Artificial intelligence device and operating method thereof

Abstract

An artificial intelligence device, according to embodiments of the present disclosure, comprises a communication unit that communicates with a display device, wherein the artificial intelligence device may involve: receiving, from the display device via the communication unit, an image generation request for a standby screen and operation state information about an electronic device linked to the display device; generating a first prompt for natural language generation on the basis of an analysis result for the image generation request and the operation state information about the electronic device; transmitting the first prompt to a first generation artificial intelligence server; receiving, in response to the first prompt, a plurality of sentences from the first generation artificial intelligence server; generating a second prompt for image generation on the basis of the plurality of sentences; transmitting the second prompt to a second generation artificial intelligence server; and receiving, from the second generation artificial intelligence server, a plurality of images corresponding to operation states of the electronic device.

Description

Artificial intelligence device and its operating method

[0001] The present disclosure relates to an artificial intelligence device, and more particularly, to an artificial intelligence device capable of generating an image corresponding to a user's speech.

[0002] Digital TV services utilizing wired or wireless networks are becoming more widespread. Digital TV services can offer a variety of services not available with existing analog broadcasting services.

[0003] For example, IPTV (Internet Protocol Television), a type of digital TV service, and smart TV services offer interactivity, allowing users to actively choose the type of program they want to watch and when. Building on this interactivity, IPTV and smart TV services can also offer a variety of additional services, such as internet search, home shopping, and online games.

[0004] The standby screen provided by existing TVs was limited in its settings, as users had to select an image from a pre-defined UI.

[0005] Additionally, the information that could be used to provide images for the standby screen on TV was limited to connected products and connection information, which were transmitted based on pre-determined rules.

[0006] The purpose of the present disclosure is to provide information and a user interface desired by a user on a standby screen without changing the software of a display device.

[0007] The purpose of this disclosure is to provide a desired image on a standby screen by simply uttering an image to generate the image.

[0008] An artificial intelligence device according to an embodiment of the present disclosure may include a communication unit that communicates with a display device, and receives a request for generating an image of a standby screen from the display device and information on the operating status of an electronic device linked with the display device through the communication unit, generates a first prompt for natural language generation based on an analysis result of the image generation request and the information on the operating status of the electronic device, transmits the first prompt to a first generation artificial intelligence server, receives a plurality of sentences from the first generation artificial intelligence server in response to the first prompt, generates a second prompt for image generation based on the plurality of sentences, transmits the second prompt to a second generation artificial intelligence server, and receives a plurality of images corresponding to the operating states of the electronic device from the second generation artificial intelligence server.

[0009] An operating method of an artificial intelligence device according to an embodiment of the present disclosure may include the steps of: receiving a request for generating an image of a standby screen from a display device and information on the operating status of an electronic device linked to the display device; generating a first prompt for natural language generation based on an analysis result of the image generation request and the information on the operating status of the electronic device; transmitting the first prompt to a first generation artificial intelligence server; receiving a plurality of sentences from the first generation artificial intelligence server in response to the first prompt; generating a second prompt for image generation based on the plurality of sentences; transmitting the second prompt to a second generation artificial intelligence server; and receiving a plurality of images corresponding to the operating states of the electronic device from the second generation artificial intelligence server.

[0010] According to embodiments of the present disclosure, information and user interfaces desired by the user can be provided on the standby screen without changing the software of the display device. Accordingly, the usability of the standby screen, which reflects the user's intentions, can be increased.

[0011] According to an embodiment of the present disclosure, a user can simply utter a word to generate an image, thereby displaying a desired image on the standby screen. This enhances interaction between the user and the display device (100).

[0012] Figure 1 is a block diagram illustrating the configuration of a display device according to one embodiment of the present invention.

[0013] Figure 2 is a block diagram of a remote control device according to one embodiment of the present invention.

[0014] Figure 3 shows an example of an actual configuration of a remote control device according to one embodiment of the present invention.

[0015] Figure 4 shows an example of utilizing a remote control device according to an embodiment of the present invention.

[0016] FIG. 5 illustrates an artificial intelligence (AI) server according to one embodiment of the present disclosure.

[0017] FIG. 6 is a ladder diagram for explaining an operation method of an AI system according to an embodiment of the present disclosure.

[0018] FIG. 7a and FIG. 7b are diagrams illustrating a process of generating images of a standby screen through a generation AI according to an embodiment of the present disclosure.

[0019] FIG. 8A and FIG. 8B are drawings explaining changes in images displayed on a standby screen according to changes in the status of an electronic device linked to a display device.

[0020] Hereinafter, embodiments related to the present invention will be described in more detail with reference to the drawings. The suffixes "module" and "part" used in the following description for components are assigned or used interchangeably solely for the convenience of writing the specification, and do not in themselves have distinct meanings or roles.

[0021] A display device according to an embodiment of the present invention is, for example, an intelligent display device that adds computer-assisted functionality to its broadcast reception function. While faithfully performing the broadcast reception function, it can also be equipped with Internet functionality and other features, providing a more user-friendly interface, such as a manual input device, touch screen, or space remote control. Furthermore, with support for wired or wireless Internet functionality, it can connect to the Internet and a computer, enabling functions such as email, web browsing, banking, or gaming. A standardized, general-purpose operating system can be used for these various functions.

[0022] Accordingly, the display device described in the present invention can perform various user-friendly functions, for example, by allowing various applications to be freely added or deleted on a general-purpose operating system kernel. More specifically, the display device can be a network TV, HBB TV, smart TV, LED TV, OLED TV, etc., and in some cases, it can also be applied to smartphones.

[0023] FIG. 1 is a block diagram illustrating the configuration of a display device according to one embodiment of the present invention.

[0024] Referring to FIG. 1, the display device (100) may include a broadcast receiving unit (130), an external device interface (135), a memory (140), a user input interface (150), a controller (170), a wireless communication interface (173), a display (180), a speaker (185), and a power supply circuit (190).

[0025] The broadcast receiving unit (130) may include a tuner (131), a demodulator (132), and a network interface (133).

[0026] The tuner (131) can select a specific broadcast channel according to a channel selection command. The tuner (131) can receive a broadcast signal for the selected specific broadcast channel.

[0027] The demodulator (132) can separate the received broadcast signal into a video signal, an audio signal, and a data signal related to the broadcast program, and can restore the separated video signal, audio signal, and data signal into a form that can be output.

[0028] The external device interface (135) can receive an application or a list of applications within an adjacent external device and transmit it to the controller (170) or memory (140).

[0029] The external device interface (135) can provide a connection path between the display device (100) and the external device. The external device interface (135) can receive one or more of images and audio output from an external device connected wirelessly or wiredly to the display device (100) and transmit them to the controller (170). The external device interface (135) can include a plurality of external input terminals. The plurality of external input terminals can include an RGB terminal, one or more HDMI (High Definition Multimedia Interface) terminals, and a component terminal.

[0030] A video signal of an external device input through an external device interface (135) can be output through a display (180). A voice signal of an external device input through an external device interface (135) can be output through a speaker (185).

[0031] An external device that can be connected to the external device interface (135) may be any one of a set-top box, a Blu-ray player, a DVD player, a game console, a sound bar, a smartphone, a PC, a USB memory, and a home theater, but this is only an example.

[0032] The network interface (133) may provide an interface for connecting the display device (100) to a wired / wireless network including the Internet. The network interface (133) may transmit or receive data to or from other users or other electronic devices via the connected network or another network linked to the connected network.

[0033] Additionally, some content data stored in the display device (100) can be transmitted to a selected user or electronic device among other users or other electronic devices pre-registered in the display device (100).

[0034] The network interface (133) can access a predetermined web page through a connected network or another network linked to the connected network. That is, by accessing a predetermined web page through a network, data can be transmitted or received with the corresponding server.

[0035] In addition, the network interface (133) can receive content or data provided by a content provider or network operator. That is, the network interface (133) can receive content such as movies, advertisements, games, VOD, broadcast signals, etc. and information related thereto provided from a content provider or network provider via a network.

[0036] Additionally, the network interface (133) can receive firmware update information and update files provided by the network operator, and transmit data to the Internet or content provider or network operator.

[0037] The network interface (133) can select and receive a desired application from among applications open to the public via a network.

[0038] The memory (140) stores a program for each signal processing and control within the controller (170), and can store signal-processed image, voice, or data signals.

[0039] In addition, the memory (140) may perform a function for temporary storage of video, audio, or data signals input from an external device interface (135) or a network interface (133), and may store information about a specific image through a channel memory function.

[0040] The memory (140) can store an application or a list of applications input from an external device interface (135) or a network interface (133).

[0041] The display device (100) can play content files (video files, still image files, music files, document files, application files, etc.) stored in the memory (140) and provide them to the user.

[0042] The user input interface (150) can transmit a signal input by the user to the controller (170) or transmit a signal from the controller (170) to the user. For example, the user input interface (150) can receive and process control signals such as power on / off, channel selection, and screen setting from the remote control device (200) according to various communication methods such as Bluetooth, Ultra Wideband (WB), ZigBee, Radio Frequency (RF) communication, or infrared (IR) communication, or process control signals from the controller (170) to be transmitted to the remote control device (200).

[0043] In addition, the user input interface (150) can transmit control signals input from local keys (not shown) such as a power key, channel key, volume key, and setting value to the controller (170).

[0044] An image signal processed by the controller (170) can be input to the display (180) and displayed as an image corresponding to the image signal. In addition, an image signal processed by the controller (170) can be input to an external output device through an external device interface (135).

[0045] The voice signal processed by the controller (170) can be output as audio to the speaker (185). In addition, the voice signal processed by the controller (170) can be input to an external output device through the external device interface (135).

[0046] In addition, the controller (170) can control the overall operation within the display device (100).

[0047] In addition, the controller (170) can control the display device (100) by a user command or internal program input through the user input interface (150), and can connect to a network to enable the user to download a desired application or application list into the display device (100).

[0048] The controller (170) enables the user-selected channel information, etc. to be output through a display (180) or speaker (185) together with processed video or audio signals.

[0049] In addition, the controller (170) allows a video signal or audio signal from an external device, for example, a camera or camcorder, input through the external device interface (135) to be output through the display (180) or speaker (185) in accordance with an external device video playback command received through the user input interface (150).

[0050] Meanwhile, the controller (170) can control the display (180) to display an image, for example, a broadcast image input through a tuner (131), an external input image input through an external device interface (135), an image input through a network interface, or an image stored in a memory (140) can be controlled to be displayed on the display (180). In this case, the image displayed on the display (180) can be a still image or a moving image, and can be a 2D image or a 3D image.

[0051] In addition, the controller (170) can control the playback of content stored in the display device (100), received broadcast content, or external input content input from outside, and the content can be in various forms such as broadcast video, external input video, audio file, still image, connected web screen, and document file.

[0052] The wireless communication interface (173) can communicate with an external device through wired or wireless communication. The wireless communication interface (173) can perform short-range communication with the external device. To this end, the wireless communication interface (173) can support short-range communication using at least one of Bluetooth™, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies. This wireless communication interface (173) can support wireless communication between the display device (100) and a wireless communication system, between the display device (100) and another display device (100), or between the display device (100) and a network in which the display device (100, or an external server) is located via a short-range wireless communication network (Wireless Area Network). The short-range wireless communication network can be a short-range wireless personal area network (Wireless Personal Area Network).

[0053] Here, the other display device (100) may be a wearable device (e.g., a smartwatch, smart glasses, a head-mounted display (HMD)) or a mobile terminal such as a smart phone that can exchange data with (or be linked to) the display device (100) according to the present invention. The wireless communication interface (173) may detect (or recognize) a wearable device capable of communication around the display device (100).

[0054] Furthermore, if the detected wearable device is a device certified to communicate with the display device (100) according to the present invention, the controller (170) can transmit at least a portion of the data processed in the display device (100) to the wearable device via the wireless communication interface (173). Accordingly, a user of the wearable device can utilize the data processed in the display device (100) via the wearable device.

[0055] The display (180) can generate a driving signal by converting a video signal, data signal, OSD signal processed by the controller (170) or a video signal, data signal, etc. received from an external device interface (135) into R, G, and B signals, respectively.

[0056] Meanwhile, since the display device (100) illustrated in FIG. 1 is merely an embodiment of the present invention, some of the illustrated components may be integrated, added, or omitted depending on the specifications of the display device (100) actually implemented.

[0057] That is, two or more components may be combined into a single component, or a single component may be subdivided into two or more components, as needed. Furthermore, the functions performed by each block are intended to illustrate embodiments of the present invention, and their specific operations or devices do not limit the scope of the present invention.

[0058] According to another embodiment of the present invention, the display device (100) may receive and play back an image through a network interface (133) or an external device interface (135) without having a tuner (131) and a demodulator (132), unlike that shown in FIG. 1.

[0059] For example, the display device (100) may be implemented separately as an image processing device, such as a set-top box, for receiving broadcast signals or contents according to various network services, and a content playback device for playing contents input from the image processing device.

[0060] In this case, the operating method of the display device according to the embodiment of the present invention described below may be performed by any one of the display device (100) described with reference to FIG. 1, as well as an image processing device such as the separated set-top box, or a content playback device having a display (180) and an audio output unit (185).

[0061] Next, a remote control device according to an embodiment of the present invention will be described with reference to FIGS. 2 and 3.

[0062] FIG. 2 is a block diagram of a remote control device according to an embodiment of the present invention, and FIG. 3 shows an example of an actual configuration of a remote control device (200) according to an embodiment of the present invention.

[0063] First, referring to FIG. 2, the remote control device (200) may include a fingerprint recognition device (210), a wireless communication circuit (220), a user input interface (230), a sensor (240), an output interface (250), a power supply circuit (260), a memory (270), a controller (280), and a microphone (290).

[0064] Referring to FIG. 2, the wireless communication circuit (220) transmits and receives signals with any one of the display devices according to the embodiments of the present invention described above.

[0065] The remote control device (200) may be equipped with an RF circuit (221) capable of transmitting and receiving signals with the display device (100) according to RF communication standards, and an IR circuit (223) capable of transmitting and receiving signals with the display device (100) according to IR communication standards. In addition, the remote control device (200) may be equipped with a Bluetooth circuit (225) capable of transmitting and receiving signals with the display device (100) according to Bluetooth communication standards. In addition, the remote control device (200) may be equipped with an NFC circuit (227) capable of transmitting and receiving signals with the display device (100) according to NFC (Near Field Communication) communication standards, and a WLAN circuit (229) capable of transmitting and receiving signals with the display device (100) according to WLAN (Wireless LAN) communication standards.

[0066] In addition, the remote control device (200) transmits a signal containing information about the movement of the remote control device (200) to the display device (100) through a wireless communication circuit (220).

[0067] Meanwhile, the remote control device (200) can receive a signal transmitted by the display device (100) through the RF circuit (221), and, if necessary, can transmit commands for power on / off, channel change, volume change, etc. to the display device (100) through the IR circuit (223).

[0068] The user input interface (230) may be configured as a keypad, buttons, a touchpad, or a touch screen. The user can input commands related to the display device (100) to the remote control device (200) by operating the user input interface (230). If the user input interface (230) includes a hard key button, the user can input commands related to the display device (100) to the remote control device (200) by pushing the hard key button. This will be described with reference to FIG. 3.

[0069] Referring to FIG. 3, the remote control device (200) may include a plurality of buttons. The plurality of buttons may include a fingerprint recognition button (212), a power button (231), a home button (232), a live button (233), an external input button (234), a volume control button (235), a voice recognition button (236), a channel change button (237), a confirmation button (238), and a back button (239).

[0070] The fingerprint recognition button (212) may be a button for recognizing a user's fingerprint. In one embodiment, the fingerprint recognition button (212) may be capable of a push operation, and may receive a push operation and a fingerprint recognition operation.

[0071] The power button (231) may be a button for turning the power of the display device (100) on / off.

[0072] The home button (232) may be a button for moving to the home screen of the display device (100).

[0073] The live button (233) may be a button for displaying a real-time broadcast program.

[0074] The external input button (234) may be a button for receiving an external input connected to the display device (100).

[0075] The volume control button (235) may be a button for adjusting the size of the volume output by the display device (100).

[0076] The voice recognition button (236) may be a button for receiving a user's voice and recognizing the received voice.

[0077] The channel change button (237) may be a button for receiving a broadcast signal of a specific broadcast channel.

[0078] The confirmation button (238) may be a button for selecting a specific function, and the back button (239) may be a button for returning to the previous screen.

[0079] Let's explain Figure 2 again.

[0080] When the user input interface (230) has a touch screen, the user can input commands related to the display device (100) using the remote control device (200) by touching the soft keys of the touch screen. In addition, the user input interface (230) may have various types of input means that can be operated by the user, such as a scroll key or a jog key, and this embodiment does not limit the scope of the present invention.

[0081] The sensor (240) may include a gyro sensor (241) or an acceleration sensor (243), and the gyro sensor (241) may sense information about the movement of the remote control device (200).

[0082] For example, the gyro sensor (241) can sense information about the operation of the remote control device (200) based on the x, y, and z axes, and the acceleration sensor (243) can sense information about the movement speed of the remote control device (200). Meanwhile, the remote control device (200) can further include a distance measuring sensor, so as to sense the distance to the display (180) of the display device (100).

[0083] The output interface (250) can output a video or audio signal corresponding to an operation of the user input interface (230) or a signal transmitted from the display device (100).

[0084] The user can recognize whether the output interface (250) is manipulating the user input interface (230) or controlling the display device (100).

[0085] For example, the output interface (250) may include an LED (251) that lights up when the user input interface (230) is operated or a signal is transmitted and received with the display device (100) via the wireless communication unit (225), a vibrator (253) that generates vibrations, a speaker (255) that outputs sound, or a display (257) that outputs images.

[0086] In addition, the power supply circuit (260) supplies power to the remote control device (200), and power waste can be reduced by stopping the power supply when the remote control device (200) does not move for a predetermined period of time.

[0087] The power supply circuit (260) can resume power supply when a predetermined key provided in the remote control device (200) is operated.

[0088] The memory (270) can store various types of programs, application data, etc. required for the control or operation of the remote control device (200).

[0089] When the remote control device (200) wirelessly transmits and receives signals through the display device (100) and the RF circuit (221), the remote control device (200) and the display device (100) transmit and receive signals through a predetermined frequency band.

[0090] The controller (280) of the remote control device (200) can store and reference information regarding the frequency band that can wirelessly transmit and receive signals with the display device (100) paired with the remote control device (200) in the memory (270).

[0091] The controller (280) controls all matters related to the control of the remote control device (200). The controller (280) can transmit a signal corresponding to a predetermined key operation of the user input interface (230) or a signal corresponding to the movement of the remote control device (200) sensed by the sensor (240) to the display device (100) via the wireless communication unit (225).

[0092] Additionally, the microphone (290) of the remote control device (200) can acquire voice.

[0093] A plurality of microphones (290) may be provided.

[0094] Next, Figure 4 is described.

[0095] Figure 4 shows an example of utilizing a remote control device according to an embodiment of the present invention.

[0096] Figure 4 (a) illustrates that a pointer (205) corresponding to a remote control device (200) is displayed on a display (180).

[0097] The user can move or rotate the remote control device (200) up and down, left and right. The pointer (205) displayed on the display (180) of the display device (100) corresponds to the movement of the remote control device (200). This remote control device (200) can be called a space remote control because, as shown in the drawing, the pointer (205) moves and is displayed according to the movement in 3D space.

[0098] Figure 4 (b) illustrates that when a user moves the remote control device (200) to the left, the pointer (205) displayed on the display (180) of the display device (100) also moves to the left correspondingly.

[0099] Information about the movement of the remote control device (200) detected through the sensor of the remote control device (200) is transmitted to the display device (100). The display device (100) can calculate the coordinates of the pointer (205) from the information about the movement of the remote control device (200). The display device (100) can display the pointer (205) to correspond to the calculated coordinates.

[0100] Figure 4 (c) illustrates a case where a user moves the remote control device (200) away from the display (180) while pressing a specific button within the remote control device (200). As a result, a selection area within the display (180) corresponding to the pointer (205) can be zoomed in and displayed in an enlarged manner.

[0101] Conversely, when the user moves the remote control device (200) closer to the display (180), the selection area within the display (180) corresponding to the pointer (205) may be zoomed out and displayed in a reduced size.

[0102] Meanwhile, when the remote control device (200) moves away from the display (180), the selection area may be zoomed out, and when the remote control device (200) moves closer to the display (180), the selection area may be zoomed in.

[0103] Additionally, when a specific button within the remote control device (200) is pressed, recognition of up, down, left, and right movements may be excluded. That is, when the remote control device (200) moves away from or toward the display (180), up, down, left, and right movements may not be recognized, and only forward and backward movements may be recognized. When a specific button within the remote control device (200) is not pressed, only the pointer (205) moves in accordance with the up, down, left, and right movements of the remote control device (200).

[0104] Meanwhile, the movement speed or movement direction of the pointer (205) can correspond to the movement speed or movement direction of the remote control device (200).

[0105] Meanwhile, the pointer in this specification refers to an object displayed on the display (180) in response to the operation of the remote control device (200). Accordingly, objects of various shapes other than the arrow shape illustrated in the drawing can be used as the pointer (205). For example, the pointer may be a concept including a point, a cursor, a prompt, a thick outline, etc. In addition, the pointer (205) may be displayed corresponding to one point on the horizontal or vertical axis on the display (180), or may be displayed corresponding to multiple points such as lines or surfaces.

[0106] FIG. 5 illustrates an artificial intelligence (AI) server according to one embodiment of the present disclosure.

[0107] Referring to FIG. 5, the AI ​​server (500) may refer to a device that trains an artificial neural network using a machine learning algorithm or uses a trained artificial neural network.

[0108] The AI ​​server (500) may be composed of multiple servers to perform distributed processing, and may be defined as a 5G network.

[0109] The AI ​​server (500) may be included as part of the configuration of the AI ​​device (100) and may perform at least part of the AI ​​processing together.

[0110] The AI ​​server (500) may include a communication unit (510), a memory (530), a learning processor (540), and a processor (560).

[0111] The communication unit (510) can transmit and receive data with an external device such as a display device (100).

[0112] The memory (530) may include a model storage unit (531).

[0113] The model storage unit (531) can store a model (or artificial neural network, 531a) that is being learned or has been learned through the learning processor (540).

[0114] A learning processor (540) can train an artificial neural network (531a) using learning data. The learning model can be used while mounted on the AI ​​server (500) of the artificial neural network, or can be mounted on an external device such as a display device (100).

[0115] The learning model may be implemented in hardware, software, or a combination of hardware and software. If part or all of the learning model is implemented in software, one or more instructions constituting the learning model may be stored in memory (530).

[0116] The processor (560) can use a learning model to infer a result value for new input data and generate a response or control command based on the inferred result value.

[0117] FIG. 6 is a ladder diagram for explaining an operation method of an AI system according to an embodiment of the present disclosure.

[0118] Referring to FIG. 6, the AI ​​system may include a display device (100), an AI server (500), a first generation AI server (610), and a second generation AI server (630).

[0119] In FIG. 6, the first generation AI server (610) and the second generation AI server (630) may be configured as one server.

[0120] Also, although FIG. 6 illustrates two generation AI servers as an example, three or more generation AI servers may be provided.

[0121] Referring to FIG. 6, the controller (170) of the display device (100) can receive a request to create an image of a standby screen (S601).

[0122] In one embodiment, the standby screen may be a screen displayed in a low power mode of the display device (100).

[0123] The image generation request may be any one of a voice command spoken by the user, a text input by the user, or a command received from a remote control device (200).

[0124] The controller (170) of the display device (100) can obtain information of an electronic device linked to the display device (100) (S603).

[0125] Information about an electronic device may include the name of the electronic device and information about the operating status of the electronic device.

[0126] The operating status information may include the operating states of the electronic device. For example, if the electronic device is a home appliance equipped with a washing and drying function, the operating status information may include: washing start, washing in progress, washing complete, drying start, drying in progress, and drying complete.

[0127] The controller (170) can receive information about an electronic device from one or more electronic devices linked to the display device (100) via a network interface (133).

[0128] In one embodiment, the controller (170) may transmit a request message to an electronic device in response to an image generation request, and may receive information from the electronic device in response to the request message.

[0129] In another embodiment, the controller (170) may transmit a request message to the electronic device before obtaining an image generation request, and may obtain information of the electronic device in advance in response to the request message.

[0130] The controller (170) of the display device (100) can transmit an image generation request and information of the electronic device to the AI ​​server (500) through the network interface (133) (S605).

[0131] The controller (170) can transmit an image generation request and information of an electronic device to the communication unit (510) of the AI ​​server (500) via a network interface (133).

[0132] The processor (560) of the AI ​​server (500) can generate a first prompt based on the received image generation request and information of the electronic device (S607).

[0133] In one embodiment, the first prompt may be a command to be input into a generation AI model loaded on the first generation AI server (610).

[0134] Generative AI models can be models that generate content in response to commands, using deep learning or machine learning.

[0135] The first prompt may be a command that causes the natural language generation AI model to generate natural language.

[0136] The processor (560) may generate a first prompt by combining the image generation request and information about the electronic device. The processor (560) may generate the first prompt by using the analysis results of the image generation request, the name of the electronic device, and information about the operating status of the electronic device.

[0137] Analysis results of image generation requests may include the user's intent.

[0138] For example, if the image generation request is a voice command uttered by the user, the processor (560) can convert the voice command into text and obtain the analysis results of the converted text using a natural language processing engine.

[0139] The analysis of voice commands may result in a request to create a specific mood image for the standby screen.

[0140] The processor (560) can generate a sentence by combining the analysis result of the image generation request, the name of the electronic device, and the operation status information of the electronic device, and can obtain the generated sentence as the first prompt.

[0141] The processor (560) of the AI ​​server (500) can transmit the generated first prompt to the first generation AI server (610) via the communication unit (510) (S609).

[0142] The first generation AI server (610) may be a server that generates natural language in response to a prompt.

[0143] The first generation AI server (610) can generate a plurality of sentences in response to the first prompt. Each of the plurality of sentences may be a sentence representing the analysis result of the image generation request and the characteristics of each operational state of the electronic device.

[0144] Each of the plurality of sentences may be a sentence that commands the generation of an image reflecting the analysis result of the image generation request and the characteristics of each operating state of the electronic device.

[0145] The processor (560) of the AI ​​server (500) can receive a plurality of sentences in response to the first prompt from the first generation AI server (610) (S611).

[0146] The first generation AI server (610) can generate a plurality of sentences in response to the first prompt. Each of the plurality of sentences may be a sentence representing the analysis result of the image generation request and the characteristics of each operational state of the electronic device.

[0147] The first generation AI server (610) can transmit a plurality of generated sentences to the AI ​​server (500).

[0148] The processor (560) of the AI ​​server (500) can generate a second prompt based on the received plurality of sentences (S613).

[0149] In one embodiment, the second prompt may be a command to be input into a generation AI model loaded on the second generation AI server (630).

[0150] The second prompt may be a command that causes the image generation AI model of the second generation AI server (630) to generate an image.

[0151] The processor (560) can generate a second prompt including a plurality of sentences in response to the first prompt and an operational state of the electronic device corresponding to each sentence.

[0152] The second prompt may contain only one of multiple sentences and the corresponding action state for that sentence.

[0153] The second prompt may include multiple sentences received from the first generation AI server (610).

[0154] The processor (560) of the AI ​​server (500) can transmit the generated second prompt to the second generation AI server (630) via the communication unit (510) (S615).

[0155] The second generation AI server (630) may be a server that generates an image in response to the second prompt.

[0156] The second generation AI server (630) may generate multiple images in response to the second prompt. Each of the multiple images may be an image representing the analysis result of the image generation request and the characteristics of each operating state of the electronic device.

[0157] The processor (560) of the AI ​​server (500) can receive multiple images in response to the second prompt from the second generation AI server (630) (S617).

[0158] The processor (560) can store the received multiple images in the memory (530) or in an external database (not shown).

[0159] The processor (560) of the AI ​​server (500) can transmit multiple images to the display device (100) via the communication unit (510) (S619).

[0160] The processor (560) can transmit multiple images to the network interface (133) of the display device (100) via the communication unit (510).

[0161] Each of the multiple images may be matched with each of the multiple motion states.

[0162] The controller (170) of the display device (100) displays a basic image among a plurality of images on the display (180) upon entering the standby screen (S621).

[0163] The default image may be an image corresponding to the basic operating state of the electronic device.

[0164] The base image may be an image corresponding to the analysis result of the image generation request and the basic operating state of the electronic device.

[0165] The controller (170) of the display device (100) can determine whether the operating status of the electronic device has changed while displaying the basic image (S623).

[0166] The controller (170) can determine whether the operating state of the electronic device has changed from the first state to the second state.

[0167] The controller (170) can periodically receive information on the operating status of the electronic device from the electronic device, and based on the received information, can determine whether the operating status of the electronic device has changed.

[0168] When the operating state of the electronic device is changed, the controller (170) of the display device (100) can display an image corresponding to the changed operating state among a plurality of images on the display (180) (S625).

[0169] When the operating state of the electronic device changes from the first state to the second state, the controller (170) can change the first image corresponding to the first state on the standby screen to the second image corresponding to the second state.

[0170] In this way, according to an embodiment of the present disclosure, information and a user interface desired by a user can be provided on a standby screen without changing the software of the display device (100).

[0171] Accordingly, the usability of the waiting screen that reflects the user's intention can be increased.

[0172] Meanwhile, the controller (170) can display an image corresponding to the operating status of the electronic device and a notification indicating the operating status on the display (180).

[0173] FIG. 7a and FIG. 7b are diagrams illustrating a process of generating images of a standby screen through a generation AI according to an embodiment of the present disclosure.

[0174] The processor (560) of the AI ​​server (500) may include a natural language prompter (561) and an image prompter (563).

[0175] The natural language prompter (561) can generate a natural language generation prompt based on the analysis results of the image generation request received from the display device (100) and information of the electronic device.

[0176] The natural language generation prompt may be a single sentence containing the analysis results of the image generation request and information about the electronic device.

[0177] The natural language prompter (651) can transmit the generated natural language generation prompt to the first generation AI (610) that generates natural language.

[0178] The first generation AI (610) can generate a natural language generation result by inputting a natural language generation prompt, and can transmit the generated natural language generation result to the image prompter (563) of the AI ​​server (500).

[0179] The natural language generation result may include a plurality of sentences representing the analysis result of the image generation request and the characteristics of each operating state of the electronic device.

[0180] The image prompter (563) can generate an image generation prompt based on the natural language generation result received from the first generation AI server (610).

[0181] The image generation prompt may be a prompt including a sentence received from the first generation AI server (610) and the operating status of the electronic device.

[0182] The image prompter (563) can transmit an image generation prompt including a sentence and the operating status of each electronic device to a second generation AI server (630) that generates the image.

[0183] The second generation AI server (630) can generate an image generation result in response to an image generation prompt and transmit the generated image generation result to the AI ​​server (500).

[0184] The image generation result may be an image generated by the image generation model of the second generation AI server (630) according to the image generation prompt.

[0185] The image prompter (563) can generate a plurality of image generation prompts corresponding to the operating state of the electronic device, and can sequentially transmit the generated plurality of image generation prompts to the second generation AI server (630).

[0186] The image prompter (563) can sequentially receive a plurality of images corresponding to each of the plurality of image generation prompts from the second generation AI server (630).

[0187] In another embodiment, the image prompter (563) can transmit multiple image generation prompts to the second generation AI server (630) at one time and receive multiple images from the second generation AI server (630) at one time.

[0188] The image creation prompt may be any one of the basic image creation prompt (563a), the first operation state image creation prompt (563b), and the second operation state image creation prompt (563c).

[0189] The basic image creation prompt (563a) may be a prompt that commands the creation of a basic image corresponding to the basic operating state of the electronic device.

[0190] The first operation state image generation prompt (563b) may be a prompt that commands the generation of an image corresponding to the first operation state of the electronic device.

[0191] The second operation state image generation prompt (563c) may be a prompt that commands the generation of an image corresponding to the second operation state of the electronic device.

[0192] The AI ​​server (500) can transmit a basic image generation prompt (563a) to the second generation AI server (630) and receive a basic image in response to the basic image generation prompt (563a).

[0193] The AI ​​server (500) can transmit a first motion state image generation prompt (563b) to the second generation AI server (630) and receive a first image in response to the first motion state image generation prompt (563b).

[0194] The AI ​​server (500) can transmit a second motion state image generation prompt (563c) to the second generation AI server (630) and receive a second image in response to the second motion state image generation prompt (563c).

[0195] The AI ​​server (500) can store the received basic image, first image, and second image in the generated image DB (700).

[0196] Figure 7b is a drawing illustrating a process of automatically generating an image suitable for a voice command according to a voice command uttered by an actual user.

[0197] In Figure 7b, the user<generate a beach-like standby screen> Assume that the utterance was made.

[0198] The display device (100) can transmit voice data corresponding to a voice command spoken by the user to the AI ​​server (500).

[0199] The display device (100) can transmit voice data and information of an electronic device linked to the display device (100) to the AI ​​server (500).

[0200] The AI ​​server (500) can also directly collect electronic devices linked to the display device (100).

[0201] The AI ​​server (500) can convert voice data into text data and obtain analysis results of the text data.

[0202] The natural language prompter (561) of the AI ​​server (500) can generate a natural language generation prompt based on the analysis results and information of the electronic device.

[0203] The analysis results may include an intention to create an image of a beach atmosphere on the waiting screen.

[0204] Information about an electronic device may include the name of the electronic device and information about the operating status of the electronic device.

[0205] The first generation AI server (610) can generate multiple sentences based on a natural language generation prompt.

[0206] Multiple sentences generated by the first generation AI server (610)<Draw Waikiki Beach> ,<Draw a clothesline in the sky in the background of this picture> and<Draw laundry hanging on a clothesline on this picture> It could be.

[0207] Each of the multiple sentences may contain a subject requesting a drawing of the analysis results and the operating status of the electronic device.

[0208] The first generation AI server (610) can transmit a plurality of generated sentences to the AI ​​server (500).

[0209] The image prompter (563) of the AI ​​server (500) can generate an image generation prompt by combining each of the received multiple sentences with the operating status of the electronic device.

[0210] Image prompter (563)<Draw Waikiki Beach> A basic image generation prompter (563a) can be generated by combining the basic state (drying completion state, drying start) of the electronic device. The basic state can be a drying completion state or a power-off state.

[0211] The second generation IA server (630) can generate a basic image (711) based on the basic image generation prompter (563a) and transmit the generated basic image (711) to the AI ​​server (500).

[0212] The base image (711) may be an image containing Waikiki Beach.

[0213] Image prompter (563)<Draw a clothesline in the sky in the background of this picture> And a first operation state image generation prompter (563b) combining the first operation state (laundry start state) of the electronic device can be generated.

[0214] The first motion state image generation prompter (563b)<Draw a clothesline in the sky in the background of this picture> , may include a first operating state of the electronic device and a first image (711).

[0215] The second generation IA server (630) can generate a first image (713) based on the first operation state image generation prompter (563b) and transmit the generated first image (713) to the AI ​​server (500).

[0216] The first image (713) may be an image including a clothesline representing Waikiki Beach and the start of laundry. The first image (713) may include more clotheslines than the base image (711).

[0217] The first image (713) may be an image that merges a clothesline representing the characteristics of the first operation state into the basic image (711).

[0218] Image prompter (563) is <<Draw laundry hanging on a clothesline on this picture> And a second operation state image generation prompter (563c) combining the second operation state (drying start state) of the electronic device can be generated.

[0219] The second motion state image generation prompter (563c)<Draw a clothesline in the sky in the background of this picture> , may include a second operating state of the electronic device and a second image (713).

[0220] The second generation IA server (630) can generate a second image (715) based on the second operation state image generation prompter (563c) and transmit the generated second image (715) to the AI ​​server (500).

[0221] The second image (715) may be an image including Waikiki Beach, a clothesline, and clothes hanging on a clothesline for drying. The second image (715) may include more clothes hanging on a clothesline compared to the first image (713).

[0222] The second image (715) may be an image that merges clothes hanging on a clothesline showing the characteristics of the second motion state with the first image (713).

[0223] The AI ​​server (500) can store the basic image (711), the first image (713), and the second image (715) in the generated image DB (700).

[0224] The AI ​​server (500) can transmit the basic image (711), the first image (713), and the second image (715) to the display device (100).

[0225] The display device (100) can store a basic image (711), a first image (713), and a second image (715) in the memory (140).

[0226] The display device (100) can display any one of the basic image (711), the first image (713), and the second image (715) according to the operating status of the electronic device.

[0227] FIG. 8A and FIG. 8B are drawings explaining changes in images displayed on a standby screen according to changes in the status of an electronic device linked to a display device.

[0228] In FIGS. 8a and 8b, the basic image (711), the first image (713), and the second image (715) may be images generated according to the process of FIGS. 6 to 7b.

[0229] The display device (100) can receive an image generation request for generating a standby screen.

[0230] The AI ​​server (500) can generate multiple images to be displayed on the standby screen according to the image generation logic of FIGS. 6 to 7b, and store the generated images in the generated image DB.

[0231] The AI ​​server (500) can transmit the generated images to the display device (100).

[0232] The display device (100) enters the standby screen.

[0233] The display device (100) can display a basic image (711) on the standby screen when the operating state of the electronic device (800) is the basic operating state. The basic operating state may be either a drying completion state or a power-off state.

[0234] The operating state of the electronic device (800) can be changed from the basic operating state to the first operating state (state 1), and the electronic device (800) can transmit information about the changed operating state to the display device (100). The first operating state can be a washing start state indicating the start of washing.

[0235] The display device (100) can display a first image (713) corresponding to the first operation state on the standby screen based on information about the changed operation state.

[0236] The operating state of the electronic device (800) can be changed from a first operating state to a second operating state (state 2), and the electronic device (800) can transmit information about the changed operating state to the display device (100). The second operating state can be a drying start state indicating the start of drying. The second operating state can be a state indicating the end of washing and the start of drying.

[0237] The display device (100) can display a second image (715) corresponding to the second operation state on the standby screen based on information about the changed operation state.

[0238] Meanwhile, the display device (100) can play images (711, 713, 715) in the form of GIF (Graphics Interchange Format) or play the images in reverse order according to changes in the operating state of the electronic device (800).

[0239] The display device (100) can sequentially play back the second image (715), the first image (713), and the basic image (711) when the operating state of the electronic device (800) is changed in the order of drying start state, drying completion state, and basic state.

[0240] In this way, according to an embodiment of the present disclosure, a user can have a desired image provided on a standby screen with just a simple utterance for image generation.

[0241] Accordingly, interaction between the user and the display device (100) can be enhanced.

[0242] According to one embodiment of the present invention, the above-described method can be implemented as processor-readable code on a medium in which a program is recorded. Examples of processor-readable media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices.

[0243] The display device described above is not limited to the configuration and method of the embodiments described above, and the embodiments may be configured by selectively combining all or part of each embodiment so that various modifications can be made.

Claims

1. In artificial intelligence devices, a communication unit for communicating with a display device; and A request for generating an image of a standby screen from the display device and information on the operating status of an electronic device linked to the display device are received through the communication unit, Generate a first prompt for natural language generation based on the analysis result of the image generation request and the operating status information of the electronic device, Transmit the above first prompt to the first generation artificial intelligence server, Receive a plurality of sentences in response to the first prompt from the first generation artificial intelligence server, Generate a second prompt for image generation based on the above multiple sentences, Transmit the above second prompt to the second generation artificial intelligence server, Receiving a plurality of images corresponding to the operating states of the electronic device from the second generation artificial intelligence server Contains a processor Artificial intelligence device.

2. In paragraph 1, The above processor Transmitting the plurality of images to the display device through the communication unit Artificial intelligence device.

3. In paragraph 1, The above processor As a result of the above analysis, a sentence generated by combining the name of the electronic device and the operation status information of the electronic device is obtained as the first prompt. Artificial intelligence device.

4. In paragraph 3, Each of the above multiple sentences A sentence that commands the creation of an image reflecting the above analysis results and the characteristics of each operating state of the electronic device. Artificial intelligence device.

5. In paragraph 4, The second prompt above is Including the above multiple sentences and the operating states of the electronic device corresponding to each sentence Artificial intelligence device.

6. In paragraph 4, The second prompt above is Including any one of the above multiple sentences and the operating state of the electronic device corresponding to any one of the above sentences Artificial intelligence device.

7. In paragraph 1, The above multiple images A basic image reflecting the above analysis results and the basic operating status of the electronic device, A first image merging the features of the first operating state of the electronic device into the above basic image, and A second image comprising a feature of a second operating state of the electronic device merged into the first image. Artificial intelligence device.

8. In paragraph 1, The above image creation request is It is a voice command spoken by the user, The above processor Converting the above voice command into text data and obtaining the analysis result based on the converted text data Artificial intelligence device.

9. In the method of operating an artificial intelligence device, A step of receiving a request for generating an image of a standby screen from a display device and information on the operating status of an electronic device linked to the display device; A step of generating a first prompt for natural language generation based on the analysis result of the image generation request and the operating status information of the electronic device; A step of transmitting the first prompt to the first generation artificial intelligence server; A step of receiving a plurality of sentences in response to the first prompt from the first generation artificial intelligence server; A step of generating a second prompt for image generation based on the plurality of sentences; a step of transmitting the second prompt to a second generation artificial intelligence server; and A step of receiving a plurality of images corresponding to the operating states of the electronic device from the second generation artificial intelligence server. How an artificial intelligence device operates.

10. In paragraph 9, further comprising a step of transmitting the plurality of images to the display device. How an artificial intelligence device operates.

11. In paragraph 9, The step of generating the above first prompt is The above analysis result includes a step of obtaining a sentence generated by combining the name of the electronic device and the operation status information of the electronic device as the first prompt. How an artificial intelligence device operates.

12. In paragraph 11, Each of the above multiple sentences A sentence that commands the creation of an image reflecting the above analysis results and the characteristics of each operating state of the electronic device. How an artificial intelligence device operates.

13. In paragraph 12, The second prompt above is Including the above multiple sentences and the operating states of the electronic device corresponding to each sentence How an artificial intelligence device operates.

14. In paragraph 12, The second prompt above is Including any one of the above multiple sentences and the operating state of the electronic device corresponding to any one of the above sentences How an artificial intelligence device operates.

15. In paragraph 9, The above multiple images A basic image reflecting the above analysis results and the basic operating status of the electronic device, A first image merging the features of the first operating state of the electronic device into the above basic image, and A second image comprising a feature of a second operating state of the electronic device merged into the first image. How an artificial intelligence device operates.

Images