Artificial intelligence device and operation method thereof

Abstract

An artificial intelligence device according to an embodiment of the present disclosure may include: a communication unit for communicating with an electronic device and one or more generative artificial intelligence (AI) servers; and a processor for: receiving a voice command uttered by a user from the electronic device; obtaining an analysis result of the received voice command; generating a prompt on the basis of the analysis result when it is determined that a response of the one or more generative AI servers is required on the basis of the obtained analysis result; transmitting the generated prompt to the one or more generative AI servers; receiving a response result corresponding to the prompt from the one or more generative AI servers; and transmitting the analysis result and the response result to the electronic device.

Description

Artificial intelligence device and its operating method

[0001] The present disclosure relates to an artificial intelligence device, and more particularly, to an artificial intelligence device capable of generating an image corresponding to a user's speech.

[0002] Digital TV services utilizing wired or wireless networks are becoming more widespread. Digital TV services can offer a variety of services not available with existing analog broadcasting services.

[0003] For example, IPTV (Internet Protocol Television), a type of digital TV service, and smart TV services offer interactivity, allowing users to actively choose the type of program they want to watch and when. Building on this interactivity, IPTV and smart TV services can also offer a variety of additional services, such as internet search, home shopping, and online games.

[0004] Recently, TVs are providing voice recognition services by analyzing voice commands given by users.

[0005] In the past, when performing voice recognition, only the voice recognition results provided by the most suitable AI server were displayed.

[0006] However, there is a problem that the voice recognition results may not match the intention of the voice spoken by the user.

[0007] The purpose of this disclosure is to provide, when providing information that matches the user's speech intent, both the response result of a server operated by a display device manufacturer and the response result of an external AI server.

[0008] The purpose of this disclosure is to provide various services to users by combining the response results of externally generated AI servers to voice commands uttered by users.

[0009] An artificial intelligence device according to an embodiment of the present disclosure may include a communication unit that communicates with an electronic device and one or more artificial intelligence (AI) generation servers; and a processor that receives a voice command uttered by a user from the electronic device, obtains an analysis result of the received voice command, and, if it is determined that a response from the one or more generation AI servers is required based on the obtained analysis result, generates a prompt based on the analysis result, transmits the generated prompt to the one or more generation AI servers, receives a response result corresponding to the prompt from the one or more generation AI servers, and transmits the analysis result and the response result to the electronic device.

[0010] An operating method of an artificial intelligence device according to an embodiment of the present disclosure may include: receiving a voice command uttered by a user from an electronic device; obtaining an analysis result of the received voice command; generating a prompt based on the analysis result when it is determined that a response from one or more generation AI servers is required based on the obtained analysis result; transmitting the generated prompt to the one or more generation AI servers; receiving a response result corresponding to the prompt from the one or more generation AI servers; and transmitting the analysis result and the response result to the electronic device.

[0011] According to embodiments of the present disclosure, an NLP service from the display device manufacturer and an external server service that provides information received from an externally generated AI can be utilized in conjunction with the user's speech intent. This can enhance the user experience by providing a variety of response results tailored to the user's speech intent.

[0012] Figure 1 is a block diagram illustrating the configuration of a display device according to one embodiment of the present invention.

[0013] Figure 2 is a block diagram of a remote control device according to one embodiment of the present invention.

[0014] Figure 3 shows an example of an actual configuration of a remote control device according to one embodiment of the present invention.

[0015] Figure 4 shows an example of utilizing a remote control device according to an embodiment of the present invention.

[0016] FIG. 5 illustrates an artificial intelligence (AI) server according to one embodiment of the present disclosure.

[0017] FIG. 6 is a ladder diagram for explaining an operation method of an AI system according to an embodiment of the present disclosure.

[0018] FIG. 7a is a diagram illustrating a process of providing analysis results of an AI server based on a user's voice command according to a conventional technology.

[0019] FIGS. 7b to 7d are drawings illustrating a process of providing an analysis result of an AI server based on a user's voice command and a first response result of a first generation AI server according to an embodiment of the present disclosure.

[0020] FIG. 8 is a ladder diagram for explaining an operation method of an AI system according to another embodiment of the present disclosure.

[0021] FIG. 9 is a diagram illustrating a process of providing analysis results of an AI server based on a user's voice command and first and second response results provided from each of the first and second generation AI servers according to an embodiment of the present disclosure.

[0022] FIG. 10 is a ladder diagram for explaining an operation method of an AI system according to another embodiment of the present disclosure.

[0023] FIGS. 11A to 11C are diagrams illustrating a process for providing a response result of another generation AI when a response result of another generation AI is required according to an embodiment of the present disclosure.

[0024] Hereinafter, embodiments related to the present invention will be described in more detail with reference to the drawings. The suffixes "module" and "part" used in the following description for components are assigned or used interchangeably solely for the convenience of writing the specification, and do not in themselves have distinct meanings or roles.

[0025] A display device according to an embodiment of the present invention is, for example, an intelligent display device that adds computer-assisted functionality to its broadcast reception function. While faithfully performing the broadcast reception function, it can also be equipped with Internet functionality and other features, providing a more user-friendly interface, such as a manual input device, touch screen, or space remote control. Furthermore, with support for wired or wireless Internet functionality, it can connect to the Internet and a computer, enabling functions such as email, web browsing, banking, or gaming. A standardized, general-purpose operating system can be used for these various functions.

[0026] Accordingly, the display device described in the present invention can perform various user-friendly functions, for example, by allowing various applications to be freely added or deleted on a general-purpose operating system kernel. More specifically, the display device can be a network TV, HBB TV, smart TV, LED TV, OLED TV, etc., and in some cases, it can also be applied to smartphones.

[0027] FIG. 1 is a block diagram illustrating the configuration of a display device according to one embodiment of the present invention.

[0028] Referring to FIG. 1, the display device (100) may include a broadcast receiving unit (130), an external device interface (135), a memory (140), a user input interface (150), a controller (170), a wireless communication interface (173), a display (180), a speaker (185), and a power supply circuit (190).

[0029] The broadcast receiving unit (130) may include a tuner (131), a demodulator (132), and a network interface (133).

[0030] The tuner (131) can select a specific broadcast channel according to a channel selection command. The tuner (131) can receive a broadcast signal for the selected specific broadcast channel.

[0031] The demodulator (132) can separate the received broadcast signal into a video signal, an audio signal, and a data signal related to the broadcast program, and can restore the separated video signal, audio signal, and data signal into a form that can be output.

[0032] The external device interface (135) can receive an application or a list of applications within an adjacent external device and transmit it to the controller (170) or memory (140).

[0033] The external device interface (135) can provide a connection path between the display device (100) and the external device. The external device interface (135) can receive one or more of images and audio output from an external device connected wirelessly or wiredly to the display device (100) and transmit them to the controller (170). The external device interface (135) can include a plurality of external input terminals. The plurality of external input terminals can include an RGB terminal, one or more HDMI (High Definition Multimedia Interface) terminals, and a component terminal.

[0034] A video signal of an external device input through an external device interface (135) can be output through a display (180). A voice signal of an external device input through an external device interface (135) can be output through a speaker (185).

[0035] An external device that can be connected to the external device interface (135) may be any one of a set-top box, a Blu-ray player, a DVD player, a game console, a sound bar, a smartphone, a PC, a USB memory, and a home theater, but this is only an example.

[0036] The network interface (133) may provide an interface for connecting the display device (100) to a wired / wireless network including the Internet. The network interface (133) may transmit or receive data to or from other users or other electronic devices via the connected network or another network linked to the connected network.

[0037] Additionally, some content data stored in the display device (100) can be transmitted to a selected user or electronic device among other users or other electronic devices pre-registered in the display device (100).

[0038] The network interface (133) can access a predetermined web page through a connected network or another network linked to the connected network. That is, by accessing a predetermined web page through a network, data can be transmitted or received with the corresponding server.

[0039] In addition, the network interface (133) can receive content or data provided by a content provider or network operator. That is, the network interface (133) can receive content such as movies, advertisements, games, VOD, broadcast signals, etc. and information related thereto provided from a content provider or network provider via a network.

[0040] Additionally, the network interface (133) can receive firmware update information and update files provided by the network operator, and can transmit data to the Internet or content provider or network operator.

[0041] The network interface (133) can select and receive a desired application from among applications open to the public via a network.

[0042] The memory (140) stores a program for each signal processing and control within the controller (170), and can store signal-processed image, voice, or data signals.

[0043] In addition, the memory (140) may perform a function for temporary storage of video, audio, or data signals input from an external device interface (135) or a network interface (133), and may store information about a specific image through a channel memory function.

[0044] The memory (140) can store an application or a list of applications input from an external device interface (135) or a network interface (133).

[0045] The display device (100) can play content files (video files, still image files, music files, document files, application files, etc.) stored in the memory (140) and provide them to the user.

[0046] The user input interface (150) can transmit a signal input by the user to the controller (170) or transmit a signal from the controller (170) to the user. For example, the user input interface (150) can receive and process control signals such as power on / off, channel selection, and screen setting from the remote control device (200) according to various communication methods such as Bluetooth, Ultra Wideband (WB), ZigBee, Radio Frequency (RF) communication, or infrared (IR) communication, or process control signals from the controller (170) to be transmitted to the remote control device (200).

[0047] In addition, the user input interface (150) can transmit control signals input from local keys (not shown) such as a power key, channel key, volume key, and setting value to the controller (170).

[0048] An image signal processed by the controller (170) can be input to the display (180) and displayed as an image corresponding to the image signal. In addition, an image signal processed by the controller (170) can be input to an external output device through an external device interface (135).

[0049] The voice signal processed by the controller (170) can be output as audio to the speaker (185). In addition, the voice signal processed by the controller (170) can be input to an external output device through the external device interface (135).

[0050] In addition, the controller (170) can control the overall operation within the display device (100).

[0051] In addition, the controller (170) can control the display device (100) by a user command or internal program input through the user input interface (150), and can connect to a network to enable the user to download a desired application or application list into the display device (100).

[0052] The controller (170) enables the user-selected channel information, etc. to be output through a display (180) or speaker (185) together with processed video or audio signals.

[0053] In addition, the controller (170) allows a video signal or audio signal from an external device, for example, a camera or camcorder, input through the external device interface (135) to be output through the display (180) or speaker (185) in accordance with an external device video playback command received through the user input interface (150).

[0054] Meanwhile, the controller (170) can control the display (180) to display an image, for example, a broadcast image input through a tuner (131), an external input image input through an external device interface (135), an image input through a network interface, or an image stored in a memory (140) can be controlled to be displayed on the display (180). In this case, the image displayed on the display (180) can be a still image or a moving image, and can be a 2D image or a 3D image.

[0055] In addition, the controller (170) can control the playback of content stored in the display device (100), received broadcast content, or external input content input from outside, and the content can be in various forms such as broadcast video, external input video, audio file, still image, connected web screen, and document file.

[0056] The wireless communication interface (173) can communicate with an external device through wired or wireless communication. The wireless communication interface (173) can perform short-range communication with the external device. To this end, the wireless communication interface (173) can support short-range communication using at least one of Bluetooth™, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies. This wireless communication interface (173) can support wireless communication between the display device (100) and a wireless communication system, between the display device (100) and another display device (100), or between the display device (100) and a network in which the display device (100, or an external server) is located via a short-range wireless communication network (Wireless Area Network). The short-range wireless communication network can be a short-range wireless personal area network (Wireless Personal Area Network).

[0057] Here, the other display device (100) may be a wearable device (e.g., a smartwatch, smart glasses, a head-mounted display (HMD)) or a mobile terminal such as a smart phone that can exchange data with (or be linked to) the display device (100) according to the present invention. The wireless communication interface (173) may detect (or recognize) a wearable device capable of communication around the display device (100).

[0058] Furthermore, if the detected wearable device is a device certified to communicate with the display device (100) according to the present invention, the controller (170) can transmit at least a portion of the data processed in the display device (100) to the wearable device via the wireless communication interface (173). Accordingly, a user of the wearable device can utilize the data processed in the display device (100) via the wearable device.

[0059] The display (180) can generate a driving signal by converting a video signal, data signal, OSD signal processed by the controller (170) or a video signal, data signal, etc. received from an external device interface (135) into R, G, and B signals, respectively.

[0060] Meanwhile, since the display device (100) illustrated in FIG. 1 is merely an embodiment of the present invention, some of the illustrated components may be integrated, added, or omitted depending on the specifications of the display device (100) actually implemented.

[0061] That is, two or more components may be combined into a single component, or a single component may be subdivided into two or more components, as needed. Furthermore, the functions performed by each block are intended to illustrate embodiments of the present invention, and their specific operations or devices do not limit the scope of the present invention.

[0062] According to another embodiment of the present invention, the display device (100) may receive and play back an image through a network interface (133) or an external device interface (135) without having a tuner (131) and a demodulator (132), unlike that shown in FIG. 1.

[0063] For example, the display device (100) may be implemented separately as an image processing device, such as a set-top box, for receiving broadcast signals or contents according to various network services, and a content playback device for playing contents input from the image processing device.

[0064] In this case, the operating method of the display device according to the embodiment of the present invention described below may be performed by any one of the display device (100) described with reference to FIG. 1, as well as an image processing device such as the separated set-top box, or a content playback device having a display (180) and an audio output unit (185).

[0065] Next, a remote control device according to an embodiment of the present invention will be described with reference to FIGS. 2 and 3.

[0066] FIG. 2 is a block diagram of a remote control device according to an embodiment of the present invention, and FIG. 3 shows an example of an actual configuration of a remote control device (200) according to an embodiment of the present invention.

[0067] First, referring to FIG. 2, the remote control device (200) may include a fingerprint recognition device (210), a wireless communication circuit (220), a user input interface (230), a sensor (240), an output interface (250), a power supply circuit (260), a memory (270), a controller (280), and a microphone (290).

[0068] Referring to FIG. 2, the wireless communication circuit (220) transmits and receives signals with any one of the display devices according to the embodiments of the present invention described above.

[0069] The remote control device (200) may be equipped with an RF circuit (221) capable of transmitting and receiving signals with the display device (100) according to RF communication standards, and an IR circuit (223) capable of transmitting and receiving signals with the display device (100) according to IR communication standards. In addition, the remote control device (200) may be equipped with a Bluetooth circuit (225) capable of transmitting and receiving signals with the display device (100) according to Bluetooth communication standards. In addition, the remote control device (200) may be equipped with an NFC circuit (227) capable of transmitting and receiving signals with the display device (100) according to NFC (Near Field Communication) communication standards, and a WLAN circuit (229) capable of transmitting and receiving signals with the display device (100) according to WLAN (Wireless LAN) communication standards.

[0070] In addition, the remote control device (200) transmits a signal containing information about the movement of the remote control device (200) to the display device (100) through a wireless communication circuit (220).

[0071] Meanwhile, the remote control device (200) can receive a signal transmitted by the display device (100) through the RF circuit (221), and, if necessary, can transmit commands for turning the power on / off, changing the channel, changing the volume, etc. to the display device (100) through the IR circuit (223).

[0072] The user input interface (230) may be configured as a keypad, buttons, a touchpad, or a touch screen. The user can input commands related to the display device (100) to the remote control device (200) by operating the user input interface (230). If the user input interface (230) includes a hard key button, the user can input commands related to the display device (100) to the remote control device (200) by pushing the hard key button. This will be described with reference to FIG. 3.

[0073] Referring to FIG. 3, the remote control device (200) may include a plurality of buttons. The plurality of buttons may include a fingerprint recognition button (212), a power button (231), a home button (232), a live button (233), an external input button (234), a volume control button (235), a voice recognition button (236), a channel change button (237), a confirmation button (238), and a back button (239).

[0074] The fingerprint recognition button (212) may be a button for recognizing a user's fingerprint. In one embodiment, the fingerprint recognition button (212) may be capable of a push operation, and may receive a push operation and a fingerprint recognition operation.

[0075] The power button (231) may be a button for turning the power of the display device (100) on / off.

[0076] The home button (232) may be a button for moving to the home screen of the display device (100).

[0077] The live button (233) may be a button for displaying a real-time broadcast program.

[0078] The external input button (234) may be a button for receiving an external input connected to the display device (100).

[0079] The volume control button (235) may be a button for adjusting the size of the volume output by the display device (100).

[0080] The voice recognition button (236) may be a button for receiving a user's voice and recognizing the received voice.

[0081] The channel change button (237) may be a button for receiving a broadcast signal of a specific broadcast channel.

[0082] The confirmation button (238) may be a button for selecting a specific function, and the back button (239) may be a button for returning to the previous screen.

[0083] Let's explain Figure 2 again.

[0084] When the user input interface (230) has a touch screen, the user can input commands related to the display device (100) using the remote control device (200) by touching the soft keys of the touch screen. In addition, the user input interface (230) may have various types of input means that can be operated by the user, such as a scroll key or a jog key, and this embodiment does not limit the scope of the present invention.

[0085] The sensor (240) may include a gyro sensor (241) or an acceleration sensor (243), and the gyro sensor (241) may sense information about the movement of the remote control device (200).

[0086] For example, the gyro sensor (241) can sense information about the operation of the remote control device (200) based on the x, y, and z axes, and the acceleration sensor (243) can sense information about the movement speed of the remote control device (200). Meanwhile, the remote control device (200) can further include a distance measuring sensor, so as to sense the distance to the display (180) of the display device (100).

[0087] The output interface (250) can output a video or audio signal corresponding to an operation of the user input interface (230) or a signal transmitted from the display device (100).

[0088] The user can recognize whether the output interface (250) is manipulating the user input interface (230) or controlling the display device (100).

[0089] For example, the output interface (250) may include an LED (251) that lights up when the user input interface (230) is operated or a signal is transmitted and received with the display device (100) via the wireless communication unit (225), a vibrator (253) that generates vibrations, a speaker (255) that outputs sound, or a display (257) that outputs images.

[0090] In addition, the power supply circuit (260) supplies power to the remote control device (200), and power waste can be reduced by stopping the power supply when the remote control device (200) does not move for a predetermined period of time.

[0091] The power supply circuit (260) can resume power supply when a predetermined key provided in the remote control device (200) is operated.

[0092] The memory (270) can store various types of programs, application data, etc. required for the control or operation of the remote control device (200).

[0093] When the remote control device (200) wirelessly transmits and receives signals through the display device (100) and the RF circuit (221), the remote control device (200) and the display device (100) transmit and receive signals through a predetermined frequency band.

[0094] The controller (280) of the remote control device (200) can store and reference information regarding the frequency band that can wirelessly transmit and receive signals with the display device (100) paired with the remote control device (200) in the memory (270).

[0095] The controller (280) controls all matters related to the control of the remote control device (200). The controller (280) can transmit a signal corresponding to a predetermined key operation of the user input interface (230) or a signal corresponding to the movement of the remote control device (200) sensed by the sensor (240) to the display device (100) via the wireless communication unit (225).

[0096] Additionally, the microphone (290) of the remote control device (200) can acquire voice.

[0097] A plurality of microphones (290) may be provided.

[0098] Next, Figure 4 is described.

[0099] Figure 4 shows an example of utilizing a remote control device according to an embodiment of the present invention.

[0100] Figure 4 (a) illustrates that a pointer (205) corresponding to a remote control device (200) is displayed on a display (180).

[0101] The user can move or rotate the remote control device (200) up and down, left and right. The pointer (205) displayed on the display (180) of the display device (100) corresponds to the movement of the remote control device (200). This remote control device (200) can be called a space remote control because, as shown in the drawing, the pointer (205) moves and is displayed according to the movement in 3D space.

[0102] Figure 4 (b) illustrates that when a user moves the remote control device (200) to the left, the pointer (205) displayed on the display (180) of the display device (100) also moves to the left correspondingly.

[0103] Information about the movement of the remote control device (200) detected through the sensor of the remote control device (200) is transmitted to the display device (100). The display device (100) can calculate the coordinates of the pointer (205) from the information about the movement of the remote control device (200). The display device (100) can display the pointer (205) to correspond to the calculated coordinates.

[0104] Figure 4 (c) illustrates a case where a user moves the remote control device (200) away from the display (180) while pressing a specific button within the remote control device (200). As a result, a selection area within the display (180) corresponding to the pointer (205) can be zoomed in and displayed in an enlarged manner.

[0105] Conversely, when the user moves the remote control device (200) closer to the display (180), the selection area within the display (180) corresponding to the pointer (205) may be zoomed out and displayed in a reduced size.

[0106] Meanwhile, when the remote control device (200) moves away from the display (180), the selection area may be zoomed out, and when the remote control device (200) moves closer to the display (180), the selection area may be zoomed in.

[0107] In addition, when a specific button within the remote control device (200) is pressed, recognition of up, down, left, and right movements may be excluded. That is, when the remote control device (200) moves away from or toward the display (180), up, down, left, and right movements may not be recognized, and only forward and backward movements may be recognized. When a specific button within the remote control device (200) is not pressed, only the pointer (205) moves in accordance with the up, down, left, and right movements of the remote control device (200).

[0108] Meanwhile, the movement speed or movement direction of the pointer (205) can correspond to the movement speed or movement direction of the remote control device (200).

[0109] Meanwhile, the pointer in this specification refers to an object displayed on the display (180) in response to the operation of the remote control device (200). Accordingly, objects of various shapes other than the arrow shape illustrated in the drawing can be used as the pointer (205). For example, the pointer may be a concept including a point, a cursor, a prompt, a thick outline, etc. In addition, the pointer (205) may be displayed corresponding to one point on the horizontal or vertical axis on the display (180), or may be displayed corresponding to multiple points such as lines or surfaces.

[0110] FIG. 5 illustrates an artificial intelligence (AI) server according to one embodiment of the present disclosure.

[0111] Referring to FIG. 5, the AI ​​server (500) may refer to a device that trains an artificial neural network using a machine learning algorithm or uses a trained artificial neural network.

[0112] The AI ​​server (500) may be composed of multiple servers to perform distributed processing, and may be defined as a 5G network.

[0113] The AI ​​server (500) may be included as part of the configuration of the AI ​​device (100) and may perform at least part of the AI ​​processing together.

[0114] The AI ​​server (500) may include a communication unit (510), a memory (530), a learning processor (540), and a processor (560).

[0115] The communication unit (510) can transmit and receive data with an external device such as a display device (100).

[0116] The memory (530) may include a model storage unit (531).

[0117] The model storage unit (531) can store a model (or artificial neural network, 531a) that is being learned or has been learned through the learning processor (540).

[0118] A learning processor (540) can train an artificial neural network (531a) using learning data. The learning model can be used while mounted on the AI ​​server (500) of the artificial neural network, or can be mounted on an external device such as a display device (100).

[0119] The learning model may be implemented in hardware, software, or a combination of hardware and software. If part or all of the learning model is implemented in software, one or more instructions constituting the learning model may be stored in memory (530).

[0120] The processor (560) can use a learning model to infer a result value for new input data and generate a response or control command based on the inferred result value.

[0121] FIG. 6 is a ladder diagram for explaining an operation method of an AI system according to an embodiment of the present disclosure.

[0122] Referring to FIG. 6, the AI ​​system may include a display device (100), an AI server (500), and a first generation AI server (610).

[0123] The AI ​​server (500) may be a natural language processing (NLP) server.

[0124] The controller (170) of the display device (100) can receive voice commands (S601).

[0125] In one embodiment, the controller (170) can receive a voice command spoken by a user through a microphone (not shown).

[0126] The controller (170) can receive a voice command from a remote control device (200). The remote control device (200) can receive a voice command spoken by a user and transmit it to the display device (100).

[0127] The controller (170) of the display device (100) can transmit voice data corresponding to a voice command to the AI ​​server (500) via the network interface (133) (S603).

[0128] The processor (560) of the AI ​​server (500) can obtain the analysis result of the voice command based on the received voice data (S605).

[0129] In one embodiment, the processor (560) can convert voice data into text data using a speech to text (STT) engine stored in the memory (530).

[0130] In another embodiment, the processor (560) can transmit voice data to an STT server and receive converted text data from the STT server.

[0131] The processor (560) can obtain analysis results of the converted text data using a natural language processing engine (NLP).

[0132] The analysis results may include intent analysis results for the intent of the voice command and search results for the intent analysis results.

[0133] For example, voice commands<Tell me the plot of A> In this case, the intent analysis result may be a request to know the plot of drama A, and the search result may be a search result about the plot of drama A.

[0134] The processor (560) of the AI ​​server (500) can determine whether a response from the generated AI is required based on the analysis result of the voice command (S607).

[0135] In one embodiment, the processor (560) may determine whether a response from the generation AI is required based on the intent analysis result of the voice command.

[0136] If the intent analysis result of the voice command is for controlling the function of the display device (100), the processor (560) may determine that a response from the generation AI is not necessary.

[0137] If the intent analysis result of the voice command is not intended to control the function of the display device (100), the processor (560) may determine that a response from the generation AI is not necessary.

[0138] If the intent analysis result of the voice command is not for the purpose of controlling the function of the display device (100), it may be for the purpose of content search.

[0139] For example, if the intent of the voice command is to control a function of the display device (100), such as controlling the screen brightness or volume of the display device (100), the processor (560) may determine that a response from the generation AI is not necessary. This is because the display device (100) can handle this function on its own without the assistance of the generation AI.

[0140] The processor (560) may determine that a response from the generation AI is required if the intent of the voice command is to search for content on the display device (100) or to search for travel itineraries for a specific region. This is because, without the assistance of the generation AI, it is difficult for the display device (100) to perform this function on its own.

[0141] If the processor (560) of the AI ​​server (500) determines that a generated AI response is not necessary, it can transmit the analysis result to the display device (100) via the communication unit (510) (S609).

[0142] The processor (560) can transmit the intent analysis result and the search result corresponding to the intent analysis result to the display device (100) through the communication unit (510).

[0143] The processor (560) can transmit only search results corresponding to the intent analysis results to the display device (100) through the communication unit (510).

[0144] The controller (170) of the display device (100) can output the received analysis results (S611).

[0145] The controller (170) can display the search results included in the analysis results on the display (180).

[0146] If the processor (560) of the AI ​​server (500) determines that a response from the generation AI is required, it can generate a prompt for a response from the generation AI (S613).

[0147] The processor (560) can generate a prompt to obtain a response result of a generation AI model equipped in the generation AI server.

[0148] Generative AI models can be models that generate content in response to commands, using deep learning or machine learning.

[0149] A prompt can be a text command that is input to a generative AI model.

[0150] A prompt can be a command that causes a natural language generation AI model to generate natural language.

[0151] The processor (560) can generate a prompt based on the intent analysis result of the voice command.

[0152] The processor (560) can generate a prompt requesting content by combining keywords included in the intent analysis result of the voice command.

[0153] Meanwhile, if the processor (560) determines that a response from the generation AI is required, it can determine a display format in which to display the analysis results and the first response results.

[0154] The display format may be a format indicating one or more of the location, size, and arrangement of the area in which the analysis results and response results of the AI ​​server (500) are to be displayed.

[0155] The processor (560) can determine one of a plurality of display formats depending on the type of the analysis result and the type of the response result, and can transmit information about the determined display format to the display device (100).

[0156] Here, type is the type of content, which can be any of text, image, video, or audio.

[0157] The processor (560) of the AI ​​server (500) can transmit a prompt to the first generation AI server (610) through the communication unit (510) (S615) and receive a first response result from the first generation AI server (610) (S617).

[0158] In one embodiment, the first generation AI server (610) may be a server that provides search results for content.

[0159] In another embodiment, the first generation AI server (610) may be a server having a natural language generation AI model that outputs natural language.

[0160] A natural language generation AI model can generate a first response result in response to a received prompt.

[0161] Natural language generation AI models can be trained using deep learning architectures such as recurrent neural networks (RNNs) and transformation models using attention mechanisms through large-scale text data.

[0162] Data for training natural language generation AI models can be collected from the web or various text sources.

[0163] A natural language generation AI model can take a received prompt as input and generate one or more sentences containing content corresponding to the intent analysis results.

[0164] The first generation AI server (610) can obtain one or more generated sentences as a first response result and transmit the obtained first response result to the AI ​​server (500).

[0165] The processor (560) of the AI ​​server (500) can transmit the analysis result of the voice command and the first response result to the display device (100) through the communication unit (510) (S619).

[0166] In one embodiment, the processor (560) can sequentially transmit the analysis result and the first response result to the display device (100) through the communication unit (510).

[0167] That is, the processor (560) may first transmit the analysis result to the display device (100) and then transmit the first response result to the display device (100) later. This is because the response time of the first generation AI server (610) may be required.

[0168] In another embodiment, the processor (560) can simultaneously transmit the analysis result and the first response result to the display device (100) through the communication unit (510).

[0169] Meanwhile, the processor (560) may filter some of the analysis results and the first response results and transmit the filtered information to the display device (100). Some of the filtered information may include information that is sexually suggestive or contains profanity.

[0170] The controller (170) of the display device (100) can output the received analysis results and the first response results (S621).

[0171] The controller (170) can display the analysis results and the first response results on the display (180).

[0172] The controller (170) can display the analysis results and the first response results on the display (180) according to the display format received from the AI ​​server (500).

[0173] FIG. 7a is a drawing illustrating a process of providing an analysis result of an AI server based on a user's voice command according to a conventional technology, and FIGS. 7b to 7d are drawings illustrating a process of providing an analysis result of an AI server based on a user's voice command and a first response result of a first generation AI server according to an embodiment of the present disclosure.

[0174] Referring to FIG. 7a, the display device (100) is playing a content image (700).

[0175] The user presses the microphone button provided on the remote control device (200),<Tell me the plot of A> You can utter a voice command called .

[0176] The remote control device (200) can transmit the received voice command to the display device (100).

[0177] The display device (100) can transmit voice data corresponding to a voice command to the AI ​​server (500) and receive a first plot search result (710) for the plot of A content included in the analysis result of the voice command from the AI ​​server (500).

[0178] The display device (100) can display the first plot search result (710) on the display (180).

[0179] The display device (100) can display the first plot search result (710) by overlaying it on the content image (700).

[0180] Referring to FIG. 7b, the display device (100) is playing a content image (700).

[0181] The user presses the microphone button provided on the remote control device (200),<Tell me the plot of A> You can utter a voice command called .

[0182] The remote control device (200) can transmit the received voice command to the display device (100).

[0183] The display device (100) can transmit voice data corresponding to a voice command to the AI ​​server (500) and receive a first plot search result (710) for the plot of A content included in the analysis result of the voice command from the AI ​​server (500).

[0184] If the AI ​​server (500) determines that the intent analysis result of the voice command is related to content search, it can generate a prompt requesting a synopsis search of content having the name A.

[0185] The AI ​​server (500) may transmit a prompt requesting a plot search of content having a name A to the first generation AI server (610).

[0186] The first generation AI server (610) can input the received prompt into the natural language generation AI model to generate a second plot search result including the plot of content A.

[0187] The first generation AI server (610) can transmit the second plot search result to the AI ​​server (500), and the AI ​​server (500) can transmit the second plot search result (730) to the display device (100).

[0188] The display device (100) can display the first plot search result (710) and the second plot search result (730) on the display (180).

[0189] The display device (100) can display the plot search results (710) by overlaying them on the content image (700).

[0190] The display device (100) can display the first plot search result (710) and the second plot search result (730) on the display (180) according to the display format received from the AI ​​server (500). Here, the display format may be a format in which the first plot search result (710) is displayed on the lower side of the display screen, and the second plot search result (730) is displayed on the upper side of the display screen.

[0191] In this way, according to an embodiment of the present disclosure, an external server service that provides information received from an NLP service of a manufacturer of a display device (100) and an externally generated AI according to a user's speech intention may be utilized.

[0192] User experience can be improved by providing a variety of response results that match the user's speech intent.

[0193] Figure 7c shows a user on the home screen or content provision screen.<Recommend comedy movie> Here is an example of a case where a voice command called .

[0194] The display device (100) can display the first search result (751) of the comedy movie received from the AI ​​server (500) and the second search result (753) of the comedy movie received from the first generation AI server (610) on the screen (750) of the display (180).

[0195] The AI ​​server (500) can generate a prompt by reflecting the intent analysis result of the voice command (comedy movie recommendation) and the user's preference information.

[0196] The user's preference information may include one or more of a preferred genre, preferred actor, or preferred director. The user's preference information may be collected by having the display device (100) transmit the information to the AI ​​server (500) in advance.

[0197] For example, a prompt might include a command to search for comedy movies in the sports genre.

[0198] The AI ​​server (500) can transmit the generated prompt to the first generation AI server (610) and receive a second search result (753) for a comedy movie of the sports genre from the first generation AI server (610).

[0199] Each of the first search result (751) and the second search result (753) may include thumbnail images of one or more comedy movies.

[0200] The screen (750) may further include feedback text (755) regarding the second search result (753). The AI ​​server (500) may transmit the second search result (753) and the feedback text (755) explaining the search intent of the second search result (753) together to the display device (100).

[0201] Meanwhile, the display device (100) can display the first search result (751) and the second search result (753) in a half view according to the display format received from the AI ​​server (500), and display the feedback text (755) at the bottom of the screen (750).

[0202] Figure 7d shows a user on the home screen or content provision screen.<Recommend 3-day trip Schedule for Rome> Here is an example of a case where a voice command called .

[0203] The display device (100) can display the third search result (771) for a travel destination in Rome received from the AI ​​server (500) and the fourth search result (773) for a travel destination in Rome received from the first generation AI server (610) on the screen (750) of the display (180).

[0204] The third search result (771) may include a plurality of videos containing information about the Rome travel destination or a URL address for each of the plurality of videos.

[0205] The AI ​​server (500) can generate a prompt by reflecting the intent analysis results of the voice command (recommended Rome travel destination) and the third search result (771). Here, the prompt may include a command requesting a travel itinerary recommendation using multiple videos about the Rome travel destination.

[0206] The AI ​​server (500) can transmit the generated prompt to the first generation AI server (610) and receive a fourth search result (773) indicating a three-day Rome travel itinerary from the first generation AI server (610).

[0207] The fourth search result (773) may include travel destination schedule information for three days.

[0208] The screen (770) may further include feedback text (775) for the fourth search result (773).

[0209] Meanwhile, the display device (100) can display the third search result (771) and the fourth search result (773) in a horizontal full view according to the display format received from the AI ​​server (500), and display the feedback text (775) at the top of the screen (770).

[0210] FIG. 8 is a ladder diagram for explaining an operation method of an AI system according to another embodiment of the present disclosure.

[0211] Referring to FIG. 8, the AI ​​system may include a display device (100), an AI server (500), a first generation AI server (610), and a second generation AI server (630).

[0212] The AI ​​server (500) may be a natural language processing (NLP) server.

[0213] The controller (170) of the display device (100) can receive voice commands (S801).

[0214] In one embodiment, the controller (170) can receive a voice command spoken by a user through a microphone (not shown).

[0215] The controller (170) can receive a voice command from a remote control device (200). The remote control device (200) can receive a voice command spoken by a user and transmit it to the display device (100).

[0216] The controller (170) of the display device (100) can transmit voice data corresponding to a voice command to the AI ​​server (500) via the network interface (133) (S803).

[0217] The processor (560) of the AI ​​server (500) can obtain the analysis result of the voice command based on the received voice data (S805).

[0218] In one embodiment, the processor (560) can convert voice data into text data using a speech to text (STT) engine stored in the memory (530).

[0219] In another embodiment, the processor (560) can transmit voice data to an STT server and receive converted text data from the STT server.

[0220] The processor (560) can obtain analysis results of the converted text data using a natural language processing engine (NLP).

[0221] The analysis results may include intent analysis results for the intent of the voice command and search results for the intent analysis results.

[0222] The processor (560) of the AI ​​server (500) can determine whether a response from the generated AI is required based on the analysis result of the voice command (S807).

[0223] In one embodiment, the processor (560) may determine whether a response from the generation AI is required based on the intent analysis result of the voice command.

[0224] If the intent analysis result of the voice command is for controlling the function of the display device (100), the processor (560) may determine that a response from the generation AI is not necessary.

[0225] If the intent analysis result of the voice command is not intended to control the function of the display device (100), the processor (560) may determine that a response from the generation AI is not necessary.

[0226] If the intent analysis result of the voice command is not for the purpose of controlling the function of the display device (100), it may be for the purpose of content search.

[0227] For example, if the intent of the voice command is to control a function of the display device (100), such as controlling the screen brightness or volume of the display device (100), the processor (560) may determine that a response from the generation AI is not necessary. This is because the display device (100) can handle this function on its own without the assistance of the generation AI.

[0228] The processor (560) may determine that a response from the generation AI is required if the intent of the voice command is to search for content on the display device (100) or to search for travel itineraries for a specific region. This is because, without the assistance of the generation AI, it is difficult for the display device (100) to perform this function on its own.

[0229] If the processor (560) of the AI ​​server (500) determines that a generated AI response is not necessary, it can transmit the analysis result to the display device (100) via the communication unit (510) (S809).

[0230] The processor (560) can transmit the intent analysis result and the search result corresponding to the intent analysis result to the display device (100) through the communication unit (510).

[0231] The processor (560) can transmit only search results corresponding to the intent analysis results to the display device (100) through the communication unit (510).

[0232] The controller (170) of the display device (100) can output the received analysis results (S811).

[0233] The controller (170) can display the search results included in the analysis results on the display (180).

[0234] If the processor (560) of the AI ​​server (500) determines that a response from the generation AI is required, it can generate a prompt for a response from the first generation AI server (610) (S813).

[0235] The processor (560) can generate a prompt to obtain a response result of the generation AI model provided in the first generation AI server (610).

[0236] Generative AI models can use deep learning or machine learning to generate content in response to commands. Commands can include one or more of text or images.

[0237] The processor (560) can generate a prompt based on the intent analysis result of the voice command.

[0238] The processor (560) can generate a prompt requesting content by combining keywords included in the intent analysis result of the voice command.

[0239] The processor (560) of the AI ​​server (500) can transmit a prompt to the first generation AI server (610) through the communication unit (510) (S815) and receive a first response result from the first generation AI server (610) (S817).

[0240] In one embodiment, the first generation AI server (610) may be a server that provides search results for content.

[0241] In another embodiment, the first generation AI server (610) may be a server having a natural language generation AI model that outputs natural language.

[0242] A natural language generation AI model can generate a first response result in response to a received prompt.

[0243] Natural language generation AI models can be trained using deep learning architectures such as recurrent neural networks (RNNs) and transformation models using attention mechanisms through large-scale text data.

[0244] Data for training natural language generation AI models can be collected from the web or various text sources.

[0245] A natural language generation AI model can take a received prompt as input and generate one or more sentences containing content corresponding to the intent analysis results.

[0246] The first generation AI server (610) can obtain one or more generated sentences as a first response result and transmit the obtained first response result to the AI ​​server (500).

[0247] The processor (560) of the AI ​​server (500) can transmit a prompt to the second generation AI server (630) through the communication unit (510) (S819) and receive a second response result from the second generation AI server (630) (S821).

[0248] That is, the processor (560) can transmit the same prompt to the first generation AI server (610) and the second generation AI server (630).

[0249] The second generation AI server (630) may be a server that provides search results for content.

[0250] In another embodiment, the second generation AI server (630) may be a server equipped with a natural language generation AI model that outputs natural language.

[0251] The second generation AI server (630) may be a server equipped with the same type of generation AI model as the first generation AI server (610).

[0252] The second generation AI server (630) can obtain one or more sentences as a second response result for the prompt and transmit the obtained second response result to the AI ​​server (500).

[0253] The processor (560) of the AI ​​server (500) can generate a third response result based on the first response result and the second response result (S823).

[0254] In one embodiment, the processor (560) may combine the first response result and the second response result to generate a third response result.

[0255] In another embodiment, the processor (560) can obtain the remaining contents excluding the overlapping portions in the first response result and the second response result as the third response result.

[0256] The processor (560) of the AI ​​server (500) can transmit the third response result to the display device (100) through the communication unit (510) (S825).

[0257] In one embodiment, the processor (560) may sequentially transmit the analysis result and the third response result to the display device (100) via the communication unit (510). That is, the processor (560) may first transmit the analysis result to the display device (100) and then transmit the third response result to the display device (100) later. This is because the response time of the first generation AI server (610) and the second generation AI server (630) may be required.

[0258] In another embodiment, the processor (560) can simultaneously transmit the analysis result and the third response result to the display device (100) through the communication unit (510).

[0259] Meanwhile, if the processor (560) determines that a response from the generation AI is required, it can determine a display format in which to display the analysis results and the third response results.

[0260] The display format may be a format that indicates the location, size, arrangement, etc. of the area where the analysis results and third response results of the AI ​​server (500) are to be displayed.

[0261] The processor (560) can determine one of a plurality of display formats depending on the type of the analysis result and the type of the third response result, and can transmit information about the determined display format to the display device (100).

[0262] Here, the type is the type of content included in the analysis results, which can be any of text, image, video, or audio.

[0263] The controller (170) of the display device (100) can output the received third response result (S827).

[0264] The controller (170) can display the analysis results and the third response results on the display (180).

[0265] The controller (170) can display the analysis results and the third response results on the display (180) according to the display format received from the AI ​​server (500).

[0266] FIG. 9 is a diagram illustrating a process of providing analysis results of an AI server based on a user's voice command and first and second response results provided from each of the first and second generation AI servers according to an embodiment of the present disclosure.

[0267] Referring to FIG. 9, the display device (100) is playing a content image (700).

[0268] The user presses the microphone button provided on the remote control device (200),<Tell me the plot of A> You can utter a voice command called .

[0269] The remote control device (200) can transmit the received voice command to the display device (100).

[0270] The display device (100) can transmit voice data corresponding to a voice command to the AI ​​server (500) and receive a first plot search result (710) for the plot of A content included in the analysis result of the voice command from the AI ​​server (500).

[0271] If the AI ​​server (500) determines that the intent analysis result of the voice command is related to content search, it can generate a prompt requesting a synopsis search of content having the name A.

[0272] The AI ​​server (500) may transmit a prompt requesting a plot search of content having a name A to each of the first generation AI server (610) and the second generation AI server (630).

[0273] Each of the first generation AI server (610) and the second generation AI server (630) may be servers operated by different operating entities.

[0274] The first generation AI server (610) can input the received prompt into the natural language generation AI model to generate a second plot search result (910) including the plot of content A.

[0275] The first generation AI server (610) can transmit the second plot search result (910) to the AI ​​server (500).

[0276] The second generation AI server (630) can input the received prompt into the natural language generation AI model to generate a third plot search result (930) including the plot of content A.

[0277] The second generation AI server (630) can transmit the third plot search result (930) to the AI ​​server (500).

[0278] The AI ​​server (500) can transmit the second plot search result (910) and the third plot search result (930) to the display device (100).

[0279] The display device (100) can display the first plot search result (710), the second plot search result (910), and the third plot search result (930) on the display (180).

[0280] The search results (900) of the generated AI including the second plot search results (910) and the third plot search results (930) may be displayed above the first plot search results (710) depending on the display format.

[0281] In this way, according to an embodiment of the present disclosure, an NLP service of the manufacturer of the display device (100) and an external server service that provides information received from a plurality of externally generated AI servers according to the user's speech intention may also be utilized.

[0282] FIG. 10 is a ladder diagram for explaining an operation method of an AI system according to another embodiment of the present disclosure.

[0283] Referring to FIG. 10, the AI ​​system may include a display device (100), an AI server (500), a first generation AI server (610), and a second generation AI server (630).

[0284] The AI ​​server (500) may be a natural language processing (NLP) server.

[0285] The controller (170) of the display device (100) can receive voice commands (S1001).

[0286] In one embodiment, the controller (170) can receive a voice command spoken by a user through a microphone (not shown).

[0287] The controller (170) can receive a voice command from a remote control device (200). The remote control device (200) can receive a voice command spoken by a user and transmit it to the display device (100).

[0288] The controller (170) of the display device (100) can transmit voice data corresponding to a voice command to the AI ​​server (500) through the network interface (133) (S1003).

[0289] The processor (560) of the AI ​​server (500) can obtain the analysis result of the voice command based on the received voice data (S1005).

[0290] In one embodiment, the processor (560) can convert voice data into text data using a speech to text (STT) engine stored in the memory (530).

[0291] In another embodiment, the processor (560) can transmit voice data to an STT server and receive converted text data from the STT server.

[0292] The processor (560) can obtain analysis results of the converted text data using a natural language processing engine (NLP).

[0293] The analysis results may include intent analysis results for the intent of the voice command and search results for the intent analysis results.

[0294] The processor (560) of the AI ​​server (500) can determine whether a response from the generated AI is required based on the analysis result of the voice command (S1007).

[0295] In one embodiment, the processor (560) may determine whether a response from the generation AI is required based on the intent analysis result of the voice command.

[0296] If the intent analysis result of the voice command is for controlling the function of the display device (100), the processor (560) may determine that a response from the generation AI is not necessary.

[0297] If the intent analysis result of the voice command is not intended to control the function of the display device (100), the processor (560) may determine that a response from the generation AI is not necessary.

[0298] If the intent analysis result of the voice command is not for the purpose of controlling the function of the display device (100), it may be for the purpose of content search.

[0299] For example, if the intent of the voice command is to control a function of the display device (100), such as controlling the screen brightness or volume of the display device (100), the processor (560) may determine that a response from the generation AI is not necessary. This is because the display device (100) can handle this function on its own without the assistance of the generation AI.

[0300] The processor (560) may determine that a response from the generation AI is required if the intent of the voice command is to search for content on the display device (100) or to search for travel itineraries for a specific region. This is because, without the assistance of the generation AI, it is difficult for the display device (100) to perform this function on its own.

[0301] If the processor (560) of the AI ​​server (500) determines that a generated AI response is not necessary, it can transmit the analysis result to the display device (100) via the communication unit (510) (S1009).

[0302] The processor (560) can transmit the intent analysis result and the search result corresponding to the intent analysis result to the display device (100) through the communication unit (510).

[0303] The processor (560) can transmit only search results corresponding to the intent analysis results to the display device (100) through the communication unit (510).

[0304] The controller (170) of the display device (100) can output the received analysis results (S1011).

[0305] The controller (170) can display the search results included in the analysis results on the display (180).

[0306] If the processor (560) of the AI ​​server (500) determines that a response from the generation AI is required, it can generate a first prompt for a response from the first generation AI server (610) (S1013).

[0307] The processor (560) can generate a first prompt to obtain a response result of the generation AI model provided in the first generation AI server (610).

[0308] Generative AI models can be models that use deep learning or machine learning to generate content in response to commands. Commands can include one or more of text or images.

[0309] The processor (560) can generate a first prompt based on the intent analysis result of the voice command.

[0310] The processor (560) can generate a first prompt requesting content by combining keywords included in the intent analysis result of the voice command.

[0311] The processor (560) of the AI ​​server (500) can transmit a prompt to the first generation AI server (610) through the communication unit (510) (S1015) and receive a first response result from the first generation AI server (610) (S1017).

[0312] In one embodiment, the first generation AI server (610) may be a server that provides search results for content.

[0313] In another embodiment, the first generation AI server (610) may be a server having a natural language generation AI model that outputs natural language.

[0314] A natural language generation AI model can generate a first response result in response to a received prompt.

[0315] Natural language generation AI models can be trained using deep learning architectures such as recurrent neural networks (RNNs) and transformation models using attention mechanisms through large-scale text data.

[0316] Data for training natural language generation AI models can be collected from the web or various text sources.

[0317] A natural language generation AI model can take a received prompt as input and generate one or more sentences containing content corresponding to the intent analysis results.

[0318] The first generation AI server (610) can obtain one or more generated sentences as a first response result and transmit the obtained first response result to the AI ​​server (500).

[0319] The processor (560) of the AI ​​server (500) can determine whether a response from another generation AI is required based on the received first response result (S1019).

[0320] In one embodiment, the processor (560) may determine that a response from another generation AI is required if the first response result only includes a specific type of information. For example, if the first response result only includes text information, the processor (560) may determine that a response from another generation AI is required to obtain image information.

[0321] In another embodiment, the processor (560) may determine that a response from another generation AI is required if an additional voice command from the user is received after providing the first response result to the display device (100).

[0322] If the first generation AI server (610) cannot provide a response to the analysis results for the additional voice command, the processor (560) may determine that a response from another generation AI is required.

[0323] If the processor (560) of the AI ​​server (500) determines that a response from another generated AI is not necessary, it can transmit the analysis result of the voice command and the first response result to the display device (100) through the communication unit (510) (S1021).

[0324] The controller (170) of the display device (100) can output the received analysis results and the first response results (S1023).

[0325] Meanwhile, if the processor (560) of the AI ​​server (500) determines that a response from another generation AI is required, it can generate a second prompt based on the first response result (S1025).

[0326] The processor (560) can generate a second prompt by combining the first response result and the analysis result of the user's additional voice command.

[0327] For example, the second prompt may include a command requesting the analysis results of an image representing the first response result and the analysis results of an additional voice command.

[0328] The processor (560) of the AI ​​server (500) can transmit a second prompt to the second generation AI server (630) through the communication unit (510) (S1027) and receive a second response result from the second generation AI server (630) (S1029).

[0329] The second generation AI server (630) may be a server equipped with a generation AI model that provides image analysis results.

[0330] The second generation AI server (630) can obtain one or more sentences as a second response result for the prompt and transmit the obtained second response result to the AI ​​server (500).

[0331] The processor (560) of the AI ​​server (500) can transmit the second response result to the display device (100) via the communication unit (510) (S1031).

[0332] The controller (170) of the display device (100) can output the received second response result (S827).

[0333] The controller (170) can display the second response result on the display (180).

[0334] FIGS. 11A to 11C are diagrams illustrating a process for providing a response result of another generation AI when a response result of another generation AI is required according to an embodiment of the present disclosure.

[0335] Figures 11a to 11c show examples of the embodiment of Figure 10.

[0336] Referring to Figure 11a, the user<Draw a picture of a cat on a piano> The first voice command is spoken to the remote control device (200).

[0337] The remote control device (200) can transmit a first voice command to the display device (100), and the display device (100) can transmit first voice data corresponding to the first voice command to the AI ​​server (500).

[0338] The AI ​​server (500) can convert first voice data into first text data and obtain analysis results of the converted first text data.

[0339] The AI ​​server (500) can transmit a plurality of images representing search results for the first voice command to the display device (100).

[0340] The display device (100) can display a plurality of search images (1110) and a text phrase indicating that the first response result of the first generation AI server (610) is waiting on the screen (1100).

[0341] The AI ​​server (500) may generate a first prompt based on the analysis result for the first voice command and transmit the generated first prompt to the first generation AI server (610). The first prompt may include a command to draw a picture of a cat on a piano.

[0342] The first generation AI server (610) can generate a plurality of response images representing <cat on piano> according to the first prompt, and transmit the generated plurality of response images (first response result) to the AI ​​server (500).

[0343] The display device (100) can display a plurality of search images (1110) and a plurality of response images (1130) on a screen (1100).

[0344] After this, the user<Set the second image on the TV> A second voice command called “” can be additionally spoken to the remote control device (200).

[0345] The remote control device (200) can transmit a second voice command to the display device (100).

[0346] The display device (100) can transmit second voice data corresponding to the second voice command to the AI ​​server (500).

[0347] The AI ​​server (500) may determine that a response from another generation AI is required based on the analysis results of the second voice data. The analysis results of the second voice data may include an intent to generate a title for the image.

[0348] The AI ​​server (500) may generate a second prompt based on the analysis results of the second voice data (setting a title for the second image) and the first response results (multiple response images). The second prompt may include a command requesting the setting of a second image included in the multiple response images and a title for the image.

[0349] The AI ​​server (500) can transmit the second prompt to the second generation AI server (630), and the second generation AI server (630) can generate a title of the image based on the second prompt.

[0350] The second generation AI server (630) can transmit the title of the generated image to the AI ​​server (500), and the AI ​​server (500) can transmit the title of the image to the display device (100).

[0351] The display device (100) can display a second image (1151) and text (1153) including the title of the image (black cat playing the piano) on the display (180).

[0352] In this way, according to an embodiment of the present disclosure, various services can be provided to the user by combining the response results of externally generated AI servers to the voice commands uttered by the user.

[0353] According to one embodiment of the present invention, the above-described method can be implemented as processor-readable code on a medium in which a program is recorded. Examples of processor-readable media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices.

[0354] The display device described above is not limited to the configuration and method of the embodiments described above, and the embodiments may be configured by selectively combining all or part of each embodiment so that various modifications can be made.

Claims

1. In artificial intelligence devices, A communication unit for communicating with an electronic device and one or more artificial intelligence (AI) servers; and A processor that receives a voice command uttered by a user from the electronic device, obtains an analysis result of the received voice command, and, if it is determined that a response from one or more generation AI servers is required based on the obtained analysis result, generates a prompt based on the analysis result, transmits the generated prompt to the one or more generation AI servers, receives a response result corresponding to the prompt from the one or more generation AI servers, and transmits the analysis result and the response result to the electronic device. Artificial intelligence device.

2. In paragraph 1, The above processor The above analysis results are first transmitted to the electronic device, and then the response results are later transmitted to the electronic device. Artificial intelligence device.

3. In paragraph 1, The above processor If the intent analysis result of the above voice command is an intent to control the function of the display device, it is determined that a response from one or more of the generating AI servers is not necessary. If the intent analysis result of the above voice command is not intended to control the function of the display device, it is determined that a response from one or more of the generating AI servers is required. Artificial intelligence device.

4. In paragraph 1, The above processor Generating the prompt based on the analysis results and the user's preference information Artificial intelligence device.

5. In paragraph 1, The above analysis results are Including the intent analysis results and search results corresponding to the intent analysis results, The above processor Generating the prompt based on the above intent analysis results and the above search results Artificial intelligence device.

6. In paragraph 1, The above processor Send the above prompt to each of the first generation AI server and the second generation AI server, Receiving a first response result from the first generation AI server and receiving a second response result from the second generation AI server Artificial intelligence device.

7. In paragraph 6, The above processor Based on the above analysis results, the first response result and the second response result, a third response result is generated, and the generated third response result is transmitted to the electronic device. Artificial intelligence device.

8. In paragraph 6, The above processor The above analysis results, the first response results and the second response results are transmitted to the electronic device. Artificial intelligence device.

9. In paragraph 1, The above processor If it is determined that a response from the first generation AI server is required based on the acquired analysis result, a first prompt is generated based on the analysis result, the generated first prompt is transmitted to the first generation AI server, and a first response result is received from the first generation AI server; and if it is determined that a response from the second generation AI server is required, a second prompt is generated based on the first response result, the second prompt is transmitted to the second generation AI server, and a second response result is received from the second generation AI server. Artificial intelligence device.

10. In paragraph 9, The above processor Receive an additional voice command from the electronic device, and determine whether a response from the second generation AI server is required based on the analysis result of the additional voice command. Artificial intelligence device.

11. In paragraph 10, The above processor If the first generation AI server cannot provide a response to the analysis result of the additional voice command based on the analysis result of the additional voice command, it is determined that a response from the second generation AI server is required. Artificial intelligence device.

12. In paragraph 11, The above processor Generating the second prompt based on the analysis result of the first response result and the additional voice command. Artificial intelligence device.

13. In paragraph 1, The above processor Transmitting information about the display format of the analysis results and the response results to the electronic device. Artificial intelligence device.

14. In paragraph 13, The above display format is A format that includes at least one of the position, size, or arrangement of the area to display the analysis results and the response results. Artificial intelligence device.

15. In the method of operating an artificial intelligence device, A step of receiving a voice command spoken by a user from an electronic device; A step of obtaining analysis results of a received voice command; A step of generating a prompt based on the analysis results, when it is determined that a response from one or more generation AI servers is required based on the acquired analysis results; A step of transmitting the generated prompt to one or more generated AI servers; A step of receiving a response result corresponding to the prompt from the one or more generating AI servers; and A step of transmitting the analysis result and the response result to the electronic device. How an artificial intelligence device operates.

Images