Display device and operation method therefor

Abstract

A display device according to an embodiment disclosed herein may comprise: a memory; a display; a network interface for communicating with a server; and a controller for acquiring user input data, transmitting the acquired user input data to the server via the network interface, receiving an analysis result of the user input data and service non-support information from the server, and storing the user input data and the analysis result in the memory in response to receiving the service non-support information.

Description

Display device and method of operation thereof

[0001] The present disclosure relates to a display device, and more particularly, to a display device for providing a target notification to a user.

[0002] Digital TV services utilizing wired or wireless networks are becoming more widespread. Digital TV services can offer a variety of services not available with existing analog broadcasting services.

[0003] For example, IPTV (Internet Protocol Television), a type of digital TV service, and smart TV services offer interactivity, allowing users to actively choose the type of program they want to watch and when. Building on this interactivity, IPTV and smart TV services can also offer a variety of additional services, such as internet search, home shopping, and online games.

[0004] Existing TVs provide content recommendation services based on the user's usage patterns, age group, and viewing history.

[0005] However, when users input voice or text, a search failure screen is displayed for unsupported functions or unprovided content, or only recommendations for similar content are provided.

[0006] Additionally, users receive information, such as emails or pop-up notifications, when using certain services through subscriptions. Typically, these services are not tailored to individual users, but are delivered to all subscribers.

[0007] These notifications, targeted at an unspecified number of people, often contain information that is not relevant to individuals. Therefore, it's necessary to tailor notifications to users based on these characteristics.

[0008] The purpose of this disclosure is to provide targeted notifications when a function or content of an unsupported display device becomes supported in the future upon user input.

[0009] The purpose of this disclosure is to provide targeted notifications to provide customized services to users.

[0010] A display device according to an embodiment of the present disclosure may include a memory, a display, a network interface for communicating with a server, and a controller for obtaining user input data, transmitting the obtained user input data to the server through the network interface, receiving an analysis result of the user input data and service non-support information from the server, and storing the user input data and the analysis result in the memory upon receiving the service non-support information.

[0011] A method of operating a display device according to an embodiment of the present disclosure may include a step of obtaining user input data, a step of transmitting the obtained user input data to a server, a step of receiving an analysis result of the user input data and service non-support information from the server, and a step of storing the user input data and the analysis result upon receiving the service non-support information.

[0012] According to an embodiment of the present disclosure, even if a desired function of a display device (100) is not currently supported, the user can be notified of the future availability of the desired function through a targeted notification. In other words, the usability of targeted notifications can be maximized by providing customized service notifications.

[0013] According to embodiments of the present disclosure, even if a user's desired content is not currently supported, they can be notified of future availability through targeted notifications. In other words, the usability of targeted notifications can be maximized by providing customized service notifications.

[0014] Figure 1 is a block diagram illustrating the configuration of a display device according to one embodiment of the present invention.

[0015] Figure 2 is a block diagram of a remote control device according to one embodiment of the present invention.

[0016] Figure 3 shows an example of an actual configuration of a remote control device according to one embodiment of the present invention.

[0017] Figure 4 shows an example of utilizing a remote control device according to an embodiment of the present invention.

[0018] FIG. 5 illustrates an artificial intelligence (AI) server according to one embodiment of the present disclosure.

[0019] FIG. 6 is a ladder diagram for explaining an operation method of a system according to an embodiment of the present disclosure.

[0020] FIG. 7 is a diagram illustrating user log information according to one embodiment of the present disclosure.

[0021] FIG. 8 is a diagram illustrating user log information stored when acquisition of analysis results for user input data fails according to one embodiment of the present disclosure.

[0022] FIGS. 9A to 9C are diagrams illustrating a process for providing a target notification when an unsupported function is changed to a supportable state according to an embodiment of the present disclosure.

[0023] FIGS. 10A to 10C are diagrams illustrating a process for providing a target notification when unsupported content is changed to a supportable state according to an embodiment of the present disclosure.

[0024] Hereinafter, embodiments related to the present invention will be described in more detail with reference to the drawings. The suffixes "module" and "part" used in the following description for components are assigned or used interchangeably solely for the convenience of writing the specification, and do not in themselves have distinct meanings or roles.

[0025] A display device according to an embodiment of the present invention is, for example, an intelligent display device that adds computer-assisted functionality to its broadcast reception function. While faithfully performing the broadcast reception function, it can also be equipped with Internet functionality and other features, providing a more user-friendly interface, such as a manual input device, touch screen, or space remote control. Furthermore, with support for wired or wireless Internet functionality, it can connect to the Internet and a computer, enabling functions such as email, web browsing, banking, or gaming. A standardized, general-purpose operating system can be used for these various functions.

[0026] Accordingly, the display device described in the present invention can perform various user-friendly functions, for example, by allowing various applications to be freely added or deleted on a general-purpose operating system kernel. More specifically, the display device can be a network TV, HBB TV, smart TV, LED TV, OLED TV, etc., and in some cases, it can also be applied to smartphones.

[0027] FIG. 1 is a block diagram illustrating the configuration of a display device according to one embodiment of the present invention.

[0028] Referring to FIG. 1, the display device (100) may include a broadcast receiving unit (130), an external device interface (135), a memory (140), a user input interface (150), a controller (170), a wireless communication interface (173), a display (180), a speaker (185), and a power supply circuit (190).

[0029] The broadcast receiving unit (130) may include a tuner (131), a demodulator (132), and a network interface (133).

[0030] The tuner (131) can select a specific broadcast channel according to a channel selection command. The tuner (131) can receive a broadcast signal for the selected specific broadcast channel.

[0031] The demodulator (132) can separate the received broadcast signal into a video signal, an audio signal, and a data signal related to the broadcast program, and can restore the separated video signal, audio signal, and data signal into a form that can be output.

[0032] The external device interface (135) can receive an application or a list of applications within an adjacent external device and transmit it to the controller (170) or memory (140).

[0033] The external device interface (135) can provide a connection path between the display device (100) and the external device. The external device interface (135) can receive one or more of images and audio output from an external device connected wirelessly or wiredly to the display device (100) and transmit them to the controller (170). The external device interface (135) can include a plurality of external input terminals. The plurality of external input terminals can include an RGB terminal, one or more HDMI (High Definition Multimedia Interface) terminals, and a component terminal.

[0034] A video signal of an external device input through an external device interface (135) can be output through a display (180). A voice signal of an external device input through an external device interface (135) can be output through a speaker (185).

[0035] An external device that can be connected to the external device interface (135) may be any one of a set-top box, a Blu-ray player, a DVD player, a game console, a sound bar, a smartphone, a PC, a USB memory, and a home theater, but this is only an example.

[0036] The network interface (133) may provide an interface for connecting the display device (100) to a wired / wireless network including the Internet. The network interface (133) may transmit or receive data to or from other users or other electronic devices via the connected network or another network linked to the connected network.

[0037] Additionally, some content data stored in the display device (100) can be transmitted to a selected user or electronic device among other users or other electronic devices pre-registered in the display device (100).

[0038] The network interface (133) can access a predetermined web page through a connected network or another network linked to the connected network. That is, by accessing a predetermined web page through a network, data can be transmitted or received with the corresponding server.

[0039] In addition, the network interface (133) can receive content or data provided by a content provider or network operator. That is, the network interface (133) can receive content such as movies, advertisements, games, VOD, broadcast signals, etc. and information related thereto provided from a content provider or network provider via a network.

[0040] Additionally, the network interface (133) can receive firmware update information and update files provided by the network operator, and can transmit data to the Internet or content provider or network operator.

[0041] The network interface (133) can select and receive a desired application from among applications open to the public via a network.

[0042] The memory (140) stores a program for each signal processing and control within the controller (170), and can store signal-processed image, voice, or data signals.

[0043] In addition, the memory (140) may perform a function for temporary storage of video, audio, or data signals input from an external device interface (135) or a network interface (133), and may store information about a specific image through a channel memory function.

[0044] The memory (140) can store an application or a list of applications input from an external device interface (135) or a network interface (133).

[0045] The display device (100) can play content files (video files, still image files, music files, document files, application files, etc.) stored in the memory (140) and provide them to the user.

[0046] The user input interface (150) can transmit a signal input by the user to the controller (170) or transmit a signal from the controller (170) to the user. For example, the user input interface (150) can receive and process control signals such as power on / off, channel selection, and screen setting from the remote control device (200) according to various communication methods such as Bluetooth, Ultra Wideband (WB), ZigBee, Radio Frequency (RF) communication, or infrared (IR) communication, or process control signals from the controller (170) to be transmitted to the remote control device (200).

[0047] In addition, the user input interface (150) can transmit control signals input from local keys (not shown) such as a power key, channel key, volume key, and setting value to the controller (170).

[0048] An image signal processed by the controller (170) can be input to the display (180) and displayed as an image corresponding to the image signal. In addition, an image signal processed by the controller (170) can be input to an external output device through an external device interface (135).

[0049] The voice signal processed by the controller (170) can be output as audio to the speaker (185). In addition, the voice signal processed by the controller (170) can be input to an external output device through the external device interface (135).

[0050] In addition, the controller (170) can control the overall operation within the display device (100).

[0051] In addition, the controller (170) can control the display device (100) by a user command or internal program input through the user input interface (150), and can connect to a network to enable the user to download a desired application or application list into the display device (100).

[0052] The controller (170) enables the user-selected channel information, etc. to be output through a display (180) or speaker (185) together with processed video or audio signals.

[0053] In addition, the controller (170) allows a video signal or audio signal from an external device, for example, a camera or camcorder, input through the external device interface (135) to be output through the display (180) or speaker (185) in accordance with an external device video playback command received through the user input interface (150).

[0054] Meanwhile, the controller (170) can control the display (180) to display an image, for example, a broadcast image input through a tuner (131), an external input image input through an external device interface (135), an image input through a network interface, or an image stored in a memory (140) can be controlled to be displayed on the display (180). In this case, the image displayed on the display (180) can be a still image or a moving image, and can be a 2D image or a 3D image.

[0055] In addition, the controller (170) can control the playback of content stored in the display device (100), received broadcast content, or external input content input from outside, and the content can be in various forms such as broadcast video, external input video, audio file, still image, connected web screen, and document file.

[0056] The wireless communication interface (173) can communicate with an external device through wired or wireless communication. The wireless communication interface (173) can perform short-range communication with the external device. To this end, the wireless communication interface (173) can support short-range communication using at least one of Bluetooth™, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies. This wireless communication interface (173) can support wireless communication between the display device (100) and a wireless communication system, between the display device (100) and another display device (100), or between the display device (100) and a network in which the display device (100, or an external server) is located via a short-range wireless communication network (Wireless Area Network). The short-range wireless communication network can be a short-range wireless personal area network (Wireless Personal Area Network).

[0057] Here, the other display device (100) may be a wearable device (e.g., a smartwatch, smart glasses, a head-mounted display (HMD)) or a mobile terminal such as a smart phone that can exchange data with (or be linked to) the display device (100) according to the present invention. The wireless communication interface (173) may detect (or recognize) a wearable device capable of communication around the display device (100).

[0058] Furthermore, if the detected wearable device is a device certified to communicate with the display device (100) according to the present invention, the controller (170) can transmit at least a portion of the data processed in the display device (100) to the wearable device via the wireless communication interface (173). Accordingly, a user of the wearable device can utilize the data processed in the display device (100) via the wearable device.

[0059] The display (180) can generate a driving signal by converting a video signal, data signal, OSD signal processed by the controller (170) or a video signal, data signal, etc. received from an external device interface (135) into R, G, and B signals, respectively.

[0060] Meanwhile, since the display device (100) illustrated in FIG. 1 is merely an embodiment of the present invention, some of the illustrated components may be integrated, added, or omitted depending on the specifications of the display device (100) actually implemented.

[0061] That is, two or more components may be combined into a single component, or a single component may be subdivided into two or more components, as needed. Furthermore, the functions performed by each block are intended to illustrate embodiments of the present invention, and their specific operations or devices do not limit the scope of the present invention.

[0062] According to another embodiment of the present invention, the display device (100) may receive and play back an image through a network interface (133) or an external device interface (135) without having a tuner (131) and a demodulator (132), unlike that shown in FIG. 1.

[0063] For example, the display device (100) may be implemented separately as an image processing device, such as a set-top box, for receiving broadcast signals or contents according to various network services, and a content playback device for playing contents input from the image processing device.

[0064] In this case, the operating method of the display device according to the embodiment of the present invention to be described below may be performed by any one of the display device (100) described with reference to FIG. 1, as well as an image processing device such as the separated set-top box, or a content playback device having a display (180) and a speaker (185).

[0065] Next, a remote control device according to an embodiment of the present invention will be described with reference to FIGS. 2 and 3.

[0066] FIG. 2 is a block diagram of a remote control device according to an embodiment of the present invention, and FIG. 3 shows an example of an actual configuration of a remote control device (200) according to an embodiment of the present invention.

[0067] First, referring to FIG. 2, the remote control device (200) may include a fingerprint recognition device (210), a wireless communication circuit (220), a user input interface (230), a sensor (240), an output interface (250), a power supply circuit (260), a memory (270), a controller (280), and a microphone (290).

[0068] Referring to FIG. 2, the wireless communication circuit (220) transmits and receives signals with any one of the display devices according to the embodiments of the present invention described above.

[0069] The remote control device (200) may be equipped with an RF circuit (221) capable of transmitting and receiving signals with the display device (100) according to RF communication standards, and an IR circuit (223) capable of transmitting and receiving signals with the display device (100) according to IR communication standards. In addition, the remote control device (200) may be equipped with a Bluetooth circuit (225) capable of transmitting and receiving signals with the display device (100) according to Bluetooth communication standards. In addition, the remote control device (200) may be equipped with an NFC circuit (227) capable of transmitting and receiving signals with the display device (100) according to NFC (Near Field Communication) communication standards, and a WLAN circuit (229) capable of transmitting and receiving signals with the display device (100) according to WLAN (Wireless LAN) communication standards.

[0070] In addition, the remote control device (200) transmits a signal containing information about the movement of the remote control device (200) to the display device (100) through a wireless communication circuit (220).

[0071] Meanwhile, the remote control device (200) can receive a signal transmitted by the display device (100) through the RF circuit (221), and, if necessary, can transmit commands for power on / off, channel change, volume change, etc. to the display device (100) through the IR circuit (223).

[0072] The user input interface (230) may be configured as a keypad, buttons, a touchpad, or a touch screen. The user can input commands related to the display device (100) to the remote control device (200) by operating the user input interface (230). If the user input interface (230) includes a hard key button, the user can input commands related to the display device (100) to the remote control device (200) by pushing the hard key button. This will be described with reference to FIG. 3.

[0073] Referring to FIG. 3, the remote control device (200) may include a plurality of buttons. The plurality of buttons may include a fingerprint recognition button (212), a power button (231), a home button (232), a live button (233), an external input button (234), a volume control button (235), a voice recognition button (236), a channel change button (237), a confirmation button (238), and a back button (239).

[0074] The fingerprint recognition button (212) may be a button for recognizing a user's fingerprint. In one embodiment, the fingerprint recognition button (212) may be capable of a push operation, and may receive a push operation and a fingerprint recognition operation.

[0075] The power button (231) may be a button for turning the power of the display device (100) on / off.

[0076] The home button (232) may be a button for moving to the home screen of the display device (100).

[0077] The live button (233) may be a button for displaying a real-time broadcast program.

[0078] The external input button (234) may be a button for receiving an external input connected to the display device (100).

[0079] The volume control button (235) may be a button for adjusting the size of the volume output by the display device (100).

[0080] The voice recognition button (236) may be a button for receiving a user's voice and recognizing the received voice.

[0081] The channel change button (237) may be a button for receiving a broadcast signal of a specific broadcast channel.

[0082] The confirmation button (238) may be a button for selecting a specific function, and the back button (239) may be a button for returning to the previous screen.

[0083] Let's explain Figure 2 again.

[0084] When the user input interface (230) has a touch screen, the user can input commands related to the display device (100) using the remote control device (200) by touching the soft keys of the touch screen. In addition, the user input interface (230) may have various types of input means that can be operated by the user, such as a scroll key or a jog key, and this embodiment does not limit the scope of the present invention.

[0085] The sensor (240) may include a gyro sensor (241) or an acceleration sensor (243), and the gyro sensor (241) may sense information about the movement of the remote control device (200).

[0086] For example, the gyro sensor (241) can sense information about the operation of the remote control device (200) based on the x, y, and z axes, and the acceleration sensor (243) can sense information about the movement speed of the remote control device (200). Meanwhile, the remote control device (200) can further include a distance measuring sensor, so as to sense the distance to the display (180) of the display device (100).

[0087] The output interface (250) can output a video or audio signal corresponding to an operation of the user input interface (230) or a signal transmitted from the display device (100).

[0088] The user can recognize whether the output interface (250) is manipulating the user input interface (230) or controlling the display device (100).

[0089] For example, the output interface (250) may include an LED (251) that lights up when the user input interface (230) is operated or a signal is transmitted and received with the display device (100) via the wireless communication unit (225), a vibrator (253) that generates vibrations, a speaker (255) that outputs sound, or a display (257) that outputs images.

[0090] In addition, the power supply circuit (260) supplies power to the remote control device (200), and power waste can be reduced by stopping the power supply when the remote control device (200) does not move for a predetermined period of time.

[0091] The power supply circuit (260) can resume power supply when a predetermined key provided in the remote control device (200) is operated.

[0092] The memory (270) can store various types of programs, application data, etc. required for the control or operation of the remote control device (200).

[0093] When the remote control device (200) wirelessly transmits and receives signals through the display device (100) and the RF circuit (221), the remote control device (200) and the display device (100) transmit and receive signals through a predetermined frequency band.

[0094] The controller (280) of the remote control device (200) can store and reference information regarding the frequency band that can wirelessly transmit and receive signals with the display device (100) paired with the remote control device (200) in the memory (270).

[0095] The controller (280) controls all matters related to the control of the remote control device (200). The controller (280) can transmit a signal corresponding to a predetermined key operation of the user input interface (230) or a signal corresponding to the movement of the remote control device (200) sensed by the sensor (240) to the display device (100) via the wireless communication unit (225).

[0096] Additionally, the microphone (290) of the remote control device (200) can acquire voice.

[0097] A plurality of microphones (290) may be provided.

[0098] Next, Figure 4 is described.

[0099] Figure 4 shows an example of utilizing a remote control device according to an embodiment of the present invention.

[0100] Figure 4 (a) illustrates that a pointer (205) corresponding to a remote control device (200) is displayed on a display (180).

[0101] The user can move or rotate the remote control device (200) up and down, left and right. The pointer (205) displayed on the display (180) of the display device (100) corresponds to the movement of the remote control device (200). This remote control device (200) can be called a space remote control because, as shown in the drawing, the pointer (205) moves and is displayed according to the movement in 3D space.

[0102] Figure 4 (b) illustrates that when a user moves the remote control device (200) to the left, the pointer (205) displayed on the display (180) of the display device (100) also moves to the left correspondingly.

[0103] Information about the movement of the remote control device (200) detected through the sensor of the remote control device (200) is transmitted to the display device (100). The display device (100) can calculate the coordinates of the pointer (205) from the information about the movement of the remote control device (200). The display device (100) can display the pointer (205) to correspond to the calculated coordinates.

[0104] Figure 4 (c) illustrates a case where a user moves the remote control device (200) away from the display (180) while pressing a specific button within the remote control device (200). As a result, a selection area within the display (180) corresponding to the pointer (205) can be zoomed in and displayed in an enlarged manner.

[0105] Conversely, when the user moves the remote control device (200) closer to the display (180), the selection area within the display (180) corresponding to the pointer (205) may be zoomed out and displayed in a reduced size.

[0106] Meanwhile, when the remote control device (200) moves away from the display (180), the selection area may be zoomed out, and when the remote control device (200) moves closer to the display (180), the selection area may be zoomed in.

[0107] Additionally, when a specific button within the remote control device (200) is pressed, recognition of up, down, left, and right movements may be excluded. That is, when the remote control device (200) moves away from or toward the display (180), up, down, left, and right movements may not be recognized, and only forward and backward movements may be recognized. When a specific button within the remote control device (200) is not pressed, only the pointer (205) moves in accordance with the up, down, left, and right movements of the remote control device (200).

[0108] Meanwhile, the movement speed or movement direction of the pointer (205) can correspond to the movement speed or movement direction of the remote control device (200).

[0109] Meanwhile, the pointer in this specification refers to an object displayed on the display (180) in response to the operation of the remote control device (200). Accordingly, objects of various shapes other than the arrow shape illustrated in the drawing can be used as the pointer (205). For example, the pointer may be a concept including a point, a cursor, a prompt, a thick outline, etc. In addition, the pointer (205) may be displayed corresponding to one point on the horizontal or vertical axis on the display (180), or may be displayed corresponding to multiple points such as lines or surfaces.

[0110] FIG. 5 illustrates an artificial intelligence (AI) server according to one embodiment of the present disclosure.

[0111] Referring to FIG. 5, the AI ​​server (500) may refer to a device that trains an artificial neural network using a machine learning algorithm or uses a trained artificial neural network.

[0112] The AI ​​server (500) may be composed of multiple servers to perform distributed processing, and may be defined as a 5G network.

[0113] The AI ​​server (500) may be included as part of the configuration of the AI ​​device (100) and may perform at least part of the AI ​​processing together.

[0114] The AI ​​server (500) may include a communication unit (510), a memory (530), a learning processor (540), and a processor (560).

[0115] The communication unit (510) can transmit and receive data with an external device such as a display device (100).

[0116] The memory (530) may include a model storage unit (531).

[0117] The model storage unit (531) can store a model (or artificial neural network, 531a) that is being learned or has been learned through the learning processor (540).

[0118] A learning processor (540) can train an artificial neural network (531a) using learning data. The learning model can be used while mounted on the AI ​​server (500) of the artificial neural network, or can be mounted on an external device such as a display device (100).

[0119] The learning model may be implemented in hardware, software, or a combination of hardware and software. If part or all of the learning model is implemented in software, one or more instructions constituting the learning model may be stored in memory (530).

[0120] The processor (560) can use a learning model to infer a result value for new input data and generate a response or control command based on the inferred result value.

[0121] FIG. 6 is a ladder diagram for explaining an operation method of a system according to an embodiment of the present disclosure.

[0122] A system according to an embodiment of the present disclosure may include a display device (100) and a server (600). The system according to an embodiment of the present disclosure may be an electronic device, such as a home appliance, installed in a home instead of the display device (100).

[0123] The server (600) may include all components included in the AI ​​server (500) illustrated in FIG. 5.

[0124] The server (600) may be a natural language processing (NLP) server that obtains analysis results by analyzing the intent of user input data.

[0125] Referring to FIG. 6, the controller (170) of the display device (100) can obtain user input data (S601).

[0126] In one embodiment, the user input data may be either voice data corresponding to a voice command spoken by the user or text data input by the user to the display device (100) via a remote control device (200).

[0127] The controller (170) can obtain user input data through the user input interface (150).

[0128] The controller (170) can receive voice data corresponding to a voice command spoken by a user from a remote control device (200).

[0129] As another example, the controller (170) can receive voice data corresponding to a voice command spoken by a user through a microphone (not shown) provided in the display device (100).

[0130] The controller (170) of the display device (100) can transmit the acquired user input data to the server (600) via the network interface (133) (S603).

[0131] The controller (170) can transmit user input data to the server (600) via the network interface (133) to provide a recognition service for user input data.

[0132] If the user input data is voice data, the controller (170) can transmit the voice data to an STT (Speech To Text) server. The STT server can convert the voice data into text data using a conversion engine and transmit the converted text data to the server (600).

[0133] The processor (560) of the server (600) can analyze user input data and determine whether analysis results for the user input data have been obtained (S605).

[0134] The processor (560) can obtain analysis results for user input data using the natural language processing engine stored in the model storage unit (531). The user input data may be text data.

[0135] A natural language processing engine may consist of an artificial neural network trained using machine learning algorithms.

[0136] A natural language processing engine can sequentially perform morphological analysis, syntax analysis, speech act analysis, and dialogue processing steps on text data to generate analysis results.

[0137] The morphological analysis step is the step of classifying text data corresponding to the user's spoken voice into morphemes, which are the smallest units that have meaning, and determining which part of speech each classified morpheme has.

[0138] The syntactic analysis step is the step that uses the results of the morphological analysis step to divide text data into noun phrases, verb phrases, adjective phrases, etc., and determines what kind of relationship exists between each divided phrase.

[0139] Through the parsing step, the subject, object, and modifiers of the speech spoken by the user can be determined.

[0140] The speech act analysis step uses the results of the syntactic analysis step to analyze the intent of the user's speech. Specifically, the speech act analysis step determines the intent of the sentence, such as whether the user is asking a question, making a request, or simply expressing emotion.

[0141] The dialogue processing stage uses the results of the speech act analysis stage to determine whether to respond to the user's utterance, respond, or ask a question for additional information.

[0142] After the conversation processing step, the natural language processing engine can generate an analysis result that includes one or more of a response, a response, or a request for additional information in response to the user's uttered intent.

[0143] The natural language processing engine may fail to analyze the intent of text data. In this case, the processor (560) may not be able to obtain analysis results for the user input data.

[0144] The natural language processing engine may not be able to output analysis results if the text data for the user input data is entered incorrectly.

[0145] If the natural language processing engine succeeds in analyzing the intent of text data, the processor (560) can obtain an analysis result indicating intent analysis.

[0146] In one embodiment, the intent for user input data may be either a request to perform a specific function of the display device (100) or a request to provide content.

[0147] A specific function of the display device (100) may indicate a function that the display device (100) can perform on its own. For example, the function of the display device (100) may be either a video recording function or a subtitle providing function.

[0148] A content provision request may be a request to provide or play content such as a movie, drama, or broadcast program.

[0149] Requests for content provision may primarily be requests for the provision of new content.

[0150] If the processor (560) of the server (600) fails to obtain the analysis result, it can transmit an analysis result non-obtainment notification indicating the failure to obtain the analysis result to the display device (100) through the communication unit (510) (S607).

[0151] The controller (170) of the display device (100) can output a notification that the received analysis results have not been acquired (S609).

[0152] The controller (170) may display a notification of non-acquisition of analysis results on the display (180). The notification of non-acquisition of analysis results may include text prompting the user to re-perform the input.

[0153] Replay of user input can be replay of text input or voice input.

[0154] Meanwhile, if the processor (560) fails to obtain an analysis result, it may transmit text data corresponding to the user input data to a streaming server providing a streaming service and receive a search result for the text data from the streaming server. The processor (560) may transmit the received search result to the display device (100) via the communication unit (510).

[0155] When the processor (560) of the server (600) obtains the analysis result of user input data (S605), it can determine whether support for the service requested by the user is possible based on the analysis result (S611).

[0156] The processor (560) can determine whether support for a service, such as a request to execute a specific function of the display device (100) or a request to provide content, is possible, which is included in the analysis results.

[0157] In one embodiment, the processor (560) can determine whether a specific function of the display device (100) included in the analysis result is supported or not.

[0158] For example, if the display device (100) does not support a recording function, the processor (560) may determine from the analysis results that the service requested by the user is not supported. If the display device (100) supports a recording function, the processor (560) may determine from the analysis results that the service requested by the user is not supported.

[0159] The processor (560) may store information about functions supported by the display device (100) in the memory (530). Based on the information about functions supported by the display device (100) stored in the memory (530), the processor (560) may determine whether support for a function included in the analysis result is possible.

[0160] As another example, if the analysis result indicates a request for provision of content A, and content A has not been officially released and is therefore unplayable, the processor (560) may determine that the service requested by the user is unavailable. If the analysis result indicates a request for provision of content A, and content A has been officially released and is therefore unplayable, the processor (560) may determine that the service requested by the user is available.

[0161] The processor (560) can inquire about the availability of content A to a content provider server or an external server, and determine whether the availability of content A included in the analysis result is possible based on a response regarding the availability of content A received from the content provider server or an external server.

[0162] If the processor (560) of the server (600) determines that support for the service requested by the user is possible, it can transmit service support information to the display device (100) via the communication unit (510) (S613).

[0163] If the analysis result is a request for the display device (100) to perform a specific function, the service support information may include a command to perform the specific function. The display device (100) may perform the function according to the command to perform the specific function.

[0164] If the analysis result is a request for specific content, the service support information may include one or more of the following: information about the content providers providing the content, and an access address for accessing the content. The service support information may also be video / audio data representing the specific content itself.

[0165] The controller (170) of the display device (100) can provide a service based on service support information (S615).

[0166] The controller (170) can perform a specific function of the display device (100) or provide specific content based on the service support information.

[0167] If the processor (560) of the server (600) determines that the service cannot be supported, it can transmit the analysis results of the user input data and information on service non-support to the display device (100) via the communication unit (510) (S617).

[0168] Service unsupported information may indicate that user input data was properly recognized (analysis results were obtained), but the service corresponding to the intent was not supported.

[0169] The controller (170) of the display device (100) can output the received analysis results and service non-support information (S619).

[0170] The controller (170) can display the received analysis results and service non-support information on the display (180) or output them as sound through a speaker (185).

[0171] The controller (170) of the display device (100) can store user log information including at least one of user input data, the type of user input data, and the accumulated number of user input data obtained by receiving service non-support information in the memory (140) (S621).

[0172] The controller (170) can store user input data and analysis results of the user input data in the memory (140) upon receiving service non-support information.

[0173] When the controller (170) receives service non-support information, it can store user log information indicating the history of attempts made by the user in the memory (40).

[0174] The type of user input data may include either a service unsupported type or an intent analysis failure type.

[0175] The service unsupported type may be a type indicating that support for a service is not possible, such as inability to provide a function of the display device (100) or inability to provide content.

[0176] The service unsupported type can be obtained based on service unsupported information received from the server (600).

[0177] The service unsupported type can be either a function unsupported type or a content unsupported type.

[0178] The intent analysis failure type may be a type indicating that intent analysis of user input data has failed. The intent analysis failure type may be obtained based on the analysis result non-acquisition notification of step S607 of FIG. 6.

[0179] The accumulated number of user input data may be the number of times user input data is acquired and accumulated.

[0180] User log information may further include analysis results.

[0181] The controller (170) can extract the accumulated number of user input data and the service unsupported type based on the user input data and the analysis results.

[0182] The controller (170) can generate user input data, accumulated count, analysis results, and unsupported service types as one log entry.

[0183] FIG. 7 is a diagram illustrating user log information according to one embodiment of the present disclosure.

[0184] Referring to FIG. 7, the controller (170) of the display device (100) can store user log information (700) in a memory (140) or a database connectable to the display device (100).

[0185] User log information (700) may include a distinction field (710) that distinguishes user input data, a type field (720) that distinguishes a service-unsupported type, a cumulative count field (730) that indicates the cumulative number of user input data, and an analysis result field (740) that describes the analysis result.

[0186] The distinction field (710) may be a field that distinguishes user input data acquired by the display device (100). The user input data may be either speech or text representing voice data. Here, only speech is described as an example.

[0187] The type field (720) may be a field indicating which service is not supported. The type field (720) may be determined based on service unsupported information received from the server (600).

[0188] The type field (720) may include a function unsupported type indicating that the display device (100) does not support a function or a content unsupported type indicating that the provision of content is not supported.

[0189] The accumulated count field (730) may be a count obtained by counting each time user input data is acquired.

[0190] The analysis result field (740) may be a field that includes information about the analysis result.

[0191] The user uttered <utterance 1> 10 times, indicating a request for a recording function, but it can be determined through user log information (700) that the function is not supported.

[0192] User log information (700) may be transmitted to the server (600) at regular intervals. The regular interval may be two weeks, but this is merely an example.

[0193] User log information (700) may include multiple log entries (701 to 705).

[0194] Each of the multiple log entries (701 to 705) may correspond to each user input data.

[0195] Each of the plurality of log entries (701 to 705) may include a spoken word, a service unsupported type, a cumulative count, and an analysis result.

[0196] Multiple log entries (701 to 705) can be classified according to the service unsupported type and stored in the memory (140).

[0197] Meanwhile, the controller (170) can store user log information even if analysis results are not obtained.

[0198] FIG. 8 is a diagram illustrating user log information stored when acquisition of analysis results for user input data fails according to one embodiment of the present disclosure.

[0199] When the controller (170) of the display device (100) receives a notification of non-acquisition of analysis results from the server (600), as in step S607 of FIG. 6, the controller (170) of the display device (100) may store user log information including one or more of user input data, the accumulated number of user input data, and additional information in a memory (140) or data base that can be connected to the display device (100).

[0200] Referring to FIG. 8, user log information (800) may include a distinction field (810) that distinguishes user input data, a type field (820) that indicates a failure in obtaining analysis results, a cumulative count field (830) that indicates the cumulative count of user input data, and an additional information field (840).

[0201] The additional information field (840) may include additional information about user input data that succeeded in the analysis result before obtaining user input data that failed to obtain the analysis result.

[0202] Additional information may be any of the following: genre of previously searched content, previously searched feature, or previously searched content type.

[0203] User log information (800) can be transmitted to the server (600).

[0204] In the case of an utterance regarding the name of a new content, if the server (600) does not know the name of the new content, acquisition of the analysis result may fail. To prepare for such a case, user log information (800) may be transmitted to the server (600) even if acquisition of the analysis result fails.

[0205] The server (600) may transmit a target notification to the display device (100) if a specific function or content of the display device (100) corresponding to the spoken word that failed to obtain the analysis result based on the log information (800) is available.

[0206] User log information (800) may include multiple log entries (801 to 803).

[0207] Each of the multiple log entries (801 to 803) may correspond to each user input data.

[0208] Each of the plurality of log entries (801 to 803) may include one or more of a utterance, a cumulative count, and additional information.

[0209] The display device (100) can transmit to the server (600) only those log items (801 to 803) whose cumulative number is greater than a certain number of times.

[0210] After that, the controller (170) of the display device (100) can transmit the stored user log information to the server (600) via the network interface (133) (S623).

[0211] The controller (170) can transmit user log information to the server (600) at a specific point in time.

[0212] The controller (170) can transmit user log information to the server (600) via the network interface (133) when the display device (100) boots up. However, this is merely an example, and the controller (170) can transmit user log information (600) to the server (600) at regular intervals.

[0213] The display device (100) and the server (600) can communicate through a cloud-based REST (Representational State Transfer) API.

[0214] A REST API can be an interface standard for securely exchanging information via HTTP.

[0215] In one embodiment, the controller (170) may transmit only a portion of the user log information to the server (600). The controller (170) may transmit only log information for user input data whose cumulative number of occurrences is greater than a certain number of occurrences to the server (600).

[0216] For example, the controller (170) may transmit to the server (600) only the log information for <utterance 1>, <utterance 2>, and <utterance 3>, which have accumulated more than 7 times among the user log information (700) illustrated in FIG. 7. That is, the controller (170) may transmit to the server (600) only some of the log items (701 to 703) among the plurality of log items (701 to 705).

[0217] User log information (800) can be classified according to the accumulated number of user input data.

[0218] The processor (560) of the server (600) can determine whether an unsupported service can be provided based on the received user log information (S625).

[0219] The processor (560) can determine whether an unsupported function has been changed to a supportable state according to a software upgrade of the display device (100).

[0220] The processor (560) can determine whether unsupported content has changed to a serviceable state as access to new content becomes available.

[0221] When an item for which service provision is possible is found among a plurality of unsupported service items, the processor (560) of the server (600) can transmit a target notification to the display device (100) via the communication unit (510) notifying that the service is available (S627).

[0222] When an unsupported function included in user log information is changed to a supportable state, the processor (560) can generate a target notification indicating that the function is supported, and transmit the generated target notification to the display device (100) via the communication unit (510).

[0223] When the state of unsupported content included in the user log information is changed to one in which provision is possible, the processor (560) can generate a target notification indicating that provision of the content is possible, and transmit the generated target notification to the display device (100) via the communication unit (510).

[0224] The processor (560) may request the content provider server or an external server to provide unsupported content based on the received user log information, and may receive the corresponding content from the content provider server or an external server in response to the request.

[0225] When the processor (560) obtains the access address of the content or the source information providing the content, it can determine that the content is supportable.

[0226] The controller (170) of the display device (100) can output a target notification received from the server (600) (S629).

[0227] The controller (170) can display the received target notification on the display (180) or output it as sound through a speaker (185).

[0228] The controller (170) can automatically update software for a specific function of the display device (100) or automatically install a related application when the target notification indicates that a specific function of the display device (100) is supported.

[0229] If the target notification indicates that specific content of the display device (100) is supported, the controller (170) may automatically update the software for providing the content or automatically install a related application. The related application may be an application corresponding to the content provider.

[0230] FIGS. 9A to 9C are diagrams illustrating a process for providing a target notification when an unsupported function is changed to a supportable state according to an embodiment of the present disclosure.

[0231] In Fig. 9a, the display device (100) is playing an image (900) on the display (180). The user<Please record it> Say a voice command requesting recording.

[0232] The display device (100) can obtain voice data corresponding to a voice command and transmit the obtained voice data to the server (600).

[0233] The server (600) can obtain text data of voice data and obtain analysis results of the obtained text data.

[0234] If the server (600) determines that the display device (100) does not support the recording function, it can transmit a service non-support notification to the display device (100).

[0235] The server (600) may store information in advance about functions supported by the display device (100).

[0236] The display device (100) can display a service unsupported notification (910) on the display (180), as illustrated in FIG. 9b.

[0237] The display device (100)<record it> The utterance called,<record it> A log entry including the cumulative acquisition count, the service unsupported type (function unsupported) and the analysis result of voice data (recording request) can be stored in the memory (140).

[0238] The display device (100) can transmit user log information including log items stored in the memory (140) to the server (600).

[0239] If recording function support is enabled based on a log entry, the server (600) can transmit a target notification to the display device (100).

[0240] The display device (100) can display the received target notification (930) on the display (180).

[0241] In this way, according to an embodiment of the present disclosure, even if a function of a display device (100) desired by a user is not currently supported, the user can be aware of a situation in which the function will be supported in the future through a target notification.

[0242] In other words, the usability of targeted notifications can be maximized as customized service notifications are provided.

[0243] FIGS. 10A to 10C are diagrams illustrating a process for providing a target notification when unsupported content is changed to a supportable state according to an embodiment of the present disclosure.

[0244] FIGS. 10A to 10C are diagrams illustrating a process for providing a target notification when provision of content that was previously impossible according to an embodiment of the present disclosure is changed to a state where provision is possible.

[0245] In Fig. 10a, the display device (100) is playing the home screen (1000) on the display (180). The user<search content A> Speak a voice command indicating the intent to search for (or provide) content.

[0246] The display device (100) can obtain voice data corresponding to a voice command and transmit the obtained voice data to the server (600).

[0247] The server (600) can obtain text data of voice data and obtain analysis results of the obtained text data.

[0248] If the server (600) fails to obtain information about content corresponding to <content A> from a new content database or content provider server, it may determine that the content cannot be supported.

[0249] The server (600) can transmit a service unsupported notification indicating that provision of <content A> is impossible to the display device (100).

[0250] The display device (100) can display a content non-support notification (1010) on the display (180), as illustrated in FIG. 10b.

[0251] The display device (100)<search content A> The utterance called,<search content A> A log entry including the cumulative acquisition count (cumulative utterance count), the service unsupported type (content unsupported), and the analysis result of voice data (content A search request) can be stored in the memory (140).

[0252] The display device (100) can transmit user log information including log items stored in the memory (140) to the server (600).

[0253] If provision of content A becomes possible based on the log entry, the server (600) can transmit a target notification to the display device (100).

[0254] The display device (100) can display the received target notification (1030) on the display (180).

[0255] The display device (100) may additionally display an indicator (1031) on the display (180) indicating that a target notification (1030) has been received.

[0256] In this way, according to an embodiment of the present disclosure, even if the content desired by the user is not currently supported, the user can be made aware of a situation in which the content will be supported in the future through a target notification.

[0257] In other words, the usability of targeted notifications can be maximized as customized service notifications are provided.

[0258] According to one embodiment of the present disclosure, the above-described method can be implemented as processor-readable code on a medium in which a program is recorded. Examples of processor-readable media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices.

[0259] The display device described above is not limited to the configuration and method of the embodiments described above, and the embodiments may be configured by selectively combining all or part of each embodiment so that various modifications can be made.

Claims

1. In the display device, memory; display; a network interface that communicates with the server; and A controller comprising: a controller for obtaining user input data, transmitting the obtained user input data to the server through the network interface, receiving analysis results of the user input data and service non-support information from the server, and storing the user input data and the analysis results in the memory upon receiving the service non-support information. Display device.

2. In paragraph 1, The above controller Extracting the accumulated number of user input data and the unsupported service type based on the user input data and the analysis results. Display device.

3. In paragraph 2, The above controller Generate the above user input data, the above accumulated number of times, the above analysis result, and the above service unsupported type as one log entry. Display device.

4. In paragraph 3, The above controller Transmitting user log information containing multiple log entries to the server via the network interface. Display device.

5. In paragraph 4, The above controller Among the above multiple log entries, only the log entries whose cumulative count is greater than a certain number are transmitted to the server through the network interface. Display device.

6. In paragraph 4, The above controller Sending the user log information to the server when the display device boots up. Display device.

7. In paragraph 2, The above service types are not supported Either a function unsupported type indicating that the function of the above display device is not supported or a content unsupported type indicating that the provision of content is not supported. Display device.

8. In paragraph 3, The above controller In response to the transmission of the above user log information, a target notification is received from the server indicating that support for a specific function or specific content of the display device is possible, Displaying the received target notification on the above display Display device.

9. In paragraph 8, The above controller Automatically update software or automatically install applications for providing the specific function or the specific content upon receiving the above target notification. Display device.

10. In paragraph 1, The above user input data is Either voice data or text data corresponding to the voice command spoken by the user Display device.

11. In paragraph 1, The above controller Receiving the service unsupported information indicating that a specific function of the display device is not supported or that the provision of specific content is impossible Display device.

12. In paragraph 1, The above controller If the above analysis result of the user input data fails to be obtained, a notification of failure to obtain the analysis result is received. Display device.

13. In paragraph 12, The above controller Storing the above user input data in the above memory Display device.

14. In paragraph 13, The above controller If the cumulative number of times the user input data is greater than a certain number of times, a type indicating a failure in obtaining the user input data and analysis results is transmitted to the server. Display device.

15. In the method of operating a display device, Step of obtaining user input data; A step of transmitting acquired user input data to the server; A step of receiving the analysis results of the user input data and service non-support information from the server; and Including a step of storing the user input data and the analysis result upon receiving the above service non-support information. How the display device operates.

Images