Display device for displaying customized recommended utterance

Abstract

This display device for displaying a customized recommended utterance comprises: a user input interface unit configured to receive a first input associated with a recommended utterance request and a second input associated with the recognition of speech data of a user by interworking with a remote control device; a display configured to display information associated with the speech data or a recommended utterance; and a processor configured to display, on the display, keywords associated with the speech data or previous speech data of the user.

Description

Display device that displays customized recommended utterances

[0001] The present disclosure relates to a display device for displaying a customized recommended utterance. More specifically, the present disclosure relates to an AI system comprising a display device for displaying a customized recommended utterance and a server.

[0002] Recently, digital TV services using wired or wireless communication networks have become commonplace. Digital TV services can provide a variety of services that were not available through existing analog broadcasting services.

[0003] For example, IPTV (Internet Protocol Television) and SMART TV services, which are types of digital TV services, provide interactivity that allows users to actively select the types of programs and viewing time.

[0004] Display devices such as digital TVs have recently been providing various services based on artificial intelligence (AI). In this regard, display devices installed in the home can provide AI services customized for multiple users. However, there is an issue that it is difficult to provide personalized recommendations because it is difficult to distinguish usage data among users who share the same display device installed in the home.

[0005] In terms of personal information exposure, there are privacy issues with recommendations as viewing history or speech and search records are exposed among users sharing the same display device. Additionally, regarding search time and complexity, directly entering keywords or searching for content requires significant time and effort, and there is an issue that it is difficult to utilize the various features provided by the display device.

[0006] The present disclosure aims to solve the aforementioned problems and other problems by providing an AI system comprising a display device and a server that display customized recommended utterances.

[0007] The present disclosure is intended to solve the issue that it is difficult to make personalized recommendations because it is difficult to distinguish usage data among users who share the same display device placed in the home.

[0008] The present disclosure is intended to address privacy issues in recommendations regarding the exposure of personal information, specifically concerning viewing history or speech / search records among users sharing the same display device.

[0009] The present disclosure is intended to address the issue that, in terms of search time and complexity, directly inputting or searching for keywords to be searched or viewed requires significant time and effort, and makes it difficult to utilize the various functions provided by display devices.

[0010] A display device for displaying a customized recommended speech according to the present disclosure comprises: a user input interface unit configured to receive a first input associated with a request for a recommended speech and a second input associated with the recognition of a user's speech data in conjunction with a remote control device; a display configured to display information or a recommended speech associated with the speech data; and a processor configured to display keywords associated with the speech data or the user's previous speech data on the display.

[0011] According to an embodiment, when the processor receives a key value corresponding to a button of the remote control device, it generates a recommended utterance request, requests the history of the previous utterance data from a database in response to the recommended utterance request, and requests a recommended utterance from a server based on the history of the utterance data and the user's account information. The processor receives the recommended utterance from the server and can control the received recommended utterance containing the keyword to be output to the display.

[0012] According to an embodiment, the processor may include a recommended utterance output module configured to execute an application program and output the recommended utterance when a first input of less than a threshold time is applied to the button; and a history management module that receives the recommended utterance request from the recommended utterance output module and receives the history of the utterance data from the database. The history management module may transmit the recommended utterance received from the server to the recommended utterance output module.

[0013] According to an embodiment, when a key value corresponding to the power button of the remote control device is received, the recommended speech output module can transmit the recommended speech request to the history management module. The history management module can transmit a greeting request to a second server and receive a greeting based on time information from the second server. The history management module can transmit the received greeting to the recommended speech output module. The recommended speech output module can output a screen containing the received greeting to the display.

[0014] According to an embodiment, when the processor is first switched from a power-off state to a power-on state on a specific date, it may output a screen containing a first greeting based on the time information to the display. If the number of times the processor is switched to the power-on state is two or more, it may output a screen containing a second greeting different from the first greeting to the display. When a first threshold time elapses in the power-on state, the processor may extract keywords based on the frequency of utterance based on the history of utterance data for each week. The processor may display the extracted keywords in the order of the frequency of utterance.

[0015] According to an embodiment, the processor may further include a voice recognition processing module configured to transmit the recognized utterance data from the remote control device to a server. When a second input exceeding a threshold time is applied to the button, the voice recognition processing module may process an analysis of the user's intent associated with the recognized utterance data in conjunction with the server. Based on the processing result of the intent analysis, the voice recognition processing module may control the display of a search result associated with the recognized utterance data on the screen of the display.

[0016] According to an embodiment, the voice recognition processing module can determine whether the recognized speech data is voiceprint data of a second user other than the user by interacting with the server. If the second user is a registered user, the processor can display an account switching screen related to whether to switch from the user's login account to the second user's second login account. When the processor receives a switching input to switch to the second login account on the account switching screen, it can switch to the second login account.

[0017] According to an embodiment, the processor may request a second recommended utterance from the server based on the second history of the second user's utterance data and the second account information of the second user. The processor may receive the second recommended utterance from the server and control the received second recommended utterance, which includes a second keyword, to be output to the display.

[0018] According to an embodiment, the history management module may store the history of the utterance data in the database based on the processing result of the intent analysis from the server. The history management module may store the history of the content information in the database based on the content information selected on the home screen of the display. The history management module may store the history of the second content information in the database based on the second content information selected on the integrated search screen of the display.

[0019] According to an embodiment, the utterance data, the content information, and the second content information stored in the database may include start time information, end time information, keywords, genre codes, intent types, and application information running on the display device. The keywords may be keywords of the utterance data, names of the content, and names of the second content.

[0020] According to an embodiment, the speech recognition processing module can control the storage of the history of the utterance data in the database. The history may include the user's account, STT result, utterance date and time, day of the week, and time information.

[0021] According to an embodiment, when the processor receives a key value corresponding to the button, it can extract account information of the current login status through the database and extract keyword information associated with the utterance used by the user based on the current day and time information. Based on the keyword information, the processor can control the received recommended utterance containing the keyword to be output to the display.

[0022] According to an embodiment, the processor may transmit account information of the current login status, the current time, and the user's connection record to the second server. The second server transmits the account information to a UPP server to receive customer segment information from the UPP server, and may transmit a welcome message generated based on the current time, the customer segment information, and the user's connection record to the processor. The processor may control the welcome message to be displayed on the display.

[0023] According to an embodiment, when the processor receives a key value corresponding to the button, it may transmit information regarding the current time, connected device, program currently being watched, and application currently being used to the second server. Based on the current time, the connected device, the program currently being watched, and the application information currently being used, the processor may receive customized use tip information from the second server. Based on the customized use tip information, the processor may output at least one sentence associated with the use tip to the display.

[0024] According to an embodiment, the processor can obtain information about the application currently in use and the user. Based on the information about the application currently in use, the history of the utterance data, and the user's account information, the processor can output at least one sentence related to the search or playback of content provided by the application to the screen of the display.

[0025] According to an embodiment, the processor can obtain status information of the connected device and environmental information of the display device and the area where the connected device is placed. Based on the status information and the environmental information, the processor can output at least one sentence associated with a usage tip that can control the operation of the connected device to the screen of the display.

[0026] According to an embodiment, the processor may receive broadcast information associated with the user's viewing program from the server. The processor may transmit the broadcast information to a third server that extracts associated keywords associated with the broadcast information. The third server may receive program details associated with the broadcast information from a fourth server of the broadcast information system. Based on the program details, the processor may output the associated keywords associated with the broadcast information to the display.

[0027] According to an embodiment, when the processor detects the first user while the first user is logged in with the first login account among a plurality of user accounts, it may request a first recommended utterance from the server based on the first history of the first user's first utterance data and the first account information. When the processor detects the second user while the second user is logged in with the second login account among a plurality of user accounts, it may request a second recommended utterance from the server based on the second history of the second user's second utterance data and the second account information. The first user may be detected through the detection of the first identifier of the first user's first device or through the voice data of the first user. The second user may be detected through the detection of the second identifier of the second user's second device or through the detection of the voice data of the second user.

[0028] According to an embodiment, if the processor detects a second user but not a first user while logged in with the first login account, it may request the second recommended utterance from the server based on the second history and the second account information. If the processor detects the first user and the second user while logged in with the first login account, it may request the first recommended utterance and the second recommended utterance from the server based on the first history, the second history, the first account information, and the second account information. The processor may display a common recommended utterance on the display based on a common keyword shared by the first recommended utterance and the second recommended utterance. The processor may display the first recommended utterance following the common recommended utterance based on the first user's first keyword. The second user may be established as a family relationship with the first user.

[0029] According to the present specification, personalized recommendations are possible by managing usage data by account, and users can be distinguished through voice recognition, enabling user-specific data learning through speech alone without a separate login process.

[0030] According to the present specification, recommendations can be received immediately by pressing an AI key anytime and anywhere, and desired functions or content can be quickly executed by significantly reducing search time through advanced logic that increases recommendation accuracy.

[0031] According to the present specification, TV can be enjoyed more immersively by providing settings or related information optimized for the content being watched and providing usage tips that make better use of the app being used.

[0032] FIG. 1 is a block diagram illustrating the configuration of a display device according to one embodiment of the present disclosure.

[0033] FIG. 2 is a drawing for explaining a content server according to an embodiment of the present disclosure.

[0034] FIG. 3 is a drawing for explaining an AI system including a display device according to an embodiment of the present disclosure.

[0035] FIG. 4 shows a flowchart of the operation between each module of a display device displaying a customized recommended utterance according to the present disclosure and a server.

[0036] FIG. 5 shows a flowchart of operations between each module of a display device configured to recognize speech data or display customized recommended speech according to the present disclosure and a server.

[0037] FIG. 6 shows the configuration of an AI system including a display device that receives a greeting to be provided to a user according to the present disclosure from a separate server and displays it on a screen.

[0038] Figure 7 shows the screens displayed on the display device over time while the power is on.

[0039] Figure 8 shows the configuration of a display device configured to store historical information for intent analysis through speech data and historical information for selected content on the home screen and integrated search screen.

[0040] FIGS. 9a and 9b show the initial screen when logging in with a specific user's login account and the screen for changing the profiles of multiple users.

[0041] FIG. 10 shows a configuration for providing a user / context-based customized recommendation service according to an embodiment.

[0042] FIG. 11 shows a configuration for providing a user / context-based customized usage tip service according to an embodiment.

[0043] FIGS. 12a and FIGS. 12b show screens of a display device that display status information of devices connected to the display device.

[0044] FIG. 13 shows the configuration of an AI system that provides a service for recommending related keywords based on broadcast information according to an embodiment.

[0045] FIG. 14 shows a flowchart of a method for displaying customized recommendation utterances performed by a display device and an AI system including the same.

[0046] FIG. 15 shows a flowchart of a method for displaying associative search results based on intent analysis of speech recognition performed by a display device and an AI system including the same.

[0047] It should be noted that technical terms used in this specification are used merely to describe specific embodiments and are not intended to limit the invention. Additionally, singular expressions used in this specification include plural expressions unless the context clearly indicates otherwise. The suffixes "module" and "part" for components used in the following description are assigned or used interchangeably solely for the ease of drafting the specification and do not inherently possess distinct meanings or roles.

[0048] In this specification, terms such as "composed of" or "comprising" should not be interpreted as necessarily including all of the various components or steps described in the specification, and should be interpreted as potentially excluding some of the components or steps, or including additional components or steps.

[0049] In addition, when describing the technology disclosed in this specification, if it is determined that a detailed description of related prior art could obscure the essence of the technology disclosed in this specification, such detailed description is omitted.

[0050] In addition, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification, and the technical concept disclosed in this specification is not limited by the attached drawings; it should be understood that they include all modifications, equivalents, and substitutions that fall within the concept and technical scope of the present invention. Furthermore, not only each of the embodiments described below, but also combinations of embodiments may fall within the concept and technical scope of the present invention as modifications, equivalents, and substitutions that fall within the concept and technical scope of the present invention.

[0051] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings.

[0052] FIG. 1 is a block diagram illustrating the configuration of a display device according to one embodiment of the present disclosure.

[0053] Referring to FIG. 1, the display device (100) may include a broadcast receiver (130), an external device interface (135), a database (140), a user input interface (150), a processor (170), a wireless communication unit (110), a voice recognition processing module (175), a display (180), an audio output unit (185), and a power supply unit (190).

[0054] The broadcast receiving unit (130) may include a tuner (131), a demodulating unit (132), and a network interface unit (133).

[0055] The tuner (131) can tune to a specific broadcast channel according to a channel tuning command. The tuner (131) can receive a broadcast signal for the tuned specific broadcast channel.

[0056] The demodulator (132) can separate the received broadcast signal into a video signal, an audio signal, and a data signal related to the broadcast program, and can restore the separated video signal, audio signal, and data signal into a form that can be output.

[0057] The network interface unit (133) may provide an interface for connecting the display device (100) to a wired / wireless network including the Internet network. The network interface unit (133) may transmit or receive data to or from other users or other electronic devices through the connected network or another network linked to the connected network.

[0058] The network interface unit (133) can access a specific web page through a connected network or another network linked to the connected network. That is, it can access a specific web page through a network and transmit or receive data with the corresponding server.

[0059] In addition, the network interface unit (133) can receive content or data provided by a content provider or network operator. That is, the network interface unit (133) can receive content such as movies, advertisements, games, VOD, broadcast signals, and related information provided by a content provider or network provider through a network.

[0060] Additionally, the network interface unit (133) can receive firmware update information and update files provided by the network operator, and can transmit data to the internet, content provider, or network operator.

[0061] The network interface unit (133) can select and receive a desired application among the applications that are open to the public through the network.

[0062] The external device interface unit (135) can receive an application or a list of applications within an adjacent external device and transmit it to a processor (170) or a database (140).

[0063] The external device interface section (135) can provide a connection path between the display device (100) and an external device. The external device interface section (135) can receive one or more of video and audio output from an external device connected to the display device (100) wirelessly or via a wired connection, and transmit them to the processor (170). The external device interface section (135) may include a plurality of external input terminals. The plurality of external input terminals may include an RGB terminal, one or more HDMI (High Definition Multimedia Interface) terminals, and a component terminal.

[0064] The video signal of an external device input through the external device interface unit (135) can be output through the display (180). The audio signal of an external device input through the external device interface unit (135) can be output through the audio output unit (185).

[0065] The external device that can be connected to the external device interface section (135) may be any one of a set-top box, Blu-ray player, DVD player, game console, soundbar, smartphone, PC, USB memory, or home theater, but this is merely an example.

[0066] In addition, some of the content data stored in the display device (100) can be transmitted to another user or other electronic device selected among other users or other electronic devices that are previously registered in the display device (100).

[0067] The database (140) can store programs for each signal processing and control within the processor (170), and can store signal-processed video, audio, or data signals.

[0068] Additionally, the database (140) may perform the function of temporarily storing video, audio, or data signals input from the external device interface unit (135) or the network interface unit (133), and may also store information regarding a predetermined image through a channel memory function.

[0069] The database (140) can store an application or a list of applications input from an external device interface unit (135) or a network interface unit (133).

[0070] The display device (100) can play content files (video files, still image files, music files, document files, application files, etc.) stored in the database (140) and provide them to the user.

[0071] The user input interface (150) can transmit a signal input by the user to the processor (170) or transmit a signal from the processor (170) to the user. For example, the user input interface (150) can receive and process control signals such as power on / off, channel selection, and screen setting from the remote control device (200) according to various communication methods such as Bluetooth, Ultra Wideband (UWB), ZigBee, Radio Frequency (RF) communication, or Infrared (IR) communication, or process to transmit control signals from the processor (170) to the remote control device (200).

[0072] Additionally, the user input interface (150) can transmit control signals input from local keys (not shown), such as a power key, channel key, volume key, and setting value, to the processor (170).

[0073] The image signal processed by the processor (170) can be input to the display (180) and displayed as an image corresponding to the image signal. Additionally, the image signal processed by the processor (170) can be input to an external output device through the external device interface section (135).

[0074] The voice signal processed by the processor (170) can be output as audio to the audio output unit (185). Additionally, the voice signal processed by the processor (170) can be input to an external output device through the external device interface unit (135).

[0075] In addition, the processor (170) can control the overall operation within the display device (100).

[0076] Additionally, the processor (170) can control the display device (100) by means of user commands or internal programs input through the user input interface (150). The processor (170) can connect to a network to enable the user to download desired applications or application lists into the display device (100). The processor (170) may be configured to execute at least one application program to control the display device (100). A first application program (10) may execute to determine whether a specific content has been purchased, played, refunded, or resold based on the viewing history and playback time of the specific content. A second application program (20) may execute to store and manage information regarding the playback history and playback time of the specific content.

[0077] The processor (170) enables the processed video or audio signal, such as channel information selected by the user, to be output through the display (180) or audio output unit (185).

[0078] Additionally, the processor (170) enables a video signal or audio signal from an external device, such as a camera or camcorder, which is input through the external device interface unit (135), to be output through the display (180) or audio output unit (185) in accordance with an external device video playback command received through the user input interface (150).

[0079] Meanwhile, the processor (170) can control the display (180) to display video, for example, broadcast video input through the tuner (131), external input video input through the external device interface unit (135), video input through the network interface unit, or video stored in the database (140) can be controlled to be displayed on the display (180). In this case, the video displayed on the display (180) may be a still image or a video, and may be a 2D image or a 3D image.

[0080] Additionally, the processor (170) can control the playback of content stored in the display device (100), received broadcast content, or external input content input from the outside, and the content may be in various forms such as broadcast video, external input video, audio file, still image, connected web screen, and document file.

[0081] The wireless communication unit (110) can communicate with an external device via wired or wireless communication. The wireless communication unit (110) can perform short-range communication with an external device. To this end, the wireless communication unit (110) can support short-range communication by using at least one of Bluetooth™, BLE (Bluetooth Low Energy), RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies. The wireless communication unit (110) can support wireless communication between the display device (100) and the wireless communication system, between the display device (100) and another display device (100), or between the display device (100) and the network where the display device (100, or external server) is located, through a wireless area network. The wireless area network may be a wireless personal area network.

[0082] Here, another display device (100) may be a wearable device (e.g., a smartwatch, smart glass, head-mounted display, or mobile terminal such as a smartphone) capable of exchanging (or interacting with) data with the display device (100) according to the present invention. A wireless communication unit (110) may detect (or recognize) a wearable device capable of communication around the display device (100). Furthermore, if the detected wearable device is an authenticated device to communicate with the display device (100) according to the present invention, the processor (170) may transmit at least a portion of the data processed in the display device (100) to the wearable device through the wireless communication unit (110). Accordingly, the user of the wearable device may use the data processed in the display device (100) through the wearable device.

[0083] The voice recognition processing module (175) can acquire audio. The voice recognition processing module (175) may include at least one microphone (not shown) and can acquire audio around the display device (100) through the microphone (not shown).

[0084] The display (180) can generate a driving signal by converting the video signal, data signal, OSD signal processed by the processor (170) or the video signal, data signal, etc. received from the external device interface unit (135) into R, G, and B signals, respectively.

[0085] Meanwhile, since the display device (100) illustrated in FIG. 1 is merely an embodiment of the present invention, some of the illustrated components may be integrated, added, or omitted depending on the specifications of the actual implemented display device (100).

[0086] That is, as needed, two or more components may be combined into a single component, or a single component may be subdivided into two or more components. In addition, the functions performed in each block are intended to explain embodiments of the present invention, and the specific operations or devices do not limit the scope of the present invention.

[0087] FIG. 2 is a drawing for explaining a server according to an embodiment of the present disclosure.

[0088] The server (300) can provide a recommendation service that recommends content or services that a viewer using the display device (100) may prefer.

[0089] The server (300) may include a communication interface (310), memory (320), and a processor (330).

[0090] The server (300) can transmit and receive data to and from at least one display device (100) via wired or wireless communication through the communication interface (310).

[0091] The memory (320) may include a database (340) or be configured to be linked with the database (340). The database (340) may store information related to content played on each device. For example, the database (340) may store content playback information, content setting information, or application installation information associated with the identification information of each device.

[0092] When the processor (330) receives a content recommendation request from a display device (100) or an external device, it can recommend content optimized for each device based on data stored in the database (340).

[0093] FIG. 3 is a drawing for explaining an AI system including a display device according to an embodiment of the present disclosure.

[0094] The AI ​​system (1000) may include at least one display device (100), at least one remote control device (200), and a server (300).

[0095] The processor (170) of the display device (100) can play content.

[0096] Additionally, the processor (170) can generate content playback information regarding the played content. Additionally, the processor (170) can generate content setting information, which is information regarding the quality, volume, and preferred channel status set when playing the content.

[0097] Content playback information may include at least one of content identification information, content genre information, content playback start time information, content playback end time information, and content total playback time information for the played content.

[0098] Content setting information may include at least one of quality information set for the content when playing the content, volume information, and preferred channel information regarding whether the user has registered the channel providing the content as a preferred channel.

[0099] The processor (170) can transmit device identification information of the display device (100), generated content playback information, and generated content setting information to the server (300) through the communication interface (175). The device identification information may be unique identification information for distinguishing it from other devices.

[0100] The server (300) can store content playback information and content setting information received from the display device (100) in the content information database (321) in association with device identification information.

[0101] Meanwhile, the processor (170) can receive a content recommendation command through the user input interface unit (150) or the voice recognition processing module (175).

[0102] When the processor (170) receives a content recommendation command, it can transmit device identification information of the display device (100) and a content recommendation request to the server (300) through the communication interface (175).

[0103] The communication interface (310) of the server (300) can receive device identification information and content recommendation requests from the display device (100).

[0104] The processor (330) of the server (300) can obtain content playback information and content setting information associated with the display device (100) from the content information database (321) based on device identification information.

[0105] The processor (330) can generate content recommendation information and recommendation setting information for the display device (100) based on content playback information and content setting information. The content recommendation information may include recommended content identification information and recommended content genre information for at least one recommended content. Additionally, the recommendation setting information may include recommended image quality setting information and preferred channel information.

[0106] The processor (330) can transmit content recommendation information and recommendation setting information to the display device (100) through the communication interface (310).

[0107] The processor (170) can receive content recommendation information and recommendation setting information from the server (300) through the communication interface (175).

[0108] The processor (170) can display at least one recommended content based on the received content recommendation information. Additionally, when a playback command for the recommended content is input through the user input interface unit (150) or the voice recognition processing module (175), the processor (170) can set the quality of the recommended content to be played based on the received recommendation setting information and play it.

[0109] The quality of recommended content is set for playback, and if a user requests a change to a preferred channel, a channel change to the preferred channel can be performed based on the preferred channel information.

[0110] Meanwhile, the display device (100) can mirror the content currently being played to an external device. The external device may include another display device or a mobile device. In this case, the mirrored content can be viewed through the external device. Therefore, viewing information regarding the mirrored content needs to serve as basic data for recommending content to the external device.

[0111] Meanwhile, when the display device (100) performs a mirroring operation to an external device, it may receive a control command from the external device to control the display device (100). The control command may include a content change command to change the content being played from the first content to the second content. When the display device (100) receives the content change command, it may play the changed content. In this case, the display device (100) needs to transmit content playback information regarding the changed content to the server (300) as information for content recommendation to the external device.

[0112] Hereinafter, a display device for displaying a customized recommend speech and an AI system including the same according to the present disclosure will be described. In this regard, FIG. 4 shows a flowchart of the operation between each module of the display device for displaying a customized recommend speech according to the present disclosure and a server.

[0113] Referring to FIGS. 1 to 4, a display device (100) linked with utterance data according to the present disclosure will be described. The display device (100) may be configured to include a user input interface unit (150), a processor (170), and a display (180).

[0114] The user input interface unit (150) may be configured to receive a first input associated with a recommended speech request in conjunction with the remote control device (200). The user input interface unit (150) may be configured to receive a second input associated with the recognition of the user's speech data in conjunction with the remote control device (200). In this regard, the first input and the second input may correspond to inputs for a specific button of the remote control device (200). For example, the first input and the second input may correspond to different types of inputs of the AI ​​shortcut key (202) of the remote control device (200). The first input may correspond to a short press for a first time period less than the threshold time of the AI ​​shortcut key (202). The second input may correspond to a long press for a second time period greater than the threshold time of the AI ​​shortcut key (202).

[0115] The display (180) may be configured to display information or recommended utterances associated with the utterance data. The processor (170) may be configured to display keywords associated with the utterance data or the user's previous utterance data on the display (180).

[0116] The processor (170) may be configured to generate a recommended speech request when it receives a key value corresponding to a button of the remote control device (200). In this regard, the button of the remote control device (200) may correspond to the AI ​​short key (202) of FIG. 3. If the AI ​​short key is pressed for a first time less than a threshold time, the processor (170) may recognize it as a first input of a short press. The processor (170) may perform an AI concierge mode that recommends customized keywords and features in response to the first input.

[0117] If the AI ​​shortcut key is pressed for a second time longer than the threshold time, the processor (170) can recognize it as a second input of long press and hold. In response to the second input, the processor (170) can perform an AI voice recognition mode that recognizes the user's speech data (voice data).

[0118] Meanwhile, the processor (170) may acquire the history of the user's previous speech data and user information to receive a recommendation speech request from the server (300) in AI concierge mode. In this regard, the history of speech data is not limited to speech data based on the user's voice recognized through the display device (100). The history of the user's speech data may include the history of speech data for other electronic devices. It may include the history of speech data for voice-recognizable home appliances or mobile terminals.

[0119] Accordingly, the processor (170) can obtain the history of speech data and user information of the voice recognized through the display device (100) and the devices (400) connected to the display device (100). The devices (400) connected to the display device (100) may be multiple home appliances linked to the display device (100). Additionally, the processor (170) can obtain the history of speech data and user information of the electronic device of a user terminal registered through the display device (100), such as a mobile terminal. In this regard, the display device (100) may operate as a home hub that controls multiple home appliances. Accordingly, the history of speech data of multiple home appliances linked to the display device (100) may be stored in the database (140).

[0120] The processor (170) may request the history of previous utterance data from the database (140) in response to a request for a recommended utterance. The processor (170) may request a recommended utterance from the server (300) based on the history of the utterance data and the user's account information.

[0121] The processor (170) may be configured to receive recommended utterances from the server (300). The processor (170) may control the received recommended utterance, which includes keywords associated with previous utterance data, to be output to the display (180).

[0122] Meanwhile, the processor (170) may be configured to include a recommended utterance output module (171) and a history management module (172). In this regard, the recommended utterance output module (171) may be implemented as an AI concierge application program that outputs recommended utterances to the screen of the display (170). The history management module (172) manages history information associated with the user's previous utterances, receives recommended utterances based on the history information and user account, and transmits them to the recommended utterance output module (171).

[0123] Meanwhile, the history management module is not limited to being implemented in the display device (100) and can be changed according to the application. A history management module (1200) that manages history information related to a user's previous utterances in the AI ​​system (1000) may be implemented as an external entity of the display device (100). The history management module (1200) may be implemented in an electronic device such as a mobile terminal, multiple home appliances linked to the display device (100), or a server. The history management module (1200) may be implemented in a mobile terminal, home appliances, a database (140, 340), a server (300), another database, another server, or an integrated server, etc.

[0124] When a button (e.g., AI shortcut key) of the remote control device (200) is pressed, the recommended speech output module (171) may be configured to execute an application program (10) to output a recommended speech. The history management module (172) may receive a recommended speech request from the recommended speech output module (171) and transmit the recommended speech request to the database (140). The history management module (172) may be configured to receive the history of speech data from the database (140). The history management module (172) may transmit the recommended speech received from the server (300) to the recommended speech output module (171).

[0125] In this regard, when a first input is applied to the AI ​​shortcut key (202) of the remote control device (200), an execution input may be transmitted (S10) from the remote control device (200) to the recommended speech output module (171). The recommended speech output module (171) may transmit a recommended speech request to the history management module (172) (S20). The history management module (172) may request history information stored in the database (140) (S30). In response to the stored history request, the database (140) may transmit selected history information among the stored history information to the history management module (172). In this regard, the selected history information may be history information corresponding to the foreground application program currently running on the display device and the login (user) account. For example, the selected history information may be history information of previous speech data in a specific application program currently running by the user among a plurality of users.

[0126] The history management module (172) can transmit a recommendation utterance request to the server (300) including user history and account information (S50). In response to the recommendation utterance request, the server (300) can select or generate a specific recommendation utterance from among the recommendation utterances stored in the database (340). The server (300) can transmit the recommendation utterance selected or generated based on the user history and account information to the history management module (172) (S60). The history management module (172) can transmit the recommendation utterance received from the server (300) to the recommendation utterance output module (171) (S70). The recommendation utterance output module (171) can output the recommendation utterance containing keywords received from the history management module (172) to the screen of the display (180) (S80). In this regard, a sentence including the user's name (e.g., Jiwon, please select an utterance you used frequently over the past week.) may be displayed on the screen.

[0127] A sentence containing the user's name may be displayed in the first area, and recommended utterances based on the user's speech history may be displayed in the second area. Recommended utterances may be displayed with keywords combined with hashtags. Recommended utterances may be displayed with sentences containing keywords combined with hashtags. The keywords of the recommended utterances may be first and second content (first and second programs) that can be viewed through the display device (100). Additionally, the keywords of the recommended utterances may be sentences spoken by the user, for example, "What is the weather like tomorrow?"

[0128] Accordingly, a display device that displays customized recommended utterances according to the present disclosure may be referred to as an AI concierge that performs AI functions. The AI ​​concierge may provide customized greetings based on user segments or situations upon power-on of the display device. Additionally, while using the display device, it may recommend recommended utterances including user and situation-specific keywords based on the user's intent and usage context. Recommended utterances may be displayed on the screen of the display device to recommend keywords, specific functions, settings, content, etc.

[0129] Meanwhile, the display device according to the present disclosure may be configured to recognize speech data or display customized recommended speech through a remote control device. In this regard, FIG. 5 shows a flowchart of the operation between each module of the display device and the server configured to recognize speech data or display customized recommended speech according to the present disclosure.

[0130] Referring to FIGS. 1 to 5, a display device (100) linked with ignition data according to the present disclosure will be described.

[0131] Regarding the recognition and loading of speech data, each component of the display device (100) can be linked with the remote control device (2000) and the server (300). When a second input exceeding a threshold time is applied to a specific button of the remote control device (2000), the user can utter the first keyword of the first program. The first keyword of the first program may be the first content "three meals a day".

[0132] The voice recognition processing module (175) can be configured to perform a voice recognition trigger operation to enable voice recognition of a user through a remote control device (2000), and to acquire speech data of the user's voice according to the trigger operation. When a second input of longer than a threshold time is applied to a specific button of the remote control device (2000), the voice recognition processing module (175) can acquire the voice "three meals a day." The processor (170) can display the sentence "This is the search result for three meals a day" in conjunction with the voice recognition processing module (175).

[0133] The processor (170) can display speech data and the results of processing the intent analysis of the speech data on a screen. The processor (170) can display the sentence "This is the search result for three meals a day" and the search result of the text corresponding to the acquired voice on the first screen of the display (170).

[0134] The processor (170) can transmit the acquired utterance data to the server (300). The server (300) can perform intent analysis processing on the transmitted utterance data. The server (300) can perform natural language processing (NLP) for intent analysis of the utterance data. The server (300) can perform intent analysis processing on the utterance data through morphological analysis, syntactic analysis, semantic analysis, and discourse analysis of the utterance data. The processor (170) can receive the results of the intent analysis processing of the utterance data from the server (300). The history management module (172) can load (store) the utterance data for which intent analysis processing has been performed into the history management module (172). The history information of the utterance data stored in the database (140) in conjunction with the history management module (172) may include utterance data, speaker account, STT (Speech to Text) information, and utterance date / day / time information.

[0135] Meanwhile, the user input interface unit (150) may be configured to receive a first input less than the threshold time of the AI ​​shortcut key (200) of the remote control device (200). The user input interface unit (150) may notify the recommendation speech output module (171) that the first input has been received. The recommendation speech output module (171) may check the account information of the current login status in conjunction with the account management module (175). In this regard, the recommendation speech output module (171) may transmit a request for the account information of the current login status to the account management module (175). In response to the account information request, the recommendation speech output module (171) may receive the account information of the current login status from the account management module (175) and identify the account of the current login status.

[0136] The recommended utterance output module (171) can obtain the history of utterance data stored in the database (140) by linking with the history management module (172). The recommended utterance output module (171) can generate recommended utterances containing specific keywords based on login account information and the history of utterance data. Recommended utterances containing specific keywords can be generated based on utterances / keywords frequently used by the user. In this regard, a sentence containing the user's name can be displayed on the second screen of the display (170). A sentence containing the user's name is displayed in the first area, and recommended utterances based on the user's utterance history can be displayed in the second area.

[0137] Meanwhile, an AI system including a display device linked with speech data according to the present disclosure can receive greetings to be provided to a user from a separate server and display them on a screen. In this regard, FIG. 6 illustrates the configuration of an AI system including a display device that receives greetings to be provided to a user from a separate server and displays them on a screen according to the present disclosure.

[0138] Referring to FIGS. 3 and 6, the power button (201) of the remote control device (200) may be pressed while in the power-off state or switched to the power-on state by a timer setting. When the power is switched to the power-on state and the AI ​​shortcut key (202) is pressed within a certain period of time, a greeting may be displayed on the screen of the display (180). When the AI ​​shortcut key (202) is pressed within a certain period of time after the power button (201) is pressed while in the power-off state, an execution input may be transmitted (S110) from the remote control device (200) to the recommended speech output module (171).

[0139] The recommended speech output module (171) can transmit a recommended speech request to the history management module (172) (S120). The history management module (172) can transmit the greeting request to a second server (300b) that performs greeting management to display a greeting when the power is on (S130). When the greeting request is transmitted, information regarding the time when the power was switched to the on state, the power state, and the number of times can be transmitted to the second server (300b). Greetings specific to the user and situation can be stored in the greeting database (340b) of the second server (300b).

[0140] The second server (300b) may request user segment information from the UPP server (300C) (S140). User (customer) segment information is information that classifies multiple users (customers) into segments (groups) based on certain criteria according to the user's gender, age, content viewed, and interests. The UPP server (300C) may be configured to store and manage information regarding user preference profiles for each user. In response to the request for user segment information, the UPP server (300C) may transmit the user segment information to the second server (300b) (S150). The second server (300b) may select a specific greeting from among the greetings stored in the database (340b), which is a greeting DB, based on user segment information, time information, and information regarding the power-on status and frequency.

[0141] A greeting including "Morning" can be selected based on the time information when the power is switched to the ON state. Depending on the number of times the power button (201) is turned on and the ON / OFF interval, the second server (300b) can select different greetings, such as an initial greeting or a greeting for the second or third time. When the power is switched to the ON state for the first time in the morning, the second server (300b) can select a greeting, "LGini, I'm glad to be with you this morning too," based on user segment information and transmit it to the history management module (172).

[0142] The history management module (172) can receive a specific greeting selected based on user segment information, time information, and the number of times the power is turned on from the second server (300b). The history management module (172) can transmit the selected specific greeting to the recommended speech output module (171). The recommended speech output module (171) can output the transmitted specific greeting to the screen of the display (180).

[0143] Referring to FIGS. 1 through 6, when a button (e.g., AI shortcut key (202))) of a remote control device (200) is pressed and a key value corresponding to the button is received, the recommended speech output module (171) can transmit a recommended speech request to the history management module (172). The history management module (172) can transmit a greeting request to the second server (300b) based on time information and frequency information, and receive a greeting from the second server (300b). The history management module (172) can transmit the received greeting to the recommended speech output module (171). The recommended speech output module (171) can output a screen containing the received greeting to the display (180).

[0144] Meanwhile, in an AI system including a display device and a server linked with speech data according to the present disclosure, the output of recommended phrases can be performed on a time basis as the power is switched to the ON state. In this regard, FIG. 7 shows screens output to a display device over time in the ON state.

[0145] Referring to FIGS. 6 and FIGS. 7(a), a greeting can be displayed on the display (180) of the display device (100) when the power is on. In this regard, a greeting can be displayed on a lower area on one side of the display (180) while the home screen is displayed as a full screen on the display (180).

[0146] Referring to FIGS. 6, FIGS. 7(a) and FIGS. 7(b), if a certain amount of time elapses without user input while a greeting is displayed, recommended phrases based on the most recent search / viewing history may be displayed on the display (180). If no input is received for a certain amount of time while a greeting is displayed, or if the content app is not launched or a voice command is not executed, recommended utterances / phrases based on the most recent search / viewing history may be displayed. For example, based on recent search history, a recommended utterance "# Find Top Gun review on YouTube" may be displayed as the first priority in the lower area of ​​one side of the display (180). Based on recent search history, a recommended utterance "# Turn on air purifier" may be displayed as the second priority in the lower area of ​​one side of the display (180). Additionally, based on recent viewing history, a recommended phrase for content or a channel "# Yonhap News" may be displayed as the third priority in the lower area of ​​one side of the display (180).

[0147] Referring to FIGS. 6, FIGS. 7(b), and FIGS. 7(c), after a certain period of time has elapsed while a recommended utterance based on the most recent search / viewing history is displayed, the user's search / viewing history can be extracted on a daily, weekly, and monthly basis. Accordingly, recommended phrases based on the user's search / viewing history on a daily, weekly, and monthly basis can be displayed on the display (180). For example, based on the search history of the past day, a recommended utterance "# What is the weather like tomorrow?" can be displayed as the first priority in the lower area of ​​one side of the display (180). Based on the viewing history of the past day, a recommended phrase for the content "# Three Meals a Day" can be displayed as the second priority in the lower area of ​​one side of the display (180). Additionally, based on the viewing history of the past day, a recommended phrase for the content "# Good Partner" can be displayed as the third priority in the lower area of ​​one side of the display (180).

[0148] Meanwhile, an AI system including a display device and a server linked with speech data according to the present disclosure may be configured to analyze the user's intent through the speech data. In this regard, FIG. 8 shows the configuration of a display device configured to store history information regarding intent analysis through speech data and history information regarding content selected on the home screen and the integrated search screen.

[0149] Referring to FIGS. 5 and 8, the voice recognition processing module (175) can recognize the user's voice saying "find me a movie" and perform intent analysis processing of the recognized voice in conjunction with the server (300). The result of the intent analysis of the recognized voice can be transmitted from the server (300) or the voice recognition processing module (175) to the history management module (172). Based on the result of the intent analysis, the history management module (172) can store time information, keywords, genre codes, intent types, and information on running applications in the database (140). According to the voice saying "find me a movie," the keyword "movie" can be extracted and the intent type (IntentType) can be designated as "search".

[0150] Meanwhile, first content information regarding the first content selected via the remote control device (200) on the home screen (180-1) of the display can be transmitted to the history management module (172). Additionally, second content information regarding the second content selected via the remote control device (200) on the integrated search screen (180-2) of the display can be transmitted to the history management module (172). Content information regarding the content selected on the home screen (180-1) and the integrated search screen (180-2) of the display can be transmitted to the history management module (172) and stored in the database (140).

[0151] Keywords for content selected on the home screen (180-1) or integrated search screen (180-2) may be selected as "Pororo" or "Baby Shark," and the IntentType may be designated as search. The IntentType of a keyword selected via voice recognition or a keyword selected on the home / integrated search screen may be classified and designated as the same search. Meanwhile, the keyword "Turn off the TV" may be extracted via voice recognition, and the IntentType of the keyword "Turn off the TV" may be classified and designated as tv_control, which turns off the power of the display device.

[0152] Movie search and playback of the first content may be a first application program such as YouTube. In this regard, voice utterance and recognition may be performed while the first application program is operating in the foreground. A first application program such as YouTube may be selected on the home screen (180-1). Meanwhile, playback of the second content may be a second application program such as web OS. Playback of the second content may be performed through a second application program of a channel service operated by the manufacturer of the display device. Accordingly, different intent analysis results or history management may be performed depending on the application currently running on the display device.

[0153] With reference to FIGS. 1 through 5 and FIG. 8, the processes for loading history information regarding voice utterances or content selected on multiple screens are described in detail. In this regard, the processor (170) may be configured to recognize and process voice data spoken by a user in conjunction with a voice recognition processing module (175). The voice recognition processing module (175) may be configured to transmit the recognized speech data from the remote control device (200) to the server (300).

[0154] The voice recognition processing module (175) can be configured to process an analysis of the user's intent based on speech data in conjunction with the server (300) when a second input exceeding a threshold time is applied to the button of the AI ​​shortcut key (202) of the remote control device (200). Based on the processing result of the intent analysis, the voice recognition processing module (175) can control the display of a search result associated with the recognized speech data on the screen of the display (180).

[0155] The history management module (172) may be configured to store the history of speech data in the database (140) based on the processing result of intent analysis from the server (300). The history management module (172) may be configured to store the history of content information in the database (140) based on content information selected on the home screen of the display (180). The history management module (172) may be configured to store the history of second content information in the database (140) based on second content information selected on the integrated search screen of the display (180).

[0156] Speech data, content information, and second content information stored in the database (140) may include various information. The information stored in the database (140) may include start time information (FromTime), end time information (ToTime), keyword, genre code, intent type, and application information (App) running on the display device (100).

[0157] The IntentType may include a voice utterance, a search performed on the home screen or integrated search screen, or an action (tv_control) associated with controlling the display device in an application associated with webOS-based live TV. The action (tv_control) associated with control may be a voice utterance associated with turning off the power of the display device in an application associated with live TV, or associated with the playback of specific content. Meanwhile, the Keyword may be at least one of the keyword of the user's utterance data, content, and the name of the second content.

[0158] Meanwhile, the speech recognition processing module (175) can control the storage of the history of speech data in the database (140). The history of speech data may include the user's account, STT results, speech date and time, day of the week, and time information. In this regard, the processor (170) can extract the account information of the current login status through the database (140) when a button (e.g., AI shortcut key) of the remote control device (200) is pressed. The processor (170) can extract keyword information associated with the speech used by the user based on the current day of the week and time information. Based on the keyword information, the processor (170) can control the output of a received recommended speech containing the keyword to the display (180).

[0159] Meanwhile, a display device for displaying customized recommended speech according to the present disclosure may be configured to switch the user's login account based on the voice data of the user currently speaking. In this regard, FIGS. 9a and 9b show an initial screen and a screen for changing the profiles of a plurality of users when logging in with a specific user's login account.

[0160] Referring to FIGS. 8 and 9a, an icon (183a) associated with the first user's first login account may be displayed on one side of the home screen (180-1) of the display (180). An icon (183b) that causes a second application program, such as WebOS, to be executed may be displayed on the other side of the home screen (180-1) of the display (180). Meanwhile, icons corresponding to services provided by multiple content servers may be displayed on the display (180) separated by category. Icons corresponding to services provided by multiple content servers may be displayed separated by categories including home office, games, music, home hub, and sports.

[0161] Referring to FIG. 9b, icons (184a, 184b, 184c, 184d) associated with multiple user profiles of multiple users may be displayed on a screen in a specific area of ​​the display (180). Depending on the selection input for the icon (183a) associated with the first login account, icons (184a, 184b, 184c, 184d) of multiple login accounts of family members may be displayed on the screen. The icon (184a) associated with the first login account of the first user corresponding to Dad may be displayed on the screen. For example, the first login account of the first user may be aaa@gmail.com. An icon (185a) for executing a logout for the first login account of the first user who is currently logged in may be displayed in the area below the icon (184a) associated with the first user's profile.

[0162] Meanwhile, an icon (184b) associated with the second login account of the second user corresponding to Chloe may be displayed on the screen. For example, the second login account of the second user may be bbb@gmail.com. An icon (184c) associated with the third login account of the third user corresponding to Mom may be displayed on the screen. Additionally, an icon (184d) for creating a fourth login account of another fourth user may be displayed on the screen.

[0163] Referring to FIGS. 1, 8 to 9a, a configuration and operation for switching a user's login account based on voice data of a user speaking in a display device that displays customized recommended speech is described.

[0164] The voice recognition processing module (175) can determine whether the recognized speech data is voice data of a second user other than the logged-in user by working in conjunction with the server (300). If the second user is a registered user, the processor (170) can display the account switching screen of FIG. 8b associated with whether to switch from the user's login account to the second user's second login account. When the processor (170) receives a switching input to switch to the second login account on the account switching screen of FIG. 8b, the second login account can be switched.

[0165] A first selection box (185a) associated with logging out the first login account may be displayed at the bottom of the first icon (184a) of the first login account of the first user corresponding to Dad. When the first selection box (185a) is selected, the first login account is switched to a logged-out state, and the second login account of the second user may be automatically switched to a logged-in state.

[0166] Meanwhile, a second selection box (185b) associated with the login of the second login account may be displayed at the bottom of the second icon (184b) of the second user's second login account corresponding to Chloe. When the second selection box (185b) is selected, the first login account may be automatically switched to a logged-out state while switching to the second user's second login account.

[0167] Meanwhile, the display device according to the present disclosure can recognize a user and display a customized recommended speech for each user. With reference to FIGS. 3 to 9, a display device for displaying a customized recommended speech for each user according to the present disclosure will be described. A processor (170) can detect whether a user is the first user while logged in with the first login account of the first user among a plurality of user accounts. When the first user is detected, the processor (170) can request a first recommended speech from the server (300) based on the first history of the first speech data of the first user and the first account information. In this regard, the detection of the first user may be achieved through the detection of the first identifier of the first user's first device, the voice data of the first user, or facial detection.

[0168] The processor (170) can detect whether the user is the second user while logged in with the second user's second login account among multiple user accounts. When the second user is detected, the processor (170) can request a second recommended speech from the server (300) based on the second history of the second user's second speech data and the second account information. In this regard, the detection of the second user may be achieved through the detection of the second identifier of the second user's second device, the second user's voiceprint data, or facial detection.

[0169] The processor (170) can determine whether the first user is not detected and the second user is detected while logged in with the first login account. If the first user is not detected and the second user is detected, the processor (170) can display a screen regarding whether to switch to the second user's second login account. If the first user is not detected and the second user is detected, the processor (170) can request a second recommended utterance from the server (300) based on the second history and second account information.

[0170] The processor (170) can determine whether the first user and the second user are detected while logged in with the first login account. If the first and second users are detected, the processor (170) can request the first recommended utterance and the second recommended utterance from the server (300) based on the first history, the second history, the first account information, and the second account information. In this regard, the processor (170) can determine whether the second user is established as a family relationship with the first user through user profile information.

[0171] When the second user is established as a family relationship with the first user, the processor (170) can display a common recommended utterance on the display (180) based on common keywords common to the first recommended utterance and the second recommended utterance. The processor (170) can display the first recommended utterance following the common recommended utterance based on the first user's first keyword. The processor (170) can display the second recommended utterance following the first recommended utterance based on the second user's second keyword.

[0172] If the first and second users are detected and the second user is not established as a family relationship with the first user, the processor (170) may display a common recommended utterance on the display (180) based on common keywords common to the first recommended utterance and the second recommended utterance. The processor (170) may determine whether to display the first recommended utterance based on the first user's first keyword based on the first user's input. When the first user input is received, the processor (170) may display the first recommended utterance on the screen following the common recommended utterance.

[0173] Meanwhile, an AI system including a display device and a server linked with speech data according to the present disclosure can provide user / context-based personalized recommendation services and usage tip services. In this regard, FIG. 10 illustrates a configuration for providing user / context-based personalized recommendation services according to an embodiment. Referring to FIG. 3, FIG. 6 and FIG. 10, when the power is switched to an ON state and the AI ​​shortcut key (202) is pressed within a certain period of time, a greeting can be displayed on the screen of the display (180). When the AI ​​shortcut key (202) is pressed, current login account information, current time, and user access record information can be transmitted from the remote control device (200) to the processor (170).

[0174] The second server (300b) can transmit login account information to the UPP server (300C). Based on the login account information, the UPP server (300C) can obtain customer segment information and transmit the obtained customer segment information to the second server (300b). The second server (300b) can generate and recommend a welcome message based on the current time, customer segment information, and user access record information. The processor (170) can transmit the welcome message received from the second server (300b) to the display (180) to control the display (180) to output the welcome message on the screen. Thus, a welcome message saying "Lelgenie, I'm glad to be with you this morning" can be displayed on the screen of the display (180).

[0175] Meanwhile, depending on the time and number of times the power is switched to the power-on state, the display device (100) may display different welcome messages, such as greetings. Referring to FIGS. 1 through 6, the processor (170) determines whether the power is switched from the power-off state to the power-on state for the first time on each date. When the power is switched to the power-on state for the first time, the processor (170) may output a screen containing a first greeting to the display (180) based on time information. When the power is switched to the power-on state for the first time, the first greeting based on time information may be generated as a sentence such as "L-Genie, I am happy to be with you this morning as well," as shown in FIG. 6.

[0176] Meanwhile, if the number of times the power is switched to the ON state is 2 or more, the processor (170) can output a screen containing a second greeting different from the first greeting to the display (180). If the number of times the power is switched to the ON state is 2 or more, the second greeting based on time information can be generated as the sentence "L-Genie, I'm glad to be able to be with you again this morning" as in FIG. 10.

[0177] Meanwhile, when the power is on and a first or second greeting is displayed, if a certain amount of time elapses, the recommended utterances of FIGS. 4 and FIGS. 5 may be displayed on the screen. Referring to FIGS. 1 to 5, the processor (170) may display a sentence containing the user's name (e.g., Jiwon, please select the utterances you used frequently during the past week.) on the screen. The sentence containing the user's name may be displayed in the first area, and recommended utterances based on the user's utterance history may be displayed in the second area.

[0178] In this regard, the processor (170) can extract keywords based on the frequency of utterances based on the history of utterance data for each week when a first threshold time has elapsed while the power is on. The processor (170) can display the extracted keywords in order of utterance frequency. Recommended utterances may be displayed with keywords combined with hashtags. Recommended utterances may be displayed with sentences containing keywords combined with hashtags. The keywords of the recommended utterances may be first and second content (first and second programs) that can be viewed through the display device (100). Additionally, the keywords of the recommended utterances may be sentences spoken by the user, for example, "What is the weather like tomorrow?"

[0179] Referring to FIGS. 1 through 10, the processor (170) can transmit account information of the current login status, the current time, and the user's connection record to the second server (300b). The second server (300b) can transmit the account information to the UPP server (300c) to receive customer segment information from the UPP server (300c). The second server (300b) can transmit a welcome message generated based on the current time, customer segment information, and the user's connection record to the processor (170). The processor (170) can control the welcome message to be output to the display (180).

[0180] Accordingly, the processor (170) can provide a context-specific recommendation function. In this regard, depending on the power-on of the display device, a context-specific greeting can be provided based on the time period in which the power is on, the frequency / interval of power on / off, and the time-specific segment information of the time period in which the display device is mainly used.

[0181] In addition, it can prioritize recommending features that are immediately usable when the display device is powered on, based on the time of day. By analyzing the applications currently in use or the content being watched while the display device is in use, it can recommend keywords most suitable for the situation. Depending on the characteristics of the currently running application, it can suggest other highly relevant applications or settings as usage tips. Through the title, genre, and metadata information of the content being watched, it can provide viewing-related keywords such as cast, director, ratings, and OST.

[0182] Meanwhile, a display device for displaying customized recommendation utterances according to the present disclosure and an AI system including the same can provide user / context-based customized usage tip services. In this regard, FIG. 11 illustrates a configuration for providing user / context-based customized usage tip services according to an embodiment. Referring to FIG. 3 and FIG. 11, when an AI shortcut key (202) is pressed, usage tip phrases considering the program being watched and the application being used can be displayed on the screen of the display (180). When the AI ​​shortcut key (202) is pressed, the processor (170) can transmit information regarding the current time, the connected device, the program currently being watched, and the application currently being used to the second server (300b). In this regard, the processor (170) can receive selected usage tip information based on the identifier and status information of the connected device.

[0183] The second server (300b) can generate and recommend customized use tip information based on the current time, connected device, program currently being watched, and application currently being used. The processor (170) can receive customized use tip information or associated use tip keywords from the second server (300b).

[0184] The processor (170) can control the display (180) to output the usage tip keywords received from the second server (300b) to the display (180). Thus, usage tip information such as "Find Top Gun reviews on YouTube," "Turn off the air purifier," and "What is the weather like tomorrow?" can be displayed on the screen of the display (180).

[0185] In this regard, if the first application program called YouTube is currently running in the foreground on the display device, the user tip information "Find Top Gun reviews on YouTube" can be displayed on the screen of the display (180).

[0186] Meanwhile, if the display device and the air purifier are connected devices, usage tip information such as "Turn off the air purifier" may be displayed on the screen of the display (180). In this regard, FIGS. 12a and FIGS. 12b show screens of the display device that display status information of devices connected to the display device.

[0187] Referring to FIGS. 12a and 12b, status information of an air purifier, a washing machine, a dryer, and a refrigerator may be displayed on the display (180) of the display device. Additionally, status information of a light, a switch, and blinds may be displayed on the display (180).

[0188] Referring to FIG. 12a, one of the devices (400) connected to the display device (100) may be an air purifier. As shown on the control screen of the home hub of the display (180), the air purifier may be in a running state (400a). Referring to FIG. 12b, the running state of the devices (400) connected to the display device (100) may be changed. As shown on the control screen of the home hub of the display (180), the air purifier may be in an off state (400b). In this regard, as shown in FIG. 11, a user tip information such as "Turn off the air purifier" may be selected on the screen of the display (180).

[0189] As illustrated in FIG. 11, when the AI ​​shortcut key of the remote control device (200) is pressed, a user-customized menu such as usage tips is provided on the screen or as voice, or the user's voice may be recognized depending on the form in which the AI ​​shortcut key is pressed. Thus, the operation of some devices that may interfere with voice recognition / output among the devices connected to the display device can be temporarily suspended. Thus, when the AI ​​shortcut key of the remote control device (200) is pressed, usage tip information such as "Turn off the air purifier" may be displayed on the screen of the display (180). Thus, as the usage tip information "Turn off the air purifier" is selected, the air purifier among the devices (400) connected to the display device (100) may be switched from an operating state (400a) to an off state (400b), as shown in FIG. 12a and FIG. 12b.

[0190] In this regard, the display screen may switch from the screen of FIG. 11 to the screen of FIG. 12b depending on user input selecting the usage tip information "Turn off the air purifier." Meanwhile, the screen switching and state switching are not limited to user input, and if a cancellation input is not received within a certain period of time, the air purifier may automatically switch to the off state (400b).

[0191] Meanwhile, based on the status information of the air purifier, a user tip information such as "Turn off the air purifier" may be displayed on the screen of the display (180). In this regard, if the air purifier has operated for a certain period of time or longer and it is determined that the concentration of fine particles in the current indoor air is below a threshold, the status information of the air purifier and the air purification status may be determined to be in a normal state. If the status information of the air purifier and the air purification status are in a normal state, a user tip information such as "Turn off the air purifier" may be displayed on the screen of the display (180).

[0192] Meanwhile, if the air purifier is off and it is determined that the concentration of fine particles in the indoor air is above a threshold, a usage tip information such as "Turn on the air purifier" may be displayed on the screen of the display (180). As the usage tip information such as "Turn on the air purifier" is selected, the display screen may switch from the screen of FIG. 11 to the screen of FIG. 12a.

[0193] Referring to FIGS. 1 to 12b, the processor (170) can detect whether a button (e.g., AI shortcut key (220)) of the remote control device (200) has been pressed. When the processor (170) detects a first input of the AI ​​shortcut key (220), it can transmit information regarding the current time, connected device, program currently being watched, and application currently being used to a second server (300b). Based on the information regarding the current time, connected device, program currently being watched, and application currently being used, the processor (170) can receive customized use tip information from the second server (300b). Based on the customized use tip information, the processor (170) can control the output of at least one sentence associated with the use tip to the display (180).

[0194] The aforementioned personalized usage tips and recommended utterances can be provided based on user-customized keywords per account. Regarding user-customized keywords per account, user data is managed based on user accounts, and account-specific preferences can be learned through voiceprint recognition during voice search. User-customized keywords can be recommended by integrating and inferring user data such as account-specific voice utterance history, viewed content, TV settings, connected home IoT devices, and preference segments classified according to usage patterns. User preferences can be analyzed based on accumulated user data by day and time, and the latest information of interest can be continuously updated based on recent data. In the case of content, optimal recommendations can be provided by utilizing content preference information from clusters that have watched the same content, while recommendations based on the user's preferred genres serve as the foundation.

[0195] Meanwhile, an AI system including a display device and a server linked with speech data according to the present disclosure may be configured to provide a broadcast information-based related keyword recommendation service. In this regard, FIG. 13 shows the configuration of an AI system that provides a broadcast information-based related keyword recommendation service according to an embodiment.

[0196] Referring to FIG. 13, the AI ​​system may be configured to include a display device (100), a server (300), a third server (500) for extracting associated keywords, and a fourth server (600) associated with an Integrated Broadcasting Information System (IBIS). The display device (100) may include a processor (170) that requests and receives information in conjunction with the servers and a display (180) that displays associated keywords.

[0197] The processor (170) may request (S210) broadcast information of a program watched from a server (300) or a database (140). The broadcast information received from the server (300) or the database (140) may be transmitted (S220) to a display device (100). The processor (170) may receive broadcast information from the server (300) or the database (140). In this regard, the entity managing the broadcast information of the program watched may be the database (140) of the server (300) or the display device (100).

[0198] The processor (170) can transmit broadcast information (S230) to a third server (500) that extracts associated keywords. The third server (500) can request program details information (S240) from a fourth server (600) or database (140) associated with the Integrated Broadcast Information System (IBIS). The fourth server (600) or database (140) can transmit program details information to the third server (500) (S250). The third server (500) can transmit the broadcast information-based associated keywords extracted based on the program details information to a display device (100) (S260). The processor (170) can transmit the associated keywords to a display (180) (or screen output unit) so that the broadcast information-based associated keywords are output to the display (180).

[0199] Referring to FIGS. 1 through 13, the processor (170) may be configured to receive broadcast information associated with a user's viewing program from a server (300). The processor (170) may be configured to transmit the broadcast information to a third server (500) that extracts associated keywords associated with the broadcast information. The third server (500) may request program details associated with the broadcast information from a fourth server (600) of the broadcast information system. In response, the third server (500) may be configured to receive program details associated with the broadcast information from the fourth server (600). The processor (170) may be configured to output associated keywords associated with the broadcast information to a display (180) based on the program details.

[0200] Meanwhile, a method for displaying customized recommended utterances performed by a display device and an AI system including the same according to the present disclosure is described. In this regard, FIG. 14 shows a flowchart of a method for displaying customized recommended utterances performed by a display device and an AI system including the same. FIG. 15 shows a flowchart of a method for displaying associative search results based on intent analysis of speech recognition performed by a display device and an AI system including the same. Referring to FIG. 1 through 15, the method for displaying recommended utterances can be performed by a processor (170) of a display device (100).

[0201] The processor (170) can generate a recommended speech request when a button (AI shortcut key (220)) of the remote control device (200) is pressed for a first time less than a threshold time and a first input is applied (S10). The recommended speech output module (171) can transmit the generated recommended speech request to the history management module (172) (S20). The history management module (172) can request the history of previous speech data from the database (140) in response to the recommended speech request (S30). The history management module (172) can receive the history of previous speech data from the database (140) (S40).

[0202] The history management module (172) can request a recommended utterance from the server (300) based on the history of the utterance data and the user's account information (S50). The history management module (172) can receive the recommended utterance from the server (300) (S60). The history management module (172) can transmit the received recommended utterance to the recommended utterance output module (171) (S70). The recommended utterance output module (171) can control the received recommended utterance containing keywords to be displayed on the screen of the display (180).

[0203] Meanwhile, when a second input is applied (S10b) by pressing a button (AI shortcut key (220)) of a remote control device (200) for a second time longer than a threshold time, the processor (170) can recognize the user's speech data (voice) through a speech recognition processing module (175). The speech recognition processing module (175) can transmit the recognized speech data to a server (300) (S20b).

[0204] The speech recognition processing module (175) and the recommended speech output module (171) can receive (S30b) the processing result of the user's intent analysis associated with the recognized speech data from the server (300). Based on the processing result of the intent analysis, the recommended speech output module (171) can control (S40b) that the search result associated with the recognized speech data is displayed on the screen of the display (180). The speech recognition processing module (175) and the recommended speech output module (171) can control (S50b) that the speech data is loaded into the database (140) through the history management module (172). Meanwhile, the speech data stored in the database (140) can have its history transferred to the history management module (172) (S40) in accordance with the speech data history request (S30) of the history management module (172).

[0205] Meanwhile, a method for displaying recommended utterances performed by a customized display device according to another aspect of the present disclosure is not limited thereto and can be applied in various ways in combination with the embodiments of FIGS. 1 to 12.

[0206] The foregoing has described a display device for displaying customized recommended utterances and an AI system including the same according to the present specification. The technical effects of the display device for displaying customized recommended utterances and the AI ​​system including the same according to the present specification may be summarized as follows, but are not limited thereto.

[0207] According to the present specification, personalized recommendations are possible by managing usage data by account, and users can be distinguished through voice recognition, enabling user-specific data learning through speech alone without a separate login process.

[0208] According to the present specification, recommendations can be received immediately by pressing an AI key anytime and anywhere, and desired functions or content can be quickly executed by significantly reducing search time through advanced logic that increases recommendation accuracy.

[0209] According to the present specification, TV can be enjoyed more immersively by providing settings or related information optimized for the content being watched and providing usage tips that make better use of the app being used.

[0210] Meanwhile, a display device that displays customized recommended utterances according to the present specification may be referred to as an AI concierge that performs AI functions. The AI ​​concierge may provide customized greetings based on user segments or situations upon power-on of the display device. Additionally, while using the display device, it may recommend recommended utterances containing user and situation-specific keywords based on the user's intent and usage context. Recommended utterances may be displayed on the screen of the display device to recommend keywords, specific functions, settings, content, etc.

[0211] Regarding user-customized keywords per account, user data is managed based on user accounts, and account-specific preferences can be learned through voiceprint recognition during voice searches. User-customized keywords can be recommended by integrating and inferring user data such as account-specific voice utterance history, viewed content, TV settings, connected home IoT devices, and preference segments classified according to usage patterns. User preferences are analyzed based on accumulated user data by day and time, and the latest information of interest can be continuously updated based on recent data. For content, while recommendations based on the user's preferred genres serve as the foundation, optimal recommendations can be provided by utilizing content preference information from clusters that have watched the same content.

[0212] Regarding context-based recommendations, context-specific greetings can be provided based on the display device's power-on status, the time of day it is powered on, the frequency / interval of power-on / power-off, and segment information regarding the time of day when the device is primarily used. Additionally, features that are immediately available for use can be recommended prioritized based on the time of day when the display device is powered on. Keywords most suitable for the situation can be recommended by analyzing the applications currently in use or the content being watched while the display device is in use. Based on the characteristics of the currently running application, other highly relevant applications or settings can be suggested as usage tips. Keywords related to viewing, such as cast, director, ratings, and OST, can be provided through the title, genre, and metadata information of the content being watched.

[0213] The foregoing disclosure may be implemented as computer-readable code on a medium on which a program is recorded. A computer-readable medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include Hard Disk Drives (HDDs), Solid State Disks (SSDs), Silicon Disk Drives (SDDs), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, etc., and also include implementations in the form of carrier waves (e.g., transmission over the Internet). Furthermore, the computer may include a control unit of a terminal. Accordingly, the foregoing detailed description should not be interpreted restrictively in all respects and should be considered exemplary. The scope of the invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the invention are included within the scope of the invention.

Claims

1. In a display device that displays a customized recommended utterance, A user input interface unit configured to receive a first input associated with a recommended speech request and a second input associated with the recognition of the user's speech data in conjunction with a remote control device; A display configured to display information or recommended utterances associated with the above-mentioned utterance data; and A processor configured to display keywords associated with the above utterance data or the user's previous utterance data on the display, and The above processor is, When a key value corresponding to a button of the above remote control device is received, the above recommended speech request is generated, and In response to the above recommended utterance request, the history of the above previous utterance data is requested from the database, and Requesting recommended utterances from the server based on the history of the above utterance data and the user's account information, and Receive the recommended utterance from the above server, and A display device that controls the received recommended utterance containing the above keywords to be output to the display.

2. In Paragraph 1, The above processor is, A recommended utterance output module configured to execute an application program and output the recommended utterance when a first input of less than a threshold time is applied to the above button; and It includes a history management module that receives the recommended utterance request from the recommended utterance output module and receives the history of the utterance data from the database, The above history management module transmits the above recommended utterance received from the server to the above recommended utterance output module, a display device.

3. In Paragraph 2, When a key value corresponding to the power button of the remote control device is received, the recommended speech output module transmits the recommended speech request to the history management module, and The above history management module transmits a greeting request to the second server and receives a greeting based on time information from the second server, and The above history management module transmits the received greeting to the above recommended utterance output module, and The above recommended speech output module is a display device that outputs a screen containing the above received greeting to the display.

4. In Paragraph 3, The above processor is, When the power is first switched from the power-off state to the power-on state on a specific date, a screen including a first greeting is output to the display based on the time information, and If the number of times the power is switched to the above-mentioned power-on state is two or more, a screen including a second greeting different from the above-mentioned first greeting is output to the above-mentioned display, and When the first threshold time elapses in the above power-on state, keywords are extracted based on the utterance frequency based on the history of utterance data for each week, and A display device that displays the above-mentioned extracted keywords in the order of the above-mentioned utterance frequencies.

5. In Paragraph 2, The processor further includes a voice recognition processing module configured to transmit the recognized speech data from the remote control device to a server, and The above voice recognition processing module is, When a second input exceeding a threshold time is applied to the above button, the analysis of the user's intent associated with the recognized utterance data is processed in conjunction with the above server, and A display device that controls the display of a search result associated with the recognized utterance data on the screen of the display based on the processing result of the above intention analysis.

6. In Paragraph 5, The above voice recognition processing module is, In conjunction with the above server, determine whether the recognized speech data is voiceprint data of a second user other than the user, and The above processor is, If the above second user is a registered user, an account switching screen related to whether to switch from the user's login account to the second user's second login account is displayed, and A display device that switches to the second login account when a switching input to switch to the second login account is received on the account switching screen.

7. In Paragraph 6, The above processor is, Requesting a second recommended utterance to the server based on the second history of the second user's utterance data and the second account information of the second user, and A display device that receives the second recommended utterance from the server and controls the received second recommended utterance, including the second keyword, to be output to the display.

8. In Paragraph 7, The above history management module is, Based on the processing result of the intent analysis from the above server, the history of the utterance data is stored in the above database, and Based on the content information selected on the home screen of the above display, the history of the content information is stored in the above database, and A display device that stores the history of the second content information in the database based on the second content information selected from the integrated search screen of the display.

9. In Paragraph 8, The utterance data, the content information, and the second content information stored in the above database include start time information, end time information, keywords, genre codes, intent types, and application information running on the display device, and A display device characterized in that the above keyword is the keyword of the above utterance data, the above content, and the name of the above second content.

10. In Paragraph 5, The above speech recognition processing module controls the history of the above speech data to be stored in the database, and The above history is a display device including the user's account, STT result, utterance time and date, day of the week and time information.

11. In Paragraph 10, The above processor is, When a key value corresponding to the above button is received, account information of the current login status is extracted through the above database, and keyword information associated with the utterance used by the above user is extracted based on the current day and time information, and A display device that controls the received recommended utterance including the keyword to be output to the display based on the keyword information.

12. In Paragraph 3, The above processor is, Transmitting the account information of the current login status, the current time, and the connection records of the said user to the said second server, and The second server transmits the account information to the UPP server to receive customer segment information from the UPP server, and transmits a welcome message generated based on the current time, the customer segment information, and the user's access record to the processor. The above processor is a display device that controls the welcome message to be output to the display.

13. In Paragraph 3, The above processor is, When a key value corresponding to the above button is received, information regarding the current time, connected device, currently watching program, and currently using application is transmitted to the above second server, and Based on the current time, the connected device, the program being watched, and the application being used, customized use tip information is received from the second server, and A display device that outputs at least one sentence associated with a usage tip to the display based on the above customized usage tip information.

14. In Paragraph 13, The above processor is, Obtaining the application information currently in use and the user, A display device that outputs at least one sentence related to the search or playback of contents provided by the application, based on the application information currently in use, the history of the utterance data, and the user's account information.

15. In Paragraph 13, The above processor is, Acquire status information of the connected device and the display device and environmental information of the area where the connected device is placed, and A display device that outputs at least one sentence associated with a usage tip capable of controlling the operation of the connected device based on the above state information and the above environment information on the screen of the display.

16. In Paragraph 3, The above processor is, Receiving broadcast information related to the user's viewing program from the server, and The broadcast information is transmitted to a third server that extracts associated keywords related to the broadcast information, and The above-mentioned third server receives program details associated with the broadcast information from the fourth server of the broadcast information system, and A display device that outputs the associated keywords associated with the broadcast information to the display based on the above program details.

17. In Paragraph 1, The above processor is, When the first user is detected while logged in with the first login account of the first user among multiple user accounts, a first recommended utterance is requested to the server based on the first history of the first user's first utterance data and the first account information, and When the second user is detected while logged in with the second login account of the second user among multiple user accounts, a second recommended utterance is requested to the server based on the second history of the second utterance data of the second user and the second account information, and Detection of the first identifier of the first device of the first user or detection of the first user is performed through the voice data of the first user, and A display device in which detection of the second user is performed through detection of the second identifier of the second device of the second user or detection of the voice data of the second user.

18. In Paragraph 17, The above processor is, If the first user is not detected and the second user is detected while logged in with the first login account, the second recommended utterance is requested to the server based on the second history and the second account information, and When the first user and the second user are detected while logged in with the first login account, the first recommended utterance and the second recommended utterance are requested to the server based on the first history, the second history, the first account information and the second account information, and Based on the common keywords of the first recommended utterance and the second recommended utterance, a common recommended utterance is displayed on the display, and Based on the first keyword of the first user, the first recommended utterance is displayed following the common recommended utterance, and A display device characterized in that the second user is established as a family relationship with the first user.

Images