Display device and system including the same

Abstract

This provides a function that allows users to input text into a display device using their own terminal. [Solution] This disclosure relates to a display device and system. A display device according to one embodiment of this disclosure includes a display, a network interface, and a controller, the controller which, in response to the activation of a virtual input interface, blocks the output of a virtual keyboard via the display, and, with the virtual input interface activated, receives a key code from a predetermined user terminal via the network interface, and displays text via the display based on a string corresponding to the received key code.

Description

Technical Field

[0001] The present disclosure relates to a display device and a system including the same, and more particularly, to a display device that provides a function of controlling a display device using a user terminal and a system including the same.

Background Art

[0002] The Internet of Things (IoT) technology refers to intelligent technologies and services that connect all things based on the Internet, that is, a network, and enable information to be communicated between people and things, and between things.

[0003] Recently, the Internet of Things technology has evolved into a smart home appliance platform that connects various electronic devices installed in a home, such as display devices, refrigerators, washing machines, air conditioners, etc., via a network, allowing a user to remotely control each electronic device or check the operating status of each electronic device in real time.

[0004] A display device is a device having a function of displaying an image that can be viewed by a user. For example, the display device can include a television (Television), a monitor, a notebook computer, etc. equipped with a liquid crystal display (LCD) using liquid crystal or an OLED display using an organic light emitting diode (hereinafter OLED).

[0005] A display device such as a recently introduced smart TV can provide not only broadcasts but also various services and functions. In order to more effectively utilize such services or functions and enhance the operational convenience of a user, remote control devices connected to the display device have evolved into various forms.

[0006] Traditionally, users controlled display devices using remote control devices that utilized infrared (IR) signals, but recently, research has been actively conducted on methods of controlling display devices using user terminals to improve user convenience. In particular, when a user tries to input text into a display device, the conventional method involves the display device showing a screen that includes a virtual keyboard, and the user has the inconvenience of having to select each key on the virtual keyboard using a pointer corresponding to the remote control device displayed on the screen.

[0007] On the other hand, when a user inputs text into a display device using a user terminal, conventionally, after the user inputs text into the user terminal, triggering the transmission of the input text typically sends a string representing the entire input text to the display device. However, in order to correct the text entered into the display device, the user must re-enter the entire text they intend to input into the user terminal, which presents a problem in that it is difficult to recommend keywords related to the text entered by the user in real time.

[0008] Furthermore, the user terminal can be configured to send a string representing the entire text entered so far to the display device each time the user enters text. However, this presents a problem: the amount of data required to send the text increases as the amount of text entered by the user increases, resulting in slower transmission speeds. [Overview of the project] [Problems that the invention aims to solve]

[0009] This disclosure aims to resolve the aforementioned issues and other issues.

[0010] Another objective is to provide a display device and a system including the same that provides the function of inputting text into the display device using a user terminal.

[0011] Another objective is to provide a display device and a system including the same that can easily modify the text displayed on the display device based on input received from a user terminal.

[0012] Another objective is to provide a display device and a system including the same that can minimize the amount of data used for text input.

[0013] Another objective is to provide a display device and a system including the same that can prevent the content displayed on the display device's screen from being obscured by a virtual keyboard when text is entered using a user terminal.

[0014] Another objective is to provide a display device and a system including the same that, in response to the cessation of text input using a user terminal, displays a screen including a virtual keyboard so that text input can be continued.

[0015] Another objective is to provide a display device and a system including the same that can guide the user to use the optimal user terminal for text input. [Means for solving the problem]

[0016] To achieve the aforementioned objective, a display device according to one embodiment of the present disclosure includes a display, a network interface, and a controller, wherein the controller can, in response to the activation of a virtual input interface, block the output of a virtual keyboard via the display, receive a key code from a predetermined user terminal via the network interface while the virtual input interface is activated, and display text via the display based on a string corresponding to the received key code.

[0017] A system according to one embodiment of the present disclosure for achieving the above objective includes a display device and at least one user terminal, wherein the display device, in response to the activation of a virtual input interface, blocks the output of a virtual keyboard via a first display of the display device, receives a key code from the user terminal while the virtual input interface is activated, displays text via the first display based on a string corresponding to the received key code, and the user terminal can output the virtual keyboard via a second display device of the user terminal and transmit the key code corresponding to the keys included in the virtual keyboard to the display device. [Effects of the Invention]

[0018] The effects of the display device and system including the same as described herein are as follows:

[0019] According to at least one embodiment of this disclosure, a function can be provided that allows text to be entered into a display device using a user terminal.

[0020] According to at least one embodiment of this disclosure, text displayed on a display device can be easily modified based on input received from a user terminal.

[0021] According to at least one embodiment of the present disclosure, the amount of data used for text input can be minimized.

[0022] According to at least one embodiment of the present disclosure, when text is input using a user terminal, it is possible to prevent the content displayed on the screen of the display device from being hidden by the virtual keyboard.

[0023] According to at least one embodiment of the present disclosure, in response to the stop of text input using a user terminal, the display device can display a screen including a virtual keyboard to continue text input.

[0024] According to at least one embodiment of the present disclosure, it is possible to guide a user to use an optimal user terminal for text input.

Brief Description of the Drawings

[0025] [Figure 1] It is a block diagram of a display device according to an embodiment of the present disclosure. [Figure 2] It is a block diagram of a remote control device according to an embodiment of the present disclosure. [Figure 3] It is a diagram showing an example of the configuration of a remote control device according to an embodiment of the present disclosure. [Figure 4] It is a diagram showing an example of utilizing a remote control device according to an embodiment of the present disclosure. [Figure 5] It is a diagram for explaining the webOS architecture according to an embodiment of the present disclosure. [Figure 6] It is a diagram for explaining the architecture of a webOS device according to an embodiment of the present disclosure. [Figure 7] It is a diagram for explaining the graphic composition flow in a webOS device according to an embodiment of the present disclosure. [Figure 8] It is a diagram showing an example of the configuration of a system according to an embodiment of the present disclosure. [Figure 9] This is a flowchart illustrating the operation method of a display device according to one embodiment of the present disclosure. [Figure 10] This is a flowchart illustrating the operation method of a display device according to one embodiment of the present disclosure. [Figure 11] This is a flowchart illustrating the operation method of a display device according to one embodiment of the present disclosure. [Figure 12] This is a flowchart illustrating the operation method of a display device according to one embodiment of the present disclosure. [Figure 13a] This figure is referenced in the description of text input using a virtual keyboard according to the embodiments of this disclosure. [Figure 13b] This figure is referenced in the description of text input using a virtual keyboard according to the embodiments of this disclosure. [Figure 14] This figure is referenced in the description of text input using a virtual keyboard according to the embodiments of this disclosure. [Figure 15] This figure is referenced in the description of text input using a virtual keyboard according to the embodiments of this disclosure. [Figure 16] This figure is referenced in the description of text input using a virtual keyboard according to the embodiments of this disclosure. [Figure 17] This figure is referenced in the description of text input using a virtual keyboard according to the embodiments of this disclosure. [Figure 18] This figure is referenced in the description of text input using a virtual keyboard according to the embodiments of this disclosure. [Figure 19] This figure is referenced in the description of text input using a virtual keyboard according to the embodiments of this disclosure. [Figure 20] This figure is referenced in the description of text input using a virtual keyboard according to the embodiments of this disclosure. [Figure 21]This figure is referenced in the description of text input using a virtual keyboard according to the embodiments of this disclosure. [Modes for carrying out the invention]

[0026] The present disclosure will be described in detail below with reference to the drawings. In order to illustrate the present disclosure clearly and concisely, the drawings omit illustrations of parts that are not relevant to the description, and the same reference numerals are used throughout the specification for parts that are identical or very similar.

[0027] The suffixes "module" and "part" used in the following description for the constituent elements are provided solely for the ease of writing this specification and do not inherently impose any particular meaning or role. Therefore, "module" and "part" may be used interchangeably.

[0028] In this application, terms such as "may include" or "may have" are intended to specify the presence of features, figures, stages, operations, components, parts, or combinations thereof as described in the specification, and should be understood not to preemptively exclude the presence or possibility of adding one or more other features, figures, stages, operations, components, parts, or combinations thereof.

[0029] Furthermore, while terms such as "first," "second," etc., are used in this specification to describe various elements, such elements are not limited by these terms. These terms are used solely to distinguish one element from another.

[0030] Figure 1 is a block diagram of a display device according to one embodiment of the present disclosure.

[0031] As shown in Figure 1, the display device 100 may include a broadcast receiving unit 130, an external device interface 135, a memory 140, a user input interface 150, a controller 170, a wireless communication interface 173, a display 180, a speaker 185, and a power supply circuit 190.

[0032] The broadcast receiving unit 130 may include a tuner 131, a demodulator 132, and a network interface 133.

[0033] The tuner 131 can select a specific broadcast channel by channel selection command. The tuner 131 can receive broadcast signals for the selected specific broadcast channel.

[0034] The demodulator 132 can separate the received broadcast signal into a video signal, an audio signal, and data signals associated with the broadcast program, and can restore the separated video signal, audio signal, and data signals into an output-ready form.

[0035] The external device interface 135 can receive applications or application lists from adjacent external devices and transmit them to the controller 170 or memory 140.

[0036] The external device interface 135 can provide a connection path between the display device 100 and an external device. The external device interface 135 can receive one or more video and audio outputs from an external device connected to the display device 100 wirelessly or via a wired connection and transmit them to the controller 170. The external device interface 135 can include multiple external input terminals. These multiple external input terminals can include RGB terminals, one or more HDMI® (High Definition Multimedia Interface) terminals, and component terminals.

[0037] Video signals from external devices input via the external device interface 135 can be output via the display 180. Audio signals from external devices input via the external device interface 135 can be output via the speaker 185.

[0038] External devices that can be connected to the external device interface 135 may include any one of the following: a set-top box, Blu-ray player, DVD player, game console, soundbar, smartphone, PC, USB memory, or home theater system; this is merely an example.

[0039] The network interface 133 can provide an interface for connecting the display device 100 to a wired / wireless network, including the Internet. The network interface 133 can transmit or receive data with other users or other electronic devices via the connected network or other networks linked to the connected network.

[0040] Furthermore, the display device 100 can transmit some of the content data stored in the display device 100 to other users or other electronic devices that have been pre-registered in the display device 100, or to selected users or selected electronic devices.

[0041] The network interface 133 can connect to a predetermined web page via the connected network or another network linked to the connected network. That is, it can connect to a predetermined web page via the network and send or receive data with the corresponding server.

[0042] Furthermore, the network interface 133 can receive content or data provided by content providers or network operators. In other words, the network interface 133 can receive content such as movies, advertisements, games, VOD, and broadcast signals, as well as related information, provided by content providers or network providers via the network.

[0043] Furthermore, the network interface 133 can receive firmware update information and update files provided by the network operator, and can transmit data to the internet, content providers, or network operators.

[0044] The network interface 133 can select and receive desired applications from among those publicly available (open) applications via the network.

[0045] The memory 140 can store programs for each signal processing and control within the controller 170, and can also store processed video, audio, or data signals.

[0046] Furthermore, the memory 140 can also function to temporarily store video, audio, or data signals input from the external device interface 135 or the network interface 133, and can also store information related to a predetermined image through its channel memory function.

[0047] Memory 140 can store applications or application lists input from the external device interface 135 or the network interface 133.

[0048] The display device 100 can play and provide to the user content files (video files, still image files, music files, document files, application files, etc.) stored in the memory 140.

[0049] The user input interface 150 can transmit signals input by the user to the controller 170, or transmit signals from the controller 170 to the user. For example, the user input interface 150 can receive and process control signals such as power on / off, channel selection, and screen settings from the remote control device 200 using various communication methods such as Bluetooth®, Ultra Wideband (WB), ZigBee®, Radio Frequency (RF) communication, or infrared (IR) communication, or it can process control signals from the controller 170 to transmit them to the remote control device 200.

[0050] Furthermore, the user input interface 150 can transmit control signals input from local keys (not shown), such as the power key, channel key, volume key, and setting value, to the controller 170.

[0051] The video signal processed by the controller 170 can be input to the display 180 and displayed as the corresponding video signal. Furthermore, the video signal processed by the controller 170 can be input to an external output device via the external device interface 135.

[0052] The audio signal processed by the controller 170 can be output as audio from the speaker 185. Furthermore, the audio signal processed by the controller 170 can be input to an external output device via the external device interface 135.

[0053] In addition, the controller 170 can control the overall operation of the display device 100.

[0054] Furthermore, the controller 170 can control the display device 100 by user commands or internal programs input via the user input interface 150, and can connect to a network to allow users to download desired applications or application lists into the display device 100.

[0055] The controller 170 ensures that the channel information selected by the user, along with the processed video or audio signal, is output via the display 180 or speaker 185.

[0056] Furthermore, the controller 170, upon receiving an external device video playback command via the user input interface 150, enables the output of video signals or audio signals from an external device, such as a camera or video camera, input via the external device interface 135, through the display 180 or speaker 185.

[0057] On the other hand, the controller 170 can control the display 180 to display video, for example, broadcast video input via the tuner 131, external input video input via the external device interface 135, video input via the network interface unit, or video stored in the memory 140 to be displayed on the display 180. In this case, the video displayed on the display 180 may be a still image or a moving image, and may be 2D video or 3D video.

[0058] Furthermore, the controller 170 can control the playback of content stored in the display device 100, received broadcast content, or external input content input from an external source. This content may take various forms, such as broadcast video, external input video, audio files, still images, connected web pages, and document files.

[0059] The wireless communication interface 173 can communicate with external devices via wired or wireless connection. The wireless communication interface 173 is capable of short-range communication with external devices. For this purpose, the wireless communication interface 173 uses Bluetooth. TM The wireless communication interface 173 can support short-range communication using at least one of the following technologies: RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus). Such a wireless communication interface 173 can support wireless communication between the display device 100 and a wireless communication system, between the display device 100 and other display devices 100, or between the display device 100 and a network where the display device (100, or an external server) is located, via a Wireless Area Network. The Near Field Network may also be a Wireless Personal Area Network.

[0060] Here, the other display device 100 may be a wearable device (e.g., a smart watch, smart glasses, HMD (head-mounted display)), a mobile terminal such as a smartphone, that is capable of exchanging (or interacting with) the display device 100 according to this disclosure. The wireless communication interface 173 can sense (or recognize) a communicable wearable device in the vicinity of the display device 100.

[0061] Furthermore, if the sensed wearable device is a device authenticated to communicate with the display device 100 according to this disclosure, the controller 170 can transmit at least a portion of the data processed in the display device 100 to the wearable device via the wireless communication interface 173. Thus, the user of the wearable device can utilize the data processed in the display device 100 through the wearable device.

[0062] The display 180 can generate drive signals by converting video signals, data signals, OSD signals processed by the controller 170, or video signals, data signals, etc., received at the external device interface 135 into R, G, and B signals, respectively.

[0063] On the other hand, the display device 100 shown in Figure 1 is merely one embodiment of the present disclosure, so some of the illustrated components may be integrated, added, or omitted depending on the specifications of the display device 100 that is actually implemented.

[0064] In other words, two or more components may be combined into one component as needed, or one component may be subdivided into two or more components. Furthermore, the functions performed in each block are for illustrative purposes only in illustrating embodiments of this disclosure, and their specific operations or devices do not limit the scope of the rights of this disclosure.

[0065] According to other embodiments of the present disclosure, the display device 100 can also receive and play video via a network interface 133 or an external device interface 135 without comprising a tuner 131 and a demodulator 132, unlike the display device 100 shown in Figure 1.

[0066] For example, the display device 100 may be implemented by separating it into a video processing device, such as a set-top box, for receiving broadcast signals or content from various network services, and a content playback device for playing the content input from the video processing device.

[0067] In this case, the operation method of the display device according to the embodiments of the present disclosure described below can be performed not only by the display device 100 as described with reference to Figure 1, but also by any one of the following: an image processing device such as the separated set-top box or a content playback device equipped with a display 180 and a speaker 185.

[0068] Next, with reference to Figures 2 and 3, a remote control device according to one embodiment of the present disclosure will be described.

[0069] Figure 2 is a block diagram of a remote control device according to one embodiment of the present disclosure, and Figure 3 is a diagram showing an example of the configuration of a remote control device according to one embodiment of the present disclosure.

[0070] First, as shown in Figure 2, the remote control device 200 may include a fingerprint recognition device 210, a wireless communication circuit 220, a user input interface 230, a sensor 240, an output interface 250, a power supply circuit 260, a memory 270, a controller 280, and a microphone 290.

[0071] As shown in Figure 2, the wireless communication circuit 220 transmits and receives signals to any one of the display devices according to the embodiments of this disclosure described above.

[0072] The remote control device 200 may include an RF circuit 221 capable of sending and receiving signals with the display device 100 according to the RF communication standard, and an IR circuit 223 capable of sending and receiving signals with the display device 100 according to the IR communication standard. The remote control device 200 may also include a Bluetooth circuit 225 capable of sending and receiving signals with the display device 100 according to the Bluetooth communication standard. Furthermore, the remote control device 200 may include an NFC (Near Field Communication) circuit 227 capable of sending and receiving signals with the display device 100 according to the NFC communication standard, and a WLAN (Wireless LAN) circuit 229 capable of sending and receiving signals with the display device 100 according to the WLAN (Wireless LAN) communication standard.

[0073] Furthermore, the remote control device 200 transmits a signal containing information about the movement of the remote control device 200 to the display device 100 via the wireless communication circuit 220.

[0074] On the other hand, the remote control device 200 can receive signals transmitted by the display device 100 via the RF circuit 221 and, if necessary, can send commands to the display device 100 via the IR circuit 223 regarding power on / off, channel change, volume change, etc.

[0075] The user input interface 230 may consist of a keypad, buttons, a touchpad, or a touchscreen. The user can input commands related to the display device 100 to the remote control device 200 by operating the user input interface 230. If the user input interface 230 includes hard key buttons, the user can input commands related to the display device 100 to the remote control device 200 by pushing the hard key buttons. This will be explained with reference to Figure 3.

[0076] As shown in Figure 3, the remote control device 200 may include multiple buttons. These buttons may include a fingerprint recognition button 212, a power button 231, a home button 232, a live button 233, an external input button 234, a volume control button 235, a voice recognition button 236, a channel change button 237, a confirmation button 238, and a back button 239.

[0077] The fingerprint recognition button 212 may be a button for recognizing the user's fingerprint. In one embodiment, the fingerprint recognition button 212 may be pushable and capable of receiving both push and fingerprint recognition actions.

[0078] The power button 231 may be a button for turning the power of the display device 100 on or off.

[0079] The home button 232 may be a button for moving to the home screen of the display device 100.

[0080] The live button 233 may be a button for displaying a live broadcast program.

[0081] The external input button 234 may be a button for receiving an external input connected to the display device 100.

[0082] The volume control button 235 may be a button for adjusting the volume level output by the display device 100.

[0083] The voice recognition button 236 may be a button for receiving and recognizing the user's voice.

[0084] The channel change button 237 may be a button for receiving the broadcast signal of a specific broadcast channel.

[0085] The confirmation button 238 is a button for selecting a specific function, and the back button 239 may be a button for returning to the previous screen.

[0086] Let's explain Figure 2 again.

[0087] If the user input interface 230 includes a touchscreen, the user can input commands related to the display device 100 to the remote control device 200 by touching the soft keys on the touchscreen. Furthermore, the user input interface 230 may include various types of input means that the user can operate, such as scroll keys and jog keys, and this embodiment does not limit the scope of the rights of this disclosure.

[0088] The sensor 240 may include a gyro sensor 241 or an accelerometer 243, and the gyro sensor 241 can sense information regarding the movement of the remote control device 200.

[0089] For example, the gyro sensor 241 can sense information regarding the operation of the remote control device 200 based on the x, y, and z axes, and the acceleration sensor 243 can sense information regarding the movement speed of the remote control device 200. On the other hand, the remote control device 200 may also be equipped with a distance measuring sensor, which can sense the distance to the display 180 of the display device 100.

[0090] The output interface 250 can output video or audio signals that correspond to the operation of the user input interface 230 or to signals transmitted from the display device 100.

[0091] The user can recognize whether the user input interface 230 is being operated or whether the display device 100 is being controlled by the output interface 250.

[0092] For example, the output interface 250 may include an LED 251 that lights up when the user input interface 230 is operated or when signals are sent and received with the display device 100 via the wireless communication circuit 220, a vibrator 253 that generates vibrations, a speaker 255 that outputs sound, or a display 257 that outputs images.

[0093] Furthermore, the power supply circuit 260 supplies power to the remote control device 200, and if the remote control device 200 does not operate for a predetermined period of time, the power supply is interrupted, thereby reducing power waste.

[0094] The power supply circuit 260 can resume power supply when a predetermined key provided on the remote control device 200 is operated.

[0095] The memory 270 can store various programs, application data, and other information necessary for controlling or operating the remote control device 200.

[0096] When the remote control device 200 transmits and receives signals wirelessly with the display device 100 via the RF circuit 221, the remote control device 200 and the display device 100 transmit and receive signals via a predetermined frequency band.

[0097] The controller 280 of the remote control device 200 can store and refer to information in its memory 270, such as the frequency band that can wirelessly transmit and receive signals with the display device 100 paired with the remote control device 200.

[0098] The controller 280 controls various matters related to the control of the remote control device 200. The controller 280 can transmit signals corresponding to predetermined key operations of the user input interface 230 or signals corresponding to the movement of the remote control device 200 sensed by the sensor 240 to the display device 100 via the wireless communication circuit 220.

[0099] Furthermore, the microphone 290 of the remote control device 200 can acquire sound.

[0100] Multiple microphones 290 may be provided.

[0101] Next, we will explain Figure 4.

[0102] Figure 4 shows an example of utilizing a remote control device according to an embodiment of the present disclosure.

[0103] Figure 4(a) illustrates that a pointer 205 corresponding to the remote control device 200 is displayed on the display 180.

[0104] The user can move or rotate the remote control device 200 up and down, left and right. The pointer 205 displayed on the display 180 of the display device 100 corresponds to the movement of the remote control device 200. Since the pointer 205 moves and is displayed according to the movement in 3D space as shown in the diagram, such a remote control device 200 may be named a spatial remote control.

[0105] Figure 4(b) illustrates that when the user moves the remote control device 200 to the left, the pointer 205 displayed on the display 180 of the display device 100 also moves to the left in response.

[0106] Information regarding the movement of the remote control device 200, as detected by the sensor of the remote control device 200, is transmitted to the display device 100. The display device 100 can calculate the coordinates of the pointer 205 from the information regarding the movement of the remote control device 200. The display device 100 can then display the pointer 205 in accordance with the calculated coordinates.

[0107] Figure 4(c) illustrates a case where a specific button on the remote control device 200 is pressed, and the user moves the remote control device 200 away from the display 180. This allows the selected area on the display 180 corresponding to the pointer 205 to be zoomed in and enlarged.

[0108] Conversely, if the user moves the remote control device 200 closer to the display 180, the selected area on the display 180 corresponding to the pointer 205 can be zoomed out and displayed in a reduced size.

[0109] On the other hand, when the remote control device 200 moves away from the display 180, the selection area is zoomed out, and when the remote control device 200 moves closer to the display 180, the selection area is zoomed in.

[0110] Furthermore, when a specific button in the remote control device 200 is pressed, the recognition of up / down and left / right movement can be excluded. That is, when the remote control device 200 moves away from or closer to the display 180, up / down, left / right movement is not recognized, and only forward / backward movement is recognized. When no specific button in the remote control device 200 is pressed, only the pointer 205 moves in accordance with the up / down, left / right movement of the remote control device 200.

[0111] On the other hand, the movement speed and direction of the pointer 205 can correspond to the movement speed and direction of the remote control device 200.

[0112] On the other hand, in this specification, a pointer refers to an object displayed on the display 180 in response to the operation of the remote control device 200. Therefore, in addition to the arrow shape shown in the drawing as the pointer 205, objects of various shapes are possible. For example, it could be a concept that includes a point, cursor, prompt, or thick outline. Furthermore, the pointer 205 can be displayed corresponding to a single point on either the horizontal or vertical axis of the display 180, or it can be displayed corresponding to multiple points such as a line or surface.

[0113] Figure 5 is a diagram illustrating a webOS architecture according to one embodiment of the present disclosure.

[0114] Referring to Figure 5, the architecture of the webOS platform is as follows:

[0115] Platforms can be broadly categorized into the kernel, the webOS core platform based on system libraries, applications, and services.

[0116] The webOS platform architecture has a layered structure, with the OS at the bottom, system libraries at the next layer, and applications at the top.

[0117] First, the lowest layer includes the Linux kernel as the OS layer, and the display device 100 can include Linux as its OS.

[0118] Above the OS layer are the BSP (Board Support Package) / HAL (Hardware Abstraction Layer) layer, the webOS core modules layer, the service layer, the Luna-Service Bus layer, the Enyo framework / NDK (Native Developer's Kit) / QT layer, and finally, at the very top, the application layer.

[0119] On the other hand, some of the layers in the aforementioned webOS layer structure can be omitted, and multiple layers can be configured as a single layer, or conversely, as a single layer.

[0120] The webOS core module layer can include components such as the Luna Surface Manager (LSM) for managing surface windows, the System & Application Manager (SAM) for managing application execution and status, and the Web Application Manager (WAM) for managing web applications based on WebKit.

[0121] LSM can manage application windows that are visible on the screen. LSM manages the display hardware, provides a buffer that can render the content required by applications, and can combine the rendering results of multiple applications and output them to the screen.

[0122] SAM can manage execution policies for systems and applications based on various conditions.

[0123] On the other hand, since webOS uses web applications as its basic application, WAM can be based on the Enyo Framework.

[0124] Application services are accessed via the Luna-service Bus, allowing new services to be registered on the bus, and applications can also search for and use the services they need.

[0125] The service layer can include services at various service levels, such as TV services and webOS services. Meanwhile, webOS services include media servers and Node.js, and Node.js services, in particular, can support JavaScript (registered trademark).

[0126] webOS services are Linux processes that implement functional logic and can communicate via a bus. This can be broadly divided into four parts: the TV process (which is either migrated from existing TVs to webOS) and services that are differentiated by the manufacturer; webOS common services; and Node.js services, which are developed as JavaScript and used via Node.js.

[0127] The application layer can include all applications that can be supported by the display device 100, such as TV applications, showcase applications, native applications, and web applications.

[0128] Applications running on webOS can be categorized into web applications, PDK (Palm Development Kit) applications, QML (Qt Meta Language or Qt Modeling Language) applications, and others, depending on how they are implemented.

[0129] Web applications can run on the WebKit Runtime, based on the WebKit engine. Such web applications can be developed and run based on the Enyo framework, or on standard HTML5, CSS (Cascading Style Sheets), and JavaScript.

[0130] A PDK application can include native applications developed in C / C++ based on a PDK provided to third-party or external developers. A PDK can refer to a development library and toolset provided to enable third parties, such as game developers, to create native applications (C / C++). For example, a PDK application can be used to develop applications where performance is critical.

[0131] A QML application is a Qt-based native application that can include basic applications provided with the webOS platform, such as a card view, home dashboard, and virtual keyboard. Here, QML can be a script-based markup language instead of C++.

[0132] On the other hand, as mentioned above, native applications are applications developed in C / C++, compiled, and run in binary form, and such native applications have the advantage of being fast.

[0133] Figure 6 is a diagram illustrating the architecture of a webOS device according to one embodiment of the present disclosure.

[0134] Figure 6 is a block diagram based on the runtime of a webOS device, which can be understood by referring to the hierarchical structure in Figure 5. The following explanation will be given with reference to Figures 5 and 6.

[0135] As shown in Figure 6, the system OS (Linux) and system libraries contain services, applications, and the webOS core module, and communication between them can be conducted via the Lunar Service Bus.

[0136] Node.js services based on HTML5, CSS, and JavaScript, such as email, contacts, and calendar; WebOS services such as logging, backup, file notify, database, activity manager, system policy, audio daemon, update, and media server; TV services such as EPG (Electronic Program Guide), PVR (Personal Video Recorder), and data broadcasting; voice recognition, Now On, notification, search, ACR (Auto Content Recognition), CBOX (Contents List Browser), wfdd, DMR, remote application, download, and SDPIF (Sony Philips Digital Interface). CP services such as Format, PDK applications, native applications such as browsers and QML applications, and Enyo framework-based UI-related TV and web applications can be processed via the Lunar Service Bus by WebOS core modules such as SAM, WAM, and LSM mentioned above. On the other hand, in the above, TV and web applications do not necessarily have to be Enyo framework-based or UI-related.

[0137] CBOX can manage lists and metadata for content from external devices such as USB, DLNA®, and the cloud connected to a TV. Furthermore, CBOX can output content listings from various content containers such as USB, DMS, DVR, and the cloud as an integrated view. CBOX can also display various types of content listings, including pictures, music, and videos, and manage their metadata. In addition, CBOX can output content from attached storage in real time. For example, when a storage device such as a USB is plugged in, CBOX can immediately output a content list for that storage device. A standardized method for content listing processing can also be defined. CBOX can also accommodate a variety of connection protocols.

[0138] SAM can be a tool for improving module complexity and scalability. For example, existing system managers handle various functions such as system UI, window management, web application runtime, and UX constraint handling in a single process, resulting in high implementation complexity. This can be resolved by separating key functions and clarifying the interfaces between them, thereby reducing implementation complexity.

[0139] LSM supports the independent development and integration of system UX elements such as card views and launchers, enabling easy adaptation to changes in product requirements. On the other hand, LSM maximizes the use of hardware resources to enable multitasking when compositing multiple application screens, such as in App-on-App, and provides a window management mechanism for multi-window and 21:9 aspect ratios.

[0140] LSM supports the implementation of system UIs based on QML, improving development productivity. QML UX, based on MVC, allows for easy configuration of screen layouts and UI component views, and also facilitates the development of code for handling user input. On the other hand, the interface between QML and webOS components is provided via QML extension plugins, and the application's graphic operations can be based on protocols such as the Wayland protocol and Luna-service calls.

[0141] As mentioned earlier, LSM is an abbreviation for Luna Surface Manager, and it can function as an Application Window Compositor.

[0142] LSM (Landing Screen Mixer) composites and outputs independently developed applications, UI components, etc., onto the screen. In relation to this, when components such as Recents applications, Showcase applications, and Launcher applications render their own content, the LSM, as a compositor, can define the output area, interaction methods, etc. In other words, the LSM, as a compositor, can handle graphic composition, focus management, input events, etc. Here, the LSM receives events, focus, etc., from an input manager, which may include remote controllers, HID devices such as mouse and keyboard, joysticks, gamepads, application remotes, and pen touch.

[0143] Thus, LSM supports a multiple window model, allowing all applications to run simultaneously due to the nature of the system UI. In connection with this, LSM can support launchers, recents, settings, notifications, system keyboard, volume UI, search, finger gestures, voice recognition (STT (Sound to Text), TTS (Text to Sound), NLP (Natural Language Processing), etc.), pattern gestures (camera, MRCU (Mobile Radio Control Unit)), live menus, ACR (Auto Content Recognition), and more.

[0144] Figure 7 is a diagram illustrating the graphic composition flow in a webOS device according to one embodiment of the present disclosure.

[0145] As shown in Figure 7, graphic composition processing can be performed via the web application manager 910, which is responsible for the UI process; the web kit 920, LSM 930, and the graphics manager (GM) 940, which are responsible for the web process.

[0146] When web application-based graphic data (or application) is generated as a UI process in the web application manager 910, the generated graphic data can be transmitted to the LSM 930 if it is not a full-screen application. On the other hand, the web application manager 910 can receive the application generated by the webkit 920 for GPU (Graphics Processing Unit) memory sharing for graphics management between the UI process and the web process, and transmit it to the LSM 930 if it is not a full-screen application as described above. If it is a full-screen application as described above, the LSM 930 can be bypassed, and in this case, it can be transmitted directly to the graphics manager 940.

[0147] The LSM930 can receive UI applications and transmit them to the Wayland Compositor via the Wayland surface, where the Wayland Compositor can process them appropriately and transmit them to the graphics manager. The graphic data transmitted from the LSM930 in this way can then be transmitted, for example, to the graphics manager's compositor via the LSM GM surface of the graphics manager 940.

[0148] On the other hand, full-screen applications, as mentioned above, are transmitted directly to the graphics manager 940 without going through the LSM930, but such applications can be processed in the graphics manager compositor via the WAM GM surface.

[0149] The graphics manager processes all graphic data within the webOS device, including not only data transmitted via the LSM GM surface and WAM GM surface, but also all graphic data transmitted via the GM surface, such as data broadcasting applications and caption applications, and processes it to ensure it is properly displayed on the screen. Here, the function of the GM compositor may be the same as or similar to that of the compositor described above.

[0150] Figure 8 shows an example of a system configuration according to one embodiment of the present disclosure.

[0151] As shown in Figure 8, the system may include an electronic device 100, a server 400, and / or a user terminal 500. The system may further include a shared machine 600.

[0152] The electronic device 100 may include a display device 100a, an air conditioner 100b, a refrigerator 100c, an air purifier 100d, a washing machine 100e, a vehicle 100f, and the like. In this disclosure, the electronic device 100 is described as a display device 100a as an example, but the present invention is not limited thereto.

[0153] The electronic device 100 can send and receive signals with the server 400, user terminal 500, and / or shared machine 600. For example, the electronic device 100 can send and receive data with at least one server 400 via a network 300 such as the Internet.

[0154] An application used for remote control of the electronic device 100 (hereinafter referred to as the remote control application) can be installed on the user terminal 500. The remote control application can also be installed on the electronic device 100. The electronic device 100 and the user terminal 500 can send and receive signals related to remote control using the remote control application.

[0155] The electronic device 100 can transmit signals related to remote control to the server 400 via the shared device 600. The electronic device 100 can receive signals related to remote control from the server 400 via the shared device 600.

[0156] The shared unit 600 can connect to a communication line installed in a location such as a home or office and transmit signals. The electronic device 100 is located within range of receiving signals from the shared unit 600.

[0157] The shared unit 600 can receive signals sent from the electronic device 100 and user terminal 500 that are connected to the shared unit 600 and transmit them externally via the communication line. The shared unit 600 can convert signals input from the outside via the communication line into wireless signals and transmit them. This allows the electronic device 100 connected to the shared unit 600 to communicate with the server 400.

[0158] According to one embodiment, when the electronic device 100 is remotely controlled via a user terminal 500, signals transmitted from the user terminal 500 can be sent to the electronic device 100 via a server 400 and / or a shared device 600. The electronic device 100 can transmit signals related to remote control to the user terminal 500 via the shared device 600 and / or the server 400. On the other hand, if the system provides the electronic device 100 with a communication protocol (e.g., Bluetooth, Wi-Fi, etc.), one-to-one communication between the electronic device 100 and the user terminal 500 is also possible.

[0159] The electronic device 100 can operate in remote control mode in response to remote control being performed using the user terminal 500. Conversely, the electronic device 100 can exit remote control mode in response to the termination of remote control using the user terminal 500.

[0160] The electronic device 100 can receive control signals from the user terminal 500 in remote control mode. On the other hand, the electronic device 100 can also receive control signals from the remote control device 200 in remote control mode.

[0161] According to one embodiment, multiple user terminals 500 can be used simultaneously for remote control of the electronic device 100. For example, with multiple user terminals 500 connected to a local area network via a shared device 600, remote control of the electronic device 100 can be performed by multiple user terminals 500 by executing a remote control application.

[0162] Figures 9 to 12 are flowcharts illustrating the operation method of a display device according to one embodiment of the present disclosure.

[0163] As shown in Figure 9, the display device 100 can receive user input in S901 operation. For example, the display device 100 can receive user input from the remote control device 200 via the user input interface 150. For example, the display device 100 can receive user input from the user terminal 500 via the network interface 133.

[0164] The display device 100 can determine, in S902 operation, whether the received user input corresponds to the use of a virtual keyboard. The virtual keyboard may be a software component that provides character input without the request of a physical key. The virtual keyboard may include multiple keys, each representing a character.

[0165] As shown in Figures 13a and 13b, the display device 100 can output a screen related to content via the display 180.

[0166] The user terminal 500 can output a screen for remote control via the display 510 in response to the execution of a remote control application that remotely controls the display device 100.

[0167] The screen for remote control output via the display 510 of the user terminal 500 may include various objects 1310, 1320, 1331, and 1333 corresponding to keypads, buttons, touchpads, etc., included in the remote control device 200. The user terminal 500 can receive user input to select an object included in the screen for remote control and transmit a control signal corresponding to the selected object to the display device 100.

[0168] The display device 100 can operate in response to control signals received from the user terminal 500.

[0169] For example, when the user terminal 500 receives input to select a channel button included in the first object 1310, it can send a control signal corresponding to the channel change button 237 to the display device 100. At this time, the display device 100 can change the broadcast channel on which it receives the broadcast signal according to the control signal received from the user terminal 500.

[0170] For example, when the user terminal 500 receives input to operate the touchpad included in the second object 1331, it can send a control signal to the display device 100 that corresponds to the movement of the remote control device 200. At this time, the display device 100 can change the position of the pointer 205 displayed via the display 180 in accordance with the control signal received from the user terminal 500.

[0171] The user can use the remote control device 200 and / or the user terminal 500 to operate the pointer 205 and select an object 1400 corresponding to text input output via the display 180 of the display device 100. At this time, the display device 100 can determine that user input corresponding to the use of a virtual keyboard has been received based on the input that selects the object 1400 corresponding to text input.

[0172] Referring again to Figure 9, the display device 100 can determine in operation S903 whether or not remote control using the user terminal 500 is performed. For example, when a remote control application is executed on the portable terminal 500 and a control signal is sent to the display device 100, the display device 100 can operate in remote control mode. At this time, the display device 100 can determine that remote control using the user terminal 500 is performed while it is operating in remote control mode.

[0173] Based on the S904 operation, the display device 100 can send a command to the user terminal 500 instructing it to use the virtual keyboard, based on the fact that remote control is being performed using the user terminal 500. For example, referring to Figure 14, the user terminal 500, upon receiving a command instructing it to use the virtual keyboard, can output an object 1340 corresponding to the virtual keyboard via the display 510. This will be explained with reference to Figure 10.

[0174] As shown in Figure 10, in operation S1001, the display device 100 can determine whether the user terminal 500 of the user account currently logged into the server 400 is to be used for remote control of the display device 100. For example, a user can log in to the server 400 with their user account by entering their user account ID and password into the display device 100. At this time, the display device 100 can compare the information of the user terminal 500 corresponding to the user account logged into the server 400 with the information of the user terminal 500 used for remote control of the display device 100 to determine whether the user terminal 500 corresponding to the user account logged into the server 400 is to be used for remote control.

[0175] In operation S1002, the display device 100 can send a command to the user terminal 500 of the user account currently logged into the server 400 to instruct it to use a virtual keyboard if the user terminal 500 of the user account currently logged into the server 400 is to be used for remote control.

[0176] In operation S1003, the display device 100 can determine whether multiple user terminals 500 are used for remote control of the display device 100 if the user terminal 500 of the user account currently logged into the server 400 is not used for remote control.

[0177] In operation S1004, if one user terminal 500 is used for remote control of the display device 100, the display device 100 can send a command to the user terminal 500 used for remote control instructing it to use a virtual keyboard.

[0178] In operation S1005, if multiple user terminals 500 are used for remote control of the display device 100, the display device 100 can send notifications regarding the use of a virtual keyboard to the multiple user terminals 500 used for remote control.

[0179] In operation S1006, the display device 100 can determine whether or not a response to a notification regarding the use of a virtual keyboard has been received from at least one of the multiple user terminals 500 used for remote control.

[0180] In operation S1007, the display device 100 can, upon receiving a response to a notification regarding the use of a virtual keyboard, send a command to the user terminal 500 that sent the response instructing it to use the virtual keyboard. At this time, if the display device 100 receives responses from two or more of the multiple user terminals 500, it can send a command to the user terminal 500 corresponding to the first response received instructing it to use the virtual keyboard.

[0181] As shown in Figure 15, when the display device 100 sends a notification regarding the use of the virtual keyboard to multiple user terminals 500a and 500b used for remote control, the multiple user terminals 500a and 500b can, based on receiving the notification regarding the virtual keyboard, output screens 1500a and 1500b via displays 510a and 510b, respectively, which contain a message asking whether or not to use the virtual keyboard.

[0182] At this time, since multiple user terminals 500a and 500b are remotely controlling the display device 100 by running a remote control application, screens 1500a and 1500b containing a message asking whether or not to use a virtual keyboard can be displayed as pop-up screens on the remote control screen.

[0183] Referring again to Figure 9, the display device 100 can activate a virtual interface associated with an input device in operation S905. Here, the virtual interface may mean an interface implemented as a virtual device, which is configured and generated in software using the resources of the display device 100. In this disclosure, the virtual interface will be described in reference to a virtual HID (Human Interface Device) associated with a keyboard. In this disclosure, the virtual interface may be named a virtual input interface.

[0184] According to one embodiment, a virtual interface can be created and installed in the BSP (Board Support Package) / HAL (Hardware Abstraction Layer) layer of the webOS of the display device 100. When the virtual interface installed in the webOS BSP layer is activated, the display device 100 can operate in the same way as when a keyboard is connected via the external device interface 135 or the wireless communication interface 173. For example, in response to the activation of the virtual interface, the display device 100 can block the output of the virtual keyboard via the display 180. For example, in response to the deactivation of the virtual interface, the display device 100 can output the virtual keyboard via the display 180.

[0185] According to one embodiment, the display device 100 can activate a virtual interface in response to the transmission of a command to the user terminal 500 instructing it to use a virtual keyboard.

[0186] According to one embodiment, the display device 100 can activate a virtual interface in response to receiving a response from the user terminal 500 to a command instructing the user terminal 500 to use a virtual keyboard. For example, in response to receiving a command instructing the user terminal 500 to use a virtual keyboard, the user terminal 500 can output an object 1340 corresponding to the virtual keyboard via the display 510. At this time, in response to the output of the object 1340 corresponding to the virtual keyboard via the display 510, the user terminal 500 can send a response to the command instructing the user terminal to use a virtual keyboard to the display device 100.

[0187] The display device 100 can determine in operation S906 whether or not user input related to the termination of the virtual keyboard has been received. Here, user input related to the termination of the virtual keyboard may mean user input related to the termination of text input. For example, user input related to the termination of the virtual keyboard may include input corresponding to the completion of text input, input corresponding to power on / off control, input corresponding to a change in broadcast channel, input corresponding to the execution of a predetermined application, input corresponding to a speech recognition function, and so on.

[0188] The display device 100 can determine, in operation S907, whether or not a key code has been received from the user terminal 500. Here, the key code may be a value corresponding to each of several keys included in the keyboard.

[0189] The display device 100 can confirm the input corresponding to the key code received from the user terminal 500 in S908 operation. For example, the display device 100 can confirm the characters, numbers, symbols, functions, etc., corresponding to the key code received from the user terminal 500.

[0190] According to one embodiment, the display device 100 can receive information about the language type along with a key code from the user terminal 500. At this time, the display device 100 can confirm the input corresponding to the key code based on the information about the language type received from the user terminal 500. For example, if the language type is Korean, the characters corresponding to a predetermined key code are If the image is JPEG2026076987000002.jpg611 and the language is English, the character corresponding to the given key code could be "a".

[0191] The display device 100 can perform operations related to the virtual keyboard based on input corresponding to key codes in S909 operation. For example, the display device 100 can determine a string corresponding to a key code received from the user terminal 500. At this time, the display device 100 can output text corresponding to the determined string via the display 180.

[0192] As shown in Figure 16, when a user selects one of the keys included in the object 1340 corresponding to the virtual keyboard output via the display 510 of the user terminal 500, the user terminal 500 can send the key code corresponding to the key selected by the user to the display device 100. The display device 100 can then display a predetermined character in the object 1400 corresponding to text input, in response to the input corresponding to the key code received from the user terminal 500 being a predetermined character.

[0193] In other words, whenever the user terminal 500 receives user input to select a key from the object 1340 corresponding to the virtual keyboard, it can send a key code corresponding to the key selected by the user to the display device 100. The display device 100 can also output text via the display 180 based on the string corresponding to the key code received from the user terminal 500.

[0194] According to one embodiment, when a key code is received from a user terminal 500, the key code can be transmitted to the LSM via the Luna-service Bus. The LSM can transmit the key code to a framework that processes key codes (e.g., Maliit). The framework that processes key codes can determine a string corresponding to the key code received from the LSM and transmit it to the LSM. The LSM can transmit the string determined by the framework that processes key codes to an application corresponding to the screen output via the display 180. Through this process, the display device 100 can determine a string corresponding to the key code and output text corresponding to the string via the display 180.

[0195] Referring again to Figure 9, the display device 100 can deactivate the virtual interface in operation S910 when it receives user input associated with the termination of the virtual keyboard. At this time, the display device 100 can send a command to the user terminal 500 instructing it to stop using the virtual keyboard. In response to receiving the command instructing it to stop using the virtual keyboard, the user terminal 500 can stop outputting the object corresponding to the virtual keyboard.

[0196] The display device 100 can perform operations in S911 operation in response to user input. For example, the display device 100 can turn off its power based on the reception of an input corresponding to the control of turning the power on / off. For example, the display device 100 can execute a predetermined application based on the reception of an input corresponding to the execution of a predetermined application and output a screen corresponding to the predetermined application via the display 180.

[0197] On the other hand, the display device 100 can determine that user input related to the termination of the virtual keyboard has been received if the key code received from the user terminal 500 corresponds to the enter key.

[0198] The display device 100 can output a virtual keyboard via the display 180, based on the fact that remote control using the user terminal 500 is not performed. This will be explained with reference to Figure 11.

[0199] As shown in Figure 11, in operation S1101, the display device 100 can determine whether at least one user terminal 500 is connected to the local network via the shared device 600 if remote control using the user terminal 500 is not performed.

[0200] In operation S1102, the display device 100 can determine whether or not a user terminal 500 connected to the local network is the user terminal 500 of a user account currently logged into the server 400, provided that at least one user terminal 500 is connected to the local network.

[0201] In operation S1103, the display device 100 can send a notification regarding the use of a virtual keyboard to the user terminal 500 of the user account currently logged into the server 400 when the user terminal 500 of the user account currently logged into the server 400 is connected to the local network.

[0202] On the other hand, in operation S1104, the display device 100 can send a notification regarding the use of a virtual keyboard to a user terminal 500 connected to the local network if the user terminal 500 of the user account currently logged into the server 400 is not connected to the local network.

[0203] As shown in Figure 17, when the display device 100 sends a notification regarding the use of the virtual keyboard to a user terminal 500 connected to the local network, the user terminal 500 can output a screen 1700 via the display 510 that contains a message asking whether or not to use the virtual keyboard, based on the receipt of the notification regarding the virtual keyboard.

[0204] At this time, since the user terminal 500 is not being used for remote control of the display device 100, a screen 1700 containing a message asking whether or not to use a virtual keyboard can be displayed as a pop-up screen on the screen currently being output via the display 510.

[0205] Referring again to Figure 11, the display device 100 can output a virtual keyboard via the display 180 in operation S1105.

[0206] For example, as shown in Figure 18, the display device 100 can output a virtual keyboard 1800 through a portion of the screen output via the display 180. In this case, the content displayed on the screen output via the display 180 may be obscured by the virtual keyboard 1800. On the other hand, the user can use the remote control device 200 to select a predetermined key 1810 corresponding to a predetermined character included in the virtual keyboard 1800 displayed on the display 180. In this case, the display device 100 can display the predetermined character on the object 1400 corresponding to text input in response to the selection of the predetermined key 1810 corresponding to the predetermined character included in the virtual keyboard 1800.

[0207] In operation S1106, the display device 100 can determine whether or not remote control using the user terminal 500 is performed. For example, the user terminal 500 can send a signal related to remote control to the display device 100 in response to the execution of a remote control application. At this time, the display device 100 can determine that remote control using the user terminal 500 is performed based on the reception of the remote control signal received from the user terminal 500.

[0208] Based on the operation in S1107, the display device 100 can send a command to the user terminal 500 instructing it to use a virtual keyboard, since remote control is being performed using the user terminal 500.

[0209] For example, as shown in Figure 19, a user terminal 500 that receives a command to use a virtual keyboard can output an object 1340 corresponding to the virtual keyboard via the display 510. At this time, the display of predetermined characters for the object 1400 corresponding to text input, which is included in the screen output via the display 180, can be maintained.

[0210] On the other hand, the display device 100 can determine whether or not user input has been received in operation S1108. For example, the display device 100 can receive user input from the remote control device 200 even when remote control using the user terminal 500 is not being performed.

[0211] In operation S1109, the display device 100 can determine whether or not user input is related to the termination of the virtual keyboard. For example, if the display device 100 receives user input that uses the pointer 205 to select an area of ​​the screen output via the display 180 that is not the virtual keyboard 1800, it can determine that user input related to the termination of the virtual keyboard has been received.

[0212] In operation S1110, the display device 100 can perform operations related to the virtual keyboard if the received user input is not user input related to the termination of the virtual keyboard.

[0213] On the other hand, in operation S1111, the display device 100 can perform an action based on user input if the received user input is related to the termination of the virtual keyboard. At this time, the display device 100 can stop the output of the virtual keyboard via the display 180.

[0214] On the other hand, if the virtual interface is activated and no user input related to key codes and the termination of the virtual keyboard is received, the display device 100 can decide whether or not to output the virtual keyboard via the display 180. This will be explained with reference to Figure 12.

[0215] As shown in Figure 12, the display device 100 can determine in operation S1201 whether or not the virtual keyboard is not being used on the user terminal 500. For example, the display device 100 can determine whether or not the virtual keyboard is not being used on the user terminal 500 based on the reception of a signal from the user terminal 500 indicating that the virtual keyboard is not being used. For example, the display device 100 can determine whether or not the virtual keyboard is not being used on the user terminal 500 based on a signal received from the server 400 regarding the status of the user terminal 500.

[0216] According to one embodiment, the user terminal 500 can send a signal to the display device 100 corresponding to the unused virtual keyboard. For example, the user terminal 500 can send a signal to the display device 100 corresponding to the unused virtual keyboard when the output of objects corresponding to the virtual keyboard via the display 510 is stopped. For example, the user terminal 500 can send a signal to the display device 100 corresponding to the unused virtual keyboard when the execution of the remote control application is terminated. For example, the user terminal 500 can send a signal to the display device 100 corresponding to the unused virtual keyboard when the output of the screen for remote control via the display 510 is stopped.

[0217] The display device 100 can determine, in operation S1202, whether or not a ping signal is received from the user terminal 500. Here, the ping signal may mean a signal indicating that the user terminal 500 is executing a remote control application and performing remote control over the display device 100. According to one embodiment, the user terminal 500 can transmit a ping signal to the display device 100 at a predetermined interval while an object corresponding to a virtual keyboard is output via the display 510. At this time, the display device 100 can monitor whether or not a ping signal is received from the user terminal 500 while the virtual interface is activated.

[0218] In operation S1203, the display device 100 can deactivate the virtual interface when the virtual keyboard is not being used on the user terminal 500 and / or when no pin signals are received.

[0219] The display device 100 can output a virtual keyboard via the display 180 in operation S1204.

[0220] The display device 100 can determine in operation S1205 whether or not the virtual keyboard is being used on the user terminal 500. For example, the display device 100 can determine that the virtual keyboard is not being used on the user terminal 500 based on the reception of a signal from the user terminal 500 corresponding to the use of the virtual keyboard.

[0221] According to one embodiment, the user terminal 500 can send a signal to the display device 100 corresponding to the use of a virtual keyboard. For example, if the user terminal 500 outputs an object corresponding to a virtual keyboard via the display 510, it can send a signal to the display device 100 corresponding to the use of a virtual keyboard. For example, if a remote control application is being executed, the user terminal 500 can send a signal to the display device 100 corresponding to the use of a virtual keyboard. For example, if the user terminal 500 outputs a screen for remote control via the display 510, it can send a signal to the display device 100 corresponding to the use of a virtual keyboard.

[0222] In operation S1206, the display device 100 can send a command to the user terminal 500 instructing it to use the virtual keyboard, based on the fact that the virtual keyboard is in use on the user terminal 500. On the other hand, the display device 100 can activate the virtual input interface in response to receiving a signal from the user terminal 500 corresponding to the use of the virtual keyboard.

[0223] On the other hand, the display device 100 can determine whether or not user input has been received in operation S1207.

[0224] In operation S1208, the display device 100 can determine whether or not user input is related to the termination of the virtual keyboard.

[0225] In operation S1209, the display device 100 can perform operations related to the virtual keyboard if the received user input is not user input related to the termination of the virtual keyboard.

[0226] On the other hand, in operation S1210, the display device 100 can perform an action based on user input if the received user input is related to the termination of the virtual keyboard. At this time, the display device 100 can stop the output of the virtual keyboard via the display 180.

[0227] As shown in Figure 20, when the home screen 2000 is displayed via the user terminal 500's display 510 while an object 1340 corresponding to the virtual keyboard is being output via the user terminal 500's display 510, the user terminal 500 can send a signal to the display device 100 corresponding to the unused virtual keyboard. At this time, based on the receipt of the signal corresponding to the unused virtual keyboard, the display device 100 can deactivate the virtual interface and output the virtual keyboard 1800 via the display 180.

[0228] As shown in Figure 21, when an object 1340 corresponding to the virtual keyboard is output via the display 510 of the user terminal 500, and the call waiting screen 2100 is output via the display 510 of the user terminal 500, the user terminal 500 can send a signal to the display device 100 corresponding to the unused virtual keyboard. At this time, based on the receipt of the signal corresponding to the unused virtual keyboard, the display device 100 can deactivate the virtual interface and output the virtual keyboard 1800 via the display 180.

[0229] As described above, according to at least one embodiment of this disclosure, a function can be provided that allows text to be entered into a display device using a user terminal.

[0230] Furthermore, according to at least one embodiment of this disclosure, the text displayed on the display device can be easily modified based on input received from a user terminal.

[0231] Furthermore, according to at least one embodiment of this disclosure, the amount of data used for text input can be minimized.

[0232] Furthermore, according to at least one embodiment of this disclosure, when text is entered using a user terminal, it is possible to prevent the content displayed on the screen of the display device from being obscured by the virtual keyboard.

[0233] Furthermore, according to at least one embodiment of this disclosure, in response to the cessation of text input using a user terminal, the display device can display a screen including a virtual keyboard to allow text input to continue.

[0234] Furthermore, according to at least one embodiment of this disclosure, it is possible to guide the user to use the optimal user terminal for text input.

[0235] Referring to Figures 1 to 21, a display device 100 according to one aspect of the present disclosure includes a display 180, a network interface 133, and a controller 170, wherein the controller 170 can, in response to the activation of a virtual input interface, block the output of a virtual keyboard via the display 180, receive a key code from a predetermined user terminal 500 via the network interface 133 while the virtual input interface is activated, and display text via the display 180 based on a string corresponding to the received key code.

[0236] Furthermore, according to one aspect of this disclosure, the controller 170 can activate the virtual input interface in response to receiving a first signal from the predetermined user terminal 500 corresponding to the use of the virtual keyboard, and can deactivate the virtual input interface in response to receiving a second signal from the predetermined user terminal 500 corresponding to the non-use of the virtual keyboard.

[0237] Furthermore, according to one aspect of this disclosure, when the virtual input interface is deactivated and the use of the virtual keyboard is requested, the controller 170 determines whether there is at least one first user terminal that controls the display device 100, and based on the existence of at least one first user terminal, it sends a command to one of the first user terminals that corresponds to a predetermined user terminal 500 to instruct the use of the virtual keyboard, or based on the absence of a first user terminal, it outputs the virtual keyboard via the display 180.

[0238] Furthermore, according to one aspect of this disclosure, the controller 170 can activate the virtual input interface in response to the transmission of a command instructing the use of the virtual keyboard.

[0239] Furthermore, according to one aspect of this disclosure, the controller 170 can activate the virtual input interface in response to receiving a response to a command instructing the use of the virtual keyboard.

[0240] Furthermore, according to one aspect of this disclosure, if there is at least one first user terminal, the controller 170 can determine whether there is a second user terminal among the at least one first user terminal that corresponds to an account logged into the server via the network interface 133, and if there is a second user terminal, it can determine the second user terminal to be the predetermined user terminal 500, and if there is no second user terminal, it can determine one of the at least one first user terminal to be the predetermined user terminal 500.

[0241] Furthermore, according to one aspect of this disclosure, if the second user terminal does not exist, the controller 170 can determine the first user terminal as the predetermined user terminal 500 if the first user terminal is a single individual, and if there are two or more first user terminals, it can send a notification regarding the use of the virtual keyboard to the plurality of first user terminals, and determine the one of the plurality of first user terminals that sent a response to the notification as the predetermined user terminal 500.

[0242] Furthermore, according to one aspect of this disclosure, the controller 170 can log in to the server with an account corresponding to the first user terminal which has been determined to be the predetermined user terminal 500, when the second user terminal does not exist.

[0243] Furthermore, according to one aspect of this disclosure, the controller 170 can maintain the activated state of the virtual input interface in response to receiving a third signal indicating the communication status from the predetermined user terminal 500 while the virtual input interface is activated, and can deactivate the virtual input interface in response to not receiving the third signal from the predetermined user terminal 500.

[0244] Furthermore, according to one aspect of this disclosure, the controller 170 further includes a user input interface 150, and based on the reception of input unrelated to the virtual keyboard via at least one of the network interface 133 and the user input interface 150, the controller 170 can, while the virtual input interface is activated, send a command to the predetermined user terminal 500 to deactivate the virtual input interface and instruct the virtual keyboard to stop using the virtual keyboard, and while the virtual input interface is deactivated and the virtual keyboard is outputting via the display 180, the controller 170 can stop the output of the virtual keyboard.

[0245] Furthermore, according to one aspect of this disclosure, the controller 170 may determine, based on the absence of the first user terminal, whether there is at least one third user terminal connected to the local area network, and if there is at least one third user terminal, it may send a notification to the third user terminal regarding the use of the virtual keyboard.

[0246] Furthermore, according to one aspect of this disclosure, if at least one third user terminal exists, the controller 170 can determine whether there is a fourth user terminal 500 among the at least one third user terminal that corresponds to an account logged into the server via the network interface 133, and if the fourth user terminal 500 exists, it can send a notification regarding the use of the virtual keyboard to the fourth user terminal 500, and if the fourth user terminal 500 does not exist, it can send a notification regarding the use of the virtual keyboard to the at least one third user terminal.

[0247] Furthermore, according to one aspect of this disclosure, the received key code is transmitted to the Luna Surface Manager (LSM) via the Luna-service Bus, the LSM transmits the received key code to a predetermined framework for processing key codes, receives a string corresponding to the received key code from the predetermined framework, and transmits the string corresponding to the received key code received from the predetermined framework to an application corresponding to a screen output via the display.

[0248] A system according to one aspect of the present disclosure includes a display device 100 and at least one user terminal 500, wherein the display device 100, in response to the activation of a virtual input interface, blocks the output of a virtual keyboard via a first display 180 of the display device 100, receives a key code from a predetermined user terminal 500 while the virtual input interface is activated, displays text via the first display 180 based on a string corresponding to the received key code, and the predetermined user terminal 500 can output the virtual keyboard via a second display device 100 of the predetermined user terminal 500 and transmit the key code corresponding to the keys included in the virtual keyboard to the display device 100.

[0249] Furthermore, according to one aspect of this disclosure, the predetermined user terminal 500 transmits a first signal to the display device 100 corresponding to the use of the virtual keyboard in response to outputting the virtual keyboard via the second display 180, and transmits a second signal to the display device 100 corresponding to the non-use of the virtual keyboard in response to the cessation of outputting the virtual keyboard via the second display 180. The display device 100 can activate the virtual input interface in response to receiving the first signal from the predetermined user terminal 500, and deactivate the virtual input interface in response to receiving the second signal from the predetermined user terminal 500.

[0250] Furthermore, according to one aspect of this disclosure, when the virtual input interface is deactivated and the use of the virtual keyboard is requested, the display device 100 determines whether there is at least one first user terminal that controls the display device 100, and based on the existence of at least one first user terminal, it sends a command to one of the first user terminals that corresponds to a predetermined user terminal 500 to instruct the use of the virtual keyboard, and based on the absence of a first user terminal, it outputs the virtual keyboard via the display 180, and the predetermined user terminal 500 can output the virtual keyboard via the second display 180 in response to receiving a command to instruct the use of the virtual keyboard.

[0251] Furthermore, according to one aspect of this disclosure, the display device 100 can activate the virtual input interface in response to the transmission of a command instructing the use of the virtual keyboard.

[0252] Furthermore, according to one aspect of this disclosure, the predetermined user terminal 500 can, in response to receiving a command instructing the use of the virtual keyboard, output the virtual keyboard via the second display 180, and in response to the output of the virtual keyboard via the virtual second display 180, transmit a response to the command instructing the use of the virtual keyboard to the display device 100, and the display device 100 can activate the virtual input interface in response to receiving the response to the command instructing the use of the virtual keyboard.

[0253] Furthermore, according to one aspect of this disclosure, if there is at least one first user terminal, the display device 100 can determine whether there is a second user terminal among the at least one first user terminal that corresponds to an account logged into the server via the display device 100, and if there is a second user terminal, it can determine that the second user terminal is the predetermined user terminal 500, and if there is no second user terminal, it can determine that one of the at least one first user terminals is the predetermined user terminal 500.

[0254] Furthermore, according to one aspect of this disclosure, if the second user terminal does not exist, the display device 100 can determine the first user terminal as a predetermined user terminal 500 if the first user terminal is a single individual, and if there are two or more first user terminals, it can send a notification regarding the use of the virtual keyboard to the plurality of first user terminals, and determine the one of the plurality of first user terminals that sends a response to the notification as the predetermined user terminal 500.

[0255] Furthermore, according to one aspect of this disclosure, the predetermined user terminal 500 transmits a third signal indicating the communication status to the display device 100 via the second display 180 while the virtual keyboard is outputting, and the display device 100 maintains the activated state of the virtual input interface in response to receiving the third signal from the predetermined user terminal 500 while the virtual input interface is activated, and can deactivate the virtual input interface in response to not receiving the third signal from the predetermined user terminal 500.

[0256] The accompanying drawings are provided solely to facilitate understanding of the embodiments disclosed herein, and are not intended to limit the technical ideas disclosed herein, and should be understood to include all modifications, equivalents, or substitutions that fall within the concept and scope of this disclosure.

[0257] On the other hand, the operation method of this disclosure can be implemented as processor-readable code on a processor-readable recording medium. The processor-readable recording medium can include any type of recording device on which processor-readable data is stored. Examples of processor-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage devices, and may also be implemented in the form of carrier waves, such as transmission over the Internet. Furthermore, the processor-readable recording media can be distributed across networked computer systems, allowing the processor-readable code to be stored and executed in a distributed manner.

[0258] Furthermore, although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications can be carried out by persons with ordinary skill in the art to which the invention pertains, without departing from the gist of the present disclosure as claimed in the claims. Moreover, such modifications should not be understood individually from the technical idea or prospects of the present disclosure.

Claims

1. In a display device, display, Network interface, and Including the controller, The aforementioned controller, To input information, an input window is displayed on the aforementioned display. A virtual keyboard is displayed on the display in order to input information into the input window. An input signal indicating the activation of the virtual keypad on the first user terminal is received from the first user terminal. In response to sensing that the first user terminal is being used to control the display device, the display of the virtual keyboard on the display is deactivated. The key code to be converted to the key selected on the virtual keypad of the first user terminal is received from the first user terminal via the network interface, The text corresponding to the received key code is displayed in the input window shown on the display. Display device.

2. The aforementioned controller, In response to sensing that the virtual keypad of the first user terminal is being used to control the display device, the display of the virtual keyboard on the display is deactivated. The display device according to claim 1.

3. The aforementioned controller, In response to receiving the input signal from the first user terminal indicating the activation of the virtual keypad on the first user terminal, the display device activates a virtual input interface that is linked to the virtual keypad on the first user terminal. The display device according to claim 1.

4. The aforementioned controller, The first user terminal determines whether the virtual keypad is idle or not. In response to the determination that the virtual keypad is idle in the first user terminal, the virtual input interface is deactivated in the display device. The display device according to claim 3.

5. The aforementioned controller, Determine whether the first user terminal controls the display device. In response to the determination that the virtual input interface is not activated, the virtual input interface is activated. The display device according to claim 3.

6. The aforementioned controller, Determine whether a wireless connection is established with the second user terminal. The system determines whether the first user terminal and the second user terminal are associated with an account logged into a server associated with the display device via the network interface. If it is determined that the first user terminal is logged into the account and the second user terminal is not logged into the account, the first user terminal is allowed to control the display device. The display device according to claim 1.

7. The aforementioned controller, Determine whether the first user terminal and the second user terminal have the authority to control the display device. Determine whether the first user terminal is wirelessly connected to the display device. Switch the user account that controls the display device to the second user terminal. The display device according to claim 6.

8. The aforementioned server, The received key code is sent to an already configured framework that processes key codes. The previously configured framework receives a string corresponding to the received key code, The string received from the already configured framework is sent to the application corresponding to the screen output via the display. The display device according to claim 6.

9. In a non-temporary computer-readable storage medium for storing instruction words, when the instruction words are executed by a processor, the processor: An input window is displayed on the screen to enter information. A virtual keyboard is displayed on the display in order to input information into the input window. An input signal indicating the activation of the virtual keypad on the first user terminal is received from the first user terminal. In response to sensing that the first user terminal is being used to control the display device, the display of the virtual keyboard on the display is deactivated. The key code to be converted to the key selected on the virtual keypad of the first user terminal is received from the first user terminal via the network interface. The text corresponding to the received key code is displayed in the input window shown on the display. Non-temporary computer-readable storage medium.

10. When the aforementioned instruction is executed by the processor, the processor will In response to sensing that the virtual keypad of the first user terminal is being used to control the display device, the display of the virtual keyboard on the display is deactivated. The non-temporary computer-readable storage medium according to claim 9.

11. When the aforementioned instruction is executed by the processor, the processor will In response to the reception of the input signal from the first user terminal indicating the activation of the virtual keypad on the first user terminal, the display device is configured to activate a virtual input interface that is linked to the virtual keypad on the first user terminal. The non-temporary computer-readable storage medium according to claim 9.

12. When the aforementioned instruction is executed by the processor, the processor will The first user terminal determines whether the virtual keypad is idle or not. In response to the determination that the virtual keypad is idle in the first user terminal, the virtual input interface is deactivated in the display device. The non-temporary computer-readable storage medium according to claim 11.

13. When the aforementioned instruction is executed by the processor, the processor will Determine whether the first user terminal controls the display device. In response to the determination that the virtual input interface is not activated, the system activates the virtual input interface. The non-temporary computer-readable storage medium according to claim 11.

14. When the aforementioned instruction is executed by the processor, the processor will Determine whether a wireless connection is established with the second user terminal. The system determines whether the first user terminal and the second user terminal are associated with an account logged into a server associated with the display device via the network interface. If it is determined that the first user terminal is logged into the account and the second user terminal is not logged into the account, the system will allow the first user terminal to control the display device. The non-temporary computer-readable storage medium according to claim 9.

15. When the aforementioned instruction is executed by the processor, the processor will Determine whether the first user terminal and the second user terminal have the authority to control the display device. Determine whether the first user terminal is wirelessly connected to the display device. The user account controlling the display device is switched to the second user terminal. A non-temporary computer-readable storage medium according to claim 14.

16. When the aforementioned instruction is executed by the processor, the processor will The server controls the transmission of the received key code to a pre-configured framework that processes key codes, the transmission of a string corresponding to the received key code from the pre-configured framework, and the transmission of the string received from the pre-configured framework to an application corresponding to the screen output via the display. A non-temporary computer-readable storage medium according to claim 14.

17. In a method for controlling a display device, The action of displaying a virtual keyboard on the screen to input information into an input window. An operation to display a virtual keyboard on the display in order to input information into the input window, An operation to receive an input signal from the first user terminal indicating the activation of the virtual keypad on the first user terminal, In response to sensing that the first user terminal is being used to control the display device, an action is taken to deactivate the display of the virtual keyboard on the display. The operation of receiving a key code, which is converted to a key selected on the virtual keypad of the first user terminal, from the first user terminal via the network interface, and The operation includes displaying text corresponding to the received key code in the input window shown on the display, A method for controlling a display device.

18. The operation further includes, in response to sensing that the virtual keypad of the first user terminal is being used to control the display device, deactivating the display of the virtual keyboard on the display, A method for controlling the display device according to claim 17.

19. The operation further includes activating a virtual input interface in the display device that is linked to the virtual keypad of the first user terminal, in response to receiving an input signal from the first user terminal indicating the activation of the virtual keypad of the first user terminal. A method for controlling the display device according to claim 17.

20. The first user terminal determines whether the virtual keypad is idle or not. The operation further includes, in response to the determination that the virtual keypad is idle in the first user terminal, the operation to deactivate the virtual input interface in the display device, A method for controlling the display device according to claim 19.

Images