Display device and operation method thereof

Abstract

A display device and an operation method thereof are disclosed according to embodiments. The disclosed display device controls a display to display screen data received from an electronic device connected to the display device, acquires gesture data by processing image data input through a camera, acquires touch data input through a touch input device, processes the gesture data and the touch data into a single human interface device (HID) signal, and transmits the HID signal to the electronic device.

Description

Display device and method of operation thereof

[0001] Various embodiments relate to a display device and a method of operating the same, and more particularly, to a display device connecting an input device and a source device and a method of operating the same.

[0002] A display device can receive and display screen data from various source devices. To control the screen data displayed on the display device, the display device can transmit input data for controlling the screen data to the source device. The display device can use a USB port to transmit the input data to the source device. When the display device acquires multiple types of input data, it can transmit the input data to the source device using a respective USB port corresponding to each type.

[0003] According to one embodiment of the present disclosure, when transmitting input data for controlling screen data displayed on the display device to an electronic device, it is intended to provide a display device capable of providing input data from a touch input device and input data from a camera to the electronic device using a single input / output interface.

[0004] The technical problems to be achieved in the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by a person having ordinary skill in the technical field to which the present disclosure belongs from the description below.

[0005] According to one embodiment, a display device may include a display, a transmit / receive interface, a memory storing one or more instructions, and a processor executing the one or more instructions stored in the memory.

[0006] According to one embodiment, at least one processor may control a display to display screen data received from an electronic device connected to the display device by executing one or more instructions.

[0007] According to one embodiment, at least one processor may process image data input through a camera to obtain gesture data by executing one or more instructions.

[0008] According to one embodiment, at least one processor may obtain touch data input through a touch input device by executing one or more instructions.

[0009] According to one embodiment, at least one processor may process gesture data and touch data into a single HID (Human Interface Device) signal by executing one or more instructions.

[0010] According to one embodiment, at least one processor may control a transmit / receive interface to transmit an HID signal to an electronic device by executing one or more instructions.

[0011] A method of operating a display device according to one embodiment may include displaying screen data received from an electronic device connected via an input / output interface.

[0012] According to one embodiment, a method of operating a display device may include an operation of processing image data input through a camera to obtain gesture data.

[0013] A method of operating a display device according to one embodiment may include obtaining touch data input via a touch input device.

[0014] A method of operating a display device according to one embodiment may include processing gesture data and touch data into a single HID (Human Interface Device) signal.

[0015] A method of operating a display device according to one embodiment may include transmitting an HID signal to an electronic device.

[0016] In one embodiment, a computer-readable recording medium having recorded thereon one or more programs executed by a processor of a display device for implementing a method of operating a display device, the method may include an operation of displaying screen data received from an electronic device connected through an input / output interface, an operation of processing image data input through a camera to obtain gesture data, an operation of obtaining touch data input through a touch input device, an operation of processing gesture data and touch data into a single HID (Human Interface Device) signal, and an operation of transmitting the HID signal to the electronic device.

[0017] According to a display device, method, and non-transitory computer-readable storage medium according to one embodiment, gesture data recognized from image data acquired from a camera is converted into touch data according to a corresponding touch input operation, and the data is provided to an electronic device using a USB interface for touch data, thereby providing the data without the need for a separate USB interface.

[0018] The effects that can be obtained from the embodiments disclosed in this disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art to which this disclosure pertains from the description below.

[0019] The above and other aspects, features and advantages of the embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings.

[0020] FIG. 1 is a reference diagram for explaining the operation of a display device and an electronic device according to one embodiment.

[0021] FIG. 2 illustrates an example of a display device system according to one embodiment.

[0022] FIG. 3 illustrates an example of a block diagram of a display device according to one embodiment.

[0023] FIG. 4 is a flowchart according to an example of a method of operating a display device according to one embodiment.

[0024] FIG. 5 illustrates an example of a method of operating a display device according to one embodiment.

[0025] FIG. 6 illustrates an example of a method of operating a display device according to one embodiment.

[0026] Figure 7 is a reference diagram for explaining the concept of HID class devices and class drivers according to the USB HID protocol.

[0027] FIG. 8 illustrates an example of a process for connecting a display device 100 to an electronic device 200 to be recognized as an HID according to the USB HID protocol according to one embodiment.

[0028] FIG. 9 illustrates an example of a table mapping gesture data and corresponding touch input actions according to one embodiment.

[0029] Fig. 10 illustrates an example of screen control according to the operation of a display device according to one embodiment.

[0030] FIG. 11 illustrates an example of a hardware configuration of a display device according to one embodiment.

[0031] FIG. 12 illustrates an example of a hardware configuration of a display device according to one embodiment.

[0032] The terms used in this specification will be briefly explained, and the present invention will be described in detail.

[0033] The terms used in this invention have been selected from widely used, current terms, taking into account their functions. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, in which case their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this invention should not be defined simply as names, but rather based on their inherent meanings and the overall content of the invention.

[0034] When a part of the specification is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless specifically stated otherwise. Furthermore, terms such as "module," "unit," and "part" used in the specification may be physically implemented by analog and / or digital circuits that include one or more of logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, etc.

[0035] Below, with reference to the attached drawings, embodiments of the present invention are described in detail so that those skilled in the art can easily implement the present invention. However, the present invention can be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description have been omitted to clearly explain the present invention, and similar parts have been designated with similar reference numerals throughout the specification.

[0036] In the embodiments of this specification, the term "user" means a person who controls a function or operation of a computing device or electronic device using a control device, and may include a viewer, an administrator, or an installer.

[0037] FIG. 1 is a reference diagram for explaining the operation of a display device and an electronic device according to one embodiment.

[0038] Referring to FIG. 1, display device 100 can display content received from various sources. For example, display device 100 can receive content from various sources and display the received content on a display. Display device 100 can be, for example, a television, a monitor, a personal computer, etc. Display device 100 can be connected to an electronic device 200 that provides content.

[0039] The electronic device 200 may refer to a device that provides content to the display device 100 in response to a request by a user input from the display device 100. The electronic device 200 may include, for example, electronic devices such as a personal computer, a game console, and a set-top box. The electronic device 200 may execute a content application in response to a request from the display device 100, and transmit a result screen according to the execution of the content application to the display device 100 via a communication network. For example, when the electronic device 200 is a game console, the electronic device 200 may execute the game application upon receiving a request to execute a game application from the display device 100, and transmit a result screen or result image rendered according to the execution of the game application to the display device 100 via a communication network. For example, if the electronic device 200 is a personal computer, when the electronic device 200 receives a request to execute a game application or movie content from the display device 100, the electronic device 200 may execute the game application or movie content and transmit a result screen rendered according to the execution of the game application or a result image played according to the execution of the movie content to the display device 100 through a communication network.

[0040] The display device 100 can transmit a request for executing a content application to the electronic device 200, and according to this execution request, can receive a result screen according to the execution of the content application from the electronic device 200 and display the received result screen on the display. At this time, since the application corresponding to the screen displayed on the display device 100 is running on the electronic device 200, in order to control the screen displayed on the display device 100, it is necessary to transmit input data for controlling the screen to the electronic device 200. Upon receiving such input data, the electronic device 200 can execute the application based on the received input data and transmit the execution result screen to the display device 100.

[0041] The display device 100 can be connected to the electronic device 200 via USB communication to transmit input data.

[0042] According to one embodiment, the display device 100 may receive input data from various input devices to control screen data displayed on the display. For example, the display device 100 may receive touch data from a touch input device such as a touch screen. For example, the display device 100 may receive touch data corresponding to touch coordinates pointing to a specific location on the screen, or a clicking or dragging action at the pointed location. For example, the display device 100 may receive image data capturing a user's gesture from a camera. By providing a method for appropriately converting image data capturing a user's gesture into touch data, the display device 100 according to an embodiment of the present disclosure can commonly transmit input data from a touch input device and input data from a camera through a single USB connection without requiring a separate USB connection for transmitting image data, by utilizing a USB connection for transmitting touch data.

[0043] According to one embodiment, the display device 100 can display screen data received from the electronic device 200.

[0044] According to one embodiment, the display device 100 can process image data input through a camera to obtain gesture data and obtain touch data input through a touch input device.

[0045] According to one embodiment, the display device 100 may process gesture data and touch data to generate one HID signal.

[0046] According to one embodiment, the display device 100 can process gesture data and touch data to generate USB HID data corresponding to a touch input operation for controlling screen data, and transmit the generated USB HID data to the electronic device 200.

[0047] According to one embodiment, the display device 100 may identify whether a recognized gesture corresponds to a predetermined gesture based on gesture data, and, when the recognized gesture is identified as corresponding to the predetermined gesture, may display a pointer at a location on the display specified by the recognized gesture.

[0048] According to one embodiment, the display device 100 can identify a touch input action based on gesture data acquired by processing image data input through a camera while a pointer is displayed.

[0049] According to one embodiment, the display device 100 may recognize a gesture based on gesture data acquired while a pointer is displayed, identify a touch input operation corresponding to the recognized gesture among one or more touch input operations, and generate the USB HID data corresponding to the identified touch input operation.

[0050] According to one embodiment, the display device 100 can detect a touch input through a touch input device and identify a touch input action based on touch data input through the touch input device.

[0051] According to one embodiment, the display device 100 may display the pointer at a location on the display specified by the direction in which the user's finger is pointing based on a recognized gesture.

[0052] FIG. 2 illustrates an example of a display device system according to one embodiment.

[0053] Referring to FIG. 2, the system may include a display device 100 and a source device 200 connected to a communication network.

[0054] The display device 100 may include a communication interface 110, an input / output interface 120, a camera 130, a touch input device 140, a display 150, a memory 160, and a processor 170. However, the display device 100 may be implemented with more components than the illustrated components and is not limited to the examples described above. For example, the display device 100 may further include a separate image processing unit for image processing of application images running on the display device 100. The communication interface 110 and the input / output interface 120 may be collectively referred to as a transmission / reception interface.

[0055] The communication interface 110 may perform communication with at least one external device. Here, 'communication' may mean an operation of transmitting and / or receiving data, signals, requests, and / or commands.

[0056] Communication interface 110 can perform wired or wireless communication with at least one external device. For example, the external device may be a source device, server, storage device, or the like capable of providing content. Alternatively, the external device may be an Internet server or the like capable of providing content. Alternatively, the external device may be an electronic device or server capable of processing or providing data.

[0057] For example, the communication interface 110 may include at least one of a communication module, a communication circuit, a communication device, an input / output port, and an input / output plug for performing wired or wireless communication with at least one external device.

[0058] The communication interface 110 may include one or more modules that enable wireless communication between the display device 100 and a wireless communication system or between the display device 100 and a network in which other devices are located.

[0059] According to one embodiment, the communication interface 110 may perform communication with the display device 100 according to a short-range communication technology. The short-range communication technology may include, for example, Bluetooth communication, Wi-Fi communication, etc. According to one embodiment, the communication interface 110 may include at least one of a Bluetooth communication module that performs communication according to a Bluetooth communication protocol and a Wi-Fi communication module that performs communication according to a Wi-Fi communication protocol.

[0060] According to one embodiment, the communication interface 110 may perform a connection operation with the communication interface 210 of the electronic device 200 under the control of the processor 170, and receive screen data from the electronic device 200.

[0061] According to one embodiment, the communication interface 110 may transmit input data for controlling screen data to the electronic device 200 under the control of the processor 170. For example, the communication interface 110 may transmit input data for controlling screen data to the electronic device 200 using wireless USB technology. The wireless USB technology may include Bluetooth USB technology or Wi-Fi USB technology.

[0062] The input / output interface 120 can receive video (e.g., moving images), audio (e.g., voice, music, etc.), and additional information from the outside of the display device 100 under the control of the processor 170. The input / output interface 120 may include one of an HDMI port (High-Definition Multimedia Interface port), a component jack, a PC port, and a USB port, or a combination of one or more thereof. In addition to the HDMI port, the input / output interface 120 may further include a DisplayPort (DP), Thunderbolt, or MHL (Mobile High-Definition Link).

[0063] According to one embodiment, the input / output interface 120 may be connected to the electronic device 200 via an HDMI port. Accordingly, the input / output interface 120 may receive screen data to be displayed on the display of the display device 100 from the electronic device 200 via the HDMI port.

[0064] According to one embodiment, the input / output interface 120 may be connected to the electronic device 200 via a USB port. Accordingly, the input / output interface 120 may transmit input data for controlling screen data to the electronic device 200 via the USB port.

[0065] Camera 130 can capture image data by capturing images of a user using one or more image sensors. Camera 130 can be built into display device 100 or can be connected to display device 100 through communication.

[0066] The touch input device 140 can detect a user's touch input and obtain touch data. The touch input device may include a touch sensor, a touch-sensitive display, a touch panel, etc. The touch input device 140 may be built into the display device 100 or may be connected to the display device 100 through communication.

[0067] According to one embodiment, the touch input device 140 may include an HID controller to output USB HID data corresponding to touch data acquired by the touch input device 140.

[0068] The display 150 can display screen data processed internally within the display device 100 or received from an external source device.

[0069] According to one embodiment, the display 150 may display screen data provided from the electronic device 200 under the control of the processor 170.

[0070] The memory 160 can store a program for processing and controlling the processor 170, and can store data input to or output from the display device 100.

[0071] The memory 160 may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk.

[0072] The processor 170 controls the overall operation of the display device 100. For example, the processor 170 may perform the functions of the display device 100 described in the present disclosure by executing one or more instructions stored in the memory 160. The processor 170 may include various processing circuits and / or multiple processors. For example, the term "processor" as used herein, including in the claims, may include various processing circuits, including at least one processor. One or more processors in at least one processor may be configured to individually and / or collectively perform the various functions described herein in a distributed manner. As used herein, "processor," "at least one processor," and "one or more processors" may be configured to perform multiple functions. However, these terms encompass, for example and without limitation, situations where one processor performs some of the functions and other processor(s) perform other parts of the functions, and situations where a single processor may perform all of the functions. Furthermore, the at least one processor may include a combination of processors that perform various functions of the disclosed functions in a distributed manner. At least one processor is capable of executing program instructions to accomplish or perform various functions.

[0073] In an embodiment of the present disclosure, the processor 170 can store one or more instructions in an internally provided memory, and control the execution of the one or more instructions stored in the internally provided memory so that the aforementioned operations are performed. That is, the processor 170 can perform a predetermined operation by executing at least one instruction or program stored in the internal memory or memory 160 provided within the processor 170.

[0074] According to one embodiment, the processor 170 may control the display to display screen data received from an electronic device connected to the display device by executing one or more instructions.

[0075] According to one embodiment, the processor 170 may process image data input through a camera to obtain gesture data by executing one or more instructions.

[0076] According to one embodiment, the processor 170 may obtain touch data input through a touch input device by executing one or more instructions.

[0077] According to one embodiment, the processor 170 may process USB HID data or the gesture data based on touch data by executing one or more instructions to generate USB HID data corresponding to a touch input operation for controlling the screen data.

[0078] According to one embodiment, the processor 170 may control an input / output interface to transmit generated USB HID data to the electronic device by executing one or more instructions.

[0079] According to one embodiment, the processor 170 may execute one or more instructions to identify whether a recognized gesture corresponds to a predetermined gesture based on the gesture data, and, upon identifying that the recognized gesture corresponds to the predetermined gesture, display a pointer at a location on the display specified by the recognized gesture.

[0080] In one embodiment, the location of the display where the pointer is displayed may indicate a location specified by the direction in which the user's finger is pointing based on the recognized gesture.

[0081] According to one embodiment, the processor 170 may generate USB HID data including coordinate information of the display specified by the recognized gesture by executing one or more instructions, and receive and display screen data on which the pointer is displayed from the electronic device in response to transmitting the USB HID data to the electronic device, when the recognized gesture is identified as corresponding to the predetermined gesture.

[0082] According to one embodiment, the processor 170 may identify the touch input operation based on gesture data acquired by processing the image data input through the camera while the pointer is displayed by executing one or more instructions.

[0083] According to one embodiment, the processor 170 may execute one or more instructions to recognize a gesture based on the gesture data acquired while the pointer is displayed, identify a touch input operation corresponding to the recognized gesture among one or more touch input operations, and generate the USB HID data corresponding to the identified touch input operation.

[0084] According to one embodiment, the processor 170 may identify the touch input operation based on the touch data input through the touch sensitive display by executing one or more instructions to detect a touch input through the touch sensitive display.

[0085] Now let's describe the electronic device 200.

[0086] The electronic device 200 may include a communication interface 210, an input / output interface 220, a memory 230, and a processor 240. However, the electronic device 200 may be implemented with more components than the illustrated components and is not limited to the examples described above. For example, the electronic device 200 may further include a separate image processing unit for image processing of application images running on the electronic device 200. The communication interface 210 and the input / output interface 220 may be collectively referred to as a transmission / reception interface.

[0087] The communication interface 210 may perform communication with at least one external device. Here, 'communication' may mean an operation of transmitting and / or receiving data, signals, requests, and / or commands.

[0088] Communication interface 210 can perform wired or wireless communication with at least one external device. For example, the external device may be a source device, server, storage device, or the like capable of providing content. Alternatively, the external device may be an Internet server capable of providing content. Alternatively, the external device may be an electronic device or server capable of processing or providing data.

[0089] For example, the communication interface 210 may include at least one of a communication module, a communication circuit, a communication device, an input / output port, and an input / output plug for performing wired or wireless communication with at least one external device.

[0090] The communication interface 210 may include one or more modules that enable wireless communication between the electronic device 200 and a wireless communication system or between the electronic device 200 and a network in which other devices are located.

[0091] According to one embodiment, the communication interface 210 may perform communication with the display device 100 according to a short-range communication technology. The short-range communication technology may include, for example, Bluetooth communication, Wi-Fi communication, etc. According to one embodiment, the communication interface 210 may include at least one of a Bluetooth communication module that performs communication according to a Bluetooth communication protocol and a Wi-Fi communication module that performs communication according to a Wi-Fi communication protocol.

[0092] The input / output interface 220 can output video (e.g., moving images), audio (e.g., voice, music, etc.), and additional information to the outside of the electronic device 200 under the control of the processor 240. The input / output interface 220 may include one of an HDMI port (High-Definition Multimedia Interface port), a component jack, a PC port, and a USB port, or a combination of one or more thereof. In addition to the HDMI port, the input / output interface 220 may further include a DisplayPort (DP), Thunderbolt, and MHL (Mobile High-Definition Link). For example, when the electronic device 200 is implemented as a game console, the electronic device 200 can provide a result screen of game content executed on the electronic device 200 to the display device 100 via HDMI, etc.

[0093] According to one embodiment, the input / output interface 220 may be connected to the display device 100 via an HDMI port. Accordingly, the input / output interface 220 may output screen data to be displayed on the display of the display device 100 via the HDMI port.

[0094] According to one embodiment, the input / output interface 220 may be connected to the display device 100 via a USB port. Accordingly, the input / output interface 220 may receive input data for controlling screen data from the display device 100 via the USB port.

[0095] The memory 230 can store a program for processing and controlling the processor 240, and can store data input to or output from the electronic device 200.

[0096] The memory 230 may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk.

[0097] The processor 240 controls the overall operation of the electronic device 200. For example, the processor 240 may perform the functions of the electronic device 200 described in the present disclosure by executing one or more instructions stored in the memory 230.

[0098] In an embodiment of the present disclosure, the processor 240 can store one or more instructions in an internally provided memory, and control the execution of the one or more instructions stored in the internally provided memory so that the aforementioned operations are performed. That is, the processor 240 can perform a predetermined operation by executing at least one instruction or program stored in an internal memory provided within the processor 240 or in the memory 230.

[0099] According to one embodiment, the processor 240 may control the input / output interface 220 to transmit screen data to be displayed on the display of the display device 100 to the display device 100 by executing one or more instructions stored in the memory 230.

[0100] According to one embodiment, the processor 240 may control the input / output interface 220 to receive input data from the display device 100 by executing one or more instructions stored in the memory 230.

[0101] According to one embodiment, the processor 240 may control screen data or execute an application based on input data received from the display device 100 by executing one or more instructions stored in the memory 230.

[0102] The block diagram of the electronic device 200 illustrated in FIG. 2 is a block diagram for one embodiment. Each component of the block diagram may be integrated, added, or omitted depending on the specifications of the electronic device 200 actually implemented. For example, two or more components may be combined into a single component, or one component may be subdivided into two or more components, as needed. In addition, the functions performed by each block are for the purpose of explaining embodiments, and the specific operations or devices thereof do not limit the scope of the present invention.

[0103] FIG. 3 illustrates an example of a block diagram of a display device according to one embodiment.

[0104] Referring to FIG. 3, the display device 100 is a device that can display images or data according to a user's request, and may include a communication interface 110, an input / output interface 120, a camera 130, a touch input device 140, a display 150, an image processing unit 155, a memory 160, a processor 170, an audio processing unit 185, and an audio output unit 180.

[0105] The communication interface 110 may perform communication with at least one external device. Here, 'communication' may mean an operation of transmitting and / or receiving data, signals, requests, and / or commands.

[0106] Communication interface 110 can perform wired or wireless communication with at least one external device. For example, the external device may be a source device, server, storage device, or the like capable of providing content. Alternatively, the external device may be an Internet server or the like capable of providing content. Alternatively, the external device may be an electronic device or server capable of processing or providing data.

[0107] For example, the communication interface 110 may include at least one of a communication module, a communication circuit, a communication device, an input / output port, and an input / output plug for performing wired or wireless communication with at least one external device.

[0108] The communication interface 110 may include one or more modules that enable wireless communication between the display device 100 and a wireless communication system or between the display device 100 and a network in which other devices are located.

[0109] The communication interface 110 may further include a mobile communication module 111 for transmitting and receiving wireless signals with at least one of a base station, an external terminal, and a server on a mobile communication network, a wireless Internet module 112 for wireless Internet access, and a short-range communication module 113. Wireless Internet technologies that may be used include WLAN (Wireless LAN) (WiFi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), and HSDPA (High Speed ​​Downlink Packet Access). In addition to the Wi-Fi communication module and the Bluetooth communication module, the short-range communication module 113 may include a communication module that uses short-range communication technologies such as BLE (Bluetooth Low Energy), RFID (Radio Frequency Identification), IrDA (infrared Data Association), UWB (Ultra Wideband), and ZigBee.

[0110] The communication interface 110 can receive screen data by being connected to the electronic device 200 through the short-range communication module 113 under the control of the processor 170.

[0111] The input / output interface 120 can receive video (e.g., moving images, etc.), audio (e.g., voice, music, etc.), and additional information (e.g., EPG, etc.) from the outside of the display device 100 under the control of the processor 170. The input / output interface 120 may include one of an HDMI port (High-Definition Multimedia Interface port) 121, a component jack 122, a PC port 123, and a USB port 124, or a combination of one or more thereof. In addition to the HDMI port, the input / output interface 120 may further include a DisplayPort (DP), Thunderbolt, or MHL (Mobile High-Definition Link).

[0112] The input / output interface 120 can receive screen data from the connected electronic device 200 by connecting to the electronic device through one or more HDMI ports.

[0113] The input / output interface 120 can transmit input data for controlling screen data to the electronic device 200 via a USB port.

[0114] Camera 130 can receive images (e.g., consecutive frames) corresponding to a user's motion, including a gesture, within the camera's recognition range. Processor 170 can use the recognition results of the received motion to select a menu displayed on the display device 100 or perform a control corresponding to the motion recognition results.

[0115] Touch input device 140 can receive touch input from a user.

[0116] The touch input device 140 may be implemented as a touch sensitive display integrated with the display 150.

[0117] Display 150 can output images or data processed in Display Device 100.

[0118] The image processing unit 155 can process an image signal and output it to the display 150 under the control of the processor 170.

[0119] The memory 160 can store a program for processing and controlling the processor 170, and can store data input to or output from the display device 100. In addition, the memory 160 can store data necessary for the operation of the display device 100.

[0120] The memory 160 may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk.

[0121] The processor 170 controls the overall operation of the display device 100. For example, the processor 170 may perform the functions of the display device 100 described in the present disclosure by executing one or more instructions stored in the memory 160.

[0122] In an embodiment of the present disclosure, the processor 170 can store one or more instructions in an internally provided memory, and control the execution of operations of the display device by executing one or more instructions stored in the internally provided memory. That is, the processor 170 can perform a predetermined operation by executing at least one instruction or program stored in the internal memory or memory 160 provided within the processor 170.

[0123] The processor 170 controls the overall operation of the display device 100 and the signal flow between internal components (not shown) of the display device 100, and performs the function of processing data. When there is a user input or a preset stored condition is satisfied, the processor 170 can execute the OS (Operating System) stored in the memory 140 and various applications.

[0124] The processor 170 may include a graphic processor (Graphic Processing Unit, not shown) for graphic processing corresponding to video. The graphic processing unit (not shown) generates a screen including various objects such as icons, images, and text using a calculation unit (not shown) and a rendering unit (not shown). The calculation unit calculates attribute values ​​such as coordinate values, shape, size, and color to be displayed for each object according to the layout of the screen using user interactions detected by a detection unit (not shown). The rendering unit generates a screen including various layouts of objects based on the attribute values ​​calculated by the calculation unit.

[0125] According to one embodiment, the processor 170 may perform operations of the display device 100 disclosed in the present disclosure by executing one or more instructions stored in the memory 160.

[0126] The audio output unit 180 can output input audio (e.g., voice, sound). In addition, the audio output unit 180 can output audio stored in the memory 140 or audio received from the electronic device 200 under the control of the processor 170. The audio output unit 180 can include at least one of a speaker, a headphone output terminal, or an S / PDIF (Sony / Philips Digital Interface:) output terminal, or a combination thereof.

[0127] The audio processing unit 185 can convert an audio signal into an analog audio signal and output it to the audio output unit 180 under the control of the processor 170.

[0128] Although not shown, the display device 100 may further include a user input interface. The user input interface may be any form of interface means capable of receiving user input. For example, the user input interface may include operation buttons arranged on a portion of the display device 100 capable of receiving user input, a touch-sensitive display configured to detect touch input, a microphone capable of receiving user voice input, and the like.

[0129] The display device 100 may be any type of device that performs a function, including a processor and memory. The display device 100 may be a fixed or portable device. For example, the display device 100 may refer to a device that has a display and can display image content, video content, game content, graphic content, etc. The display device 100 may output or display image or content received from the source device 300. The display device 100 may include various types of electronic devices that can receive and output content, such as televisions such as network TVs, smart TVs, Internet TVs, web TVs, and IPTVs; computers such as desktops, laptops, and tablets; smart phones, cellular phones, game players, music players, video players, medical equipment, and home appliances. The display device 100 may be referred to as a display device in that it receives and displays content, and may also be referred to as a content receiving device, a sink device, an electronic device, a computing device, etc.

[0130] The block diagram of the display device 100 illustrated in FIG. 2 is a block diagram for one embodiment. Each component of the block diagram may be integrated, added, or omitted depending on the specifications of the display device 100 actually implemented. For example, two or more components may be combined into one component, or one component may be subdivided into two or more components, as needed. In addition, the functions performed by each block are for the purpose of explaining embodiments, and the specific operations or devices thereof do not limit the scope of the present invention.

[0131] FIG. 4 is a flowchart according to an example of a method of operating a display device according to one embodiment.

[0132] Referring to FIG. 4, in operation 410, the display device 100 can receive and display screen data from an electronic device connected via wired or wireless communication.

[0133] According to one embodiment, the display device 100 may receive and display screen data from an electronic device connected via wireless communication such as Bluetooth communication or Wi-Fi communication, or may receive and display screen data from an electronic device connected via an input / output interface such as HDMI.

[0134] In operation 420, the display device 100 can process image data input through a camera to obtain gesture data.

[0135] According to one embodiment, the display device 100 may receive image data input through a camera built into the display device 100 or a camera connected through a communication interface. The image data may include an image of a user making a gesture in front of the display device 100.

[0136] According to one embodiment, the display device 100 can perform gesture recognition by performing image processing on image data input through a camera to obtain gesture data.

[0137] In operation 430, the display device 100 can obtain touch data input through a touch input device.

[0138] According to one embodiment, the display device 100 can obtain touch data input through a touch input device such as a touch sensor, a touch panel, or a touch sensitive display.

[0139] At operation 440, the display device 100 may process gesture data and touch data to generate USB HID data corresponding to a touch input operation for controlling screen data.

[0140] According to one embodiment, when the display device 100 acquires touch data, it may generate USB HID data corresponding to the touch data.

[0141] According to one embodiment, the display device 100 may, when acquiring gesture data, determine whether the acquired gesture data corresponds to a predetermined gesture.

[0142] According to one embodiment, the display device 100 can determine whether the subsequently acquired gesture data corresponds to a touch input operation for controlling screen data if it is determined that the acquired gesture data corresponds to a predetermined gesture.

[0143] According to one embodiment, the display device 100 may convert the acquired gesture into a pointing motion if it is determined that the acquired gesture corresponds to a predetermined gesture.

[0144] According to one embodiment, the display device 100 determines whether gesture data acquired through a camera while a pointer is displayed on the display corresponds to a gesture corresponding to one or more touch input actions, and if it is determined that the acquired gesture data corresponds to a gesture corresponding to one touch input action, the display device 100 may generate USB HID data based on the touch input action corresponding to the acquired gesture data.

[0145] At operation 450, the display device 100 can transmit the generated USB HID data to the electronic device.

[0146] FIG. 5 illustrates an example of a method of operating a display device according to one embodiment.

[0147] Referring to FIG. 5, in operation 510, the display device 100 and the electronic device 200 can perform a connection operation for transmitting and receiving screen data.

[0148] According to one embodiment, the display device 100 may be connected to an electronic device 200 that provides screen data using a wired or wireless communication technology. The wireless communication technology may include, for example, Bluetooth communication technology or Wi-Fi communication technology.

[0149] According to one embodiment, the display device 100 can be connected to the electronic device 200 using an HDMI port.

[0150] In operation 520, the display device 100 and the electronic device 200 can perform a connection operation for transmitting and receiving input data.

[0151] According to one embodiment, the display device 100 may be connected to the electronic device 200 using wired or wireless communication technology to provide input data. The wired communication technology may include, for example, a USB interface. The wireless communication technology may include wireless USB communication technology, and Bluetooth or Wi-Fi communication may be used for wireless communication.

[0152] In one embodiment, when connecting using a USB interface, the upstream USB port of the display device 100 may be connected to the downstream USB port of the electronic device 200 using a USB cable. With this connection, the display device 100 may be recognized as an HID by the electronic device 200. The Universal Serial Bus (USB) is a concept that groups devices with similar data reporting characteristics into device classes and has a single class driver for each group, instead of having separate software drivers for each new computer peripheral. Additionally, devices have the ability to explain to the class driver what control they have and how to report data. An HID (Human Interface Device) device may represent a device that provides services for inputting and outputting human data to and from a host. Examples of HIDs may include a mouse, a joystick, a gamepad, a keyboard, or a voltmeter and temperature sensor. The electronic device 200 and the display device 100 can be connected by plugging in a USB cable connected to an upstream USB port of the display device 100 to a downstream USB port of the electronic device 200. When the electronic device 200 and the display device 100 are connected with the USB cable, the electronic device 200 can recognize that a device, i.e., the display device 100, is plugged in. The electronic device 200 can request device information from the display device 100. Then, the display device 100 can transmit a device descriptor containing information about the display device to the electronic device 200. Specifically, the information about the display device can include information about a touch input device involved in transmitting input data from the display device 100.Information about the touch input device contained in the device descriptor may include information about the vendor ID of the touch input device, the product ID of the touch input device, and the firmware version of the touch input device. The electronic device 200 may recognize information about the touch input device from the device descriptor and load an appropriate HID class driver for the touch input device. The HID class driver may select a configuration for the device, i.e., a configuration for the touch input device, and transmit a configuration setting request to the display device 100. The display device 100, specifically, the touch input device, may be configured according to the configuration setting. By this operation, the display device 100, specifically, the touch input device, may be recognized as an HID by the electronic device 200 connected to its upstream USB port.

[0153] Figure 7 is a reference diagram for explaining the concept of HID class devices and class drivers according to the USB HID protocol.

[0154] Referring to FIG. 7, HID class device 710 and HID class driver 730 transmit and receive device descriptors and data reports.

[0155] Instead of requiring separate software drivers for each new computer peripheral, USB (Universal Serial Bus) groups devices with similar data reporting characteristics into device classes, each with a single class driver. Devices also have the ability to tell the class driver what control they have and how they report data.

[0156] The HID (Human Interface Device) class device 710 may represent a device that provides services for inputting and outputting human data to and from the host 720. Examples of HIDs may include a mouse, joystick, gamepad, keyboard, or voltmeter and temperature sensor. The host 720 may represent a device that utilizes or requests the services of the HID. The host 720 may include a personal computer, a handheld computer, a game console, a data logger, and the like.

[0157] Information about a USB device is stored in segments called descriptors in the USB device's memory.

[0158] The HID class device 710 retrieves and transmits all data using the corresponding HID class driver 730. The path and retrieval of the data can be accomplished by examining the device's descriptor and data report.

[0159] The report descriptor 740 describes the data generated by the HID class device 710 and what that data measures. For example, the report descriptor 840 may define items describing the location and button states of the HID class device. By examining the items in the report descriptor 740, the HID class driver 830 can determine the size and structure of the data report from the HID class device 710.

[0160] FIG. 8 illustrates an example of a process for connecting a display device 100 to an electronic device 200 to be recognized as an HID according to the USB HID protocol according to one embodiment.

[0161] Referring to FIG. 8, in operation 810, an upstream USB port of the display device 100 and the electronic device 200 can be connected with a USB cable. That is, the display device and the electronic device 200 can be connected by plugging in a USB cable connected to the display device 100 to a downstream USB port (i.e., host) of the electronic device 200.

[0162] In operation 820, when the electronic device 200 and the display device 100 are connected with a USB cable, the electronic device 200 can recognize that the device, i.e., the display device 100, is plugged in.

[0163] At operation 830, the electronic device 200 may transmit a request to set a unique address of the device to the display device 100.

[0164] At operation 840, the electronic device 200 may request device information from the display device 100.

[0165] In operation 850, the display device 100 may transmit a device descriptor containing information about a touch input device to the electronic device 200. The information about the touch input device contained in the device descriptor may include a vendor ID, a product ID, and firmware version information of the touch input device.

[0166] At operation 860, the electronic device 200 can recognize information about a touch input device included in the display device 100 from the device descriptor and load an appropriate HID class driver for the touch input device.

[0167] In operation 870, the HID class driver can select a configuration for the device, i.e., a configuration for a touch input device.

[0168] At operation 880, the HID class driver may send a configuration request to display device 100.

[0169] In operation 890, the touch input device of the display device 100 may be configured according to configuration settings.

[0170] By this action, the display device 100 can be recognized as a HID to the electronic device 200 connected to its upstream USB port.

[0171] In this way, the display device 100 and the electronic device 200 can perform a connection operation for transmitting and receiving input data.

[0172] Returning to FIG. 5 again, at operation 530, electronic device 200 may transmit screen data to display device 100.

[0173] According to one embodiment, the electronic device 200 may execute an application and transmit the resulting screen data to the display device 100.

[0174] At operation 540, the display device 100 may receive screen data from the electronic device 200 and display the screen data at operation 550.

[0175] At operation 560, the display device 100 can obtain input data.

[0176] According to one embodiment, the input data may include touch data acquired through a touch input device included in the display device 100. The touch data may include data that the touch input device can receive, such as pointing coordinates where a user points at the touch input device using a finger or a pointing device, an input of clicking at the pointing location, an input of dragging at the pointing location, and the like.

[0177] In one embodiment, the input data may include image data acquired through a camera. The image data may include data captured by the camera of a user's movements or gestures.

[0178] At step 570, the display device 100 can determine whether the acquired input data is touch data. If the determination result is not touch data, the process can proceed to step 600. If the determination result is touch data, the process can proceed to step 580.

[0179] At operation 580, the display device 100 may generate USB HID data corresponding to the touch data.

[0180] According to one embodiment, the display device 100 may generate a USB HID containing information representing touch data, i.e., touch coordinates or touch-related input, for example, a click input or a drag input.

[0181] At operation 590, the display device 100 may transmit USB HID data to the electronic device 200.

[0182] In operation 595, the electronic device 200 may control screen data based on USB HID data received from the display device 100. For example, if the USB HID data includes pointing coordinates, a pointer may be displayed at the location of the pointing coordinates. For example, if the USB HID data includes a click input, an operation may be performed to activate an item corresponding to the pointing location. As a result of controlling screen data based on the USB HID data in this way, the screen data may be transmitted to the display device 100 and displayed on the display device 100.

[0183] FIG. 6 illustrates an example of a method of operating a display device according to one embodiment.

[0184] If the input data is determined to be not touch data in operation 570 of FIG. 5, the process proceeds to operation 600, and with reference to FIG. 6, in operation 601, the display device 100 can recognize a gesture based on gesture data.

[0185] In operation 602, the display device 100 can determine whether the recognized gesture corresponds to a predetermined gesture.

[0186] Since a pointing motion must be performed on the display device to convert a gesture input into a touch input, this is to determine whether such a pointing motion has occurred. In other words, a predetermined gesture can serve as a standard gesture for determining whether a pointing motion has occurred. For example, a predetermined gesture for determining whether a pointing motion has occurred may include a motion in which a user points at the display device 100 with an index finger. Of course, this is merely an example, and the predetermined gesture for determining a pointing motion may be determined in various ways.

[0187] If the recognized gesture does not correspond to a predetermined gesture as a result of the judgment, the operation can be terminated.

[0188] If the recognized gesture is judged to correspond to a predetermined gesture as a result of the judgment, the process can proceed to action 603.

[0189] In operation 603, the display device 100 can obtain coordinates of the display specified by the recognized gesture.

[0190] According to one embodiment, the display device 100 may obtain coordinates that can be identified by the recognized gesture by analyzing the recognized gesture. For example, if the predetermined gesture is a motion of pointing at the display device 100 with a thumb, the display device 100 may obtain coordinates where the thumb meets the display device in the direction pointed at.

[0191] At operation 604, the display device 100 may generate a USB HID containing coordinate information.

[0192] At operation 605, the display device 100 may transmit USB HID data to the electronic device 200.

[0193] At operation 606, the electronic device 200 may display a pointer on the screen based on USB HID data containing coordinate information.

[0194] In operation 607, the electronic device 200 may transmit screen data showing a pointer on the screen to the display device 100.

[0195] In operation 608, the display device 100 can display screen data with a pointer displayed.

[0196] In operation 609, the display device 100 can acquire gesture data by processing image data input through the camera while the pointer is displayed.

[0197] At operation 610, the display device 100 can identify a touch input action corresponding to gesture data.

[0198] According to one embodiment, the display device 100 may include information mapping each corresponding touch input action to one or more gesture data, and may obtain a touch input action corresponding to the gesture data by referring to the mapping information.

[0199] FIG. 9 illustrates an example of a table mapping gesture data and corresponding touch input actions according to one embodiment.

[0200] Referring to FIG. 9, table 900 may include one or more gestures and a touch input action corresponding to each gesture. For example, a touch input action corresponding to a gesture of extending an index finger may be a pointer exposure (display). For example, a touch input action corresponding to a gesture of making a fist and then opening it may be a click. For example, a touch input action corresponding to a gesture of moving a palm in an open state may be a drag. For example, a touch input action corresponding to a gesture of swiping a hand to the left may be a backward movement or a movement of a pointer to the left. For example, a touch input action corresponding to a gesture of swiping a hand to the right may be a forward movement or a movement of a pointer to the right.

[0201] Returning to FIG. 6 again, if a touch input action corresponding to a gesture is identified using table 900 as illustrated in FIG. 9 at operation 610, then at operation 611, the display device 100 can generate USB HID data corresponding to the identified touch input action.

[0202] At operation 612, the display device 100 may transmit the identified USB HID data to the electronic device 200.

[0203] In operation 613, the electronic device 200 can control screen data based on the USB HID data. For example, if the USB HID data includes data corresponding to a click input, the electronic device 200 can obtain a result screen that appears according to an action of clicking an item corresponding to the position where the pointer is displayed. For example, if the USB HID data includes data corresponding to an input of moving the pointer to the right, the electronic device 200 can obtain a result screen that appears according to an action of moving the pointer to the right.

[0204] At operation 614, the electronic device 200 may transmit the resulting screen data to the display device 100.

[0205] In operation 615, the display device 100 may display the result screen data received from the electronic device 200.

[0206] Fig. 10 illustrates an example of screen control according to the operation of a display device according to one embodiment.

[0207] Referring to FIG. 10, the display device 100 can display screen data 1000A received from the electronic device 200 on the display. For example, the screen data 1000A can include a first menu 1010, a second menu 1020, and a third menu 1030.

[0208] The display device 100 captures an image of a user using a camera to obtain image data, and can recognize a gesture by analyzing the image data at this time. If the display device 100 determines that the gesture recognition result corresponds to a predetermined gesture, for example, if the gesture is recognized as a gesture in which the user points at the display device 100 with an index finger, the display device 100 can obtain display coordinate information specified by the user's gesture, generate a USB HID including the obtained coordinate information, and transmit the same to the electronic device 200. Accordingly, the electronic device 200 can generate screen data 1000B that displays a pointer at the coordinates included in the USB HID, and transmit the same to the display device 100.

[0209] Accordingly, the display device 100 can display screen data 1000B in which a pointer is displayed in the first menu 1010. When the user sees the screen data 1000B and makes a motion of moving the hand to the right, the camera of the display device 100 can capture this to obtain image data and analyze the image data to recognize the gesture. The display device 100 can identify a touch input motion of moving the pointer to the right in response to the gesture of the user moving the hand to the right, for example, by referring to the table of FIG. 9. The display device 100 can generate a USB HID including an input of moving the pointer to the right and transmit it to the electronic device 200. The electronic device 200 can generate a result screen 1000C in which the pointer 1050 is displayed in the second menu 1020 according to the control of moving the pointer to the right according to the information included in the USB HID and transmit it to the display device 100. Accordingly, the display device 100 can display the screen 1000C as a result of the pointer 1050 moving to the second menu 1020.

[0210] FIG. 11 illustrates an example of a hardware configuration of a display device according to one embodiment.

[0211] Referring to FIG. 11, the touch input device 140 may include an HID controller 145. The HID controller 145 may generate USB HID data based on touch data acquired from the touch input device 140 and output the generated data to the processor 170.

[0212] The processor 170 can receive image data from the camera 130 and USB HID data from the touch input device 140.

[0213] According to one embodiment, when the processor 170 receives USB HID data from the touch input device 140, it can control the input / output unit 120 to transmit the USB HID data to the electronic device 200.

[0214] According to one embodiment, when the processor 170 receives image data from the camera 130, the processor 170 may convert the received image data into touch data corresponding to the recognized gesture if the recognized gesture satisfies a predetermined condition, thereby generating USB HID data based on the converted touch data. In addition, the processor may control the input / output unit 120 to transmit the USB HID data generated based on the gesture in this way.

[0215] FIG. 12 illustrates an example of a hardware configuration of a display device according to one embodiment.

[0216] Referring to FIG. 12, compared to the example shown in FIG. 11, it further includes a switch 1200.

[0217] The touch input device 140 may include an HID controller 145. The HID controller 145 may generate USB HID data based on touch data acquired from the touch input device 140 and output the data to the switch 1200.

[0218] Switch 1200 can transmit USB HID data received from touch input device 140 to input / output unit 120.

[0219] According to one embodiment, when the processor 170 receives image data from the camera 130, the processor 170 may convert the received image data into touch data corresponding to the recognized gesture if the recognized gesture satisfies a predetermined condition, thereby generating USB HID data based on the converted touch data. The processor may then output the USB HID data generated based on the gesture to the switch 1200.

[0220] In one embodiment, when the processor 170 outputs USB HID data generated based on a gesture to the switch 1200, the switch 1200 can control the switch 1200 to output the USB HID data generated based on the gesture to the input / output unit 120.

[0221] Some embodiments may also be implemented in the form of a recording medium containing computer-executable instructions, such as program modules, executed by a computer. Computer-readable media may be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Computer-readable media may also include computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data.

[0222] The disclosed embodiments can be implemented as a S / W program including commands stored in a computer-readable storage media.

[0223] A computer may include an electronic device according to the disclosed embodiments, as a device capable of calling a command stored from a storage medium and performing an operation according to the called command according to the disclosed embodiments.

[0224] Computer-readable storage media may be provided in the form of non-transitory storage media. Here, "non-transitory" simply means that the storage medium does not contain signals and is tangible, but does not distinguish between whether data is stored semi-permanently or temporarily on the storage medium.

[0225] Additionally, the control method according to the disclosed embodiments may be provided as a computer program product. The computer program product may be traded as a product between a seller and a buyer.

[0226] A computer program product may include a software program and a computer-readable storage medium on which the software program is stored. For example, the computer program product may include a product in the form of a software program (e.g., a downloadable app) distributed electronically by a device manufacturer or through an electronic marketplace (e.g., Google Play Store, App Store). For electronic distribution, at least a portion of the software program may be stored on a storage medium or temporarily created. In this case, the storage medium may be a storage medium of a manufacturer's server, an electronic marketplace server, or a relay server that temporarily stores the software program.

[0227] In a system comprising a server and a device, the computer program product may include a storage medium of the server or a storage medium of the device. Alternatively, if a third device (e.g., a smartphone) exists that is communicatively connected to the server or device, the computer program product may include a storage medium of the third device. Alternatively, the computer program product may include a software program itself that is transmitted from the server to the device or the third device, or from the third device to the device.

[0228] In this case, one of the server, the device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments. Alternatively, two or more of the server, the device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments in a distributed manner.

[0229] For example, a server (e.g., a cloud server or an artificial intelligence server, etc.) may execute a computer program product stored on the server to control a device in communication with the server to perform a method according to the disclosed embodiments.

[0230] As another example, a third device may execute a computer program product to control a device in communication with the third device to perform a method according to the disclosed embodiments. When the third device executes the computer program product, the third device may download the computer program product from a server and execute the downloaded computer program product. Alternatively, the third device may execute a computer program product provided in a preloaded state to perform the method according to the disclosed embodiments.

[0231] Additionally, in this specification, a “part” may be a hardware component such as a processor or circuit, and / or a software component executed by a hardware component such as a processor.

[0232] The above description of the present disclosure is provided for illustrative purposes only, and those skilled in the art will readily appreciate that modifications to other specific forms can be made without altering the technical spirit or essential features of the present disclosure. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, components described as being single may be implemented in a distributed manner, and similarly, components described as being distributed may be implemented in a combined manner.

[0233] The scope of the present disclosure is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present disclosure.

Claims

1. In the display device 100, Display 150, Transmit / receive interface 110, 120, Memory 160 storing one or more instructions, and At least one processor 170 for executing one or more instructions stored in the memory; The at least one processor 170 executes the one or more instructions, Controlling the display to display screen data received from an electronic device connected to the display device; Process the image data input through camera 130 to obtain gesture data, Acquire touch data input through touch input device 140, The above gesture data and the above touch data are processed into one HID (Human Interface Device) signal, A display device that controls the transmission / reception interface to transmit the HID signal to the electronic device.

2. In paragraph 1, The at least one processor 170 executes the one or more instructions, A display device that obtains the HID signal by processing the touch data and the gesture data to generate USB HID data corresponding to a touch input operation for controlling the screen data.

3. In any one of paragraphs 1 and 2, The at least one processor 170 executes the one or more instructions, Based on the above gesture data, identify whether the recognized gesture corresponds to a predetermined gesture, A display device that displays a pointer at a location on the display specified by the recognized gesture, when the recognized gesture is identified as corresponding to the predetermined gesture.

4. In any one of paragraphs 1 to 3, The at least one processor 170 executes the one or more instructions, A display device in which the position of the display on which the above pointer is displayed indicates a position specified by the direction pointed by the user's finger based on the recognized gesture.

5. In any one of paragraphs 1 to 4, The at least one processor 170 executes the one or more instructions, When the recognized gesture is identified as corresponding to the predetermined gesture, USB HID data including coordinate information of the display specified by the recognized gesture is generated, A display device that receives and displays screen data on which the pointer is displayed from the electronic device in response to transmitting the USB HID data to the electronic device.

6. In any one of paragraphs 1 to 5, The at least one processor 170 executes the one or more instructions, A display device that identifies the touch input operation based on gesture data acquired by processing the image data input through the camera while the pointer is displayed.

7. In any one of paragraphs 1 to 6, The at least one processor 170 executes the one or more instructions, Recognizes a gesture based on the gesture data acquired while the above pointer is displayed, Identifying a touch input action corresponding to the recognized gesture among one or more touch input actions, A display device that generates the USB HID data corresponding to the identified touch input action.

8. In any one of paragraphs 1 to 7, The at least one processor 170 executes the one or more instructions, A display device that detects a touch input through the touch sensitive display and identifies the touch input operation based on the touch data input through the touch sensitive display.

9. In any one of paragraphs 1 to 8, The above camera is built into or connected to the display device via communication, The above touch input device is a display device built into or connected to the display device via communication.

10. A method for operating a display device 100, An action of displaying screen data received from an electronic device connected to the display device; An action to acquire gesture data by processing image data input through a camera. An action to acquire touch data input through a touch input device. An operation of processing the above gesture data and the above touch data into one HID (Human Interface Device) signal, and A method comprising the action of transmitting said HID signal to said electronic device.

11. In paragraph 10, A method further comprising an operation of acquiring the HID signal by processing the touch data and the gesture data to generate USB HID data corresponding to a touch input operation for controlling the screen data.

12. In any one of paragraphs 10 to 11, An action of identifying whether a recognized gesture corresponds to a predetermined gesture based on the above gesture data, and A method further comprising: displaying a pointer at a location on the display specified by the recognized gesture, upon identifying that the recognized gesture corresponds to the predetermined gesture.

13. In any one of paragraphs 10 to 12, A method further comprising an action of indicating a location on the display where the pointer is displayed based on the recognized gesture, the location being specified by the direction in which the user's finger is pointing.

14. In any one of paragraphs 10 to 13, An operation of generating USB HID data including coordinate information of the display specified by the recognized gesture, upon identifying that the recognized gesture corresponds to the predetermined gesture; and A method further comprising an action of receiving and displaying screen data on which the pointer is displayed from the electronic device in response to transmitting the USB HID data to the electronic device.

15. A non-transitory computer-readable recording medium having recorded thereon one or more instructions executed by a processor of a display device, wherein when executed by the processor of the display device, the one or more instructions cause the display device to: Displaying screen data received from an electronic device connected to the display device; Processes image data input through the camera to obtain gesture data, Acquire touch data input through a touch input device, The above gesture data and the above touch data are processed into one HID (Human Interface Device) signal, A non-transitory computer-readable recording medium transmitting the HID signal to the electronic device.

Images