Projector and method for transmitting touch input

Abstract

Various embodiments of the present disclosure relate to a projector and a method for transmitting a touch input to a remote display device. To this end, the projector may: determine whether the projector is communicatively connected to a display device; and, if the projector is communicatively connected to the display device, receive, from the display device, a third output screen including at least one user interface object, project an image of the third output screen, obtain a user touch input with respect to the third output screen, and transmit third control data corresponding to the user touch input to the display device.

Description

Projector and its touch input transmission method

[0001] The present disclosure relates to a method for transmitting touch input to a projector and a remote display device.

[0002] Electronic devices equipped with touchscreens allow users to control the device through an intuitive and user-friendly interface simply by touching the touchscreen. However, for large-screen display devices where users view content from a distance, it can be inconvenient for users to approach and touch the display directly. Therefore, a user interface can be provided via a remote control, or by mirroring the display device's screen onto the touchscreen-equipped electronic device.

[0003] With the widespread adoption of smart TVs and the popularization of OTT (Over The Top) services, the content and additional services provided by display devices have rapidly increased. However, it is inconvenient to search and navigate these contents using only the existing user interface via remote control or screen mirroring.

[0004] The present disclosure relates to a method for transmitting touch input to a projector and a remote display device. Specifically, various embodiments of the present disclosure relate to a touch-sensitive projector that provides a screen capable of controlling a remote display device, a display device linked to the projector, and a method thereof, and more particularly, to a projector, a display device, and a method thereof that project a graphical user interface (GUI) of a remote display device with the same aspect ratio, or project a user interface object (e.g., a keyboard, various pop-up windows, etc.) that can cover a main screen (e.g., a content screen) on the remote display device.

[0005] In order to solve the above-described technical problem, according to one embodiment of the present disclosure, a display device may include: a display; a communication interface for performing data communication with a projector; a memory for storing at least one command; and at least one processor electrically connected to the memory and for executing the at least one command. The at least one processor may determine whether a communication connection with the projector is established using the communication interface. When the communication connection with the projector is established, the at least one processor may determine whether a first output screen includes at least one user interface object, and if the first output screen includes the at least one user interface object, generate a second output screen excluding the at least one user interface object from the first output screen, and control the display to display the second output screen.

[0006] According to one embodiment, the at least one processor, when in communication connection with the projector, can control the communication interface to generate a third output screen including the at least one user interface object and transmit the third output screen to the projector.

[0007] According to one embodiment, the at least one processor may control the communication interface to transmit screen ratio information of the display device to the projector when the display device is in communication connection with the projector.

[0008] According to one embodiment, the at least one processor, when in communication connection with the projector, can control the communication interface to generate at least one identification information capable of identifying the at least one user interface object and transmit the at least one identification information to the projector.

[0009] According to one embodiment, the at least one processor, when in communication with the projector, can control the communication interface to generate control information instructing standard keyboard output and transmit the control information to the projector.

[0010] In addition, according to one embodiment of the present disclosure, a projector may include an image projection unit; a communication interface for performing data communication with a display device; a memory for storing at least one command; and at least one processor electrically connected to the memory and for executing the at least one command. The at least one processor may determine whether a communication connection with the display device is established using the communication interface. When the communication connection with the display device is established, the at least one processor may control the communication interface to receive a third output screen including at least one user interface object from the display device, control the image projection unit to project the third output screen, obtain a user touch input for the third output screen, and control the communication interface to transmit third control data corresponding to the user touch input to the display device.

[0011] According to one embodiment, the at least one processor, when in communication connection with the display device, controls the communication interface to receive screen ratio information of the display device from the display device, and controls the image projector to project the third output screen with the same screen ratio as the screen ratio information of the display device.

[0012] According to one embodiment, the at least one processor, when the display device is in communication connection with the display device, controls the communication interface to receive at least one identification information capable of identifying at least one user interface object from the display device, generate a fourth output screen including at least one user interface object corresponding to the at least one identification information, control the image projector to project the fourth output screen, acquire a user touch input for the fourth output screen, and control the communication interface to transmit fourth control data corresponding to the user touch input to the display device.

[0013] According to one embodiment, the at least one processor, when in communication connection with the display device, controls the communication interface to receive control information instructing a standard keyboard output from the display device, generate a fifth output screen including a standard keyboard corresponding to the control information, control the image projector to project the fifth output screen, obtain a user touch input for the fifth output screen, and control the communication interface to transmit fifth control data corresponding to the user touch input to the display device.

[0014] In addition, according to one embodiment of the present disclosure, a method for driving a display device may include an operation of determining whether a communication connection with a projector is established; an operation of determining whether at least one user interface object is included in a first output screen when the communication connection with the projector is established; an operation of generating a second output screen excluding the at least one user interface object from the first output screen when the at least one user interface object is included in the first output screen; and an operation of displaying the second output screen.

[0015] According to one embodiment, the method for driving the display device may further include, when the display device is in communication connection with the projector, an operation of generating a third output screen including the at least one user interface object; and an operation of transmitting the third output screen to the projector.

[0016] According to one embodiment, the method for driving the display device may further include an operation of transmitting screen ratio information of the display device to the projector when the display device is in communication connection with the projector.

[0017] According to one embodiment, the method for driving the display device may further include, when the display device is in communication connection with the projector, generating at least one piece of identification information capable of identifying the at least one user interface object; and transmitting the at least one piece of identification information to the projector.

[0018] According to one embodiment, the method for driving the display device may further include, when the display device is connected to the projector, generating control information for instructing standard keyboard output; and transmitting the control information to the projector.

[0019] In addition, according to one embodiment of the present disclosure, a method for driving a projector may include an operation of determining whether a communication connection is established with a display device; an operation of receiving a third output screen including at least one user interface object from the display device if the communication connection is established with the display device; an operation of projecting an image on the third output screen; an operation of obtaining a user touch input for the third output screen; and an operation of transmitting third control data corresponding to the user touch input to the display device.

[0020] According to one embodiment, the method for driving the projector may further include, when the display device is connected to the display device, receiving screen ratio information of the display device from the display device; and projecting an image on the third output screen with the same screen ratio as the screen ratio information of the display device.

[0021] According to one embodiment, the method for driving the projector may further include, when the display device is connected to the display device in communication with the display device, receiving at least one identification information capable of identifying at least one user interface object from the display device; generating a fourth output screen including at least one user interface object corresponding to the at least one identification information; projecting an image on the fourth output screen; acquiring a user touch input for the fourth output screen; and transmitting fourth control data corresponding to the user touch input to the display device.

[0022] According to one embodiment, the method for driving the projector may further include, when the display device is connected to the display device in communication with the display device, receiving control information instructing a standard keyboard output from the display device; generating a fifth output screen including a standard keyboard corresponding to the control information; projecting an image on the fifth output screen; acquiring a user touch input for the fifth output screen; and transmitting fifth control data corresponding to the user touch input to the display device.

[0023] In addition, according to one embodiment of the present disclosure, a computer-readable recording medium having recorded thereon a program for performing the method may be included.

[0024] According to various embodiments of the present disclosure, it is possible to provide an effect that eliminates the inconvenience felt by a user when controlling a remote display device that operates based on a complex GUI through a focus movement device such as a remote control.

[0025] Furthermore, according to various embodiments of the present disclosure, by projecting the GUI of a remote display device from a projector in a user-friendly size with the same screen ratio, a user interface of an efficient size for touch input can be provided, and the disadvantages in terms of cost efficiency that arise when directly implementing touch recognition on a display device that is trending toward large screens can be overcome. Furthermore, since the user can input by touch through a projection screen that is nearby, an easy and intuitive user interface for the remote display device can be provided, and the disadvantages in usability that arise when the user directly touches the remote display device can be overcome.

[0026] Additionally, the remote display device can improve the user experience in terms of both content immersion and user interface by outputting the main screen, such as content, excluding user interface objects (e.g., keyboard, command window, etc.), and projecting user interface objects only on the projector.

[0027] The effects that can be obtained from the exemplary embodiments of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly derived and understood by those skilled in the art to which the exemplary embodiments of the present disclosure pertain from the following description. In other words, unintended effects resulting from implementing the exemplary embodiments of the present disclosure can also be derived by those skilled in the art from the exemplary embodiments of the present disclosure.

[0028] FIG. 1 and FIG. 2 are exemplary diagrams showing a projector according to one embodiment of the present disclosure projecting the same GUI as a display device at the same screen ratio.

[0029] FIG. 3 is an exemplary diagram showing a display device according to one embodiment of the present disclosure that displays content and a projector that projects a GUI screen obtained from the display device.

[0030] FIG. 4 is an example diagram of a projector projecting a keyboard obtained from a display device according to one embodiment of the present disclosure.

[0031] FIG. 5 is an example diagram of a projector projecting a self-rendered keyboard screen according to one embodiment of the present disclosure.

[0032] FIG. 6 is an example diagram of a projector projecting a user interface object screen rendered by itself according to one embodiment of the present disclosure.

[0033] FIG. 7 illustrates a block diagram of a display device according to one embodiment of the present disclosure.

[0034] FIG. 8 illustrates a block diagram of a projector according to one embodiment of the present disclosure.

[0035] FIG. 9 is a schematic flowchart of a method for driving a display device according to one embodiment of the present disclosure.

[0036] FIG. 10 is a schematic flowchart of a projector driving method according to one embodiment of the present disclosure.

[0037] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0038] FIG. 1 and FIG. 2 are exemplary diagrams showing a projector according to one embodiment of the present disclosure projecting the same GUI as a display device. FIG. 1 and FIG. 2 are exemplary diagrams showing a projector according to one embodiment of the present disclosure projecting the GUI of a display device with the same screen ratio as the screen ratio of the display device.

[0039] Referring to FIGS. 1 and 2, the projector (100) can project the same GUI screen (120, 220) as the GUI (130, 230) of the display device (110). The projector (100) can project the GUI screen (120, 220) with the same screen ratio as the screen ratio of the display device (110). For example, referring to FIG. 1, if the screen ratio of the display device (110) is 16:9 (horizontal:vertical), the projector (100) can project the GUI screen (120) at 16:9. For example, referring to FIG. 2, if the screen ratio of the display device (110) is 9:16 (horizontal:vertical), the projector (100) can project the GUI screen (220) at 9:16. The GUI screen (120, 220) may include at least one user interface object. The at least one user interface object may include, but is not limited to, UI objects associated with the user interface, such as a related content search window, a text input window, a content icon, and various other pop-up windows.

[0040] According to one embodiment, the projector (100) can remotely control the display device (110) by obtaining user input (e.g., touch gesture input) for at least one user interface object and transmitting control information corresponding to the user input to the display device (110).

[0041] To this end, according to one embodiment, the display device (110) can determine whether it is communicatively connected with the projector (100). If it is communicatively connected with the projector (100), the display device (110) can determine whether at least one user interface object is included in the output screen. The display device (110) can generate a third output screen (120, 220) including the at least one user interface object. The display device (110) can transmit the third output screen (120, 220) to the projector (100). The display device (110) can transmit screen ratio information of the display device (110) to the projector (100).

[0042] According to one embodiment, the projector (100) can determine whether it is communicatively connected with the display device (110). If it is communicatively connected with the display device (110), the projector (100) can receive a third output screen (120, 220) including at least one user interface object from the display device (110). The projector (100) can project the third output screen (120, 220) and obtain a user touch input (e.g., a touch gesture input) for the third output screen (120, 220). The projector (100) can transmit third control data corresponding to the user touch input to the display device (110), thereby remotely controlling the display device (110).

[0043] FIG. 3 is an exemplary diagram showing a display device displaying content and a projector projecting a GUI screen obtained from the display device according to one embodiment of the present disclosure.

[0044] Referring to FIG. 3, the display device (110) can display content (330). When the projector (100) is connected to the display device (110) and the display device (110) wants to display a GUI screen (320), the display device (110) can transmit the GUI screen (320) to the projector (100), and the projector (100) can project the received GUI screen (320). For example, the GUI screen (320) may be a screen that can be displayed by overlaying the content on the display device (110), but is not limited thereto. The GUI screen (320) can include at least one user interface object. The at least one user interface object includes, but is not limited to, a UI object associated with a user interface, such as a related content search window, a text input window, a content icon, and various other pop-up windows.

[0045] According to one embodiment, the projector (100) can remotely control the display device (110) by obtaining user input (e.g., touch gesture input) for at least one user interface object and transmitting control information corresponding to the user input to the display device (110).

[0046] To this end, according to one embodiment, the display device (110) can determine whether it is communicatively connected with the projector (100). If it is communicatively connected with the projector (100), the display device (110) can determine whether at least one user interface object is included in the first output screen. If the at least one user interface object is included in the first output screen, the display device (110) can generate a second output screen (330) excluding the at least one user interface object from the first output screen. The display device (110) can display the second output screen (330). The display device (110) can generate a third output screen (320) including the at least one user interface object. The display device (110) can transmit the third output screen (320) to the projector.

[0047] According to one embodiment, the projector (100) can determine whether it is communicatively connected to the display device (110). If it is communicatively connected to the display device (110), the projector (100) can receive a third output screen (320) including at least one user interface object from the display device. The projector (100) can project the third output screen (320) and obtain a user input (e.g., a touch gesture input) for the third output screen (320). The projector (100) can transmit third control data corresponding to the user input to the display device (110), thereby remotely controlling the display device (110). For example, the third control data can include coordinate values ​​on a user input plane, a code value corresponding to a specific UI object, or a key value.

[0048] FIG. 4 is an example diagram of a projector projecting a keyboard obtained from a display device according to one embodiment of the present disclosure.

[0049] Referring to FIG. 4, the display device (110) can display content (430). When the projector (100) is connected to the display device (110) and the display device (110) wants to display a keyboard screen (420), the display device (110) can transmit the keyboard screen (420) to the projector (100), and the projector (100) can project the received keyboard screen (420). The keyboard screen (420) can include, but is not limited to, any one of a QWERTY keyboard, a DVORAK keyboard, a COLEMAK keyboard, a QWERTZ keyboard, an AZERTY keyboard, a Korean 2-set keyboard, a Korean 3-set keyboard, or a JIS (Japanese Industrial Standard) layout keyboard, or a keyboard layout generated in the display device (110) in a non-standard manner.

[0050] According to one embodiment, the projector (100) can remotely control the display device (110) by obtaining user input (e.g., touch gesture input) on the keyboard screen (420) and transmitting control information corresponding to the user input (e.g., input key code value) to the display device (110).

[0051] FIG. 5 is an example diagram of a projector projecting a self-rendered keyboard screen according to one embodiment of the present disclosure.

[0052] Referring to FIG. 5, the display device (110) can display a screen (530) requesting a key input from a user. In this example, the display device (110) displays the screen (530) requesting a key input from a user, but the display device (110) can display various screens, such as but not limited to content. When the projector (100) is communicatively connected to the display device (110) and the display device (110) wants to display a keyboard screen, the display device (110) can generate control information instructing a standard keyboard output. The control information instructing the standard keyboard output can include, but is not limited to, control information instructing any one of a QWERTY keyboard, a DVORAK keyboard, a COLEMAK keyboard, a QWERTZ keyboard, an AZERTY keyboard, a Korean 2-set keyboard, a Korean 3-set keyboard, or a JIS (Japanese Industrial Standard) layout keyboard. The display device (110) can transmit the above control information to the projector (100).

[0053] In one embodiment, the projector (100) may receive control information from the display device (110) that directs standard keyboard output. The projector (100) may render and generate a fifth output screen (520) that includes a standard keyboard corresponding to the control information. The projector (100) may project the fifth output screen.

[0054] According to one embodiment, the projector (100) can remotely control the display device (110) by obtaining user input (e.g., touch gesture input) for the fifth output screen (520) and transmitting fifth control data (e.g., input key code value) corresponding to the user input to the display device (110).

[0055] FIG. 6 is an example diagram of a projector projecting a user interface object screen rendered by itself according to one embodiment of the present disclosure.

[0056] Referring to FIG. 6, the display device (110) can display content (630). When the projector (100) is connected to the display device (110) and the display device (110) intends to display a GUI screen, the display device (110) can generate at least one piece of identification information that can identify at least one user interface object constituting the GUI screen. For example, the at least one piece of identification information can include at least one piece of identification information for each of the UI objects of the related content search window and the text input window. The display device (110) can transmit the at least one piece of identification information to the projector (100).

[0057] According to one embodiment, the projector (100) can determine whether it is communicatively connected to the display device (110). If communicatively connected to the display device (110), the projector (100) can receive from the display device (110) at least one piece of identification information that can identify at least one user interface object. The projector (100) can render and generate a fourth output screen (620) that includes at least one user interface object corresponding to the at least one piece of identification information. For example, if the at least one piece of identification information identifies each of the UI objects of the related content search window and the text input window, the projector (100) can render and generate an output screen that includes the related content search window and the text input window. The projector (100) can project the fourth output screen (620) and obtain a user input (e.g., a touch gesture input) for the fourth output screen (620). The projector (100) can transmit fourth control data corresponding to the user input to the display device (110), thereby remotely controlling the display device (110). For example, the fourth control data can include coordinate values ​​on the user input plane, a code value corresponding to a specific UI object, or a key value.

[0058] FIG. 7 illustrates a block diagram of a display device according to one embodiment of the present disclosure.

[0059] Referring to FIG. 7, a display device (700) may include a display (710) for displaying an output image, a memory (720) for storing at least one command, at least one processor (730) for executing the at least one command, an image input unit (740), and a communication interface (750) for transmitting and receiving data and signals with at least one external device. The display device (700) may include additional components in addition to the illustrated components, or may omit at least one of the illustrated components.

[0060] According to one embodiment, the display device (700) may be, but is not limited to, a smartphone, a tablet PC, a PC, a smart TV, a mobile phone, a personal digital assistant (PDA), a laptop, a media player, a micro server, a digital broadcasting terminal, a navigation device, a kiosk, a home appliance, and other mobile or non-mobile computing devices. In addition, the display device (700) may be a wearable terminal, such as a watch or glasses, that can perform various computing functions, such as video viewing and communication. The display device (700) may be a terminal in various forms without being limited to the above-described contents.

[0061] According to one embodiment, the display (710) may perform functions for outputting information in the form of numbers, characters, images, and / or graphics. The display (710) may include at least one hardware module for outputting. The at least one hardware module may include, for example, at least one of a Liquid Crystal Display (LCD), a Light Emitting Diode (LED), a Light Emitting Polymer Display (LPD), an Organic Light Emitting Diode (OLED), an Active Matrix Organic Light Emitting Diode (AMOLED), or a Flexible LED (FLED). The display (710) may display a screen corresponding to data received from the processor (730). The display (710) may be referred to as a 'display unit', an 'output unit', or other terms having equivalent technical meanings thereto.

[0062] According to one embodiment, the memory (720) is a storage medium used by the display device (700) and can store data such as at least one command or setting information corresponding to at least one program. The program can include an operating system (OS) program and various application programs.

[0063] For example, the memory (720) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., an SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read only memory (ROM), an electrically erasable programmable ROM (EEPROM), a programmable ROM (PROM), a magnetic memory, a magnetic disk, and an optical disk.

[0064] According to one embodiment, the communication interface (750) may provide a wired / wireless communication interface that enables communication with an external device. The communication interface (750) may include at least one of a wired Ethernet, a wireless LAN communication unit, and a short-range communication unit. The wireless LAN communication unit may include, for example, Wi-Fi, and may support the wireless LAN standard (IEEE802.11x) of the Institute of Electrical and Electronics Engineers (IEEE). The wireless LAN communication unit may be wirelessly connected to an AP (Access Point) under the control of the processor (730). The short-range communication unit may wirelessly perform short-range communication with an external device under the control of the processor (730). Short-range communication may include Bluetooth, Bluetooth Low Energy, Infrared Data Association (IrDA), Ultra Wide Band (UWB), WiFi Direct, and Near Field Communication (NFC). The external device may include a server device that provides video services, etc., and a mobile terminal (e.g., a smartphone, tablet, etc.). According to one embodiment, the communication interface (750) may perform data communication with the projector (800).

[0065] According to one embodiment, the video input unit (740) can receive viewing images and viewing image information through a tuner (not shown), an input / output unit (not shown), or a communication interface (750). The video input unit (740) can include at least one of the tuner and the input / output unit. The tuner can select only the frequency of a broadcast channel to be received by the display device (700) from among many radio wave components through amplification, mixing, resonance, etc. of a broadcast signal received wirelessly or wiredly. The broadcast signal can include video, audio, and additional data (e.g., an Electronic Program Guide (EPG)). The tuner can receive broadcast channels (or viewing images) from various broadcast sources such as terrestrial broadcasting, cable broadcasting, satellite broadcasting, and Internet broadcasting. The tuner can be implemented as an integrated unit with the display device (700) or as a separate tuner electrically connected to the display device (700). The input / output unit may include at least one of an HDMI (High Definition Multimedia Interface) input port, a component input jack, a PC input port, and a USB input jack, which can receive images and image information from an external device of the display device (700) under the control of the processor (730). It will be understood by those skilled in the art that the input / output unit may be added, deleted, and / or changed depending on the performance and structure of the display device (700).

[0066] According to one embodiment, the processor (730) may execute at least one instruction stored in the memory (720) to perform operations or data processing related to control and / or communication of at least one other component of the display device (700).

[0067] For example, the processor (730) may include at least one of a central processing unit (CPU), a graphics processing unit (GPU), a micro controller unit (MCU), a sensor hub, a supplementary processor, a communication processor, an application processor, an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA), and may have multiple cores.

[0068] According to one embodiment, the processor (730) may determine whether it is connected to the projector (800) using the communication interface (750). If it is connected to the projector (800), the processor (730) may determine whether at least one user interface object is included in the first output screen. The at least one user interface object may include, but is not limited to, a UI object associated with a user interface, such as a related content search window, a text input window, a content icon, and various other pop-up windows. If the at least one user interface object is included in the first output screen, the processor (730) may generate a second output screen excluding the at least one user interface object from the first output screen. The processor (730) may control the display (710) to display the second output screen. The processor (730) may generate a third output screen including the at least one user interface object. The processor (730) may control the communication interface (750) to transmit the third output screen to the projector (800).

[0069] According to one embodiment, when the processor (730) is connected to the projector (800), the processor (730) can control the communication interface (750) to transmit screen ratio information displayed on the display device (700) to the projector (800). For example, when the screen ratio information of the display device (700) is 16:9 (horizontal:vertical), the processor (730) can control the communication interface (750) to transmit screen ratio information corresponding to 16:9 to the projector (800).

[0070] According to one embodiment, when connected to a projector (800), the processor (730) may generate at least one piece of identification information that can identify at least one user interface object. For example, the at least one piece of identification information may include at least one piece of identification information for each of the UI objects of the related content search window and the text input window. The processor (730) may control the communication interface (750) to transmit the at least one piece of identification information to the projector (800).

[0071] According to one embodiment, when connected to a projector (800), the processor (730) may generate control information instructing standard keyboard output. The control information instructing standard keyboard output may include, but is not limited to, control information instructing any one of a QWERTY keyboard, a DVORAK keyboard, a COLEMAK keyboard, a QWERTZ keyboard, an AZERTY keyboard, a Korean 2-set keyboard, a Korean 3-set keyboard, or a JIS (Japanese Industrial Standard) layout keyboard. The processor (730) may control the communication interface (750) to transmit the control information to the projector (800).

[0072] FIG. 8 illustrates a block diagram of a projector according to one embodiment of the present disclosure.

[0073] Referring to FIG. 8, the projector (800) may include an image projection unit (810), a memory (820), a processor (830), a camera (840), and a communication interface (850). The projector (800) may further include at least one of a user interface (not shown) and an input / output interface (not shown). The projector (800) may include additional components in addition to the illustrated components, or may omit at least one of the illustrated components.

[0074] A projector (800) may refer to an image electronic device that projects (or projects, projects) an image. Specifically, the projector (800) may be an optical device that enlarges and projects an input image onto a screen (or wall).

[0075] The image projection unit (810) may include a light source lamp (not shown) and a lens (not shown). The light source lamp may refer to a device that outputs light. Light output from the light source lamp may pass through the lens and be projected onto a screen (or wall).

[0076] The camera (840) is a configuration for capturing a subject and generating a captured image, wherein the captured image is a concept including both a moving image and a still image. The camera (840) may include a lens and an image sensor. The types of lenses include general-purpose lenses, wide-angle lenses, zoom lenses, etc., and may be determined according to the type, characteristics, and usage environment of the projector (800). The image sensor may include a complementary metal oxide semiconductor (CMOS) and a charge-coupled device (CCD). The camera (840) may include a combination of at least one of an infrared sensor, a 3D sensor, and an ultrasonic sensor, but is not limited thereto.

[0077] The camera (840) outputs incident light as an image signal. Specifically, the camera (840) may be equipped with a lens, pixels, and an AD converter. The lens collects light from a subject to form an optical image in an imaging area, and the pixels may output an image signal in an analog form from the image input through the lens. In addition, the AD converter may convert the analog form of the image signal into a digital form and output it. In particular, the camera (840) is arranged to capture at least one direction among the front, side, and rear of the projector (800), and may capture a user present around the projector (800) to generate a captured image. There may be one camera (840) or multiple cameras. When capturing an image using multiple cameras, three-dimensional movement can be analyzed more precisely, which may be effective in analyzing a user's touch input gesture.

[0078] According to one embodiment, the camera (840) can acquire a user's touch input gesture for an image projected by the image projector (810). Specifically, the camera (840) can detect an image or video of the user's hand, and track and detect the movement of the user's hand. For example, the camera (840) can detect at least one of the following: the size of the user's hand or fingers, the position of the user's hand, the number of the user's hands, whether the user's hand is left or right, the distance between the user's right and left hands, and a gesture made by the user's body (e.g., hand).

[0079] According to one embodiment, the camera (840) can detect a user touch gesture input for an image projected on a user input plane. The camera (840) can obtain depth information of a user finger image and three-dimensional depth information of the user input plane. The camera (840) can detect a three-dimensional depth between the user finger image and the projected user input plane image. The processor (830) can detect which finger has touched which UI object or key in the projected image based on the three-dimensional depth, or can detect whether a specific coordinate on the user input plane has been touched.

[0080] The processor (830) can perform overall control operations of the projector (800). Specifically, the processor (830) has a function of controlling the overall operations of the projector (800). The processor (830) can be embedded and included within the projector (800) or can be communicatively connected to the projector (800) using a communication interface (850).

[0081] The processor (830) may be implemented as a digital signal processor (DSP), a microprocessor, or a time controller (TCON) that processes digital signals. However, the present invention is not limited thereto, and may include one or more of a central processing unit (CPU), a micro controller unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a graphics processing unit (GPU), a communication processor (CP), or an ARM processor, or may be defined by the relevant terminology. In addition, the processor (830) may be implemented as a system on chip (SoC) or large scale integration (LSI) having a processing algorithm built in, or may be implemented in the form of a field programmable gate array (FPGA). In addition, the processor (830) may perform various functions by executing computer executable instructions stored in the memory (820).

[0082] The memory (820) may be implemented as an internal memory such as a ROM (e.g., an electrically erasable programmable read-only memory (EEPROM)) or RAM included in the processor (830), or may be implemented as a separate memory from the processor (830). In this case, the memory (820) may be implemented as a memory embedded in the projector (800) or as a memory that can be detachably attached to the projector (800) depending on the purpose of data storage. For example, data for operating the projector (800) may be stored in a memory embedded in the projector (800), and data for the expansion function of the projector (800) may be stored in a memory that can be detachably attached to the projector (800).

[0083] Meanwhile, in the case of memory embedded in the projector (800), it may be implemented as at least one of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD)), and in the case of memory that can be detachably attached to the projector (800), it may be implemented in the form of a memory card (e.g., compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD), extreme digital (xD), multi-media card (MMC), etc.), external memory that can be connected to a USB port (e.g., USB memory), etc.

[0084] The user interface (not shown) may include at least one device selected from the group consisting of a button, a touchpad, a mouse, a keyboard, and a touchscreen. Here, the button may be a mechanical button, a touchpad, a wheel, or any other type of button formed on any area of ​​the front, side, or back of the main body of the projector (800).

[0085] The communication interface (850) may provide a wired / wireless communication interface that enables communication with an external device. The communication interface (850) may transmit and receive (or input / output) at least one of an audio and a video signal. The communication interface (850) may include at least one of a wired communication unit, a wireless LAN communication unit, and a short-range communication unit. The wired communication unit may be at least one interface of a High Definition Multimedia Interface (HDMI), a Mobile High-Definition Link (MHL), a Universal Serial Bus (USB), a Display Port (DP), a Thunderbolt, a Video Graphics Array (VGA) port, an RGB port, a D-subminiature (D-SUB), and a Digital Visual Interface (DVI). The wireless LAN communication unit may include, for example, Wi-Fi, and may support the wireless LAN standard (IEEE802.11x) of the Institute of Electrical and Electronics Engineers (IEEE). The wireless LAN communication unit can be wirelessly connected to an AP (Access Point) under the control of the processor (830). The short-range communication unit can wirelessly perform short-range communication with an external device under the control of the processor (830). Short-range communication may include Bluetooth, Bluetooth Low Energy, Infrared Data Association (IrDA), Ultra Wide Band (UWB), WiFi Direct, and Near Field Communication (NFC). The external device may include a display device (700) that provides images, etc. According to one embodiment, the communication interface (850) may perform data communication with the display device (700).

[0086] The projector (800) may include a speaker (not shown). The speaker (not shown) may output various audio data input through the communication interface (850) as well as various notification sounds or voice messages.

[0087] According to one embodiment, the processor (830) may determine whether it is communicatively connected to the display device (700) using the communication interface (850). If it is communicatively connected to the display device (700), the processor (830) may control the communication interface (850) to receive a third output screen including at least one user interface object from the display device (700). The processor (830) may control the image projector (810) to project the third output screen. The processor (830) may obtain a user touch input for the third output screen through the camera (840). The processor (830) may control the communication interface (850) to transmit third control data corresponding to the user touch input to the display device (700). For example, the third control data may include coordinate values ​​on a user input plane, a code value corresponding to a specific UI object, or a key value.

[0088] According to one embodiment, when the processor (830) is connected to the display device (700), the processor (830) can control the communication interface (850) to receive screen ratio information of the display device (700) from the display device (700). The processor (830) can control the image projector (810) to project the third output screen with the same screen ratio as the screen ratio information of the display device (700). For example, when screen ratio information corresponding to 9:16 (horizontal:vertical) is received from the display device (110), the processor (830) can control the image projector (810) to project the third output screen with 9:16.

[0089] According to one embodiment, when the processor (830) is connected to the display device (700), the processor (830) may control the communication interface (850) to receive from the display device (700) at least one identification information capable of identifying at least one user interface object. The processor (830) may generate a fourth output screen including at least one user interface object corresponding to the at least one identification information. For example, when the at least one identification information identifies each of the UI objects of the related content search window and the text input window, the processor (830) may render and generate an output screen including the related content search window and the text input window. The processor (830) may control the image projector (810) to project the fourth output screen. The processor (830) may acquire a user touch input for the fourth output screen through the camera (840). The processor (830) may control the communication interface (850) to transmit fourth control data corresponding to the user touch input to the display device (700). For example, the fourth control data may include coordinate values ​​on a user input plane, a code value corresponding to a specific UI object, or a key value.

[0090] According to one embodiment, when the processor (830) is connected to the display device (700), the processor (830) can control the communication interface (850) to receive control information instructing a standard keyboard output from the display device (700). The control information instructing the standard keyboard output may include, but is not limited to, control information instructing any one of a QWERTY keyboard, a DVORAK keyboard, a COLEMAK keyboard, a QWERTZ keyboard, an AZERTY keyboard, a Korean 2-set keyboard, a Korean 3-set keyboard, or a JIS (Japanese Industrial Standard) layout keyboard. The processor (830) can generate a fifth output screen including a standard keyboard corresponding to the control information. The processor (830) can control the image projector (810) to project the fifth output screen. The processor (830) can acquire a user touch input for the fifth output screen through a camera (840). The processor (830) may control the communication interface to transmit fifth control data corresponding to the user touch input to the display device (700). For example, the fifth control data may include a specific key value input by the user through touch on a projected standard keyboard screen.

[0091] FIG. 9 is a schematic flowchart of a method for driving a display device according to an embodiment of the present disclosure. The display device of FIG. 9 may correspond to the display device (700) of FIG. 7. In the operation of the display device described in FIG. 9, portions overlapping with those described in FIG. 7 may be omitted. Some of the operations illustrated in FIG. 9 may be omitted, operations not illustrated in FIG. 9 may be added, and one or more of the operations may be executed in a different order.

[0092] In operation 910 according to one embodiment, the display device (700) can determine whether it is connected to the projector (800). If it is connected to the projector (800), the process moves to operation 920, and if it is not connected to the projector, the process can end.

[0093] In operation 920 according to one embodiment, the display device (700) may determine whether at least one user interface object is included in the first output screen. If at least one user interface object is included in the first output screen, the process may proceed to operation 930, and if not, the process may end.

[0094] In operation 930 according to one embodiment, the display device (700) can generate a second output screen excluding the at least one user interface object from the first output screen.

[0095] In operation 940 according to one embodiment, the display device (700) can display the second output screen.

[0096] In operation 950 according to one embodiment, the display device (700) can generate a third output screen including at least one user interface object.

[0097] In operation 960 according to one embodiment, the display device (700) can transmit the third output screen to the projector (800).

[0098] According to one embodiment, the display device (700) can transmit screen ratio information of the display device (700) to the projector (800).

[0099] FIG. 10 is a schematic flowchart of a projector driving method according to one embodiment of the present disclosure. The projector of FIG. 10 may correspond to the projector (800) of FIG. 8. In the operation of the projector (800) described in FIG. 10, portions overlapping with those described in FIG. 8 may be omitted. Some of the operations illustrated in FIG. 10 may be omitted, operations not illustrated in FIG. 10 may be added, and one or more of the operations may be executed in a different order.

[0100] In operation 1010 according to one embodiment, the projector (800) can determine whether it is connected to the display device (700). If it is connected to the display device (700), the process can proceed to operation 1020, and if it is not connected to the display device (700), the process can end.

[0101] In operation 1020 according to one embodiment, the projector (800) may receive a third output screen including at least one user interface object from the display device (700).

[0102] In operation 1030 according to one embodiment, the projector (800) can project an image on the third output screen.

[0103] According to one embodiment, the projector (800) can receive screen ratio information of the display device (700) from the display device (700). The projector (800) can project an image on the third output screen with the same screen ratio as the screen ratio information received from the display device (700).

[0104] In operation 1040 according to one embodiment, the projector (800) can obtain a user touch input for the third output screen.

[0105] In operation 1050 according to one embodiment, the projector (800) may transmit third control data corresponding to the user touch input to the display device (700). For example, the third control data may include coordinate values ​​on the user input plane, a code value corresponding to a specific UI object, or a key value.

[0106] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, display devices, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0107] The various embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. For example, a component expressed in the singular should be understood to include a concept including plural components unless the context clearly indicates only the singular. It should be understood that the term "and / or" used in this document encompasses any and all possible combinations of one or more of the listed items. The terms "comprise," "have," "consist of," and the like used in this disclosure are intended to specify only the presence of a feature, component, part, or combination thereof described in this disclosure, and the use of such terms does not exclude the presence or addition of one or more other features, components, parts, or combinations thereof. In this document, phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can each include any one of the items listed together in that phrase, or all possible combinations thereof. Terms such as "first", "second", or "first" or "second" may be used merely to distinguish the corresponding element from other corresponding elements and do not limit the corresponding elements in any other respect (e.g., importance or order).

[0108] The terms "part" or "module" used in various embodiments of this document may include units implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. The "part" or "module" may be an integrally formed component or a minimum unit or part of the component that performs one or more functions. For example, according to one embodiment, the "part" or "module" may be implemented in the form of an application-specific integrated circuit (ASIC).

[0109] The term “if” as used in various embodiments of this document may be interpreted to mean “when”, “when”, “in response to determining”, or “in response to detecting”, depending on the context. Similarly, “if it is determined that” or “if ~ is detected” may be interpreted to mean “upon determining”, “in response to determining”, or “upon detecting”, or “in response to detecting”, depending on the context.

[0110] The program executed by the display device (700) and projector (800) described in this document may be implemented as hardware components, software components, and / or a combination of hardware components and software components. The program may be executed by any system capable of executing computer-readable instructions.

[0111] Software may include a computer program, code, instructions, or a combination of one or more of these, which can configure a processing device to perform a desired operation or command the processing device, either independently or collectively. Software may be implemented as a computer program including instructions stored on a computer-readable storage medium. Examples of the computer-readable storage medium include magnetic storage media (e.g., read-only memory (ROM), random-access memory (RAM), floppy disks, hard disks, etc.) and optical reading media (e.g., CD-ROMs, digital versatile discs (DVDs)). The computer-readable storage medium may be distributed across network-connected computer systems so that the computer-readable code can be stored and executed in a distributed manner. Computer programs can be distributed online (e.g., by download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0112] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In the display device, display; A communication interface that performs data communication with the projector; a memory storing at least one instruction; and At least one processor electrically connected to said memory and configured to execute said at least one instruction; At least one processor of the above, Using the above communication interface, determine whether communication is established with the projector, When connected to the above projector: Determine whether the first output screen contains at least one user interface object, If the first output screen includes at least one user interface object, a second output screen is generated excluding the at least one user interface object from the first output screen, A display device that controls the display to display the second output screen.

2. In paragraph 1, At least one processor of the above, When connected to the above projector: Generate a third output screen including at least one user interface object; A display device that controls the communication interface to transmit the third output screen to the projector.

3. In paragraph 2, At least one processor of the above, When connected to the above projector: A display device that controls the communication interface to transmit screen ratio information of the display device to the projector.

4. In paragraph 1, At least one processor of the above, When connected to the above projector: Generate at least one identifying information capable of identifying at least one user interface object; A display device that controls the communication interface to transmit at least one piece of identification information to the projector.

5. In paragraph 1, At least one processor of the above, When connected to the above projector: Generates control information that directs standard keyboard output, A display device that controls the communication interface to transmit the above control information to the projector.

6. In the projector, Image projection area; A communication interface that performs data communication with a display device; Memory that stores at least one instruction; At least one processor electrically connected to said memory and configured to execute said at least one instruction; At least one processor of the above, Using the above communication interface, determine whether communication is connected to the display device, When connected to the above display device: Controlling the communication interface to receive a third output screen including at least one user interface object from the display device; Controlling the image projector to project the third output screen; Obtain user touch input for the third output screen, A projector that controls the communication interface to transmit third control data corresponding to the user touch input to the display device.

7. In paragraph 6, At least one processor of the above, When connected to the above display device: Controlling the communication interface to receive screen ratio information of the display device from the display device, A projector that controls the image projection unit to project the third output screen with the same screen ratio as the screen ratio information of the display device.

8. In paragraph 6, At least one processor of the above, When connected to the above display device: Controlling the communication interface to receive from the display device at least one identification information capable of identifying at least one user interface object; Generate a fourth output screen including at least one user interface object corresponding to at least one of the above identification pieces of information, Controlling the image projector to project the fourth output screen; Obtain user touch input for the above fourth output screen, A projector that controls the communication interface to transmit fourth control data corresponding to the user touch input to the display device.

9. In paragraph 6, At least one processor of the above, When connected to the above display device: Controlling the communication interface to receive control information directing standard keyboard output from the display device; Generate a fifth output screen including a standard keyboard corresponding to the above control information, Controlling the image projector to project the fifth output screen; Obtain user touch input for the fifth output screen, A projector that controls the communication interface to transmit fifth control data corresponding to the user touch input to the display device.

10. In a method of driving a display device, Action to determine whether communication connection with the projector has been established; When connected to the above projector: An action to determine whether the first output screen contains at least one user interface object; An operation of generating a second output screen excluding the at least one user interface object from the first output screen, when the at least one user interface object is included in the first output screen; and A method comprising an action of displaying the second output screen.

11. In paragraph 10, When connected to the above projector: An operation of generating a third output screen including at least one user interface object; and A method further comprising the action of transmitting the third output screen to the projector.

12. In paragraph 11, When connected to the above projector: A method further comprising the action of transmitting screen ratio information of the display device to the projector.

13. In the method of operating the projector, An action to determine whether a communication connection has been established with a display device; When connected to the above display device: An operation of receiving a third output screen including at least one user interface object from the display device; An action of projecting an image onto the third output screen; An operation for obtaining user touch input for the third output screen; and A method comprising the action of transmitting third control data corresponding to the user touch input to the display device.

14. In paragraph 13, When connected to the above display device: An operation of receiving screen ratio information of the display device from the display device; and A method further comprising an action of projecting an image of the third output screen with the same screen ratio information as that of the screen ratio information of the display device.

15. A computer-readable recording medium having recorded thereon at least one program for implementing the method described in any one of claims 10 to 14.

Images