Electronic device and operation method of electronic device

Abstract

A display device according to an embodiment of the present disclosure may comprise: a first communication unit for communicating on the basis of a first wireless communication technology; a second communication unit for communicating on the basis of a second wireless communication technology; and a processor connected to the first communication unit and the second communication unit. The display device may be configured to: receive, through the first communication unit, first image data from a first electronic device for transmitting data received from a source device; receive a connection request from a remote control device for transmitting input information of the source device, through the second communication unit; perform pairing between the display device and the remote control device in response to receiving the connection request; and transmit information on the pairing between the display device and the remote control device to the first electronic device through the first communication unit. The first electronic device and the display device may be connected to the same network.

Description

Electronic devices and methods of operating electronic devices

[0001] Embodiments disclosed in this document relate to electronic devices and methods of operating electronic devices.

[0002] With the advancement of Internet of Things (IoT) technology, IoT devices (e.g., smart TVs, wireless video transmitters and receivers, game consoles, etc.) can be connected to each other and controlled by remote control devices (e.g., smartphones, remote controls, etc.). IoT devices can be connected via wires, but can also communicate data using short-range wireless communication technologies such as Wi-Fi and Bluetooth. As technology for wireless connection between IoT devices continues to advance, devices can now be moved to different locations according to user needs, rather than being placed in a single fixed space. Furthermore, as the types of external devices connected to display devices (e.g., TVs) diversify (e.g., set-top boxes, high-definition multimedia interface (HDMI) devices, game consoles, etc.), wireless video transmitters and receivers can be used to facilitate the connection of external devices and display devices.

[0003] If communication between a remote control device (e.g., a game pad) and a source device (e.g., a game console device) is not performed smoothly due to reasons such as the user's movement, it may be difficult to transmit information about user input generated by the remote control device to the source device.

[0004] Based on the discussion described above, the present disclosure provides a device and method for enabling data communication between a remote control device and a source device through various paths, as well as a wireless communication link formed between the remote control device and the source device.

[0005] Additionally, the present disclosure provides a device and method for adaptively selecting a path for transmitting information regarding user input generated from a remote control device according to a user's location.

[0006] A display device according to one embodiment of the present disclosure may include a first communication unit for communicating based on a first wireless communication technology, a second communication unit for communicating based on a second wireless communication technology, and a processor connected to the first communication unit and the second communication unit. The processor according to one embodiment may receive first image data from a first electronic device for transmitting data received from a source device through the first communication unit. The processor according to one embodiment may receive a connection request from a remote control device for transmitting input information of the source device through the second communication unit. The processor according to one embodiment may perform pairing of the display device and the remote control device in response to receiving the connection request. The processor according to one embodiment may transmit pairing information of the display device and the remote control device to the first electronic device through the first communication unit. In one embodiment, the first electronic device and the display device may be connected to the same network.

[0007] A method of operating a display device according to one embodiment of the present disclosure may include an operation of receiving first image data from a first electronic device for transmitting data received from a source device using a first wireless communication technology. A method of operating a display device according to one embodiment may include an operation of receiving a connection request from a remote control device for transmitting input information of the source device using a second wireless communication technology. A method of operating a display device according to one embodiment may include an operation of performing pairing of the display device and the remote control device in response to receiving the connection request. A method of operating a display device according to one embodiment may include an operation of transmitting pairing information of the display device and the remote control device to the first electronic device via the first communication unit. A method of operating a display device according to one embodiment may include connecting the first electronic device and the display device to the same network.

[0008] Embodiments of the present disclosure provide an effect that allows information regarding a user's input to be transmitted to the source device even when the connection between the remote control device and the source device is disconnected due to the user's movement or the like.

[0009] The effects that can be obtained from the present disclosure are not limited to the effects mentioned in the various embodiments, and other effects that are not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure belongs from the description below.

[0010] The foregoing and other aspects, features, and advantages of specific embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the drawings.

[0011] FIG. 1 illustrates a block configuration of an electronic device according to various embodiments.

[0012] FIG. 2 illustrates an example of an IoT environment including a display device according to various embodiments.

[0013] FIG. 3 illustrates an example of a block configuration of a display device, a source device, a remote control device, and a wireless video transmission / reception device according to various embodiments.

[0014] FIG. 4 illustrates an example of connection of a remote control device according to movement of the remote control device according to various embodiments.

[0015] Figure 5 illustrates the operation flow of a display device according to various embodiments.

[0016] FIG. 6 illustrates a signaling flow of a remote control device, a source device, a first electronic device, and a display device according to various embodiments.

[0017] In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0018] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0019] Figure 1 illustrates a block diagram of an electronic device according to various embodiments. Furthermore, the electronic device (100) may be a wearable terminal, such as a watch or glasses, capable of performing various computing functions, such as video viewing and communication. The electronic device (100) may be a terminal of various forms, including various circuit units, without limitation to the above-described elements.

[0020] According to one embodiment, the memory (120) is a storage medium used by the electronic device (100) and can store data such as at least one command (121) or setting information corresponding to at least one program. The program can include an operating system (OS) program and various application programs.

[0021] In one embodiment, the memory (120) may store pairing information of an external electronic device positioned adjacent to the electronic device (100). In one embodiment, the pairing information may include device information of the external electronic device, information about other external electronic devices or remote control devices paired with the external electronic device, information about a method by which the external electronic device and the other external electronic devices or remote control devices are paired (e.g., Bluetooth, Wi-Fi), information about a pairing history between the external electronic device and the other external electronic devices or remote control devices, etc.

[0022] In one embodiment, the memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., an SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read only memory (ROM), an electrically erasable programmable ROM (EEPROM), a programmable ROM (PROM), a magnetic memory, a magnetic disk, and an optical disk.

[0023] According to one embodiment, the video input unit (130) including various circuit units can receive video and video information through a tuner, an input / output unit, or a communication unit (150) including various circuit units. In one embodiment, the communication unit (150) can be implemented as a processor in hardware. The video input unit (130) can include at least one of the tuner and the input / output unit. The tuner can select only the frequency of a broadcast channel to be received by the electronic device (100) among many radio wave components through amplification, mixing, resonance, etc. of a broadcast signal received wired and / or wirelessly. The broadcast signal can include video, audio, and additional data (e.g., an Electronic Program Guide (EPG)). The tuner can receive real-time broadcast channels (or real-time viewing images) from various broadcast sources such as terrestrial broadcasting, cable broadcasting, satellite broadcasting, and Internet broadcasting. The above tuner may be implemented as an integral part of the electronic device (100) or as a separate tuner electrically connected to the electronic device (100). The input / output unit may include at least one of an HDMI (High Definition Multimedia Interface) input port, a component input jack, a PC input port, and a USB input jack, which can receive images and image information from an external device of the electronic device (100) under the control of the processor (110). It is obvious to those skilled in the art that the input / output unit may be added, deleted, and / or changed depending on the performance and structure of the electronic device (100).

[0024] According to one embodiment, the display (140) may perform functions for outputting information in the form of numbers, characters, images, and / or graphics. The display (140) may include at least one hardware module for outputting. The at least one hardware module may include, for example, at least one of a Liquid Crystal Display (LCD), a Light Emitting Diode (LED), a Light Emitting Polymer Display (LPD), an Organic Light Emitting Diode (OLED), an Active Matrix Organic Light Emitting Diode (AMOLED), or a Flexible LED (FLED). The display (140) may display a screen corresponding to data received from the processor (110). The display (140) may be referred to as an “output unit,” a “display unit,” or other terms having equivalent technical meanings thereto. The “screen” may include an image displayed on the display of an electronic device. The image may be referred to by terms such as a frame. Various types of objects such as icons, text, photos, videos, widgets, etc. may be displayed on the screen.

[0025] According to one embodiment, the communication unit (150) may provide a wired / wireless communication interface that enables communication with an external device. The communication unit (150) may include at least one of a wired Ethernet, a wireless LAN communication unit, and a short-range communication unit. The wireless LAN communication unit may include, for example, Wi-Fi, and may support the wireless LAN standard (IEEE802.11x) of the Institute of Electrical and Electronics Engineers (IEEE). The wireless LAN communication unit may be wirelessly connected to an AP (Access Point) under the control of the processor (110). The AP may include a device that enables devices to be connected using a related standard using Wi-Fi in a computer network. The short-range communication unit may wirelessly perform short-range communication with an external device under the control of the processor (110). Short-range communication may include Bluetooth, Bluetooth Low Energy, Infrared Data Association (IrDA), Ultra Wide Band (UWB), and Near Field Communication (NFC). The above external devices may include server devices that provide video services, etc., and mobile terminals (e.g., phones, tablets, etc.).

[0026] According to one embodiment, a processor (110) including various circuit units may execute at least one instruction (121) stored in a memory (120) to perform calculations or data processing related to control and / or communication of at least one other component of an electronic device (100). The processor (110) may include at least one of a central processing unit (CPU), a graphics processing unit (GPU), a micro controller unit (MCU), a sensor hub, a supplementary processor, a communication processor, an application processor, an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA), and may have multiple cores.

[0027] In one embodiment, the processor (110) may control at least one other component (e.g., a hardware or software component) of the electronic device (100) connected to the processor (110), for example, by executing software, and may perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculation, the processor (110) may store commands or data received from other components in a volatile memory, process the commands or data stored in the volatile memory, and store result data in a non-volatile memory. According to one embodiment, the processor (110) may include a main processor (e.g., a central processing unit or an application processor) or an auxiliary processor (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together therewith. For example, when the electronic device (100) includes a main processor and an auxiliary processor, the auxiliary processor may be configured to use lower power than the main processor or to be specialized for a given function. The auxiliary processor may be implemented separately from the main processor or as part of it.

[0028] In one embodiment, the processor (110) may obtain image frame data from at least one of the memory (120), the image input unit (130), or the communication unit (150). The processor (110) may receive the image frame data from at least one of the memory (120), the image input unit (130), or the communication unit (150). The image frame data may include data regarding frames constituting an image. For example, the image frame data may be identified from the memory (120) (e.g., a recorded and stored image). For example, the image frame data may include data obtained from the communication unit (150) or the image input unit (130) (e.g., a real-time streaming image).

[0029] FIG. 2 illustrates an example of an IoT environment including a display device according to various embodiments. The display device (210) of FIG. 2 may be a device corresponding to the electronic device (100) of FIG. 1.

[0030] In one embodiment, the IoT environment (200) may include a display device (210), a remote control device (220), a wireless video transmission / reception device (230), and a source device (240). (Here, each of the devices may include various circuitry.)

[0031] In one embodiment, the display device (210) may include a device corresponding to the electronic device (100) of FIG. 1.

[0032] In one embodiment, the display device (210) can display a screen based on data received from a source device (240) via a wireless video transmission / reception device (230). The display device (210) can display data regarding game graphics received from the source device (240). A user can control a remote control device (220) based on the screen displayed on the display device (210).

[0033] In one embodiment, the remote control device (220) may include a device for remotely controlling the display device (210) and the source device (240). For example, the remote control device (220) may include a remote control or a game pad device.

[0034] In one embodiment, the remote control device (220) may include a plurality of physical objects (e.g., buttons, a touchpad, a joystick, etc.) for receiving user input. For example, the remote control device (220) may include objects for controlling screens displayed on the source device (240) and the display device (210) (e.g., power on, off, object selection, volume control), and objects for controlling game graphic screens (e.g., direction control, icon selection, etc.).

[0035] In one embodiment, a control signal generated based on a user input inputted into a remote control device (220) may be transmitted to a game source device. The remote control device (220) may transmit the control signal to the source device (240) via a wireless communication link (203) with the source device (240).

[0036] In one embodiment, a control signal generated based on a user input inputted into a remote control device (220) may be transmitted to a display device (210). The remote control device (220) may transmit the control signal to the display device (210) via a wireless communication link (202) with the display device.

[0037] In one embodiment, the wireless video transmission / reception device (230) includes a transceiver circuit and can connect a source device (240) and a display device (210). Referring to FIG. 2, the wireless video transmission / reception device (230) can be connected to the source device (240) by wire (205) or wirelessly (204). For example, the wireless video transmission / reception device (230) can be connected to the source device (240) via an HDMI (high definition multimedia interface) connection. For example, the wireless video transmission / reception device (230) can be connected wirelessly (201) to the display device (210).

[0038] In one embodiment, the wireless video transmission / reception device (230) may be connected to a device (e.g., a set-top box, an HDMI device) for transmitting other input sources in addition to a source device (240) such as a game console. The wireless video transmission / reception device (230) may simplify the connection between various media source devices and display devices that may be included in the IoT environment (200).

[0039] In one embodiment, the source device (240) may include a device for executing game software and processing game data.

[0040] In one embodiment, the source device (240) includes communication and processing circuitry and can receive control signals from the remote control device (220). The source device (240) can identify a user's command based on the control signal received from the remote control device (220) and control interactions within the game. For example, the user can operate the remote control device (220) to move a character within the game screen or perform a task within the game. The control signal generated by the user's operation can be transmitted to the source device (240), and the source device (240) can transmit data regarding a screen to be displayed in the game based on the received control signal to the display device (210) via the wireless video transmission / reception device (230).

[0041] FIG. 3 illustrates an example of a block configuration of a display device, a source device, a remote control device, and a wireless video transmission / reception device according to various embodiments. The display device (310) of FIG. 3 may include a device corresponding to the display device (210) of FIG. 2. The wireless video transmission / reception device (330) of FIG. 3 may include a device corresponding to the wireless video transmission / reception device (230) of FIG. 2. The remote control device (320) of FIG. 3 may include a device corresponding to the remote control device (220) of FIG. 2. The source device (340) of FIG. 3 may include a device corresponding to the source device (240) of FIG. 2.

[0042] According to one embodiment, the display device (310) may include a first communication unit (316), a second communication unit (312), a control unit (314), and a display unit (318).

[0043] In one embodiment, the first communication unit (310) may communicate with the first communication unit (336) of the wireless video transmission / reception device (330) using a first short-range wireless communication technology. The first short-range wireless communication technology may include WiFi and WiFi Direct.

[0044] In one embodiment, the second communication unit (312) may communicate with the second communication unit (332) of the wireless video transmission / reception device (330) using a second short-range wireless communication technology. The second short-range wireless communication technology may include Bluetooth.

[0045] In one embodiment, the control unit (314) can control the first communication unit (316), the second communication unit (312), and the display unit (318).

[0046] In one embodiment, the display unit (318) can display the screen of the display device (310).

[0047] According to one embodiment, the remote control device (320) may include a communication unit (322) and an input signal processing unit (324).

[0048] In one embodiment, the communication unit (322) may communicate with the first communication unit (316) of the display device (310) and the communication unit (344) of the source device (340) using a first short-range wireless communication technology.

[0049] In one embodiment, the communication unit (322) can transmit a control signal generated from the remote control device (320) to the source device (340) and the display device (310).

[0050] In one embodiment, the input signal processing unit (324) can generate a control signal based on a user input input to the remote control device (320).

[0051] According to one embodiment, the wireless video transmission and reception device (330) may include a second communication unit (332), a control unit (334), a first communication unit (336), and an HDMI connection unit (338).

[0052] In one embodiment, the second communication unit (332) can communicate with the second communication unit (312) of the display device (310). The second communication unit (332) can perform communication using a second short-range wireless communication technology.

[0053] In one embodiment, the first communication unit (336) can communicate with the communication unit (344) of the source device. The first communication unit (336) can perform communication using a first short-range wireless communication technology.

[0054] In one embodiment, the control unit (334) can control the second communication unit (332) and the first communication unit (336).

[0055] In one embodiment, the wireless video transmitter / receiver device (330) may be connected to an HDMI connector (346) of a source device (340) via an HDMI connector (338).

[0056] According to one embodiment, the source device (340) may include a control unit (342), a communication unit (344), and an HDMI connection unit (346).

[0057] In one embodiment, the communication unit (344) can perform data communication with the first communication unit (336) of the wireless video transmission / reception device (330) and the communication unit (322) of the remote control device (320).

[0058] In one embodiment, the communication unit (344) can transmit and receive data with the wireless video transmission / reception device (330) and the remote control device (320) using the first short-range wireless communication technology.

[0059] In one embodiment, the control unit (342) can control the communication unit (344).

[0060] In one embodiment, it can be connected to the HDMI connection (338) of the wireless video transmission / reception device (330) via the HDMI connection (346).

[0061] In one embodiment, the connection between the HDMI connection (346) and the HDMI connection (338) may include the wired connection of FIG. 2.

[0062] In one embodiment, the connection of the communication unit (344) and the first communication unit (336), i.e., the wireless communication link formed through the communication unit (344) and the first communication unit (336), may include the wireless connection (204) of FIG. 2.

[0063] In one embodiment, the connection of the second communication unit (332) and the second communication unit (312), i.e., the wireless communication link formed through the second communication unit (332) and the second communication unit (312), may include the wireless connection (201) of FIG. 2.

[0064] In one embodiment, the connection of the communication unit (344) and the communication unit (322), i.e., the wireless communication link formed through the communication unit (344) and the communication unit (322), may include the wireless connection (203) of FIG. 2.

[0065] In one embodiment, the connection of the communication unit (322) and the first communication unit (316), i.e., the wireless communication link formed through the communication unit (322) and the first communication unit (316), may include the wireless connection (204) of FIG. 2.

[0066] FIG. 4 illustrates an example of connection (400) of a remote control device according to movement of the remote control device according to various embodiments.

[0067] Referring to FIG. 4, the remote control device (320) is a device operated by a user, and the position of the remote control device (320) can also change according to the user's movement.

[0068] In one embodiment, the first state (401) may include a state in which a remote control device (320) and a source device (340) are connected, and the remote control device (320) transmits a control signal to the source device (340).

[0069] In one embodiment, a wireless communication link may be established between the remote control device (320) and the source device (340). For example, the remote control device (320) and the source device (340) may be Bluetooth paired.

[0070] In one embodiment, the source device (340) includes communication and processing circuitry and can generate screen data including information about a screen to be displayed on the display device (310) based on a control signal received from a remote control device (320).

[0071] In one embodiment, the source device (340) can transmit the generated screen data to the display device (310) via the wireless video transmission / reception device (330).

[0072] In one embodiment, the second state (402) may include a state in which the remote control device (320) is disconnected from the source device (340). The connection between the remote control device (320) and the source device (340) may be disconnected due to movement of the user using the remote control device (320) (e.g., moving into a room containing the display device (310)), an obstacle (e.g., a wall, another user, or an object) between the source device (340) and the remote control device (320). In the second state (402), the source device (340) may no longer be able to receive control signals from the remote control device (320).

[0073] In one embodiment, the third state (403) may include a state in which a wireless communication link is established between the remote control device (320) and the display device (310). For example, if the connection between the remote control device (320) and the source device (340) is broken due to an obstacle, the remote control device (320) may transmit a control signal to the source device (340) via the display device (310) and the wireless video transmission / reception device (330).

[0074] In one embodiment, the remote control device (320) may, in response to identifying (or recognizing) that the remote control device (320) is disconnected (e.g., disconnected) from the source device (340), search for other connectable devices. As a result of searching for other connectable devices, the remote control device (320) may establish a connection with the display device (310). For example, the remote control device (320) may perform pairing with the display device (310).

[0075] FIG. 5 illustrates an operation flow (500) of a display device according to various embodiments. The display device of FIG. 5 may include a device corresponding to the electronic device (100) of FIG. 1, the display device (210) of FIG. 2, and the display device (310) of FIGS. 3 and 4. The source device of FIG. 5 may include a device corresponding to the source device (240) of FIG. 2 and the source device (340) of FIGS. 3 and 4. The remote control device of FIG. 5 may include a device corresponding to the remote control device (220) of FIG. 2 and the remote control device (320) of FIGS. 3 and 4. The first electronic device of FIG. 5 may include a device corresponding to the wireless video transmission / reception device (230) of FIG. 2 and the wireless video transmission / reception device (330) of FIGS. 3 and 4. The operation of FIG. 5 may include all of the operation contents of the display device according to the first state, the second state, and the third state of FIG. 4.

[0076] According to one embodiment, in operation 510, the display device may receive first image data from a first electronic device for transmitting data received from a source device through a first communication unit.

[0077] In one embodiment, the first communication unit may communicate data with the source device using a first short-range wireless communication technology (e.g., Wi-Fi).

[0078] In one embodiment, a first electronic device for transmitting data received from a source device may include a wireless video transceiver. In one embodiment, the first electronic device may be connected to other devices (e.g., a set-top box, a PC, etc.) that can provide input sources to the display device in addition to the source device.

[0079] In one embodiment, the first electronic device may be connected to the source device via a wired connection. The first electronic device may be connected to the source device via an HDMI cable. For example, the first electronic device may be connected to the source device via HDMI CEC.

[0080] In one embodiment, the first electronic device may establish a wireless communication link with the source device. The first electronic device may establish the wireless communication link with the source device using a second short-range wireless communication technology. For example, the first electronic device may pair with the source device via Bluetooth.

[0081] In one embodiment, the first image data may include audio / video data generated from a source device. The first image data may include data about software (e.g., a game) running on the source device. The first image data may include data generated based on user input identified on the remote control device. The first image data may include both video data and audio data.

[0082] In one embodiment, a source device can receive data from a server device.

[0083] According to one embodiment, in operation 520, the display device may receive a connection request from a remote control device for transmitting input information of the source device through the second communication unit.

[0084] In one embodiment, the remote control device may include a device for transmitting information regarding user input to a source device. The remote control device may include a remote control, a smartphone, or a gamepad device.

[0085] In one embodiment, the remote control device may include objects for identifying user input. For example, the remote control device may include physical buttons and operating elements. For example, the remote control device may include a display. The remote control device may receive user touch input through the display.

[0086] In one embodiment, the second communication unit may communicate data with the remote control device using a second short-range wireless communication technology (e.g., Bluetooth).

[0087] In one embodiment, when the remote control device is connected to a source device and is communicating data with the source device, and the connection between the remote control device and the source device is disconnected (e.g., when a user using the remote control device moves a certain distance from the source device and the pairing is broken), the remote control device may transmit a connection request to another IoT device (e.g., a display device) nearby. The remote control device may transmit information regarding a user input input into the remote control device to the source device. If the remote control device is connected to the source device, the remote control device may transmit information regarding the user input directly to the source device. However, if the user of the remote control device moves away from the source device, direct communication between the remote control device and the source device may become impossible. In this case, the remote control device may not transmit information regarding the user input directly to the source device, but may transmit information regarding the user input to the source device through a display device (e.g., a TV) and a first electronic device (e.g., a wireless video transmission / reception device) located nearby. To this end, the remote control device can, in response to identifying that the remote control device and the source device are disconnected, search for display devices or wireless video transmitters and receivers present in the vicinity and request a wireless connection to a connectable display device or wireless video transmitter and receiver.

[0088] In one embodiment, the display device can monitor connection requests received from adjacent devices. If a connection request is identified during the monitoring process, Bluetooth pairing can be performed with the device that sent the connection request.

[0089] In one embodiment, the display device may broadcast an advertising packet at predetermined intervals. Upon receiving the advertising packet from the display device, the remote control device may transmit a connection request to the display device in response to identifying that the remote control device has been disconnected from the source device.

[0090] In one embodiment, the connection request may include medium access control (MAC) information of the remote control device, identification information of the remote control device, and information for a Bluetooth connection of the remote control device.

[0091] According to one embodiment, in operation 530, the display device may perform pairing with the remote control device in response to receiving a connection request.

[0092] In one embodiment, the display device can perform Bluetooth pairing with a remote control device.

[0093] In one embodiment, the display device and the remote control device may perform an authentication procedure.

[0094] In one embodiment, if the display device and the remote control device have a history of previous connection, the authentication procedure may not be performed.

[0095] In one embodiment, the display device may establish a wireless communication link for data communication between the display device and the remote control device as a result of pairing with the remote control device. The display device may receive information regarding user input and control signals from the remote control device via the wireless communication link.

[0096] According to one embodiment, in operation 540, the display device can transmit pairing information between the display device and the remote control device to the first electronic device through the first communication unit.

[0097] In one embodiment, the display device may transmit MAC information and identification information of the remote control device identified during the pairing process with the remote control device performed in operation 530 to the first electronic device.

[0098] In one embodiment, the first electronic device may transmit pairing information of the received remote control device to the source device. The source device may determine whether to process data based on information regarding user input of the remote control device received from the first electronic device, based on the pairing information of the remote control device received from the first electronic device.

[0099] In one embodiment, the first electronic device and the display device may be connected to the same network. In one embodiment, the display device may receive data regarding user input from the remote control device via the second communication unit. In one embodiment, the display device may transmit data regarding the user input to the first electronic device via the first communication unit.

[0100] In one embodiment, data regarding the user input can be transmitted via a back channel of the display device and the first electronic device.

[0101] In one embodiment, the first electronic device may transmit a connection request to the source device. The first electronic device may request a Bluetooth connection to the source device.

[0102] In one embodiment, if the connection between the first electronic device and the source device is successful, the first electronic device can communicate data with the source device via a wireless communication link.

[0103] In one embodiment, if the connection between the first electronic device and the source device fails, the first electronic device can communicate data with the source device via a wire (e.g., HDMI CEC).

[0104] In one embodiment, the display device may receive, through the first communication unit, second image data generated based on data regarding the user input from the first electronic device. The second image data may include image data generated based on information regarding the user input transmitted from a remote control device to the source device via the display device and the first electronic device.

[0105] In one embodiment, the display device can display the screen based on the second image data.

[0106] FIG. 6 illustrates a signaling flow (600) of a remote control device, a source device, a first electronic device, and a display device according to various embodiments.

[0107] In one embodiment, at operation 602, the remote control device (320) may transmit control data to the source device (340). The control data may include information regarding a user input identified at the remote control device (320).

[0108] According to one embodiment, in operation 604, the source device (340) may transmit image data to the first electronic device (330). The image data may include data regarding an execution screen of software (e.g., a game) running on the source device (340).

[0109] According to one embodiment, in operation 606, the first electronic device (330) may transmit image data to the display device (310).

[0110] According to one embodiment, in operation 608, the display device (310) can display a screen. The display device (310) can display the screen based on image data received from the first electronic device (330).

[0111] According to one embodiment, in operation 610, the remote control device (320) may identify that the connection between the remote control device (320) and the source device (340) is disconnected. If the remote control device (320) moves away from the source device (340) by a certain distance or more due to a user's movement or the like, or if an obstacle is created between the remote control device (320) and the source device (340), the connection between the remote control device (320) and the source device (340) may be disconnected.

[0112] According to one embodiment, at operation 612, the source device (340) may identify a disconnection with the remote control device (320). The source device (340) may also identify a disconnection with the remote control device (320).

[0113] According to one embodiment, in operation 614, the remote control device (320) includes communication and processing circuitry and can transmit a connection request to the display device (310). Since the remote control device (320) cannot directly transmit information about a user input identified in the remote control device (320) to the source device (340), the remote control device (320) can transmit the connection request to the display device (310) in order to transmit information about the user input to the display device (310) located adjacent to the remote control device (320). The operation content according to operation 614 can include all of the operation content corresponding to operation 520.

[0114] According to one embodiment, in operation 616, the remote control device (320) and the display device (310) may perform a pairing operation. In the pairing procedure, the remote control device (320) and the display device (310) may exchange information necessary for Bluetooth pairing or connection formation. For example, the remote control device (320) may transmit the MAC address of the remote control device (320), identification information of the remote control device (320), etc. to the display device (310).

[0115] According to one embodiment, in operation 618, the display device (310) may transmit pairing information to the first electronic device (330). The pairing information may include information regarding pairing of the display device (310) and the remote control device (320) (e.g., identification information of the remote control device (320), MAC address, etc.).

[0116] According to one embodiment, in operation 620, the first electronic device (330) may transmit pairing information to the source device (340).

[0117] According to one embodiment, in operation 622, the first electronic device (330) may transmit a connection request to the source device (340). The connection request may include a signal that the first electronic device (330) transmits to the source device (340) for a Bluetooth connection between the first electronic device (330) and the source device (340).

[0118] According to one embodiment, at operation 624, the source device (340) may determine a communication method with the first electronic device (330).

[0119] In one embodiment, if the connection between the source device (340) and the first electronic device (330) is successful, the source device (340) can communicate with the first electronic device (330) based on Bluetooth technology.

[0120] In one embodiment, if the connection between the source device (340) and the first electronic device (330) fails, the source device (340) can communicate with the first electronic device (330) via a wire (e.g., HDMI-CEC).

[0121] According to one embodiment, at operation 626, the remote control device (320) may transmit control data to the display device (310). The control data may include information regarding a user input identified to the remote control device (320).

[0122] According to one embodiment, at operation 628, the display device (310) may transmit a control signal to the first electronic device (330). The control signal may include information regarding a user input transmitted from the remote control device (320) to the display device (310).

[0123] According to one embodiment, in operation 630, the first electronic device (330) may transmit control data to the source device (340). The control data may include information regarding a user input transmitted by the remote control device (320) to the display device (310).

[0124] According to one embodiment, in operation 632, the source device (340) may transmit image data to the first electronic device (330). The image data may include screen display data that the source device (340) generates based on the control data received in operation 630.

[0125] According to one embodiment, in operation 634, the first electronic device (330) may transmit image data to the display device (310).

[0126] According to one embodiment, in operation 636, the display device (310) may display a screen. The display device (310) may display the screen based on the image data received in operation 634.

[0127] A display device according to one embodiment of the present disclosure may include a first communication unit for communicating based on a first wireless communication technology, a second communication unit for communicating based on a second wireless communication technology, and a processor connected to the first communication unit and the second communication unit. The processor according to one embodiment may receive first image data from a first electronic device for transmitting data received from a source device through the first communication unit. The processor according to one embodiment may receive a connection request from a remote control device for transmitting input information of the source device through the second communication unit. The processor according to one embodiment may perform pairing of the display device and the remote control device in response to receiving the connection request. The processor according to one embodiment may transmit pairing information of the display device and the remote control device to the first electronic device through the first communication unit. In one embodiment, the first electronic device and the display device may be connected to the same network.

[0128] According to one embodiment, the processor may receive data regarding user input from the remote control device via the second communication unit. According to one embodiment, the processor may transmit data regarding the user input to the first electronic device via the first communication unit.

[0129] A processor according to one embodiment may transmit data regarding the user input via a back channel of the display device and the first electronic device.

[0130] An electronic device according to one embodiment further includes a display unit connected to a processor, wherein the processor receives, through the first communication unit, second image data generated based on data regarding the user input from the first electronic device, and displays the screen based on the second image data.

[0131] In one embodiment, the first wireless communication technology may include Wi-Fi, and the second wireless communication technology may include Bluetooth.

[0132] In one embodiment, the pairing information of the display device and the remote control device may include a MAC address of the remote control device and identification information of the remote control device.

[0133] In one embodiment, the processor, in response to receiving the connection request, may display a user interface for performing pairing of the display device and the remote control device.

[0134] In one embodiment, the processor may, in response to identifying that the source device and the display device are connected, transmit information regarding the user input to the source device.

[0135] A method of operating a display device according to one embodiment of the present disclosure may include an operation of receiving first image data from a first electronic device for transmitting data received from a source device using a first wireless communication technology. A method of operating a display device according to one embodiment may include an operation of receiving a connection request from a remote control device for transmitting input information of the source device using a second wireless communication technology. A method of operating a display device according to one embodiment may include an operation of performing pairing of the display device and the remote control device in response to receiving the connection request. A method of operating a display device according to one embodiment may include an operation of transmitting pairing information of the display device and the remote control device to the first electronic device via the first communication unit. A method of operating a display device according to one embodiment may include connecting the first electronic device and the display device to the same network.

[0136] A method of operating a display device according to one embodiment may include an operation of receiving data regarding a user input from the remote control device using the second wireless communication technology. A method of operating a display device according to one embodiment may include an operation of transmitting data regarding the user input to the first electronic device using the first wireless communication technology.

[0137] A method of operating a display device according to one embodiment may include transmitting data regarding the user input through a back channel of the display device and the first electronic device.

[0138] A method of operating a display device according to one embodiment may include an operation of receiving second image data generated based on data regarding the user input from the first electronic device using the first wireless communication technology. A method of operating a display device according to one embodiment may include an operation of displaying a screen based on the second image data.

[0139] In one embodiment, the first wireless communication technology may include Wi-Fi, and the second wireless communication technology may include Bluetooth.

[0140] In one embodiment, the pairing information of the display device and the remote control device may include a MAC address of the remote control device and identification information of the remote control device.

[0141] A method of operating a display device according to one embodiment may include, in response to receiving the connection request, displaying a user interface for performing pairing of the display device and the remote control device.

[0142] A method of operating a display device according to one embodiment may include, in response to identifying that the source device and the display device are connected, transmitting information regarding the user input to the source device.

[0143] A method of operating a remote control device according to one embodiment of the present disclosure may include performing data communication with a source device using a first short-range wireless communication technology. A method of operating a remote control device according to one embodiment of the present disclosure may include transmitting a connection request to a display device in response to identifying that a connection between the remote control device and the source device is released. A method of operating a remote control device according to one embodiment of the present disclosure may include forming a wireless communication link between the remote control device and the display device using the first short-range wireless communication technology. A method of operating a remote control device according to one embodiment of the present disclosure may include transmitting information regarding a user input input to the remote control device to the display device via the wireless communication link.

[0144] In one embodiment, the first short-range wireless communication technology may include Bluetooth.

[0145] A method of operating a remote control device according to one embodiment of the present disclosure may include an operation of transmitting a connection request to the display device in response to receiving an advertising packet from the display device.

[0146] A method of operating a remote control device according to one embodiment of the present disclosure may include an operation of transmitting a connection request to the source device in response to identifying that a connection between the remote control device and the display device has been lost.

[0147] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, display devices, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0148] The various embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. For example, a component expressed in the singular should be understood to include a concept including plural components unless the context clearly indicates only the singular. It should be understood that the term "and / or" used in this document encompasses any and all possible combinations of one or more of the listed items. The terms "comprise," "have," "consist of," and the like used in this disclosure are intended to specify only the presence of a feature, component, part, or combination thereof described in this disclosure, and the use of such terms does not exclude the presence or addition of one or more other features, components, parts, or combinations thereof. In this document, phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can each include any one of the items listed together in that phrase, or all possible combinations thereof. Terms such as "first", "second", or "first" or "second" may be used merely to distinguish the corresponding element from other corresponding elements and do not limit the corresponding elements in any other respect (e.g., importance or order).

[0149] The terms "part" or "module" used in various embodiments of this document may include units implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. The "part" or "module" may be an integrally formed component or a minimum unit or part of the component that performs one or more functions. For example, according to one embodiment, the "part" or "module" may be implemented in the form of an application-specific integrated circuit (ASIC).

[0150] The term "if" as used in various embodiments of this document may be interpreted to mean "when", "when", "in response to determining", or "in response to detecting", depending on the context. Similarly, "if it is determined that" or "if ~ is detected" may be interpreted to mean "upon determining", "in response to determining", or "upon detecting", or "in response to detecting", depending on the context.

[0151] The program executed by the server device described in this document may be implemented as hardware components, software components, and / or a combination of hardware components and software components. The program may be executed by any system capable of executing computer-readable instructions.

[0152] Software may include a computer program, code, instructions, or a combination of one or more of these, which can configure a processing device to perform a desired operation or command the processing device, either independently or collectively. Software may be implemented as a computer program including instructions stored on a computer-readable storage medium. Examples of the computer-readable storage medium include magnetic storage media (e.g., read-only memory (ROM), random-access memory (RAM), floppy disks, hard disks, etc.) and optical reading media (e.g., CD-ROMs, digital versatile discs (DVDs)). The computer-readable storage medium may be distributed across network-connected computer systems so that the computer-readable code can be stored and executed in a distributed manner. Computer programs can be distributed online (e.g., by download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0153] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In the display device, A first communication unit communicating based on a first wireless communication technology; A second communication unit communicating based on a second wireless communication technology; Including a processor connected to the first communication unit and the second communication unit, The above processor: From a first electronic device for transmitting data received from a source device, first image data is received through the first communication unit, A connection request is received from a remote control device for transmitting input information of the source device through the second communication unit, In response to receiving the above connection request, pairing of the display device and the remote control device is performed, It is configured to transmit pairing information of the display device and the remote control device to the first electronic device through the first communication unit, A device wherein the first electronic device and the display device are connected to the same network.

2. In claim 1, The above processor: From the above remote control device, data regarding user input is received through the second communication unit, A device configured to transmit data regarding the user input through the first communication unit to the first electronic device.

3. In claim 2, The above processor, A device configured to transmit data regarding the user input through a back channel of the display device and the first electronic device and a back channel of the display device.

4. In claim 2, Further comprising a display unit connected to the above processor, The above processor, Through the first communication unit, second image data generated based on data regarding the user input is received from the first electronic device, A device configured to display the screen based on the second image data.

5. In claim 1, The above first wireless communication technology includes Wi-Fi, The second wireless communication technology is a device including Bluetooth.

6. In claim 3, A device wherein the pairing information includes a MAC (medium access control) address of the remote control device and identification information of the remote control device.

7. In claim 1, The above processor: A device configured to display a user interface for performing pairing of the display device and the remote control device in response to receiving the connection request.

8. In claim 1, The above processor: A device configured to transmit information regarding the user input to the source device in response to identifying that the source device and the display device are connected.

9. In the method of operating the display device, An operation of receiving first image data from a first electronic device using a first wireless communication technology for transmitting data received from a source device, An action of receiving a connection request from a remote control device for transmitting input information of the source device, using a second wireless communication technology; In response to receiving the above connection request, performing pairing of the display device and the remote control device; and Including an operation of transmitting pairing information of the display device and the remote control device to the first electronic device through the first communication unit, A method wherein the first electronic device and the display device are connected to the same network.

10. In claim 9, An action of receiving data regarding user input from said remote control device using said second wireless communication technology; A method comprising the action of transmitting data regarding the user input to the first electronic device using the first wireless communication technology.

11. In claim 10, A method comprising the operation of transmitting data regarding the user input through a back channel of the display device and the first electronic device and a back channel of the display device.

12. In claim 10, An operation of receiving second image data generated based on data regarding the user input from the first electronic device using the first wireless communication technology; A method comprising an action of displaying a screen based on the second image data.

13. In the method of operating a remote control device, An operation of performing data communication using a source device and a first short-range wireless communication technology; In response to identifying that the remote control device and the source device are disconnected, the act of transmitting a connection request to the display device; An operation of forming a wireless communication link between the remote control device and the display device using the first short-range wireless communication technology; A method comprising the action of transmitting information regarding user input entered into the remote control device via the wireless communication link to the display device.

14. In claim 13, A method wherein the first short-range wireless communication technology includes Bluetooth.

15. In claim 13, A method comprising the action of transmitting a connection request to a display device in response to receiving an advertising packet from the display device.

Images