Display device and operating method thereof

Abstract

This display device comprises: one or more sensors; a communication interface; a memory for storing one or more instructions; and at least one processor for executing the one or more instructions stored in the memory, wherein the at least one processor executes the one or more instructions so that: a user event corresponding to a control command of the display device is detected through the one or more sensors; an electronic device corresponding to the user event is identified through the communication interface; the display device transmits identification information of an application running therein and an application-related information request to the electronic device; the application-related information is received from the electronic device; and an application-related operation is performed on the basis of the received application-related information.

Description

Display device and method of operation thereof

[0001] The present disclosure relates to a display device and a method of operating the display device. More specifically, the present disclosure relates to a method of controlling the display device using an electronic device.

[0002] Display devices like TVs are primarily used for viewing media. Smart TVs allow the installation of applications for viewing media. After installing applications according to the user's choice, the TV can launch them by selecting the corresponding object.

[0003] Recent display devices have introduced a feature that allows users to mirror their terminal screens by tapping the bezel. Mirroring technology allows the display device to display the image intended for the user's terminal. While tapping is an intuitive way to control a display device, its utility has been limited because it only operates with a specific function (i.e., mirroring) in certain situations.

[0004] According to one embodiment of the present disclosure, a display device includes one or more sensors, a communication interface, a memory storing one or more instructions, and at least one processor executing one or more instructions stored in the memory. The at least one processor detects a user event through the one or more sensors by executing the one or more instructions. The at least one processor identifies an electronic device corresponding to the user event through the communication interface. The at least one processor transmits identification information of an application being executed by the display device to the electronic device. The at least one processor receives application-related information from the electronic device in response to the identification information of the application. The at least one processor performs an application-related operation based on the received application-related information.

[0005] A method of operating a display device according to one embodiment of the present disclosure includes the steps of detecting a user event through one or more sensors, identifying an electronic device corresponding to the user event through a communication interface, transmitting identification information of an application being executed by the display device to the electronic device, receiving application-related information from the electronic device in response to the identification information of the application, and performing an application-related operation based on the received application-related information.

[0006] In one embodiment of the present disclosure, a computer-readable recording medium having recorded thereon a program for performing a method of operating a display device on a computer is provided.

[0007] The present disclosure can be readily understood by the combination of the following detailed description and the accompanying drawings, wherein reference numerals refer to structural elements.

[0008] FIG. 1 is a drawing for explaining a control service of a display device according to one embodiment of the present disclosure.

[0009] FIG. 2 is a flowchart for explaining the operation of a display device according to one embodiment of the present disclosure.

[0010] FIG. 3 is a flowchart illustrating an operation method between a display device and an electronic device according to one embodiment of the present disclosure.

[0011] FIG. 4 is a diagram for explaining the operation of a display device running a Home application according to one embodiment of the present disclosure.

[0012] FIG. 5 is a diagram for explaining the operation of a display device running an OOBE application according to one embodiment of the present disclosure.

[0013] FIG. 6 is a diagram illustrating the operation of a display device running a manufacturing mode application according to one embodiment of the present disclosure.

[0014] FIG. 7 is a diagram for explaining the operation of a display device running a multi-view application according to one embodiment of the present disclosure.

[0015] FIG. 8 is a flowchart illustrating an operation method between a display device and an electronic device executing a second application according to one embodiment of the present disclosure.

[0016] FIG. 9 is a diagram illustrating the operation of a display device including a plurality of sensors and executing a second application according to one embodiment of the present disclosure.

[0017] FIG. 10 is a diagram for explaining the operation of a display device executing a second application according to one embodiment of the present disclosure.

[0018] FIG. 11 is a flowchart illustrating an operation method between a display device and an electronic device that includes a plurality of sensors and is executing a first application according to one embodiment of the present disclosure.

[0019] FIG. 12 is a diagram for explaining the operation of a display device running a multi-view application according to one embodiment of the present disclosure.

[0020] FIG. 13 is a diagram for explaining the operation of a display device in response to multiple user events according to one embodiment of the present disclosure.

[0021] FIG. 14 is an example of an interface for notifying a control service of a display device according to one embodiment of the present disclosure.

[0022] FIG. 15 is a drawing for explaining the operation of a display device in an environment in which the display device executes multiple applications according to one embodiment of the present disclosure.

[0023] FIG. 16 is a schematic block diagram of a display device and an electronic device according to one embodiment of the present disclosure.

[0024] FIG. 17 is a detailed block diagram of a display device according to one embodiment of the present disclosure.

[0025] In this disclosure, the expression “at least one of a, b or c” may refer to “a”, “b”, “c”, “a and b”, “a and c”, “b and c”, “all of a, b and c”, or variations thereof.

[0026] Below, embodiments of the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein.

[0027] Although the functions mentioned in this disclosure are described using currently used general terms, they may refer to various other terms depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Therefore, the terms used in this disclosure should not be interpreted solely based on their names, but rather based on the meanings of the terms and the overall content of this disclosure.

[0028] Additionally, the terminology used in this disclosure is for the purpose of describing specific embodiments only and is not intended to limit the present disclosure.

[0029] When a part of the specification is said to "include" a component, this does not exclude other components, but rather implies the inclusion of other components, unless otherwise specifically stated. Furthermore, terms such as "part," "module," etc., used throughout the specification refer to a unit that processes at least one function or operation, which may be implemented in hardware, software, or a combination of hardware and software.

[0030] Below, with reference to the attached drawings, embodiments of the present invention are described in detail so that those skilled in the art can easily implement the present invention. However, the present invention can be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description have been omitted to clearly explain the present invention, and similar parts have been designated with similar reference numerals throughout the specification.

[0031] In this disclosure, a "processor" may include various processing circuits and / or multiple processors. For example, the term "processor" as used herein, including in the claims, may include various processing circuits, including at least one processor. One or more processors in at least one processor may be configured to perform various functions described herein, individually and / or collectively, in a distributed fashion. As used herein, "processor," "at least one processor," and "one or more processors" may be configured to perform various functions. However, these terms encompass, without limitation, situations where one processor performs some of the functions and other processor(s) perform other parts of the functions, and situations where a single processor may perform all of the functions. Furthermore, at least one processor may include a combination of processors that perform various functions of the disclosed functions in a distributed manner. At least one processor may execute program instructions to achieve or perform various functions.

[0032] In this disclosure, the term “user” means a person who controls a system, function, or operation, and may include a developer, an administrator, an installer, or a repair technician.

[0033] In the present disclosure, a “user event” may correspond to a user’s action or a user’s control command for controlling a display device using a user device.

[0034] In the present disclosure, the display device can execute various types of applications. Each application may require different information to proceed to the next step. For example, the type of application mentioned in FIG. 3 may be distinct from the type of application mentioned in FIG. 8. In this case, the application mentioned in FIG. 3 may be referred to as a first application, and the application mentioned in FIG. 8 may be referred to as a second application. For example, the first application may be an application that requires application-related information to perform the next operation, and the second application may be an application that does not require application-related information. The application-related information may be information that is preset or stored by an application installed on a user device other than the display device.

[0035] The present disclosure will be described in detail with reference to the attached drawings below.

[0036] FIG. 1 is a drawing for explaining a control service of a display device according to one embodiment of the present disclosure.

[0037] Referring to FIG. 1, a display device (100) according to one embodiment of the present disclosure may include at least one of a TV, a smartphone, a tablet PC, a digital camera, a camcorder, a laptop computer, a desktop, an e-book reader, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a navigation device, an MP3 player, a wearable device, a smart watch, a home network system, a security system, a medical device, a head-mounted display (HMD), a hemispherical display, a large display, and a projector display.

[0038] A display device (100) according to one embodiment of the present disclosure may be equipped with an operating system (OS) and an Internet connection function. In this case, the display device (100) may be referred to as a smart TV or an Internet TV. In addition to the original function of a TV, which is to output real-time broadcast programs, the display device (100) may execute various types of applications (or apps) using the operating system installed therein. Various types of applications may be installed on the display device (100) by default. Alternatively, the display device (100) may connect to the Internet, search for an application requested by the user, and install the application under the control of the user. By executing the application, the display device (100) may provide various functions or services provided by the application.

[0039] A display device (100) according to one embodiment of the present disclosure can provide a control service. The control service may be a service that connects a user's device (e.g., an electronic device (200)) to the display device (100) and allows the user to control the display device (100) using the user's device. The user may perform a user event that connects the display device (100) and the electronic device (200). For example, the user event may be, but is not limited to, an action of tapping the electronic device (200) on the display device (100).

[0040] According to one embodiment of the present disclosure, a display device (100) and an electronic device (200) can each detect a tap action and identify the device on which the tap action occurred. The display device (100) and the electronic device (200) can be connected to each other via a wireless communication interface. The display device (100) can identify an application that is currently being executed. The display device (100) can be in a state of displaying an execution screen (10) of the application. The display device (100) can identify application-related information, which is information required for the identified application to operate in the next process. The display device (100) can request application-related information from the electronic device (200). The electronic device (200) can transmit the application-related information to the display device (100). Here, the application-related information transmitted by the electronic device (200) to the display device (100) may include information that is pre-set or pre-stored before a user event occurs in the electronic device (200), and may include information necessary for the application running on the display device (100) to proceed to the next process. Upon receiving the application-related information, the display device (100) may perform an application-related operation. For example, the display device (100) may receive the application-related information and change the state or operation of the application. The display device (100) may receive the application-related information and perform an operation for the application to proceed to the next step.

[0041] A display device (100) according to one embodiment of the present disclosure can be controlled through a tap operation using an electronic device (200). For example, a user can tap the display device (100) displaying an execution screen of an application with the electronic device (200). Through the tap operation, the user can control the application being executed on the display device (100) using the same information as information previously stored in the electronic device (200). Since the user can perform the next operation without going through multiple steps (or depth) for the application being executed on the display device (100), convenience can be increased. Even without a control device (e.g., a remote controller) of the display device (100), the user can control various types of applications being executed on the display device (100) in various ways using the electronic device (200).

[0042] In other words, tap gestures are not only utilized as functions specific to specific situations of the display device (100), but can also be utilized for various functions depending on the type of application running on the display device (100). Since users can run various applications in various scenarios through simple and intuitive tap gestures, the user experience can be improved.

[0043] FIG. 2 is a flowchart for explaining an operating method of a display device according to one embodiment of the present disclosure.

[0044] Referring to FIG. 2, operations 210 to 250 may be performed by the processor of the display device (100). The operating method of the display device (100) is not limited to that illustrated in FIG. 2, and any one of the operations illustrated in FIG. 2 may be omitted, or operations not illustrated in FIG. 2 may be further included.

[0045] In operation 210, the display device (100) can detect a user event through a sensor.

[0046] In one embodiment of the present disclosure, a user event may correspond to a user's action or a user's control command for controlling a display device (100) using an electronic device (200).

[0047] In one embodiment of the present disclosure, a user event may be an action of tapping an electronic device (200) on a display device (100). By tapping the electronic device (200) on the display device (100), the user can use the electronic device (200) as a control device for controlling the display device (100). However, the tapping action is only one of several actions that can be implemented, and the user event may be applied in various ways, such as gestures, touch, and voice recognition.

[0048] In one embodiment of the present disclosure, the display device (100) may include one or more sensors. The display device (100) may determine whether a user event has occurred based on whether a sensing value received through one or more sensors is equal to or greater than a threshold value. Here, the threshold value may correspond to a value defined by a tap operation on the display device (100) using the electronic device (200).

[0049] In operation 220, the display device (100) can identify an electronic device (200) that responds to a user event through a communication interface.

[0050] In one embodiment of the present disclosure, the electronic device (200) transmits a wireless connection request, and the display device (100) can complete a wireless connection with the electronic device (200) by responding to the wireless connection request. The wireless protocol used for the wireless connection may, for example, utilize the BLE (Bluetooth Low Energy) profile. For example, the display device (100) and the electronic device (200) may each establish a communication channel using the GATT (Generic Attribute Profile). The display device (100) and the electronic device (200) may each exchange data through the established communication channel.

[0051] In one embodiment of the present disclosure, the display device (100) can scan the electronic device (200) via a communication interface. The display device (100) can receive an advertising packet including identification information of the electronic device (200) (e.g., a Media Access Control address (MAC) address) from the electronic device (200) via the communication interface. The display device (100) can store the received identification information of the electronic device (200).

[0052] In operation 230, the display device (100) can transmit identification information of the application running on the display device (100) to the electronic device (200).

[0053] In one embodiment of the present disclosure, the display device (100) can identify an application that the display device (100) is running. The display device (100) can transmit identification information of the identified application (e.g., a name or identifier of the application) to the electronic device (200) through a communication interface.

[0054] In one embodiment of the present disclosure, the display device (100) can identify application-related information, which is information necessary for an identified application to proceed to the next step. The display device (100) can transmit the application-related information along with the application's identification information to the electronic device (200).

[0055] In one embodiment of the present disclosure, the display device (100) transmits only the application's identification information to the electronic device (200), and the electronic device (200) can identify application-related information based on the application's identification information. Alternatively, the display device (100) according to one embodiment of the present disclosure can identify application-related information based on the application's identification information, and transmit the application's identification information and a request for application-related information together to the electronic device (200).

[0056] In one embodiment of the present disclosure, the display device (100) may be in a state of displaying an execution screen of an application.

[0057] In operation 240, the display device (100) can receive application-related information from the electronic device (200). The display device (100) can receive application-related information from the electronic device (200) via a communication interface.

[0058] In one embodiment of the present disclosure, application-related information may include information that is preset or pre-stored through an application installed in the electronic device (200).

[0059] In operation 250, the display device (100) can perform an application-related operation based on the received application-related information.

[0060] In one embodiment of the present disclosure, the display device (100) may change the state or operation of an application when receiving application-related information from an electronic device (200) while executing the application. Based on the application-related information, the display device (100) may perform an operation for the application to move to the next stage.

[0061] In one embodiment of the present disclosure, the display device (100) may display an execution screen corresponding to a changed state of the application based on application-related information. The display device (100) may display an execution screen corresponding to the next step of the application.

[0062] FIG. 3 is a flowchart illustrating an operation method between a display device and an electronic device according to one embodiment of the present disclosure.

[0063] Referring to FIG. 3, in operation 310, the display device (100) and the electronic device (200) can detect a user event through a sensor.

[0064] In one embodiment of the present disclosure, the display device (100) and the electronic device (200) can each detect a tap action through a sensor. For example, the sensor may include an acceleration sensor, a gyro sensor, an RGB camera, etc. However, the present invention is not limited thereto, and the display device (100) and the electronic device (200) can each detect a user event through a communication interface. For example, the communication interface may include Near Field Communication (NFC), Bluetooth Low Energy (BLE), etc.

[0065] For example, the display device (100) and the electronic device (200) can each detect shaking due to a tapping motion using an acceleration sensor. The display device (100) and the electronic device (200) can each determine whether a tapping motion has occurred based on whether a sensing value received by the acceleration sensor is greater than or equal to a threshold value. Here, the threshold value may correspond to a value defined as a tapping motion for the display device (100) using the electronic device (200).

[0066] For example, the display device (100) and the electronic device (200) can each determine whether a tap operation has occurred by utilizing a change in a wireless signal. For example, when the display device (100) and the electronic device (200) come into proximity, the strength of the wireless signal (e.g., RSSI, Received Signal Strength Indicator) according to BLE communication may increase, and when the electronic device (200) moves away, the strength of the wireless signal may decrease. Each of the display device (100) and the electronic device (200) can determine that a tap operation has occurred when the strength of the wireless signal increases or exceeds a threshold strength.

[0067] For example, the display device (100) can determine whether a tap operation has occurred using an RGB camera. The display device (100) can acquire an image of the user through the RGB camera and analyze the user's motion using motion recognition technology. The motion recognition technology may be a technology provided in the display device (100), but is not limited thereto. In this case, the electronic device (200) can detect a tap operation using a gyro sensor provided in the electronic device (200) and transmit information indicating that a tap operation has occurred to the display device (100). The electronic device (200) can improve the accuracy of motion recognition by compensating for situations in which the display device (100) may not recognize the tap operation using the motion recognition technology. The display device (100) can detect a tap operation even when the user remotely taps the electronic device (200) instead of directly tapping the display device (100) by using the RGB camera. Accordingly, the user's leanback experience can be maintained.

[0068] In operation 320, the display device (100) and the electronic device (200) can identify the device where the user event occurred by detecting the user event. The display device (100) and the electronic device (200) can be connected to each other via a wireless communication interface.

[0069] In one embodiment of the present disclosure, the display device (100) and the electronic device (200) can each identify which device a user event occurred on. For example, the display device (100) and the electronic device (200) can each use a BLE profile to identify which device a user event occurred on. For example, the display device (100) and the electronic device (200) can each establish a communication channel using the Generic Attribute Profile (GATT). The display device (100) and the electronic device (200) can each exchange data through the established communication channel.

[0070] For example, when the electronic device (200) detects a user event, the electronic device (200) can broadcast an advertising packet to the display device (100), which is a central device, via BLE to inform of its presence. When the display device (100) detects a user event, the electronic device (200) can scan peripheral devices via BLE and discover the electronic device (200). For example, the electronic device (200) can transmit an advertising packet including identification information of the electronic device (200) (e.g., a Media Access Control address (MAC address)) to the display device (100). The display device (100) can receive the advertising packet from the electronic device (200) and store identification information of the electronic device (200) (e.g., a MAC address) included in the advertising packet.

[0071] However, without limitation thereto, according to one embodiment, the display device (100) may broadcast an advertising packet including identification information of the display device (100) to announce its presence, and the electronic device (200) may scan the display device (100). In this case, the advertising packet may further include identification information of the application running on the display device (100) (e.g., the name of the application, etc.), but is not limited thereto.

[0072] However, without being limited thereto, the display device (100) and the electronic device (200) may identify each other by using a common server to which the devices are linked.

[0073] In operation 330, the display device (100) can identify the application that the display device (100) is running and the necessary application-related information.

[0074] Here, the application running on the display device (100) may include various types of applications installed by default on the display device (100), applications consisting of multiple steps such as OOBE (Out Of Box Experience) applications, third-party applications, etc. For example, the application may correspond to a software program designed to perform functions such as OOBE, network settings, user account settings, multi-view, accessibility settings, home, video player, TV viewer, volume control, settings, mobile camera, screen sharing, internet browser, video streaming, music playback, and factory mode. Meanwhile, the application mentioned in FIG. 3 may be referred to as a first application as an application that requires application-related information to perform the following operation.

[0075] Here, application-related information refers to information related to the application, such as data necessary to set, acquire, or adjust the operation or user experience of the application described above, or source information providing the data. Application-related information may refer to information necessary to change the state or operation of the application, or information necessary for the application to move to the next step or operation. Application-related information may not be stored on the display device (100) prior to a user event, but rather may be information previously stored only on the electronic device (200).

[0076] In one embodiment of the present disclosure, application-related information may include at least one of user account information, network configuration information, user configuration information, accessibility information, authentication information for entering manufacturing mode, content information, or source information providing content. Detailed examples of application-related information are described in FIGS. 4 to 7.

[0077] In one embodiment of the present disclosure, the display device (100) can obtain identification information (e.g., the name or identifier of the application) of the currently running application through a monitoring module for monitoring the currently running application. The display device (100) can obtain information required for the currently running application to move to the next stage, i.e., application-related information, through the monitoring module. The monitoring module can include one or more instructions for identifying the currently running application. In addition, the monitoring module can include one or more instructions for identifying information required for the currently running application to move to the next stage based on the identified application. The monitoring module can track data required by the application for each stage. Alternatively, for example, the monitoring module can identify characters or logos output on the currently running display screen to identify what information is currently required.

[0078] Additionally, in one embodiment of the present disclosure, the display device (100) can identify information about the current stage of the application (e.g., network setting screen) and determine necessary application-related information (e.g., network setting information) in the case of an application consisting of multiple stages, such as an OOBE application.

[0079] Alternatively, in one embodiment of the present disclosure, when the display device (100) executes a third-party application provided by a manufacturer separate from the manufacturer of the display device (100), the display device (100) may give priority to the electronic device (200) and allow the electronic device (200) to determine what content to provide to the display device (100). In this case, the display device (100) may only transmit identification information of the application and may not transmit a request for application-related information.

[0080] In one embodiment of the present disclosure, the display device (100) can display an execution screen of an application. While a user performs a tap operation on the display device (100) using the electronic device (200), the display device (100) may be in a state of displaying the execution screen of the application.

[0081] In operation 340, the display device (100) may transmit identification information of the running application and a request for application-related information to the electronic device (200). For example, the display device (100) may transmit the request for application-related information along with identification information of the running application to the electronic device (200) via a wireless protocol.

[0082] Here, the application identification information is unique information that can specify the type or function of the application, and may include, but is not limited to, the name or identifier of the application.

[0083] In one embodiment of the present disclosure, the display device (100) can identify application-related information and transmit a request for application-related information to the electronic device (200).

[0084] In operation 350, the electronic device (200) may transmit a reception response to the display device (100) upon receiving identification information of the running application and a request for application-related information from the display device (100). For example, the electronic device (200) may transmit the reception response to the display device (100) via a wireless protocol. Meanwhile, the reception response may be expressed as a signal indicating that a signal has been received, such as success or ack. In one embodiment, operation 350 may be omitted.

[0085] In operation 360, the electronic device (200) may output a user event occurrence notification. For example, the user event occurrence notification may include information indicating that the display device (100) can be controlled using the electronic device (200), information regarding subsequent operations of the display device (100) to be controlled according to the user event, etc. The user event occurrence notification may be output through an output interface such as a speaker or display.

[0086] For example, the electronic device (200) may notify the user of the occurrence of a user event upon detecting the user event through a sensor. Alternatively, for example, the electronic device (200) may notify the user of the occurrence of a user event upon receiving identification information of an application running on the display device (100) and a request for application-related information. Accordingly, the user of the electronic device (200) may confirm that he or she can control the display device (100) using the electronic device (200) by generating a user event (e.g., a tap operation). In one embodiment, operation 360 may be omitted.

[0087] In operation 370, the electronic device (200) may transmit application-related information to the display device (100).

[0088] In one embodiment of the present disclosure, upon receiving an application-related information request, the electronic device (200) can identify application-related information. The electronic device (200) can transmit the identified application-related information to the display device (100) via a wireless protocol.

[0089] In one embodiment of the present disclosure, application-related information may include information that is preset or pre-stored through an application executed on the electronic device (200). For example, application-related information may include user account information registered on the electronic device (200), network setting information, user setting information, accessibility information, authentication information, content information to be output on the display device (100), source information providing the content, and the like. This will be described in detail with reference to FIGS. 4 to 7 .

[0090] In operation 380, the display device (100) can perform an application-related operation based on application-related information received from the electronic device (200).

[0091] Here, application-related actions may indicate changing the state or operation of the application, or signal processing for the application to move to the next step or operation. In one embodiment of the present disclosure, the display device (100) may change the state or operation of the application when receiving application-related information from the electronic device (200) while executing the application. In addition, the display device (100) may perform signal processing for executing the next step of the application.

[0092] Additionally, application-related actions may include processing execution screens to indicate a changed state or operation of the application. In one embodiment of the present disclosure, the display device (100) may display an execution screen corresponding to the changed state of the application based on application-related information. Furthermore, the display device (100) may display an execution screen corresponding to the next step (or next operation) of the application.

[0093] For example, when the display device (100) is running an application related to user account settings, the display device (100) can send a user account information request to the electronic device (200) and receive the user account information from the electronic device (200). When the display device (100) receives the user account information, it can log in to the user account using the entered information and provide personalized content. This is described in FIG. 4.

[0094] In one embodiment of the present disclosure, a user can tap a display device (100) displaying an execution screen of an application with an electronic device (200). Through the tapping action, the user can control the application being executed on the display device (100) using the same information previously stored in the electronic device (200). Since the user can perform the next operation on the application being executed on the display device (100) without going through multiple steps (or depth), convenience can be increased. Even in a situation where the user does not have a separate control device (e.g., a remote control) for the display device (100), the user can control the display device (100) in various ways using the electronic device (200).

[0095] FIG. 4 is a diagram for explaining the operation of a display device running a Home application according to one embodiment of the present disclosure.

[0096] Referring to FIG. 4, it is assumed that the display device (100) executes the Home application. The display device (100) may display the execution screen (400) of the Home application. The Home application may be an application for managing and controlling the display device (100). The Home application may require user account information, etc., to provide a personalized user experience. The user account information may include an ID, password, authentication token, etc. set by the user.

[0097] When a tap action is performed between the display device (100) and the electronic device (200), the display device (100) can identify that the currently running application is a Home application. Upon identifying the Home application, the display device (100) can transmit a user account information request to the electronic device (200). The display device (100) can transmit the user account information request along with the name of the application (e.g., Home app) to the electronic device (200) via a communication interface (operation 410).

[0098] Upon receiving a request for user account information, the electronic device (200) can identify user account information registered in the cloud server. The electronic device (200) can transmit the identified user account information to the display device (100) via a communication interface (operation 420). The electronic device (200) can perform encryption on the user account information to be transmitted to the display device (100).

[0099] A user account may be an account previously registered by a user on a cloud server. For example, a user may access a cloud server via an electronic device (200) and create a user account by entering an ID and password. The electronic device (200) may transmit its identification information (e.g., a serial number or MAC address) to the cloud server. The cloud server may register the electronic device (200) by linking the identification information of the electronic device (200) to the user account. The cloud server may receive user account information and identification information of the electronic device (200) from the electronic device (200) and authenticate the information, thereby allowing the electronic device (200) to access the cloud server.

[0100] The display device (100) can receive user account information from the electronic device (200) via a communication interface. Upon receiving the user account information, the display device (100) can log in to a user account through a Single Sign-On (SSO) module or add a new user account (operation 430). For example, the display device (100) can request authentication of the user account information through the SSO module to log in to a user account. The display device (100) can perform a procedure for registering itself with a cloud server using the user account information received from the electronic device (200). The cloud server can register the display device (100) with a user account.

[0101] A user can use a user account used in the electronic device (200) on the display device (100) by tapping the display device (100) displaying the Home application with the electronic device (200). For example, the display device (100) can add a user account received from the electronic device (200) as a new account on the display device (100) or switch the login to the received user account.

[0102] FIG. 5 is a diagram for explaining the operation of a display device running an OOBE application according to one embodiment of the present disclosure.

[0103] Referring to FIG. 5, it is assumed that the display device (100) executes an OOBE application. The display device (100) can display an execution screen (500) of the OOBE application. The OOBE application is an application for managing the initial settings and initial user experience of the display device (100), and may require initial settings such as basic settings, user account settings, network connections, and accessibility settings. The OOBE application may include at least one of an application related to user account settings, an application related to network settings, and an application related to accessibility settings.

[0104] When a tap operation is performed between the display device (100) and the electronic device (200), the display device (100) can identify that the currently running application is an OOBE application. Upon identifying the OOBE application, the display device (100) can transmit a setting information request to the electronic device (200). The setting information request can include at least one of basic setting information, user account information, network setting information, or accessibility information. The display device (100) can transmit the setting information request along with the name of the application (e.g., OOBE app) to the electronic device (200) via a communication interface (operation 510).

[0105] In one embodiment, the display device (100) can identify information about the current stage of an OOBE application consisting of multiple stages. The display device (100) can determine necessary configuration information based on the information about the current stage of the OOBE application. The display device (100) can request at least one of basic configuration information, user account information, network configuration information, or accessibility information from the electronic device (200).

[0106] Upon receiving a request for configuration information, the electronic device (200) may transmit the configuration information to the display device (100) (operation 520). The basic configuration information may represent personalized configuration information for the user. The user account information represents account information registered by the user on a cloud server, as described in FIG. 5. The network configuration information may include connection information for an Access Point (AP), such as an SSID (Service Set Identifier), a password, and the like. The accessibility information may represent information regarding whether an accessibility function is activated. For example, the accessibility function may include a screen reader function that reads the content displayed on the screen aloud, a screen magnification function, a subtitle generation function, a voice recognition function, and the like. The electronic device (200) may encrypt the configuration information to be transmitted to the display device (100).

[0107] The display device (100) can receive configuration information from the electronic device (200) via a communication interface. Upon receiving the configuration information, the display device (100) can automatically perform configuration for the display device (100) (operation 530). For example, the display device (100) can use the basic configuration information to set the basic theme, language, etc. of the display device (100). The display device (100) can use the user account information to log in to a user account. The display device (100) can use the network configuration information to access an AP and connect to a network. The display device (100) can use the accessibility information to determine whether to activate an accessibility function.

[0108] A user can perform initial settings for the display device (100) in the same manner as the settings for the electronic device (200) by tapping the display device (100) displaying the OOBE application with the electronic device (200). Since the user can move on to the next step without going through multiple steps (or depths) for OOBE settings, convenience can be increased.

[0109] FIG. 6 is a diagram illustrating the operation of a display device running a manufacturing mode application according to one embodiment of the present disclosure.

[0110] Referring to FIG. 6, it is assumed that the display device (100) executes a factory mode application. The display device (100) may display an execution screen (600) of the factory mode application. The factory mode application may represent a dedicated application for a repairer to enter the factory mode to resolve a defect that occurred in the display device (100). To execute the factory mode application, authentication information that can authenticate the repairer may be required. When the display device (100) collects the repairer's authentication information, the display device (100) may enter the factory mode through the factory mode application. In the factory mode, the repairer may confirm a defect that occurred in the display device (100), collect information for resolving the defect, or resolve the defect.

[0111] When a tap operation is performed between the display device (100) and the electronic device (200), the display device (100) can identify that the currently running application is a manufacturing mode application. Upon identifying the manufacturing mode application, the display device (100) can transmit an authentication information request to the electronic device (200). The display device (100) can transmit the authentication information request along with the name of the application (e.g., Factory mode app) to the electronic device (200) via a communication interface (operation 610). Here, the electronic device (200) may be a repairman's terminal, but is not limited thereto.

[0112] Upon receiving a request for authentication information, the electronic device (200) may transmit the authentication information to the display device (100) (operation 620). The authentication information may be information stored in an enabler plugin of the electronic device (200). The authentication information may indicate credential information required to enter manufacturing mode. The electronic device (200) may encrypt the authentication information to be transmitted to the display device (100).

[0113] The display device (100) can receive authentication information from the electronic device (200) via a communication interface. Upon receiving the authentication information, the display device (100) can verify the authentication information to confirm its validity and enter manufacturing mode (operation 630). By entering manufacturing mode, the display device (100) can provide information for identifying a defect or resolving a defect. For example, a repairman can enter manufacturing mode of the display device (100) by tapping the repairman's terminal on the display device (100). The display device (100) can display information for identifying a defect in the display device (100) or resolving a defect. Accordingly, the repairman can omit the authentication procedure for identifying a defect in the display device (100) or collecting related information, thereby increasing convenience.

[0114] FIG. 7 is a diagram for explaining the operation of a display device running a multi-view application according to one embodiment of the present disclosure.

[0115] Referring to FIG. 7, it is assumed that a display device (100) executes a multiview application. The display device (100) can display an execution screen (700) of the multiview application. A multiview application can refer to an application that provides multiple contents simultaneously on a single display. A multiview application can divide the display into multiple areas and display different contents in each area. For example, the contents can include contents provided by TV channels, streaming services, images, videos, audio, video calls, cameras, and various input sources (e.g., HDMI, USB, etc.). A multiview application may require source information regarding contents to be output to each area of ​​the display device (100).

[0116] When a tap operation is performed between the display device (100) and the electronic device (200), the display device (100) can identify that the currently running application is a multiview application. Upon identifying the multiview application, the display device (100) can transmit a source information request to the electronic device (200). The display device (100) can transmit the source information request along with the name of the application (e.g., multiview app) to the electronic device (200) via a communication interface (operation 710).

[0117] Upon receiving a source information request, the electronic device (200) may transmit the source information to the display device (100) (operation 720). The display device (100) may receive the source information from the electronic device (200) via a communication interface. For example, the display device (100) may receive the source information through a method such as mirroring or casting. Mirroring and casting technologies are technologies that allow an image to be displayed on the electronic device (200) to be displayed on the display device (100). In the case of mirroring, the electronic device (200) that transmits content processes the content, and the display device (100) that receives the content may receive and output the processed content. In the case of casting, the electronic device (200) may specify the content, and the display device (100) that receives the content may directly process and output the content. For example, when an electronic device (200) requests a server to cast a video being played on the electronic device (200), the server transmits source information, such as the URL of the video, to the display device (100), and the display device (100) can access the URL and play the video.

[0118] The display device (100) can output content using source information (operation 730). The display device (100) can simultaneously output one or more contents received from the electronic device (200) to each area. A user can display the contents displayed on the electronic device (200) on the display device (100) by tapping the electronic device (200) on the display device (100).

[0119] Here, the content provided by the electronic device (200) to the display device (100) is exemplified as being video, but is not limited thereto, and the electronic device (200) may provide source information regarding TV channels, streaming services, images, videos, audio, video calls, etc. through a tap operation with the display device (100). In addition, when a tap operation is performed between the electronic device (200) and the display device (100), input sources (e.g., HDMI 1, HDMI 2) connected to the display device (100) may be switched without the electronic device (200) sending separate source information.

[0120] FIG. 8 is a flowchart illustrating an operation method between a display device and an electronic device executing a second application according to one embodiment of the present disclosure.

[0121] Referring to FIG. 8, in operation 810, the display device (100) can detect a user event through at least one sensor among a plurality of sensors.

[0122] A display device (100) according to one embodiment of the present disclosure may include a plurality of sensors. The plurality of sensors may be located at different locations of the display device (100). For example, a first sensor may be located on a first side (e.g., upper side, lower side, left side, right side, etc.) of the display device (100), and a second sensor may be located on a second side (e.g., upper side, lower side, left side, right side, etc.) of the display device (100).

[0123] A display device (100) according to one embodiment of the present disclosure can detect the location at which a user event occurred through at least one sensor among a plurality of sensors. For example, the display device (100) can receive a sensing value from at least one sensor and determine that a user event occurred at the location of at least one sensor.

[0124] Alternatively, for example, the display device (100) may receive sensing values ​​from each of a plurality of sensors, determine at least one sensor that has obtained a sensing value greater than or equal to a threshold value, and determine that a user event has occurred at a location of at least one sensor. Here, the defined value may correspond to a value defined as a tap operation on the display device (100) using the electronic device (200).

[0125] In operation 815, the electronic device (200) can detect a user event through a sensor.

[0126] In operation 820, the display device (100) and the electronic device (200) can identify the device where the user event occurred by detecting the user event. Operation 820 may correspond to operation 320 of FIG. 3.

[0127] At operation 830, the display device (100) can identify the running application. At operation 840, the display device (100) can transmit identification information of the running application to the electronic device (200).

[0128] A display device (100) according to one embodiment of the present disclosure can obtain application identification information through a monitoring module. The monitoring module may include one or more instructions for monitoring a currently running application and monitoring information required for the currently running application to move to the next stage.

[0129] In one embodiment of the present disclosure, it is assumed that the display device (100) executes an application (i.e., a second application) that can be controlled using a signal generated by the display device (100). For example, the second application may correspond to a software program designed to perform functions such as volume or channel adjustment, setting menu manipulation, etc. of the display device (100). For example, the display device (100) may generate a remote control input signal corresponding to a virtual remote control key and control the display device (100) when a user event is detected. The display device (100) may generate a remote control input signal corresponding to a virtual remote control key through software, and perform a predetermined operation by processing the remote control input signal.

[0130] In one embodiment of the present disclosure, if the display device (100) identifies that a second application is running, the display device (100) may omit the operation of requesting application-related information from the electronic device (200). For example, the display device (100) may transmit only identification information of the running application to the electronic device (200), or transmit a dummy signal (or NULL, or NONE) together with the identification information of the running application.

[0131] This is different from the embodiment of FIG. 3, which includes an operation in which the display device (100) requests application-related information from the electronic device (200) (operation 340) and performs the next step of the application using the application-related information received from the electronic device (200) (operations 370 and 380).

[0132] In operation 850, the electronic device (200) may transmit a reception response to the display device (100) upon receiving the application identification information from the display device (100). In this case, the electronic device (200) may omit the operation of transmitting application-related information. For example, the electronic device (200) may transmit only the reception response to the display device (100), or transmit a dummy signal (or NULL, or NONE) together with the reception response.

[0133] In operation 860, the electronic device (200) may output a user event occurrence notification. Operation 860 corresponds to operation 360 of FIG. 3.

[0134] In operation 870, the display device (100) may perform a predetermined application-related operation based on at least one sensor that detects a user event.

[0135] A display device (100) according to one embodiment of the present disclosure can generate an input signal based on an identified application. The display device (100) can perform a predetermined application-related operation in response to the input signal. For example, when an application for volume adjustment is running, the display device (100) can generate and process an input signal for volume adjustment, thereby adjusting the volume.

[0136] A display device (100) according to one embodiment of the present disclosure can perform different operations depending on the location where a user event occurs. For example, the display device (100) can generate different input signals depending on which of a plurality of sensors detects the user event.

[0137] Here, the different actions may be actions determined based on the sensor's position. For example, the different actions may include volume up / down, channel up / down, brightness or contrast increase / decrease in the settings menu, or up / down movement in the settings menu. The different input signals may represent remote control input signals for executing the different actions described above.

[0138] According to one embodiment of the present disclosure, a display device (100) may perform a first operation determined in response to a position of a first sensor when a first sensor detects a user event. The first operation may be an operation associated with a first side of the display device (100). For example, when a second sensor detects a user event, the display device (100) may perform a second operation determined in response to a position of the second sensor. The second operation may be an operation associated with a second side of the display device (100).

[0139] For example, if a user taps the upper side of the display device (100) using an electronic device (200) while the display device (100) is running an application that provides TV channels, the volume of the display device (100) may increase, and if the user taps the lower side, the volume may decrease.

[0140] Meanwhile, the plurality of sensors according to one embodiment of the present disclosure may be replaced with a plurality of antennas. For example, the display device (100) may include a plurality of antennas. The antenna may be a wireless communication antenna for receiving a wireless communication signal (e.g., a BLE signal or an UWB signal, etc.) transmitted from an antenna of an external device. For example, each of the plurality of antennas provided in the display device (100) may receive a wireless communication signal transmitted from the electronic device (200). The plurality of antennas may be located at different locations (e.g., upper, lower, left, right, etc.) of the display device (100). The display device (100) may identify a user event through a wireless communication signal received from at least one of the plurality of antennas. The display device (100) may perform a predetermined application-related operation in response to the location of at least one antenna that has identified the user event.

[0141] FIG. 9 is a diagram illustrating the operation of a display device including a plurality of sensors and executing a second application according to one embodiment of the present disclosure.

[0142] Referring to FIG. 9, it is assumed that the display device (100) executes a second application.

[0143] Referring to 901, the display device (100) can display an execution screen (910) of an application for menu settings. Referring to 902, the display device (100) can display an execution screen (920) of an application providing TV channels. The display device (100) can include a plurality of sensors, for example, a first sensor (930) and a second sensor (940). The first sensor (930) can be located on a first side (for example, an upper side) of the display device (100). The second sensor (940) can be located on a second side (for example, a lower side) of the display device (100).

[0144] A user can use the electronic device (200) as a replacement for a remote control key by tapping the display device (100) displaying an application with the electronic device (200). For example, in 1001, when the user taps the first side of the display device (100) using the electronic device (200) while the display device (100) is executing an application that provides a settings menu, the first sensor (930) can detect the user's tap action. The display device (100) can perform a contrast increase action based on the tap action detected by the first sensor (930). Alternatively, when the user taps the second side of the display device (100) using the electronic device (200), the second sensor (940) can detect the user's tap action. The display device (100) can perform a contrast decrease action based on the tap action detected by the second sensor (940).

[0145] For example, in 902, when the user taps the first side of the display device (100) using the electronic device (200) while the display device (100) is executing an application that provides a TV channel, the display device (100) may perform a channel up operation based on the tap operation detected by the first sensor (930). Alternatively, when the user taps the second side of the display device (100) using the electronic device (200), the display device (100) may perform a channel down operation based on the tap operation detected by the second sensor (940).

[0146] FIG. 10 is a diagram for explaining the operation of a display device executing a second application according to one embodiment of the present disclosure.

[0147] Referring to FIG. 10, an application controllable using a remote control input signal (i.e., a second application) may include an OTT application. The display device (100) may output an execution screen (1000) of the OTT application. In FIG. 10, the display device (100) may not include multiple sensors.

[0148] For example, if a user taps the display device (100) using an electronic device (200) while the display device (100) is running an OTT application, the display device (100) may perform a play / pause toggle operation. Upon detecting the tap operation, the display device (100) may generate a play / pause toggle signal. The display device (100) may switch the media running in the OTT application between a play state and a pause state through the toggle signal.

[0149] In this case, the display device (100) may not request application-related information from the electronic device (200). For example, the display device (100) may transmit only the application's identification information (e.g., OTT app) to the electronic device (200), or transmit a dummy signal (or NULL, or NONE) instead of a request for application-related information together with the application's identification information (e.g., OTT app). The electronic device (200) may transmit only a reception response to the display device (100), or transmit a dummy signal (or NULL, or NONE) together with the reception response.

[0150] FIG. 11 is a flowchart illustrating an operation method between a display device and an electronic device that includes a plurality of sensors and is executing a first application according to one embodiment of the present disclosure.

[0151] Referring to FIG. 11, in operation 1110, the display device (100) may detect a user event through at least one sensor among a plurality of sensors. In operation 1115, the electronic device (200) may detect a user event through the sensor. Operation 1110 may correspond to operation 810 of FIG. 8. Operation 1115 may correspond to operation 815 of FIG. 8.

[0152] In operation 1120, the display device (100) and the electronic device (200) can identify the device where the user event occurred by detecting the user event. Operation 1120 may correspond to operation 820 of FIG. 8.

[0153] In operation 1130, the display device (100) can identify the running application and required application-related information. The application mentioned in FIG. 11 may be referred to as a first application. Operation 1130 may correspond to operation 330 of FIG. 3.

[0154] For example, the display device (100) can execute a multi-view application, as illustrated in FIG. 12. The display device (100) can identify that the multi-view application requires source information to proceed to the next step or operation. When the display device (100) receives the source information, it can output different content corresponding to the source information to each of the plurality of areas of the display.

[0155] In operation 1140, the display device (100) may transmit identification information of the application running on the electronic device (200) and a request for application-related information. Operation 1140 may correspond to operation 340 of FIG. 3.

[0156] In operation 1150, the electronic device (200) may transmit a reception response to the display device (100) upon receiving identification information of the running application and a request for application-related information from the display device (100). Operation 1150 may correspond to operation 350 of FIG. 3.

[0157] In operation 1160, the electronic device (200) may output a user event occurrence notification. Operation 1160 may correspond to operation 360 of FIG. 3.

[0158] In operation 1170, the electronic device (200) may transmit application-related information to the display device (100). The display device (100) may receive application-related information from the electronic device (200). Operation 1170 may correspond to operation 370 of FIG. 3.

[0159] For example, the electronic device (200) can transmit source information about content output from the electronic device (200) to the display device (100), as illustrated in FIG. 12. The display device (100) can receive source information about content from the electronic device (200).

[0160] In operation 1180, the display device (100) may perform a predetermined application-related operation based on at least one sensor that detects a user event and application-related information.

[0161] A display device (100) according to one embodiment of the present disclosure can perform different actions depending on the location where a user event occurs. For example, the display device (100) can perform different actions depending on which sensor detected the user event. Here, the different actions may be actions determined in response to the location of the sensor.

[0162] For example, when the display device (100) is executing a multi-view application for displaying different content in each of a plurality of areas of the display, as illustrated in FIG. 12, the display device (100) can display content corresponding to source information in an area of ​​the display corresponding to the position of the sensor.

[0163] Meanwhile, the plurality of sensors according to one embodiment of the present disclosure may be replaced with a plurality of antennas, as described in FIG. 8.

[0164] FIG. 12 is a diagram for explaining the operation of a display device running a multi-view application according to one embodiment of the present disclosure.

[0165] Referring to FIG. 12, it is assumed that a display device (100) executes a multi-view application. The display device (100) can display a first execution screen (1210) of the multi-view application. The first execution screen (1210) can include a plurality of areas, for example, a first area (1201) and a second area (1202). Different content can be displayed in each of the plurality of areas. For example, the first area (1201) can be located on a first side (for example, the right side) of the display, and the second area (1202) can be located on a second side (for example, the left side) of the display.

[0166] The display device (100) may include a plurality of sensors, for example, a first sensor (1230) and a second sensor (1240). The first sensor (1230) may be located on a first side (e.g., a right side) of the display device (100). The second sensor (1240) may be located on a second side (e.g., a left side) of the display device (100).

[0167] The display device (100) can detect a tap operation of the electronic device (200) through the first sensor (1230) or the second sensor (1240). FIG. 12 illustrates an example in which a user taps the first side of the display device (100) using the electronic device (200). In this case, the first sensor (1230) detects the user's tap operation on the first side, and the display device (100) can identify that the user's tap operation has occurred on the first side.

[0168] When the display device (100) detects a tap operation of the electronic device (200) while executing a multi-view application, the display device (100) may transmit a source information request to the electronic device (200). In response to the source information request, the electronic device (200) may transmit source information including applications, programs, contents, sources, etc. being executed on the electronic device (200) to the display device (100). For example, the source information may include identification information of the contents being executed on the electronic device (200) and information necessary for executing the contents (for example, a URL of a web page in the case of a web browser). For example, the source information may be provided to the display device (100) in a mirroring or casting manner. Examples of source information have been described with reference to FIG. 8.

[0169] The display device (100) can control the display so that content corresponding to the source information is displayed in an area of ​​the display corresponding to the location where the tap action is identified. For example, upon receiving the source information, the display device (100) can display a second execution screen (1220) with a third area (1205) added to the first side (e.g., the right side) of the display. The third area (1205) can display content (e.g., a web page) corresponding to the source information received from the electronic device (200).

[0170] The second execution screen (1220) may include a plurality of areas, for example, a first area (1203) displaying content corresponding to the first area (1201), a second area (1204) displaying content corresponding to the second area (1202), and a third area (1205) displaying content corresponding to the source information. As the third area (1205) is added, the positions of the first area (1201) and the second area (1202) displayed in the existing first execution screen (1210) may change. For example, in the second execution screen (1220), the first area (1203) may move to the lower part of the second side of the display, and the second area (1204) may move to the upper part of the second side of the display, but is not limited thereto.

[0171] Meanwhile, unlike the illustrated embodiment, when a user taps the second side of the display device (100) using the electronic device (200), the display device (100) may identify that the user's tapping action has occurred on the second side. In this case, the display device (100) may display content corresponding to the source information in an area corresponding to the second side of the display.

[0172] FIG. 13 is a diagram for explaining the operation of a display device in response to multiple user events according to one embodiment of the present disclosure.

[0173] Referring to FIG. 13, a display device (100) according to an embodiment of the present disclosure may perform different operations depending on the number of times a user event occurs. For example, an operation performed when a user event occurs once may be different from an operation performed when the user events occur consecutively. For example, the display device (100) may detect user events consecutively. The display device (100) may perform different operations depending on the number of times the user events are detected consecutively. Here, detecting a user event consecutively may mean detecting a user event two or more times within a set period of time.

[0174] In Fig. 13, an example is provided in which a display device (100) displays an execution screen (1300) of a multi-view application. In this case, the first tap operation may be set to correspond to mirroring, and the second tap operation may be set to correspond to casting. The display device (100) may detect consecutive tap operations. When the display device (100) detects one tap operation, it may perform a mirroring operation. When the display device (100) detects two consecutive tap operations, it may perform a casting operation. The function set for each tap operation may be changed by directly modifying or specifying it by the user.

[0175] For example, when the first tap operation is detected, the display device (100) may request application-related information for mirroring from the electronic device (200). For example, the display device (100) may request content from the electronic device (200). The display device (100) may receive and output content processed in the electronic device (200) from the electronic device (200).

[0176] Alternatively, for example, if a second tap operation is detected consecutively after a first tap operation, the display device (100) may request application-related information for casting from the electronic device (200). For example, the display device (100) may request source information that provides content to the electronic device (200). The display device (100) may receive the source information from the electronic device (200), process the content, and output the content.

[0177] Additionally, although not shown, according to one embodiment of the present disclosure, the first tap action may correspond to an application-related action, and the second tap action may correspond to an action desired by the user or an application execution action. The user may pre-register a desired action or an application execution action and set the second tap action as a shortcut for that action. For example, the user may set actions such as screen share or music share as shortcuts for the second tap action.

[0178] The user can set a desired action as a shortcut key during the initial setup through the OOBE application. Alternatively, the user can set a desired action as a shortcut key by performing the initial two-tap operation, and the display device (100) can then perform the action set as a shortcut key based on two consecutive tap operations. Alternatively, if an action has already been set as a shortcut key, the user can change the desired action to a shortcut key by performing three consecutive tap operations.

[0179] FIG. 14 is an example of an interface for notifying a control service of a display device according to one embodiment of the present disclosure.

[0180] Referring to FIG. 14, a display device (100) according to an embodiment of the present disclosure may display an interface (1400) for notifying a control service. The interface (1400) may include visual information indicating that an application can be controlled based on a user event. For example, the visual information may indicate that the display device (100) can perform an application-related operation when a tap operation is performed. For example, the visual information may indicate a location where a tap is possible. For example, the visual information may be expressed to be distinguished from other areas of the display using items, colors, patterns, etc. The display device (100) may display explanatory information around the visual information to indicate which operation is performed in response to the tap operation. The explanatory information may be a tooltip.

[0181] When a display device (100) according to an embodiment of the present disclosure includes a plurality of sensors, for example, a first sensor (1430) and a second sensor (1440), the display device (100) can display different visual information (e.g., 1410, 1420) in areas corresponding to the positions of each of the plurality of sensors. Each piece of visual information can be displayed in a tappable position. For example, the display device (100) can display the first visual information (1410) in an area (e.g., the right side) corresponding to the position of the first sensor (1430). When the right side is tapped around the first visual information (1410), the display device (100) can display descriptive information (e.g., tap to cast) indicating that a casting operation is performed. Alternatively, for example, the display device (100) can display the second visual information (1420) in an area (e.g., the left side) corresponding to the position of the second sensor (1440). The display device (100) may display descriptive information (e.g., tap to mirror) indicating that a mirroring operation is performed when the left side of the second visual information (1420) is tapped.

[0182] A display device (100) according to one embodiment of the present disclosure can display visual information at the start-up stage of an application. The display device (100) can display the visual information when the application is first executed. Alternatively, the display device (100) can display the visual information when the application is first executed two or three times. However, the timing of displaying the visual information is not limited to the examples described above.

[0183] FIG. 15 is a drawing for explaining the operation of a display device in an environment in which the display device executes multiple applications according to one embodiment of the present disclosure.

[0184] Referring to FIG. 15, a display device (100) according to an embodiment of the present disclosure may provide a multi-window service. The multi-window service may be a service that displays the execution screen of each application in each area of ​​the display when multiple applications are executed simultaneously. In this case, the execution screens of each application may be displayed as overlays in the z-axis direction. For example, the display device (100) may display multiple applications in the following order: an execution screen (1510) of a first application, an execution screen (1520) of a second application, and an execution screen (1530) of a third application. The execution screen (1510) of the first application may be located in the uppermost layer in the z-axis direction.

[0185] When the display device (100) detects a user event in a multi-window environment, the application located at the topmost layer in the z-axis direction may perform an action in response to the user event. For example, the first application may perform an application-related action in response to the user event.

[0186] FIG. 16 is a schematic block diagram of a display device and an electronic device according to one embodiment of the present disclosure.

[0187] Referring to FIG. 16, the display device (100) may include a processor (110), a memory (120), a communication interface (130), a sensor (140), and a display (150). However, the display device (100) may be implemented with more or fewer components than the illustrated components, and is not limited to the examples described above.

[0188] The memory (120) can store a program for processing and controlling the processor (110), and can store data input to or output from the display device (100). The memory (120) can include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk.

[0189] In one embodiment of the present disclosure, the memory (120) may store an application for providing a certain function or service.

[0190] In one embodiment of the present disclosure, the memory (120) may store one or more instructions for executing a monitoring module. The monitoring module may include one or more instructions for identifying a currently running application and identifying information required for the currently running application to proceed to the next step.

[0191] In one embodiment of the present disclosure, the memory (120) may store one or more instructions for executing a motion recognition module. The motion recognition module may detect whether a user event has occurred in an image of the user acquired through a camera.

[0192] In one embodiment of the present disclosure, the memory (120) may store identification information of the electronic device (200) received from the electronic device (200). The memory (120) may store application-related information received from the electronic device (200). The processor (110) may store information of each of the previously connected electronic devices (e.g., identification information and application-related information of the electronic device (200)) in the memory (120). When the display device is tapped with an electronic device having a previous connection history, the processor (110) may immediately execute an application-related operation.

[0193] The processor (110) controls the overall operation of the display device (100). For example, the processor (110) may perform the functions of the display device (100) described in the present disclosure by executing one or more instructions stored in the memory (120). In this case, the memory (120) may store one or more instructions executable by the processor (110). In addition, the processor (110) may store one or more instructions in an internally provided memory, and execute one or more instructions stored in the internally provided memory to control the above-described operations to be performed. That is, the processor (110) may perform a predetermined operation by executing at least one instruction or program stored in the internal memory provided in the processor (110) or the memory (120).

[0194] The processor (110) may include one or more processors. In this case, one or more processors may be a general-purpose processor such as a CPU (Central Processing Unit), an AP (Application Processor), a DSP (Digital Signal Processor), a graphics-only processor such as a GPU (Graphics Processing Unit), a VPU (Vision Processing Unit), or an AI-only processor such as an NPU (Neural Processing Unit). If one or more processors are AI-only processors, the AI-only processors may be designed with a hardware structure specialized for processing a specific AI model.

[0195] The communication interface (130) can transmit and receive data or signals with external devices, peripheral devices, servers, electronic devices, etc. under the control of the processor (110). The communication interface (130) can include at least one antenna for wirelessly communicating with other devices. The antenna can receive a wireless communication signal transmitted from an antenna of an external device. For example, the communication interface (130) can include a Wi-Fi module, a Bluetooth communication module, a BLE (Bluetooth Low Energy) communication module, a near field communication module (NFC), an infrared (IrDA) communication module, a wireless communication module, a LAN module, an Ethernet module, a wired communication module, etc. In this case, each communication module can be implemented in the form of at least one hardware chip. The Wi-Fi module and the Bluetooth module perform communication in the Wi-Fi method and the Bluetooth method, respectively. When using a Wi-Fi module or a Bluetooth module, the processor (110) first transmits and receives various connection information such as SSID and session key, and after establishing a communication connection using this, can transmit and receive various information. The wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as Zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), 5G (5th Generation), etc.

[0196] In one embodiment of the present disclosure, the processor (110) can determine whether a tap operation has occurred by utilizing a change in a wireless signal. For example, when an electronic device (200) approaches a display device (100), the strength of a wireless signal (e.g., RSSI, Received Signal Strength Indicator) due to BLE communication may increase, and when the electronic device (200) moves away, the strength of the wireless signal may decrease. The processor (110) can determine that a tap operation has occurred when the strength of the wireless signal increases or when the strength of the wireless signal is greater than or equal to a threshold strength.

[0197] The sensor (140) can detect the presence, location, and type of user actions, such as spatial gestures, touches, and other user interactions. For example, the sensor (140) may include one or more sensors. For example, the sensor (140) may include a distance sensor, an image sensor, a gesture sensor, an illuminance sensor, a gyro sensor, an acceleration sensor, and the like.

[0198] In one embodiment of the present disclosure, the image sensor may include an RGB camera, and may capture an image of a user through the RGB camera, convert the captured image into sensing data, and transmit the data to the processor (110). The processor (110) may apply the image of the user to a motion recognition module to detect a user event.

[0199] The sensor (140) may not be built into the display device (100), but may be implemented as a camera of a terminal connected to the display device (100), a camera connected via USB, a sensor, a peripheral IoT (Internet of Things) device, etc.

[0200] In one embodiment of the present disclosure, the processor (110) can detect shaking due to a tapping motion using an acceleration sensor. The processor (110) can determine whether a tapping motion has occurred based on whether a sensing value received by the acceleration sensor is greater than or equal to a threshold value.

[0201] The display (150) can output images or data processed in the display device (100) under the control of the processor (110).

[0202] According to one embodiment of the present disclosure, one or more processors (110) may detect a user event through a sensor (140) by executing one or more instructions. The processor (110) may identify an electronic device (200) corresponding to the user event through a communication interface (130). The processor (110) may transmit identification information of an application being executed by the display device (100) to the electronic device (200). The processor (110) may receive application-related information from the electronic device (200) corresponding to the identification information of the application. The processor (110) may perform an application-related operation based on the received application-related information.

[0203] The electronic device (200) may include a processor (201), a memory (202), a communication interface (203), and a sensor (204).

[0204] The processor (201) controls the overall operation of the electronic device (200). For example, the processor (201) may perform the functions of the electronic device (200) described in the present disclosure by executing one or more instructions stored in the memory (202).

[0205] The memory (202) can store a program for processing and controlling the processor (201), and can store data input to or output from the electronic device (200).

[0206] In one embodiment of the present disclosure, the memory (202) may store an application for providing a certain function or service.

[0207] In one embodiment of the present disclosure, the memory (202) may store information that is preset or pre-stored through an application executed on the electronic device (200). In the present disclosure, this may be referred to as application-related information.

[0208] The communication interface (203) may include one or more modules that enable wireless communication between the electronic device (200) and the network where the display device (100) is located. The communication interface (203) may transmit and receive data or signals to and from the display device (100). The communication interface (203) may include at least one antenna for wirelessly communicating with another device. For example, the communication interface (203) may include a Wi-Fi module, a Bluetooth communication module, a BLE (Bluetooth Low Energy) communication module, a Near Field Communication module (NFC), an infrared (IrDA) communication module, a wireless communication module, a LAN module, an Ethernet module, a wired communication module, etc.

[0209] In one embodiment of the present disclosure, the processor (201) can determine whether a tap operation has occurred by utilizing a change in a wireless signal. For example, when an electronic device (200) approaches a display device (100), the strength of a wireless signal (e.g., RSSI, Received Signal Strength Indicator) due to BLE communication may increase, and when the electronic device (200) moves away, the strength of the wireless signal may decrease. The processor (201) can determine that a tap operation has occurred when the strength of the wireless signal increases or when the strength of the wireless signal is greater than or equal to a threshold strength.

[0210] The sensor (204) can detect the presence, location, and type of user actions, such as spatial gestures, touches, and other user interactions. For example, the sensor (204) may include one or more sensors. For example, the sensor (204) may include a distance sensor, an image sensor, a gesture sensor, an illumination sensor, a gyro sensor, an acceleration sensor, and the like.

[0211] In one embodiment of the present disclosure, the processor (201) can detect shaking due to a tapping motion using an acceleration sensor. The processor (201) can determine whether a tapping motion has occurred based on whether a sensing value received by the acceleration sensor is greater than or equal to a threshold value.

[0212] In one embodiment of the present disclosure, when a user holds an electronic device (200) equipped with a gyro sensor in his / her hand and performs an action corresponding to a user event, the gyro sensor may recognize the user event. Alternatively, when a user wears a wearable device such as a watch, bracelet, or ring and makes a gesture, the wearable device may recognize the user event.

[0213] According to one embodiment of the present disclosure, a processor (201) can detect a user event through a sensor (204) by executing one or more instructions. The processor (201) can identify a display device (100) corresponding to the user event through a communication interface (203). The processor (201) can receive identification information of an application being executed and a request for application-related information from the display device (100). The processor (201) can output a notification regarding the occurrence of a user event. The processor (201) can identify application-related information and transmit the application-related information to the display device (100).

[0214] FIG. 17 is a detailed block diagram of a display device according to one embodiment of the present disclosure.

[0215] Referring to FIG. 17, the display device (100) may include a tuner unit (1740), a processor (110), a display (150), a communication interface (130), a detection unit (1730), an input / output unit (1770), a video processing unit (1780), an audio processing unit (1785), an audio output unit (1760), a memory (120), and a power supply unit (1795). Any details overlapping with the description of FIG. 16 are omitted.

[0216] A tuner unit (1740) according to one embodiment can select and tune only the frequency of a channel to be received by the display device (100) among many radio wave components through amplification, mixing, resonance, etc. of a broadcast signal received wired or wirelessly. The broadcast signal includes audio, video, and additional information (e.g., EPG (Electronic Program Guide)).

[0217] The tuner unit (1740) can receive broadcast signals from various sources, such as terrestrial broadcasting, cable broadcasting, satellite broadcasting, and Internet broadcasting. The tuner unit (1740) can also receive broadcast signals from sources, such as analog broadcasting or digital broadcasting.

[0218] According to one embodiment, a detection unit (1730) detects a user's voice, a user's image, or a user's interaction, and may include a microphone (1731), a camera unit (1732), and a light receiving unit (1733).

[0219] The microphone (1731) receives the user's spoken voice. The microphone (1731) can convert the received voice into an electrical signal and output it to the processor (110).

[0220] The optical receiver (1733) receives an optical signal (including a control signal) from an external control device through an optical window (not shown) of a bezel of the display (150), etc. The optical receiver (1733) can receive an optical signal corresponding to a user input (e.g., touch, pressing, touch gesture, voice, or motion) from the control device. A control signal can be extracted from the received optical signal under the control of the processor (110).

[0221] The input / output unit (1770) according to one embodiment can receive video (e.g., moving images, etc.), audio (e.g., voice, music, etc.), and additional information (e.g., EPG, etc.) from the outside of the display device (100). The input / output unit (1770) can include any one of a High-Definition Multimedia Interface (HDMI), a Mobile High-Definition Link (MHL), a Universal Serial Bus (USB), a Display Port (DP), a Thunderbolt, a Video Graphics Array (VGA) port, an RGB port, a D-subminiature (D-SUB), a Digital Visual Interface (DVI), a component jack, and a PC port.

[0222] A video processing unit (1780) according to one embodiment performs processing on video data received by the display device (100). The video processing unit (1780) can perform various image processing operations, such as decoding, scaling, noise filtering, frame rate conversion, and resolution conversion on the video data.

[0223] According to one embodiment, the memory (120) can store various data, programs or applications for driving and controlling the display device (100).

[0224] Additionally, the program stored in the memory (120) may include one or more instructions. The program (one or more instructions) or application stored in the memory (120) may be executed by the processor (110).

[0225] According to one embodiment, a processor (110) may acquire an image by executing one or more instructions stored in a memory (120). The image may be an image previously stored in the memory (120) or an image received from an external device via a tuner unit (1740) or a communication unit (1750). In addition, the image may be an image on which various image processing such as decoding, scaling, noise filtering, frame rate conversion, and resolution conversion are performed in a video processing unit (1780).

[0226] A display (150) according to one embodiment converts image signals, data signals, OSD signals, control signals, etc. processed by a processor (110) to generate a driving signal. The display (150) may be implemented as a PDP, LCD, OLED, flexible display, etc., and may also be implemented as a 3D display. In addition, the display (150) may be configured as a touch screen and may be used as an input device in addition to an output device.

[0227] The audio processing unit (1785) processes audio data. The audio processing unit (1785) may perform various processing operations, such as decoding, amplification, and noise filtering, on audio data. Meanwhile, the audio processing unit (1785) may include multiple audio processing modules to process audio corresponding to multiple contents.

[0228] The audio output unit (1760) outputs audio included in a broadcast signal received through the tuner unit (1740) under the control of the processor (110). The audio output unit (1760) can output audio (e.g., voice, sound) input through the communication unit (1750) or the input / output unit (1770). In addition, the audio output unit (1760) can output audio stored in the memory (120) under the control of the processor (110). The audio output unit (1760) can include at least one of a speaker, a headphone output terminal, or a S / PDIF (Sony / Philips Digital Interface:) output terminal.

[0229] The power supply unit (1795) supplies power input from an external power source to components inside the display device (100) under the control of the processor (110). In addition, the power supply unit (1795) can supply power output from one or more batteries (not shown) located inside the display device (100) to the internal components under the control of the processor (110).

[0230] The memory (120) can store various data, programs, or applications for driving and controlling the display device (100) under the control of the processor (110). The memory (120) can include a broadcast reception module (not shown), a channel control module, a volume control module, a communication control module, a voice recognition module, a motion recognition module, an optical reception module, a display control module, an audio control module, an external input control module, a power control module, a power control module for an external device connected wirelessly (e.g., Bluetooth), a voice database (DB), or a motion database (DB). The processor (110) can perform each function using the software stored in the memory (120).

[0231] According to one embodiment of the present disclosure, a display device includes one or more sensors, a communication interface, a memory storing one or more instructions, and at least one processor executing one or more instructions stored in the memory. The at least one processor detects a user event through the one or more sensors by executing the one or more instructions. The at least one processor identifies an electronic device corresponding to the user event through the communication interface. The at least one processor transmits identification information of an application being executed by the display device to the electronic device. The at least one processor receives application-related information from the electronic device in response to the identification information of the application. The at least one processor performs an application-related operation based on the received application-related information.

[0232] According to one embodiment of the present disclosure, the at least one processor may determine whether the user event has occurred based on whether a sensing value received through the at least one sensor is equal to or greater than a threshold value defined as a tap operation on the display device using the electronic device by executing the at least one instruction.

[0233] According to one embodiment of the present disclosure, the at least one processor may scan the electronic device through the communication interface by executing the one or more instructions. The at least one processor may receive an advertising packet containing identification information of the electronic device from the electronic device through the communication interface. The at least one processor may store the identification information of the electronic device.

[0234] According to one embodiment of the present disclosure, the at least one processor may transmit a request for the application-related information together with identification information of the application to the electronic device by executing the one or more instructions. The application-related information may include information necessary for the running application to perform an operation of a next process.

[0235] According to one embodiment of the present disclosure, the application-related information may include at least one of user account information, network setting information, user setting information, accessibility information, authentication information for entering manufacturing mode, content information, or source information providing content. According to one embodiment of the present disclosure, the at least one processor may, by executing the one or more instructions, perform an operation for the application to move on to the next step based on the application-related information. The at least one processor may display an execution screen corresponding to the next step through a display.

[0236] According to one embodiment of the present disclosure, the one or more sensors may include a plurality of sensors positioned at different locations of the display device. According to one embodiment of the present disclosure, the at least one processor may detect a user event through at least one sensor among the plurality of sensors by executing the one or more instructions. The at least one processor may identify an electronic device corresponding to the user event through the communication interface. The at least one processor may identify an application being executed by the display device. The at least one processor may perform a predetermined application-related operation corresponding to the location of the at least one sensor that detected the user event.

[0237] According to one embodiment of the present disclosure, the one or more sensors may include a first sensor located on a first side of the display device and a second sensor located on a second side. According to one embodiment of the present disclosure, the at least one processor may perform a first operation associated with the first side when the first sensor detects the user event by executing the one or more instructions. The at least one processor may perform a second operation associated with the second side when the second sensor detects the user event.

[0238] According to one embodiment of the present disclosure, the one or more sensors may include a plurality of sensors positioned at different locations of the display device. According to one embodiment of the present disclosure, the at least one processor may detect a user event through at least one sensor among the plurality of sensors by executing the one or more instructions. The at least one processor may identify an electronic device corresponding to the user event through the communication interface. The at least one processor may transmit identification information of an application being executed by the display device and a request for application-related information to the electronic device. The at least one processor may receive the application-related information from the electronic device. The at least one processor may perform a predetermined application-related operation in response to the location of the at least one sensor that detected the user event.

[0239] According to one embodiment of the present disclosure, the at least one processor may receive source information about content from the electronic device when the display device executes an application for displaying different content on each of a plurality of areas of the display by executing the one or more instructions. The at least one processor may display content corresponding to the source information on an area of ​​the display corresponding to a position of the at least one sensor.

[0240] A display device according to one embodiment of the present disclosure may be characterized in that it performs different operations depending on the number of times the user event is continuously detected.

[0241] According to one embodiment of the present disclosure, the at least one processor may, by executing the one or more instructions, display visual information indicating that the application is controlled based on the user event and descriptive information indicating what action is performed in response to the user event.

[0242] A method of operating a display device according to one embodiment of the present disclosure includes the steps of detecting a user event through one or more sensors, identifying an electronic device corresponding to the user event through a communication interface, transmitting identification information of an application being executed by the display device to the electronic device, receiving application-related information from the electronic device in response to the identification information of the application, and performing an application-related operation based on the received application-related information.

[0243] The step of detecting the user event according to one embodiment of the present disclosure may include a step of determining whether the user event has occurred based on whether a sensing value received through the one or more sensors is equal to or greater than a threshold value defined as a tap operation on the display device using the electronic device.

[0244] The step of identifying the electronic device corresponding to the user event according to one embodiment of the present disclosure may include the step of scanning the electronic device through the communication interface, the step of receiving an advertisement packet including identification information of the electronic device from the electronic device through the communication interface, and the step of storing the identification information of the electronic device.

[0245] According to one embodiment of the present disclosure, the step of transmitting identification information of a running application by the display device to the electronic device may include the step of transmitting a request for application-related information together with the identification information of the application to the electronic device. The application-related information may include information necessary for the running application to perform the operation of the next process.

[0246] According to one embodiment of the present disclosure, the step of performing an application-related operation based on the received application-related information may include the step of performing an operation for the application to move to a next step based on the application-related information, and the step of displaying an execution screen corresponding to the next step through a display.

[0247] A method of operating a display device according to one embodiment of the present disclosure may further include a step of detecting a user event through at least one sensor among a plurality of sensors positioned at different locations of the display device, a step of identifying an electronic device corresponding to the user event through the communication interface, a step of identifying an application being executed by the display device, and a step of performing a predetermined application-related operation corresponding to a location of the at least one sensor that detected the user event.

[0248] A method for operating a display device according to one embodiment of the present disclosure may further include a step of detecting a user event through at least one sensor among a plurality of sensors positioned at different locations of the display device, a step of identifying an electronic device corresponding to the user event through the communication interface, a step of transmitting an application-related information request including identification information of an application being executed by the display device to the electronic device, a step of receiving the application-related information from the electronic device, and a step of performing a determined application-related operation corresponding to a location of the at least one sensor that detected the user event.

[0249] The step of receiving the application-related information from the electronic device according to one embodiment of the present disclosure may include the step of receiving source information about the content from the electronic device when the display device executes an application for displaying different content in each of a plurality of areas of the display.

[0250] The step of performing a predetermined application-related action corresponding to a location of the at least one sensor that detected the user event according to one embodiment of the present disclosure may include a step of displaying content corresponding to the source information in an area of ​​the display corresponding to the location of the at least one sensor.

[0251] A method of operating a display device according to one embodiment of the present disclosure may further include a step of displaying visual information indicating that the application is controlled based on the user event and explanatory information indicating what operation is performed in response to the user event.

[0252] A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is permanently stored in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0253] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

Claims

1. In the display device, One or more sensors; communication interface; Memory that stores one or more instructions; and At least one processor configured to execute one or more instructions stored in the memory, The at least one processor, by executing the one or more instructions, Detecting user events through one or more of the above sensors, Identifying an electronic device responding to the user event through the communication interface; The display device transmits identification information of the application being run to the electronic device, Receiving application-related information from the electronic device in response to the identification information of the application; A display device that performs application-related actions based on the received application-related information.

2. In paragraph 1, The at least one processor, by executing the one or more instructions, A display device that determines whether the user event has occurred based on whether a sensing value received through the one or more sensors is greater than or equal to a threshold value defined by a tap operation on the display device using the electronic device.

3. In paragraph 1 or 2, The at least one processor, by executing the one or more instructions, Scanning the electronic device through the communication interface, Receiving an advertising packet including identification information of the electronic device from the electronic device through the communication interface, A display device that stores identification information of the electronic device.

4. In any one of paragraphs 1 to 3, The at least one processor, by executing the one or more instructions, Transmitting a request for application-related information together with identification information of said application to said electronic device, The above application-related information is a display device that includes information necessary for the running application to execute the operation of the next process.

5. In paragraph 4, A display device, wherein the application-related information includes at least one of user account information, network setting information, user setting information, accessibility information, authentication information for entering manufacturing mode, content information, or source information providing content.

6. In any one of paragraphs 1 to 5, The at least one processor, by executing the one or more instructions, Based on the above application-related information, the application performs an action to move to the next step, A display device that displays an execution screen corresponding to the above next step through a display.

7. In any one of paragraphs 1 to 6, The one or more sensors include a plurality of sensors located at different locations of the display device, The at least one processor, by executing the one or more instructions, Detecting a user event through at least one sensor among the above plurality of sensors, Identifying an electronic device responding to the user event through the communication interface; The above display device identifies the application running, A display device that performs a predetermined application-related action in response to the location of at least one sensor that detects the user event.

8. In paragraph 7, The plurality of sensors include a first sensor located on a first side of the display device and a second sensor located on a second side, The at least one processor, by executing the one or more instructions, When the first sensor detects the user event, a first action associated with the first side is performed, A display device that performs a second action associated with the second side when the second sensor detects the user event.

9. In any one of paragraphs 1 to 8, The one or more sensors include a plurality of sensors located at different locations of the display device, The at least one processor, by executing the one or more instructions, Detecting a user event through at least one sensor among the above plurality of sensors, Identifying an electronic device responding to the user event through the communication interface; The display device transmits identification information of the application running on the display device and a request for application-related information to the electronic device, Receive the application-related information from the electronic device, A display device that performs a predetermined application-related action in response to the location of at least one sensor that detects the user event.

10. In paragraph 9, The at least one processor, by executing the one or more instructions, When the display device runs an application to display different content on each of multiple areas of the display, Receive source information about content from the electronic device; A display device that displays content corresponding to the source information in an area of ​​the display corresponding to the location of at least one sensor.

11. In any one of paragraphs 1 to 10, A display device characterized in that different actions are performed depending on the number of times the above user event is detected consecutively.

12. In any one of paragraphs 1 to 11, The at least one processor, by executing the one or more instructions, A display device that displays visual information indicating that the application is controlled based on the user event and descriptive information indicating what action is performed in response to the user event.

13. In the method of operating the display device, A step of detecting a user event through one or more sensors; A step of identifying an electronic device corresponding to the user event through a communication interface; A step of transmitting identification information of an application running on the display device to the electronic device; A step of receiving application-related information from the electronic device in response to the identification information of the application; and A method comprising the step of performing an application-related action based on the received application-related information.

14. In paragraph 13, The step of detecting the above user event is: A method comprising a step of determining whether the user event has occurred based on whether a sensing value received through the one or more sensors is greater than or equal to a threshold value defined by a tap operation on the display device using the electronic device.

15. A step of detecting a user event through one or more sensors; A step of identifying an electronic device corresponding to the user event through a communication interface; A step of transmitting identification information of an application running on the display device to the electronic device; A step of receiving application-related information from the electronic device in response to the identification information of the application; and A computer-readable recording medium having recorded thereon a program for performing a method of operating a display device on a computer, the method comprising the step of performing an application-related operation based on the received application-related information.

Images