Electronic device for providing virtual keyboard corresponding to keyboard guide, and control method therefor

Abstract

An electronic device for providing a virtual keyboard corresponding to a keyboard guide and a control method therefor are provided. The electronic device according to one embodiment of the present disclosure may comprise instructions configured to: establish, on the basis of a first user input for at least one user interface, a communication connection with an external electronic device corresponding to the at least one user interface; on the basis of the establishment of the communication connection, display, on a touch screen display, a virtual keyboard so as to correspond to a keyboard guide; and transmit, on the basis of a second user input for the virtual keyboard, information about at least one visual object to the at least one external electronic device such that the at least one visual object corresponding to the second user input is displayed on the at least one external electronic device.

Description

Electronic device providing a virtual keyboard corresponding to a keyboard guide and a method for controlling the same

[0001] The present disclosure relates to an electronic device that provides a virtual keyboard corresponding to a keyboard guide and a method for controlling the same.

[0002] Various services and additional features provided through electronic devices, such as portable devices like smartphones (e.g., Samsung® Galaxy®), are gradually increasing. To enhance the utility value of these electronic devices and satisfy the needs of diverse users, telecommunications service providers or electronic device manufacturers are competitively developing devices to offer various functions and differentiate themselves from competitors. Consequently, the various functions provided through electronic devices are also becoming increasingly sophisticated.

[0003] Recently, there has been a trend of increasing numbers of electronic devices owned by individual users. For example, users tend to own various electronic devices such as tablet PCs and wearable devices in addition to smartphones. While tablet PCs can provide a virtual keyboard, the size of the keys included in the virtual keyboard may not be suitable for text input, or users may need to purchase a separate wireless keyboard to provide a keyboard input environment that suits them. Consequently, users must spend a significant amount of money to create a keyboard input environment that suits them, and they may also face the inconvenience of having to carry a bulky wireless keyboard along with their tablet PC. As another example, in the case of IoT devices such as TVs, voice recognition functions are provided separately for text input; however, such voice recognition suffers from frequent errors, such as the input of unintended words during the recognition process.

[0004] According to one embodiment of the present disclosure, an electronic device may be provided that provides a user with an environment in which input of content for another electronic device (e.g., tablet PC) connected to an electronic device (e.g., smartphone) can be easily performed by providing a keyboard guide made of a material that is easy for the user to carry and a virtual keyboard corresponding to the keyboard guide through an electronic device (e.g., smartphone).

[0005] According to one embodiment of the present disclosure, when a virtual keyboard corresponding to a keyboard guide is provided through an electronic device (e.g., a tablet PC), the currently displayed screen is divided to provide the virtual keyboard, thereby enabling efficient use of the resources of an electronic device (e.g., a tablet PC) having a screen size relatively larger than that of a smartphone.

[0006] According to one embodiment of the present disclosure, when a virtual keyboard corresponding to a keyboard guide is provided through an electronic device (e.g., a tablet PC), an electronic device may be provided that provides an environment in which input of content to an electronic device (e.g., a tablet PC) can be easily performed by dividing the currently displayed screen to provide the virtual keyboard.

[0007] According to one embodiment of the present disclosure, a method for controlling an electronic device may be provided, which provides a user with an environment in which input of content for another electronic device (e.g., tablet PC) connected to an electronic device (e.g., smartphone) can be easily performed by providing a keyboard guide made of a material that is easy for the user to carry and a virtual keyboard corresponding to the keyboard guide through an electronic device (e.g., smartphone).

[0008] According to one embodiment of the present disclosure, when a virtual keyboard corresponding to a keyboard guide is provided through an electronic device (e.g., a tablet PC), a method for controlling an electronic device is provided that allows for efficient use of resources of an electronic device (e.g., a tablet PC) having a screen that is relatively larger than a smartphone screen by dividing the currently displayed screen to provide the virtual keyboard.

[0009] According to one embodiment of the present disclosure, a method for controlling an electronic device may be provided, wherein a virtual keyboard corresponding to a keyboard guide is provided through an electronic device (e.g., a tablet PC), and the virtual keyboard is provided by dividing the currently displayed screen to provide the electronic device (e.g., a tablet PC), thereby providing an environment in which input of content to the electronic device (e.g., a tablet PC) can be easily performed by the user.

[0010] An electronic device according to one embodiment of the present disclosure comprises a communication circuit, a touchscreen display, at least one processor, and a memory configured to store a plurality of instructions. The plurality of instructions may include, when executed by the at least one processor, instructions configured to cause the electronic device to detect a keyboard guide located on the touchscreen display, and based on the detection of the keyboard guide, to display at least one user interface representing at least one external electronic device located around the electronic device through the touchscreen display so as to correspond to the keyboard guide, to establish a communication connection with the external electronic device corresponding to the at least one user interface based on a first user input for the at least one user interface, to display a virtual keyboard corresponding to the keyboard guide through the touchscreen display based on the establishment of the communication connection, and to transmit information about at least one visual object to the at least one external electronic device so as to display at least one visual object corresponding to the second user input on the at least one external electronic device based on a second user input for the virtual keyboard.

[0011] An electronic device according to one embodiment of the present disclosure comprises a touchscreen display, at least one processor, and a memory configured to store a plurality of instructions, wherein the plurality of instructions may include instructions configured to control the first application based on user input on the virtual keyboard, wherein, when executed by the at least one processor, the electronic device displays a first execution screen of a first application through the touchscreen display according to a first size, detects that a keyboard guide is located on the first execution screen, and based on the detection of the keyboard guide, displays a virtual keyboard corresponding to the keyboard guide through the touchscreen display while reducing the size of the first execution screen from the first size to a second size, and the first execution screen reduced to the second size is displayed together with the virtual keyboard so as not to overlap with it.

[0012] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments of the present document.

[0013] FIG. 2 is an exemplary drawing for explaining the function or operation of an electronic device according to one embodiment of the present disclosure providing a virtual keyboard and transmitting information input to an external electronic device according to user input on the provided virtual keyboard.

[0014] FIGS. 3a and FIGS. 3b are exemplary drawings for illustrating a keyboard guide according to one embodiment of the present disclosure.

[0015] FIG. 4 is an exemplary drawing for explaining the function or operation of an electronic device detecting a keyboard guide according to one embodiment of the present disclosure.

[0016] FIG. 5 is an exemplary drawing for explaining a function or operation of acquiring user input to display a virtual keyboard when an electronic device according to one embodiment of the present disclosure detects a keyboard guide.

[0017] FIGS. 6a and 6b are exemplary drawings for explaining a function or operation in which an electronic device according to one embodiment of the present disclosure displays an icon representing an external electronic device that can be connected to the electronic device, and establishes a communication connection with a selected external electronic device based on user input regarding the displayed icon.

[0018] FIG. 7 is an exemplary drawing for explaining the function or operation of an electronic device according to one embodiment of the present disclosure transmitting typing information entered through a virtual keyboard to at least one external electronic device connected to the electronic device so as to be operable.

[0019] FIGS. 8a and FIGS. 8b are exemplary drawings for explaining a function or operation in which an electronic device according to one embodiment of the present disclosure detects the separation of a keyboard guide and displays a guidance message indicating a transition from keyboard mode to normal mode based on the detection.

[0020] FIG. 9 is an exemplary drawing for explaining a function or operation in which an electronic device according to one embodiment of the present disclosure displays a bar set to control the attributes of a virtual keyboard displayed in correspondence with the keyboard guide around a detected keyboard guide.

[0021] FIG. 10 is an exemplary drawing for explaining a function or operation of changing the display position of a bar based on user input for a bar displayed by an electronic device according to one embodiment of the present disclosure.

[0022] FIG. 11 is an exemplary drawing for explaining the function or operation of an electronic device according to one embodiment of the present disclosure displaying a remote control key as a virtual keyboard according to the type of external electronic device detected.

[0023] FIG. 12 is an example drawing for explaining the function or operation illustrated in FIG. 11 from the perspective of a user interface.

[0024] FIG. 13 is an exemplary drawing for explaining the function or operation of an electronic device according to one embodiment of the present disclosure that divides the screen based on the detection of a keyboard guide to display an execution screen of a first application and a virtual keyboard.

[0025] FIGS. 14, FIGS. 15, and FIGS. 16 are example drawings for explaining the function or operation described in FIG. 13 from the perspective of a user interface.

[0026] FIG. 17 is an exemplary drawing for explaining the function or operation of an electronic device according to one embodiment of the present disclosure dividing a display area into a plurality of regions based on the detection of a keyboard guide.

[0027] FIG. 18 is an exemplary drawing for explaining a function or operation in which an electronic device according to one embodiment of the present disclosure controls an electronic device based on user input for an icon displayed around a keyboard guide or as a virtual keyboard.

[0028] FIGS. 19, FIGS. 20, and FIGS. 21 are example drawings for explaining the function or operation described in FIG. 18 from the perspective of a user interface.

[0029] FIG. 22 is an exemplary drawing for explaining the function or operation of an electronic device according to one embodiment of the present disclosure that reconfigures and displays a virtual keyboard based on the movement of a keyboard guide.

[0030] FIG. 23 is an example drawing for explaining the function or operation described in FIG. 22 from the perspective of a user interface.

[0031] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments.

[0032] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0033] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0034] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0035] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0036] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0037] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0038] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0039] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0040] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0041] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0042] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0043] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0044] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0045] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0046] The power management module (188) can manage the power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0047] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0048] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0049] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0050] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0051] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0052] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0053] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0054] FIG. 2 is an exemplary drawing for explaining a function or operation in which an electronic device (101) according to one embodiment of the present disclosure provides a virtual keyboard (710) and transmits information input to an external electronic device (e.g., a first external electronic device (720) and / or a second external electronic device (730)) according to user input on the provided virtual keyboard (710).

[0055] Referring to FIG. 2, an electronic device (101) according to one embodiment of the present disclosure (e.g., processor (120) of FIG. 1) can detect a keyboard guide located on a touchscreen display (e.g., display module (160) of FIG. 1) in operation 210. The electronic device (101) according to one embodiment of the present disclosure can operate in a designated mode (e.g., keyboard mode) to detect the keyboard guide. The electronic device (101) according to one embodiment of the present disclosure can enter the designated mode according to user input for a designated application (e.g., a settings application). The electronic device (101) according to one embodiment of the present disclosure can display a first button set to enter the keyboard mode and / or a second button set to select the method of the keyboard to be displayed through the electronic device (101) via the designated application. The electronic device (101) according to one embodiment of the present disclosure can enter the keyboard mode when user input for the first button and / or the second button is obtained. A keyboard mode according to one embodiment of the present disclosure may include a mode configured to display a virtual keyboard (710) corresponding to the keyboard guide (310) when contact or proximity to a plurality of tips (e.g., a first tip (312), a second tip (314), a third tip (316), and / or a fourth tip (318)) of the keyboard guide (310) (or keyboard frame) is detected. An electronic device (101) according to one embodiment of the present disclosure may identify that the keyboard guide (310) has come into contact with or is in proximity to the electronic device (101) when a plurality of contacts are detected at different locations within a specified time (e.g., 0.5 seconds) in the keyboard mode. However, an electronic device (101) according to one embodiment of the present disclosure may identify that the keyboard guide (310) has come into contact with or is in proximity to the electronic device (101) when a plurality of contacts are detected at different locations within a specified time (e.g., 0.5 seconds) without entering the specified mode.In this case, even if the electronic device (101) is not in keyboard mode, the electronic device (101) may display a virtual keyboard (710) corresponding to the keyboard guide (310).

[0056] FIGS. 3a and 3b are exemplary drawings for illustrating a keyboard guide (310) according to one embodiment of the present disclosure. FIG. 4 is an exemplary drawing for illustrating a function or operation in which an electronic device (101) according to one embodiment of the present disclosure detects the keyboard guide (310). Referring to FIGS. 3a and 3b, the keyboard guide (310) according to one embodiment of the present disclosure may be manufactured from at least partially flexible material (e.g., silicone and / or TPU). Accordingly, the keyboard guide (310) according to one embodiment of the present disclosure may be folded, bent, or rolled by an external force. The keyboard guide (310) according to one embodiment of the present disclosure may have a plurality of tips (e.g., a first tip (312), a second tip (314), a third tip (316), and / or a fourth tip (318)). A plurality of tips (e.g., a first tip (312), a second tip (314), a third tip (316), and / or a fourth tip (318)) according to one embodiment of the present disclosure may be provided in a manner that is attached to one surface of the keyboard guide (310). A plurality of tips (e.g., a first tip (312), a second tip (314), a third tip (316), and / or a fourth tip (318)) according to one embodiment of the present disclosure may be attached to the keyboard guide (310) so as to be detachable, or they may be attached fixedly (e.g., not detachable). The shape of a plurality of tips (e.g., a first tip (312), a second tip (314), a third tip (316), and / or a fourth tip (318)) according to one embodiment of the present disclosure may include, but is not limited to, a semicircular or semi-elliptical shape protruding outward. A plurality of tips (e.g., a first tip (312), a second tip (314), a third tip (316), and / or a fourth tip (318)) according to one embodiment of the present disclosure may include a resonant circuit.A resonant circuit according to one embodiment of the present disclosure may be resonated based on an electromagnetic field signal generated from a digitizer (e.g., a display module (160)) of an electronic device (101), and may radiate an electromagnetic resonance (EMR) input signal (or a magnetic field) by resonance. An electronic device (101) according to one embodiment of the present disclosure may identify the position of a keyboard guide (310) on the electronic device (101) (e.g., the position of each of a plurality of tips) using the electromagnetic resonance input signal. For example, the electronic device (101) may identify the position of the keyboard guide (310) based on the magnitude of the induced electromotive force (e.g., output current) generated by the electromagnetic resonance input signal in each of a plurality of channels (e.g., a plurality of loop coils) within the digitizer. According to one embodiment of the present disclosure, the electronic device (101) may generate a signal based on an electric field based on an electrically coupled resonance (ECR) method. The resonant circuit of the keyboard guide (310) according to one embodiment of the present disclosure may be resonated by an electric field. The electronic device (101) according to one embodiment of the present disclosure may detect potentials in a plurality of channels (e.g., electrodes) due to resonance in the keyboard guide (310), and may determine the position of the keyboard guide (310) based on the potentials. Alternatively, the keyboard guide (310) according to one embodiment of the present disclosure may be implemented using an active electrostatic (AES) method. The electronic device (101) according to one embodiment of the present disclosure may detect the keyboard guide (310) based on a change in capacitance (self-capacitance or mutual capacitance) associated with at least one electrode of the touch panel. In this case, the keyboard guide (310) may not include a resonant circuit.The term "panel" as used in this disclosure may be used as a term encompassing a digitizer and a touch screen panel (TSP). A resonant circuit according to one embodiment of this disclosure may include a coil (or inductor) and / or a capacitor. A resonant circuit according to one embodiment of this disclosure may resonate based on an input electric field and / or magnetic field (e.g., an electric field and / or magnetic field generated from a digitizer of an electronic device (101). When a keyboard guide (310) according to one embodiment of this disclosure transmits a signal by means of an EMR method, the keyboard guide (310) may generate a signal including a resonant frequency based on an electromagnetic field generated from an inductive panel of an electronic device (101). In the case where the keyboard guide (310) according to one embodiment of the present disclosure transmits a signal by the AES method, the keyboard guide (310) can generate a signal by using capacitance coupling with the electronic device (101). In the case where the keyboard guide (310) transmits a signal by the ECR method, the keyboard guide (310) can generate a signal including a resonant frequency based on the electric field generated from the capacitive device of the electronic device. A resonant circuit according to one embodiment may be used to change the strength or frequency of the electromagnetic field according to the user's operation state. For example, the resonant circuit may provide various resonant frequencies depending on the connection combination of a plurality of capacitors, or may provide various resonant frequencies based on a variable inductor and / or a variable capacitor.Referring to FIG. 4, a plurality of tips of the keyboard guide (310) (e.g., a first tip (312), a second tip (314), a third tip (316), and / or a fourth tip (318)) may be positioned on a touchscreen display of the electronic device (101). An electronic device (101) according to one embodiment of the present disclosure may identify that the keyboard guide (310) has come into contact with or is in close proximity to the electronic device (101) when a plurality of contacts are detected at different locations within a specified time (e.g., 0.5 seconds) in keyboard mode. According to one embodiment of the present disclosure, the distance between a plurality of tips (e.g., a first tip (312) and / or a second tip (314)) positioned on one side of the keyboard guide (310) and the distance between a plurality of tips (e.g., a third tip (316) and / or a fourth tip (318)) positioned on the other side of the keyboard guide (310) may be substantially the same.

[0057] FIG. 5 is an example drawing for explaining a function or operation of acquiring user input to display a virtual keyboard when an electronic device (101) according to one embodiment of the present disclosure detects a keyboard guide (310).

[0058] Referring to FIG. 5, an electronic device (101) according to one embodiment of the present disclosure may obtain a first user input from a user after detecting a keyboard guide (310). The first user input according to one embodiment of the present disclosure may include a direct touch input to a touchscreen display (e.g., a display module (160)). Based on the acquisition of the first user input, the electronic device (101) according to one embodiment of the present disclosure may determine the display direction of a virtual keyboard (710) to be displayed on the electronic device (101). For example, the electronic device (101) according to one embodiment of the present disclosure may identify the location or point where the first user input was acquired as the location or point where a space bar included in the virtual keyboard (710) is displayed. An electronic device (101) according to one embodiment of the present disclosure may display a space bar on an area including a location or point where a first user input is obtained, and may display other characters of a virtual keyboard (710) corresponding to the keyboard guide (310) based on this (e.g., display other characters so that other characters of the virtual keyboard (710) are displayed above the space bar). According to one embodiment of the present disclosure, the distance between a plurality of tips (e.g., a first tip (312) and / or a second tip (314)) disposed on one side of the keyboard guide (310) and the distance between a plurality of tips (e.g., a third tip (316) and / or a fourth tip (318)) disposed on the other side of the keyboard guide (310) may be substantially different from each other. For example, the distance (d1) between a plurality of tips (e.g., a first tip (312) and / or a second tip (314)) placed on one side of the keyboard guide (310) may be longer than the distance (d2) between a plurality of tips (e.g., a third tip (316) and / or a fourth tip (318)) placed on the other side of the keyboard guide (310).An electronic device (101) according to one embodiment of the present disclosure may identify or determine the direction in which a space bar is displayed within a virtual keyboard (710) based on the distance between a plurality of tips (e.g., d1 and d2). For example, an electronic device (101) according to one embodiment of the present disclosure may display a virtual keyboard (710) such that a space bar is displayed on the side where the distance between the plurality of tips is short (e.g., the direction in which the third tip (316) and / or the fourth tip (318) is positioned) even if no user input is obtained. According to one embodiment of the present disclosure, the shape of a plurality of tips (e.g., the first tip (312) and / or the second tip (314)) positioned on one side of the keyboard guide (310) and the shape of a plurality of tips (e.g., the third tip (316) and / or the fourth tip (318)) positioned on the other side of the keyboard guide (310) may be substantially different from each other. An electronic device (101) according to one embodiment of the present disclosure may identify or determine the direction in which a space bar is displayed within a virtual keyboard (710) based on the shape of a plurality of tips. For example, an electronic device (101) according to one embodiment of the present disclosure may display a virtual keyboard (710) such that a space bar is displayed on a surface where at least one tip having a first shape is placed (e.g., the direction in which a third tip (316) and / or a fourth tip (318) is placed) even if no user input is obtained. An electronic device (101) according to one embodiment of the present disclosure may provide a guidance message indicating that user input (e.g., touch input) is required to display the virtual keyboard (710) (or an arrangement of keys) when a keyboard guide (310) is detected.

[0059] The size of a virtual keyboard (710) according to one embodiment of the present disclosure may be predetermined. In other words, information regarding at least one virtual keyboard (710) having a size specified to correspond to the size of a keyboard guide (310) may be stored in memory (130). An electronic device (101) according to one embodiment of the present disclosure may select any one virtual keyboard (710) or an arrangement of keys corresponding to the size of a keyboard guide (310) from among at least one virtual keyboard (710) stored in memory (130). For example, an electronic device (101) according to one embodiment of the present disclosure may identify the distance between a plurality of tips (e.g., a first tip (312), a second tip (314), a third tip (316), and / or a fourth tip (318)) and select any one virtual keyboard (710) or an arrangement of keys having a size corresponding to the identified distance. In the case where information regarding a virtual keyboard (710) or an array of keys having a size corresponding to the size of a keyboard guide (310) is not stored in memory (130) (e.g., the horizontal length of the keyboard guide (310) has a first length, but only information regarding a virtual keyboard (710) or an array of keys corresponding to a second length is stored in memory (130)), an electronic device (101) according to one embodiment of the present disclosure may display the virtual keyboard (710) or an array of keys stored in memory (130) as a virtual keyboard (710) by adjusting the size of the virtual keyboard (710) or an array of keys stored in memory (130) to correspond to the distance between a plurality of identified tips (e.g., a first tip (312), a second tip (314), a third tip (316), and / or a fourth tip (318)).

[0060] Returning to FIG. 2, an electronic device (101) (e.g., processor (120) of FIG. 1) according to one embodiment of the present disclosure may, in operation 220, display at least one user interface (e.g., first icon (610) and / or second icon (620)) representing at least one external electronic device (e.g., first external electronic device (720) and / or second external electronic device (730)) located around the electronic device (101) via a touchscreen display based on detection of a keyboard guide (310). FIG. 6a and FIG. 6b are exemplary drawings for explaining a function or operation in which an electronic device according to one embodiment of the present disclosure displays an icon representing an external electronic device that can be connected to the electronic device, and establishes a communication connection with the selected external electronic device based on user input for the displayed icon.

[0061] Referring to FIGS. 6a and 6b, an electronic device (101) according to one embodiment of the present disclosure may broadcast a message for a near-field communication connection based on the detection of a keyboard guide (310). An electronic device (101) according to one embodiment of the present disclosure may receive a response message from an external electronic device (e.g., a first external electronic device (720) as a TV and / or a second external electronic device (730) as a tablet PC) based on the message broadcast from the electronic device (101). An electronic device (101) according to one embodiment of the present disclosure may identify the type of the external electronic device (e.g., a first external electronic device (720) as a TV and / or a second external electronic device (730) as a tablet PC) using information included in the received response message (e.g., information about the unique number of the external electronic device, information about the type of the external electronic device, information about the communication method supported by the external electronic device). An electronic device (101) according to one embodiment of the present disclosure may display a user interface corresponding to the type based on the identified type (e.g., a first icon (610) including a TV shape and / or a second icon (620) including a tablet PC shape). The user interface according to one embodiment of the present disclosure (e.g., a first icon (610) including a TV shape and / or a second icon (620) including a tablet PC shape) may be shown to the user through a hole (e.g., a second hole (310b) and / or a third hole (310c)) formed in the keyboard guide (310). In other words, the user interface according to one embodiment of the present disclosure (e.g., a first icon (610) including a TV shape and / or a second icon (620) including a tablet PC shape) may be displayed to correspond to the keyboard guide (310).An electronic device (101) according to one embodiment of the present disclosure may obtain a selection input for at least one of the displayed user interfaces (e.g., a first icon (610) including a TV shape and / or a second icon (620) including a tablet PC shape) as illustrated in FIG. 6b. The selection input according to one embodiment of the present disclosure may include, but is not limited to, a touch input from a user (510). Additionally, a function or operation of broadcasting a message for a near-field communication connection according to one embodiment of the present disclosure may be performed prior to operation 220. Furthermore, when the electronic device (101) according to one embodiment of the present disclosure is connected to a plurality of external electronic devices (e.g., a session is established), the device that is the target of text input among the plurality of external electronic devices may be changed through input to a designated key included in the virtual keyboard (710).

[0062] Returning to FIG. 2, an electronic device (101) (e.g., processor (120) of FIG. 1) according to one embodiment of the present disclosure may, in operation 230, establish a communication connection with an external electronic device (e.g., first external electronic device (720)) corresponding to at least one user interface (e.g., first icon (610) including a TV shape and / or second icon (620) including a tablet PC shape) based on user input for at least one user interface (e.g., first icon (610) including a TV shape). The electronic device (101) according to one embodiment of the present disclosure may transmit a session establishment request message to the first external electronic device (720) in response to a response message received from the first external electronic device (720). An electronic device (101) according to one embodiment of the present disclosure can establish a session for communication with a first external electronic device (720) based on a session establishment request message transmitted to a first external electronic device (720).

[0063] An electronic device (101) according to one embodiment of the present disclosure (e.g., processor (120) of FIG. 1) may, in operation 240, display a virtual keyboard (710) corresponding to a keyboard guide (310) through a touchscreen display based on the establishment of a communication connection. A plurality of keys included in the virtual keyboard (720) or array of keys displayed by the electronic device (101) according to one embodiment of the present disclosure may be shown to a user through a plurality of holes (e.g., a first hole (310a), a second hole (310b), and / or a third hole (310c)) formed in the keyboard guide (310).

[0064] An electronic device (101) (e.g., processor (120) of FIG. 1) according to one embodiment of the present disclosure may, in operation 250, transmit information about at least one visual object (e.g., text) corresponding to the second user input based on the second user input to the virtual keyboard (710) to an external electronic device (e.g., first external electronic device (720)) so that the visual object (e.g., text) corresponding to the second user input is displayed on the external electronic device (e.g., first external electronic device (720)). FIG. 7 is an exemplary drawing for explaining a function or operation in which an electronic device (101) according to one embodiment of the present disclosure transmits typing information entered through the virtual keyboard (710) to at least one external electronic device (e.g., first external electronic device (720) and / or second external electronic device (730)) connected to the electronic device (101) so as to be operable. Referring to FIG. 7, an electronic device (101) according to one embodiment of the present disclosure can obtain user input for a virtual keyboard (710) displayed through the electronic device (101). An electronic device (101) according to one embodiment of the present disclosure can transmit typing information corresponding to the user input for the virtual keyboard (710) to at least one external electronic device (e.g., a first external electronic device (720) and / or a second external electronic device (730)). Typing information according to one embodiment of the present disclosure can be displayed through the external electronic device (e.g., a first external electronic device (720) and / or a second external electronic device (730)).

[0065] FIGS. 8a and FIGS. 8b are exemplary drawings for explaining a function or operation in which an electronic device (101) according to one embodiment of the present disclosure detects the separation of a keyboard guide (370) and, based on the detection, displays a guidance message (e.g., a first guidance message (810)) that guides switching from keyboard mode to normal mode.

[0066] Referring to FIGS. 8a and 8b, an electronic device (101) according to one embodiment of the present disclosure can detect the separation of a keyboard guide (310) in keyboard mode. For example, an electronic device (101) according to one embodiment of the present disclosure can detect that at least one tip (e.g., first tip (312) and / or third tip (316)) among a plurality of tips (e.g., first tip (312), second tip (314), third tip (316), and / or fourth tip (318)) located on the electronic device (101) as shown in FIG. 8a is separated from the electronic device (101) (e.g., release of contact). An electronic device (101) according to one embodiment of the present disclosure may determine that the keyboard guide (310) is separated from the electronic device (101) if a specified number of tips among a plurality of tips (e.g., a first tip (312), a second tip (314), a third tip (316), and / or a fourth tip (318)) are not detected for a specified period of time or longer. Based on detecting that the keyboard guide (310) is separated from the electronic device (101), the electronic device (101) according to one embodiment of the present disclosure may provide a guidance message (e.g., a first guidance message (810)) configured to obtain user input for releasing the keyboard mode, as illustrated in FIG. 8b. The electronic device (101) according to one embodiment of the present disclosure may release or maintain the keyboard mode according to user input for the first guidance message (810). An electronic device (101) according to one embodiment of the present disclosure can release a session established between the electronic device (101) and at least one external electronic device connected to be operable, based on the release of keyboard mode.

[0067] FIG. 9 is an exemplary drawing for explaining a function or operation in which an electronic device (101) according to one embodiment of the present disclosure displays a bar (e.g., a first bar (900)) configured to control the attributes of a virtual keyboard (710) displayed in correspondence with the keyboard guide (310) around a detected keyboard guide (310). Referring to FIG. 9, the electronic device (101) according to one embodiment of the present disclosure may display a first bar (900) around the keyboard guide (310) based on the detection of the keyboard guide (310). The first bar (900) according to one embodiment of the present disclosure may include a first user interface (902) indicating the type of a first visual effect (902a) (e.g., a specified color and / or gradient effect) provided around each key included in the virtual keyboard (710), and / or a second user interface (904) indicating the attributes of the first visual effect (902a). An electronic device (101) according to one embodiment of the present disclosure may determine a first visual effect (902a) provided through a virtual keyboard (710) according to user input to a first user interface (902). An electronic device (101) according to one embodiment of the present disclosure may determine a mode of expression (e.g., blinking, disappearing after a specified time, always displayed) of the first visual effect (902a) provided through a virtual keyboard (710) according to user input to a second user interface (904). An electronic device (101) according to one embodiment of the present disclosure may provide a virtual keyboard (710) based on the acquisition of user input to the first user interface (902) and / or the second user interface (904). An electronic device (101) according to one embodiment of the present disclosure may provide a third button configured to display a first bar (900) through a designated application (e.g., a setting application and / or a keyboard skin application provided by the manufacturer of the keyboard guide (310)).An electronic device (101) according to one embodiment of the present disclosure may display or not display a first bar (900) depending on user input to a third button. The size of the first bar (900) according to one embodiment of the present disclosure may be substantially equal to the distance between the centers of a plurality of tips (e.g., a first tip (312) and a second tip (314)), but is not limited thereto. An electronic device (101) according to one embodiment of the present disclosure may provide a second visual effect (910) around a keyboard guide (310). The term “tip” as used in the present disclosure may be used interchangeably / substitutely with the term “foot”.

[0068] FIG. 10 is an exemplary drawing for explaining a function or operation in which an electronic device (101) according to one embodiment of the present disclosure changes the display position of a bar (e.g., a first bar (900)) based on user input regarding the bar (e.g., a first bar (900)). Referring to FIG. 10, an electronic device (101) according to one embodiment of the present disclosure can obtain user input (e.g., a long touch and a drag gesture) regarding the first bar (900). Based on obtaining user input (e.g., a long touch and a drag gesture) regarding the first bar (900), the electronic device (101) according to one embodiment of the present disclosure can change the display position of the first bar (900) as shown in FIG. 10.

[0069] FIG. 11 is an exemplary drawing for explaining a function or operation in which an electronic device (101) according to one embodiment of the present disclosure displays a remote control key as a virtual keyboard (710) depending on the type of detected external electronic device (e.g., a first external electronic device (720) and / or a second external electronic device (730)). FIG. 12 is an exemplary drawing for explaining the function or operation illustrated in FIG. 11 in terms of a user interface.

[0070] Referring to FIGS. 11 and 12, an electronic device (101) according to one embodiment of the present disclosure (e.g., processor (120) of FIG. 1) can detect a keyboard guide located on a touchscreen display in operation 1110. For example, an electronic device (101) according to one embodiment of the present disclosure can detect a keyboard guide (310) located on a touchscreen display in keyboard mode.

[0071] An electronic device (101) according to one embodiment of the present disclosure may, in operation 1120, based on the detection of a keyboard guide (310), display at least one user interface (e.g., a first icon (610) and / or a second icon (620)) representing at least one external electronic device located around the electronic device (101) via a touchscreen display to correspond to the keyboard guide (310). An electronic device (101) according to one embodiment of the present disclosure may, based on the type of the identified external electronic device (e.g., a first external electronic device (720)), display a user interface corresponding to the identified type (e.g., a first icon (610) including a TV shape and / or a second icon (620) including a tablet PC shape). A user interface according to one embodiment of the present disclosure (e.g., a first icon (610) including a TV shape and / or a second icon (620) including a tablet PC shape) can be shown to the user through a hole (e.g., a second hole (310b) and / or a third hole (310c)) formed in the keyboard guide (310).

[0072] An electronic device (101) according to one embodiment of the present disclosure may, in operation 1130, obtain a first user input for at least one user interface (e.g., a first icon (610)). Based on the obtained first user input, the electronic device (101) according to one embodiment of the present disclosure may establish a communication connection with an external electronic device (e.g., a first external electronic device (720)) corresponding to at least one user interface (e.g., a first icon (610) including a TV shape).

[0073] An electronic device (101) according to one embodiment of the present disclosure can identify the type of an external electronic device (e.g., a first external electronic device (720)) corresponding to a first user input in operation 1140. An electronic device (101) according to one embodiment of the present disclosure can identify the type of an external electronic device (e.g., a first external electronic device (720)) by using information included in a response message obtained from an external electronic device.

[0074] An electronic device (101) according to one embodiment of the present disclosure may display a virtual keyboard or at least one remote control key corresponding to a keyboard guide (310) based on the identified type in operation 1150. An electronic device (101) according to one embodiment of the present disclosure may display a virtual keyboard (710) including additional keys (e.g., number keys and / or shortcut keys) when the external electronic device is identified as a device including a keyboard as the type of the identified external electronic device (e.g., first external electronic device (720)). An electronic device (101) according to one embodiment of the present disclosure may display smart remote control keys as a virtual keyboard (710) as shown in FIG. 12 when the external electronic device is identified as a TV. Smart remote control keys according to one embodiment of the present disclosure may include at least one first set of shortcut keys (1210) corresponding to a specific broadcaster and / or a second set of shortcut keys (1220) configured to execute a specified application according to user input. A second set of shortcut keys (1220) according to one embodiment of the present disclosure may include at least one icon associated with a designated application (e.g., a third icon (1222a), a fourth icon (1222b), a fifth icon (1222c), a sixth icon (1222d), and / or a seventh icon (1222e)). According to one embodiment of the present disclosure, at least one icon associated with a designated application (e.g., a third icon (1222a), a fourth icon (1222b), a fifth icon (1222c), a sixth icon (1222d), and / or a seventh icon (1222e)) may include, for example, a shortcut key for executing at least one preferred channel selected by the user among the broadcast channels provided by the designated application (e.g., a video streaming application).An electronic device (101) according to one embodiment of the present disclosure may transmit information (e.g., user icon selection information) designated to an external electronic device (e.g., first external electronic device (720)) and / or a server so that a function or operation corresponding to the user input is performed through an external electronic device (e.g., first external electronic device (720)) in accordance with user input to a smart remote control.

[0075] FIG. 13 is an exemplary drawing for explaining a function or operation in which an electronic device (101) according to one embodiment of the present disclosure divides the screen based on the detection of a keyboard guide (310) to display an execution screen of a first application and a virtual keyboard (710). FIG. 14, FIG. 15 and FIG. 16 are exemplary drawings for explaining the function or operation described in FIG. 13 from the perspective of a user interface. FIG. 17 is an exemplary drawing for explaining a function or operation in which an electronic device (101) according to one embodiment of the present disclosure divides a display area into a plurality of regions (e.g., a first region (1710), a second region (1720), and / or a third region (1730)) based on the detection of a keyboard guide (310).

[0076] Referring to FIG. 13, an electronic device (101) according to one embodiment of the present disclosure (e.g., processor (120) of FIG. 1) can display a first execution screen of a first application according to a first size through a touchscreen display in operation 1310. An electronic device (101) according to one embodiment of the present disclosure can display an execution screen of a video streaming application in full screen, as shown in FIG. 14.

[0077] An electronic device (101) according to one embodiment of the present disclosure can detect that a keyboard guide (310) is positioned on a first execution screen in operation 1320. An electronic device (101) according to one embodiment of the present disclosure can identify that a plurality of tips (e.g., a first tip (312), a second tip (314), a third tip (316), and / or a fourth tip (318)) are in contact with or near the electronic device (101) in key mode mode, as illustrated in FIG. 15.

[0078] An electronic device (101) according to one embodiment of the present disclosure may, in operation 1330, display a virtual keyboard (710) corresponding to the keyboard guide (310) through a touchscreen display while reducing the size of a first execution screen from a first size to a second size based on the detection of the keyboard guide (310). An electronic device (101) according to one embodiment of the present disclosure may, as shown in FIG. 16, display a first execution screen as a full screen and then reduce it to a second size while displaying the virtual keyboard (710) corresponding to the keyboard guide (310) together with the reduced first execution screen (1410). An electronic device (101) according to one embodiment of the present disclosure may, in addition to the first execution screen (1410) of a first application having a reduced size, display a second execution screen (1420) of a second application together. According to one embodiment of the present document, an electronic device (101) may display only a first execution screen (1410) of a first application having a reduced size and a virtual keyboard (710) when it is determined that a second execution screen (1420) of a second application (e.g., the most recently executed application other than the first application) cannot be displayed depending on the size of the display (e.g., display module (160)) of the electronic device (101) and the size of the keyboard guide (310). To this end, the electronic device (101) according to one embodiment of the present document may identify the size of the keyboard guide (310), and when the identified size of the keyboard guide (310) is substantially the same as the length (e.g., horizontal length) of the display module (160) or differs within a specified error range, it may display only a first execution screen (1410) of a first application having a reduced size and a virtual keyboard (710).Alternatively, an electronic device (101) according to one embodiment of the present document may determine in advance (e.g., before the keyboard guide (310) is detected) whether to display only the first execution screen (1410) of the first application having a reduced size and the virtual keyboard (710) based on information about the size of the display module (130) and information about the size of the keyboard guide (310) that are stored in advance in memory (130). An electronic device (101) according to one embodiment of the present document may divide the display area of ​​the display module (160) into a plurality of areas (e.g., a first area (1710), a second area (1720) and / or a third area (1730)), as shown in FIG. 17, and display the execution screen of the application in each area (e.g., a first area (1710), a second area (1720) and / or a third area (1730)). The number and / or size of a plurality of regions according to one embodiment of the present disclosure may be predetermined or specified by a user through a designated application (e.g., a configuration application).

[0079] An electronic device (101) according to one embodiment of the present disclosure can control a first application based on user input to a virtual keyboard (710) in operation 1340. When user input to a virtual keyboard (710) is obtained, the electronic device (101) according to one embodiment of the present disclosure can control an active application (e.g., a first application) (e.g., displaying typed characters on the execution screen of the first application). The electronic device (101) according to one embodiment of the present disclosure can switch the active application based on user input to the application execution screen.

[0080] FIG. 18 is an exemplary drawing for explaining a function or operation in which an electronic device (101) according to one embodiment of the present disclosure controls the electronic device (101) based on user input for a bar (e.g., second bar (1910)) displayed around a keyboard guide (310) or an icon (e.g., eighth icon (1911), ninth icon (1912), tenth icon (1913), eleventh icon (1914), twelveth icon (1915)) displayed as a virtual keyboard (710). FIG. 19, FIG. 20 and FIG. 21 are exemplary drawings for explaining the function or operation described in FIG. 18 from the perspective of a user interface.

[0081] Referring to FIG. 18, an electronic device (101) according to one embodiment of the present disclosure can display a first execution screen of a first application according to a first size through a touchscreen display (e.g., a display module (160)) in operation 1810. An electronic device (101) according to one embodiment of the present disclosure can display an execution screen of a video streaming application in full screen, as shown in FIG. 14.

[0082] An electronic device (101) according to one embodiment of the present disclosure can detect that a keyboard guide (310) is positioned on a first execution screen in operation 1820. An electronic device (101) according to one embodiment of the present disclosure can identify that a plurality of tips (e.g., a first tip (312), a second tip (314), a third tip (316), and / or a fourth tip (318)) are in contact with or near the electronic device (101) in key mode mode, as illustrated in FIG. 15.

[0083] An electronic device (101) according to one embodiment of the present disclosure may, in operation 1830, display a virtual keyboard (710) corresponding to the keyboard guide (310) through a touchscreen display while reducing the size of a first execution screen from a first size to a second size based on the detection of the keyboard guide (310). An electronic device (101) according to one embodiment of the present disclosure may, as illustrated in FIG. 19, display a first execution screen as a full screen and then reduce it to a second size while displaying a virtual keyboard (710) corresponding to the keyboard guide (310) together with the reduced first execution screen (1410). An electronic device (101) according to one embodiment of the present disclosure may, in addition to the first execution screen (1410) of a first application having a reduced size, display a second execution screen (1420) of a second application and / or a third execution screen (1430) of a third application together.

[0084] An electronic device (101) according to one embodiment of the present disclosure may, in operation 1840, display a bar (e.g., a second bar (1910)) including at least one visual object around a virtual keyboard (710). An electronic device (101) according to one embodiment of the present disclosure may, in operation 1850, activate an application to be controlled by the virtual keyboard based on user input regarding the displayed bar (e.g., the second bar (1910)). A second bar (1910) according to one embodiment of the present disclosure may include a 13th icon (1911) set to control a first execution screen (1410), a 14th icon (1912) set to control a second execution screen (1420), a 15th icon (1913) set to control a third execution screen (1430), a 16th icon (1914) set to control a first external electronic device (720) connected to the electronic device for operation, a 17th icon (1915) set to control a second external electronic device (730) connected to the electronic device for operation, and / or a text display area (1916) set to display text typed by a user. An electronic device (101) according to one embodiment of the present disclosure may activate an application execution screen corresponding to a designated icon based on user input for a designated icon (e.g., the 15th icon (1913)). In other words, an electronic device (101) according to one embodiment of the present disclosure can control an application execution screen corresponding to a designated icon through user input to a virtual keyboard (710). Alternatively, an electronic device (101) according to one embodiment of the present disclosure can control an external electronic device corresponding to a designated icon according to user input to a designated icon (e.g., the 17th icon (1915)).For example, an electronic device (101) according to one embodiment of the present disclosure may transmit information about the detected typing input to an external electronic device corresponding to the designated icon when a typing input by a user is detected while a designated icon (e.g., the 17th icon (1915)) is selected. An external electronic device according to one embodiment of the present disclosure (e.g., the second external electronic device (730)) may display information about the received typing input (e.g., display a typed string).

[0085] An electronic device (101) according to one embodiment of the present disclosure may provide a virtual keyboard (710) (e.g., to be displayed to the user through a hole in the keyboard guide (310)) a 13th icon (1911) set to control a first execution screen (1410), a 14th icon (1912) set to control a second execution screen (1420), a 15th icon (1913) set to control a third execution screen (1430), a 16th icon (1914) set to control a first external electronic device (720) connected to the electronic device to be operable, and / or a 17th icon (1915) set to control a second external electronic device (730) connected to the electronic device to be operable.

[0086] An electronic device (101) according to one embodiment of the present disclosure may display a second bar (1910) around a keyboard guide (310) that includes at least one visual object for controlling a currently active application (e.g., a video streaming application), as illustrated in FIG. 21. In this case, the electronic device (101) according to one embodiment of the present disclosure may include, but is not limited to, an icon for stopping the video, an icon for speeding up the playback speed, and / or an icon for sharing the video being played with another external device, as at least one visual object for controlling the active application (e.g., a video streaming application).

[0087] FIG. 22 is an exemplary drawing for explaining a function or operation in which an electronic device (101) according to one embodiment of the present disclosure reconfigures and displays a virtual keyboard (710) based on the movement of a keyboard guide (310). FIG. 23 is an exemplary drawing for explaining the function or operation described in FIG. 22 from the perspective of a user interface.

[0088] Referring to FIG. 22, an electronic device (101) according to one embodiment of the present disclosure can detect the movement of a keyboard guide in operation 2210, in a state where a virtual keyboard and an execution screen of at least one application are displayed so as not to overlap each other. An electronic device (101) according to one embodiment of the present disclosure can display at least one icon (e.g., 18th icon (2311), 19th icon (2312), 20th icon (2313), and / or 21st icon (2314)) for controlling an application (e.g., a game application) as a virtual keyboard (710) according to the type of application, as shown in FIG. 23, to correspond to the keyboard guide (310). An electronic device (101) according to one embodiment of the present disclosure may display at least one icon for controlling another application (e.g., a 22nd icon (2315), a 23rd icon (2316), a 24th icon (2317), and / or a 25th icon (2318)) in addition to at least one icon for controlling an application (e.g., a game application) (e.g., a 18th icon (2311), a 19th icon (2312), a 20th icon (2313), and / or a 21st icon (2314)) in correspondence with a keyboard guide (310), in addition to at least one icon for controlling an application (e.g., a video playback application) (e.g., a 22nd icon (2315), a 23rd icon (2316), a 24th icon (2317), and / or a 25th icon (2318)), in accordance with FIG. 23, when there are multiple applications running. An electronic device (101) according to one embodiment of the present disclosure may detect movement of the keyboard guide (310) while at least one icon is displayed. An electronic device (101) according to one embodiment of the present disclosure can detect the movement of a keyboard guide (310) by detecting the movement of at least some of the tips (e.g., first tip (312), second tip (314), third tip (316) and / or fourth tip (318)) among a plurality of tips (e.g., first tip (312) and third tip (316)).

[0089] An electronic device (101) according to one embodiment of the present disclosure may reconstruct and display a virtual keyboard (710) based on the movement of a keyboard guide (310) detected in operation 2220. An electronic device (101) according to one embodiment of the present disclosure may display at least one icon (e.g., a 22nd icon (2315), a 23rd icon (2316), a 24th icon (2317), and / or a 25th icon (2318)) by moving it according to the movement of the keyboard guide (310). In this case, the size of the execution screen of an application displayed around the keyboard guide (310) may be enlarged or reduced according to the movement of the keyboard guide (310). An electronic device (101) according to one embodiment of the present disclosure may not display some of the icons of at least one icon (e.g., 22nd icon (2315), 23rd icon (2316), 24th icon (2317), and / or 25th icon (2318)) depending on the movement of the keyboard guide (310).

[0090] An electronic device according to one embodiment of the present disclosure (e.g., the electronic device (101) of FIG. 1) comprises a communication circuit, a touchscreen display (e.g., the display module (160) of FIG. 1), at least one processor (e.g., the processor (120) of FIG. 1), and a memory configured to store a plurality of instructions (e.g., the memory (130) of FIG. 1). When the plurality of instructions are executed by the at least one processor, the electronic device detects a keyboard guide located on the touchscreen display, and based on the detection of the keyboard guide, displays at least one user interface representing at least one external electronic device located around the electronic device through the touchscreen display so as to correspond to the keyboard guide, establishes a communication connection with the external electronic device corresponding to the at least one user interface based on a first user input for the at least one user interface, and based on the establishment of the communication connection, displays a virtual keyboard through the touchscreen display so as to correspond to the keyboard guide, and based on a second user input for the virtual keyboard, It may include instructions configured to transmit information about at least one visual object to at least one external electronic device so that at least one visual object corresponding to the second user input is displayed on the at least one external electronic device.

[0091] An electronic device according to one embodiment of the present disclosure comprises a touchscreen display, at least one processor, and a memory configured to store a plurality of instructions, wherein the plurality of instructions may include instructions configured to control the first application based on user input on the virtual keyboard, wherein, when executed by the at least one processor, the electronic device displays a first execution screen of a first application through the touchscreen display according to a first size, detects that a keyboard guide is located on the first execution screen, and based on the detection of the keyboard guide, displays a virtual keyboard corresponding to the keyboard guide through the touchscreen display while reducing the size of the first execution screen from the first size to a second size, and the first execution screen reduced to the second size is displayed together with the virtual keyboard so as not to overlap with it.

[0092] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0093] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0094] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0095] Various embodiments of this document may be implemented as software (e.g., program (2540)) comprising one or more instructions stored in a storage medium (e.g., internal memory (2536) or external memory (2538)) readable by a machine (e.g., electronic device (2501)). For example, a processor of the machine (e.g., electronic device (2501)) may call at least one of the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, "non-transient" simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0096] According to one embodiment, the method according to the various embodiments disclosed herein may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0097] According to one embodiment of this document, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to one embodiment of this document, one or more of the components or operations among the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to the integration. According to one embodiment of this document, operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device, communication circuit, touchscreen display, At least one processor, and The electronic device includes a memory configured to store a plurality of instructions, wherein the plurality of instructions, when executed by the at least one processor, cause: Detecting a keyboard guide located on the above touchscreen display, and Based on the detection of the above keyboard guide, at least one user interface representing at least one external electronic device located around the electronic device is displayed through the touchscreen display to correspond to the keyboard guide, and Based on a first user input for the at least one user interface, a communication connection is established with an external electronic device corresponding to the at least one user interface, and Based on the establishment of the above communication connection, a virtual keyboard is displayed through the touchscreen display to correspond to the keyboard guide, and An electronic device characterized by including instructions configured to transmit information about at least one visual object to at least one external electronic device so that, based on a second user input to the virtual keyboard, at least one visual object corresponding to the second user input is displayed on the at least one external electronic device.

2. In Paragraph 1, An electronic device characterized in that the plurality of instructions further include instructions configured to cause the electronic device to provide a designated visual effect around each of the plurality of keys included in the virtual keyboard when executed by the at least one processor.

3. In Paragraph 1 or 2, The electronic device is characterized in that the plurality of instructions further include an instruction configured to cause the electronic device to display at least one menu for changing the properties of the specified visual effect around the keyboard guide when executed by the at least one processor.

4. In any one of paragraphs 1 through 3, The electronic device is characterized in that the plurality of instructions further include instructions configured to cause the electronic device to change the display position of the at least one menu based on user input for the at least one menu when executed by the at least one processor.

5. In any one of paragraphs 1 through 4, The above plurality of instructions, when executed by the at least one processor, cause the electronic device: An instruction set to determine the type of the virtual keypad according to the type of the external electronic device, and An electronic device characterized by further including instructions set to display the virtual keypad based on the type of the virtual keypad determined above.

6. In any one of paragraphs 1 through 5, The above plurality of instructions, when executed by the at least one processor, cause the electronic device: An instruction set to detect when the above keyboard guide is separated from the electronic device, and An electronic device characterized by further including an instruction configured to display a notification message for obtaining user input for releasing a mode configured to display the virtual keyboard based on detecting the separation.

7. In electronic devices, touchscreen display, At least one processor, and The electronic device includes a memory configured to store a plurality of instructions, wherein the plurality of instructions, when executed by the at least one processor, cause: Through the above touchscreen display, a first execution screen of a first application is displayed according to a first size, and Detecting that a keyboard guide is located on the first execution screen, and Based on the detection of the keyboard guide, a virtual keyboard corresponding to the keyboard guide is displayed through the touchscreen display while reducing the size of the first execution screen from the first size to the second size, and the first execution screen reduced to the second size is displayed together with the virtual keyboard so as not to overlap. An electronic device characterized by including instructions configured to control the first application based on user input to the virtual keyboard.

8. In Paragraph 7, The above plurality of instructions, when executed by the at least one processor, cause the electronic device: An instruction configured to display the second execution screen of a second application together with the first execution screen and the virtual keypad reduced to the second size, such that the second execution screen does not overlap with the first execution screen and the virtual keypad, and An electronic device characterized by further including an instruction configured to display a first bar, comprising a plurality of icons configured to obtain selection input for the first application or the second application, and a first area configured to display text information entered through the virtual keypad, around the virtual keypad.

9. In Paragraph 7 or 8, The above plurality of instructions, when executed by the at least one processor, cause the electronic device: An instruction configured to display the second execution screen of a second application together with the first execution screen and the virtual keypad reduced to the second size, such that the second execution screen does not overlap with the first execution screen and the virtual keypad, and An electronic device characterized by further including an instruction configured to display a plurality of icons configured to obtain a selection input for the first application or the second application as the virtual keyboard.

10. In any one of paragraphs 7 through 9, The above plurality of instructions, when executed by the at least one processor, cause the electronic device: An instruction set to identify the type of the first application or the second application, and An electronic device characterized by further including instructions configured to display around the keyboard guide at least one first bar including a first icon for controlling the first application, or at least one second bar including a second icon for controlling the second application, based on the identified type.

11. In any one of paragraphs 7 through 10, The above plurality of instructions, when executed by the at least one processor, cause the electronic device: An instruction set to detect movement of the keyboard guide while the first execution screen or the second execution screen is displayed, and An electronic device characterized by further including instructions configured to adjust the display size of the first execution screen or the second execution screen and to rearrange the virtual keyboard based on the detected movement.

12. In any one of paragraphs 7 through 11, The above electronic device further includes a communication circuit, The above plurality of instructions, when executed by the at least one processor, cause the electronic device: An instruction set to display at least one third icon representing at least one external electronic device operably connected to the electronic device, and included in the first bar. An electronic device characterized by further including an instruction configured to transmit text information to the at least one external electronic device through the communication circuit, based on user input for the at least one third icon, so that text information entered through the virtual keyboard is displayed through the at least one external electronic device.

13. In any one of paragraphs 7 through 12, An electronic device characterized in that the first execution screen reduced to the second size, the second execution screen of the second application, and the virtual keyboard are displayed to occupy the entire display area of ​​the touchscreen display without overlapping each other.

14. In a non-transient recording medium configured to store a plurality of computer-executable instructions, said plurality of instructions are, when executed by at least one processor of an electronic device, given to said electronic device: Through the touchscreen display of the electronic device, a first execution screen of a first application is displayed according to a first size, and Detecting that a keyboard guide is located on the first execution screen, and Based on the detection of the keyboard guide, a virtual keyboard corresponding to the keyboard guide is displayed through the touchscreen display while reducing the size of the first execution screen from the first size to the second size, and the first execution screen reduced to the second size is displayed together with the virtual keyboard so as not to overlap. A non-transient recording medium characterized by including instructions configured to control the first application based on user input to the virtual keyboard.

15. In Paragraph 14, The above plurality of instructions, when executed by the at least one processor, cause the electronic device: An instruction configured to display the second execution screen of a second application together with the first execution screen and the virtual keypad reduced to the second size, such that the second execution screen does not overlap with the first execution screen and the virtual keypad, and A non-transient recording medium characterized by further including an instruction to display a first bar, which includes a plurality of icons set to obtain selection input for the first application or the second application around the virtual keypad, and a first area set to display text information entered through the virtual keypad.

Images