Electronic device for recognizing gestures performed by a stylus pen and operating method thereof

Abstract

An electronic device for identifying gestures performed by a stylus and an operating method thereof are provided. The electronic device includes a sensing panel; at least one processor electrically connected to the sensing panel and a communication module; and a memory electrically connected to the at least one processor. The memory may store instructions, which are configured to cause the at least one processor to perform the following operations when executed by the at least one processor: based on detecting a pen signal from the stylus through the sensing panel, perform a first operation based on the detected pen signal recognition; and based on not detecting a pen signal from the stylus through the sensing panel, receive at least one communication signal through the communication module, recognize a gesture based on information about the position of the stylus included in the received at least one communication signal, and perform a second operation corresponding to the recognized gesture. Various other embodiments are possible.

Description

Technical Field

[0001] The present disclosure relates to an electronic device for recognizing a gesture performed by a stylus pen and an operating method thereof. More specifically, the present disclosure relates to recognizing a gesture performed using a stylus pen based on information included in a communication signal received from the stylus pen and being able to perform an operation corresponding to the recognized gesture. Background Art

[0002] Electronic devices have been developed to receive various inputs from users through a specific input device (e.g., a stylus pen) connected thereto via wireless communication. The electronic device can recognize a position designated thereon by an input device having a pen function (the term "stylus pen" may be used for convenience of description in this disclosure) and can execute a function corresponding to the position.

[0003] The electronic device may detect a magnetic field generated by the stylus using an electromagnetic resonance (hereinafter referred to as "electromagnetic resonance (EMR)") method. The electronic device may recognize the position of the stylus based on an induced electromotive force generated by the magnetic field for each channel.

[0004] The stylus pen may be connected to the electronic device via short-range communication, such as Bluetooth Low Energy (BLE). The stylus pen may transmit information, such as the pressed state of a button arranged on the housing of the stylus pen, to the electronic device via short-range communication, and the electronic device may perform a specific operation based on the received information.

[0005] The above information is presented as background information only to assist with understanding the present disclosure. No determination has been made, and no assertion is made, as to whether any of the above may be applicable as prior art to the present disclosure. Summary of the invention

[0006] Depending on whether a button provided on the stylus is pressed, the electronic device may perform a specific operation. However, only a few operations of the electronic device may be mapped to turning on or off a button provided in the stylus. Therefore, it may be difficult to effectively control applications that support various operations using the stylus.

[0007] Aspects of the present disclosure are intended to at least solve the above-mentioned problems and / or disadvantages, and to at least provide the advantages described below. Therefore, one aspect of the present disclosure is to provide an electronic device and an operating method thereof, which are capable of recognizing a gesture performed using a stylus based on information included in a communication signal received from the stylus, and are capable of performing an operation corresponding to the recognized gesture.

[0008] Additional aspects will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the presented embodiments.

[0009] According to one aspect of the present disclosure, an electronic device is provided. The electronic device includes: a sensing panel; at least one processor electrically connected to the sensing panel and a communication module; and a memory electrically connected to the at least one processor. The memory may store instructions, which are configured to cause the at least one processor to perform the following operations when executed by the at least one processor: based on detecting a pen signal from a stylus through the sensing panel, perform a first operation based on the detected pen signal recognition, and based on not detecting a pen signal from the stylus through the sensing panel: receiving at least one communication signal through the communication module, recognizing a gesture based on information about the position of the stylus included in the received at least one communication signal, and performing a second operation corresponding to the recognized gesture.

[0010] According to another aspect of the present disclosure, an electronic device is provided. The electronic device includes: a communication module; at least one processor electrically connected to the communication module; and a memory electrically connected to the at least one processor. The memory may store instructions, which are configured to cause the at least one processor to perform the following operations when executed by the at least one processor: receiving a first communication signal including first information indicating the start of gesture input from a stylus through the communication module; receiving multiple second communication signals from the stylus through the communication module, and identifying one or more candidate gestures based on information about the position of the stylus included in the received multiple second communication signals; receiving a third communication signal through the communication module after receiving the multiple second communication signals, identifying a gesture that satisfies a specific condition in the identified one or more candidate gestures based on identifying the second information indicating the termination of gesture input included in the third communication signal, and performing an operation corresponding to the identified gesture.

[0011] According to another aspect of the present disclosure, a method for operating an electronic device is provided. The method includes, based on detecting a pen signal from a stylus through a sensing panel of the electronic device, performing a first operation based on recognition of the detected pen signal, and based on not detecting the pen signal from the stylus through the sensing panel, the method may include: receiving at least one communication signal through a communication module of the electronic device, recognizing a gesture based on information about a position of the stylus included in the received at least one communication signal, and performing a second operation corresponding to the recognized gesture.

[0012] According to various embodiments, an electronic device capable of recognizing a gesture performed using a stylus pen based on information included in a communication signal received from the stylus pen and capable of performing an operation corresponding to the recognized gesture, and an operating method of the electronic device can be provided.

[0013] Other aspects, advantages and salient features of the present disclosure will become apparent to those skilled in the art from the following detailed description which discloses various embodiments of the present disclosure in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following description in conjunction with the accompanying drawings, in which:

[0015] Figure 1 is a block diagram illustrating an electronic device in a network environment according to an embodiment of the present disclosure;

[0016] Figure 2 is a perspective view showing an electronic device including a stylus pen according to an embodiment of the present disclosure;

[0017] Figure 3A is a block diagram illustrating a stylus according to an embodiment of the present disclosure;

[0018] Figure 3B is an exploded perspective view showing a stylus according to an embodiment of the present disclosure;

[0019] Figure 4 is a diagram showing a configuration of an electronic device and a stylus pen according to an embodiment of the present disclosure;

[0020] Figure 5 is a diagram for describing a configuration of an electronic device according to an embodiment of the present disclosure;

[0021] Fig. 6A is a flowchart for describing operations of a stylus pen and an electronic device when the stylus pen is inserted into the electronic device according to an embodiment of the present disclosure;

[0022] Figure 6B is a flowchart for describing operations of a stylus pen and an electronic device when the stylus pen is removed from the electronic device according to an embodiment of the present disclosure;

[0023] Figure 6C is a diagram illustrating operations of an electronic device and a stylus when the stylus is inserted into the electronic device according to an embodiment of the present disclosure;

[0024] Figure 7 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure;

[0025] Fig. 8A is a flowchart for describing a method of operating an electronic device according to an embodiment of the present disclosure;

[0026] Figure 8B is a flowchart for describing the operation of an electronic device and a stylus according to an embodiment of the present disclosure;

[0027] Fig. 9 is a diagram for describing an operation performed according to a distance between an electronic device and a stylus according to an embodiment of the present disclosure;

[0028] Fig.10 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure;

[0029] Fig.11 is a diagram for describing a detector according to an embodiment of the present disclosure;

[0030] Fig.12 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure;

[0031] Fig.13 is a diagram for describing switching between states set for candidate gesture determination according to an embodiment of the present disclosure;

[0032] Fig.14 is a diagram for describing a detection process performed by a swipe gesture detector according to an embodiment of the present disclosure;

[0033] Fig.15 is a diagram for describing a detection process performed by a circle gesture detector according to various embodiments of the present disclosure;

[0034] Fig.16 is a diagram for describing a detection process performed by a shake gesture detector according to an embodiment of the present disclosure;

[0035] Fig.17 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure;

[0036] Fig.18 is a diagram illustrating movement of a stylus pen according to an embodiment of the present disclosure;

[0037] Fig.19 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure;

[0038] Fig. 20A is a diagram illustrating an electronic device that performs an operation corresponding to a gesture according to an embodiment of the present disclosure;

[0039] Fig. 20B is a diagram illustrating a screen displayed by an electronic device according to an embodiment of the present disclosure;

[0040] Fig. 20C is a diagram illustrating a screen displayed by an electronic device according to an embodiment of the present disclosure;

[0041] Fig.20Dis a diagram illustrating a screen displayed by an electronic device according to an embodiment of the present disclosure;

[0042] Fig.20E is a diagram illustrating a screen displayed by an electronic device according to an embodiment of the present disclosure;

[0043] Fig.20F is a diagram illustrating a screen displayed by an electronic device according to an embodiment of the present disclosure;

[0044] Fig.21 is a diagram illustrating a gesture guide screen of an electronic device according to an embodiment of the present disclosure;

[0045] Fig.22A is a diagram illustrating a setting screen of an electronic device according to an embodiment of the present disclosure;

[0046] Fig. 22B is a diagram illustrating a setting screen of an electronic device according to an embodiment of the present disclosure;

[0047] Fig. 22C is a diagram illustrating a setting screen of an electronic device according to an embodiment of the present disclosure;

[0048] Fig.23A is a flowchart for describing a method of operating a stylus according to an embodiment of the present disclosure; and

[0049] Fig. 23B is a flowchart for describing a method of operating a stylus pen according to an embodiment of the present disclosure.

[0050] Throughout the drawings, it should be noted that like reference numbers are used to depict the same or similar elements, features, and structures. DETAILED DESCRIPTION

[0051] The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of the various embodiments of the present disclosure as defined by the claims and their equivalents. The description includes various specific details to assist in this understanding, but these are to be considered as merely exemplary. Therefore, it will be appreciated by those of ordinary skill in the art that various changes and modifications may be made to the various embodiments described herein without departing from the scope and spirit of the present disclosure. In addition, descriptions of well-known functions and configurations may be omitted for clarity and brevity.

[0052] The terms and words used in the following description and claims are not limited to the bibliographic meanings, but are merely used by the inventor to enable a clear and consistent understanding of the present disclosure. Therefore, it should be apparent to those skilled in the art that the following description of various embodiments of the present disclosure is provided for illustrative purposes only and not for the purpose of limiting the present disclosure as defined by the appended claims and their equivalents.

[0053] It should be understood that the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to "a component surface" includes reference to one or more of such surfaces.

[0054] Figure 1 is a block diagram illustrating electronic devices in a network environment according to an embodiment of the present disclosure.

[0055] Figure 1 1 is a block diagram illustrating an electronic device 101 in a network environment 100 according to various embodiments. Figure 1 , the electronic device 101 in the network environment 100 may communicate with the electronic device 102 via the first network 198 (e.g., a short-range wireless communication network), or communicate with the electronic device 104 or the server 108 via the second network 199 (e.g., a long-range wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 via the server 108. According to an embodiment, the electronic device 101 may include a processor 120, a memory 130, an input device 150, a sound output device 155, a display device 160, an audio module 170, a sensor module 176, an interface 177, a haptic module 179, a camera module 180, a power management module 188, a battery 189, a communication module 190, a user identification module (SIM) 196, or an antenna module 197. In some embodiments, at least one of the components (e.g., the display device 160 or the camera module 180) may be omitted from the electronic device 101, or one or more other components may be added to the electronic device 101. In some embodiments, some of the components may be implemented as a single integrated circuit.For example, the sensor module 176 (eg, a fingerprint sensor, an iris sensor, or an illumination sensor) may be implemented to be embedded in the display device 160 (eg, a display).

[0056] The processor 120 may run, for example, software (e.g., program 140) to control at least one other component (e.g., hardware component or software component) of the electronic device 101 connected to the processor 120, and may perform various data processing or calculations. According to one embodiment, as at least part of the data processing or calculation, the processor 120 may load a command or data received from another component (e.g., sensor module 176 or communication module 190) into the volatile memory 132, process the command or data stored in the volatile memory 132, and store the resultant data in the non-volatile memory 134. According to an embodiment, the processor 120 may include a main processor 121 (e.g., a central processing unit (CPU) or an application processor (AP)) and an auxiliary processor 123 (e.g., a graphics processing unit (GPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that is independent of or combined with the main processor 121 in operation. Additionally or alternatively, the auxiliary processor 123 may be adapted to consume less power than the main processor 121, or adapted to be specifically used for a specified function. The auxiliary processor 123 may be implemented separately from the main processor 121 , or as part of the main processor 121 .

[0057] When the main processor 121 is in an inactive (e.g., sleep) state, the auxiliary processor 123 (rather than the main processor 121) may control at least some of the functions or states related to at least one component (e.g., the display device 160, the sensor module 176, or the communication module 190) among the components of the electronic device 101, or when the main processor 121 is in an active state (e.g., running an application), the auxiliary processor 123 may control at least some of the functions or states related to at least one component (e.g., the display device 160, the sensor module 176, or the communication module 190) together with the main processor 121. According to an embodiment, the auxiliary processor 123 (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., the camera module 180 or the communication module 190) that is functionally related to the auxiliary processor 123.

[0058] The memory 130 may store various data used by at least one component of the electronic device 101 (e.g., the processor 120 or the sensor module 176). The various data may include, for example, software (e.g., the program 140) and input data or output data for commands related thereto. The memory 130 may include a volatile memory 132 or a nonvolatile memory 134. The nonvolatile memory 134 may include an internal memory 136 or an external memory 138.

[0059] The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system (OS) 142 , middleware 144 , or applications 146 .

[0060] The input device 150 may receive commands or data from outside the electronic device 101 (e.g., a user) to be used by other components of the electronic device 101 (e.g., the processor 120). The input device 150 may include, for example, a microphone, a mouse, a keyboard, or a digital pen (e.g., a stylus).

[0061] The sound output device 155 can output a sound signal to the outside of the electronic device 101. The sound output device 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as playing multimedia or playing records, and the receiver can be used for incoming calls. Depending on the embodiment, the receiver can be implemented as a separate part from the speaker, or as a part of the speaker.

[0062] The display device 160 may visually provide information to the outside of the electronic device 101 (e.g., a user). The display device 160 may include, for example, a display, a holographic device, or a projector, and a control circuit for controlling a corresponding one of the display, the holographic device, and the projector. According to an embodiment, the display device 160 may include a touch circuit adapted to detect a touch or a sensor circuit (e.g., a pressure sensor) adapted to measure the strength of a force caused by a touch.

[0063] The audio module 170 may convert sound into an electrical signal, or vice versa. According to an embodiment, the audio module 170 may obtain sound via the input device 150, or output sound via the sound output device 155 or an external electronic device (e.g., the electronic device 102 (e.g., a speaker or earphone)) directly or wirelessly coupled to the electronic device 101.

[0064] The sensor module 176 may detect an operating state (e.g., power or temperature) of the electronic device 101 or an environmental state (e.g., a state of a user) outside the electronic device 101, and then generate an electrical signal or data value corresponding to the detected state. According to an embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illumination sensor.

[0065] The interface 177 may support one or more specific protocols to be used to connect the electronic device 101 directly (e.g., wired) or wirelessly to an external electronic device (e.g., the electronic device 102). According to an embodiment, the interface 177 may include, for example, a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0066] The connection end 178 may include a connector, wherein the electronic device 101 can be physically connected to an external electronic device (e.g., the electronic device 102) via the connector. According to an embodiment, the connection end 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0067] The haptic module 179 may convert the electrical signal into mechanical stimulation (eg, vibration or motion) or electrical stimulation that can be recognized by the user via his sense of touch or kinesthetic sense. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0068] The camera module 180 may capture still images or moving images. According to an embodiment, the camera module 180 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0069] The power management module 188 may manage power supply to the electronic device 101. According to an embodiment, the power management module 188 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0070] The battery 189 may power at least one component of the electronic device 101. According to an embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0071] The communication module 190 may support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108), and perform communication via the established communication channel. The communication module 190 may include one or more communication processors capable of operating independently from the processor 120 (e.g., an application processor (AP)) and supporting direct (e.g., wired) communication or wireless communication. According to an embodiment, the communication module 190 may include a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). A corresponding one of these communication modules can communicate with an external electronic device via a first network 198 (e.g., a short-range communication network such as Bluetooth, Wireless Fidelity (Wi-Fi) Direct, or Infrared Data Association (IrDA)) or a second network 199 (e.g., a long-range communication network such as a cellular network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules can be implemented as a single component (e.g., a single chip), or these various types of communication modules can be implemented as multiple components separated from each other (e.g., multiple chips). The wireless communication module 192 can identify and authenticate the electronic device 101 in a communication network (such as the first network 198 or the second network 199) using user information (e.g., an International Mobile Subscriber Identity (IMSI)) stored in the user identification module 196.

[0072] The antenna module 197 may transmit or receive a signal or power to or from the outside of the electronic device 101 (e.g., an external electronic device). According to an embodiment, the antenna module 197 may include an antenna including a radiating element formed of a conductive material or a conductive pattern formed in or on a substrate (e.g., a PCB). According to an embodiment, the antenna module 197 may include a plurality of antennas. In this case, at least one antenna suitable for a communication scheme used in a communication network (such as the first network 198 or the second network 199) may be selected from the plurality of antennas by, for example, the communication module 190 (e.g., the wireless communication module 192). A signal or power may then be transmitted or received between the communication module 190 and the external electronic device via the selected at least one antenna. According to an embodiment, another component (e.g., a radio frequency integrated circuit (RFIC)) other than the radiating element may be additionally formed as a part of the antenna module 197.

[0073] At least some of the above components may be connected to each other via an inter-peripheral communication scheme (e.g., a bus, a general purpose input output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)) and communicatively transmit signals (e.g., commands or data) therebetween.

[0074] According to an embodiment, a command or data may be sent or received between the electronic device 101 and the external electronic device 104 via the server 108 connected to the second network 199. Each of the electronic device 102 and the electronic device 104 may be a device of the same type as the electronic device 101, or a device of a different type from the electronic device 101. According to an embodiment, all or some operations to be executed in the electronic device 101 may be executed in one or more of the external electronic device 102, the external electronic device 104, or the server 108. For example, if the electronic device 101 should automatically execute a function or service or should execute a function or service in response to a request from a user or another device, the electronic device 101 may request the one or more external electronic devices to execute at least part of the function or service instead of executing the function or service, or the electronic device 101 may request the one or more external electronic devices to execute at least part of the function or service in addition to executing the function or service. The one or more external electronic devices that receive the request may execute the requested at least part of the function or service, or execute another function or another service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may provide the result as at least a partial reply to the request with or without further processing the result. To this end, for example, cloud computing technology, distributed computing technology, or client-server computing technology may be used.

[0075] Figure 2 FIG. 2 is a diagram showing an electronic device (eg, Figure 1 According to various embodiments, the stylus in this document can be used with Figure 1 The input device 150 corresponds to rather than to Figure 1 Corresponding to the electronic device 102.

[0076] Reference Figure 2 , the electronic device 101 of various embodiments may include Figure 12. The electronic device 101 includes a housing 210, and may include a structure in which the stylus 201 can be inserted. The electronic device 101 includes a housing 210, and may include a hole 211 in a portion of the housing 210 (e.g., a portion of a side surface 210a). The electronic device 101 may include a first internal space 212, which is a receiving space connected to the hole 211, and the stylus 201 may be inserted into the first internal space 212. According to the illustrated embodiment, the stylus 201 may include a first button 201a that can be pressed at one end thereof, so that the stylus 201 can be easily taken out from the first internal space 212 of the electronic device 101. When the first button 201a is pressed, a repulsion mechanism associated with the first button 201a (e.g., a repulsion mechanism operated by at least one elastic member (e.g., a spring)) may be activated to separate the stylus 201 from the internal space 212.

[0077] Figure 3A FIG. 2 is a diagram showing a stylus pen (eg, Figure 2 Block diagram of the stylus 201 in FIG.

[0078] Reference Figure 3A According to an embodiment, the stylus 201 may include a processor 220, a memory 230, a resonant circuit 287, a charging circuit 288, a battery 289, a communication circuit 290, an antenna 297, a trigger circuit 298, and / or a sensor 299. In some embodiments, the processor 220, at least a portion of the resonant circuit 287, and / or at least a portion of the communication circuit 290 of the stylus 201 may be constructed on a printed circuit board or in the form of a chip. The processor 220, the resonant circuit 287, and / or the communication circuit 290 may be electrically connected to the memory 230, the charging circuit 288, the battery 289, the antenna 297, the trigger circuit 298, and / or the sensor 299.

[0079] The processor 220 according to various embodiments may include a general-purpose processor configured to execute a customized hardware module or software (e.g., an application). The processor 220 may include a hardware component (function) or a software element (program) including at least one of various sensors, a data measurement module, an input / output interface, a module for managing the state or environment of the stylus 201, or a communication module provided in the stylus. The processor 220 may include, for example, at least one of hardware, software, firmware, and a combination of two or more thereof. According to an embodiment, the processor 220 may be configured to send an indication button (e.g., Figure 3B The information may include information on a pressed state of a button 337 of the handwriting control unit 201, sensing information acquired by the sensor 229, and / or information calculated based on the sensing information (eg, information associated with the position of the stylus pen 201).

[0080] The resonant circuit 287 according to various embodiments may resonate based on an electromagnetic field signal generated by a digitizer (e.g., display device 160) of the electronic device 101, and may radiate an electromagnetic resonance (EMR) input signal (or magnetic field). The electronic device 101 may use the electromagnetic resonance input signal to identify the position of the stylus 201 on the electronic device 101. For example, the electronic device 101 may identify the position of the stylus 201 based on the size of the induced electromotive force (e.g., output current) generated by the electromagnetic resonance input signal in each of a plurality of channels (e.g., a plurality of loop coils) within the digitizer. In addition, the electronic device 101 and the stylus 201 have been described above as operating based on the EMR method. However, this is only an example, and the electronic device 101 may generate a signal based on an electric field by an electrically coupled resonance (ECR) method. The resonant circuit of the stylus 201 may be resonated by the electric field. The electronic device 101 may identify the potential generated in a plurality of channels (e.g., electrodes) due to the resonance in the stylus 201, and may also identify the position of the stylus 201 based on the potential. The stylus 201 may be implemented by an active electrostatic (AES) method, and one of ordinary skill in the art will appreciate that there is no limitation on the type of implementation.

[0081] According to various embodiments, the memory 230 may store information associated with the operation of the stylus 201. For example, the information may include information for communicating with the electronic device 101 and frequency information associated with the input operation of the stylus 201. In addition, the memory 230 may store a program (or application, algorithm, or processing loop) for calculating information (e.g., coordinate information and / or displacement information) about the position of the stylus 201 based on the sensing data of the sensor 299. The memory 230 may also store a communication stack of the communication circuit 290. Depending on the implementation, the communication circuit 290 and / or the processor 220 may include a dedicated memory.

[0082] The resonant circuit 287 according to various embodiments may include a coil (or inductor) and / or a capacitor. The resonant circuit 287 may resonate based on an electric field and / or a magnetic field input thereto (e.g., an electric field and / or a magnetic field generated by a digitizer of the electronic device 101). When the stylus 201 sends a signal by the EMR method, the stylus 201 may generate a signal including a resonant frequency based on an electromagnetic field generated by a sensing panel of the electronic device 101. When the stylus 201 sends a signal by the AES method, the stylus 201 may generate a signal using capacitive coupling with the electronic device 101. When the stylus 201 sends a signal by the ECR method, the stylus 201 may generate a signal including a resonant frequency based on an electric field generated by a capacitive device of the electronic device 101. According to an embodiment, the resonant circuit 287 may be used to change the strength or frequency of the electromagnetic field according to the operation state of the user. For example, the resonant circuit 287 may provide various frequencies for identifying a suspended input, a drawing input, a button input, or an erase input. For example, the resonance circuit 287 may provide various resonance frequencies by connecting a plurality of capacitors in various combinations, or may provide various resonance frequencies based on a variable inductor and / or a variable capacitor.

[0083] When the charging circuit 288 according to various embodiments is connected to the resonance circuit 287 based on the switching circuit, the resonance signal generated by the resonance circuit 287 may be rectified into a DC signal and may be provided to the battery 289. According to an embodiment, the stylus 201 may recognize whether the stylus 201 is inserted into the electronic device 101 using the voltage level of the DC signal detected in the charging circuit 288.

[0084] According to various embodiments, the battery 289 may be configured to store power required to operate the stylus 201. The battery 289 may include, for example, a lithium-ion battery or a capacitor, and may be rechargeable or replaceable. According to an embodiment, the battery 289 may be charged using power (e.g., a DC signal (DC power)) supplied from the charging circuit 288.

[0085] The communication circuit 290 according to various embodiments may be configured to perform a wireless communication function between the stylus 201 and the communication module 190 of the electronic device 101. According to an embodiment, the communication circuit 290 may send state information, input information, and / or position-related information of the stylus 201 to the electronic device 101 using a short-range communication method. For example, the communication circuit 290 may send direction information (e.g., motion sensor data) of the stylus 201 acquired through the trigger circuit 298, voice information input through the microphone, or information about the remaining amount of the battery 289 to the electronic device 101. For example, the communication circuit 290 may send sensing data acquired from the sensor 299 and / or information associated with the position of the stylus 201 identified based on the sensing data to the electronic device 101. For example, the communication circuit 290 may send information about a button (e.g., Figure 3B The information of the state of the button 337 of the electronic device 101 is transmitted to the electronic device 101. As an example, the short-range communication method may include at least one of Bluetooth, Bluetooth Low Energy (BLE), or wireless LAN.

[0086] The antenna 297 according to various embodiments may be used to transmit / receive signals or power to / from the outside (e.g., the electronic device 101). According to an embodiment, the stylus pen 201 may include a plurality of antennas 297, and at least one antenna 297 suitable for a communication method may be selected from among the plurality of antennas 297. Through the selected at least one antenna 297, the communication circuit 290 may exchange signals or power with an external electronic device.

[0087] According to various embodiments, the trigger circuit 298 may include at least one button or sensor circuit. According to an embodiment, the processor 220 may identify an input method (e.g., touch or push) or a type of button of the stylus 201 (e.g., an EMR button or a BLE button). According to an embodiment, the sensor circuit may generate an electrical signal or data value corresponding to an internal operating state or an external environmental state of the stylus 201. For example, the sensor circuit may include at least one of a motion sensor, a remaining battery capacity detection sensor, a pressure sensor, an optical sensor, a temperature sensor, a geomagnetic sensor, and a biometric sensor. According to an embodiment, the trigger circuit 298 may send a trigger signal to the electronic device 101 using an input signal of a button or a signal acquired by a sensor.

[0088] The sensor 299 according to various embodiments may include an acceleration sensor (e.g., an accelerometer), a gyroscope sensor, and / or a geomagnetic sensor. The acceleration sensor may sense information about the linear movement of the stylus 201. The gyroscope sensor may sense information related to the rotation of the stylus 201. The geomagnetic sensor may sense information about the tilt state (e.g., orientation) of the stylus 201. The processor 220 may send the information acquired from the sensor 299 to the electronic device 101 via the communication circuit 290. Optionally, based on the information obtained from the sensor 299, the processor 220 may send information associated with the position of the stylus 201 (e.g., the coordinates of the stylus 201 and / or the displacement of the stylus 201) to the electronic device 101 via the communication circuit 290.

[0089] Figure 3B FIG. 2 is a diagram showing a stylus pen (eg, Figure 2 Exploded perspective view of the stylus 201 in FIG.

[0090] Reference Figure 3B , the stylus pen 201 may include a pen case 300 forming the exterior of the stylus pen 201 and internal components inside the pen case 300. In the illustrated embodiment, the internal components may be inserted into the pen case 300 in a single assembly operation in a state where several components installed inside the stylus pen 201 are assembled together.

[0091] The pen case 300 has an elongated shape between the first end 300a, the middle portion 300c, and the second end 300b, and may include a second internal space 301 therein. The pen case 300 may have an elliptical cross section having a major axis and a minor axis and may be formed as a whole in an elliptical cylinder shape. Figure 2 As described above, the first internal space 212 of the electronic device 101 may also be formed to have an elliptical cross-section corresponding to the shape of the pen housing 300. According to various embodiments, at least a portion of the pen housing 300 may include a synthetic resin (e.g., plastic) and / or a metal material (e.g., aluminum). According to an embodiment, the first end portion 300a of the pen housing 300 may be made of a synthetic resin material. Various embodiments may be applied to the material of the pen housing 300.

[0092] The internal component may have an elongated shape corresponding to the shape of the pen case 300. The internal component may be roughly divided into three parts along its length direction. For example, the internal component may include a coil portion 310 disposed at a position corresponding to the first end 300a of the pen case 300, a pop-up portion 320 disposed at a position corresponding to the second end 300b of the pen case 300, and a circuit board portion 330 disposed at a position corresponding to the body of the pen case 300.

[0093] The coil part 310 may include a pen tip 311 exposed to the outside of the first end 300a when the internal component is fully inserted into the pen case 300, a sealing ring 312, a coil 313 wound multiple times, and / or a writing pressure sensor 314 configured to obtain a pressure change depending on the compression of the pen tip 311. The sealing ring 312 may include epoxy resin, rubber, polyurethane, or silicone resin. The sealing ring 312 may be provided for waterproof and dustproof purposes, and may protect the coil part 310 and the circuit board part 330 from flooding or dust. According to an embodiment, the coil 313 may form a resonant frequency in a set frequency band (e.g., 500kHz), and may be combined with at least one device (e.g., a capacitive element (capacitor)) so as to adjust the resonant frequency formed by the coil 313 within a predetermined range.

[0094] The pop-up portion 320 may include a pop-up portion for ejecting a signal from an electronic device (e.g., Figure 2 According to an embodiment, the pop-up portion 320 may include a rod 321, a pop-up member 322 disposed around the rod 321 and forming an overall appearance of the pop-up portion 320, and a button unit 323 (for example, Figure 2 321 and the pop-up member 322. When the internal components are fully inserted into the pen housing 300, the portion including the rod 321 and the pop-up member 322 is surrounded by the second end 300b of the pen housing 300, and at least a portion of the button unit 323 may be exposed to the outside of the second end 300b. A plurality of components (not shown) (e.g., a cam member or an elastic member) may be arranged in the pop-up member 322 to form a push-pull structure. In an embodiment, the button unit 323 may be substantially coupled to the rod 321 to reciprocate relative to the pop-up member 322. According to various embodiments, the button unit 323 may include a button having a locking structure to allow the user to pull out the stylus 201 using a fingernail. According to an embodiment, the stylus 201 may provide another input method by including a sensor for detecting the linear reciprocating motion of the rod 321.

[0095] The circuit board portion 330 may include a printed circuit board 332, a base 331 surrounding at least one surface of the printed circuit board 332, and an antenna. According to an embodiment, a substrate placement portion 333 on which the printed circuit board 332 is arranged is formed on the top surface of the base 331, and the printed circuit board 332 may be fixed in a state of being placed on the substrate placement portion 333. According to an embodiment, the printed circuit board 332 may include a first surface and a second surface, wherein a variable capacitor or a switch 334 connected to the coil 313 may be arranged on the first surface, and a charging circuit, a battery 336, or a communication circuit may be arranged on the second surface. Here, according to an embodiment, the first surface and the second surface of the printed circuit board 332 may be different stacking surfaces in a vertical stacking structure. According to another embodiment, the first surface and the second surface may be different parts arranged along the longitudinal direction of the printed circuit board 332. The battery 336 may include a double layer capacitor (EDLC). The charging circuit is located between the coil 313 and the battery, and may include a voltage detector circuit and a rectifier. Here, the battery 336 may not necessarily be arranged on the second surface of the printed circuit board 332. The position of the battery 336 may be variously arranged according to various mounting structures of the circuit board 330 , and may be arranged at a position different from that shown in the drawings.

[0096] The antenna may include Figure 3B The antenna structure 339 in the example shown and / or the antenna embedded in the printed circuit board 332. According to various embodiments, the switch 334 may be provided on the printed circuit board 332. The second button 337 provided on the stylus pen 201 is used to push the switch 334, and may be exposed to the outside through the side opening 302 in the pen case 300. When the second button 337 is supported by the support member 338 and no external force acts on the second button 337, the support member 338 provides an elastic restoring force to restore or maintain the second button 337 in a state of being arranged in a predetermined position.

[0097] The circuit board portion 330 may include, for example, a sealing ring (such as an O-ring). According to an embodiment, an O-ring made of an elastomer may be arranged at opposite ends of the base 331 to form a sealing structure between the base 331 and the pen case 300. In some embodiments, the support member 338 may be partially in close contact with the inner wall of the pen case 300 around the side opening 302 to form a sealing structure. For example, the circuit board portion 330 may also include a waterproof and dustproof structure similar to the sealing ring 312 of the coil portion 310 on at least a portion thereof.

[0098] The stylus pen 201 may include a battery seating portion 333a in which a battery 336 is arranged on a top surface of the base 331. The battery 336 mountable in the battery seating portion 333a may include, for example, a cylindrical type battery.

[0099] The stylus pen 201 may include a microphone (not shown) and / or a speaker. The microphone and / or the speaker may be directly connected to the printed circuit board 332, or may be connected to a separate flexible printed circuit board (FPCB) (not shown) connected to the printed circuit board 332. According to various embodiments, the microphone and / or the speaker may be arranged at a position parallel to the second button 337 in the longitudinal direction of the stylus pen 201.

[0100] Figure 4 is a diagram illustrating a configuration of an electronic device and a stylus pen according to an embodiment of the present disclosure.

[0101] Reference Figure 4 , the electronic device 101 may include a pen controller 410. The pen controller 410 may include, for example, at least one amplifier connected to at least one coil 411 or 412. The pen controller 410 may include at least one coil 411 or 412, and may provide charging power to the stylus 201 through at least one coil 411 or 412. Optionally, the sensing panel controller (e.g., Figure 5 The sensing panel controller 511 in the embodiment may provide a signal having a pattern through at least one coil 411 or 412. The pattern is shared with the stylus 201 in advance and may also include a pattern for controlling the stylus 201, for example, including a charging start indication pattern, a charging termination indication pattern, and a detection pattern. Although two coils 411 and 412 for providing a charging signal or a signal having a pattern for control are shown, this is only an example, and the number of coils is not limited thereto.

[0102] According to various embodiments, the resonant circuit 420 (eg, Figure 3A The resonant circuit 287 in the electronic device 101 may include a coil 421, at least one capacitor 422 or 423, and / or a switch 424. When the switch 424 is in an off state, the coil 421 and the capacitor 422 may constitute a resonant circuit, and when the switch 424 is in an on state, the coil 421 and the capacitors 422 and 423 may constitute a resonant circuit. Therefore, the resonant frequency of the resonant circuit 420 may change depending on the on / off state of the switch 424. For example, the electronic device 101 may recognize the on / off state of the switch 424 based on the frequency of the signal from the stylus 201.

[0103] According to various embodiments, at least one rectifier 431 or 435 may rectify an alternating current (AC) waveform signal VPEN output from the resonance circuit 420 and may output the rectified signal. chg_ctrl) 432 can receive the rectified signal VM output from the rectifier 431. Based on the rectified signal VM, the charging switch controller 432 can identify whether the signal generated by the resonant circuit 420 is a signal for charging or a signal for position detection. For example, the charging switch controller 432 can identify whether the signal generated by the resonant circuit 420 is a signal for charging or a signal for position detection based on, for example, the voltage of the rectified signal VM. When it is identified that the signal generated by the resonant circuit 420 is a signal for charging, the charging switch controller 432 can turn on the charging switch (SW chg ) 436 is controlled to be in an on state. The charging switch controller 432 may send a charging start signal (chg_on) to the charging switch 436. In this case, the rectified signal VIN may be sent to the battery 437 (eg, Figure 3A The battery 289 in FIG. 4 is a battery 437 that can be charged using the received rectified signal VIN. An overvoltage protection circuit (OVP) 433 can identify the battery voltage VBAT and can control the charging switch 436 to be in an off state when the battery voltage exceeds an overvoltage threshold.

[0104] According to various embodiments, when it is identified that the battery voltage exceeds the operating voltage threshold, the load switch controller (SW L _ctrl)434 can control the load switch (SW L ) 438 is turned on. When the load switch 438 is turned on, power from the battery 437 can be transmitted to the BLE communication circuit / controller (BLE+ controller) 439 (e.g., Figure 3A The BLE communication circuit / controller 439 may operate using the received power. When the distance between the stylus 201 and the electronic device 101 is greater than the threshold distance, the button control circuit 440 may send information about the input of the button (e.g., button 337) to the BLE communication circuit / controller 439. The BLE communication circuit / controller 439 may communicate with the BLE communication circuit / controller 439 via the antenna 441 (e.g., Figure 3A The antenna 297 in the sensor 450 sends the received information about the button input to the electronic device 101. Figure 3AThe sensor 299 in the electronic device 101 may include a gyro sensor 451 and / or an accelerometer 452. The sensing data acquired by the gyro sensor 451 and / or the accelerometer 452 may be sent to the BLE communication circuit / controller 439. The BLE communication circuit / controller 439 may send a communication signal including the received sensing data to the electronic device 101 via the antenna 441. Optionally, the BLE communication circuit / controller 439 may identify information associated with the identified position of the stylus 201 (e.g., coordinates and / or displacement of the stylus 201) based on the received sensing data. The BLE communication circuit / controller 439 may send the identified information associated with the position of the stylus 201 to the electronic device 101 via the antenna 441. According to various embodiments, when the stylus 201 is pulled out from the electronic device 101, the BLE communication circuit / controller 439 may activate the accelerometer 452. The BLE communication circuit / controller 439 may activate the gyro sensor 451 when a button (e.g., button 337) is pressed. In addition, the activation time is only illustrative, and there is no limitation on the activation time of each sensor. In addition, the sensor 450 may further include a geomagnetic sensor. When only the accelerometer 452 is activated, the stylus 201 may provide the electronic device 101 with acceleration information measured by the accelerometer 452, and the electronic device 101 may operate based on both the position of the stylus 201 and the acceleration information, and the position of the stylus 201 is recognized based on the pen signal.

[0105] Figure 5 is a diagram for describing a configuration of an electronic device according to an embodiment of the present disclosure.

[0106] Reference Figure 5, the electronic device 101 may include a sensing panel controller 511, a processor 512 (e.g., processor 120), a Bluetooth controller 513 (e.g., communication module 190), and / or an antenna 514. The electronic device 101 may include a sensing panel 503, a display component 502 arranged on the sensing panel 503, and / or a window 501 arranged on the display component 502. Depending on the implementation, when the sensing panel 503 is implemented as a digital converter, a touch sensing panel (e.g., a touch screen panel (TSP)) may also be arranged above or below the sensing panel 503. Depending on the implementation, the touch sensing panel may be located above the display component 502. The sensing panel 503 may be implemented as a digital converter and may include a plurality of loop coils. According to various embodiments, when implemented as a digital converter, the sensing panel 503 may include an element (e.g., an amplifier) ​​for applying an electrical signal (e.g., a transmission signal) to the loop coil. The sensing panel 503 may include an element (e.g., an amplifier, a capacitor, and an ADC) for processing a signal (e.g., an input signal) output from the loop coil. The sensing panel 503 may recognize the position of the stylus 201 based on the magnitude of the signal output from each loop coil (e.g., the digital value converted for each channel), and may output the position of the stylus 201 to the processor 120. Optionally, according to the implementation, the processor 120 may recognize the position of the stylus 201 based on the magnitude of the signal output from each loop coil (e.g., the digital value converted for each channel). For example, the sensing panel 503 may apply a current to at least one of the loop coils, and at least one coil may form a magnetic field. The stylus 201 may resonate by the magnetic field formed around it, and a magnetic field may be formed by the stylus 201 due to the resonance. Due to the magnetic field formed by the stylus 201, a current may be output from each loop coil. The electronic device 101 may recognize the position of the stylus 201 based on the magnitude of the current of the corresponding channel of the loop coil (e.g., the digital value converted). In order to determine the position of the stylus 201, the loop coil may include a coil extending in one axis (e.g., x-axis) direction and a coil extending in another axis (e.g., y-axis) direction, but there is no limitation on its arrangement form. The sensing panel controller 511 may apply a transmission signal Tx to at least some of the plurality of loop coils of the sensing panel 503, and the loop coils receiving the transmission signal Tx may form a magnetic field. The sensing panel controller 511 may receive a reception signal Rx output from at least some of the plurality of loop coils in a time-division manner. The sensing panel controller 511 may identify the position of the stylus 201 based on the reception signal Rx, and may send the position of the stylus 201 to the processor 512. Optionally, when the electronic device 101 operates in an ECR method, the sensing panel 503 is implemented as a touch sensing panel, and may identify the position of the stylus 201 based on an output signal from an electrode of the touch sensing panel.Optionally, when the electronic device 101 detects the pen using a touch sensing panel, the pen may be detected based on changes in capacitance (mutual capacitance and / or self capacitance) at electrodes of the touch sensing panel. Hardware capable of sensing a pen signal from a stylus or a touch sensing panel of a digitizer may be referred to as a sensing panel 503. In various embodiments, the electronic device 101 may detect the stylus (or the position of the stylus) by identifying an electrical signal generated by the pen based on an AES method.

[0107] The sensing panel controller 511 may also identify whether the stylus 201 is inserted into (or coupled to) the electronic device 101 based on the received signal, and may send the result of the identification to the processor 512. According to the implementation, the sensing panel controller 511 may be implemented integrally with the sensing panel 503. The processor 512 may send a signal for wireless charging based on whether the stylus is inserted. The processor 512 may control the Bluetooth controller 513 based on whether the stylus is inserted, and when the wireless communication connection is not established, the processor 512 may perform control so that the stylus 201 is connected for wireless communication. In addition, when the stylus 201 is installed, the processor 512 may send charging capacity information to the electronic device 101, and when the stylus 201 is removed, the processor 512 may send information about button pressing and sensor data to the electronic device 101. In addition, based on the data received from the stylus 201, the processor 512 may perform control so that the charging signal and the control signal are sent to the sensing panel controller 511. Based on the data received from the stylus 201, the processor 512 may recognize a gesture and may perform an operation corresponding to the gesture. The processor 512 may send a function mapped to the gesture to an application. The Bluetooth controller 513 may send / receive information through the stylus 201 and the antenna 514. The display part 502 may include an element for displaying a screen. The window 501 may be made of a transparent material capable of accommodating the display part 502.

[0108] Fig. 6A is a flowchart for describing operations of a stylus pen and an electronic device when the stylus pen is inserted into the electronic device according to an embodiment of the present disclosure.

[0109] Reference Fig. 6A, in operation 601, the stylus 201 may be inserted into the accommodation space in the electronic device 101. However, the insertion into the accommodation space is merely illustrative, and the electronic device 101 may also include an area (or space) in addition to the storage space where the stylus 201 can be installed (or attached), in which case the stylus 201 may be detachably attached to the corresponding area (or space). It should be understood by a person of ordinary skill in the art that the operation of the stylus 201 in the present disclosure when it is located in the accommodation space may also be performed when the stylus 201 is attached to the installation area (or space) in another embodiment. For example, a user may insert the stylus 201 into the accommodation space in the electronic device 101, and considering that the corresponding operation is not an active operation of the stylus 201, the corresponding operation is indicated by a dotted line. The description will be made with reference to the case where the stylus 201 is inserted into the electronic device 101 before a communication connection between the stylus 201 and the electronic device 101 is established. Fig. 6A Embodiment. In the present disclosure, when the electronic device 101 or the stylus 201 performs a specific operation, it may indicate that the processor 120 included in the electronic device 101 or the processor 220 included in the stylus 201 performs the specific operation. When the electronic device 101 or the stylus 201 performs a specific operation, it may indicate that the processor 120 included in the electronic device 101 or the processor 220 included in the stylus 201 causes other hardware to perform the specific operation. Optionally, when the electronic device 101 or the stylus 201 performs a specific operation, it may indicate that an instruction stored in a memory is executed or an instruction is stored in a memory, so that the processor 120 included in the electronic device 101 or the processor 220 included in the stylus 201 performs the specific operation.

[0110] According to various embodiments, at operation 603, the electronic device 101 may detect the insertion of the stylus 201. For example, the electronic device 101 may detect the insertion of the stylus 201 based on a received signal received from the stylus 201 in response to a transmitted signal transmitted by the digitizer. It will be understood by those of ordinary skill in the art that the method of detecting the insertion is not limited. At operation 605, the electronic device 101 may perform an initialization operation, and may send, for example, a reset command to the stylus 201. At operation 607, the stylus 201 may perform a reset operation. For example, the stylus 201 may release an existing BLE connection. At operation 609, the stylus 201 may perform an advertising operation. For example, the stylus 201 may broadcast an advertising signal. At operation 611, the electronic device 101 may identify the inserted stylus 201. The electronic device 101 may identify the inserted stylus 201 based on the received advertising signal. At operation 613, the electronic device 101 may request a communication connection. For example, the electronic device 101 may transmit a connection request signal in response to the advertisement signal. In operation 615 , the stylus pen 201 may establish a communication connection with the electronic device 101 .

[0111] Figure 6B is a flowchart for describing operations of a stylus pen and an electronic device when the stylus pen is removed from the electronic device according to an embodiment of the present disclosure.

[0112] Reference Figure 6B , at operation 621, the stylus 201 may be removed from the accommodation space in the electronic device 101. For example, the user may take out the stylus 201 from the accommodation space in the electronic device 101. At operation 623, the electronic device 101 may detect the removal of the stylus 201. For example, the electronic device 101 may detect the removal of the stylus 201 based on not receiving a signal from the stylus 201, but there is no limitation on the method for detecting the removal. At operation 625, the stylus 201 may detect the removal of the stylus 201. For example, the stylus 201 may detect the removal of the stylus 201 based on not receiving a signal from the electronic device 101, but there is no limitation on the method for detecting the removal. When separation is detected, the stylus 201 may exchange parameters (e.g., connection interval and / or slave device waiting time) with the electronic device 101.

[0113] According to various embodiments, at operation 627, the stylus pen 201 may activate the acceleration sensor based on the removal detection. At operation 629, the stylus pen 201 may sense acceleration information of the stylus pen 201 via the activated acceleration sensor. Although not shown, the stylus pen 201 may send the sensed acceleration information to the electronic device 101. In various embodiments, the electronic device 101 may perform an operation based on the received acceleration information. In various embodiments, the stylus pen 201 may activate the acceleration sensor, and the gyro sensor consuming relatively high power may be set to remain in a deactivated state.

[0114] According to various embodiments, at operation 631, the stylus pen 201 may recognize an input through a button (e.g., button 337). At operation 633, when the button input is recognized, the stylus pen 201 may activate the gyro sensor. At operation 635, the stylus pen 201 may sense rotation information via the activated gyro sensor. At operation 637, the stylus pen 201 may send information based on the sensing result. For example, the stylus pen 201 may send sensing information acquired via the acceleration sensor and the gyro sensor to the electronic device 101. Optionally, based on the sensing information acquired via the acceleration sensor and the gyro sensor, the stylus pen 201 may recognize the coordinates (e.g., two-dimensional coordinates or three-dimensional coordinates) of the stylus pen 201, and may send the recognized coordinates to the electronic device 101. Optionally, based on the sensing information acquired via the acceleration sensor and the gyroscope sensor, the stylus 201 may identify displacement information about the coordinates (e.g., two-dimensional coordinates or three-dimensional coordinates) of the stylus 201, and may send the identified displacement information to the electronic device 101. At operation 639, the electronic device 101 may perform an operation based on the received information. When the sensing information is received, the electronic device 101 may identify the position information of the stylus 201 based on the sensing information, may identify a gesture corresponding to the position information, and may perform an operation corresponding to the gesture. When the position information of the stylus 201 is received, the electronic device 101 may identify a gesture corresponding to the position information, and may perform an operation corresponding to the gesture. For example, the stylus 201 may send information to the electronic device 101 until the input of the pen button is released. The electronic device 101 may recognize a gesture based on the position information of the stylus 201, which has been recognized until the release of the button input is detected. When the release of the button input is detected, the stylus 201 may deactivate the gyroscope sensor again. However, in various embodiments, the stylus 201 may activate both the gyro sensor and the acceleration sensor from the time the removal is detected. In this case, the position information of the stylus 201 before the button input may be used to correct the gesture direction, and the gesture recognition accuracy may be improved. For example, the electronic device 101 may recognize the initial direction information of the stylus 201, and may recognize the gesture using displacement based on the initial direction information.

[0115] Figure 6C is a diagram illustrating operations of an electronic device and a stylus pen when the stylus pen is inserted into the electronic device according to an embodiment of the present disclosure.

[0116] Reference Figure 6C, at operation 661, the stylus 201 may be inserted into the accommodation space in the electronic device 101. For example, the stylus 201 may be removed from the accommodation space after the initial insertion and then may be reinserted. At operation 663, the electronic device 101 may detect the insertion of the stylus 201. At operation 665, the electronic device 101 may instruct the stylus 201 to activate charging. The electronic device 101 may indicate charging activation based on, for example, sending a signal with a pattern through the accommodation space coil or sending a communication signal through the communication module. At operation 667, the electronic device 101 may start the charging mode. At operation 669, the stylus 201 may detect the insertion of the stylus 201. The stylus 201 may recognize whether the stylus 201 is inserted based on information received from the electronic device 101 or the amplitude of the voltage applied to the resonant circuit (or rectifier output terminal) of the stylus 201. At operation 671, the stylus 201 may deactivate the sensor. The stylus 201 may disable some sensors, or may be set to ignore sensor deactivation. At operation 673, the electronic device 101 and the stylus 201 may perform a charging operation. In various embodiments, the charging operation of operation 673 may be performed immediately after the charging mode of operation 667 is started, and there is no limitation on its execution time.

[0117] Figure 7 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0118] Reference Figure 7 , at operation 701, the electronic device 101 may identify a gesture start event. For example, the electronic device 101 may identify a gesture start event based on information included in a communication signal from the stylus 201. The gesture start event may be, for example, detection of a pressed state of a button (e.g., button 337) provided on the stylus 201. The button may be implemented as any one of a physical key, a touch key, a motion key, a pressure key, and a keyless solution, and there is no limitation on its implementation form. After identifying the pressed state of the button, the stylus 201 may send a communication signal including information indicating the pressed state of the button to the electronic device 101.

[0119] According to various embodiments, at operation 703, the electronic device 101 may identify whether a pen signal is detected. For example, the electronic device 101 may receive an EMR signal by an EMR method or an ECR signal by an ECR method from the stylus 201. Alternatively, the electronic device 101 may identify an electrical signal generated by the stylus by an AES method. For example, the electronic device 101 may apply a transmission signal to a loop coil included in the sensing panel 503. After applying the transmission signal, when the intensity of a signal output from the loop coil included in the sensing panel 503 is greater than or equal to a threshold intensity, the electronic device 101 may identify that a pen signal is detected. When the intensity of a signal output from the loop coil included in the sensing panel 503 is less than a threshold intensity, the electronic device 101 may identify that a pen signal is not detected. Alternatively, the electronic device 101 may perform detection of the stylus 201 or position detection of the stylus 201 by identifying a capacitance change caused by a touch of the stylus on the touch sensing panel, which may also be referred to as detecting a pen signal. Optionally, the electronic device 101 may also perform detection of the stylus pen 201 or position detection of the stylus pen 201 based on detection of an electrical signal generated by the pen according to the AES method, which may also be referred to as detecting a pen signal.

[0120] According to various embodiments, when it is recognized that a pen signal is detected (operation 703: Yes), at operation 705, the electronic device 101 may perform a first operation corresponding to the pen signal. The electronic device 101 may recognize the position of the stylus 201 on the electronic device 101 (e.g., the display device 160) based on the pen signal, and may perform a first operation corresponding to the recognized position of the stylus 201. For example, the electronic device 101 may perform a function corresponding to an icon displayed on the display device 160. Alternatively, the electronic device 101 may display a drawing object on the display device 160 according to the movement of the stylus 201. The electronic device 101 may perform a function corresponding to a state of pressing a button of the pen, and there is no restriction on the first operation. Alternatively, when the stylus 201 is within a recognizable range, the electronic device 101 may control a drawing function. For example, the stylus 201 may detect movement after activating an acceleration sensor, and may send the result of the detection to the electronic device 101. In another example, information related to movement occurring when the side surface of the stylus pen 201 is touched may be transmitted to the electronic device 101. When the stylus pen 201 is implemented in a non-button type, information related to movement occurring when the side surface is touched may be transmitted to the electronic device 101.

[0121] Based on the received information, the electronic device 101 according to various embodiments may provide at least one icon panel (e.g., composed of icons capable of changing the pen function) related to the pen function (e.g., thickness and / or color), or may change the pen function (e.g., thickness and / or color). Optionally, unlike a physical button input (a change in the pen signal frequency), the electronic device 101 may display additional information about the content being displayed through movement detected by an acceleration sensor of the stylus 201 (e.g., movement during a touch input of the side surface of the stylus).

[0122] According to various embodiments, when the pen signal is not detected (operation 703: No), at operation 707, the electronic device 101 may receive information about the position of the stylus 201 (e.g., information about the motion sensor data, coordinates based on the motion sensor data, and / or displacement based on the motion sensor data about the position of the stylus 201). The electronic device 101 may receive at least one communication signal from the stylus 201, and may recognize the information about the position of the stylus 201 included in each of the one or more received communication signals. The information about the position of the stylus 201 may include the coordinates or displacement of the stylus 201. The electronic device 101 may recognize the time series position of the stylus 201. At operation 709, the electronic device 101 may check a gesture based at least in part on the information about the position of the stylus 201. Optionally, the electronic device 101 may receive information about the position of the stylus 201 even when the pen signal is detected, but in this case, the signal may be ignored. At operation 711, the electronic device 101 may perform a second operation corresponding to the recognized gesture. The second operation may be set to be different from the first operation. The electronic device 101 may map an operation corresponding to the gesture, and may recognize and execute the mapped operation.

[0123] Fig. 8A is a flowchart for describing a method of operating an electronic device according to an embodiment of the present disclosure.

[0124] Reference Fig. 8A , at operation 801, the electronic device 101 may recognize that a button (e.g., button 337) of the stylus pen 201 is pressed. The electronic device 101 may receive a communication signal including information indicating a pressed state of the button from the stylus pen 201. At operation 803, the electronic device 101 may recognize whether a pen signal is detected. When it is recognized that a pen signal is detected (operation 803: Yes), at operation 805, the electronic device 101 may perform a first operation corresponding to the pen signal and / or the pressing of the button. When it is recognized that a pen signal is not detected (operation 803: No), at operation 807, the electronic device 101 may receive information about the position of the stylus pen 201. The electronic device 101 may recognize information about the time series position of the stylus pen 201.

[0125] According to various embodiments, at operation 809, the electronic device 101 may identify whether the button of the stylus 201 is released. When the button is released, the stylus 201 according to various embodiments may send a communication signal including information indicating a release state to the electronic device 101. When the release of the button is not recognized (operation 809: No), the electronic device 101 may continue to receive information about the position of the stylus 201. Therefore, the electronic device 101 may acquire information about the time series position of the stylus 201. When the release of the button is recognized (operation 809: Yes), at operation 811, the electronic device 101 may recognize a gesture based at least in part on the received information about the position of the stylus 201. At operation 813, the electronic device 101 may perform a second operation corresponding to the recognized gesture. As described above, the electronic device 101 may recognize a gesture using information about the position during the period in which the button of the stylus 201 is pressed.

[0126] Figure 8B is a flowchart for describing operations of an electronic device and a stylus pen according to an embodiment of the present disclosure.

[0127] Reference Figure 8B, at operation 821, the stylus pen 201 may recognize that a button (e.g., button 337) of the stylus pen 201 is pressed. It is assumed that the stylus pen 201 has established a communication connection with the electronic device 101. At operation 823, the stylus pen 201 may send a communication signal including information about the pressing of the button of the stylus pen. At operation 825, the electronic device 101 may start gesture recognition based on the received information about the pressing of the button of the pen. The stylus pen 201 may recognize, for example, information about the displacement of the stylus pen 201 based on information sensed by the gyro sensor and the acceleration sensor. The stylus pen 201 may recognize the displacement in a two-dimensional (or three-dimensional) coordinate system based on the sensed information measured during a unit time. The stylus 201 may send a first displacement (ΔX1, ΔY1) at operation 827, a second displacement (ΔX2, ΔY2) at operation 829, a third displacement (ΔX3, ΔY3) at operation 831, and an nth displacement (ΔXn, ΔYn) at operation 833. Based on the received displacements, the electronic device 101 may identify the time series position of the stylus 201 in the coordinate system. For example, the electronic device 101 may identify the second position by applying (e.g., adding) the first displacement to the first position in the coordinate system. The electronic device 101 may identify multiple positions in the coordinate system by sequentially applying the displacements. At operation 835, the stylus 201 may detect the release of a button of the stylus 201. At operation 837, the stylus 201 may send information about the release of the button of the stylus 201 to the electronic device 101. At operation 839, the electronic device 101 may recognize a gesture based on the received information. The electronic device 101 may recognize a gesture using multiple positions recognized in the coordinate system. A detailed example of gesture recognition will be described later.

[0128] exist Figure 8B In the embodiment of FIG. 1 , although the stylus pen 201 is shown as sending information including displacement, this is only an example. The stylus pen 201 may send the sensing information to the electronic device 101 without change, or may send position information (eg, coordinate values) to the electronic device 101. Figure 8B , it is shown that information indicating the pressing or releasing of a button is transmitted in a communication signal different from information associated with the position of the stylus pen 201. However, this is also only an example, and these two pieces of information may be transmitted using one signal.

[0129] Fig. 9 is a diagram for describing an operation performed according to a distance between an electronic device and a stylus pen according to an embodiment of the present disclosure.

[0130] Reference Fig. 9In operation 901, the electronic device 101 and the stylus pen 201 may establish pairing (e.g., communication connection). When the stylus pen 201 is located within the recognizable area of ​​the electronic device 101, the electronic device 101 may detect a pen signal from the stylus pen 201. When the stylus pen 201 is located outside the recognizable area of ​​the electronic device 101, the electronic device 101 may not be able to detect the pen signal from the stylus pen 201.

[0131] When the stylus 201 is outside the recognizable area of ​​the electronic device 101, the user 900 may push the button 337 of the stylus 201. The user 900 may release the button 337 after moving the stylus 201 between a plurality of points 911, 912, 913, 914, 915, 916, 917, and 918 while the button 337 is pressed. The stylus 201 may activate the gyro sensor and the acceleration sensor while the button is pressed, and may recognize information about the position based on the sensing data acquired by the activated sensors. For example, the stylus 201 may recognize the coordinates of the stylus 201, and may send the coordinates to the electronic device 101. The electronic device 101 may recognize a gesture (e.g., a gesture of moving to the right) based on the received coordinates of the stylus 201. The electronic device 101 may perform an operation corresponding to the gesture, and may display a screen 931 indicating the result of the operation. Depending on the implementation, the electronic device 101 may execute a function mapped to a gesture, or may execute cursor movement in an air mouse mode. For example, when it is recognized that the stylus 201 does not move and a button (e.g., button 337) provided on the stylus 201 remains in a pressed state, the electronic device 101 may change the mode from the gesture mode to the air mouse mode, or vice versa. The electronic device 101 may perform a mode change in response to a long press of a button.

[0132] When the stylus 201 is located in the recognizable area of ​​the electronic device 101, the user 900 may push the button 337 of the stylus 201. In addition, the user 900 may bring the stylus 201 into contact with the first point 903 of the electronic device 101. The user 900 may move the stylus 201 from the first point 903. The electronic device 101 may recognize the time series change of the position of the stylus 201 based on the pen signal from the stylus 201. Based on the position change, the electronic device 101 may display, for example, a screen 904 indicating a drawing result. The electronic device 101 may display a screen corresponding to the hovering function and a screen corresponding to the drawing result. Optionally, when the stylus 201 is located in the recognizable range, the electronic device 101 may control the drawing function. For example, the stylus 201 may detect movement after activating the acceleration sensor, and the result of the detection may be sent to the electronic device 101. In an example, the electronic device 101 may receive information about the movement of the pen that occurs when the side surface of the stylus 201 is touched. Based on the received information, the electronic device 101 may provide at least one icon panel (e.g., composed of icons capable of changing the pen function) related to the pen function (e.g., thickness and / or color), or may change the pen function (e.g., thickness and / or color). Alternatively, unlike a physical button input (a change in the pen signal frequency), the electronic device 101 may display additional information about the content being displayed through movement detected by an acceleration sensor of the stylus 201 (e.g., a side surface touch input of the stylus).

[0133] Within the recognizable range, the electronic device 101 may activate the position gesture engine based on the detection of the pen signal, and may perform an operation based on the position of the stylus identified using the activated position gesture engine. For example, based on the pen signal identified in the recognizable range, the electronic device 101 may identify the position of the stylus 201, and may perform an operation based on the identified position. In addition, the electronic device 101 may use the acceleration sensing information of the stylus 201 received in addition to perform additional operations, or may perform operations identified using the identified position and acceleration sensing information together. Additional operations may include, for example, providing at least one icon panel (e.g., composed of icons capable of changing the pen function) associated with the pen function (e.g., thickness and / or color), changing the pen function (e.g., thickness and / or color), or displaying additional information about the content being displayed. Based on the fact that no pen signal is detected outside the recognizable range, the electronic device 101 may activate the direction gesture engine, and may use the activated direction gesture engine to recognize gestures. The direction gesture engine may include, for example, a circle gesture detector capable of detecting a circle gesture and a swipe gesture detector capable of detecting a swipe gesture, and there is no limitation on the type of gesture or the type of detector for detecting the gesture. The detector will be described later.

[0134] Fig.10 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure. Fig.11 describe Fig.10 Embodiment of the invention.

[0135] Fig.11 is a diagram for describing a detector according to an embodiment of the present disclosure.

[0136] Reference Fig.10 In operation 1001, the electronic device 101 may identify a gesture start event. In operation 1003, the electronic device 101 may receive information about the position of the stylus. In operation 1005, the electronic device 101 may identify a candidate gesture based at least in part on the information about the position of the stylus.

[0137] Reference Fig.11 , the electronic device 101 may receive coordinates from the stylus 201 via the antenna 191 and the communication module 190. The electronic device 101 may provide the received coordinates to the swipe gesture detector 1110 and the circle gesture detector 1120. When the electronic device 101 receives raw data or displacement, the electronic device 101 may convert the data or displacement into coordinates, and may provide the coordinates to the detectors 1110 and 1120. The detector may output a specific gesture based on the received coordinates using, for example, a determination algorithm executed by the processor 120. For example, the swipe gesture detector 1110 may output a left swipe gesture 1111, a right swipe gesture 1112, an upward swipe gesture 1113, or a downward swipe gesture 1114 as a candidate gesture based on the received coordinates. When there is no gesture that satisfies the condition, the swipe gesture detector 1110 may not output a gesture. Based on the received coordinates, the circle gesture detector 1120 may output at least one of a clockwise gesture 1121 or a counterclockwise gesture 1122 as a candidate gesture. When there is no gesture that satisfies the condition, the circle gesture detector 1120 may not output a gesture. The detection algorithms of the swipe gesture detector 1110 and the circle gesture detector 1120 will be described later. The swipe gesture detector 1110 and the circle gesture detector 1120 may be executed at least simultaneously, but may be executed sequentially depending on the implementation. Optionally, all detectors 1110 and detectors 1120 may be executed, but only detectors corresponding to gestures supported by the application executed in the foreground may be executed. In addition, it can be understood by a person of ordinary skill in the art that the types and numbers of detectors 1110 and detectors 1120 are illustrative and not limited thereto.

[0138] According to various embodiments, in operation 1007, the electronic device 101 may identify whether a gesture termination event is detected. Before a gesture termination event is detected (operation 1007: No), the electronic device 101 may maintain the operation of identifying candidate gestures. For example, the electronic device 101 may provide the newly received coordinates to each of the detector 1110 and the detector 1120 while maintaining the execution of the detector 1110 and the detector 1120. When a termination event is detected (operation 1007: Yes), in operation 1009, the electronic device 101 may select a gesture from the candidate gestures. For example, Fig.11 As shown, the processor 120 may execute a gesture determiner 1130, and the gesture determiner 1130 may be an algorithm for selecting any one from one or more candidate gestures. In various embodiments, the electronic device 101 may operate in a manner that ultimately selects any one of the one or more candidate gestures, or when a gesture termination event is recognized when the candidate gesture is changed and managed in real time, the electronic device 101 may operate to select a candidate gesture as a gesture. There is no restriction on the gesture selection method. In operation 1011, the electronic device 101 may execute a function corresponding to the selected gesture. For example, the electronic device 101 may send a gesture or a function execution command mapped to the gesture to any one of the applications 1141, 1142, 1143, and 1144 included in the loaded application set 1140. The application that receives the gesture or the function execution command mapped to the gesture may be preset or selected as an application running in the foreground. The selection algorithm of the gesture determiner 1130 will be described in detail later. The function mapped to the gesture for each application may be different. Table 1 shows the functions of the gestures mapped to each application according to various embodiments.

[0139] Table 1

[0140]

[0141] Referring to Table 1, for a state (e.g., a state before imaging or a state during imaging) of an application (e.g., a camera application), the electronic device 101 may map different functions to gestures. Alternatively, for a specific application (e.g., a music application), a function may not be mapped to a specific gesture (e.g., an up / down swipe gesture).

[0142] Fig.12 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0143] Reference Fig.12, in operation 1201, the electronic device 101 may identify the coordinates of the position of the stylus 201. In operation 1203, the electronic device 101 may perform scoring using at least one parameter based on the coordinates of the position of the stylus 201. For example, the electronic device 101 may identify the direction of displacement between adjacent coordinates, the magnitude of displacement, and the direction and magnitude of the reference starting point of any coordinate as parameters in all coordinates, and may perform scoring using the parameters. The parameters and scoring methods may be set differently for each detector. In operation 1205, the electronic device 101 may identify whether the scoring result satisfies a specific condition. When the specific condition is met (operation 1205: yes), the electronic device 101 may identify a candidate gesture in operation 1207. When the specific condition is not met (operation 1205: no), the electronic device 101 may identify whether a gesture termination event is detected in operation 1211. For example, the electronic device 101 may identify the corresponding gesture, and then may perform calculations on the coordinates continuously input thereafter. The electronic device 101 may store the recognized gesture, and then may update the stored gesture to another gesture based on the continuously input coordinates. That is, the electronic device 101 may perform scoring until a specific gesture does not satisfy a specific condition, and when another scoring result does not satisfy the specific condition, the electronic device 101 may determine that the corresponding gesture is not detected. As will be described later, the electronic device 101 may be in a non-application state (e.g., Fig.13 The electronic device 101 may identify the previously updated gesture as a candidate gesture.

[0144] According to various embodiments, at operation 1211, the electronic device 101 may identify whether a gesture termination event is detected. When a gesture termination event is not detected (operation 1211: No), the electronic device 101 may return to operation 1201 and may identify new coordinates of the stylus 201. The electronic device 101 may also perform scoring by identifying parameters of the new coordinates, and may determine whether to classify the corresponding gesture as a candidate gesture based on the result of the scoring. When a gesture termination event is detected (operation 1211: Yes), the electronic device 101 may select a candidate gesture as a gesture at operation 1213. As described above, when coordinates are accumulated, a specific gesture may be classified as a candidate gesture, and may then be managed as not detected due to additional coordinates.

[0145] Fig.13 is a diagram for describing switching between states set for candidate gesture determination according to an embodiment of the present disclosure.

[0146] Reference Fig.13, the electronic device 101 may set an idle state 1301, an interested state 1302, a satisfied state 1303, and a non-application (N / A) state 1304 in response to a specific gesture. The idle state 1301 may be a standby state for receiving coordinates when paired with the stylus 201. When the coordinates of a position are received from the stylus 201, the electronic device 101 may enter the interested state 1302. In the interested state 1302, the electronic device 101 may perform scoring using parameters identified from the coordinates. When the scoring result satisfies a specific condition, the electronic device 101 may identify a gesture corresponding to the satisfied state 1303, and may then continue to perform calculations on the input coordinates. That is, the electronic device 101 may perform scoring until the specific condition is not met. In addition, when the scoring result in the interested state 1302 does not meet a specific condition, the electronic device 101 may classify the corresponding gesture as a non-application state 1304. In addition, when the other coordinates do not satisfy the scoring result in the satisfied state 1303, the electronic device 101 may store the gesture that satisfies the previous scoring as a candidate gesture (the last gesture), and the corresponding gesture may be classified as the non-application state 1304. After entering the non-application state 1304, the electronic device 101 may enter the interested state 1302 when the coordinates are received again. Subsequently, when the scoring is performed by using the parameters identified from the continuously input coordinates to meet the specific conditions, the electronic device 101 may enter the satisfied state 1303 of the corresponding gesture. After entering the satisfied state 1303, when the scoring is continued until the specific conditions are not met and the conditions are thus not met, the electronic device 101 may store the gesture that satisfies the previous scoring as a candidate gesture (ongoing gesture). In this case, the electronic device 101 may compare the guide distance (e.g., moving distance) of the ongoing gesture with the guide distance (e.g., moving distance) of the stored gesture (the last gesture), and may store the gesture with the longer guide distance as a candidate gesture. The comparison of the parameters between the ongoing gesture and the last gesture may be referred to as the parameter comparison between the candidate gestures.

[0147] According to various embodiments, the electronic device 101 may select any one of the detected candidate gestures when a termination event is detected, rather than replacing and storing the candidate gestures in real time. For example, the electronic device 101 may recognize at least one gesture detected before a termination event (e.g., a button release), and may select any one of one or more gestures when a termination event is detected. Those of ordinary skill in the art will appreciate that there is no limitation on the method of determining the candidate gestures.

[0148] Fig.14 is a diagram for describing a detection process performed by a swipe gesture detector according to an embodiment of the present disclosure.

[0149] Table 2 shows a scoring method and a determination criterion for each parameter corresponding to a swipe gesture according to various embodiments.

[0150] Table 2

[0151]

[0152]

[0153] Reference Fig.14 , based on the scoring method in Table 2, Fig.14 For example, the user may move the stylus 201 while pressing the button 337 of the stylus 201. In the state where the button 337 is pressed, the electronic device 101 may identify the first coordinate 1401, the second coordinate 1402, the third coordinate 1403, the fourth coordinate 1404, the fifth coordinate 1405, and the sixth coordinate 1406. For the first coordinate 1401, it is recognized that the stylus moves "b" or more to the right with reference to the reference coordinate (0,0). Therefore, it can be recognized that the conditions of the first and second items of "NOK" are not satisfied but the condition of "OK" is satisfied. Therefore, the electronic device 101 may perform scoring to increase the number of OKs by 1 for the first coordinate 1401. For the second coordinate 1402, it is recognized that the stylus moves "b" or more to the left and "a" or more in the right angle direction. It can be recognized that the condition of the first item of "NOK" is satisfied and the exception condition of the first item of "NOK" is satisfied (when the moving distance in the traveling direction is greater than the traveling distance in the right angle direction). Therefore, the electronic device 101 may perform a scoring of increasing the number of OK by 1 for the second coordinate 1402. Through the above process, the electronic device 101 may perform a scoring of increasing each of the number of NOK, OK, and critical by 1 for each of the third coordinate 1403, the fourth coordinate 1405, the fifth coordinate 1405, and the sixth coordinate 1406. In particular, with reference to the fourth coordinate 1404, it may be recognized that the displacement in the x-axis direction of the fifth coordinate 1405 is to the left, and the scoring may be performed to increase the number of critical corresponding to the reverse movement by 1. The electronic device 101 may recognize that the number of critical is 4 or less as a threshold value and the number of NOK is 10 or less as a threshold value. In addition, the electronic device 101 may recognize that the moving distance in the right direction is d or more. Based on the fact that all the conditions of the first and second items of the "determination criteria" are met, the electronic device 101 may recognize that the gesture corresponding to the coordinates of the stylus 201 is a swipe to the right.

[0154] Fig.15 is a diagram for describing a detection process performed by a circle gesture detector according to an embodiment of the present disclosure.

[0155] Table 3 shows a scoring method and a determination criterion for each parameter corresponding to a circle gesture according to various embodiments.

[0156] Table 3

[0157]

[0158] Reference Fig.15 , based on the scoring method in Table 3, Fig.15 For example, the user may move the stylus 201 while pressing the button 337 of the stylus 201. In the state where the button 337 is pressed, the electronic device 101 may identify the first coordinate 1501, the second coordinate 1502, the third coordinate 1503, the fourth coordinate 1504, the fifth coordinate 1505, the sixth coordinate 1506, the seventh coordinate 1507, the eighth coordinate 1508, the ninth coordinate 1509, the tenth coordinate 1510, and the eleventh coordinate 1511. It can be recognized that the angle formed by the reference coordinate (0,0), the first coordinate 1501, and the second coordinate 1502 is formed counterclockwise, does not satisfy the condition of "NOK" in Table 3, but satisfies the condition of "OK". The electronic device 101 may perform scoring to increase the number of OK by 1 for the second coordinate 1502. In addition, the electronic device 101 may recognize that the angle formed by the second coordinate 1502, the third coordinate 1503, and the fourth coordinate 1504 is a straight line, and therefore, the scoring may be performed to increase the number of NOK by 1 for the fourth coordinate 1504. In addition, the electronic device 101 may recognize that the angle formed by the fifth coordinate 1505, the sixth coordinate 1506, and the seventh coordinate 1507 corresponds to the clockwise direction, and the scoring may be performed to increase the number of critical by 1 for the seventh coordinate 1507. The electronic device 101 may sequentially score until the eleventh coordinate 1511, and may recognize OK (8), NOK (1), and critical (1) as cumulative scoring results. The electronic device 101 may recognize that the corresponding scoring results satisfy the conditions of the first and second items of the "determination standard" of Table 3. In addition, the electronic device 101 may recognize w as the maximum deviation in the x-axis direction of all coordinates and h as the maximum deviation in the y-axis direction of all coordinates, and may confirm that w / h and h / w ​​are both 0.35 or more. Therefore, the electronic device 3 can recognize that the condition of the third item of the "determination standard" is satisfied. The electronic device 101 can recognize that the angle a formed between the reference coordinate (0,0) as the starting point, the sixth coordinate 1506 as the middle point, and the eleventh coordinate 1511 as the end point is less than 90 degrees, and thus can recognize that the condition of the fourth item of the "determination standard" is satisfied. Based on the fact that the conditions of the first to fourth items of the "determination standard" are satisfied, the electronic device 101 can recognize that the gesture is a counterclockwise circle gesture.

[0159] In addition, the conditions of Tables 2 and 3 above are merely illustrative and may be modified. In addition, the electronic device 101 may apply different scoring methods and determination conditions depending on the state of the electronic device 101 or the user. For example, different scoring methods and determination conditions may be applied to a situation where the electronic device 101 is stationary, a situation where the user of the electronic device is walking, and a situation where the user of the electronic device is running, respectively. In addition, the electronic device 101 may adjust the scoring method and determination conditions based on the gesture habits of each user, and may apply the adjusted scoring method and determination conditions later.

[0160] Fig.16 is a diagram for describing a detection process performed by a shake gesture detector according to an embodiment of the present disclosure.

[0161] Reference Fig.16 According to various embodiments based on information about the position of the button 337 of the stylus 201 in a state where the button 337 is pressed, the electronic device 101 may recognize a first coordinate 1601, a second coordinate 1602, a third coordinate 1603, a fourth coordinate 1604, a fifth coordinate 1605, a sixth coordinate 1606, a seventh coordinate 1607, an eighth coordinate 1608, a ninth coordinate 1609, a tenth coordinate 1610, an eleventh coordinate 1611, a twelfth coordinate 1612, a thirteenth coordinate 1613, a fourteenth coordinate 1614, a fifteenth coordinate 1615, a sixteenth coordinate 1616, a seventeenth coordinate 1617, and an eighteenth coordinate 1618. The electronic device 101 may recognize that there are three or more cases in which three angles (e.g., a, b, c, and d) formed by three adjacent coordinate groups are 90 degrees or less. The electronic device 101 may recognize that the deviation between the center points of the angles of 90 degrees or less (e.g., the fourth coordinate 1604, the eighth coordinate 1608, the twelfth coordinate 1612, and the fifteenth coordinate 1615) is greater than w. The electronic device 101 may recognize that the type of gesture is a shaking gesture based on satisfying the above conditions. In addition, those of ordinary skill in the art will understand that the above criteria for determining a shaking gesture are merely illustrative and may also be modified.

[0162] Fig.17 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure. Fig.18 describe Fig.17 Embodiment of the invention.

[0163] Fig.18 is a diagram illustrating movement of a stylus pen according to an embodiment of the present disclosure.

[0164] Reference Fig.17, at operation 1701, the electronic device 101 may recognize at least one candidate gesture. For example, the electronic device 101 may recognize a plurality of time-series coordinates of the stylus 201. The electronic device 101 may input the plurality of recognized coordinates into a plurality of gesture detectors (or detection algorithms). When the scoring result satisfies a specific determination condition, the gesture detector may output a corresponding gesture, and the electronic device 101 may recognize at least one candidate gesture.

[0165] Reference Fig.18 , according to the movement of the stylus pen 201 in the state where the button 337 is pressed, the electronic device 101 may recognize the first coordinate 1801, the second coordinate 1803, the third coordinate 1805, the fourth coordinate 1807, the fifth coordinate 1809, the sixth coordinate 1811, the seventh coordinate 1813, the eighth coordinate 1815, the ninth coordinate 1817, the tenth coordinate 1819, and the eleventh coordinate 1821. The electronic device 101 may input the first coordinate 1801 to the eleventh coordinate 1821 into the circle gesture detector, and may recognize the clockwise circle gesture as a candidate gesture. The electronic device 101 may input the first coordinate 1801 to the eleventh coordinate 1821 into the swipe gesture detector, and may recognize the right swipe gesture as a candidate gesture.

[0166] According to various embodiments, in operation 1703, the electronic device 101 may identify at least one travel distance corresponding to each of one or more candidate gestures. For example, the electronic device 101 may confirm that the travel distance of the clockwise circle gesture is "a" and the travel distance of the right swipe gesture is "b". In operation 1705, the electronic device 101 may select a gesture based on the result of comparing one or more travel distances. For example, the electronic device 101 may select a gesture with the longest travel distance among the candidate gestures based on a simple comparison of the travel distances. Optionally, the electronic device 101 may weight the travel distances and may select a gesture based on the result of comparing the weighted travel distances. For example, the electronic device 101 may be set to give a higher weight to gestures detected later in sequence. According to various embodiments, the electronic device 101 may select a gesture among the candidate gestures based on the scoring result used in the gesture detection process. For example, a gesture with the best scoring result may be selected. In addition, the electronic device 101 may be set to perform gesture processing when the corresponding gesture accounts for more than 50% of the total travel distance.

[0167] Fig.19 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0168] Reference Fig.19, in operation 1901, the electronic device 101 may recognize a gesture. In operation 1903, the electronic device 101 may recognize whether the application is an application that can perform gesture commands. For example, the electronic device 101 may recognize whether the application being executed in the foreground is an application that can perform gesture commands. In the case where the application is an application where gesture commands are impossible (operation 1903: No), in operation 1905, the electronic device 101 may output a message indicating that the application is not capable of performing gesture commands. When the application is an application where gesture commands can be performed (operation 1903: Yes), in operation 1907, the electronic device 101 may recognize whether the application can process the recognized gesture. When it is recognized that the application cannot process the recognized gesture (operation 1907: No), in operation 1909, the electronic device 101 may output a message indicating that it is impossible to process the recognized gesture. When it is recognized that the application can process the recognized gesture (operation 1907: Yes), in operation 1911, the electronic device 101 may recognize whether there is a function mapped to the recognized gesture. When there is no function mapped to the recognized gesture (operation 1911: No), the electronic device 101 may output a message indicating that there is no function mapped to the recognized gesture in operation 1913. When there is a function mapped to the recognized gesture (operation 1911: Yes), the electronic device 101 may provide the mapped function in operation 1915. According to various embodiments, for example, when an undefined gesture is detected a certain number of times or more (e.g., twice), the electronic device 101 may display a screen providing a gesture guide.

[0169] Fig. 20A is a diagram illustrating an electronic device that performs an operation corresponding to a gesture according to an embodiment of the present disclosure.

[0170] Reference Fig. 20A , according to various embodiments, the electronic device 101 may receive information about multiple positions of the stylus 201 according to the movement of the stylus 201 in a state where the button 337 is pressed. The electronic device 101 may recognize a circle gesture and a swipe gesture as candidate gestures, and may ultimately select the swipe gesture. The electronic device 101 may recognize that the function of the swipe gesture mapped to the gallery application is a gallery image switch. Therefore, the electronic device 101 may switch from the first image 2001, which is the execution screen of the gallery application, to a screen including at least a portion of the first image 2001 and at least a portion of the second image 2002, and may display a screen including at least a portion of the first image 2001 and at least a portion of the second image 2002. Although not shown, the electronic device 101 may then display the entire second image 2002, and therefore, image switching may be performed.

[0171] Fig. 20B is a diagram illustrating a screen displayed by an electronic device according to an embodiment of the present disclosure.

[0172] Fig. 20C is a diagram illustrating a screen displayed by an electronic device according to an embodiment of the present disclosure.

[0173] Fig.20D is a diagram illustrating a screen displayed by an electronic device according to an embodiment of the present disclosure.

[0174] Fig.20E is a diagram illustrating a screen displayed by an electronic device according to an embodiment of the present disclosure.

[0175] Fig.20F is a diagram illustrating a screen displayed by an electronic device according to an embodiment of the present disclosure.

[0176] Reference Fig. 20B , the electronic device 101 according to various embodiments may display a first icon 2021 on the application execution screen 2020. The electronic device 101 may display the first icon 2021 on the application execution screen 2020 based on execution of an application (e.g., a gallery application) that supports gesture control and / or button control through the stylus 201. The first icon 2021 may indicate that control through the stylus 201 is possible.

[0177] Reference Fig. 20C According to various embodiments, the electronic device 101 may display the second icon 2022 on the application execution screen 2020. For example, the electronic device 101 may recognize a gesture and may display the second icon 2022 corresponding to the recognized gesture. Fig. 20C In the embodiment of the present invention, it is shown that the electronic device 101 displays the second icon 2022 corresponding to the left swipe gesture based on the confirmation of the left swipe gesture, but this is only exemplary. The electronic device 101 can display icons corresponding to various recognized gestures. The electronic device 101 can perform the function corresponding to the gesture. When the predetermined time exceeds or the operation corresponding to the gesture is completed, the electronic device 101 can change the second icon 2022 to the first icon 2021, and can display the first icon 2021.

[0178] Reference Fig.20D , according to various embodiments, the electronic device 101 may display a third icon 2041 on the application execution screen 2020. When gesture recognition fails, the electronic device 101 may display the third icon 2041. For example, when gesture recognition failure is recognized a threshold number of times (e.g., 3 times), the electronic device 101 may display the third icon 2041. However, the threshold number of times is not limited. The electronic device 101 may display a message window 2042 indicating that recognition of the gesture has failed and asking whether to switch to an application state for providing help. When a selection of the message window 2042 is recognized, the electronic device 101 may provide a screen (e.g., Fig.212102), which provides a description of the supported gestures.

[0179] Reference Fig.20E , the electronic device 101 according to various embodiments may display a message window 2043 indicating that the gesture is not supported on the application execution screen 2020. For example, the gallery application of the electronic device 101 may be set to provide functions corresponding to the left swipe gesture, the right swipe gesture, the upward swipe gesture, and the downward swipe gesture. When the electronic device 101 recognizes the circle gesture, the electronic device 101 may recognize that there is no function mapped to the recognized gesture in the application (e.g., the application running in the foreground). In response to this, the electronic device 101 may display a message window 2043 indicating that the recognized gesture is not supported.

[0180] Reference Fig.20F , the electronic device 101 according to various embodiments may display a message window 2050 for describing the function mapped to the gesture on the application execution screen 2020. For example, the message window 2050 may include information indicating that a detailed item may be provided in response to an upward swipe gesture and a downward swipe gesture, a next item may be provided in response to a left swipe gesture, and a previous item may be provided in response to a right swipe gesture. The message window 2050 may include information indicating that no function is mapped to a clockwise circle gesture or a counterclockwise circle gesture. For example, the electronic device 101 may display the message window 2050 based on the detection of a hovering event of the icon 2021. However, it will be understood by those of ordinary skill in the art that there is no limitation on the calling event of the message window 2050.

[0181] Fig.21 is a diagram illustrating a gesture guide screen of an electronic device according to an embodiment of the present disclosure.

[0182] The electronic device 101 according to various embodiments may display, for example, a screen 2101 for guiding an upward swipe gesture and a screen 2102 for guiding a counterclockwise circle gesture. The screen 2101 and the screen 2102 may be images or video images. The electronic device 101 may receive information about the position from the stylus 201, and may determine whether a gesture corresponding to the guide screen is detected based on the received information. When a gesture is detected, the electronic device 101 may output a screen indicating a successful attempt, and when gesture detection fails, the electronic device 101 may output a screen requesting a retry. In various embodiments, the electronic device 101 may use information about the position of the stylus 201 detected during the display of the guide screen to identify and store a gesture detection algorithm dedicated to the user. Later, when it is recognized that the user is using the stylus 201, the electronic device 101 may use the gesture detection algorithm stored for each user.

[0183] Fig.22Ais a diagram illustrating a setting screen of an electronic device according to an embodiment of the present disclosure.

[0184] Fig. 22B is a diagram illustrating a setting screen of an electronic device according to an embodiment of the present disclosure.

[0185] Fig. 22C is a diagram illustrating a setting screen of an electronic device according to an embodiment of the present disclosure.

[0186] Reference Fig.22A , the electronic device 101 may display a first screen 2210 for selecting an application. The first screen 2210 may include an icon selection area 2220, and the icon selection area 2220 may include a plurality of icons 2221, 2222, 2223, 2224, 2225, 2226, 2227, and 2228 set to respectively execute applications. The electronic device 101 may provide a function of adding icons, and the user may add or delete an application that the user desires to control by gestures.

[0187] Reference Fig. 22B , the electronic device 101 may display a second screen 2230, through which a gesture of a specific application may be selected. For example, when an icon 2221 corresponding to a camera application of the first screen 2210 is selected, the second screen 2230 may be provided. The second screen 2230 may include an object 2231 representing the selected application and a toggle switch 2232 for setting an activation gesture control. In addition, the second screen 2230 may include objects 2235, 2236, 2237, and 2238 corresponding to the gesture, and the objects 2235, 2236, 2237, and 2238 may include text for a function currently mapped to the corresponding gesture. The second screen 2230 may include objects 2233 and 2234 for setting a function of a pen button and gesture control. When the objects 2233 and 2234 are selected, the electronic device 101 may provide a screen that enables setting a function corresponding to a corresponding pen button control command (e.g., single press / double press).

[0188] Reference Fig. 22C, based on the user input for selecting the object 2235, the electronic device 101 may display a selection window 2240 on the second screen 2230 in response to the upward gesture, on which a function that can be mapped is selectable. The selection window 2240 may include text indicating a function mapped in response to a current corresponding gesture (e.g., an upward gesture), and one or more objects 2242, 2243, 2244, 2245, and 2246, each of which indicates a corresponding one of the selectable functions. When any one of the one or more objects 2242, 2243, 2244, 2245, and 2246 is selected and an upward gesture is subsequently detected, the electronic device 101 may provide a function corresponding to the newly selected object.

[0189] Fig.23A is a flowchart for describing a method of operating a stylus pen according to an embodiment of the present disclosure.

[0190] Reference Fig.23A , at operation 2301, the stylus 201 may recognize the pressing of a button (e.g., button 337). At operation 2303, the stylus 201 may send information indicating the pressing of the button to the electronic device 101 through the formed communication connection. At operation 2305, the stylus 201 may recognize information about the position of the stylus 201 (e.g., coordinate information of the stylus 201 or displacement information of the stylus 201 in the coordinate system) based on sensing data recognized from one or more sensors. For example, when recognizing the pressing of the button, the stylus 201 may activate both the gyro sensor and the acceleration sensor to recognize information about the position. At operation 2307, the stylus 201 may send the recognized information to the electronic device 101. At operation 2309, the stylus 201 may recognize whether the release of the button is detected. When the release of the button is not detected (operation 2309: No), the stylus 201 may re-recognize the information about the position based on the sensing data, and then the re-recognized information may be repeatedly sent to the electronic device 101. When the release of the button is detected (operation 2309 : Yes), the stylus pen 201 may transmit information indicating the release of the button to the electronic device 101 at operation 2311 .

[0191] Fig. 23B is a flowchart for describing a method of operating a stylus pen according to an embodiment of the present disclosure.

[0192] Reference Fig. 23B, at operation 2321, the stylus 201 may recognize the pressing of a button (e.g., button 337). At operation 2323, the stylus 201 may recognize information about the position of the stylus 201 based on sensing data recognized from one or more sensors. At operation 2325, the stylus 201 may recognize whether the release of the button is detected. When the release of the button is not detected (operation 2325: No), the stylus 201 may collect sensing data and may repeatedly recognize information about the position of the stylus 201, whereby the stylus 201 may acquire information about the time series position of the stylus 201. When the release of the button is detected (operation 2325: Yes), at operation 2327, the stylus 201 may recognize a gesture based on the information about the position. The stylus 201 may recognize a gesture by a method at least partially the same as the method in which the electronic device 101 recognizes a gesture from the information about the position described above. At operation 2329, the stylus 201 may send information indicating the recognized information to the electronic device 101. The electronic device 101 may perform an operation corresponding to the information indicating the received gesture.

[0193] According to various embodiments, the electronic device 101 may include: a sensing panel 503; a communication module 190; at least one processor 120 electrically connected to the sensing panel 503 and the communication module 190; and a memory 130 electrically connected to the at least one processor 120. The memory 130 may store instructions configured to, when executed by the at least one processor 120, cause the at least one processor 120 to perform the following operations: based on detecting a pen signal from the stylus 201 through the sensing panel, perform a first operation based on recognition of the detected pen signal, and based on not detecting a pen signal from the stylus 201 through the sensing panel 503, receive at least one communication signal through the communication module 190, recognize a gesture included in the received at least one communication signal based on information about the position of the stylus 201, and perform a second operation corresponding to the recognized gesture.

[0194] According to various embodiments, the instructions may be configured to cause the at least one processor 120 to perform the following operations when executed by the at least one processor 120: receiving a first communication signal including first information indicating a start gesture input through the communication module 190 before receiving at least one communication signal; receiving a second communication signal including second information indicating a termination gesture input through the communication module 190 after receiving at least one communication signal; and recognizing a gesture based on identifying the second information included in the received second communication signal.

[0195] According to various embodiments, the first information includes information indicating that the button of the stylus pen 201 is in a pressed state, and the second information includes information indicating that the button of the stylus pen 201 is in a released state.

[0196] According to various embodiments, the information about the position of the stylus 201 may include at least one of sensing data acquired by a sensor of the stylus 201 , coordinates of the stylus 201 identified based on the acquired sensing data, or displacement of the stylus 201 identified based on the acquired sensing data.

[0197] According to various embodiments, the instructions may be configured to, when executed by the at least one processor 120, cause the at least one processor 120 to perform the following operations: identify one or more candidate gestures using information about the position of the stylus 201, and identify a gesture among the identified one or more candidate gestures.

[0198] According to various embodiments, the instructions may be configured to cause the at least one processor 120 to perform the following operations when executed by the at least one processor 120: input information about the position of the stylus 201 into multiple gesture detection algorithms, each of the multiple gesture detection algorithms determines whether a corresponding gesture has occurred, and identifies one or more gestures corresponding to one or more detection algorithms having processing results that satisfy specific conditions among the processing results of the multiple gesture detection algorithms as one or more candidate gestures.

[0199] According to various embodiments, the instructions may be configured to cause the at least one processor 120 to perform the following operations when executed by the at least one processor 120: identifying the travel distance of each of one or more candidate gestures, and identifying the candidate gesture corresponding to the maximum travel distance among the travel distances of the one or more candidate gestures as a gesture.

[0200] According to various embodiments, the instructions may also be configured to cause the at least one processor 120, when executed by the at least one processor 120, to recognize a gesture based on at least one of: a weight of each of one or more candidate gestures, a score of each of one or more candidate gestures, or a ratio of a travel distance of each of one or more candidate gestures to a total distance of the one or more candidate gestures.

[0201] According to various embodiments, the instructions may be configured to, when executed by the at least one processor 120 , cause the at least one processor 120 to perform the following operations: executing a function mapped to the recognized gesture as a second operation in an application running in the foreground of the electronic device 101 .

[0202] According to various embodiments, the instructions may be configured to cause the at least one processor 120 to perform the following operations when executed by the at least one processor 120: receiving acceleration information of the stylus 201 through the communication module 190 when a pen signal from the stylus 201 is detected, and identifying a first operation based on the detected pen signal and the acceleration information of the stylus 201.

[0203] According to various embodiments, the instructions may be configured to cause the at least one processor 120 to perform the following operations when executed by the at least one processor 120: based on a pen signal being detected, activating a position gesture engine and using the activated position gesture engine, and performing a first operation based on the position of the indicated stylus 201 identified by using the activated position gesture engine, and based on no pen signal being detected, activating a direction gesture engine, and recognizing a gesture by using the activated direction gesture engine.

[0204] According to various embodiments, the first operation includes at least one of: providing a pen function or providing additional information about the content being displayed.

[0205] According to various embodiments, the electronic device 101 may include: a communication module 190; at least one processor 120 electrically connected to the communication module 190; and a memory 130 electrically connected to the at least one processor 120. The memory 130 may store instructions configured to cause the at least one processor 120 to perform the following operations when executed by the at least one processor: receiving a first communication signal including first information indicating a start gesture input from the stylus 201 through the communication module 190; receiving a plurality of second communication signals from the stylus 201 through the communication module 190, identifying one or more candidate gestures based on information about the position of the stylus 201 included in the received plurality of second communication signals; receiving a third communication signal through the communication module 190 after receiving the plurality of second communication signals, identifying a gesture satisfying a specific condition among the identified one or more candidate gestures based on identifying the second information indicating a termination gesture input included in the third communication signal, and performing an operation corresponding to the identified gesture.

[0206] According to various embodiments, the first information includes information indicating that the button of the stylus pen 201 is in a pressed state, and the second information includes information indicating that the button of the stylus pen 201 is in a released state.

[0207] According to various embodiments, the information about the position of the stylus 201 may include at least one of sensing data acquired by a sensor of the stylus 201 , coordinates of the stylus 201 recognized based on the acquired sensing data, or displacement of the stylus 201 recognized based on the acquired sensing data.

[0208] According to various embodiments, the instructions may be configured to cause the at least one processor 120 to perform the following operations when executed by the at least one processor 120: input information about the position of the stylus 201 into multiple gesture detection algorithms, each of the multiple gesture detection algorithms determines whether a corresponding gesture has occurred, and identifies one or more gestures among the processing results of the multiple gesture detection algorithms that correspond to one or more detection algorithms having processing results that satisfy another specific condition as one or more candidate gestures.

[0209] According to various embodiments, the instructions may be configured to cause the at least one processor 120 to perform the following operations when executed by the at least one processor 120: identifying the travel distance of each of one or more candidate gestures, and identifying the candidate gesture corresponding to the maximum travel distance among the travel distances of the one or more candidate gestures as a gesture.

[0210] According to various embodiments, the instructions may be configured to, when executed by at least one processor 120, cause the at least one processor 120 to further recognize a gesture based on at least one of: a weight of each of one or more candidate gestures, a score of each of one or more candidate gestures, or a ratio of a travel distance of each of one or more candidate gestures relative to a total distance of one or more candidate gestures.

[0211] According to various embodiments, the method of operating the electronic device 101 may include: based on detecting a pen signal from a stylus through the sensing panel 503 of the electronic device 101, performing a first operation based on recognition of the detected pen signal, and based on not detecting a pen signal from the stylus 201 through the sensing panel 503, the method may include: receiving at least one communication signal through the communication module 190 of the electronic device 101, recognizing a gesture based on information about the position of the stylus 201 included in the at least one received communication signal, and performing a second operation corresponding to the recognized gesture.

[0212] According to various embodiments, the recognition of the gesture may include: recognizing one or more candidate gestures using information about the position of the stylus 201 , and recognizing a gesture among the recognized one or more candidate gestures.

[0213] The electronic device according to various embodiments may be one of various types of electronic devices. The electronic device may include, for example, a portable communication device (e.g., a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a household appliance. According to an embodiment of the present disclosure, the electronic device is not limited to those electronic devices described above.

[0214] It should be understood that the various embodiments of the present disclosure and the terms used therein are not intended to limit the technical features set forth herein to specific embodiments, but include various changes, equivalents or alternative forms for the corresponding embodiments. For the description of the accompanying drawings, similar reference numerals may be used to refer to similar or related elements. It will be understood that the nouns in the singular form corresponding to the term may include one or more things unless the relevant context clearly indicates otherwise. As used herein, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C" and "at least one of A, B or C" may include all possible combinations of items listed together with the corresponding one of the multiple phrases. As used herein, terms such as "1st" and "2nd" or "first" and "second" may be used to simply distinguish the corresponding component from another component, and do not limit the component in other aspects (e.g., importance or order). It will be understood that if an element (e.g., a first element) is referred to as being “combined with another element (e.g., the second element)”, “combined to another element (e.g., the second element)”, “connected with another element (e.g., the second element)”, or “connected to another element (e.g., the second element)” when the terms “operably” or “communicatively” are used or when the terms “operably” or “communicatively” are not used, it means that the element may be directly (e.g., wired) connected to the other element, wirelessly connected to the other element, or connected to the other element via a third element.

[0215] As used herein, the term "module" may include units implemented in hardware, software, or firmware, and may be used interchangeably with other terms (e.g., "logic," "logic block," "portion," or "circuit"). A module may be a single integrated component adapted to perform one or more functions or a minimum unit or portion of the single integrated component. For example, according to an embodiment, a module may be implemented in the form of an application specific integrated circuit (ASIC).

[0216] The various embodiments described herein may be implemented as software (e.g., a program) including one or more instructions stored in a storage medium (e.g., an internal memory 136 or an external memory 138) that can be read by a machine (e.g., a main device or a task execution device). For example, a processor of the machine (e.g., a main device or a task execution device) may call at least one of the one or more instructions stored in the storage medium and execute the at least one instruction. This enables the machine to operate to perform at least one function according to at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. A machine-readable storage medium may be provided in the form of a non-transitory storage medium. Among them, the term "non-transitory" only means that the storage medium is a tangible device and does not include a signal (e.g., an electromagnetic wave), but the term does not distinguish between data being semi-permanently stored in a storage medium and data being temporarily stored in a storage medium.

[0217] According to an embodiment, the method according to various embodiments of the present disclosure may be included and provided in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be released in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)), or may be downloaded via an application store (e.g., Play Store TM ) The computer program product may be published (e.g., downloaded or uploaded) online, or the computer program product may be distributed (e.g., downloaded or uploaded) directly between two user devices (e.g., smart phones). If published online, at least part of the computer program product may be temporarily generated, or at least part of the computer program product may be at least temporarily stored in a machine-readable storage medium (such as a memory of a manufacturer's server, an application store's server, or a forwarding server).

[0218] According to various embodiments, each component (e.g., module or program) in the above-mentioned components may include a single entity or multiple entities. According to various embodiments, one or more components in the above-mentioned components may be omitted, or one or more other components may be added. Alternatively or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, according to various embodiments, the integrated component may still perform the one or more functions of each component in the multiple components in the same or similar manner as a corresponding component in the multiple components before integration. According to various embodiments, the operations performed by a module, program or another component may be performed sequentially, in parallel, repeatedly or in a heuristic manner, or one or more operations in the operations may be run or omitted in different orders, or one or more other operations may be added.

[0219] While the present disclosure has been shown and described with reference to various embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the claims and their equivalents.

Claims

1. An electronic device, comprising: Sensing panel; Communication circuits; at least one processor electrically connected to the sensing panel and the communication circuit; as well as a memory electrically connected to the at least one processor, The memory is configured to store instructions, which, when executed by the at least one processor, cause the electronic device to perform the following operations: Based on the pen signal from the stylus being detected by the sensing panel: identifying a position of the stylus on the electronic device based on the pen signal, and performing a first operation corresponding to the identified position of the stylus on the electronic device; and Based on the pen signal from the stylus not being detected by the sensing panel: recognizing a gesture based on information regarding a position of the stylus included in at least one communication signal received from the stylus through the communication circuit; and A second operation different from the first operation and corresponding to the recognized gesture is performed.

2. The electronic device according to claim 1, wherein: When executed by the at least one processor, the instructions further cause the electronic device to perform the following operations: receiving, through the communication circuit, a first communication signal including first information about the position of the stylus pen, and the first information indicates the start of a gesture input, receiving, through the communication circuit, a second communication signal including second information indicating termination of the gesture input, and A gesture is recognized based on recognizing the second information included in the received second communication signal.

3. The electronic device as claimed in claim 2, in, The first information includes information indicating that a button of the stylus pen is in a pressed state, and The second information includes information indicating that the button of the stylus pen is in a released state.

4. The electronic device according to claim 1, wherein: The information about the position of the stylus pen includes at least one of the following: The sensing data acquired by the sensor of the stylus, The coordinates of the stylus pen identified based on the acquired sensing data, or The displacement of the stylus is identified based on the acquired sensing data.

5. The electronic device as claimed in claim 1, wherein: When executed by the at least one processor, the instructions further cause the electronic device to perform the following operations: using information about the position of the stylus to identify one or more candidate gestures, and A gesture is identified among the identified one or more candidate gestures.

6. The electronic device as claimed in claim 5, wherein: When executed by the at least one processor, the instructions further cause the electronic device to perform the following operations: inputting information about the position of the stylus into a plurality of gesture detection algorithms, wherein each gesture detection algorithm of the plurality of gesture detection algorithms determines whether a corresponding gesture has occurred, and One or more gestures corresponding to one or more detection algorithms having processing results satisfying a specific condition among the processing results of the plurality of gesture detection algorithms are identified as the one or more candidate gestures.

7. The electronic device as claimed in claim 5, wherein: When executed by the at least one processor, the instructions further cause the electronic device to perform the following operations: identifying a travel distance for each of the one or more candidate gestures, and A candidate gesture corresponding to a maximum travel distance among the travel distances of the one or more candidate gestures is recognized as a gesture.

8. The electronic device as claimed in claim 7, wherein: When executed by the at least one processor, the instructions further cause the electronic device to perform the following operations: A gesture is identified based on at least one of: a weight of each of the one or more candidate gestures, a score of each of the one or more candidate gestures, or a ratio of a travel distance of each of the one or more candidate gestures relative to a total distance of the one or more candidate gestures.

9. The electronic device as claimed in claim 1, wherein: When executed by the at least one processor, the instructions further cause the electronic device to perform the following operations: The function mapped to the recognized gesture is executed in an application running in the foreground of the electronic device as the second operation.

10. The electronic device according to claim 1, wherein: When executed by the at least one processor, the instructions further cause the electronic device to perform the following operations: When a pen signal from the stylus is detected, acceleration information of the stylus is received through the communication circuit, and The first operation is recognized based on the detected pen signal and acceleration information of the stylus.

11. The electronic device according to claim 1, wherein: When executed by the at least one processor, the instructions further cause the electronic device to perform the following operations: Based on the detected pen signal: activating a position gesture engine and using the activated position gesture engine, and performing the first operation based on the position of the stylus on the electronic device recognized by using the activated position gesture engine, and Based on the failure to detect the pen signal, a directional gesture engine is activated, and a gesture is recognized by using the activated directional gesture engine.

12. The electronic device according to claim 1, wherein: The first operation includes at least one of providing a pen function or providing additional information about content being displayed.

13. A method of operating an electronic device, comprising: Based on detecting a pen signal from the stylus pen through a sensing panel of the electronic device, identifying a position of the stylus pen on the electronic device based on the pen signal, and performing a first operation corresponding to the identified position of the stylus on the electronic device; as well as Based on the pen signal from the stylus not being detected by the sensing panel, performing the following operations: recognizing a gesture based on information regarding a position of the stylus included in at least one communication signal received from the stylus, and A second operation different from the first operation and corresponding to the recognized gesture is performed.

14. The method of claim 13, wherein: The information about the position of the stylus pen includes at least one of the following: The sensing data acquired by the sensor of the stylus, The coordinates of the stylus pen identified based on the acquired sensing data, or The displacement of the stylus is identified based on the acquired sensing data.

15. The method of claim 13, wherein: The steps to recognize a gesture include: using information about the position of the stylus to identify one or more candidate gestures; and A gesture is identified among the identified one or more candidate gestures.

Images