Method and apparatus for recognizing gesture using accessory electronic device

Abstract

The present disclosure provides an electronic device. The electronic device may be configured to: receive first sensor data obtained by a gyro sensor and second sensor data obtained by a passive infrared sensor from a wearable electronic device through a communication circuit; calculate a y-axis angle of a finger of a user wearing the wearable electronic device on the basis of the first sensor data; compensate for the y-axis angle on the basis of a predetermined reference point in response to detecting that the finger wearing the wearable device is in a folded state on the basis of the second sensor data; and generate a control signal corresponding to the compensated y-axis angle to control a screen output through a display.

Description

Method and device for recognizing gestures using an accessory electronic device

[0001] The present disclosure relates to a method and device for recognizing gestures using an accessory electronic device.

[0002] Video see-through (VST) vision is a technology that displays real-time video captured by a camera on a display, overlaying virtual objects on top. Users can simultaneously view the real world and virtual elements seen through the camera, making the virtual objects appear seamlessly integrated within the real world. VST vision utilizes a camera to detect hands and recognize gestures. However, vision-based gesture recognition has several technical limitations. For example, its reliance on the camera's field of view (FOV) limits the recognition range and can lead to detection errors due to environmental changes, such as the need for sufficient light. Identifying the target of recognition can be difficult when multiple hands are within the gesture recognition range. Occlusion can occur, where the user's hand is partially or completely obscured by another object. Furthermore, because VST vision overlays virtual objects based on a real-time video feed, very low latency is required for real-time gesture recognition. However, complex gesture recognition algorithms can hinder real-time processing.

[0003] A wearable electronic device according to one embodiment of the present disclosure comprises: a gyro sensor for detecting rotational motion in a roll direction, a pitch direction, and a yaw direction; a passive infrared sensor for detecting infrared; a communication circuit; a memory for storing at least one program; And at least one processor electrically connected to the memory and executing at least one command of a program stored in the memory, wherein the at least one processor: in response to a first movement of a finger worn by the wearable electronic device, based on at least one sensor data acquired according to a pitch direction rotation of the gyro sensor, recognizes a first angular change in the y-axis direction within a predetermined sensing range, and in response to a part of the body being sensed in a first direction by the passive infrared sensor while the at least one sensor data is acquired, corrects at least a part of the first angular change based on a predetermined reference point, and calculates a movement of a first y-axis value within the sensing range corresponding to the corrected first angular change, and in response to the movement of the first y-axis value being in an increasing direction, identifies the first movement as a scroll up gesture, and in response to the movement of the first y-axis value being in a decreasing direction, identifies the first movement as a scroll down gesture.

[0004] According to another embodiment of the present disclosure, an electronic device includes: a camera; a display; a communication circuit; a memory storing at least one program; and at least one processor electrically connected to the memory and executing at least one command of the program stored in the memory, wherein the at least one processor receives first sensor data acquired by a gyro sensor and second sensor data acquired by a passive infrared sensor from a wearable electronic device through the communication circuit, calculates a y-axis angle of a finger of a user wearing the wearable electronic device based on the first sensor data, and in response to detecting that the worn finger is in a folded state based on the second sensor data, compensates for the y-axis angle based on a predetermined reference point, and generates a control signal corresponding to the compensated y-axis angle, thereby controlling a screen output through the display.

[0005] In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

[0006] FIG. 1 illustrates an electronic device for recognizing a gesture using an accessory electronic device according to one embodiment of the present disclosure.

[0007] FIG. 2 illustrates a configuration of an accessory electronic device according to various embodiments of the present disclosure.

[0008] FIG. 3 is an example of recognizing a scroll gesture using an accessory electronic device according to one embodiment of the present disclosure.

[0009] FIG. 4 is a flowchart of a method for an electronic device to recognize a gesture using an accessory electronic device according to an embodiment of the present disclosure.

[0010] FIG. 5 is an example of display screen control by a two-dimensional pointing gesture according to one embodiment of the present disclosure.

[0011] FIG. 6A is an example of gyro sensor data and PIR sensor data of an accessory electronic device according to one embodiment of the present disclosure.

[0012] FIG. 6b is an example of display screen control by a scroll down gesture according to one embodiment of the present disclosure.

[0013] FIG. 7A illustrates example operations for recognizing a gesture recognition range of an accessory electronic device according to an embodiment of the present disclosure.

[0014] FIG. 7b is an example of a calibration method for a control range of an electronic device according to a gesture recognition range according to an embodiment of the present disclosure.

[0015] FIG. 8 is an example of identifying whether there is movement on the surface of an accessory electronic device according to one embodiment of the present disclosure.

[0016] FIG. 9 is an example of a control operation according to a scroll gesture while floating on the floor of an accessory electronic device according to an embodiment of the present disclosure.

[0017] FIG. 10 is an example of a tap operation of an accessory electronic device according to one embodiment of the present disclosure.

[0018] FIG. 11 is a flowchart of a gesture recognition method using an accessory electronic device according to an embodiment of the present disclosure.

[0019] FIG. 12 is a flowchart of a method for analyzing sensor data by an accessory electronic device according to an embodiment of the present disclosure.

[0020] FIG. 13 is an example of gesture recognition corresponding to micro-movements of an accessory electronic device according to an embodiment of the present disclosure.

[0021] FIG. 14 is an example of gesture recognition corresponding to rapid movement of an accessory electronic device according to one embodiment of the present disclosure.

[0022] FIG. 15A is an example of a PIR sensor of an accessory electronic device according to one embodiment of the present disclosure.

[0023] FIG. 15b is an example of a front view and a side view of an accessory electronic device according to one embodiment of the present disclosure.

[0024] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0025] An embodiment of the present disclosure is described below with reference to the attached drawings.

[0026] FIG. 1 illustrates an electronic device for recognizing a gesture using an accessory electronic device according to one embodiment of the present disclosure.

[0027] An accessory electronic device (201) according to one embodiment is wearable on a user's body (e.g., a finger) and can detect the user's biometric information while being worn by the user using a biometric sensor.

[0028] According to an embodiment, an electronic device (101) can recognize various gestures made by a part of a body (e.g., a finger) on which the accessory electronic device (201) is worn by using an accessory electronic device (201). For example, the electronic device (101) may be a head mounted display device (HMD). The HMD device (101) is a wearable form on a user's body (e.g., a head) and can output virtual reality (VR) or augmented reality (AR) images through a display. The HMD device (101) can recognize various gestures by using the accessory electronic device (201) as a control signal for controlling a display screen. While the user wears the HMD device (101) on his / her head and the accessory electronic device (201) on his / her right index finger, he / she can make two-dimensional pointing, tapping, long pressing, short pressing, scrolling up, scrolling down, swiping, or pinching, thereby executing a function corresponding to each gesture. For example, the HMD device (101) can use finger gestures to control the movement of a cursor displayed on a display screen. The HMD device (101) can recognize a two-dimensional pointing motion by a finger using an accessory electronic device (201) and control the position of the cursor on the display to move in response to the two-dimensional pointing motion.

[0029] An accessory electronic device (201) according to one embodiment may include a gyro sensor for detecting rotational motion in a roll direction, a pitch direction, and a yaw direction; a passive infrared sensor for detecting infrared; a communication circuit; a memory for storing at least one program; and at least one processor electrically connected to the memory and executing at least one command of the program stored in the memory.

[0030] In one embodiment, an accessory electronic device (201) may be configured to, in response to a first movement of a finger worn by the wearable electronic device (201), recognize a first angular change in the y-axis direction within a predetermined sensing range based on at least one sensor data acquired according to a pitch direction rotation of a gyro sensor of the wearable electronic device (201), correct at least a part of the first angular change based on a predetermined reference point in response to a part of the body being sensed in a first direction by a passive infrared sensor of the wearable electronic device (201) while the at least one sensor data is acquired, calculate a movement of a first y-axis value within the sensing range corresponding to the corrected first angular change, and identify the first movement as a scroll-up gesture in response to the movement of the first y-axis value being in an increasing direction, and identify the first movement as a scroll-down gesture in response to the movement of the first y-axis value being in a decreasing direction.

[0031] An accessory electronic device (201) according to one embodiment may identify the first movement as a movement in contact with a surface in response to a micro-vibration being detected by the gyro sensor, and identify the first movement as a movement in a lifted state from the surface in response to a micro-vibration not being detected by the gyro sensor.

[0032] An accessory electronic device (201) according to one embodiment can identify that the first movement corresponds to a long press gesture when the finger of a user wearing the wearable electronic device is detected by the passive infrared sensor and the duration of the movement exceeds a threshold time.

[0033] An accessory electronic device (201) according to one embodiment may detect a vibration value generated when a finger on which the wearable electronic device is worn comes into contact with an adjacent finger using the gyro sensor in response to a second movement by a finger on which the wearable electronic device is worn and an adjacent finger, and may identify that the second movement corresponds to a tap gesture in response to detecting the adjacent finger in a second direction using the passive infrared sensor.

[0034] An accessory electronic device (201) according to one embodiment may identify that the second movement corresponds to a long tap gesture when the time for which the adjacent finger is detected in the second direction by the passive infrared sensor exceeds a threshold time.

[0035] In one embodiment, the accessory electronic device (201) may identify the first movement as a gesture of lifting the worn finger in response to the first angular change being in a direction greater than a predetermined threshold angle.

[0036] An accessory electronic device (201) according to one embodiment may identify one or more detailed gestures by reflecting the degree to which the worn finger is lifted, depending on the degree to which the change in the first angle is greater than the determined threshold angle.

[0037] According to one embodiment, the determined sensing range may be determined by a motion of maximally moving a finger of a user wearing the wearable electronic device from side to side while keeping the finger in a pinned state and a motion of maximally bending the finger inward while keeping the finger in a completely pinned state.

[0038] According to one embodiment, the passive infrared sensor may include a plurality of thermoelectric sensors and may identify an infrared entry direction of the first movement detected by the plurality of thermoelectric sensors.

[0039] According to one embodiment, the passive infrared sensor includes a lens, and the sensing range and direction of the thermoelectric sensor can be determined in response to the position and direction of the lens.

[0040] An electronic device (101) according to one embodiment may include a camera; a display; a communication circuit; a memory storing at least one program; and at least one processor electrically connected to the memory and executing at least one command of the program stored in the memory.

[0041] According to one embodiment, an electronic device (101) receives first sensor data acquired by a gyro sensor and second sensor data acquired by a passive infrared sensor from a wearable electronic device through the communication circuit, calculates a y-axis angle of a finger of a user wearing the wearable electronic device based on the first sensor data, and in response to detecting that the finger is in a folded state based on the second sensor data, compensates for the y-axis angle based on a predetermined reference point, and generates a control signal corresponding to the compensated y-axis angle to control a screen output through the display.

[0042] In one embodiment, the electronic device (101) can detect that the worn finger is in a folded state in response to the second sensor data including an infrared detection signal in a direction in which the wearable electronic device is worn, and can detect that the worn finger is in a pinned state in response to the second sensor data not including an infrared detection signal.

[0043] An electronic device (101) according to one embodiment may identify that the worn finger is in contact with a surface in response to a micro-vibration being detected based on the first sensor data, and may identify that the worn finger is floating on the surface in response to a micro-vibration not being detected based on the first sensor data.

[0044] An electronic device (101) according to one embodiment may identify a long press gesture in response to a duration in which the worn finger is detected exceeding a predetermined threshold time based on the first sensor data.

[0045] According to one embodiment, the electronic device (101) may detect vibrations generated when the worn finger and the finger on which the wearable electronic device is worn and the adjacent finger come into contact with each other based on the first sensor data, identify a direction in which the adjacent finger is sensed based on the second sensor data, and identify a tap gesture by the worn finger and the adjacent finger in response to the direction in which the adjacent finger is sensed being different from the direction in which the wearable electronic device is worn.

[0046] FIG. 2 illustrates a configuration of an accessory electronic device according to various embodiments of the present disclosure.

[0047] Referring to FIG. 2, according to one embodiment, an accessory electronic device (201) that is connected to an electronic device (101) supports biometric information sensing functions, touch functions, and wireless communication functions, and may refer to an electronic device that can be worn on a user's body. The accessory electronic device (201) may also be referred to as a wearable electronic device.

[0048] The accessory electronic device (201) illustrated in FIG. 2 is illustrated as a ring type (e.g., a smart ring) worn on a finger by a user, but is not limited thereto, and may be implemented as other types of accessory electronic devices, such as a watch type (e.g., a smart watch) or a band type (e.g., a smart band).

[0049] An accessory electronic device (201) according to one embodiment may include a first annular housing (2010) (e.g., an outer ring housing, a first ring housing, or a first housing portion) and a second annular housing (2020) (e.g., an inner ring housing, a second ring housing, or a second housing portion) coupled to the first housing (2010) and including an opening. The opening may be formed to a size such that a user's finger can be inserted therein. For example, the first housing (2010) may be formed of a material that is resistant to external impact or scratches, such as a metal material, ceramic, or stainless steel. The first housing (2010) may also undergo a separate fixing or coating process for color implementation. The second housing (2020) may be formed of the same material as the first housing (2010), or may be formed of a material such as a molding material, plastic, or glass for sensing. The second housing (2020) may be formed to comprise at least a portion of a metallic material for biometric measurements.

[0050] An accessory electronic device (201) according to one embodiment may include a processor (210), a memory (220), a communication module (230), an antenna (235), a battery (240), a charging interface (245), at least one biometric sensor (250), a touch sensor (260), an inertial sensor (270), a temperature sensor (280), and a power management integrated circuit (PMIC) (290) disposed in a space between a first housing (2010) and a second housing (2020). Some of the components may be disposed on a substrate (295) (e.g., FPCB, flexible printed circuit board) having flexibility to correspond to the curvature of the accessory electronic device (201).

[0051] According to some embodiments, the accessory electronic device (201) may further include other components (e.g., a display, an ultrasonic sensor, an audio output device) in addition to the components illustrated.

[0052] A communication module (230) according to one embodiment may include various hardware and / or software configurations for supporting wireless communication with an external electronic device (hereinafter, HMD device (101) of FIG. 1). The wearable electronic device (201) may transmit and receive various data or control commands with the electronic device (101) via the communication module (230) via wired / wireless communication. In one embodiment, the communication module may support short-range wireless communication. Short-range wireless communication includes at least one of Bluetooth, BLE (Bluetooth Low Energy), ZigBee, ANT+, Wi-Fi, Cellular (LTE, 5G, 6G, NB-IoT), NFC (near field communication), RFID (radio frequency identification), UWB (ultra wide band), GNSS (global navigation satellite system) or / and MST (magnetic secure transmission), but is not limited thereto. According to some embodiments, the communication module (230) may be implemented in an integrated form with the processor (210).

[0053] According to one embodiment, the antenna (235) may be connected to the communication module (230) via the substrate (295). The wearable electronic device (201) may transmit or receive communication signals / data to the outside via the antenna (225). The antenna (235) may include a single or multiple antennas. In some embodiments, a part of the first housing (2010) (e.g., a metal member) may be designed to be used as the antenna (235).

[0054] According to one embodiment, the battery (240) may be formed in a curved shape to have a curvature corresponding to the curvature of the space between the first housing (2010) and the second housing (2020). The battery (240) may be configured such that multiple battery packs are separated and arranged. The battery (240) may be connected to a charging interface (245).

[0055] According to one embodiment, a charging interface (245) may be electrically connected to a PMIC (290) mounted on a substrate (295) via the substrate (295). The charging interface (245) may support wired charging (terminal) or wireless charging (WPC, NFC) methods for charging.

[0056] At least one biometric sensor (250) according to one embodiment can obtain various biometric information of a user using an optical signal. For example, the biometric sensor (250) may be a photoplethysmogram (PPG) sensor or an optical sensor that can obtain various biometric information such as heart rate and blood circulation by measuring a plethysmogram according to an optical signal, but is not limited thereto. The biometric sensor (250) can obtain biometric information such as heart rate (HR), blood pressure, saturation of percutaneous oxygen (SpO2), galvanic skin response (GSR), electrocardiography (ECG), blood flow velocity, and bioelectrical impedance, but is not limited thereto.

[0057] In some embodiments, the biometric sensor (250) may include a fingerprint sensor.

[0058] A biometric sensor (250) according to one embodiment may include a sensor controller (250a), a plurality of emitters (250b) for outputting optical signals, and a plurality of receivers (250c) for receiving optical signals. The emitters (250b) may include light-emitting elements that emit light of various wavelengths or colors (e.g., green, red) to measure a biometric signal. The emitters (250b) may be formed by at least one of a light-emitting diode (LED), a semiconductor laser diode (LD), an infrared (IR) diode, and a VCSEL. The receivers (250c) may be formed by a photodiode (PD) or a complementary metal-oxide-semiconductor (CMOS) camera. The receivers (250c) may convert a received optical signal through an analog-to-digital converter (ADC) and store the converted signal in a processor (210) or a memory (220). The sensor controller (250a) can control the light emitting unit (250b) and the light receiving unit (250c).

[0059] A biometric sensor (250) according to one embodiment may include a passive infrared sensor (250d). The passive infrared sensor (250d) does not emit light and can only passively detect a certain range of infrared light. For example, the passive infrared sensor (250d) can detect a temperature range emitted by a human body. It differs from a light receiving unit (250c) that detects the return of an optical signal output by a light emitting unit (250b), and also from a proximity sensor that emits energy and measures the time it takes for it to be reflected and returned.

[0060] A passive infrared sensor (250d) according to one embodiment may include a 2X2 sized thermoelectric sensor. When the passive infrared sensor (250d) detects a part of the body passing through the 2X2 sized thermoelectric sensor, the passive infrared sensor (250d) may identify an entry direction. For example, the passive infrared sensor (250d) may distinguish between horizontal and vertical entry, and may distinguish between entry and exit. The passive infrared sensor (250d) is arranged toward the first housing (2010) opposite to the user's wearing direction, and a part for infrared detection may be exposed to the outside of the first housing (2010). The passive infrared sensor (250d) may have a thermoelectric sensor arranged toward the first housing (2010) and may be equipped with a lens (e.g., F-lens) to adjust the detection angle and temperature range of the sensor. The passive infrared sensor (250d) can be appropriately positioned and oriented so that the passive infrared sensing area faces the direction of the fingernail when worn on the user's finger, so that the thermoelectric sensor can effectively recognize the user's finger folding motion. A portion (e.g., the lens) of the passive infrared sensor (250d) can be exposed to the outside of the first housing (2010). The passive infrared sensor (250d) can be implemented transparently for smooth sensing, but can also be implemented so that it is not visible from the field of view using a translucent material for aesthetic reasons.

[0061] A touch sensor (260) according to one embodiment can detect a touch signal of a user touching a wearable electronic device (201). The touch sensor (260) can be formed using at least one of a pressure type, an electrostatic type, an optical type, or an ultrasonic type, for example.

[0062] In some embodiments, the touch sensor (260) may be omitted.

[0063] An inertial sensor (270) according to one embodiment can obtain movement information of a wearable electronic device (201). For example, the inertial sensor (270) can detect motion, gesture, impact, posture, and / or activity (e.g., sedentary, moving, sports). The inertial sensor (270) may be formed as a 3-axis accelerometer, but is not limited thereto, and may also be formed as a 6-axis sensor including an accelerometer and a gyroscope.

[0064] A gyroscope sensor (hereinafter, referred to as a gyro sensor) (270) according to one embodiment can detect the movement of a finger (e.g., an index finger) wearing a wearable electronic device (201). However, in the case of a two-dimensional movement, such as placing the palm on the floor and moving only the finger, it is difficult to distinguish between a state where the finger is pinched and a state where the finger is folded because there is almost no rotation in the pinch direction.

[0065] A temperature sensor (280) according to one embodiment can measure the body temperature of a user or the temperature of a component (e.g., an electronic component) included in a wearable electronic device (201). The temperature sensor (280) can be formed in a contact or non-contact manner and may vary depending on the design. The wearable electronic device (201) can record the temperature information recorded through the temperature sensor (280) in memory or, under processor control, use it to measure the body temperature of the user, estimate skin temperature, or estimate situational awareness.

[0066] According to one embodiment, a PMIC (290) can manage power delivered from a battery (40) to each component of a wearable electronic device (201).

[0067] The memory (220) according to one embodiment may store various instructions that may be performed by the processor (210). Such instructions may include arithmetic and logical operations, data movement, or control commands such as input / output that may be recognized by the processor (210).

[0068] According to one embodiment, the processor (210) is a configuration capable of performing calculations or data processing related to control and / or communication of each component of the accessory electronic device (201), and may be composed of one or more processors. The calculation and data processing functions that the processor (210) may implement on the accessory electronic device (201) are not limited, but in this document, various operations for supporting gesture recognition in conjunction with the electronic device (101) can be processed.

[0069] FIG. 3 is an example of recognizing a scroll gesture using an accessory electronic device according to one embodiment of the present disclosure.

[0070] According to one embodiment, the accessory electronic device (201) can recognize various types of gestures made by a hand wearing the accessory electronic device (201). The accessory electronic device (201) can detect the movement of the accessory electronic device (201) in the x-axis direction based on sensor data according to the roll direction rotation acquired by the gyro sensor. For example, the accessory electronic device (201) can recognize a swipe gesture from left to right using the gyro sensor.

[0071] According to one embodiment, the accessory electronic device (201) can recognize a two-dimensional pointing gesture. The accessory electronic device (201) can recognize a pointing position by a hand based on a sensing range corresponding to a two-dimensional plane. Here, the sensing range may refer to a range of movement of a finger while the palm is fixed. The hand gesture may be, for example, a movement of moving a finger left and right while the palm is fixed on the floor (or surface), a movement of tapping a fingertip on the floor, or a movement of moving a finger up and down, similar to operating a mouse.

[0072] Referring to FIG. 3, the accessory electronic device (201) can recognize the movement of the index finger moving up and down along the floor while the palm is fixed to the floor using a gyro sensor (e.g., the gyro sensor (270) of FIG. 2) and a passive infrared (PIR) sensor (e.g., the passive infrared sensor (250d) of FIG. 2). The sensing range for the up and down movement is determined in response to the maximum range of the index finger moving up and down while the palm is fixed to the floor. For example, the y-axis sensing range is a maximum Y max From minimum Y min It can be.

[0073] When a finger is folded within the Y-axis movement range (sensing range), the folded portion can be detected by the PIR sensor (250d). Referring to Fig. 3, the PIR detection area corresponds to a portion of the sensing range.

[0074] Referring to the first screen (310), the accessory electronic device (201) may receive a gyro sensor (270) value and a PIR sensor (250d) value to detect a state in which the index finger worn by the accessory electronic device (201) is open at a first time point (T1). In one embodiment, when infrared rays are detected by the PIR sensor (250d), the sensor value may be any value greater than or equal to 1. When infrared rays are not detected by the PIR sensor (250d), the sensor value may be 0 or any value close to 0. When the PIR sensor value is 0, the accessory electronic device (201) may recognize that the index finger is open. The accessory electronic device (201) may convert the gyro sensor value into an angle and determine a position corresponding to the converted angle within the y-axis sensing range. In the case of the first screen (310), the value obtained by converting the gyro sensor value into an angle is a first angle (α°).

[0075] The finger of the first screen (310) can be moved from a state of being pinched and folded downwards to a state of being folded and folded downwards on the second screen (320). At this time, the first angle (α°) converted into an angle by the value of the gyro sensor (270) can gradually decrease and become 0, and then gradually increase and become the first angle (α°) again. This is because the pinch direction angle of the accessory electronic device (201) is the same whether the index finger is pinched or folded. The accessory electronic device (201) can recognize that a part of the body (e.g., the tip of the index finger) is detected by the sensing part of the accessory electronic device (201) (a certain range in the downward direction of the worn finger) based on the PIR sensor value.

[0076] According to one embodiment, the accessory electronic device (201) may set a point where the gyro sensor value becomes 0 as a reference point. The finger may be extended or folded around the reference point. In one embodiment, the PIR detection area corresponding to a PIR sensor value of 1 may be located in a direction relative to the reference point.

[0077] Referring to the second screen (320), the accessory electronic device (201) is at a second point in time (T n ) to detect a folded state of the index finger worn by the accessory electronic device (201), the accessory electronic device (201) may receive a gyro sensor (270) value and a PIR sensor (250d) value. The accessory electronic device (201) may recognize that the index finger is folded when the PIR sensor value is 1. The accessory electronic device (201) may convert the gyro sensor value into an angle and determine a position corresponding to the converted angle within the y-axis sensing range. In the case of the second screen (320), the value obtained by converting the gyro sensor value into an angle is a first angle (α°). The accessory electronic device (201) may detect a folded state of the index finger using the PIR sensor (250d), and in response thereto, compensate for the first angle value using a determined reference point. In the case of FIG. 3, the reference point may be set to 0, and the upward direction of the y-axis may be considered a positive number, and the downward direction of the y-axis may be considered a negative number. The accessory electronic device (201) can determine the Y-axis position by considering both the gyro sensor value and the PIR sensor value, maintaining the first angle as is for the first screen (310), and changing the sign of the first angle to a negative number for the second screen (320).

[0078] FIG. 4 is a flowchart of a method for an electronic device to recognize a gesture using an accessory electronic device according to an embodiment of the present disclosure.

[0079] An electronic device according to one embodiment (e.g., a head mounted display device (101) of FIG. 1) can recognize a gesture of a finger wearing the accessory electronic device (101) based on a gyro sensor value and a PIR sensor value of the accessory electronic device (101).

[0080] In operation 410, the electronic device (101) may perform a preparation state for gesture recognition. For example, the electronic device (101) may perform a preparation state for gesture recognition in response to entering an application capable of 2D content control, playback control, etc. For gesture recognition, the gyro sensor (270) and the PIR sensor (250d) of the accessory electronic device (101) may be activated.

[0081] In operation 420, the electronic device (101) can detect a trigger gesture corresponding to a gesture recognition function using the accessory electronic device (201). For example, the trigger gesture may be set to a double tap when located on the floor (or surface) (surface state). Or, when lifted from the floor (in air state), a motion distinct from a daily motion (e.g., flicking the index finger) may be set.

[0082] In operation 430, the electronic device (101) may receive first data from the gyro sensor (270) of the accessory electronic device (201) in response to detecting a trigger gesture. The electronic device (101) may receive roll, pitch, and yaw sensor data acquired by the gyro sensor (270) to identify a recognizable gesture in the entered application. However, for some gestures, the yaw sensor data may not be used for gesture recognition. For example, a two-dimensional pointing gesture may skip the yaw sensor data.

[0083] In operation 440, the electronic device (101) may receive second data from the passive infrared sensor (250d) of the accessory electronic device (201) in response to detecting a trigger gesture. The electronic device (101) may receive infrared detection data acquired by the PIR sensor (250d) to identify a gesture related to a finger folding movement in an entered application. The electronic device (101) may distinguish between a pinned state and a folded state of the finger based on the PIR sensor value, distinguish the shape of the finger folded (when the index finger is folded or when the thumb is moved toward the index finger), and identify the time for which a part of the finger is maintained in a detected state.

[0084] In operation 450, the electronic device (101) can recognize a gesture based on first data acquired by the gyro sensor (270) and second data acquired by the PIR sensor (250d). For example, the electronic device (101) can identify an x-axis angle and a y-axis angle based on the first data, and determine whether to compensate for the y-axis angle based on the second data. The electronic device (101) can identify a two-dimensional pointing position for controlling the display based on an x-axis position of a two-dimensional display area corresponding to the x-axis angle and a y-axis position of the two-dimensional display area corresponding to the compensated y-axis angle.

[0085] FIG. 5 is an example of display screen control by a two-dimensional pointing gesture according to one embodiment of the present disclosure.

[0086] According to one embodiment, the electronic device (101) can recognize a finger gesture using an accessory electronic device (201) and use it as a control signal for a display screen.

[0087] An electronic device (101) according to an embodiment can recognize a two-dimensional pointing gesture of a finger as a control signal for the movement of a cursor (501) on a display screen (510). The electronic device (101) can recognize a gesture (521) of moving an index finger from left to right using an accessory electronic device (201), and move the cursor (501) on the display screen (510) from ① to ②. The electronic device (101) can detect a finger folding motion using a PIR sensor (250d), and recognize a gesture (522) of moving an index finger from up to down, and move the cursor (501) on the display screen (510) from ② to ③.

[0088] FIG. 6A is an example of gyro sensor data and PIR sensor data of an accessory electronic device according to one embodiment of the present disclosure.

[0089] FIG. 6b is an example of display screen control by a scroll down gesture according to one embodiment of the present disclosure.

[0090] According to one embodiment, an accessory electronic device (e.g., accessory electronic device (201) of FIG. 1) can detect a folding motion of a finger (e.g., an index finger) wearing the accessory electronic device (201) using a PIR sensor (250d).

[0091] This is a graph showing gyro sensor data (621) and passive infrared sensor data (622) over time, corresponding to operations 611 to 613 of FIG. 6A.

[0092] At motion 611 (time T1), the gyro sensor value is α. The PIR sensor value is 0.

[0093] At action 612 (time point T2), the gyro sensor value is 0. The PIR sensor value is 0.

[0094] At motion 613 (time point T3), the gyro sensor value is α. The PIR sensor value is 1.

[0095] According to one embodiment, the accessory electronic device (201) can recognize that the index finger is in a pinned state and calculate the y-coordinate (α) based on the gyro sensor value (α) and the PIR sensor value (0) according to operation 611. The accessory electronic device (201) can recognize that the index finger is in a pinned state and calculate the y-coordinate (0) based on the gyro sensor value (0) and the PIR sensor value (0) according to operation 612. The accessory electronic device (201) can recognize that the index finger is in a folded state and calculate the y-coordinate (-α) based on the gyro sensor value (α) and the PIR sensor value (1) according to operation 613.

[0096] An accessory electronic device (201) according to an embodiment can recognize a scroll down gesture for continuous motions (motions 611 to 613). The accessory electronic device (201) can identify that the scroll down gesture is a motion that touches a surface in response to a micro-vibration being detected by the gyro sensor (270). The accessory electronic device (201) can identify that the scroll down gesture is a motion that touches a surface in response to a micro-vibration not being detected by the gyro sensor (270). The accessory electronic device (201) can identify that the continuous motion is a scroll down gesture that touches a surface (floor).

[0097] Referring to FIG. 6B, an electronic device (e.g., electronic device (101) of FIG. 1) can receive a recognized scroll down gesture as a control signal using an accessory electronic device (201) and control a display screen (620) to scroll down (621). The y coordinate (α) on the sensing area of ​​FIG. 6A is a first point (y) of the display screen (620). α ) corresponds to the second point (y) of the display screen (620), and the y coordinate (-α) corresponds to the second point (y) of the display screen (620). -α ) can be corresponded to.

[0098] According to one embodiment, the electronic device (101) can receive gyro sensor values ​​and PIR sensor values ​​from the accessory electronic device (201) and identify a gesture corresponding thereto. According to another embodiment, the electronic device (101) can receive identified gesture information from the accessory electronic device (201). In one embodiment, at least one of the electronic device (101) or the accessory electronic device (201) can identify a gesture based on the gyro sensor values ​​and PIR sensor values ​​of the accessory electronic device (201).

[0099] FIG. 7A illustrates example operations for recognizing a gesture recognition range of an accessory electronic device according to an embodiment of the present disclosure.

[0100] FIG. 7b is an example of a calibration method for a control range of an electronic device according to a gesture recognition range according to an embodiment of the present disclosure.

[0101] Referring to FIGS. 7A and 7B together, an accessory electronic device (e.g., accessory electronic device (201) of FIG. 1) can determine a range (740) within which it can recognize a gesture while performing a predetermined movement while being worn by a user.

[0102] In operation 710, a user wearing an accessory electronic device (201) on an index finger can perform an operation of moving the index finger as far left and right as possible while keeping the palm fixed and while extending the index finger as much as possible. The accessory electronic device (201) can determine the x-axis motion range from the leftmost point (①) (741) to the rightmost point (②) (742) while extending the finger as much as possible by operation 710.

[0103] In operation 720, a user wearing an accessory electronic device (201) on an index finger can perform an operation of moving the index finger to the left and right as much as possible while keeping the palm fixed and bending the index finger to the maximum. The accessory electronic device (201) can determine the x-axis motion range from the leftmost point (③) (743) to the rightmost point (④) (744) while the finger is bent (folded) by operation 720.

[0104] In operation 730, a user wearing an accessory electronic device (201) on an index finger can perform an operation of bending the index finger from a maximally straightened state to a maximally bent state while keeping the palm fixed. The accessory electronic device (201) can determine a y-axis motion range from the uppermost point (⑤) to the lowermost point (⑥) by operation 730.

[0105] Referring to FIG. 7B, the electronic device (101) can map the control range (750) of the display screen of the electronic device (101) to correspond to the recognition range (740) of the accessory electronic device (201). ①②③④ of the gesture recognition range (740) can correspond to ①②③④ of the control range (750).

[0106] For example, the gesture recognition range (740) may be set to the coordinates (-40°, 50°) of the first point (741), the coordinates (40°, 50°) of the second point (742), the coordinates (-25°, 20°) of the third point (743), and the coordinates (25°, -20°) of the fourth point (744). The X-axis coordinate value may represent the left and right tilt angle (angle) acquired by the gyro sensor. The Y-axis coordinate value may represent the bending angle (angle) acquired by the gyro sensor (270). Accordingly, the X-axis angle range is 80°, and the Y-axis angle range is 70°.

[0107] For example, the x-axis screen range of the electronic device (101) may be set to 1920, and the y-axis screen range may be set to 1080.

[0108] When the gesture recognition range (740) is mapped to the control range (750), the coordinates (-40°, 50°) of the first point (741) can be (0, 1080), the coordinates (40°, 50°) of the second point (742) can be (1920, 1080), the coordinates (-25°, 20°) of the third point (743) can be (0,0), and the coordinates (25°, -20°) of the fourth point (744) can be (1920, 0). Accordingly, the upper x-axis resolution of the display can be 1920 / 80° = 24 / °, the lower x-axis resolution can be 1920 / 50° = 38.4 / °, and the y-axis resolution can be 1080 / 70° = 15.4 / °.

[0109] According to one embodiment, the electronic device (101) can set a display control range according to a gesture recognition range, and then detect gesture speed, etc. according to use and reflect it in the setting.

[0110] FIG. 8 is an example of identifying whether there is movement on the surface of an accessory electronic device according to an embodiment of the present disclosure.

[0111] An accessory electronic device according to one embodiment (e.g., the accessory electronic device (201) of FIG. 1) can identify whether a gesture is made in contact with a surface (surface) or in air (in air) when worn by a user. In one embodiment, the accessory electronic device (201) can identify a state of contact with a surface in response to a micro-vibration detected by a gyro sensor (270).

[0112] In another embodiment, the accessory electronic device (201) can identify whether it is floating from a surface based on the maximum angle (α°) at which it touches the surface. The accessory electronic device (201) can identify whether it is floating from a surface in response to a current y-axis angle (A) acquired by a gyro sensor being greater than the maximum angle (α°).

[0113] Referring to FIG. 8, in operation 810, the current angle (A) of the index finger wearing the accessory electronic device (201) is greater than the maximum angle (α°) and corresponds to a state of being lifted off the floor. In the state of being lifted off the floor, when the index finger is lowered, as in operation 820, the current angle (A) of the index finger wearing the accessory electronic device (201) matches the maximum angle (α°) and corresponds to a state of being in contact with the floor.

[0114] FIG. 9 is an example of a control operation according to a scroll gesture while floating on the floor of an accessory electronic device according to an embodiment of the present disclosure.

[0115] According to one embodiment, an accessory electronic device (e.g., accessory electronic device (201) of FIG. 1) can recognize a scroll up or down gesture when the current y-axis angle (A) acquired by the gyro sensor (270) is greater than the maximum angle (α°).

[0116] In the 910 motion, the index finger wearing the accessory electronic device (201) can perform a motion of raising the index finger further upward while floating off the surface.

[0117] According to one embodiment, the accessory electronic device (201) can recognize a scroll-up gesture in response to a signal detected by the PIR sensor (250d) being absent and a movement of the y-axis angle acquired by the gyro sensor becoming greater than a maximum angle. Accordingly, the electronic device (e.g., the electronic device (101) of FIG. 1) can control the cursor (901) of the display screen (911) to scroll up only the screen in a fixed state in response to recognizing the scroll-up gesture while floating on a surface.

[0118] In the 920 motion, the index finger wearing the accessory electronic device (201) can perform a motion of folding the index finger while floating off the surface.

[0119] According to one embodiment, the accessory electronic device (201) can recognize a scroll down gesture in response to a part of the body being detected by the PIR sensor (250d) while the movement of the y-axis angle acquired by the gyro sensor (270) decreases from a state greater than the maximum angle. Accordingly, the electronic device (101) can control the cursor (901) of the display screen (921) to scroll up only the screen in a fixed state in response to recognizing the scroll down gesture while floating on the surface.

[0120] According to one embodiment, by combining the sensor values ​​of the gyro sensor (270) and the PIR sensor (250d), scrolling up / down while floating from a surface and scrolling up / down while touching the surface can be distinguished. Since each gesture can be recognized as a different gesture, the control signal corresponding to each gesture can be set differently.

[0121] FIG. 10 is an example of a tap operation of an accessory electronic device according to one embodiment of the present disclosure.

[0122] According to one embodiment, an accessory electronic device (e.g., the accessory electronic device (101) of FIG. 1) can perform a tap action by using a finger (e.g., an index finger) wearing the accessory electronic device (101) together with another finger (e.g., a thumb).

[0123] The first tap gesture (motions 1011 to 1013) is a movement in which the thumb approaches the index finger while the index finger wearing the accessory electronic device (201) is fixed, and the accessory electronic device (201) can recognize the tap gesture in response to the gyro sensor value being constant and the thumb being detected by the PIR sensor (250d) of the accessory electronic device (201).

[0124] The second tap motion (motions 1021 to 1023) is a motion in which both the index finger and the thumb touch each other, and the accessory electronic device (201) detects vibration at the moment when the index finger and the thumb touch each other by the gyro sensor value, and in response to the thumb being detected by the PIR sensor (250d) of the accessory electronic device (201), can recognize the tap gesture by the thumb and the index finger.

[0125] According to one embodiment, when detecting the first tap motion and the second tap motion, the direction in which the PIR sensor (250d) detects the folding motion of the index finger and the direction in which it detects the adjacent motion of the thumb with respect to the detection area (1040) may be different. For example, the direction in which the index finger is detected may be in the direction of the fingertip, and the direction in which the thumb is detected may be perpendicular to the direction of the fingertip.

[0126] In one embodiment, the accessory electronic device (201) can distinguish between an entry (IN) motion in which a body part is detected in the detection area (1040) and an exit (OUT) motion in which a body part is not detected. The accessory electronic device (201) can identify how long a body part is detected in the detection area (1040). In one embodiment, the accessory electronic device (201) can recognize by changing a tap into a long press gesture in response to detection by the PIR sensor (250d) continuing for a threshold time or longer.

[0127] FIG. 11 is a flowchart of a method for identifying a gesture state by an accessory electronic device according to an embodiment of the present disclosure.

[0128] According to one embodiment, an accessory electronic device (e.g., the accessory electronic device (201) of FIG. 1) can recognize a hand gesture made by wearing the accessory electronic device (201) using a gyro sensor (270) and a PIR sensor (250d). The operation of the accessory electronic device (201) of FIG. 11 can be at least partially performed by an electronic device (e.g., the head mounted display device (101) of FIG. 1) according to one embodiment. The electronic device (101) can receive sensor data of the gyro sensor and sensor data of the PIR sensor from the accessory electronic device (201) to recognize a hand gesture made by wearing the accessory electronic device (201).

[0129] In operation 1110, according to one embodiment, the accessory electronic device (201) may activate a motion detection sensor. The motion detection sensor may include at least one of a gyro sensor (270) or a PIR sensor (250d). For example, the accessory electronic device (201) may activate the motion detection sensor in response to being worn by a user.

[0130] In operation 1120, according to one embodiment, the accessory electronic device (201) may detect a gyro sensor value according to the movement of a finger (e.g., an index finger) wearing the accessory electronic device (201). If the gyro sensor value is not detected, the gyro sensor may remain in a standby state.

[0131] In operation 1130, according to one embodiment, the accessory electronic device (201) can identify whether there is a micro-vibration in response to a sensor value detected by the gyro sensor (270). When a user moves a finger on a surface, a micro-vibration may occur due to the part where the finger contacts the floor. The gyro sensor (270) can detect the micro-vibration.

[0132] In operation 1140, according to one embodiment, the accessory electronic device (201) can recognize that the surface motion is a surface motion in contact with a surface in response to detecting a micro-vibration.

[0133] In operation 1150, according to one embodiment, the accessory electronic device (201) can recognize that the operation is an air in surface operation in response to not detecting micro-vibrations.

[0134] FIG. 12 is a flowchart of a method for analyzing sensor data by an accessory electronic device according to an embodiment of the present disclosure.

[0135] An accessory electronic device (e.g., the accessory electronic device (201) of FIG. 1) according to an embodiment can recognize continuous gestures by analyzing real-time gyro sensor values ​​and PIR sensor values. The operation of the accessory electronic device (201) of FIG. 12 can be at least partially performed by an electronic device (e.g., the head mounted display device (101) of FIG. 1) according to an embodiment. The electronic device (101) can receive sensor data of a gyro sensor and sensor data of a PIR sensor from the accessory electronic device (201) and recognize continuous gestures of a hand wearing the accessory electronic device (201).

[0136] In operation 1210, according to one embodiment, the accessory electronic device (201) can identify whether a real-time gyro sensor value is detected.

[0137] In operation 1220, according to one embodiment, the accessory electronic device (201) may, in response to the gyro sensor value being detected, identify whether a PIR sensor value is detected.

[0138] In operation 1230, according to one embodiment, the accessory electronic device (201) can identify that the finger wearing the accessory electronic device (201) is in a pinned state when the gyro sensor value is detected and the PIR sensor value is not detected.

[0139] In operation 1240, according to one embodiment, the accessory electronic device (201) can identify whether a change in PIR sensor data is detected when a finger (index finger) wearing the accessory electronic device (201) is pinned, i.e., whether a part of the body is detected.

[0140] In operation 1250, according to one embodiment, the accessory electronic device (201) can identify a scroll down gesture when a part of the body is detected by the PIR sensor (250d) while the index finger is in contact with the surface. Thereafter, according to one embodiment, the accessory electronic device (201) can repeat operation 1210.

[0141] In operation 1231, according to one embodiment, the accessory electronic device (201) can identify that the finger (index finger) wearing the accessory electronic device (201) is folded when the gyro sensor value is detected and the PIR sensor value is also detected.

[0142] In operation 1241, according to one embodiment, the accessory electronic device (201) can identify whether a change in PIR sensor data is detected while the index finger is folded, i.e., whether a part of the body is no longer detected.

[0143] In operation 1251, according to one embodiment, the accessory electronic device (201) can identify that the scroll-up gesture is made when the index finger is in contact with the surface and the body part is no longer detected by the PIR sensor (250d) while the index finger is folded. Thereafter, according to one embodiment, the accessory electronic device (201) can repeat operation 1210.

[0144] According to one embodiment, the accessory electronic device (201) can perform real-time gesture recognition while repeating operations 1210 to 1250 or 1251.

[0145] FIG. 13 is an example of gesture recognition corresponding to micro-movements of an accessory electronic device according to an embodiment of the present disclosure.

[0146] According to one embodiment, when an accessory electronic device (e.g., the accessory electronic device (201) of FIG. 1) wishes to control fine movements with hand gestures during content playback, the accessory electronic device may recognize scrolling movements while in contact with a surface (floor) and control fine movements of a playback bar on a playback screen. Among various gestures, gestures corresponding to fine movements may be used to generate fine control signals.

[0147] According to one embodiment, the accessory electronic device (201) can recognize a fine scrolling up gesture (1321) or a scrolling down gesture (1322) made by a finger wearing the accessory electronic device (201) while in contact with a surface, and transmit the same to an HMD device (e.g., the electronic device (101) of FIG. 1). The HMD device (101) can control the movement of a playback bar (1311) of a playback screen (1310) to move by a fine degree according to the received scrolling signal.

[0148] FIG. 14 is an example of gesture recognition corresponding to rapid movement of an accessory electronic device according to one embodiment of the present disclosure.

[0149] In one embodiment, an accessory electronic device (e.g., the accessory electronic device (201) of FIG. 1) may recognize scrolling movements while floating from a surface and control the playback speed when a user wishes to control rapid movements with hand gestures during content playback. Among various gestures, gestures that are differentiated by stage may be used to generate step-by-step control signals by dividing the speed into stages.

[0150] According to one embodiment, the accessory electronic device (201) can recognize a gesture of scrolling a finger wearing the accessory electronic device (201) while floating on a surface, and moving the scrolling range in several steps. The accessory electronic device (201) can recognize a gesture of lifting an index finger further upward as a gesture for forward speed search on a playback screen (1410). The accessory electronic device (201) can identify a gesture of lifting an index finger when a change in an angle detected by a gyro sensor (270) is in a direction greater than a predetermined threshold angle.

[0151] In operation 1431, the accessory electronic device (201) may recognize a gesture of raising a hand to a y-axis angle of 0° by the gyro sensor (270) and generate a first stage control signal (e.g., double-speed forward navigation).

[0152] In operation 1441, the accessory electronic device (201) may recognize a gesture of raising a hand at a y-axis angle of β1° by the gyro sensor (270), and generate a second stage control signal (e.g., 4x forward speed search).

[0153] In operation 1451, the accessory electronic device (201) can recognize a gesture of raising a hand at a y-axis angle of γ1° by the gyro sensor (270) and generate a third-stage control signal (e.g., 8x forward speed search).

[0154] An accessory electronic device (201) according to one embodiment can identify a gesture of scrolling the index finger downwards when the change in angle detected by the gyro sensor (270) is in a direction greater than a predetermined threshold angle, when the gesture is a folding motion of the index finger by the PIR sensor (250d).

[0155] In operation 1432, the accessory electronic device (201) can recognize a hand folding gesture by the PIR sensor when the y-axis angle by the gyro sensor (270) is 0°, and generate a first stage control signal (e.g., double-speed backward search).

[0156] In operation 1442, the accessory electronic device (201) can recognize a hand folding gesture by the PIR sensor (250d) in a state where the y-axis angle by the gyro sensor (270) is β1°, and generate a second stage control signal (e.g., 4x backward speed search).

[0157] In operation 1452, the accessory electronic device (201) can recognize a gesture of folding the hand by the PIR sensor in a state where the y-axis angle by the gyro sensor (270) is γ1°, and generate a third stage control signal (e.g., 8x backward speed search).

[0158] FIG. 15A is an example of a PIR sensor of an accessory electronic device according to one embodiment of the present disclosure.

[0159] FIG. 15b is an example of a front view and a side view of an accessory electronic device according to one embodiment of the present disclosure.

[0160] An accessory electronic device according to one embodiment (e.g., accessory electronic device (201) of FIG. 2) may include a passive infrared sensor (250d). The passive infrared sensor (250d) according to one embodiment may include a thermoelectric sensor (sensor die) (1502) having a size of 2X2. In FIG. 15A, the size of the thermoelectric sensor (1502) is exemplary, and as the size increases, the infrared detection area may become wider.

[0161] A passive infrared sensor (250d) according to one embodiment may be provided with a lens (e.g., F-lens) (1501) to adjust the sensing angle and temperature range of the sensor. In order to appropriately adjust the sensing range (1510) of the thermoelectric sensor (1502), the position and direction (angle) of the lens (1501) may be appropriately adjusted.

[0162] Referring to FIG. 15b, on the front side of the accessory electronic device (201), the sensing range (1510) from the lens (1501) may be directed toward the front side (arrow).

[0163] On the side surface of the accessory electronic device (201), a sensing range (1520) from the lens (1501) may be positioned along the wearing direction (arrow). In order to detect the folding motion of a finger wearing the accessory electronic device (201), a sensing range directed toward the fingertip, such as on the side surface, may be required.

[0164] The embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0165] The term "module" used in the embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0166] One embodiment of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0167] According to one embodiment, the method according to one embodiment disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0168] According to one embodiment, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and arranged in other components. According to one embodiment, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to one embodiment, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In wearable electronic devices, A gyro sensor that detects rotational motion in the roll, pitch, and yaw directions; A passive infrared sensor that detects infrared radiation; communication circuit; memory for storing at least one program; and At least one processor electrically connected to the memory and executing at least one instruction of a program stored in the memory, At least one processor of the above: In response to a first movement of a finger worn by the wearable electronic device, the wearable electronic device recognizes a first angular change in the y-axis direction within a predetermined sensing range based on at least one sensor data acquired according to a pitch direction rotation of the gyro sensor, In response to the detection of a part of the body in a first direction by the passive infrared sensor while the at least one sensor data is acquired, correcting at least a part of the first angle change based on a set reference point, and calculating a movement of the first y-axis value within the sensing range corresponding to the corrected first angle change, and In response to the movement of the first y-axis value being in an increasing direction, the first movement is identified as a scroll up gesture, A wearable electronic device that identifies the first movement as a scroll down gesture in response to the movement of the first y-axis value being in a decreasing direction.

2. In paragraph 1, At least one processor, In response to the micro-vibration being detected by the above gyro sensor, the first movement is identified as a movement in contact with the surface, A wearable electronic device, wherein the first movement is identified as a movement in a state of being lifted from a surface in response to the micro-vibration not being detected by the gyro sensor.

3. In paragraph 1, At least one processor, A wearable electronic device that identifies the first movement as a long press gesture when the finger of a user wearing the wearable electronic device is detected by the passive infrared sensor and the duration of the movement exceeds a threshold time.

4. In paragraph 1, At least one processor, A wearable electronic device, wherein the wearable electronic device detects a vibration value generated when the finger on which the wearable electronic device is worn comes into contact with the adjacent finger using the gyro sensor in response to a second movement by the worn finger and the adjacent finger, and in response to detecting the adjacent finger in a second direction using the passive infrared sensor, identifies that the second movement corresponds to a tap gesture.

5. In paragraph 4, At least one processor, A wearable electronic device that identifies the second movement as a long tap gesture when the time for which the adjacent finger is detected in the second direction by the passive infrared sensor exceeds a threshold time.

6. In paragraph 1, At least one processor, A wearable electronic device, wherein the first movement is identified as a gesture of lifting the worn finger in response to the first angular change being in a direction greater than a predetermined threshold angle.

7. In paragraph 6, At least one processor, A wearable electronic device that identifies one or more detailed gestures by reflecting the degree to which the worn finger is lifted, depending on the degree to which the change in the first angle is greater than the determined threshold angle.

8. In paragraph 1, A wearable electronic device, wherein the above-determined sensing range is determined by a motion of maximally moving the user's finger left and right while pinning the finger and a motion of maximally bending the finger inward while fully pinning the finger.

9. In paragraph 1, The above passive infrared sensor comprises a plurality of thermoelectric sensors, A wearable electronic device that identifies the direction of entry of infrared rays of the first movement detected by the plurality of thermoelectric sensors.

10. In paragraph 9, The above passive infrared sensor includes a lens, A wearable electronic device, wherein the sensing range and direction of the thermoelectric sensor are determined in response to the position and direction of the lens.

11. In electronic devices, camera; display; communication circuit; memory for storing at least one program; and At least one processor electrically connected to the memory and executing at least one instruction of a program stored in the memory, At least one processor, Receive first sensor data acquired by a gyro sensor and second sensor data acquired by a passive infrared sensor from a wearable electronic device through the above communication circuit, Based on the first sensor data, the wearable electronic device calculates the y-axis angle of the user's finger, In response to detecting that the finger is folded based on the second sensor data, the y-axis angle is compensated based on a predetermined reference point, and An electronic device that generates a control signal corresponding to the compensated y-axis angle, thereby controlling a screen output through the display.

12. In paragraph 11, At least one processor, In response to the second sensor data including an infrared detection signal in the direction in which the wearable electronic device is worn, detecting that the worn finger is folded, An electronic device that detects that the worn finger is in a pinned state in response to the second sensor data not including an infrared detection signal.

13. In paragraph 11, At least one processor, In response to the detection of micro-vibrations based on the first sensor data, identify that the worn finger is in contact with the surface, An electronic device that identifies that the worn finger is floating on a surface in response to no micro-vibration being detected based on the first sensor data.

14. In paragraph 11, At least one processor, An electronic device that identifies a long press gesture in response to a duration in which the worn finger is detected exceeding a predetermined threshold time based on the first sensor data.

15. In paragraph 11, At least one processor above Based on the first sensor data, the worn finger and the wearable electronic device detect vibrations generated when the worn finger and an adjacent finger come into contact with each other, Based on the second sensor data, identify the direction in which the adjacent finger is detected, An electronic device that identifies a tap gesture by the worn finger and the adjacent finger in response to the direction in which the adjacent finger is detected being different from the direction in which the wearable electronic device is worn.

Images