Wearable device and method for providing feedback of wearable device

Abstract

According to one disclosed embodiment, a method for providing feedback for a wearable device includes: detecting a user's movement; determining a body part to which a feedback signal is to be sent based on the detected user movement; generating the feedback signal; and outputting the feedback signal, wherein the feedback signal is an electrical signal for stimulating muscles of the body part.

Description

[0001] This application is a divisional application of a patent application with application number 201680027770.2, filed on March 7, 2016, and entitled “Wearable device and method for providing feedback for a wearable device.” Technical Field

[0002] The present inventive concept relates to a wearable device and a method for providing feedback via the wearable device. Background Art

[0003] In human-computer interaction (HCI), users have traditionally used devices such as keyboards and mice to perform input operations. In recent years, input operations can be performed by mapping a virtual user interface (UI) onto space to recognize user movements.

[0004] For the results of the input processing and specific processing operations described above, the electronic device can provide feedback to the user. The electronic device can provide feedback to the user about the completion of the input operation or feedback about the specific results by generating vibration or sound or by displaying specific graphic elements on the screen display. Summary of the Invention

[0005] Technical issues

[0006] Embodiments disclosed herein relate to a feedback device configured to enhance realism by simulating physical sensations through a method and apparatus for sending feedback signals to body parts corresponding to user movements.

[0007] Technical Solution

[0008] A method for providing feedback performed by a wearable device is provided, the method comprising: detecting user movement; determining a body part to which a feedback signal is to be sent based on the detected user movement; generating a feedback signal; and outputting the feedback signal, wherein the feedback signal is an electrical signal for stimulating muscles of the body part.

[0009] A wearable device is provided, comprising: an electrode sheet located on a user's body; an input interface configured to detect the user's movement; a controller configured to determine a body part to which a feedback signal is to be sent based on the detected user's movement; and an output interface configured to output the feedback signal via the electrode sheet located on the body part, wherein the feedback signal is an electrical signal for stimulating muscles in the body part. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a schematic diagram for describing a method for transmitting a feedback signal according to an embodiment.

[0011] Figure 2 It is a flowchart of a method for providing feedback according to an embodiment.

[0012] Figure 3 It is a flowchart of a method for providing feedback according to an embodiment.

[0013] Figure 4a It is a block diagram of a wearable device according to an embodiment.

[0014] Figure 4b It is a block diagram for describing a method of detecting a user's movement by using depth data and electromyogram (EMG) signals.

[0015] Figure 4c It is a block diagram for describing a method of detecting a user's movement by using depth data and EMG signals.

[0016] Figure 5 It is a view for describing an example of an EMG signal generated during contraction and relaxation of a muscle.

[0017] Figure 6 It is a schematic diagram for describing a wearable device according to an embodiment.

[0018] Figure 7 It is a diagram for describing the electrode type of a wearable device according to another embodiment.

[0019] Figure 8 It is a diagram for describing the electrode type of a wearable device according to another embodiment.

[0020] Figure 9 It is a diagram for describing the electrode type of a wearable device according to another embodiment.

[0021] Figures 10a to 10d It is a view for describing the muscle connected to the body part to which a feedback signal is to be sent through a wearable device.

[0022] Figure 11 It is a schematic diagram for describing a method of sending a feedback signal according to an embodiment.

[0023] Figure 12a It is a flowchart of a method for sending a feedback signal according to an embodiment.

[0024] Figure 12b It is a flowchart of a method for sending a feedback signal according to an embodiment.

[0025] Figure 13 It is a view for describing an operation of changing a user's movement by sending a feedback signal.

[0026] Figure 14 It is a view of a smart garment according to an embodiment.

[0027] Figure 15 is a view of a smart garment according to another embodiment.

[0028] Figure 16a is a flowchart of a method for outputting a feedback signal according to an embodiment.

[0029] Figure 16b is a flowchart of a method for outputting a feedback signal according to an embodiment.

[0030] Figure 17 is a block diagram of a wearable device according to an embodiment.

[0031] Figure 18 is a block diagram of a wearable device according to another embodiment.

[0032] Figure 19 is a flowchart of a method performed by a wearable device to detect movement of the musculoskeletal system by detecting a potential.

[0033] Best Mode

[0034] According to one aspect of the inventive concept, there is provided a method performed by a wearable device to provide feedback, the method including: detecting user movement; determining a body part to which a feedback signal is to be sent based on the detected user movement; generating a feedback signal; and outputting the feedback signal, wherein the feedback signal is an electrical signal for stimulating a muscle of the body part.

[0035] According to another aspect of the inventive concept, there is provided a wearable device including: electrode pads located on a user's body; an input interface configured to detect user movement; a controller configured to determine a body part to which a feedback signal is to be sent based on the detected user movement; and an output interface configured to output a feedback signal via an electrode pad located on the body part, wherein the feedback signal is an electrical signal for stimulating a muscle of the body part. Detailed Description

[0036] Figure 1 is a schematic diagram for describing a method of sending a feedback signal according to an embodiment. Refer to Figure 1 , the wearable device 14 may detect a body part of the user that has performed a spatial input operation and may send a feedback signal to the detected body part.

[0037] The smart glasses 12 may detect the air conditioner 11. The smart glasses 12 may detect an electronic device that the user wants to control by tracking the user's both eyes.

[0038] The smart glasses 12 can display a control menu 13 configured to control the air conditioner 11. For example, the smart glasses 12 can display a control menu 13 configured to set the temperature of the air conditioner 11. The smart glasses 12 can display the control menu 13 in a virtual space. Thus, a user wearing the smart glasses 12 can see the control menu 13 via the smart glasses 12.

[0039] The user can touch the virtual screen. The user can perform an input operation by touching the virtual screen via the smart glasses 12. For example, the user can increase or decrease the temperature of the air conditioner 11 by touching a button displayed on the virtual screen.

[0040] The smart glasses 12 can control the air conditioner 11 according to the user input. The smart glasses 12 can detect the user movement used in the input operation and can recognize the user movement as a control signal to increase or decrease the temperature of the air conditioner 11.

[0041] The wearable device 14 can provide a feedback signal to the user. The wearable device 14 can receive the user movement from the smart glasses 12 and send the feedback signal to a body part of the user based on the user movement. The feedback signal can be an electrical signal for stimulating the muscles of the body part. For example, when the user has touched the virtual screen with the index finger, the smart glasses 12 can detect the user's index finger, can detect an event that the index finger approaches a specific user interface (UI) of the virtual screen by extracting a depth value, and can send the detected event and the depth value to the wearable device 14 as a detection result. Based on the sent data, the wearable device 14 can send an electrical signal to the muscle or muscle bundle connected to the user's index finger. The wearable device 14 can send an electro - muscular stimulation to the muscle or muscle bundle so as to provide the user with a physical feedback that the user has touched the virtual menu via the repulsive force of the musculoskeletal system.

[0042] When the user performs an input operation via a movement or gesture in the virtual space, the user can physically recognize the input operation even if the user does not actually physically contact the space. Thus, for the user's movement or gesture, the wearable device 14 can provide the user with a physical feedback on the completion of the input operation or a physical feedback such as a depth experience.

[0043] Figure 2 is a flowchart of a method for providing feedback according to an embodiment. Referring to Figure 2 , the wearable device 100 can send an electrical signal to the feedback part based on the user movement received from the electronic device 200.

[0044] In operation S210, the electronic device 200 may detect user movement. The electronic device 200 may detect user movement by using a camera, a depth camera, an infrared sensor, an ultrasonic sensor, a motion sensor, etc. For example, the electronic device 200 may detect the finger used by the user to perform a spatial input operation, what kind of movement the user is making, etc.

[0045] In operation S220, the electronic device 200 may send the detection result to the wearable device 100.

[0046] In operation S230, the wearable device 100 may determine a feedback part based on the detection result. The feedback part represents a part of the user's body. For example, the feedback part may include the finger, fingertip, neck, back, knee, etc. of the user used in the input operation. The wearable device 100 may determine the part of the user's body to which the feedback signal is to be sent based on the user movement. The wearable device 100 may determine the feedback part based on the detection result received from the electronic device 200 and the measured surface electromyogram signal (hereinafter referred to as the sEMG signal).

[0047] In operation S240, the wearable device 100 may determine the feedback intensity. The feedback intensity refers to the amplitude of the electrical signal sent by the wearable device 100 to the user. The wearable device 100 may differentially set the feedback intensity based on the user, the external environment (such as the current temperature, humidity, etc.), the user settings, or the learning result. Similarly, the wearable device 100 may set the feedback intensity by considering at least two of the user, the external environment (such as the current temperature, humidity, etc.), the user settings, and the learning result.

[0048] In operation S250, the wearable device 100 may determine the feedback direction. The feedback direction refers to the direction in which the wearable device 100 sends the electrical signal to the user. The wearable device 100 may sequentially send the electrical signal to multiple body parts. The feedback direction refers to the direction of the sequentially output electrical signals. The wearable device 100 may differentially set the feedback direction based on the user, the external environment (such as the current temperature, humidity, etc.), the user settings, or the learning result. Similarly, the wearable device 100 may set the feedback direction by considering at least two of the user, the external environment (such as the current temperature, humidity, etc.), the user settings, and the learning result.

[0049] In operation S260, the wearable device 100 may determine a feedback period. The feedback period may refer to the time interval at which the wearable device 100 outputs a feedback signal. Alternatively, the feedback period may refer to the mode at which the wearable device 100 outputs a feedback signal. The wearable device 100 may differentially set the feedback period based on the user, the external environment (such as, current temperature, humidity, etc.), user settings, or learning results. Similarly, the wearable device 100 may set the feedback period by considering at least two of the user, the external environment (such as, current temperature, humidity, etc.), user settings, and learning results.

[0050] In operation S270, the wearable device 100 may send an electrical signal to the feedback part. For example, the wearable device 100 may generate an electrical muscle stimulation signal and transmit the electrical muscle stimulation signal via an electrode patch. The electrode patch may be located on or attached to the user's body. The wearable device 100 may send the feedback signal to the user based on the method determined in operations S230 to S260. In other words, the wearable device 100 may send the feedback signal to the determined feedback part based on the determined intensity, direction, and period.

[0051] According to the present embodiment, the feedback part is determined in the above order (S230), the feedback intensity is determined (S240), the feedback direction is determined (S250), and the feedback period is determined (S260). However, the order of operations S230 to S260 may be changed, and operations S230 to S260 may be performed simultaneously according to specific circumstances.

[0052] Figure 3 is a flowchart of a method for providing feedback according to an embodiment. Referring to Figure 3 , the wearable device 100 may output an electrical signal based on the result of the determined feedback method received from the electronic device 200.

[0053] According to Figure 2 the flowchart illustrated in, the wearable device 100 determines all these feedback methods. However, according to Figure 3 the flowchart illustrated in, the electronic device 200 may determine the feedback method and send the determined feedback method to the wearable device 100. Accordingly, in operation S370, the wearable device 100 may output an electrical signal based on the received feedback method.

[0054] Figure 3 Operations S310 to S360 of

[0055] When the wearable device 100 includes only an electrical signal generator, the electronic device 200 (rather than the wearable device 100) may determine the feedback method. Accordingly, the wearable device 100 may be manufactured to have a simple structure.

[0056] Figure 4a is a block diagram of a wearable device 400 according to an embodiment. Referring to Figure 4a , the wearable device 400 may include an input interface 410, a controller 420, a communicator 430, a memory 440, and an output interface 450.

[0057] The input interface 410 may detect user movement. The input interface 410 may detect user movement by measuring sEMG signals generated from the user's muscles. Similarly, the input interface 410 may detect user movement by photographing the user movement. Similarly, the input interface 410 may detect user movement by using a variety of sensors.

[0058] The input interface 410 may include a sensor 411, an analog-to-digital conversion (ADC) filter 412, and a camera 413. The sensor 411 may include a temperature sensor, a humidity sensor, an EMG signal measuring device, etc. The EMG signal measuring device may include a conductive electrode, an induction electrode, etc., and may measure electrical signals generated from the user's muscles.

[0059] The ADC filter 412 may extract only the activation signal of the user's muscles from the signals received from the conductive electrode or the induction electrode, and may convert the extracted activation signal into a digital signal.

[0060] The camera 413 may photograph the user and output the captured image to the controller 420. For example, the camera 413 may be a depth camera, and the depth camera may obtain the depth information of the user. Accordingly, the input interface 410 may compare the change in the depth value received from the depth camera, extract the user's contour, and track the change in the user's movement.

[0061] The input interface 410 may further include a sensor configured to track the wearing position of the wearable device 400. For example, the input interface 410 may further include an acceleration sensor, a gyro sensor, a motion sensor, an EMG sensor, an infrared sensor, an ultrasonic sensor, etc., and may detect user movement by using these sensors.

[0062] The input interface 410 may detect user movement by using the signals received from at least two sensors. For example, the input interface 410 may detect the overall movement of the user by using the image captured by the camera 413 and may detect the detailed movement of the user by using the data detected by the EMG sensor.

[0063] The wearable device 400 can change its wearing position over time. Thus, the input interface 410 can track the wearing position of the wearable device 400 by using a variety of sensors and output the tracking result to the controller 420.

[0064] The controller 420 can operate and control a signal processing module, an electrode selection module, a signal generation module, a learning module, a wearing position determination module, and a ground control module. The signal processing module, the electrode selection module, the signal generation module, the learning module, the wearing position determination module, and the ground control module can be stored in the memory 440. The controller 420 can output a feedback signal to the user by controlling the output interface 450 based on the signal received from the input interface 410.

[0065] The controller 420 can not only output a feedback signal based on the user's movement, but also track the change in the tension of the user's muscles and use this tension change as an input. The change in the tension of the muscles can be measured via the measured sEMG signal. For example, the user can virtually obtain an object via a gripping gesture of the hand on an object displayed in a virtual space. After that, when the controller 420 senses an increase in the muscle tension, even if the user's gesture does not change, the controller 420 can determine that the user has applied a grasping force to the object and can execute an event related to applying a grasping force to the virtual object. Thus, the controller 420 can detect the user's intention based on the sEMG signal and process events based on the user's intention, thereby interacting with the user.

[0066] The controller 420 can determine a feedback method based on the signal received from the input interface 410. The signal processing module can determine the body part to which the feedback signal is to be sent, as well as the intensity, direction, and period of the feedback signal.

[0067] The controller 420 can select the electrodes to which the feedback signal is to be output based on the determined feedback method. The electrode selection module can select at least one electrode or a group of electrodes. The electrode selection module can select the electrode pads to be turned on or off, or select the electrodes to be turned on or off.

[0068] The signal generation module can generate a feedback signal via a predefined algorithm and reference values. Similarly, the signal generation module can dynamically change the feedback signal based on the value calculated via the learning module.

[0069] The learning module can calculate the intensity, direction, period, etc. of the feedback signal suitable for the user based on the user's movement received via the input interface 410 or the communicator 430, and the data received from the reference database (DB) 442. The learning module can send data such as the calculated intensity, direction, period, etc. to the signal processing module.

[0070] The wearing position determination module can track the wearing position of the wearable device 400 and determine the current wearing position. The wearing position determination module can control a device configured to prevent a change in the wearing position.

[0071] The ground control module can prevent an electro-muscular stimulation (EMS) signal from being emitted to an unintended body part. The ground control module can maintain a ground state around the electrode to which the EMS signal is to be emitted. The ground control module can maintain the ground state around the electrode to which the EMS signal is to be emitted by outputting a signal to a ground electrode around the electrode to which the EMS signal is to be emitted. The ground control module can control the on / off state of the ground electrode by outputting a signal to the ground electrode.

[0072] The communicator 430 can receive a detection result of a user's movement from an electronic device other than the wearable device 400. The communicator 430 can send data to and receive data from the other electronic device via Zigbee, Wifi, Bluetooth, etc.

[0073] Under the control of the controller 420, the memory 440 can store a DB 411 related to learning data (hereinafter referred to as "learning DB") and a DB 442 related to a reference (hereinafter referred to as "reference DB"). The learning data DB 441 can store learning data on a user's usage characteristics and muscle activation. The learning data DB 441 can output the optimal EMS intensity, optimal EMS direction, optimal EMS period, etc. for each user to the controller 420. Similarly, the learning data DB 441 can store the sensitivity and muscle activation degree learned for each user.

[0074] The reference DB 442 can provide a standard reference muscle activation degree to the controller 420. The reference DB 442 can store a reference model based on a user's height, weight, etc.

[0075] The output interface 450 may include a digital-to-analog conversion (DAC) filter 451, a motor / pump 452, an EMS transmitter 453, and an EMS eliminator 454. The DAC filter 451 may convert the digital signal received from the controller 420 into an analog signal. The motor / pump 452 may control a device configured to prevent a change in the wearing position. The EMS transmitter 453 may emit an EMS signal via an electrode. Corresponding to the EMS transmitter 453, the EMS eliminator 454 may generate an activation pattern and sequence of a ground electrode configured to prevent the emission of the EMS signal. The EMS eliminator 454 may appropriately eliminate the EMS signal based on the intensity and direction of the electrical signal emitted from the EMS transmitter 453 and the type of the target muscle. Also, the EMS eliminator 454 may prevent conductive signals propagating through the inner and outer portions of the user's skin from invading the area where non-target muscles are located, thereby controlling the propagation of the EMS signal caused by the conductivity of the skin and the human body.

[0076] Figure 4b is a block diagram for describing a method of detecting a user's movement by using depth data and an EMG signal. Referring to Figure 4b , the wearable device 400 may detect the user's movement by receiving depth data from the synchronization device 500.

[0077] The synchronization device 500 may be a device wirelessly or wiredly connected to the wearable device 400. The synchronization device 500 may receive data from the wearable device 400 or send data to the wearable device 400.

[0078] The synchronization device 500 may include a depth camera 510, a controller 520, and a communicator 530. The depth camera 510 may capture an image of the user and generate depth data. The depth data may be output to the controller 520 and sent to the wearable device 400 via the communicator 530.

[0079] The depth camera 510 may use the time-of-flight (TOF) method and the pattern change capture method. According to the TOF method, the depth is calculated by identifying the phase change of the reflected wave. The depth camera 510 may use infrared rays. In an environment where the infrared rays are strongly incident, the recognition rate of the depth camera 510 may be reduced. Therefore, the wearable device 400 may detect the user's movement by using not only the depth data but also the sEMG signal measured by the input interface 410.

[0080] For parts where the depth measurement via the depth camera 510 is unreliable, the wearable device 400 can use EMG signals to supplement the depth data. The input interface 410 can measure the EMG signals by using electrode patches located on the user's local musculoskeletal system. The wearable device 400 can estimate the movement of the user's musculoskeletal system by filtering the sEMG signals and extracting the patterns or maximum and minimum values of the sEMG signals. Thus, the wearable device 400 can detect the user's movement by supplementing the depth data based on the estimated movement of the musculoskeletal system.

[0081] The controller 420 can detect the user's movement by using the input data received from the input interface 410 and the depth data. For example, the controller 420 can detect the overall user movement by using the depth data and can detect the detailed user movement by using the input data. The controller 420 can extract the user's contour by using the depth data and track the change of the movement state by using the input data. The input data can be EMG signals. The controller 420 can detect the movement by analyzing the EMG signals. Since the controller 420 uses the depth data and the input data, the controller 420 can accurately detect the user's movement.

[0082] Figure 4c is a block diagram for describing a method of detecting the user's movement by using the depth data and the EMG signals. The synchronization device 500 can perform calculations by using the input data and the depth data and output the calculation results to the wearable device 400. The calculation results can refer to the user's movement. In other words, the synchronization device 500 can detect the user's movement by using the input data and the depth data and output the detection results to the wearable device 400. For example, the wearable device 400 can measure the sEMG signals generated due to the movement of the user's muscles and send the sEMG signals to the synchronization device 500. The synchronization device 500 can supplement the depth data or provide additional information by using the received sEMG signals.

[0083] The synchronization device 500 can track the user's emotional state or position change by analyzing the patterns of the sEMG signals or the changes in the sEMG signals. Alternatively, the synchronization device 500 can track the user's emotional state or position change by analyzing the patterns of the sEMG signals or comparing the changes in the sEMG signals with a reference.

[0084] The controller 520 may receive depth data from the depth camera 510 and may receive input data from the wearable device 400. The controller 520 may perform calculations by using the depth data and the input data and may output the calculation result to the wearable device 400 again. Accordingly, the wearable device 400 does not have to perform the calculation and may control the output interface 450 to output a feedback signal based on the received calculation result.

[0085] Figure 5 is a view for describing an example of EMG signals 21 and 22 generated during contraction and relaxation of the muscle 20. As Figure 5 shown, the EMG signal 21 generated when the muscle 20 contracts and the EMG signal 22 generated when the muscle 20 relaxes are different from each other. The wearable device 400 may detect a user's movement via the position, pattern, etc. of the EMG signals 21 and 22 measured. Similarly, the wearable device 400 may induce contraction or relaxation of the muscle 20 by transmitting a signal having the same pattern as the EMG signal 21 or 22.

[0086] Figure 6 is a schematic diagram for describing a wearable device 600 according to an embodiment. Referring to Figure 6 , the wearable device 600 may be of a type that can be worn on the wrist. The wearable device 600 may include a Velcro 620 and electrode pads 610. A user may fix the wearable device 600 to the user's wrist by using the Velcro 620. The user may fix the wearable device 600 such that the electrode pads 610 are in contact with the user's skin.

[0087] When the user wears the wearable device 600, the electrode pads 610 are in contact with the user's body. Similarly, when the user wears the wearable device 600 on the wrist, the electrode pads 610 are located in a region where the muscles of the fingers are separated. Accordingly, the wearable device 600 may output an EMS signal to the muscles of the finger to which the feedback signal is to be sent. For Figure 6 , the case of wearing the wearable device 600 on the wrist is described. However, the wearable device 600 may be worn on various parts of the user's body, such as the finger, shoulder, thigh, etc. Accordingly, the wearable device 600 may be worn on the body part to which the feedback signal is to be sent and the wearable device 600 may send a feedback signal based on the user's movement.

[0088] The electrode pads 610 may sense an EMG signal generated based on the user's movement. The electrode pads 610 may sense an EMG signal generated from the body via the electrodes 611. Accordingly, the wearable device 600 may detect the user's movement according to the position of the electrodes 611 via which the EMG signal is sensed.

[0089] The electrode patch 610 may include electrodes 611 that output EMS signals to a user's wrist. In other words, the electrode patch 610 may include a plurality of electrodes 611. Based on the muscles, the electrodes 611 may have different distributions, shapes, sizes, etc.

[0090] The electrode patch 610 may be used for inputting and outputting signals. In other words, the wearable device 600 may output EMS signals or receive EMG signals via the electrode patch 610.

[0091] The wearable device 600 may send feedback signals to individual fingers. The electrode patch 610 may output EMS signals via some of the plurality of electrodes 611. The wearable device 600 may determine the finger to which the EMS signal is to be sent and determine the electrode 611 in contact with the muscle of the determined finger. The wearable device 600 may output the EMS signal via the determined electrode 611.

[0092] Figure 6 The case where the electrode patch 610 has a linear shape is shown. However, the shape of the electrode patch 610 may vary so that the electrode patch 610 can easily come into contact with the user's body. Also, the shape and position of the muscles are different for each body part of the user. Therefore, the electrode patch 610 may be formed in different shapes for each body part where the electrode patch 610 is located, or the electrode patch 610 may include electrodes 611 having different shapes. According to another embodiment, the electrode patch 610 may be formed to be coupled to the body.

[0093] The wearable device 600 may detect a finger by detecting a user's movement, and send a feedback signal to the finger of the user for performing a spatial input operation or to the muscle connected to the finger. The position where the EMG signal is generated, the format of the EMG signal, the intensity of the EMG signal, etc. may all vary based on the user's movement. Therefore, the wearable device 600 may detect the user's movement based on the sensed EMG signal. The wearable device 600 may select the electrode 611 to which the feedback signal is to be sent based on the user's movement, and also output the feedback signal (i.e., an electrical signal) to the selected electrode 611. The wearable device 600 may output the feedback signal to a plurality of electrodes 611, and may differentially set the time point at which the electrical signal is output to the plurality of electrodes 611, the duration of the electrical signal, etc.

[0094] Figure 7 is a diagram for describing the electrode type of the wearable device 600 according to another embodiment. Refer to Figure 7 , the wearable device 600 may further include a ground electrode 630. The ground electrode 630 may be formed around the electrode patch 610. The ground electrode 630 may prevent the EMS signal output to the electrode patch 610 from being emitted to the surrounding environment.

[0095] Figure 8 is a diagram for describing the types of electrodes of the wearable device 600 according to another embodiment. Referring to Figure 8 , the wearable device 600 may include a plurality of electrode sheets 610. These plurality of electrode sheets 610 may be electrically separated from each other via a ground electrode 630. The electrode sheet 610 may have a square shape. The wearable device 600 may determine the electrode sheet 610 to which the EMS signal is to be output and may determine one electrode among the electrodes 611 included in the determined electrode sheet 610 to which the EMS signal is to be output.

[0096] Figure 9 is a diagram for describing the types of electrodes of the wearable device 600 according to another embodiment. Referring to Figure 9 , the wearable device 600 may include a polygonal electrode sheet 610. Figure 9 shows, for example, a hexagonal electrode sheet 610. However, the electrode sheet 610 may have a pentagonal, heptagonal or circular shape. Each hexagonal electrode sheet 610 may be surrounded by a ground electrode 630.

[0097] Figures 10a to 10d is a view for describing the muscles connected to the body part to which the feedback signal is to be sent through the wearable device 600. Referring to Figures 10a to 10d , the wearable device 600 may output a feedback signal so that the thumb, index finger, middle finger, little finger, etc. feel a physical repulsive force. Figure 10a is a view for describing the muscles connected to the thumb, Figure 10b is a view for describing the muscles connected to the index finger, Figure 10c is a view for describing the muscles connected to the middle finger, and Figure 10c is a view for describing the muscles connected to the little finger.

[0098] The wearable device 600 may detect a user's movement and determine which one of the thumb, index finger, middle finger, and little finger the user has used to perform a spatial input operation. For example, when the wearable device 600 determines that the user has performed a spatial input operation by using the thumb, the wearable device 600 may send a feedback signal to any muscle through which the thumb feels a physical repulsive force.

[0099] The wearable device 600 may send a feedback signal to the muscles related to the movements of the thumb, index finger, middle finger, and little finger shown in Figures 10a to 10d so that these fingers can feel a physical repulsive force.

[0100] The wearable device 600 can be in contact with or around the muscles that move the fingers and output an EMS signal to the selected electrodes. The wearable device 600 can determine the intensity, direction, or period of the EMS signal based on the user and output the EMS signal based on the determined intensity, direction, or period.

[0101] Figures 10a to 10d An embodiment is shown in which a feedback signal is output to the muscles related to the movement of the fingers. However, when the user performs a spatial input operation using other body parts besides the fingers, the wearable device 600 can send a feedback signal to the muscles related to the other body parts that have performed the spatial input operation, so that the user can feel a physical repulsive force.

[0102] Figure 11 is a schematic diagram for describing a method of sending a feedback signal according to an embodiment. Refer to Figure 11 , the belt-type feedback device 1110 or the clothing-type feedback device 1120 can determine whether the user's movement corresponds to a reference movement and can send a feedback signal to the body part that does not correspond to the reference movement based on the determination result. The belt-type feedback device 1110 or the clothing-type feedback device 1120 can be an example of a wearable device.

[0103] The belt-type feedback device 1110 or the clothing-type feedback device 1120 can directly detect the user's movement or can receive the detected user's movement from an electronic device located around the user. The belt-type feedback device 1110 or the clothing-type feedback device 1120 can detect the user's movement via an input interface. For example, the belt-type feedback device 1110 or the clothing-type feedback device 1120 can detect the user's movement via the EMG signal received through the electrodes. The belt-type feedback device 1110 or the clothing-type feedback device 1120 can include electrode pads located on the user's body and can sense the EMG signal via the electrodes included in the electrode pads. The controller of the belt-type feedback device 1110 or the clothing-type feedback device 1120 can detect the user's movement based on the body part where the EMG signal is measured, the intensity of the EMG signal, the format of the EMG signal, etc.

[0104] As another example, the belt-type feedback device 1110 or the clothing-type feedback device 1120 can detect the user's movement via a depth image captured by a depth camera through a synchronization device. The depth camera can obtain a depth image by measuring the distance between each body part of the user and the depth camera based on the movement of the user. When the movement of the user changes, the distance between each body part of the user and the depth camera changes. The depth camera can obtain a depth image whenever the movement of the user changes.

[0105] The belt-type feedback device 1110 or the clothing-type feedback device 1120 may receive a depth image from the synchronization device. A controller of the belt-type feedback device 1110 or the clothing-type feedback device 1120 may detect a user's movement via the received depth image. The synchronization device may send the result of detecting the user's movement by using the depth image to the belt-type feedback device 1110 or the clothing-type feedback device 1120. The synchronization device may be an additional device located in front of the user.

[0106] The synchronization device may be connected to the belt-type feedback device 1110 or the clothing-type feedback device 1120 wirelessly or wiredly, and may send data to and receive data from the belt-type feedback device 1110 or the clothing-type feedback device 1120.

[0107] The belt-type feedback device 1110 or the clothing-type feedback device 1120 may compare the user's movement with a reference movement, or may receive a comparison result from the synchronization device. The belt-type feedback device 1110 or the clothing-type feedback device 1120 may determine a body part where the user's movement and the reference movement are different from each other. For example, the belt-type feedback device 1110 or the clothing-type feedback device 1120 may calculate an error between the user's movement and the reference movement by calculating a difference between position coordinates of feature points of the user's movement and the reference movement.

[0108] The belt-type feedback device 1110 or the clothing-type feedback device 1120 can determine the body part to which the electrical signal is to be sent based on the calculated error. The belt-type feedback device 1110 or the clothing-type feedback device 1120 can output an electrical signal to stimulate the determined body part. The electrical signal can be sent to the body part determined based on the error between the user's movement and the reference movement, or the electrical signal can be sent to the muscles around the determined body part, and the electrical signal can be output to reduce the error between the user's movement and the reference movement in order to correct the user's movement. The belt-type feedback device 1110 or the clothing-type feedback device 1120 can output an electrical signal to reduce the error. Alternatively, the belt-type feedback device 1110 or the clothing-type feedback device 1120 can output an electrical signal until the error becomes 0. In other words, the belt-type feedback device 1110 or the clothing-type feedback device 1120 can update the error by re-detecting the user's movement while outputting the electrical signal, and can output the electrical signal via the electrode pads until the error becomes 0. As another example, the belt-type feedback device 1110 or the clothing-type feedback device 1120 can output an electrical signal until the error becomes less than a predetermined threshold. Since the user's movement and the reference movement cannot perfectly correspond to each other, the belt-type feedback device 1110 or the clothing-type feedback device 1120 can set a threshold and output an electrical signal until the error becomes less than the threshold. Here, the threshold can be set to a value such that even if the error is not 0, feedback does not need to be provided.

[0109] Figure 12a is a flowchart of a method for outputting a feedback signal according to an embodiment. Referring to Figure 12a , the wearable device can provide feedback to the user by comparing the user's movement and the reference movement.

[0110] In operation S1210, the wearable device can detect the user's movement. The wearable device can detect the user's movement by using sensors.

[0111] The wearable device can receive the user's physical information from the user or can directly scan the user's body to obtain physical information. Physical information refers to the user's height, appearance, weight, etc.

[0112] The wearable device can measure the user's body ratio. The wearable device can detect the user's feature points (e.g., neck, left elbow, right knee, etc.) and measure the user's body ratio based on the distances between the feature points.

[0113] In operation S1220, the wearable device can determine whether the user's movement corresponds to the reference movement. When it is determined that the user's movement corresponds to the reference movement, the method ends; and when it is determined that the user's movement does not correspond to the reference movement, the method can proceed to operation S1230.

[0114] The user can select a reference motion. Alternatively, the wearable device can scan the user's body and can determine the reference model most similar to the user's body and select the motion included in the reference model that is most similar to the user's motion.

[0115] The wearable device can compare the selected motion with the user's motion. The wearable device can compare the motion of the user's body with the motion of the reference model. The wearable device can compare the feature points of the user's body with the feature points of the reference model, where the feature points of the reference model correspond to the feature points of the user's body. The wearable device can calculate the difference between the feature points of the user's body and the feature points of the reference model. More specifically, the wearable device can make the user's body correspond to the reference model and store the amount of change relative to each feature point to obtain the amount of change of the user's body.

[0116] In operation S1230, the wearable device can provide feedback. When the user's motion does not correspond to the reference motion, the wearable device can send a feedback signal to the user. The wearable device can send the feedback signal to the body part where the user's motion and the reference motion do not correspond to each other, thereby providing the user with feedback on which body part does not correspond to the reference motion. After the wearable device provides feedback, the method can enter operation S1210.

[0117] The wearable device can provide feedback until the wearable device determines that the user's motion corresponds to the reference motion.

[0118] Figure 12b It is a flowchart of a method for sending a feedback signal according to an embodiment.

[0119] In operation S1201, the synchronization device can obtain a depth image via a depth camera. The synchronization device can capture the user based on a predetermined period to obtain a depth image. The synchronization device can detect the user's motion not only using the depth camera but also using various sensors, cameras, etc.

[0120] In operation S1202, the wearable device can receive the depth image from the synchronization device.

[0121] In operation S1203, the wearable device can detect the user's motion via the depth image and can calculate the error between the user's motion and the reference motion.

[0122] In operation S1204, the wearable device can determine whether an error has occurred. The wearable device can determine whether an error has occurred for each body part.

[0123] In operation S1205, the wearable device can determine the electrodes located on the body part where the error has occurred.

[0124] In operation S1206, the wearable device may output an electrical signal to the determined electrodes. The wearable device may output an EMS signal via the electrodes to provide a feedback signal to the user.

[0125] When the user's movement and the reference movement do not correspond to each other, operations S1201 to S1206 may be repeatedly executed, and when the user's movement and the reference movement correspond to each other, the method may end.

[0126] Figure 13 is a view for describing an operation of changing the user's movement by sending a feedback signal. Refer to Figure 13 , when the user performs a yoga movement of bending their back, the user's movement may not be correct. For example, the degree of muscle contraction and relaxation of the user does not correspond to the reference movement. The wearable device may determine whether the user's movement is correct and send a feedback signal to the user.

[0127] The wearable device may detect the user's yoga movement via a depth image. The synchronization device may be located directly in front of the user to capture the user's yoga movement and obtain a depth image. The synchronization device may send the depth image to the wearable device. The wearable device may detect the user's yoga movement via the depth image and may compare the user's yoga movement with the reference movement to determine whether the user's yoga movement is correct. Alternatively, the wearable device may detect the user's yoga movement via the electrical signal sensed by the electrode patches located on the user's body and detect the user's yoga movement by combining the depth image and the electrical signal.

[0128] The wearable device may determine whether the user's movement is correct by comparing the detected yoga movement with the reference movement. For example, the direction in which the user bends their back may be opposite to the direction of the reference movement, and the wearable device may determine the user's back as the body part to which the feedback signal is to be sent. Alternatively, the movement of the user's leg or arm may not correspond to the reference movement, and the wearable device may determine the leg or arm as the body part to which the feedback signal is to be sent.

[0129] The wearable device may send a feedback signal to positions 1 to 3 to change the user's movement. When the user performs movement A indicated by the solid line and the reference movement is movement B indicated by the dashed line, the wearable device may send a feedback signal to positions 1 to 3. The wearable device may send a feedback signal to positions 1 to 3 until the user changes their movement to movement B indicated by the dashed line. The wearable device may determine the direction of the feedback signal in the order from position 1 to position 3 and may reduce the amplitude of the feedback signal in the order from position 1 to position 3.

[0130] According to this embodiment, an example of a user performing yoga movements is described. However, this embodiment can be applied to other sports games that require movement correction or other games that require physical movement.

[0131] Figure 14 FIG. 4 is a view of the smart clothing 1700 according to an embodiment. The smart clothing 1700 may include a plurality of feedback devices 1710.

[0132] The smart clothing 1700 can detect user movement by measuring EMG signals via the feedback devices 1710. The smart clothing 1700 can detect user movement based on the position of the feedback devices 1710 via which the EMG signals are measured, the pattern of the EMG signals, and the like.

[0133] The feedback devices 1710 can be arranged in a circular shape in the smart clothing 1700. Accordingly, the feedback devices 1710 can be in contact with the user's body. The smart clothing 1700 can send a feedback signal to a target body part via the feedback devices 1710. The feedback devices 1710 can generate an EMS signal and output the EMS signal to the user via electrodes.

[0134] The feedback devices 1710 can be a plurality of electrode patches, and each electrode patch can include at least one electrode. The smart clothing 1700 can determine the electrode patch to which the feedback signal is to be output and can activate only the determined electrode patch. The smart clothing 1700 can output the feedback signal to the activated electrode patch. Similarly, the smart clothing 1700 can output the feedback signal only to some of the plurality of electrodes included in one electrode patch to output the feedback signal more precisely.

[0135] Figure 15 FIG. 17 is a view of the smart clothing 1800 according to another embodiment. The smart clothing 1800 may include a plurality of feedback devices 1810, and these feedback devices 1810 can be arranged in the smart clothing 1800 in a horizontal direction, a vertical direction, or in a grid shape. The feedback devices 1810 can be a plurality of electrode patches, and each electrode patch can include a plurality of electrodes. Figure 15 FIG. 19 illustrates a square electrode patch. However, the electrode patch can have various shapes, such as a circular shape, a polygonal shape, and the like.

[0136] The smart clothing 1800 can determine the electrode patch or electrode to which the feedback signal is to be output and can activate only the determined electrode patch or electrode. The smart clothing 1800 can output the feedback signal to the activated electrode patch or electrode to output the feedback signal more precisely.

[0137] Figure 16a FIG. 26 is a flowchart of a method for outputting a feedback signal according to an embodiment.

[0138] In operation S1910, the wearable device can detect user movement. The wearable device can detect user movement by using sensors. Similarly, the wearable device can receive the detection result of user movement from another electronic device.

[0139] In operation S1920, the wearable device can determine the body part to which the feedback signal is to be sent based on the user movement. The wearable device can determine the user body part used for the spatial input operation by the user, or can determine the body part where there is an error between the user movement and the reference movement.

[0140] In operation S1930, the wearable device can send a feedback signal to the determined body part. The wearable device can send a vibration, sound, or electrical signal to the determined body part.

[0141] Figure 16b It is a flowchart of a method for outputting a feedback signal according to an embodiment.

[0142] In operation S1901, the wearable device can measure the EMG signal via the electrode patch. The EMG signal is generated from the muscle based on the user movement. The electrode patch can sense the EMG signal and send the sensed EMG signal to the controller of the wearable device.

[0143] In operation S1902, the wearable device can analyze the position of the measured EMG signal and the pattern of the EMG signal. The wearable device can determine which electrode patch the EMG signal is measured via and analyze the pattern of the measured EMG signal. The wearable device can predict the user movement based on the analysis result.

[0144] In operation S1903, the wearable device can determine the body part to which the EMS signal is to be sent. Based on the user movement, the wearable device can determine the body part to which the EMS signal is to be sent to provide feedback to the user.

[0145] In operation S1904, the wearable device can determine the electrode patch located on the determined body part.

[0146] In operation S1905, the wearable device can activate the determined electrode patch. Alternatively, the wearable device can only activate some of the electrode patches included.

[0147] In operation S1906, the wearable device can output an EMS signal to the determined electrode patch. The wearable device can determine the intensity, pattern, duration, etc. of the EMS signal, and can output the EMS signal with the determined intensity and pattern to the electrode patch during the determined duration.

[0148] Figure 17It is a block diagram of a wearable device 2000 according to an embodiment. Referring to Figure 17 , the wearable device 2000 may include an input interface 2010, a processor 2020, an output interface 2030, and electrode pads 2040.

[0149] The input interface 2010 may detect user movement by using sensors. The input interface 2010 may capture the user movement or may measure signals generated from the user's muscles. For example, the input interface 2010 may sense EMG signals from electrodes located on the user's body, and may detect user movement by analyzing the body part where the EMG signals are sensed and the pattern of the EMG signals.

[0150] The processor 2020 may determine the body part to which a feedback signal is to be sent based on the user movement. The processor 2020 may determine the body part used by the user for a spatial input operation. Alternatively, the processor 2020 may compare a reference movement with the user movement received from the input interface 2010 to determine the body part to which the feedback signal is to be sent. The processor 2020 may output the feedback signal to the muscle or muscle bundle connected to the determined body part so that the determined body part senses a repulsive force.

[0151] The processor 2020 may calculate the error between the user movement and the reference movement and may control the output interface 2030 to output the feedback signal to the determined body part until the error becomes equal to or less than a threshold.

[0152] The processor 2020 may determine the intensity, direction, and period of the feedback signal. The processor 2020 may determine the intensity, direction, and period of the feedback signal based on the learning data stored for each user.

[0153] The output interface 2030 may send the feedback signal to the determined body part. The output interface 2030 may output an electrical signal for stimulating the muscle in the determined body part to the electrode pads 2040 located on the determined body part. For example, the output interface 2030 may generate an EMS signal and send the EMS signal to the body via the electrode pads 2040.

[0154] When the output interface 2030 outputs the feedback signal, the output interface 2030 may output power to the ground electrode around the electrode to which the feedback signal is output, so as to prevent the feedback signal from being emitted to other electrodes or other muscles.

[0155] The electrode pads 2040 may include at least one polygonal electrode pad and a ground electrode. The polygonal electrode pad may be arranged to be surrounded by the ground electrode. For example, the polygonal electrode pad may be hexagonal. However, the polygonal electrode may have various shapes.

[0156] The electrode patch 2040 can be in contact with the body or can be located on the body. Similarly, the electrode patch 2040 can be attached in clothing.

[0157] Figure 18 is a block diagram of a wearable device 2000 according to another embodiment. Referring to Figure 18 , the electrode patch 2040 can be connected to the input interface 2010 and the output interface 2030 via a switch 2050.

[0158] When the electrode patch 2040 is connected to the input interface 2010, the input 210 can receive an EMG signal via the electrode patch 2040. When the electrode patch 2040 is connected to the output interface 2030, the output interface 2030 can output an EMS signal via the electrode patch 2040. Thus, the electrode patch 2040 can be used for both input and output signals, and the input interface 2010 and the output interface 2030 can share the electrode patch 2040.

[0159] Figure 19 is a flowchart of a method performed by the wearable device 2000 to detect movement of the musculoskeletal system by detecting potential.

[0160] In operation S2210, the mode of the wearable device 2000 can be switched. The wearable device 2000 can switch the electrode patch 2040 to an input mode between the input mode and the output mode so as to connect the input interface 2010 to the electrode patch 2040.

[0161] In operation S2220, the input interface 2010 can maintain a standby state until a potential is detected.

[0162] In operation S2230, the input interface 2010 can determine whether an activated potential is detected.

[0163] In operation S2240, the input interface 2010 can determine whether the measured data is greater than a threshold. In other words, the input interface 2010 can determine whether the measured potential is greater than the threshold.

[0164] In operation S2250, the processor 2020 can detect the movement of the local musculoskeletal system by detecting the pattern of the measured potential.

[0165] In operation S2270, the processor 2020 can detect the amount and vector of movement by using the measured data.

[0166] In operation S2280, the processor 2020 can detect the pattern of the measured potential.

[0167] In operation S2290, the processor 2020 may detect the movement of the global musculoskeletal system by integrating data. The processor 2020 may detect the movement of the overall musculoskeletal system by integrating the amount of movement, vector, and potential pattern.

[0168] In operation S2260, the wearable device 2000 may send the detection result to another device, such as the synchronization device 500. The wearable device 2000 may send the results of detecting the movement of the local musculoskeletal system and the movement of the global musculoskeletal system to another device.

[0169] Referring to Figure 19 It is described that the wearable device 2000 sends the detection result to another device. However, the wearable device 2000 may also receive the measured data from another device and may use the detection result and the received measured data to detect the user's movement.

[0170] The wearable device according to an embodiment may send a feedback signal to the body part used by the user for the input operation.

[0171] The wearable device according to an embodiment may send an electrical signal to the body part used by the user for the input operation.

[0172] The wearable device according to an embodiment may send an electrical signal to the body part by comparing the user's movement with a reference movement.

[0173] The device described herein may include a processor, a memory for storing program data and executing the program data, a permanent storage device such as a disk drive, a communication port for handling communication with an external device, and a user interface device such as a touch panel, keys, buttons, etc. Any processing may be implemented as a software module or algorithm and may be stored on a computer-readable medium as program instructions or computer-readable code executable by the processor. These computer-readable media are such as magnetic storage media (e.g., read-only memory (ROM), random access memory (RAM), floppy disks, hard disks, etc.) and optical data reading devices (e.g., CD-ROM, digital versatile disc (DVD), etc.). The non-computer-readable storage media may also be distributed in a networked computer system so that the computer-readable code is stored and executed in a distributed manner. The medium may be read by a computer, stored in the memory, and executed by the processor.

[0174] The present disclosure may be described in terms of functional block components and various processing steps. These functional blocks may be implemented by any number of hardware and / or software components configured to perform the specified functions. For example, the present disclosure may employ various integrated circuit components, such as memory elements, processing elements, logic elements, look-up tables, etc., which may implement various functions under the control of one or more microprocessors or other control devices. Similarly, in the case where elements of the present disclosure are implemented by using software programming or software elements, any programming or scripting language (such as C, C++, Java, assembly, etc.) may be utilized to implement the present disclosure, and various algorithms may be implemented by using any combination of data results, objects, processes, routines, or other programming elements. Moreover, the present disclosure may employ any number of conventional techniques for electronic configuration, signal processing and / or control, data processing, etc. The terms “mechanism”, “element”, “unit” and “component” are used in a broad sense and are not limited to mechanical or physical embodiments, but may include software routines in conjunction with processors, etc.

[0175] The specific embodiments shown and described herein are illustrative examples of the present disclosure and are not intended to limit the scope of the present disclosure in any way. For the sake of brevity, conventional electronic devices, control systems, software development, and other functional aspects of the system (and components of the individual operating components of the system) may not be described in detail. Moreover, the connecting lines or connectors shown in the various figures provided herein are intended to represent exemplary functional relationships and / or physical or logical couplings between the various elements. It should be noted that in actual devices, there may be many alternative or additional functional relationships, physical connections or logical connections.

[0176] In the context of describing the present disclosure (especially in the context of the following claims), the use of the terms “a”, “an”, “the” and similar referents should be construed to cover both the singular and the plural. Moreover, the recitation of ranges of values herein is merely intended to be a shorthand method of referring individually to each separate value falling within the range. Unless otherwise indicated herein, each separate value is incorporated into the specification as if it were individually recited herein. Finally, unless otherwise indicated herein or clearly contradicted by context, the steps of all methods described herein may be performed in any suitable order. Unless otherwise required, the use of any and all examples or exemplary language (e.g., “such as”) provided herein is merely intended to better illustrate the present disclosure and does not constitute a limitation on the scope of the present disclosure. Various modifications and variations can be readily envisioned by those of ordinary skill in the art without departing from the spirit and scope of the present disclosure.

Claims

1. A method for providing a feedback signal by an electronic device, the method comprising: Detecting user movement; Identifying an interaction between the user movement and a user interface provided by the electronic device; Based on the detected interaction, determining whether to output a feedback signal; Based on determining to output the feedback signal, determining a body part of the user to which the feedback signal is to be sent based on the detected interaction; Determining at least one of an intensity, a direction, or a period of the feedback signal; Based on the determined body part and at least one of the intensity of the feedback signal, the direction of the feedback signal, or the period of the feedback signal, generating a control signal to cause the feedback signal to be sent to the determined body part; and Providing the control signal to output the feedback signal to the determined body part.

2. The method according to claim 1, Among them, The detecting the interaction with the user interface includes detecting a user touch input to a virtual object on the user interface, and wherein, the determining whether to output the feedback signal includes determining whether to output the feedback signal based on whether a user touch input to the virtual object is detected.

3. The method according to claim 2, wherein The detecting the user touch input to the virtual object on the user interface includes: Detecting an event that a user finger touches the virtual object; Detecting a depth value of the user finger obtained by a depth camera of the electronic device; and Based on the detected event and the detected depth value, detecting the user touch input to the virtual object.

4. The method according to claim 1, wherein, The detecting the user movement includes: Sensing at least one electromyogram (EMG) signal from at least one electrode located on at least one body part of the user; and Detecting the user movement for interacting with the user interface by analyzing at least one body part from which the at least one EMG signal is sensed and a pattern of the at least one EMG signal.

5. The method according to claim 1, wherein, The determining the body part includes: Determining an error between the detected user movement and a reference movement by comparing the detected user movement with the reference movement; and Based on the determined error between the detected user movement and the reference movement, determining the body part.

6. The method according to claim 1, wherein, The determining the body part includes: Extracting at least one feature point from an image of the user's body obtained by a camera of the electronic device; Determining a body ratio of the user based on the extracted at least one feature point; Selecting a reference movement based on the body ratio of the user; and Based on an error between the detected user movement and the reference movement, determining the body part.

7. The method according to claim 1, the method further comprising controlling another electronic device connected to the electronic device based on the interaction with the user interface.

8. The method according to claim 1, wherein The feedback signal is an electrical signal for stimulating muscles of a body part of the user.

9. The method according to claim 8, Among them, The provided control signal is transmitted to another electronic device, and Wherein, the other electronic device is configured to output the feedback signal to a body part of the user by controlling an electrode located on the body part based on the control signal.

10. The method according to claim 1, Among them, the interaction includes spatial input performed by a body part of the user, and wherein, determining the body part of the user includes: determining that the body part for performing the spatial input is the body part to which the feedback signal is to be sent.

11. An electronic device, the electronic device includes: at least one sensor; a communication interface; and at least one processor, the at least one processor is configured to: detect user movement; identify an interaction between the user movement and a user interface provided by the electronic device; determine whether to output a feedback signal based on the detected interaction with the user interface; based on determining to output the feedback signal, determine the body part of the user to which the feedback signal is to be sent based on the detected interaction; determine at least one of the intensity, direction, or period of the feedback signal; generate a control signal based on the determined body part and at least one of the intensity of the feedback signal, the direction of the feedback signal, or the period of the feedback signal, so that the feedback signal is sent to the determined body part; and control the communication interface to provide the control signal to output the feedback signal to the determined body part.

12. The electronic device according to claim 11, wherein, The at least one processor is further configured to: detect a user touch input to a virtual object on the user interface; and determine whether to output the feedback signal based on whether the user touch input to the virtual object is detected.

13. The electronic device according to claim 12, wherein, The at least one processor is further configured to: detect an event that a user finger touches the virtual object; detect a depth value of the user finger obtained by using a depth camera of the electronic device; and detect a user touch input to the virtual object based on the detected event and the detected depth value.

14. The electronic device according to claim 11, wherein, The at least one processor is further configured to: sense at least one electromyogram (EMG) signal from at least one electrode located on at least one body part of the user; and detect the user movement for interacting with the user interface by analyzing at least one body part from which the at least one EMG signal is sensed and the pattern of the at least one EMG signal.

15. The electronic device according to claim 11, wherein, The at least one processor is further configured to: determine an error between the detected user movement and a reference movement by comparing the detected user movement with the reference movement; and determine the body part based on the determined error between the detected user movement and the reference movement.

16. The electronic device according to claim 11, wherein, The at least one processor is further configured to: extract at least one feature point from an image of the user's body obtained by using a camera of the electronic device; determine the body ratio of the user based on the extracted at least one feature point; select a reference movement based on the body ratio of the user; and determine the body part based on an error between the detected user movement and the reference movement.

17. The electronic device according to claim 11, wherein, The at least one processor is further configured to control the communication interface to provide the control signal to another electronic device, the another electronic device being configured to output the feedback signal to a body part of the user by controlling an electrode located on the body part based on the control signal.

18. The electronic device according to claim 11, wherein, The feedback signal is an electrical signal for stimulating a muscle of a body part of the user.

19. The electronic device according to claim 11, Among them, the interaction includes spatial input performed by a body part of the user, and wherein the at least one processor is further configured to: determine that the body part for performing the spatial input is the body part to which the feedback signal is to be sent.

20. A computer-readable recording medium configured to store one or more computer programs including instructions that, when executed by at least one processor, cause the at least one processor to control: detect user movement; identify an interaction between the user movement and a user interface provided by an electronic device; determine whether to output a feedback signal based on the detected interaction; based on determining to output the feedback signal, determine a body part of the user to which the feedback signal is to be sent based on the detected interaction; determine at least one of an intensity, a direction, or a period of the feedback signal; generate a control signal based on the determined body part and at least one of the intensity of the feedback signal, the direction of the feedback signal, or the period of the feedback signal, so that the feedback signal is sent to the determined body part; and provide the control signal to output the feedback signal to the determined body part.

Images