Electronic device and operation method thereof

Abstract

An electronic device and an operation method thereof are provided. The electronic device comprises: a sensor for acquiring a gesture of a user; a memory for storing one or more instructions; and one or more processors for executing the one or more instructions stored in the memory. The one or more processors execute the one or more instructions to: when a first gesture acquired through the sensor does not correspond to a preset gesture, store, in a database, data related to that the first gesture does not correspond to the preset gesture; and when a second gesture acquired through the sensor after the first gesture corresponds to the preset gesture, set a target action mapped to the second gesture as a target action of the first gesture.

Description

Electronic device and method of operation thereof

[0001] The disclosed various embodiments relate to electronic devices and methods of operating the same, and more particularly to electronic devices that recognize user gestures and methods of operating the same.

[0002] Recent electronic devices offer gesture recognition services that recognize user gestures and perform actions corresponding to the recognized gestures. For example, a user can preset specific gestures and corresponding actions on the electronic device. If the electronic device recognizes a user gesture entered into the device as a preset gesture, it can perform an action corresponding to the preset gesture.

[0003] However, if a non-preset gesture is input to an electronic device, the electronic device may not recognize the gesture. Furthermore, if the user's physical or environmental limitations prevent them from expressing the preset gestures on the electronic device, the gesture recognition service may not be available.

[0004] An electronic device according to one embodiment of the present disclosure includes a sensor for acquiring a user's gesture, a memory for storing one or more instructions, and one or more processors for executing the one or more instructions stored in the memory. The one or more processors execute the one or more instructions so that, if a first gesture acquired through the sensor does not correspond to a preset gesture, data related to the first gesture not corresponding to the preset gesture is stored in a database, and if a second gesture acquired through the sensor after the first gesture corresponds to the preset gesture, a target action mapped to the second gesture is set as the target action of the first gesture.

[0005] A method of operating an electronic device according to one embodiment of the present disclosure includes, if a first gesture acquired through a sensor does not correspond to a preset gesture, storing data related to the first gesture not corresponding to the preset gesture in a database, and if a second gesture acquired through the sensor after the first gesture corresponds to the preset gesture, setting a target action mapped to the second gesture as the target action of the first gesture.

[0006] FIG. 1 is a drawing for explaining an operation of an electronic device providing a gesture recognition service according to one embodiment.

[0007] FIG. 2 is a schematic block diagram of an electronic device according to one embodiment.

[0008] FIG. 3 is a detailed block diagram of a memory of an electronic device according to one embodiment.

[0009] FIG. 4 is a diagram showing a gesture DB and an action DB according to one embodiment.

[0010] FIG. 5 is a diagram showing a history DB according to one embodiment.

[0011] FIG. 6 is a flowchart illustrating an operation method for an electronic device to provide a gesture recognition service according to one embodiment.

[0012] FIG. 7A is a flowchart illustrating an operation performed by an electronic device according to one embodiment when the electronic device identifies a gesture that has failed to be recognized according to the first type.

[0013] FIG. 7B is a flowchart illustrating an operation performed by an electronic device according to one embodiment when the electronic device identifies a gesture that has failed to be recognized according to the first type. FIG. 8 is an example illustrating an operation performed by an electronic device according to one embodiment when the electronic device identifies a gesture that has failed to be recognized according to the first type.

[0014] FIG. 9 is a diagram showing the history DB, gesture DB, and action DB in the first type illustrated in FIG. 8.

[0015] FIG. 10 is a flowchart illustrating an operation performed when an electronic device according to one embodiment identifies a gesture that has failed to be recognized according to a second type.

[0016] FIG. 11 is an example showing an action performed when an electronic device according to one embodiment identifies a gesture that has failed to be recognized according to a second type.

[0017] FIG. 12 is an example showing an action performed when an electronic device according to one embodiment identifies a gesture that failed to be recognized according to a second type.

[0018] FIG. 13 is a diagram showing the history DB, gesture DB, and action DB in the second type illustrated in FIGS. 11 and 12.

[0019] FIG. 14 is an example showing an action performed when an electronic device according to one embodiment identifies a gesture that failed to be recognized according to a second type.

[0020] FIG. 15 is a diagram showing the history DB, gesture DB, and action DB in the second type illustrated in FIG. 14.

[0021] FIG. 16 is a flowchart illustrating an operation performed by an electronic device according to one embodiment when the electronic device identifies a gesture that has failed to be recognized according to the first type or the second type.

[0022] FIG. 17a is a flowchart illustrating the operation of an electronic device and a server according to one embodiment.

[0023] FIG. 17b is a flowchart illustrating the operation of an electronic device and a server according to one embodiment.

[0024] Fig. 18 is a detailed block diagram of an electronic device according to one embodiment.

[0025] In this disclosure, the expression “at least one of a, b or c” may refer to “a”, “b”, “c”, “a and b”, “a and c”, “b and c”, “all of a, b and c”, or variations thereof.

[0026] Below, embodiments of the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein.

[0027] Although the functions mentioned in this disclosure are described using currently used general terms, they may refer to various other terms depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Therefore, the terms used in this disclosure should not be interpreted solely based on their names, but rather based on the meanings of the terms and the overall content of this disclosure.

[0028] Additionally, the terminology used in this disclosure is for the purpose of describing specific embodiments only and is not intended to limit the present disclosure.

[0029] When a part of the specification is said to "include" a component, this does not exclude other components, but rather implies the inclusion of other components, unless otherwise specifically stated. Furthermore, terms such as "part," "module," etc., used throughout the specification refer to a unit that processes at least one function or operation, which may be implemented in hardware, software, or a combination of hardware and software.

[0030] Below, with reference to the attached drawings, embodiments of the present invention are described in detail so that those skilled in the art can easily implement the present invention. However, the present invention can be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description have been omitted to clearly explain the present invention, and similar parts have been designated with similar reference numerals throughout the specification.

[0031] In the embodiments of this specification, the term “user” means a person who controls a system, function or operation, and may include a developer, administrator or installer.

[0032] The present disclosure will be described in detail with reference to the attached drawings below.

[0033] FIG. 1 is a drawing for explaining an operation of an electronic device providing a gesture recognition service according to one embodiment.

[0034] Referring to FIG. 1, an electronic device (100) according to one embodiment of the present disclosure may provide a gesture recognition service. The gesture recognition service according to one embodiment of the present disclosure may be a service that recognizes a user's gesture and provides an action corresponding to the recognized user's gesture. The action corresponding to the gesture is an action that can be activated by the gesture and may include one or more operations that can be activated according to the recognition of the gesture. A user of the electronic device (100) can control the electronic device (100) through gestures without a separate input device.

[0035] An electronic device (100) according to one embodiment of the present disclosure can provide personalized gestures for each user of the electronic device (100). For example, the electronic device (100) can store information regarding preset gestures. For example, the electronic device (100) can map and store preset gestures and target actions corresponding to each preset gesture. Furthermore, for example, a user can set various gestures and corresponding target actions for the electronic device (100).

[0036] An electronic device (100) according to one embodiment of the present disclosure may include at least one of a desktop, a smartphone, a tablet personal computer (PC), a mobile phone, a video phone, an e-book reader, a laptop personal computer (PC), a netbook computer, a digital camera, a personal digital assistant (PDA), a portable multimedia player (PMP), a camcorder, a navigation system, a wearable device, a smart watch, a home network system, a security system, a medical device, a head-mounted display (HMD), a hemispherical display, a large display, and a projector display.

[0037] Alternatively, the electronic device (100) according to one embodiment of the present disclosure may be a device that plays content and provides the content to an external display device. For example, the content may refer to multimedia content and may include, but is not limited to, images, videos, audio, text, games, applications, etc. The electronic device (100) may include a set-top box (STB), a Blu-ray Disc (Blu-ray Disc) player, a DVD (Digital Versatile Disk) player, a game device, a digital camera, a camcorder, a streaming device, a home theater, etc. In this case, the electronic device (100) may be connected to the external display device through wired or wireless communication. For example, the electronic device (100) may be connected to the external display device through an input / output unit such as an HDMI port. However, the present disclosure is not limited thereto, and the electronic device (100) may also be connected to the external display device through short-range wireless communication such as wireless LAN (e.g., Wi-Fi), Bluetooth, or long-range wireless communication.

[0038] In one embodiment of the present disclosure, an electronic device (100) can store information regarding a palm open gesture (10) and information regarding an action mapped to the palm open gesture (10) (20). For example, the palm open gesture (10) can be mapped to a gesture recognition mode launching action. When the electronic device (100) receives a palm open gesture (10) from a user, the electronic device (100) can recognize the palm open gesture (10) and launch the gesture recognition mode. The electronic device (100) can provide a gesture recognition service to the user according to the gesture recognition mode.

[0039] In one embodiment of the present disclosure, when an electronic device (100) acquires a new gesture (50) that is not a preset gesture in the electronic device (100), the electronic device (100) may analyze the new gesture (50) based on information about preset gestures and predict an appropriate target action for the new gesture (50). For example, the electronic device (100) may suggest an optimal action for the new gesture (50) based on at least one of the following: the time for which the new gesture (50) is maintained, the number of times the new gesture (50) is identified, the presence or absence of a gesture identified subsequent to the new gesture (50), characteristic information of the new gesture (50), or the similarity between the new gesture (50) and the preset gesture. In the present disclosure, when the new gesture (50) does not correspond to a preset gesture, the new gesture (50) may be referred to as an 'unrecognized gesture', a 'recognition failure gesture', or a 'first gesture'. Additionally, if a new gesture (50) corresponds to a preset gesture, the new gesture (50) may be referred to as a 'recognized gesture', a 'recognition success gesture', or a 'second gesture'.

[0040] In one embodiment of the present disclosure, when the electronic device (100) identifies a new gesture (50) that does not correspond to a preset gesture, and determines that the gesture identified subsequent to the new gesture (50) corresponds to a preset gesture, the electronic device (100) may set a target action mapped to a subsequent gesture as the target action of the new gesture (50). For example, when the electronic device (100) identifies a subsequent gesture corresponding to a preset gesture after identifying a new gesture (50), the electronic device (100) may determine that the user intends to use the new gesture (50) as the same action as the subsequent gesture. Accordingly, the electronic device (100) may predict that the target action of the new gesture (50) is the same as the target action mapped to the subsequent gesture. This will be described in detail with reference to FIGS. 7A to 9 and FIG. 16.

[0041] Alternatively, in one embodiment of the present disclosure, if the electronic device (100) identifies a new gesture (50) that does not correspond to a preset gesture, and if the new gesture (50) continues to be identified, the electronic device (100) may set a target action of the new gesture (50) based on the similarity between the new gesture (50) and preset gestures. For example, if the electronic device (100) determines that the new gesture (50) is maintained for a preset time or is identified more than a preset number of times, the electronic device (100) may consider that the user is mistaking the new gesture (50) for a preset gesture. For example, if the similarity between the new gesture (50) and the preset gesture is high, the electronic device (100) may set the target action of the new gesture (50) to be the same as the target action mapped to the preset gesture. For example, if the similarity between the new gesture (50) and the preset gesture is low, the electronic device (100) may provide a candidate action list indicating information about target actions set by other users for the new gesture (50). This will be explained in detail in Figs. 10 to 16.

[0042] In one embodiment of the present disclosure, the electronic device (100) can predict a target action for a new gesture (50) and provide the predicted target action to the user. For example, the electronic device (100) can provide the predicted target action to the user through an output interface such as a display or a speaker. For example, the electronic device (100) can provide a user interface (60) such as “A new gesture has been detected. Would you like to set the new gesture as the “A” action?” Alternatively, in one embodiment, when the electronic device (100) is connected to an external display device, the electronic device (100) can provide information regarding the predicted target action to the external display device through wired or wireless communication.

[0043] An electronic device (100) according to one embodiment of the present disclosure can recognize a user's gesture, and if the recognized user's gesture corresponds to a preset gesture, can provide an action mapped to the preset gesture. Even if the user's gesture does not correspond to a preset gesture, the electronic device (100) can suggest a user-customized action for a new gesture. The electronic device (100) can suggest an optimal action for a new gesture based on information about the user's gesture and information about the preset gesture, thereby enhancing interaction between the electronic device (100) and the user. In addition, user convenience of the electronic device (100) can be enhanced.

[0044] FIG. 2 is a schematic block diagram of an electronic device according to one embodiment.

[0045] Referring to FIG. 2, the electronic device (100) may include a processor (110), a memory (120), a communication unit (130), and a sensor (140). However, the electronic device (100) may be implemented with fewer components than the illustrated components, or may be implemented with more components than the illustrated components, and is not limited to the examples described above.

[0046] The memory (120) can store a program for processing and controlling the processor (110), and can store data input to or output from the electronic device (100). The memory (120) can include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk.

[0047] The processor (110) controls the overall operation of the electronic device (100). For example, the processor (110) may perform the functions of the electronic device (100) described in the present disclosure by executing one or more instructions stored in the memory (120). In this case, the memory (120) may store one or more instructions executable by the processor (110).

[0048] Additionally, in one embodiment of the present disclosure, the processor (110) may store one or more instructions in an internally provided memory, and may control the execution of the above-described operations by executing one or more instructions stored in the internally provided memory. That is, the processor (110) may perform a predetermined operation by executing at least one instruction or program stored in an internal memory or memory (120) provided within the processor (110).

[0049] The processor (110) may include one or more processors. At this time, one or more processors may be general-purpose processors such as a CPU, AP, or DSP (Digital Signal Processor), graphics-only processors such as a GPU or VPU (Vision Processing Unit), or artificial intelligence-only processors such as an NPU. If one or more processors are artificial intelligence-only processors, the artificial intelligence-only processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0050] The communicator (130) can transmit and receive data or signals with external devices, peripheral devices, servers, display devices, etc. For example, the communicator (130) can include a Wi-Fi module, a Bluetooth module, an infrared communication module, a wireless communication module, a LAN module, an Ethernet module, a wired communication module, etc. In this case, each communication module can be implemented in the form of at least one hardware chip. The Wi-Fi module and the Bluetooth module perform communication in the Wi-Fi method and the Bluetooth method, respectively. When using the Wi-Fi module or the Bluetooth module, various connection information such as the SSID and session key are first transmitted and received, and after establishing a communication connection using this, various information can be transmitted and received. The wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), 5G (5th Generation), etc.

[0051] The communication unit (130) can periodically receive a candidate action list from the server, which indicates information about target actions set by other users for user gestures. The communication unit (130) can update information about gestures and actions by periodically transmitting new gestures and target actions mapped to the new gestures to the server.

[0052] The sensor (140) can detect the presence, location, and type of the user's spatial gestures, touches, and other user interactions. For example, the sensor (140) may include one or more sensors. For example, the sensor (140) may include a distance sensor, an image sensor, a gesture sensor, an illuminance sensor, and the like. For example, the distance sensor may include various sensors that detect the distance between the electronic device (100) and the user, such as an ultrasonic sensor, an IR (Infrared Radiation) sensor, and a TOF (Time of Flight) sensor. For example, the image sensor may include a camera. The image sensor can capture an image of the user through the camera, convert the captured image into sensing data, and transmit the data to the processor (110).

[0053] The sensor (140) may not be built into the electronic device (100), but may be implemented as a camera of a terminal connected to the electronic device (100), a camera connected via USB, a peripheral IoT (Internet of Things) device, etc.

[0054] According to one embodiment, one or more processors (110) execute one or more instructions so that, if a first gesture acquired through a sensor (140) does not correspond to a preset gesture, data related to the first gesture not corresponding to the preset gesture is stored in a database. One or more processors (110) execute one or more instructions so that, if a second gesture acquired through a sensor (140) after the first gesture corresponds to a preset gesture, a target action mapped to the second gesture is set as a target action of the first gesture.

[0055] According to one embodiment, the electronic device (100) can propose a target action appropriate for the user's gesture based on the user's gesture and subsequent gestures, even if the user's gesture does not correspond to a preset gesture. Accordingly, user convenience for the electronic device (100) can be enhanced.

[0056] According to one embodiment, one or more processors (110) may store data regarding whether a user's gesture (305) obtained through a sensor (140) corresponds to a preset gesture in a database by executing one or more instructions. One or more processors (110) may detect whether data related to whether a user's gesture (305) does not correspond to a preset gesture is stored in the database.

[0057] One or more processors (110) according to one embodiment can identify whether the first gesture is stored in the history DB (360) a preset number of times for a preset time by executing one or more instructions.

[0058] According to one embodiment, one or more processors (110) may execute one or more instructions to set a target action mapped to the second gesture as the target action of the first gesture if the similarity between the acquired first gesture and the second gesture is greater than or equal to a first threshold value based on the second gesture corresponding to a preset gesture.

[0059] According to one embodiment, the electronic device (100) may consider the recognition failure result and, if it determines that the first and second gestures are completely unrelated, may stop the target action prediction operation. Accordingly, the electronic device (100) can efficiently provide gesture recognition services.

[0060] One or more processors (110) according to one embodiment may set a target action of the first gesture based on a similarity between the first gesture and a preset gesture, based on a time for which the first gesture is maintained or a number of times the first gesture is identified, by executing one or more instructions.

[0061] According to one embodiment, an electronic device (100) can propose a target action appropriate for a user's gesture based on the similarity between the user's gesture and the preset gesture, even if the user's gesture does not correspond to a preset gesture. Accordingly, user convenience of the electronic device (100) can be enhanced.

[0062] According to one embodiment, one or more processors (110) may execute one or more instructions to set a target action mapped to a preset gesture as a target action of the first gesture if the similarity between the first gesture and the preset gesture is greater than or equal to a second threshold value.

[0063] According to one embodiment, one or more processors (110) may execute one or more instructions to provide a candidate action list indicating information about a target action set by another user for the first gesture as a target action of the first gesture, if the similarity between the first gesture and the preset gesture is less than a second threshold value.

[0064] According to one embodiment, an electronic device (100) can propose an optimal target action for a user's gesture by suggesting target actions mapped by other users as candidate actions. Accordingly, user convenience for the electronic device (100) can be enhanced.

[0065] An electronic device (100) according to one embodiment may further include a communication unit (130). One or more processors (110) may execute one or more instructions to request a candidate action list for the first gesture from an external device through the communication unit (130) if there is no candidate action list for the first gesture.

[0066] According to one embodiment, one or more processors (110) can be controlled to provide a UI (User Interface) including an inquiry corresponding to setting of a target action for the first gesture, by executing one or more instructions, if the similarity between the first gesture and the preset gesture is less than a second threshold value.

[0067] According to one embodiment, one or more processors (110) can identify whether a user's gesture (305) corresponds to a preset gesture by executing one or more instructions. If the one or more processors (110) identify that the user's gesture (305) corresponds to a preset gesture by executing one or more instructions, the one or more processors (110) can identify a target action mapped to the preset gesture.

[0068] According to one embodiment, one or more processors (110) may acquire feature information (325) corresponding to the user's gesture (305) based on the acquired user's gesture (305) by executing one or more instructions. If the similarity between the feature information (325) and the feature information of the preset gesture is greater than or equal to a reference value, the one or more processors (110) may identify that the user's gesture (305) corresponds to the preset gesture. If the similarity is less than the reference value, the one or more processors (110) may identify that the user's gesture (305) does not correspond to the preset gesture.

[0069] According to one embodiment, one or more processors (110) may receive information from an external device via a communication unit (130) as to whether a user's gesture (305) corresponds to a preset gesture by executing one or more instructions. Upon receiving the information as to whether the user's gesture corresponds to the preset gesture, one or more processors (110) may set a target action corresponding to the user's gesture.

[0070] According to one embodiment, the history DB (360) may store any one of the following: a similarity between a user's gesture (305) and preset gestures, information about a gesture among preset gestures that has the highest similarity to the user's gesture (305), a maximum value of the similarity, a result of recognition failure related to a first gesture not corresponding to a preset gesture, or a result of recognition success related to a second gesture corresponding to a preset gesture.

[0071] One or more processors (110) can update the first gesture and the target action mapped to the first gesture by executing one or more instructions.

[0072] One or more processors (110) can provide a target action for the first gesture by executing one or more instructions, based on at least one of the time the first gesture is maintained, the number of times the first gesture is identified, the presence of a second gesture identified subsequent to the first gesture, characteristic information of the first gesture, or a similarity between the first gesture and a preset gesture, upon identifying the first gesture.

[0073] FIG. 3 is a detailed block diagram of a memory of an electronic device according to one embodiment.

[0074] When FIG. 2 is connected to FIG. 3, the memory (120) of the electronic device (100) can store a data capture module (310), a feature extraction module (320), a gesture control module (330), a gesture database (340), an action database (350), and a history database (360).

[0075] The programs stored in the memory (120) can be classified into multiple modules according to their functions. Specifically, each of the data capture module (310), feature extraction module (320), and gesture control module (330) included in the memory (120) may include appropriate software, logic, circuits, interfaces, and / or codes that can be operated to perform a predetermined function.

[0076] The data capture module (310) can receive sensing data converted from a user's gesture (305) through a sensor (140). For example, when a user's gesture (305) is detected through a camera, the data capture module (310) can capture an image related to the user's gesture (305) on a frame-by-frame basis. For example, the data capture module (310) can store data in which an image related to the user's gesture (305) is captured.

[0077] The feature extraction module (320) can extract feature information (325) representing the characteristics of a gesture based on data captured through the data capture module (310). For example, the feature information (325) can include embedding data. For example, the embedding data can include 1024 data, but is not limited thereto. The feature extraction module (320) can extract feature information representing the characteristics of a gesture using one or more neural networks using a known algorithm or artificial intelligence. The feature extraction module (320) can provide the feature information (325) to the gesture control module (330).

[0078] The feature extraction module (320) can extract various feature information based on the characteristics of various gestures. For example, the feature information may vary depending on the characteristics of the gesture. For example, if the characteristics of the gestures are the same, the same feature information may be extracted. If the characteristics of the gestures are different, different feature information may be extracted. If the characteristics of the gestures are similar, similar feature information may be extracted. For example, the feature extraction module (320) can cluster gestures having the same or similar features based on embedded data representing the extracted feature information. For example, when the embedded data is projected onto a two-dimensional area, gestures having the same or similar features may be grouped by area. For example, thumb up gestures may have the same or similar embedding data (or feature information), a thumb up gesture may have similar embedding data (or feature information) to an index finger up gesture, and a thumb up gesture may have different embedding data (or feature information) from a palm open gesture. The embedding data may be used to determine the similarity between gestures through the gesture control module (330).

[0079] The gesture control module (330) can receive feature information (325) extracted from the feature extraction module (320). If the user's gesture (305) corresponds to a preset gesture, the gesture control module (330) can extract information about an action mapped to the preset gesture. If the user's gesture (305) does not correspond to a preset gesture, the gesture control module (330) can predict a target action suitable for the user's gesture (305). In the present disclosure, a gesture that does not correspond to a preset gesture among the user's gestures (305) may be referred to as an 'unrecognized gesture', a 'recognition failure gesture', or a 'first gesture'. In addition, a gesture that corresponds to a preset gesture among the user's gestures (305) may be referred to as a 'recognized gesture', a 'recognition success gesture', or a 'second gesture'.

[0080] The gesture control module (330) includes a gesture recognition module (332), a gesture monitoring module (334), an action prediction module (336), and an update module (338).

[0081] The gesture recognition module (332) can obtain feature information (325) about the user's gesture (305) extracted through the feature extraction module (320). The gesture recognition module (332) can determine whether the user's gesture (305) corresponds to a preset gesture. For example, the gesture recognition module (332) can determine whether the user's gesture (305) corresponds to a preset gesture by determining the similarity between the user's gesture (305) and the preset gesture. For example, the gesture recognition module (332) can determine the similarity based on the feature information (325) about the user's gesture (305) and the feature information about the preset gesture stored in the gesture DB (340). For example, the similarity can be calculated using the cosine similarity of Mathematical Formula 1. However, the similarity calculation method is not limited to Mathematical Formula 1.

[0082] [Mathematical Formula 1]

[0083]

[0084] In mathematical expression 1, A may be embedded data representing feature information (325) for a user's gesture (305), and B may be embedded data representing feature information for a preset gesture stored in a gesture DB (340).

[0085] The gesture recognition module (332) may determine that the user's gesture (305) corresponds to a preset gesture based on the determination that the similarity is greater than or equal to a reference value. In this case, the user's gesture (305) corresponding to the preset gesture may be a second gesture. The gesture recognition module (332) may determine that the user's gesture (305) does not correspond to a preset gesture based on the determination that the similarity is less than a reference value. In this case, the user's gesture (305) that does not correspond to a preset gesture may be a first gesture. In one embodiment, the reference value may be a predefined value and may be modified as needed. In one embodiment, the reference value for the gesture recognition module (332) to determine the recognition result is exemplified as 0.75, but is not limited thereto.

[0086] The gesture recognition module (332) can identify a target action mapped to the user's gesture (305) based on the determination that the similarity is greater than or equal to a reference value. For example, the gesture recognition module (332) can identify information about an action mapped to a preset gesture stored in the gesture DB (340) and identify a target action stored in the action DB (350). For example, in FIGS. 4 and 5 , if the first input user's gesture (Index 1) has a similarity (e.g., 0.88) with Gesture_1 that is greater than or equal to a reference value (e.g., 0.75), the electronic device (100) can identify that the action ID of the action mapped to Gesture_1 is Action_3 through the gesture DB (340). The electronic device (100) can identify that the target action corresponding to Action_3 is Channel Up through the action DB (350). The electronic device (100) can perform a Channel Up operation in response to the first input user gesture (Index 1).

[0087] The gesture recognition module (332) can store the result of recognition regarding whether the user's gesture (305) corresponds to a preset gesture in the history DB (360).

[0088] For example, if the user's gesture (305) corresponds to a preset gesture, the gesture recognition module (332) may store data related to the preset gesture in the history DB (360). For example, the data related to the user's gesture corresponding to the preset gesture may be expressed as a result of recognition success. For example, the electronic device (100) may store a recognition success result regarding the second gesture corresponding to the preset gesture in the history DB (360).

[0089] For example, if the user's gesture (305) does not correspond to a preset gesture, the gesture recognition module (332) may store data related to the fact that the user's gesture does not correspond to a preset gesture in the history DB (360). For example, the data related to the fact that the user's gesture does not correspond to a preset gesture may be expressed as a result of recognition failure. For example, the electronic device (100) may store a result of recognition failure regarding the fact that the first gesture does not correspond to a preset gesture in the history DB (360).

[0090] Meanwhile, in the present disclosure, a preset gesture refers to a gesture in which feature information about the gesture is stored in the gesture DB (340) and action information mapped to the gesture is stored. Accordingly, the gesture recognition module (332) can determine that a gesture in which feature information about the gesture is not stored in the gesture DB (340) does not correspond to a preset gesture. In addition, the gesture recognition module (332) can determine that a gesture in which feature information about the gesture is stored in the gesture DB (340) but mapped action information is not stored (e.g., Gesture_4 of the gesture DB (340) in FIG. 4) does not correspond to a preset gesture. Accordingly, the gesture recognition module (332) can store a recognition failure result for a gesture in which feature information about the gesture is stored in the gesture DB (340) but mapped action information is not stored in the history DB (360). Accordingly, the gesture monitoring module (334) can monitor the recognition failure result for a gesture in which feature information about the gesture is stored in the gesture DB (340) but mapped action information is not stored. Accordingly, the action prediction module (336) can predict the target action for a gesture in which feature information about the gesture is stored in the gesture DB (340) but mapped action information is not stored.

[0091] The history DB (360) may include a result of recognition regarding whether the user's gesture (305) corresponds to a preset gesture, and the recognition result may include at least one of a result of recognition failure for the first gesture or a result of recognition success for the second gesture. The history DB (360) may include, as a recognition result, the similarity between the user's gesture (305) and the preset gesture determined by the gesture recognition module (332), information regarding the gesture with the highest similarity to the user's gesture (305) among the preset gestures, and the maximum similarity (Max similarity). For example, when the user's gesture (305) corresponds to a preset gesture, the recognition success result for the second gesture may be stored as Index 1, 2, 3, and 4 of FIG. 5. For example, if the user's gesture does not correspond to a preset gesture, the recognition failure result for the first gesture may be stored as Index 756, 757, 758, and 759 in FIG. 5.

[0092] The gesture monitoring module (334) can monitor the recognition results stored in the history DB (360) through the gesture recognition module (332) in the background. The gesture monitoring module (334) can monitor whether a first gesture that does not correspond to a preset gesture is stored in the history DB (360). If the gesture monitoring module (334) identifies a recognition failure result for the first gesture, it can perform an operation to predict a target action for the first gesture. For example, the gesture monitoring module (334) can provide an operation initiation signal to the action prediction module (336).

[0093] The gesture monitoring module (334) can identify whether the first gesture is stored in the history DB (360) a preset number of times for a preset period of time. If the gesture monitoring module (334) identifies the first gesture stored in the history DB (360) a preset number of times for a preset period of time, the gesture monitoring module (334) can provide an action initiation signal to the action prediction module (336). For example, if four first gestures, for example, Index 756, 757, 758, and 759, are stored in the history DB (360) for 1 second in FIG. 5, the gesture monitoring module (334) can provide an action initiation signal to the action prediction module (336) to predict target actions for the four first gestures. Alternatively, for example, if four first gestures are stored in the history DB (360) for 1 hour in FIG. 5, the gesture monitoring module (334) may not provide an action initiation signal to the action prediction module (336). In this case, if there are four failed recognition results for gestures within an hour, the failures did not occur over a short period of time, so predicting the target action may not be necessary. However, the preset time and number of times are merely examples for convenience of explanation and are not limited to the examples described.

[0094] In one embodiment, the gesture monitoring module (334) may determine the type of recognition failure to predict an action for the first gesture when the first gesture is identified. For example, the gesture monitoring module (334) may determine the type of recognition failure based on at least one of the time the first gesture is maintained, the number of times the first gesture is identified, the presence or absence of a second gesture identified subsequent to the first gesture, feature information of the first gesture, or the similarity between the first gesture and a preset gesture. For example, the time the first gesture is maintained may be obtained through a time stamp recorded in the history DB (360), and the number of times the first gesture is identified may be obtained through the number of indexes recorded in the history DB (360). The presence or absence of a second gesture identified subsequent to the first gesture may be obtained through the history DB (360). Feature information of the first gesture may be obtained through the feature extraction module (320). The similarity between the first gesture and the preset gesture can be obtained through the history DB (360).

[0095] The action prediction module (336) can predict a target action for the first gesture and suggest the target action to the user as the gesture monitoring module (334) identifies the first gesture. The action prediction module (336) can provide the target action for the first gesture based on at least one of the duration for which the first gesture is maintained, the number of times the first gesture is identified, the presence of a second gesture identified subsequent to the first gesture, characteristic information of the first gesture, or the similarity between the first gesture and a preset gesture. For example, the action prediction module (336) can set the target action of the first gesture according to the first type based on the presence of a second gesture identified subsequent to the first gesture. For example, the action prediction module (336) can set the target action of the first gesture according to the second type based on the duration for which the first gesture is maintained or the number of times the first gesture is identified.

[0096] In one embodiment, the action prediction module (336) may identify a second gesture corresponding to a preset gesture as a subsequent gesture of the first gesture according to the first type. Based on the identification of the second gesture subsequent to the first gesture, the action prediction module (336) may set a target action mapped to the second gesture as the target action of the first gesture. In one embodiment, if the action prediction module (336) identifies a subsequent gesture corresponding to a preset gesture according to the first type after identifying the first gesture, it may be determined that the user intends to use the first gesture as the same action as the subsequent gesture. For example, the first type may indicate a case where the time for which the first gesture is maintained before changing to the second gesture is within a preset time, or a case where the number of times the first gesture is identified before changing to the second gesture is within a preset number of times. For example, the first type may indicate that a first gesture changes to a second gesture within 5 seconds, or that the first gesture is identified 5 times per unit time and then changes to a second gesture. Accordingly, the action prediction module (336) may predict that the target action of the first gesture is the same as the target action mapped to the second gesture.

[0097] Alternatively, in one embodiment, the action prediction module (336) may identify a similarity between the first gesture and the second gesture when the second gesture is identified as a subsequent gesture of the first gesture according to the first type. The action prediction module (336) may set a target action mapped to the second gesture as the target action of the first gesture based on the similarity between the first gesture and the second gesture being greater than or equal to a threshold value. The action prediction module (336) may stop the target action prediction operation for the first gesture based on the similarity between the first gesture and the second gesture being less than the threshold value. In one embodiment, the threshold value set by the action prediction module (336) to predict the target action for the first gesture may be lower than, greater than, or equal to a reference value of 0.75.

[0098] In one embodiment, the action prediction module (336) may set a target action for the first gesture based on the similarity between the first gesture and preset gestures, if it is determined that the first gesture continues to be identified according to the second type. For example, the second type may indicate that the first gesture is maintained for a preset period of time or that the first gesture is identified more than a preset number of times. For example, the second type may indicate that the first gesture is maintained for 10 seconds or that the first gesture is identified more than 10 times per unit time.

[0099] In one embodiment, the action prediction module (336) may identify the similarity between the first gesture and the preset gesture if it is determined that the first gesture is identified according to the second type. The action prediction module (336) may set the target action of the first gesture to be the same as the target action mapped to the preset gesture if the similarity between the first gesture and the preset gesture is greater than or equal to a threshold value. For example, the action prediction module (336) may provide a candidate action list indicating information about target actions set by other users for the first gesture if the similarity between the first gesture and the preset gesture is less than the threshold value. In one embodiment, the threshold value for the action prediction module (336) to predict the target action for the first gesture may be exemplified as 0.6. In one embodiment, the threshold value (e.g., 0.6) set by the action prediction module (336) may be less than a reference value (e.g., 0.75) set by the gesture recognition module (332).

[0100] The update module (338) can transmit and receive data to and from the server (200) via the communication unit (130). The update module (338) can update the target action for the first gesture predicted by the first gesture and action prediction module (336) to the server (200). The update module (338) can receive a candidate action list indicating information about target actions set by other users from the server (200). For example, the candidate action list can include one or more target actions for gestures set by other users.

[0101] Fig. 4 is a diagram illustrating a gesture DB and an action DB according to one embodiment. Fig. 5 is a diagram illustrating a history DB according to one embodiment.

[0102] Referring to FIGS. 4 and 5, the electronic device (100) can store a gesture DB (340), an action DB (350), and a history DB (360) in the memory (120).

[0103] The gesture DB (340) may include information about gestures, information about actions mapped to gestures, information about candidate actions set by other users for gestures, and feature information about gestures.

[0104] For example, the gesture DB (340) may include a gesture ID (341), an action ID (342), a candidate ID (343), and embedding data (344). For example, the gesture ID (341) may correspond to information about a gesture in the electronic device (100), the action ID (342) may correspond to information about an action mapped to the gesture, the candidate ID (343) may correspond to information about a candidate action set by another user for the gesture, and the embedding data (344) may correspond to feature information about the gesture.

[0105] For example, gesture ID (341) indicates gesture identifiers (ID), such as Gesture_1, Gesture_2, etc. For example, embedding data (344) indicates 1024 embedding data indicating the characteristics of gestures. For example, gestures stored in gesture DB (340) may include one-hand gestures, such as a thumb up gesture while making a fist, an index finger up gesture while making a fist, a palm open gesture, and a fist clench gesture. Alternatively, for example, gestures stored in gesture DB (340) may include two-hand gestures or may include gestures using body parts other than hands. Additionally, for example, gestures stored in the gesture DB (340) may include motion gestures such as swiping, zoom-in, and zoom-out in addition to static gestures.

[0106] For example, the action ID (342) may display Action_1, Action_2, etc., which are identifiers of actions mapped to the gesture ID (341), or information (Empty) indicating that there is no target action mapped to the gesture ID (341). For example, the gesture DB (340) may store a gesture (e.g., Gesture_1) in which feature information about the gesture is stored and action information mapped to the gesture is stored, and a gesture (e.g., Gesture_4) in which feature information about the gesture is stored but mapped action information is not stored.

[0107] For example, candidate ID (343) indicates the identifier of the candidate action corresponding to gesture ID (341).

[0108] For example, according to the gesture DB (340), Gesture_1 is mapped to Action_3 and can have Action_3, Action_1, and Action_5 as candidate actions.

[0109] The action DB (350) may include information about actions mapped to gestures set in the electronic device (100). For example, the action DB (350) may include an action ID (351) as information about the action and may include a target action (352).

[0110] For example, the action ID (351) displays the identifiers of the action, such as Action_1, Action_2, etc. For example, the target action (352) displays various actions performed on the electronic device (100). For example, the target action (352) may include functions such as Launch Home, Channel Down, Channel Up, Launch Netflix, Mute, etc. In addition, for example, the target action (352) may include a function to perform controlling a character or object in a specific application.

[0111] In one embodiment, the electronic device (100) can determine similarity based on feature information (325) about the user's gesture (305) and feature information about preset gestures stored in the gesture DB (340). The electronic device (100) can identify preset gestures that have a high similarity to the user's gesture (305), identify information about actions mapped to the preset gestures through the gesture DB (340), and identify target actions through the action DB (350).

[0112] The history DB (360) can store a history of user gestures input into the electronic device (100). For example, the history DB (360) can store a result of recognition regarding whether the user's gesture corresponds to a preset gesture. It can include a similarity (361) between the user's gesture input into the electronic device (100) and the preset gesture, information (362) regarding the gesture with the highest similarity to the user's gesture (305) among the preset gestures, and a maximum similarity (Max similarity) (363). For example, the similarity (361) can be calculated based on embedding data representing characteristic information of the user's gesture and embedding data representing characteristic information of the preset gesture.

[0113] For example, the history DB (360) can store the results of recognition success and recognition failure for the user's gesture. For example, if the user's gesture corresponds to a preset gesture, the results of recognition success may be displayed as Index 1, 2, 3, and 4. For example, if the user's gesture does not correspond to a preset gesture, the results of recognition failure may be displayed as Index 756, 757, 758, and 759.

[0114] For example, according to the history DB (360), the first input user's gesture (Index 1) may have a similarity of 0.88 with Gesture_1. In one embodiment, if the gesture recognition module (332) assumes that the standard value of the similarity for determining the recognition result is 0.75, the first input user's gesture (Index 1) corresponds to Gesture_1 and may be identified as having been successfully recognized. In one embodiment, the electronic device (100) may identify that the action ID of the action mapped to Gesture_1 is Action_3 through the gesture DB (340). The electronic device (100) may identify that the target action corresponding to Action_3 is Channel Up through the action DB (350). Therefore, the electronic device (100) may perform the Channel Up operation in response to the first input user's gesture (Index 1).

[0115] For example, according to the history DB (360), since the similarity between the 756th input user gesture (Index 756) and the preset gesture is less than the reference value (ex. 0.75), the 756th input user gesture (Index 756) does not correspond to the preset gesture and may be identified as having failed recognition. For example, if the 756th input user gesture (Index 756) is a gesture of raising the index finger (index finger up), the electronic device (100) may determine that the similarity with Gesture_1 corresponding to the gesture of raising the thumb (thumb up) is 0.31. This is because the embedding data of the gesture of raising the index finger (index finger up) is different from the embedding data of the gesture of raising the thumb (thumb up). An electronic device (100) according to one embodiment can predict a target action suitable for the 756th input user gesture (Index 756).

[0116] Fig. 6 is a flowchart illustrating an operation method for an electronic device to provide a gesture recognition service according to one embodiment. Fig. 6 is described in conjunction with Figs. 2 to 5.

[0117] Referring to FIG. 6, in operation 610, an electronic device (100) according to one embodiment can obtain a user's gesture.

[0118] For example, the electronic device (100) can obtain the user's gesture (305) through the sensor (140). For example, the electronic device (100) can obtain the user's gesture (305) through the camera. The electronic device (100) can capture an image related to the user's gesture (305) on a frame-by-frame basis. The electronic device (100) can store data in which the image related to the user's gesture (305) is captured.

[0119] In operation 620, the electronic device (100) according to one embodiment can identify whether the user's gesture corresponds to a preset gesture.

[0120] For example, the electronic device (100) may acquire feature information (325) representing the characteristics of the user's gesture (305) based on acquiring the user's gesture (305). For example, the feature information (325) may include embedding data. For example, if the features of the gestures are the same, the same feature information may be extracted, if the features of the gestures are different, different feature information may be extracted, and if the features of the gestures are similar, similar feature information may be extracted.

[0121] For example, the electronic device (100) can determine the similarity between the characteristic information (325) of the user's gesture (305) and the characteristic information of the preset gesture. By determining the similarity between the user's gesture (305) and the preset gesture, the electronic device (100) can determine whether the user's gesture (305) corresponds to the preset gesture.

[0122] For example, if the similarity is greater than a threshold value, the electronic device (100) may identify the user's gesture (305) as corresponding to a preset gesture. In the present disclosure, a gesture that does not correspond to a preset gesture among the user's gestures (305) may be referred to as an 'unrecognized gesture', a 'recognition failure gesture', or a 'first gesture'.

[0123] For example, if the similarity is less than a threshold value, the electronic device (100) may identify that the user's gesture (305) does not correspond to a preset gesture. In the present disclosure, a gesture corresponding to a preset gesture among the user's gestures (305) may be referred to as a 'recognized gesture', a 'recognition success gesture', or a 'second gesture'. In one embodiment, the threshold value may be a predefined reference value and may be modified as needed.

[0124] Meanwhile, in one embodiment, the entity that identifies whether the user's gesture corresponds to a preset gesture is exemplified as the electronic device (100), but is not limited thereto. For example, the electronic device (100) may receive information regarding whether the user's gesture corresponds to a preset gesture from an external device, such as a server, via the communication unit (130). This is described in detail in FIG. 17b.

[0125] In operation 630, the electronic device (100) according to one embodiment may identify a target action mapped to the preset gesture when the user's gesture is identified as corresponding to the preset gesture.

[0126] For example, if the electronic device (100) determines that the user's gesture (305) corresponds to a preset gesture because the similarity is greater than a threshold value, the electronic device (100) can identify a target action mapped to the user's gesture (305). The electronic device (100) can identify information about an action mapped to a preset gesture stored in the gesture DB (340) and can identify a target action stored in the action DB (350). For example, in FIGS. 4 and 5 , if the first input user's gesture (Index 1) has a similarity (e.g., 0.88) greater than or equal to a reference value (e.g., 0.75) with Gesture_1, the electronic device (100) can identify that the action ID of the action mapped to Gesture_1 is Action_3 through the gesture DB (340). The electronic device (100) can identify that the target action corresponding to Action_3 is Channel Up through the action DB (350). The electronic device (100) may perform a Channel Up operation in response to the first input user gesture (Index 1). Alternatively, the electronic device (100) may transmit information regarding the Channel Up operation to an external device in response to the first input user gesture (Index 1).

[0127] By providing a gesture recognition service according to one embodiment of the present invention, an electronic device (100) can control the electronic device (100) without an input device. Accordingly, user convenience can be enhanced.

[0128] Meanwhile, according to one embodiment, the electronic device (100) may receive information regarding a target action mapped to a user's gesture from a server via the communication unit (130). For example, the electronic device (100) may receive information regarding whether the user's gesture corresponds to a preset gesture and information regarding a target action mapped to the preset gesture from the server. This is described in detail in FIG. 17b.

[0129] In operation 640, the electronic device (100) according to one embodiment may store the result of recognition success for the user's gesture in a database. In one embodiment, the database may correspond to a history DB (360).

[0130] For example, if the user's gesture (305) corresponds to a preset gesture and is successfully recognized, the electronic device (100) may store the recognition success result for the user's gesture in the history DB (360). For example, the electronic device (100) may store the recognition success result for the user's gesture as in Index 1, 2, 3, and 4 of FIG. 5.

[0131] In operation 650, the electronic device (100) according to one embodiment may store a result of recognition failure for the user's gesture in a database based on determining that the user's gesture does not correspond to a preset gesture.

[0132] For example, if the electronic device (100) fails to recognize the user's gesture (305) because it does not correspond to a preset gesture, the electronic device (100) may store the result of the recognition failure for the user's gesture in the history DB (360). For example, the electronic device (100) may store the result of the recognition failure for the user's gesture as in Index 756, 757, 758, and 759 of FIG. 5.

[0133] For example, the electronic device (100) can identify whether there is a recognition failure result for the user's gesture, the type of recognition failure, etc., by monitoring the recognition results stored in the history DB (360).

[0134] FIG. 7A is a flowchart illustrating an operation performed by an electronic device according to one embodiment when the electronic device identifies a gesture that has failed to be recognized according to the first type.

[0135] Referring to FIG. 7A, in operation 710, if the first gesture does not correspond to a preset gesture, the electronic device (100) according to one embodiment may store data related to the first gesture not corresponding to the preset gesture in a database. In one embodiment, the database may correspond to a history DB (360). For example, data related to the user's gesture not corresponding to the preset gesture may be expressed as a result of recognition failure.

[0136] In the present disclosure, a user's gesture that does not correspond to a preset gesture may be referred to as an "unrecognized gesture," a "recognition failure gesture," or a "first gesture." For example, the electronic device (100) may store a recognition failure result regarding the first gesture not corresponding to a preset gesture in the history DB (360). For example, the first gesture may be acquired through a sensor (140), such as a camera.

[0137] For example, the electronic device (100) can determine the similarity between the characteristic information of the user's gesture and the characteristic information of the preset gesture. By determining the similarity between the user's gesture and the preset gesture, the electronic device (100) can determine whether the user's gesture (305) corresponds to the preset gesture.

[0138] For example, the electronic device (100) may determine that the user's gesture (305) does not correspond to a preset gesture based on the determination that the similarity is less than a threshold value. For example, if the recognition of the user's gesture fails because it does not correspond to a preset gesture, the electronic device (100) may store the recognition failure result for the user's gesture in the history DB (360). For example, the electronic device (100) may store the recognition failure result for the user's gesture as in Index 756, 757, 758, and 759 of FIG. 5. For example, the history DB (360) may include a recognition result regarding whether the user's gesture corresponds to a preset gesture. For example, the history DB (360) may store data related to the user's gesture not corresponding to a preset gesture. For example, the history DB (360) may store the recognition failure result for the first gesture. Additionally, for example, the history DB (360) may include at least one of the following as a recognition result: the similarity between the user's gesture and a preset gesture, information about the gesture with the highest similarity to the user's gesture among the preset gestures, or the maximum similarity.

[0139] In one embodiment, the electronic device (100) can monitor whether there are recognition failure results, the type of recognition failure, etc. by storing the recognition results in the history DB (360). For example, the electronic device (100) can monitor the recognition results stored in the history DB (360) in the background. For example, the electronic device (100) can periodically monitor whether a first gesture that does not correspond to a gesture preset in the history DB (360) is stored in the history DB (360).

[0140] For example, when the electronic device (100) identifies that the first gesture is stored in the history DB (360), the electronic device (100) may determine the type of recognition failure to predict an action for the first gesture. For example, the electronic device (100) may determine the type of recognition failure based on at least one of the time that the first gesture is maintained, the number of times the first gesture is identified, the presence or absence of a second gesture identified subsequent to the first gesture, characteristic information of the first gesture, or the similarity between the first gesture and a preset gesture. For example, the electronic device (100) may set the target action of the first gesture according to the first type based on the presence or absence of a second gesture identified subsequent to the first gesture. In one embodiment, the electronic device (100) may identify whether the first gesture is stored in the history DB (360) a preset number of times for a preset time.

[0141] In operation 720, the electronic device (100) according to one embodiment may set a target action mapped to the second gesture as the target action of the first gesture if the second gesture acquired after the first gesture corresponds to a preset gesture. For example, the second gesture may be acquired through a sensor (140), such as a camera. The second gesture may be a subsequent gesture of the first gesture.

[0142] In the present disclosure, a gesture corresponding to a preset gesture among the user's gestures may be referred to as a 'recognized gesture', a 'recognition success gesture', or a 'second gesture'.

[0143] In one embodiment, the electronic device (100) may identify a second gesture corresponding to a preset gesture as a subsequent gesture to the first gesture. For example, the electronic device (100) may determine that the user's gesture corresponds to a preset gesture based on a similarity determined to be above a threshold value.

[0144] In one embodiment, the electronic device (100) may, based on identifying a second gesture subsequent to a first gesture, set a target action mapped to the second gesture as the target action of the first gesture. In one embodiment, if the electronic device (100) identifies a subsequent gesture corresponding to a gesture preset according to a first type after identifying the first gesture, the electronic device (100) may determine that the user intends to use the first gesture as the same action as the subsequent gesture. Accordingly, the electronic device (100) may predict that the target action of the first gesture is the same as the target action mapped to the second gesture.

[0145] Alternatively, in one embodiment, when the electronic device (100) identifies a second gesture as a subsequent gesture of the first gesture according to the first type, the electronic device (100) may identify the similarity between the first gesture and the second gesture. Based on the similarity between the first gesture and the second gesture being greater than or equal to a threshold value, the electronic device (100) may set the target action mapped to the second gesture as the target action of the first gesture. Based on the similarity between the first gesture and the second gesture being less than the threshold value, the electronic device (100) may stop the target action prediction operation for the first gesture. In one embodiment, the threshold value set by the electronic device (100) to predict the target action for the first gesture may be lower than or equal to the reference value of 0.75. According to one embodiment, if the electronic device (100) determines that the first gesture and the second gesture are completely unrelated, taking into account the recognition failure result, the electronic device (100) may stop the target action prediction operation. Accordingly, the electronic device (100) may efficiently provide a gesture recognition service.

[0146] In one embodiment, the electronic device (100) may store data related to whether a user's gesture corresponds to a preset gesture in the history DB (360). For example, data related to whether a user's gesture corresponds to a preset gesture may be expressed as a result of recognition success. For example, the electronic device (100) may store a recognition success result regarding whether a second gesture corresponds to a preset gesture in the history DB (360). For example, the electronic device (100) may store recognition success results for the user's gesture as in Indexes 1, 2, 3, and 4 of FIG. 5.

[0147] According to one embodiment, the electronic device (100) can propose a target action appropriate for the user's gesture based on the user's gesture and subsequent gestures, even if the user's gesture does not correspond to a preset gesture. Accordingly, user convenience for the electronic device (100) can be enhanced.

[0148] FIG. 7B is a flowchart illustrating an operation performed by an electronic device according to one embodiment when it identifies a gesture that has failed to be recognized according to the first type. FIG. 7B may further include a monitoring step compared to FIG. 7A.

[0149] Referring to FIG. 7B, in operation 730, an electronic device (100) according to one embodiment may store a recognition result regarding whether a user's gesture corresponds to a preset gesture in a database. In one embodiment, the database may correspond to a history DB (360).

[0150] For example, the electronic device (100) can determine the similarity between the characteristic information of the user's gesture and the characteristic information of the preset gesture. By determining the similarity between the user's gesture and the preset gesture, the electronic device (100) can determine whether the user's gesture (305) corresponds to the preset gesture.

[0151] For example, the electronic device (100) may determine that the user's gesture corresponds to a preset gesture based on the determination that the similarity is greater than or equal to a threshold value. For example, the electronic device (100) may store data related to the user's gesture corresponding to the preset gesture in the history DB (360). For example, the data related to the user's gesture corresponding to the preset gesture may be, for example, the electronic device (100) may store a recognition failure result regarding the first gesture not corresponding to the preset gesture in the history DB (360). For example, the electronic device (100) may determine that the user's gesture (305) does not correspond to a preset gesture based on the determination that the similarity is less than a threshold value. For example, the electronic device (100) may store data related to the user's gesture not corresponding to the preset gesture in the history DB (360). For example, the electronic device (100) may store a recognition failure result regarding the first gesture not corresponding to the preset gesture in the history DB (360). For example, the history DB (360) may include a recognition result regarding whether the user's gesture corresponds to a preset gesture, and the recognition result may include at least one of a recognition failure result or a recognition success result. The history DB (360) may include, as a recognition result, at least one of the similarity between the user's gesture and the preset gesture, information regarding the gesture with the highest similarity to the user's gesture among the preset gestures, or the maximum similarity.

[0152] For example, the electronic device (100) can monitor whether there is a recognition failure result, the type of recognition failure, etc. by storing the recognition result in the history DB (360).

[0153] In operation 740, the electronic device (100) according to one embodiment can detect whether a first gesture that does not correspond to a preset gesture is stored in the database. The electronic device (100) can monitor the recognition results stored in the history DB (360) in the background. The electronic device (100) can periodically monitor whether a first gesture that does not correspond to a preset gesture (an unrecognized gesture, or a gesture that failed to be recognized) is stored in the history DB (360).

[0154] In operation 750, an electronic device (100) according to one embodiment may identify a first gesture. The electronic device (100) may identify that a first gesture (unrecognized gesture, or recognition failure gesture) that does not correspond to a preset gesture is stored in the history DB (360).

[0155] For example, when the electronic device (100) identifies that the first gesture is stored in the history DB (360), the electronic device (100) may determine the type of recognition failure to predict an action for the first gesture. For example, the electronic device (100) may determine the type of recognition failure based on at least one of the time the first gesture is maintained, the number of times the first gesture is identified, the presence or absence of a second gesture identified subsequent to the first gesture, characteristic information of the first gesture, or the similarity between the first gesture and a preset gesture. For example, the electronic device (100) may set the target action of the first gesture according to the first type based on the presence or absence of a second gesture identified subsequent to the first gesture.

[0156] In one embodiment, the electronic device (100) can identify whether the first gesture is stored in the history DB (360) a preset number of times for a preset period of time. For example, if four first gestures, for example, Index 756, 757, 758, and 759, are stored in the history DB (360) for 1 second in FIG. 5 , the electronic device (100) can perform an operation to predict the target action for the first gesture. For example, if four first gestures are stored in the history DB (360) for 1 hour in FIG. 5 , the electronic device (100) may not perform an operation to predict the target action for the first gesture. For example, if there are recognition failure results for four gestures in 1 hour, the operation to predict the target action may be unnecessary because the recognition failure did not occur for a short period of time. Therefore, the electronic device (100) can selectively predict the target action for the first gesture according to the type of recognition failure. However, the preset time and preset number of times are only examples for the convenience of explanation and are not limited to the examples described.

[0157] In operation 760, the electronic device (100) according to one embodiment may set a target action mapped to the second gesture as the target action of the first gesture if the second gesture obtained after the first gesture corresponds to a preset gesture.

[0158] In one embodiment, the electronic device (100) may identify a second gesture corresponding to a preset gesture as a subsequent gesture of the first gesture according to the first type. Based on the identification of the second gesture subsequent to the first gesture, the electronic device (100) may set a target action mapped to the second gesture as the target action of the first gesture. In one embodiment, if the electronic device (100) identifies a subsequent gesture corresponding to a preset gesture according to the first type after identifying the first gesture, the electronic device (100) may determine that the user intends to use the first gesture as the same action as the subsequent gesture.

[0159] For example, the first type may indicate a case where the first gesture is maintained for a preset period of time before changing to a second gesture, or where the first gesture is identified for a preset number of times before changing to a second gesture. For example, the first type may indicate a case where the first gesture changes to the second gesture within 5 seconds, or where the first gesture is identified 5 times per unit time and then changes to the second gesture. Accordingly, the electronic device (100) may predict that the target action of the first gesture is the same as the target action mapped to the second gesture.

[0160] Alternatively, in one embodiment, when the electronic device (100) identifies a second gesture as a subsequent gesture of the first gesture according to the first type, the electronic device (100) may identify the similarity between the first gesture and the second gesture. Based on the similarity between the first gesture and the second gesture being greater than or equal to a threshold value, the electronic device (100) may set the target action mapped to the second gesture as the target action of the first gesture. Based on the similarity between the first gesture and the second gesture being less than the threshold value, the electronic device (100) may stop the target action prediction operation for the first gesture. In one embodiment, the threshold value set by the electronic device (100) to predict the target action for the first gesture may be lower than or equal to the reference value of 0.75. According to one embodiment, if the electronic device (100) determines that the first gesture and the second gesture are completely unrelated, taking into account the recognition failure result, the electronic device (100) may stop the target action prediction operation. Accordingly, the electronic device (100) may efficiently provide a gesture recognition service.

[0161] According to one embodiment, the electronic device (100) can propose a target action appropriate for the user's gesture based on the user's gesture and subsequent gestures, even if the user's gesture does not correspond to a preset gesture. Accordingly, user convenience for the electronic device (100) can be enhanced.

[0162] The operation of Fig. 7 is described in more detail with reference to Figs. 8 and 9.

[0163] FIG. 8 is an example of an operation performed by an electronic device according to one embodiment when it identifies a gesture that has failed to be recognized according to the first type. FIG. 9 is a diagram illustrating a history DB, a gesture DB, and an action DB in the first type exemplified in FIG. 8.

[0164] Referring to FIGS. 8 and 9, in the gesture DB (340) and action DB (350) of the electronic device (100) according to one embodiment, it is assumed that the gesture of raising the thumb (Gesture_1) is mapped to the action of raising the channel (Channel up), and the gesture of raising the index finger is not stored in the gesture DB (340).

[0165] For example, referring to FIG. 8, a user may attempt a gesture of raising an index finger (820) to raise a channel of content, and then attempt a gesture of raising a thumb (830) as a subsequent gesture. In this case, the electronic device (100) may identify that the gesture of raising an index finger (820) does not correspond to a preset gesture. The electronic device (100) may set a channel raising action (Channel Up) mapped to the thumb raising gesture (830) as a target action of the gesture of raising an index finger (820). For example, the electronic device (100) may provide an interface (810) such as “A new gesture has been detected. Would you like to set the new gesture as the action “Channel UP”?” Based on the user’s response, the electronic device (100) may set the target action of the gesture of raising an index finger (820) as the action of raising a channel (Channel Up).

[0166] The electronic device (100) can perform the above-described prediction operation by monitoring the history DB (360). For example, referring to FIG. 9, the electronic device (100) can monitor four gestures indicating a recognition failure result (910), for example, Index 756, 757, 758, and 756, in the history DB (360). If the four gestures indicating the recognition failure result (910) are identified for 1 second, the electronic device (100) can predict the target action for the four gestures indicating the recognition failure result (910). For example, the four gestures indicating the recognition failure result (910) can be a gesture (820) of raising an index finger.

[0167] For example, after identifying four gestures indicating a recognition failure result (910), the electronic device (100) can identify a subsequent gesture indicating a recognition success result (920). For example, the subsequent gesture indicating a recognition success result (910) may be a thumb-up gesture (830). In this case, the electronic device (100) can set the target action (Channel Up) mapped to the subsequent gesture indicating a recognition success result (920) as the target action of the four gestures indicating a recognition failure result (910). Accordingly, the electronic device (100) can set the target action of the index finger-up gesture (820) to the channel-up action (Channel Up).

[0168] FIG. 10 is a flowchart illustrating an operation performed when an electronic device according to one embodiment identifies a gesture that has failed to be recognized according to a second type.

[0169] Referring to FIG. 10, in operation 1010, an electronic device (100) according to one embodiment may store a recognition result regarding whether a user's gesture corresponds to a preset gesture in a database. Operation 1010 may correspond to operation 730 of FIG. 7b.

[0170] For example, the electronic device (100) can determine the similarity between the characteristic information of the user's gesture and the characteristic information of the preset gesture. By determining the similarity between the user's gesture and the preset gesture, the electronic device (100) can determine whether the user's gesture (305) corresponds to the preset gesture. For example, the electronic device (100) can determine that the user's gesture corresponds to the preset gesture based on the determination that the similarity is greater than or equal to a threshold value. For example, the electronic device (100) can store data related to whether the user's gesture corresponds to the preset gesture in the history DB (360). For example, the electronic device (100) can determine that the user's gesture (305) does not correspond to the preset gesture based on the determination that the similarity is less than the threshold value. For example, if the user's gesture does not correspond to the preset gesture, the electronic device (100) can store data related to whether the user's gesture does not correspond to the preset gesture in the history DB (360).

[0171] In operation 1020, the electronic device (100) according to one embodiment may detect whether a first gesture that does not correspond to a preset gesture is stored in the database. Operation 1020 may correspond to operation 740 of FIG. 7B.

[0172] For example, the electronic device (100) can monitor whether there is a recognition failure result, the type of recognition failure, etc. by storing the recognition result in the history DB (360).

[0173] For example, the electronic device (100) can monitor whether a first gesture that does not correspond to a preset gesture is stored in the history DB. The electronic device (100) can monitor the recognition result stored in the history DB (360) in the background. The electronic device (100) can periodically monitor whether a first gesture that does not correspond to a preset gesture (an unrecognized gesture, or a gesture that fails to be recognized) is stored in the history DB (360).

[0174] In operation 1030, the electronic device (100) according to one embodiment may identify a first gesture. The electronic device (100) may identify that a first gesture that does not correspond to a preset gesture (an unrecognized gesture, or a gesture that fails to be recognized) is stored in the history DB (360). Operation 1030 may correspond to operation 750 of FIG. 7B.

[0175] When the electronic device (100) identifies a first gesture, it can determine the type of recognition failure to predict an action for the first gesture. For example, the electronic device (100) can determine the type of recognition failure based on at least one of the following: the time for which the first gesture is maintained, the number of times the first gesture is identified, the presence of a second gesture identified subsequent to the first gesture, characteristic information of the first gesture, or the similarity between the first gesture and a preset gesture. For example, the electronic device (100) can set the target action of the first gesture according to the second type based on the time for which the first gesture is maintained or the number of times the first gesture is identified.

[0176] In operation 1040, the electronic device (100) according to one embodiment may identify a similarity between the first gesture and the preset gesture based on the time the first gesture is maintained or the number of times the first gesture is identified.

[0177] In one embodiment, if the electronic device (100) determines that the first gesture continues to be identified according to the second type, the electronic device may set the target action of the first gesture based on the similarity between the first gesture and preset gestures. For example, the second type may indicate that the first gesture is maintained for a preset period of time or that the first gesture is identified more than a preset number of times. For example, the second type may indicate that the first gesture is maintained for 10 seconds or that the first gesture is identified more than 10 times per unit time.

[0178] In one embodiment, the electronic device (100) may identify a similarity between the first gesture and the preset gesture if it is determined that the first gesture continues to be identified according to the second type.

[0179] In operation 1050, the electronic device (100) according to one embodiment may set a target action mapped to the preset gesture as the target action of the first gesture if the similarity between the first gesture and the preset gesture is greater than or equal to a threshold value. Operation 1050 is described in more detail with reference to FIGS. 11, 12, and 13.

[0180] For example, if the similarity between the first gesture and the preset gesture is greater than or equal to a threshold value, the electronic device (100) may set the target action of the first gesture to be the same as the target action mapped to the preset gesture. In one embodiment, the threshold value for predicting the target action for the first gesture by the electronic device (100) may be exemplified as 0.6, but is not limited thereto. The threshold value for predicting the target action for the first gesture by the electronic device (100) may be less than or equal to the reference value of 0.75.

[0181] The electronic device (100) can identify a target action mapped to a preset gesture based on information stored in the gesture DB (340) and the action DB (350). The electronic device (100) can set the target action of the first gesture to be the same as the target action mapped to the preset gesture.

[0182] Alternatively, in one embodiment, the electronic device (100) may identify the top n gestures with a high degree of similarity between the first gesture and the preset gesture, and provide a list of target actions mapped to the top n gestures (where n is a natural number). The electronic device (100) may set the target action for the first gesture based on the user's response.

[0183] In operation 1060, if the similarity between the first gesture and the preset gesture is less than a threshold value, the electronic device (100) according to one embodiment may provide a candidate action list indicating information about a target action set by another user for the first gesture as the target action of the first gesture. Operation 1060 will be described in more detail with reference to FIGS. 14 and 15.

[0184] In one embodiment, the electronic device (100) may periodically receive a list of candidate actions from the server (200). The electronic device (100) may store the list of candidate actions in a gesture DB (340).

[0185] In one embodiment, the electronic device (100) may identify a candidate action list for a first gesture stored in the gesture DB (340). In one embodiment, if the electronic device (100) determines that a candidate action list for the first gesture exists in the gesture DB (340), the electronic device (100) may provide the candidate action list for the first gesture as a target action for the first gesture. In one embodiment, the first gesture may be a gesture for which feature information about the gesture is stored in the gesture DB (340), but for which a mapped target action does not exist (Action ID: Empty). The gesture DB (340) may store the candidate action list for the first gesture. For example, the first gesture may be Gestrue_4 of the gesture DB (340).

[0186] Alternatively, in one embodiment, if the electronic device (100) determines that there is no candidate action list for the first gesture in the gesture DB (340), it may request the server (200) for a candidate action list for the first gesture. The electronic device (100) may provide the candidate action list for the first gesture received from the server (200) as a target action for the first gesture.

[0187] Alternatively, in one embodiment, if the electronic device (100) determines that there is no candidate action list for the first gesture in the gesture DB (340), it may provide an interface that inquires whether to set a target action for the first gesture. Based on the user response, the electronic device (100) may automatically enter a screen for setting a target action for a new gesture.

[0188] According to one embodiment, an electronic device (100) can propose a target action appropriate for a user's gesture based on the similarity between the user's gesture and the preset gesture, even if the user's gesture does not correspond to a preset gesture. Accordingly, user convenience of the electronic device (100) can be enhanced.

[0189] FIG. 11 is an example of an operation performed by an electronic device according to an embodiment when the electronic device identifies a gesture that has failed to be recognized according to the second type. FIG. 12 is an example of an operation performed by an electronic device according to an embodiment when the electronic device identifies a gesture that has failed to be recognized according to the second type. FIG. 13 is a diagram illustrating a history DB, a gesture DB, and an action DB in the second type exemplified in FIGS. 11 and 12.

[0190] Referring to FIGS. 11 to 13, in the gesture DB (340) and the action DB (350) of the electronic device (100) according to one embodiment, it is assumed that a gesture of raising a thumb (Gesture_1) is mapped to an action of raising a channel (Channel up), a gesture of making a fist (Gesture_3) is mapped to a mute action (Mute), and a gesture of raising an index finger is not stored in the gesture DB (340).

[0191] For example, referring to FIGS. 11 and 13, a user may continuously attempt to raise an index finger gesture (1120) for a predetermined period of time to raise a channel of content. In this case, the electronic device (100) may identify the similarity between the index finger raising gesture (1120) and preset gestures, and may suggest a target action for the index finger raising gesture (1120) based on the similarity. For example, the electronic device (100) may determine that the similarity (0.62) between the index finger raising gesture (1120) (Index 757) and the thumb raising gesture (Gesture_1) is greater than a threshold value. In this case, the electronic device (100) may set an action (Channel Up) for raising a channel mapped to the thumb raising gesture (Gesture_1) as the target action of the index finger raising gesture (1120). For example, the electronic device (100) may provide an interface (1110) such as, “A new gesture has been detected. Would you like to set the new gesture as the “Channel UP” action?” Based on the user’s response, the electronic device (100) may set the target action of the index finger raising gesture (1120) as the channel raising action (Channel Up).

[0192] The electronic device (100) may perform the above-described prediction operation by monitoring the history DB (360). For example, referring to FIG. 13, if the history DB (360) identifies that Index 757 indicating a recognition failure result (1310) has been maintained for 10 seconds, the electronic device (100) may predict a target action for a gesture indicating a recognition failure result (1310). For example, the gesture indicating a recognition failure result (1310) may be a gesture (1120) of raising an index finger.

[0193] For example, the electronic device (100) may identify the similarity between a gesture indicating a recognition failure result (1310) and a preset gesture, and if the similarity is greater than a threshold value (e.g., 0.6), the electronic device (100) may set the target action (Channel Up) mapped to the preset gesture as the target action of the gesture indicating the recognition failure result (1310). Accordingly, the electronic device (100) may set the target action of the gesture (1120) of raising the index finger as the action of raising the channel (Channel Up).

[0194] Also, referring to FIGS. 12 and 13 , in one embodiment, the electronic device (100) can identify the similarity between a gesture indicating a recognition failure result (1310) and a preset gesture, identify the top n gestures with high similarity, and provide a list of target actions mapped to the top n gestures (where n is a natural number). For example, the electronic device (100) can identify Gesture_1 and Gesture_3, which are the top two gestures with high similarity to the gesture indicating a recognition failure result (1310). The electronic device (100) can suggest the target action (Channel Up) mapped to Gesture_1 and the target action (Mute) mapped to Gesture_3 as the target action of the gesture (1220) of raising the index finger. For example, the electronic device (100) can provide an interface (1210) such as “A new gesture has been detected. What action would you like to set the new gesture to? Channel Up / Mute.” The electronic device (100) can set a target action for a gesture (1220) of raising an index finger based on the user's response.

[0195] FIG. 14 is an example of an operation performed by an electronic device according to one embodiment when it identifies a gesture that has failed to be recognized according to the second type. FIG. 15 is a diagram illustrating a history DB, a gesture DB, and an action DB in the second type exemplified in FIG. 14.

[0196] Referring to FIGS. 14 and 15, in the gesture DB (340) of the electronic device (100) according to one embodiment, it is assumed that the V gesture (Gesture_4) has feature information stored about the gesture, but no mapped target action exists. In the gesture DB (340), the V gesture (Gesture_4) may have a candidate action list (Action_1, Action_2, Action_3) representing target actions set by other users for the V gesture (Gesture_4).

[0197] A user may continuously attempt a V gesture (1420) that does not correspond to a preset gesture for a preset period of time. For example, if the recognition result for the V gesture (1420) is stored in Index 760 of the history DB (360), the electronic device (100) may identify that Index 760, which indicates a recognition failure result (1510), has been maintained for 10 seconds. The electronic device (100) may predict a target action for the V gesture (1420) that indicates a recognition failure result (1510).

[0198] The electronic device (100) may propose a target action for the V gesture (1420) based on the similarity between the V gesture (1420) and preset gestures. The electronic device (100) may determine that the similarity between the V gesture (1420) and the preset gestures is less than a threshold value. For example, in Index 760, the similarity between the V gesture (1420) and the V gesture (Gesture_4) is greater than the threshold value, but since there is no target action mapped to the V gesture (Gesture_4), the V gesture (Gesture_4) does not correspond to a preset gesture. Therefore, the electronic device (100) may determine that the similarity between the V gesture (1420) and the preset gestures is less than the threshold value.

[0199] If the electronic device (100) determines that the similarity of the V gesture (1420) with a preset gesture is less than a threshold value and that a candidate action list exists, the electronic device (100) may provide a candidate action list (Action_1, Action_2, Action_3) for the V gesture (Gesture_4) as a target action for the V gesture (1420). For example, the electronic device (100) may provide an action for entering home (Launch Home), an action for raising a channel (Channel Up), and an action for lowering a channel (Channel Down) as a candidate action list for the V gesture (Gesture_4) as the target action of the V gesture (1420). For example, the electronic device (100) may provide an interface (1410) such as “A new gesture has been detected. Which action would you like to set the new gesture to? Channel Up / Channel Down / Launch Home.” The electronic device (100) can set the target action of the V gesture (1420) based on the user's response.

[0200] FIG. 16 is a flowchart illustrating an operation performed by an electronic device according to one embodiment when the electronic device identifies a gesture that has failed to be recognized according to the first type or the second type.

[0201] Referring to FIG. 16, in operation 1610, an electronic device (100) according to one embodiment may detect whether a first gesture that does not correspond to a preset gesture is stored in a database. Operation 1610 may correspond to operation 730 of FIG. 7B or operation 1010 of FIG. 10.

[0202] In operation 1620, the electronic device (100) according to one embodiment may identify a first gesture. Operation 1620 may correspond to operation 740 of FIG. 7B or operation 1020 of FIG. 10.

[0203] When the electronic device (100) identifies a first gesture, it can determine the type of recognition failure to predict an action for the first gesture. For example, the electronic device (100) can determine the type of recognition failure based on at least one of the duration for which the first gesture is maintained, the number of times the first gesture is identified, the presence or absence of a second gesture identified subsequent to the first gesture, characteristic information of the first gesture, or the similarity between the first gesture and a preset gesture. For example, the electronic device (100) can set the target action of the first gesture according to the first type based on the presence or absence of a second gesture identified subsequent to the first gesture. For example, the electronic device (100) can set the target action of the first gesture according to the second type based on the duration for which the first gesture is maintained or the number of times the first gesture is identified.

[0204] In one embodiment, the conditions for corresponding to the first type may be the same as or different from the conditions for corresponding to the second type. For example, the time for which the first gesture is maintained according to the first type may be the same as or shorter than the time for which the first gesture is maintained according to the second type. Furthermore, for example, the number of times the first gesture is identified according to the first type may be the same as or fewer than the number of times the second gesture is identified according to the second type.

[0205] In operation 1630, the electronic device (100) according to one embodiment can identify a second gesture corresponding to a preset gesture according to the first type.

[0206] The electronic device (100) may identify a second gesture corresponding to a preset gesture as a subsequent gesture to the first gesture according to the first type. For example, the first type may indicate a case where the time for which the first gesture is maintained before changing to the second gesture is within a preset time, or a case where the first gesture is identified within a preset number of times before changing to the second gesture. For example, the first type may indicate a case where the first gesture changes to the second gesture within 5 seconds, or a case where the first gesture is identified 5 times per unit time and then changes to the second gesture.

[0207] In operation 1640, the electronic device (100) according to one embodiment may set the target action mapped to the second gesture as the target action of the first gesture.

[0208] The electronic device (100) may set the target action mapped to the second gesture as the target action of the first gesture based on the identification of the second gesture following the first gesture according to the first type. Accordingly, the electronic device (100) may predict that the target action of the first gesture is the same as the target action mapped to the second gesture.

[0209] Alternatively, in one embodiment, when the electronic device (100) identifies a second gesture as a subsequent gesture of the first gesture according to the first type, the electronic device (100) may identify the similarity between the first gesture and the second gesture. Based on the similarity between the first gesture and the second gesture being greater than or equal to a threshold value, the electronic device (100) may set the target action mapped to the second gesture as the target action of the first gesture. Based on the similarity between the first gesture and the second gesture being less than the threshold value, the electronic device (100) may stop the target action prediction operation for the first gesture. In one embodiment, the threshold value set by the electronic device (100) to predict the target action for the first gesture may be lower than, greater than, or equal to the reference value of 0.75.

[0210] In operation 1650, the electronic device (100) according to one embodiment may identify the first gesture for a preset time or a preset number of times or more according to the second type.

[0211] If the electronic device (100) determines that the first gesture continues to be identified according to the second type, the electronic device may set a target action for the first gesture based on the similarity between the first gesture and preset gestures. For example, the second type may indicate a case where the first gesture is maintained for a preset period of time or the first gesture is identified more than a preset number of times. For example, the second type may indicate a case where the first gesture is maintained for 10 seconds or the first gesture is identified more than 10 times per unit time.

[0212] In operation 1660, the electronic device (100) according to one embodiment may identify the similarity between the first gesture and the preset gesture. If the electronic device (100) determines that the first gesture is still identified according to the second type, the electronic device (100) may identify the similarity between the first gesture and the preset gesture.

[0213] In operation 1670, the electronic device (100) according to one embodiment may set a target action mapped to the preset gesture as the target action of the first gesture if the similarity between the first gesture and the preset gesture is greater than or equal to a threshold value.

[0214] For example, if the similarity between the first gesture and the preset gesture is greater than or equal to a threshold value, the electronic device (100) may set the target action of the first gesture to be the same as the target action mapped to the preset gesture. In one embodiment, the threshold value for predicting the target action for the first gesture by the electronic device (100) may be exemplified as 0.6, but is not limited thereto. The threshold value for predicting the target action for the first gesture by the electronic device (100) may be less than or equal to the reference value of 0.75.

[0215] In operation 1680, the electronic device (100) according to one embodiment may provide a candidate action list indicating information about a target action set by another user for the first gesture as a target action of the first gesture if the similarity between the first gesture and the preset gesture is less than a threshold value.

[0216] For example, if the similarity between the first gesture and the preset gesture is below a threshold value, the electronic device (100) may provide a candidate action list indicating information about target actions set by other users for the first gesture. The candidate action list may be periodically received from the server via the communication unit (130).

[0217] In operation 1690, the electronic device (100) according to one embodiment may update the first gesture and the target action mapped to the first gesture. The electronic device (100) may transmit the first gesture and the target action mapped to the first gesture to the server via the communication unit (130).

[0218] FIG. 17 is a flowchart illustrating the operation of an electronic device and a server according to one embodiment.

[0219] Referring to FIG. 17, in operation 1705, a server (200) according to one embodiment may transmit a candidate action list of another user to an electronic device (100). The server (200) may periodically transmit the candidate action list to the electronic device (100). The electronic device (100) may store the candidate action list in a gesture DB (340).

[0220] In operation 1710, an electronic device (100) according to one embodiment may monitor a history DB (360). Operation 1710 may correspond to operation 740 of FIG. 7B or operation 1020 of FIG. 10.

[0221] In operation 1715, the electronic device (100) according to one embodiment may identify a first gesture that does not correspond to a preset gesture. The electronic device (100) may identify the first gesture for a preset time or a preset number of times or more according to the second type. If the electronic device (100) determines that the first gesture continues to be identified according to the second type, the electronic device (100) may set a target action of the first gesture based on the similarity between the first gesture and the preset gestures. Operation 1715 may correspond to operation 1030 of FIG. 10.

[0222] In operation 1720, the electronic device (100) according to one embodiment may identify a similarity between the first gesture and a preset gesture. Operation 1720 may correspond to operation 1040 of FIG. 10.

[0223] In operation 1725, the electronic device (100) according to one embodiment may set a target action mapped to the preset gesture as the target action of the first gesture if the similarity between the first gesture and the preset gesture is greater than or equal to a threshold value. The electronic device (100) may identify the target action mapped to the preset gesture based on information stored in the gesture DB (340) and the action DB (350). The electronic device (100) may set the target action of the first gesture to be the same as the target action mapped to the preset gesture. Operation 1725 may correspond to operation 1050 of FIG. 10.

[0224] In operation 1730, the electronic device (100) according to one embodiment may identify whether a candidate action list for the first gesture exists if the similarity between the first gesture and the preset gesture is less than a threshold value. The electronic device (100) may identify the candidate action list for the first gesture stored in the gesture DB (340).

[0225] In operation 1735, if it is determined that the similarity between the first gesture and the preset gesture is less than a threshold value and that a candidate action list for the first gesture exists, the electronic device (100) according to one embodiment may provide the candidate action list for the first gesture as a target action for the first gesture. The electronic device (100) may provide the candidate action list for the first gesture as a target action for the first gesture. Operation 1730 may correspond to operation 1060 of FIG. 10.

[0226] For example, the first gesture may be a gesture for which feature information about the gesture is stored in the gesture DB (340), but for which no mapped target action exists (Action ID: Empty). The first gesture may have a candidate action list indicating target actions set by other users for the first gesture. The electronic device (100) may provide a candidate action list for the first gesture for which feature information about the gesture is stored in the gesture DB (340), but for which no mapped action information is stored.

[0227] In operation 1740, the electronic device (100) according to one embodiment may request a candidate action list for the first gesture from the server (200) if it is determined that the similarity between the first gesture and the preset gesture is less than a threshold value and that there is no candidate action list for the first gesture.

[0228] For example, the first gesture may be a gesture for which feature information about the gesture is not stored in the gesture DB (340). The electronic device (100) may request a list of candidate actions from the server (200) for a gesture for which feature information about the first gesture is not stored in the gesture DB (340).

[0229] In operation 1745, the server (200) according to one embodiment may transmit a candidate action list for a first gesture to the electronic device (100) based on a request received from the electronic device (100).

[0230] In operation 1735, the electronic device (100) according to one embodiment may provide the candidate action list for the first gesture as a target action of the first gesture based on receiving the candidate action list for the first gesture from the server (200). For example, the electronic device (100) may newly store the first gesture and the candidate action list for the first gesture received from the server (200) in the gesture DB (340). The electronic device (100) may provide the candidate action list for a gesture for which feature information regarding the gesture is not stored in the gesture DB (340).

[0231] In operation 1750, an electronic device (100) according to one embodiment may update a first gesture and a target action to a server (200). The electronic device (100) may store the first gesture and the target action mapped to the first gesture, and transmit them to the server (200) via the communication unit (130).

[0232] FIG. 17b is a flowchart illustrating the operation of an electronic device and a server according to one embodiment.

[0233] Referring to FIG. 17B, in operation 1760, an electronic device (100) according to an embodiment may acquire a user's gesture through a sensor (140). The sensor (140) may receive the user's gesture through a camera and transmit it to the electronic device (100). The electronic device (100) may receive the user's gesture from the sensor (140). The user's gesture may be acquired as a captured image.

[0234] In operation 1765, the electronic device (100) according to one embodiment may transmit data corresponding to the user's gesture to the server (200) via the communication unit (140). The data corresponding to the user's gesture may include image data regarding the user's gesture. In addition, the data corresponding to the user's gesture may include data that has been secured to protect the user's personal information. The electronic device (100) may perform security processing on the user's gesture. The electronic device (100) may transmit the data corresponding to the user's gesture to the server (200).

[0235] In operation 1770, the server (200) according to one embodiment can identify whether the user's gesture corresponds to a preset gesture. For example, the server (200) can identify whether the user's gesture corresponds to a preset gesture based on the similarity between the user's gesture and the preset gesture. For example, if the similarity is below a threshold value, the server (200) can identify that the user's gesture does not correspond to the preset gesture. For example, if the similarity is above a threshold value, the server (200) can identify that the user's gesture corresponds to a preset gesture.

[0236] Additionally, in one embodiment, the server (200) may identify target action information corresponding to the user's gesture. If the user's gesture corresponds to a preset gesture, the server (200) may identify target action information corresponding to the user's gesture.

[0237] In operation 1775, the server (200) according to one embodiment may transmit information to the electronic device (100) regarding whether the user's gesture corresponds to a preset gesture.

[0238] Additionally, in one embodiment, the server (200) may transmit information about whether a preset gesture corresponds to the user's gesture and target action information corresponding to the user's gesture to the electronic device (100).

[0239] In operation 1780, the electronic device (100) according to one embodiment may set a target action corresponding to the user's gesture upon receiving information regarding whether the user's gesture corresponds to a preset gesture. For example, upon receiving information from the server (200) regarding whether a second gesture following a first gesture corresponds to a preset gesture, the electronic device (100) may set a target action mapped to the second gesture as the target action of the first gesture.

[0240] Additionally, the electronic device (100) according to one embodiment may perform a target action corresponding to the user's gesture upon receiving information regarding whether the user's gesture corresponds to a preset gesture and target action information corresponding to the user's gesture. In one embodiment, the electronic device (100) may omit an operation of identifying a target action corresponding to the user's gesture.

[0241] Fig. 18 is a detailed block diagram of an electronic device according to one embodiment.

[0242] The electronic device (1800) of FIG. 18 may be an example of the electronic device (100) of FIG. 2.

[0243] Referring to FIG. 18, an electronic device (1800) may include a processor (1801) and a memory (1802). The processor (1801) and the memory (1802) included in the electronic device (1800) may correspond to the processor (110) and the memory (120) included in the electronic device (100) of FIG. 2. In addition, the communication unit (820) included in the electronic device (1800) may correspond to the communication unit (130) included in the electronic device (100) of FIG. 2, and the sensor (1832) included in the electronic device (1800) may correspond to the sensor (140) included in the electronic device (100) of FIG. 2.

[0244] An electronic device (1800) according to one embodiment may further include, in addition to a processor (1801) and a memory (1802), a tuner unit (1810), a communication unit (1820), a detection unit (1830), an input / output unit (1840), a video processing unit (1850), a display (1860), an audio processing unit (1870), an audio output unit (1880), and a user input unit (1890).

[0245] The tuner unit (1810) can select and tune only the frequency of the channel to be received by the electronic device (1800) among many radio wave components through amplification, mixing, resonance, etc. of broadcast content received via wire or wireless. The content received through the tuner unit (1810) is decoded and separated into audio, video, and / or additional information. The separated audio, video, and / or additional information can be stored in the memory (1802) under the control of the processor (1801).

[0246] The communication unit (1820) can connect the electronic device (1800) to peripheral devices, external devices, a server (200), a display device (300), a remote control device (400), a mobile terminal, etc., under the control of the processor (1801). The communication unit (1820) can include at least one communication module capable of performing wireless communication. The communication unit (1820) can include at least one of a wireless LAN module (1821), a Bluetooth module (1822), and a wired Ethernet (1823) in accordance with the performance and structure of the electronic device (1800).

[0247] The Bluetooth module (1822) can receive a Bluetooth signal transmitted from a peripheral device according to the Bluetooth communication standard. The Bluetooth module (1822) can be a BLE (Bluetooth Low Energy) communication module and can receive a BLE signal. The Bluetooth module (1822) can constantly or temporarily scan the BLE signal to detect whether a BLE signal is received. The wireless LAN module (1821) can transmit and receive a Wi-Fi signal with a peripheral device according to the Wi-Fi communication standard.

[0248] In one embodiment, the communication module for communicating with the server (200), the communication module for communicating with the display device (300), the communication module for communicating with the remote control device (400), and the communication module for communicating with the mobile terminal may be different or may be the same.

[0249] The detection unit (1830) detects the user's voice, the user's image, or the user's interaction, and may include a microphone (1831), a sensor (1832), and a light receiving unit (1833).

[0250] The microphone (1831) can receive an audio signal including a user's uttered voice or noise, and can convert the received audio signal into an electrical signal and output it to the processor (1801).

[0251] In one embodiment, the microphone (1831) may be provided in a remote control device (400), such as a remote control, a mobile terminal, or an AI speaker. For example, the mobile terminal may execute an application for remotely controlling the electronic device (1800). In this case, the microphone (1831) provided in the remote control device (400) may receive an audio signal including a user's spoken voice or noise, and the remote control device (400) may convert the audio signal into a control signal and transmit it to the electronic device (1800). The electronic device (1800) may receive a control signal from the remote control device (400) through the communication unit (1830).

[0252] The sensor (1832) detects the user's image, or the user's interaction, gesture, and touch, and may include a distance sensor, an image sensor, a gesture sensor, an illumination sensor, and the like. The distance sensor may include various sensors that detect the distance between the electronic device (1800) and the user, such as an ultrasonic sensor, an IR (Infrared Radiation) sensor, and a TOF (Time Of Flight) sensor. The distance sensor detects the distance from the user and can transmit the sensing data to the processor (1801). The image sensor can capture the user's gesture by taking a picture using a camera or the like and transmit it to the processor (1801). The gesture sensor can detect the movement speed or direction using an acceleration sensor or a gyro sensor. The illumination sensor can detect the ambient illumination.

[0253] The optical receiver (1833) can receive an optical signal (including a control signal). The optical receiver (1833) can receive an optical signal corresponding to a user input (e.g., touch, press, touch gesture, voice, or motion) from a control device such as a remote control or a mobile phone.

[0254] The input / output unit (1840) can receive video (e.g., dynamic image signal or still image signal), audio (e.g., voice signal or music signal), and additional information from an external device, etc., under the control of the processor (1801). The input / output unit (1840) can include a port for outputting video and audio together, and can also include a port for separate output of video and audio.

[0255] The input / output unit (1840) may include one of an HDMI port (High-Definition Multimedia Interface port, 1841), a component jack (component jack, 1842), a PC port (PC port, 1843), and a USB port (USB port, 1844). The input / output unit (1840) may include a combination of an HDMI port (1841), a component jack (1842), a PC port (1843), and a USB port (1844). In addition, the input / output unit (1840) may include one of a DP (Display Port), a Thunderbolt port, a VGA port, an RGB port, a D-SUB, and a DVI.

[0256] In one embodiment, when the electronic device (100) is a content providing device such as a set-top box, the input / output unit (1840) can output video, audio, and additional information to the display device (300) under the control of the processor (1801).

[0257] The video processing unit (1850) processes image data to be displayed by the display (1860) and can perform various image processing operations such as decoding, rendering, scaling, noise filtering, frame rate conversion, and resolution conversion on the image data.

[0258] The display (1860) can display content received from a broadcasting station, an external server, or an external device such as an external storage medium. The content may be a media signal, including a video signal, an image, a text signal, etc.

[0259] In one embodiment, the display (1860) may output information about a target action predicted for a user's gesture to the screen. The processor (1801) may control the display (1860) to output information about a target action predicted for the user's gesture to the screen.

[0260] In one embodiment, when the electronic device (1800) is connected to the display device (300), the processor (1801) can transmit information about a predicted target action for the user's gesture to the display device (300).

[0261] The audio processing unit (1870) performs processing on audio data. The audio processing unit (1870) can perform various processing such as decoding, amplification, and noise filtering on audio data.

[0262] The audio output unit (1880) can output audio included in content received through the tuner unit (1810) under the control of the processor (1801), audio input through the communication unit (1820) or the input / output unit (1840), and audio stored in the memory (1802). The audio output unit (1880) can include at least one of a speaker (1881), a headphone (1882), or an S / PDIF (Sony / Philips Digital Interface: output terminal) (1883).

[0263] In one embodiment, the audio output unit (1880) may output information regarding a target action predicted for a user's gesture as audio. The processor (1801) may control the audio output unit (1880) to output information regarding a target action predicted for a user's gesture as audio.

[0264] The user input unit (1890) can receive user input for controlling the electronic device (1800). The user input unit (1890) can include various types of user input devices, including, but not limited to, a touch panel that detects a user's touch, a button that receives a user's push operation, a wheel that receives a user's rotation operation, a keyboard, a dome switch, a microphone for voice recognition, a motion detection sensor that senses motion, etc. When a remote control device (400) or other mobile terminal controls the electronic device (1800), the user input unit (1890) can receive a control signal received from the remote control device (400).

[0265] The server (200) can transmit and receive data with the electronic device (1800) through the communication unit. In one embodiment, the server (200) may separately include a gesture server that stores information stored in the gesture DB (340) and an action server that stores information stored in the action DB (350). The server (200) may receive mapping information between new gestures and target actions from the electronic device (1800). The server (200) may store a candidate action list indicating information about target actions set by other users. The server (200) may store the candidate action list by listing the most frequently mapped actions for each gesture. The server (200) may periodically provide the candidate action list to the electronic device (1800). Alternatively, when the server (200) receives a request for the candidate action list from the electronic device (1800), it may provide the candidate action list to the electronic device (1800).

[0266] An operating method of an electronic device (100) according to one embodiment includes a step (710) of storing data related to the fact that the first gesture does not correspond to a preset gesture in a database if a first gesture acquired through a sensor (140) does not correspond to a preset gesture, and a step (720) of setting a target action mapped to the second gesture as a target action of the first gesture if a second gesture acquired through the sensor (140) after the first gesture corresponds to a preset gesture.

[0267] According to one embodiment, the electronic device (100) can propose a target action appropriate for the user's gesture based on the user's gesture and subsequent gestures, even if the user's gesture does not correspond to a preset gesture. Accordingly, user convenience for the electronic device (100) can be enhanced.

[0268] A method of operating an electronic device (100) according to one embodiment may further include a step of storing data regarding whether a user's gesture (305) obtained through a sensor (140) corresponds to a preset gesture in a database, and a step of detecting whether data related to whether the user's gesture (305) does not correspond to a preset gesture is stored in the database.

[0269] The method of operating an electronic device (100) according to one embodiment may further include a step of identifying whether the first gesture is stored in the database a preset number of times during a preset time.

[0270] The step (720) of setting a target action mapped to a second gesture according to one embodiment as a target action of the first gesture may include a step of identifying a similarity between the first gesture and the second gesture based on identifying the second gesture, and a step of setting a target action mapped to the second gesture as the target action of the first gesture based on the similarity between the first gesture and the second gesture being greater than or equal to a first threshold value.

[0271] According to one embodiment, the electronic device (100) may consider the recognition failure result and, if it determines that the first and second gestures are completely unrelated, may stop the target action prediction operation. Accordingly, the electronic device (100) can efficiently provide gesture recognition services.

[0272] A method of operating an electronic device according to one embodiment may further include a step of setting a target action of the first gesture based on a similarity between the first gesture and a preset gesture, based on a time for which the first gesture is maintained or a number of times the first gesture is identified.

[0273] According to one embodiment, an electronic device (100) can propose a target action appropriate for a user's gesture based on the similarity between the user's gesture and the preset gesture, even if the user's gesture does not correspond to a preset gesture. Accordingly, user convenience of the electronic device (100) can be enhanced.

[0274] The method of operating an electronic device according to one embodiment may further include a step (1050) of setting a target action mapped to the preset gesture as the target action of the first gesture if the similarity between the first gesture and the preset gesture is greater than or equal to a second threshold value.

[0275] The method of operating an electronic device according to one embodiment may further include a step (1060) of providing a candidate action list indicating information about a target action set by another user for the first gesture as a target action of the first gesture, if the similarity between the first gesture and the preset gesture is less than a second threshold value.

[0276] According to one embodiment, an electronic device (100) can propose an optimal target action for a user's gesture by suggesting target actions mapped by other users as candidate actions. Accordingly, user convenience for the electronic device (100) can be enhanced.

[0277] The method of operating an electronic device (100) according to one embodiment may further include a step (620) of identifying whether a user's gesture (305) corresponds to a preset gesture.

[0278] A method of operating an electronic device (100) according to one embodiment may further include a step (630) of identifying a target action mapped to a preset gesture when a user's gesture (305) is identified as corresponding to a preset gesture.

[0279] The operating method of the electronic device (100) according to one embodiment may further include a step of acquiring feature information (325) corresponding to the user's gesture (305) based on the acquired user's gesture (305). The operating method of the electronic device (100) may further include a step of identifying the user's gesture (305) as corresponding to the preset gesture if the similarity between the feature information (325) and the feature information of the preset gesture is greater than or equal to a reference value. The operating method of the electronic device (100) may further include a step of determining that the user's gesture (305) does not correspond to the preset gesture based on the determination that the similarity is less than the reference value.

[0280] According to one embodiment, the operating method of the electronic device (100) may further include a step of receiving information from an external device through the communication unit (130) as to whether the user's gesture (305) corresponds to a preset gesture. The operating method of the electronic device (100) may further include a step of setting a target action corresponding to the user's gesture upon receiving the information as to whether the user's gesture corresponds to a preset gesture. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the 'non-transitory storage medium' only means that it is a tangible device and does not include a signal (e.g., electromagnetic wave), and this term does not distinguish between cases where data is stored semi-permanently in the storage medium and cases where data is stored temporarily. For example, the 'non-transitory storage medium' may include a buffer in which data is temporarily stored.

[0281] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

Claims

1. In an electronic device (100), A sensor (140) for acquiring a user's gesture (305); A memory (120) storing one or more instructions; and comprising one or more processors (110) for executing one or more instructions stored in the memory (120); The one or more processors (110) execute the one or more instructions, If the first gesture acquired through the above sensor (140) does not correspond to a preset gesture, data related to the first gesture not corresponding to the preset gesture is stored in the database, An electronic device (100) that sets a target action mapped to the second gesture as the target action of the first gesture if the second gesture acquired through the sensor (140) after the first gesture corresponds to the preset gesture.

2. In paragraph 1, The one or more processors (110) execute the one or more instructions, Data on whether the user's gesture (305) obtained through the sensor (140) corresponds to the preset gesture is stored in the database, An electronic device (100) that detects whether data related to the user's gesture (305) not corresponding to the preset gesture is stored in the database.

3. In paragraph 1 or 2, The one or more processors (110) execute the one or more instructions, An electronic device (100) that sets a target action mapped to the second gesture as the target action of the first gesture if the similarity between the acquired first gesture and the second gesture is greater than or equal to a first threshold value based on the second gesture corresponding to the preset gesture.

4. In any one of paragraphs 1 to 3, The one or more processors (110) execute the one or more instructions, An electronic device (100) that sets a target action of the first gesture based on the similarity between the first gesture and the preset gesture, based on the time for which the first gesture is maintained or the number of times the first gesture is identified.

5. In paragraph 4, The one or more processors (110) execute the one or more instructions, If the similarity between the first gesture and the preset gesture is greater than or equal to the second threshold value, the target action mapped to the preset gesture is set as the target action of the first gesture, An electronic device (100) that provides a candidate action list corresponding to information about a target action set by another user for the first gesture as a target action of the first gesture if the similarity between the first gesture and the preset gesture is less than the second threshold value.

6. In paragraph 5, The above electronic device (100) further includes a communication unit (130), The one or more processors (110) execute the one or more instructions, An electronic device (100) that requests a candidate action list for the first gesture from an external device through the communication unit (130) if there is no candidate action list for the first gesture.

7. In paragraph 4, The one or more processors (110) execute the one or more instructions, An electronic device (100) that controls to provide a UI (User Interface) including an inquiry corresponding to setting of a target action for the first gesture if the similarity between the first gesture and the preset gesture is less than a second threshold value.

8. In any one of paragraphs 1 to 7, The one or more processors (110) execute the one or more instructions, Identify whether the user's gesture (305) corresponds to the preset gesture, An electronic device (100) that identifies a target action mapped to the preset gesture when the user's gesture (305) is identified as corresponding to the preset gesture.

9. In any one of paragraphs 1 to 8, The one or more processors (110) execute the one or more instructions, Based on the user's gesture (305) obtained above, feature information (325) corresponding to the user's gesture (305) is obtained, If the similarity between the above feature information (325) and the feature information of the preset gesture is greater than or equal to a reference value, the user's gesture (305) is identified as corresponding to the preset gesture. An electronic device (100) that identifies the user's gesture (305) as not corresponding to the preset gesture if the similarity is less than the reference value.

10. In any one of paragraphs 1 to 9, The above electronic device (100) further includes a communication unit (130), The one or more processors (110) execute the one or more instructions, Receive information about whether the user's gesture (305) corresponds to the preset gesture through the communication unit (130) from an external device, An electronic device (100) that sets a target action corresponding to the user's gesture upon receiving information about whether the gesture corresponds to the above-described preset gesture.

11. In any one of paragraphs 1 to 10, The one or more processors (110) execute the one or more instructions, An electronic device (100) that updates the first gesture and the target action mapped to the first gesture.

12. In any one of paragraphs 1 to 11, The one or more processors (110) execute the one or more instructions, An electronic device (100) that provides a target action for the first gesture based on at least one of: a time period during which the first gesture is maintained, a number of times the first gesture is identified, the presence of the second gesture identified subsequent to the first gesture, characteristic information of the first gesture, or a similarity between the first gesture and the preset gesture, upon identifying the first gesture.

13. In the operating method of an electronic device (100), If the first gesture acquired through the sensor (140) does not correspond to a preset gesture, a step (710) of storing data related to the first gesture not corresponding to the preset gesture in a database; and A method including a step (720) of setting a target action mapped to the second gesture as the target action of the first gesture if the second gesture acquired through the sensor (140) after the first gesture corresponds to the preset gesture.

14. In paragraph 13, The above method of operation is, A step of storing data on whether the user's gesture (305) obtained through the sensor (140) corresponds to the preset gesture in the database; and A method further comprising the step of detecting whether data related to the user's gesture (305) not corresponding to the preset gesture is stored in the database.

15. In clause 13 or 14, The above method of operation is, A method further comprising the step of setting a target action of the first gesture based on a similarity between the first gesture and the preset gesture, based on a time for which the first gesture is maintained or a number of times the first gesture is identified.

Images