Electronic device and control method therefor

Abstract

Disclosed is an electronic device. The electronic device according to the present disclosure may comprise a camera, a display, a memory for storing instructions, and at least one processor including processing circuitry, wherein the instructions, when executed individually or collectively by the at least one processor, cause the electronic device to: identify a gesture input of a user on the basis of an image acquired by means of the camera; on the basis of a plurality of gestures set for executing a plurality of functions of the electronic device, identify whether the gesture input is a gesture set for executing a function; when it is identified that the gesture input is a gesture set for executing the function, identify, on the basis of the number of times the gesture input is input, whether the gesture input is a gesture embodied to the user; when it is identified that the gesture input is a gesture embodied to the user, execute a function set for the gesture input among the plurality of functions; and when it is identified that the identified gesture is a gesture that is not embodied to the user, display, on the display, a first interface screen for inquiring about execution of a function set for the gesture input.

Description

Electronic device and its control method

[0001] The present disclosure relates to an electronic device controlled using gestures and a method for controlling the same.

[0002] Users can operate the TV using gestures without a remote control. Previously, TVs could only be operated using fixed gestures, but with the advancement of gesture recognition technology, users can now control the TV by setting their own desired gestures.

[0003] Although users have set their desired gestures, there is a possibility that they may forget them. Consequently, there is a need for a method to distinguish between gestures that users can internalize and continue to use, and those that cannot be used because they have not been internalized.

[0004] An electronic device according to the present disclosure comprises at least one processor including a camera, a display, a memory for storing instructions, and a processing circuitry, wherein when the instructions are executed individually or collectively by the at least one processor, the electronic device identifies a user’s gesture input based on an image acquired through the camera, identifies whether the gesture input is a gesture set for the execution of a function based on a plurality of gestures set for the execution of a plurality of functions of the electronic device, and if the gesture input is identified as a gesture set for the execution of a function, identifies whether the gesture input is a gesture embodied by the user based on the number of times the gesture input is input, and if the gesture input is identified as a gesture embodied by the user, executes the function set for the gesture input among the plurality of functions, and if the identified gesture is identified as a gesture not embodied by the user, displays a first interface screen on the display for inquiring about the execution of the function set for the gesture input.

[0005] In addition, the memory stores information regarding the number of times a positive response to an execution inquiry for a function corresponding to a gesture is continuously entered for each of the plurality of gestures, and

[0006] When the above instructions are executed individually or collectively by the at least one processor, the electronic device may identify the number of times a positive response to a query for the execution of a function set in the gesture input is continuously entered based on information stored in the memory, and identify whether the gesture input is a gesture embodied by the user based on the identified number.

[0007] Additionally, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may identify the gesture input as a gesture embodied by the user if the number of consecutive positive responses to a query for the execution of a function set in the gesture input is greater than or equal to a preset number, and identify the gesture input as a gesture not embodied by the user if the number of consecutive positive responses to a query for the execution of a function set in the gesture input is less than the preset number.

[0008] When the above instructions are executed individually or collectively by the at least one processor, the electronic device may update the number of positive responses corresponding to the gesture input stored in the memory when a positive response is received to a query for the execution of a function set in the gesture input while the first interface screen is displayed.

[0009] Additionally, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may display a second interface screen on the display for inquiring about the function to be set in the gesture input when a negative response to an inquiry about the execution of the function set in the gesture input is received while the first interface screen is displayed, and may display a third interface screen on the display containing information related to the function to be set in the gesture input when user input regarding the function to be set in the gesture input is received.

[0010] The third interface screen above may include at least one of a gesture already set on the electronic device for a function to be set for the gesture input, a gesture set by another user for a function to be set for the gesture input, the gesture input, and a function set by another user for the gesture input.

[0011] Additionally, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may identify the similarity between the gesture input and a gesture set by another user for a function set on the gesture input, and adjust the preset number of times to identify whether the gesture input is a gesture embodied by the user based on the identified similarity.

[0012] When the above instructions are executed individually or collectively by the at least one processor, the electronic device may reduce the preset number of times for identifying whether the gesture input is a gesture embodied by the user if the identified similarity is greater than or equal to a preset threshold, and increase the preset number of times for identifying whether the gesture input is a gesture embodied by the user if the identified similarity is less than a preset threshold.

[0013] Additionally, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may display a second interface screen on the display for querying the function to be set for the gesture input when the gesture input is identified as not being a gesture set for the execution of a function, and may display a third interface screen on the display containing information related to the function to be set for the gesture input when user input regarding the function to be set for the gesture input is received.

[0014] In addition, regarding the control method of an electronic device,

[0015] The method may include: a step of identifying a user’s gesture input based on an image acquired through a camera; a step of identifying whether the gesture input is a gesture set for the execution of a function based on a plurality of gestures set for the execution of a plurality of functions of the electronic device; a step of identifying whether the gesture input is a gesture embodied by the user based on the number of times the gesture input has been input, if the gesture input is identified as a gesture set for the execution of a function; a step of executing a function set for the gesture input among the plurality of functions if the gesture input is identified as a gesture embodied by the user; and a step of displaying a first interface screen on a display for inquiring about the execution of a function set for the gesture input if the identified gesture is identified as a gesture not embodied by the user.

[0016] In addition, a non-transient computer-readable recording medium storing one or more instructions executed by a processor of an electronic device to perform an operation of the electronic device may include: a step of identifying a user’s gesture input based on an image acquired through a camera; a step of identifying whether the gesture input is a gesture set for the execution of a function based on a plurality of gestures set for the execution of a plurality of functions of the electronic device; a step of identifying whether the gesture input is a gesture embodied by the user based on the number of times the gesture input has been entered, if the gesture input is identified as a gesture set for the execution of a function; a step of executing a function set for the gesture input among the plurality of functions if the gesture input is identified as a gesture embodied by the user; and a step of displaying a first interface screen on a display for inquiring about the execution of a function set for the gesture input if the identified gesture is identified as a gesture not embodied by the user.

[0017] FIG. 1 is a drawing for explaining a method of controlling an electronic device according to at least one embodiment of the present disclosure.

[0018] FIG. 2 is a drawing for explaining the configuration of an electronic device according to at least one embodiment of the present disclosure.

[0019] FIG. 3 is a drawing for explaining the detailed configuration of an electronic device according to at least one embodiment of the present disclosure.

[0020] FIG. 4 is a flowchart illustrating a method for controlling an electronic device according to at least one embodiment of the present disclosure.

[0021] FIG. 5 is a table for explaining the types of gestures according to at least one embodiment of the present disclosure.

[0022] FIG. 6 is a drawing for illustrating an interface screen for inquiring about function selection according to at least one embodiment of the present disclosure.

[0023] FIG. 7 is a drawing illustrating a case where a user incorrectly inputs a gesture according to at least one embodiment of the present disclosure.

[0024] FIG. 8 is a drawing illustrating an interface screen for inquiring about the execution of a desired function by a user according to at least one embodiment of the present disclosure.

[0025] FIG. 9 is a drawing for explaining communication between an electronic device and a server according to at least one embodiment of the present disclosure.

[0026] FIG. 10 is a drawing for illustrating an interface screen for resetting a gesture according to at least one embodiment of the present disclosure.

[0027] FIGS. 11 to 14 are drawings for explaining a method for setting criteria for determining whether a gesture according to at least one embodiment of the present disclosure has been learned by a user.

[0028] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments.

[0029] In relation to the description of the drawings, similar reference numerals may be used for similar or related components.

[0030] The singular form of the noun corresponding to an item may include one or plural items, unless the relevant context clearly indicates otherwise.

[0031] In this document, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0032] Terms such as "first," "second," or "first" or "second" may be used simply to distinguish a component from another corresponding component and do not limit the components in other aspects (e.g., importance or order).

[0033] Where any (e.g., 1st) component is referred to as "coupled" or "connected" to another (e.g., 2nd) component, with or without the terms "functionally" or "communicationly," it means that the component may be connected to the other component directly (e.g., via a wire), wirelessly, or through a third component.

[0034] Terms such as “include” or “have” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in this document, and do not preclude the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0035] When it is said that one component is “connected,” “combined,” “supported,” or “in contact” with another component, this includes not only cases where the components are directly connected, combined, supported, or in contact, but also cases where they are indirectly connected, combined, supported, or in contact through a third component.

[0036] When it is said that a component is located “on” another component, this includes not only cases where one component is in contact with the other, but also cases where another component exists between the two components.

[0037] The term “and / or” includes a combination of multiple related described components or any of the multiple related described components.

[0038] The present disclosure will be described in more detail below with reference to the attached drawings.

[0039] FIG. 1 is a drawing for explaining the operation of an electronic device according to one embodiment of the present disclosure.

[0040] The electronic device (100) according to the present disclosure may be a television with a built-in camera (110). However, the present disclosure is not limited thereto, and the electronic device (100) may be various types of devices including a camera (110), such as a smartphone, tablet PC, laptop PC, etc.

[0041] Referring to FIG. 1, the electronic device (100) can acquire an image by taking a picture of a user (1) using a camera (110), identify the user's gesture from the image, and perform a function corresponding to the gesture. For example, the electronic device (100) can identify a function corresponding to the identified gesture among a plurality of functions corresponding to a plurality of gestures, and perform the identified function. For example, if the electronic device (100) has a screen capture function set for a gesture where the user (1) makes a V shape with their hand, the electronic device (100) can execute a screen capture function for the content being played.

[0042] In one embodiment, the electronic device (100) can identify whether a gesture taken by the user (1) is a gesture embodied in the user (1).

[0043] The fact that a gesture is embodied in the user may include the user recognizing that a function is executed on the electronic device (100) according to the gesture the user has taken.

[0044] If the electronic device (100) identifies that the user's gesture is a gesture embodied in the user (1), it can execute a function corresponding to the gesture. However, if the electronic device (100) identifies that the user's gesture is not a gesture embodied in the user (1), it can make the gesture embodied in the user or set a different function for the gesture.

[0045] According to the present disclosure, the electronic device (100) can freely set a gesture for controlling the electronic device and provide the user with the opportunity to reset the gesture even if the user forgets the gesture set while using the electronic device. Accordingly, a high degree of freedom can be granted to the user when setting the gesture. In addition, according to the present disclosure, the electronic device (100) can distinguish whether the already set gesture is a gesture embodied by the user, thereby enabling effective management of the set gesture.

[0046] FIG. 2 is a drawing for explaining the configuration of an electronic device according to one embodiment of the present disclosure.

[0047] Referring to FIG. 2, the electronic device (100) may include a camera (110), a display (120), at least one memory (130) (hereinafter referred to as memory (130)), and at least one processor (140) (hereinafter referred to as processor (140)).

[0048] The camera (110) can generate an image. For example, the camera may include an RGB camera. The camera (110) can acquire an image by capturing the front of the display (120). If a user is in front of the display (120), the user may be captured in the image taken by the camera (110).

[0049] The camera (110) may be placed at various locations on the electronic device (100). For example, the camera (110) may be placed in the bezel area of ​​the display (130). The camera (110) may be placed in the upper central area, lower central area, left central area, or right central area of ​​the bezel area of ​​the display (130), but the present disclosure is not limited to this example.

[0050] The camera (110) may include a lens assembly and an image sensor. However, such a configuration is exemplary, and it is understood that new configurations may be added or some configurations omitted in addition to such configurations when carrying out the present disclosure. For example, the camera (110) may further include an image signal processor (ISP). The image signal processor may be configured as at least part of the processor (140).

[0051] A lens assembly can collect light incident from the outside. A lens assembly may include one or more lenses. For example, a lens assembly can refract light incident from the outside. The refracted light can be focused on an image sensor.

[0052] An image sensor can generate an image corresponding to light received through a lens assembly. The image sensor can be implemented as a CCD (charged coupled device) sensor or a CMOS (complementary metal oxide semiconductor) sensor. The image sensor may include a pixel array composed of multiple pixels. For example, the image sensor can generate an image by converting light into an electrical image signal using multiple pixels. The image signal can be amplified and converted into a digital signal, after which it can be processed. The image generated by the image sensor can be provided to a processor (140).

[0053] The display (120) can output visualized information to the user. The visualized information may include visual objects displayed on the screen of the display (120). For example, the visual objects may include images, UI (user interface) elements, etc. The UI elements may include icons, GUI (graphic user interface), etc.

[0054] The display (120) may be implemented as a display including a self-emissive element or as a display including a non-emissive element and a backlight. For example, the display (110) may be implemented as various types of displays such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, an LED (Light Emitting Diodes) display, a micro LED display, a Mini LED display, a QLED (Quantum dot light-emitting diodes) display, etc.

[0055] According to various embodiments of the present disclosure, the memory (130) may store data necessary for the electronic device (100) to operate. Depending on the purpose of data storage, the memory (130) may be implemented as a memory embedded in the display device (100) (e.g., volatile memory (e.g., semi-permanent memory such as RAM (random access memory)), non-volatile memory (e.g., permanent memory such as ROM (read-only memory)), flash memory, hard drive or solid-state drive, etc.), or as a memory that is detachable from the display device (100) (e.g., memory card, external memory, etc.).

[0056] Instructions may be stored in the memory (130). The processor (140) may perform the operation of the electronic device (100) according to various embodiments of the present disclosure by executing the instructions in the memory (130) individually or collectively. Additionally, programs and data for operating the electronic device (100) may be stored in the memory (130). For example, the memory (130) may store one or more software applications, such as operating system (or system) software applications, firmware software applications, driver software applications, plugin (e.g., add-in, add-on, and / or applet) software applications, and / or any other suitable software applications.

[0057] Meanwhile, in the present disclosure, the term memory (130) may be used to include memory (130), ROM, RAM, or a memory card (e.g., micro SD card, memory stick) mounted in the processor (140). In addition, various information necessary within the scope of achieving the purpose of the present disclosure may be stored in the memory (130), and the information stored in the memory (130) may be updated as it is received from an external device or input by a user.

[0058] The processor (140) can control the overall operations of the electronic device (100). For example, the processor (140) can cause other components of the electronic device (100) to perform various operations by executing instructions stored in memory (130). For example, the processor (140) can control the operation of the electronic device (100) by operatively connecting with the camera (110), the display (120), and memory (130). Additionally, the processor (140) can control the operation of the electronic device (100) according to the present disclosure by executing one or more instructions stored in memory (130). The processor (140) may be composed of one or more processors.

[0059] The processor (140) may be implemented as one or more integrated circuit (or circuitry) chips and may perform various data processing operations. The processor (140) may include at least one electrical circuit and may process instructions (or programs, data, etc.) stored in memory (140) individually or collectively. The processor (140) may include a processor assembly comprising one or more processing circuits. The processor (140) may include any processing circuit that is operative to control the performance and operation of one or more components of the electronic device (100) (e.g., camera (110), display (120), memory (130)). For example, the processor (140) (e.g., application processor (AP)) may be implemented as a system on chip (SoC) (e.g., a single chip or chipset). For example, the processor (140) may be implemented with a plurality of cores (or at least one core circuit), a plurality of chips, or a plurality of chipsets. For example, the processor (140) may include one or more processing circuits. For example, the processor (140) may include one or more processing circuits configured to perform the various functions of the present disclosure individually and / or collectively.

[0060] The processor (140) may include one or more of a CPU (Central Processing Unit), GPU (Graphics Processing Unit), APU (Accelerated Processing Unit), MIC (Many Integrated Core), DSP (Digital Signal Processor), NPU (Neural Processing Unit), hardware accelerator, or machine learning accelerator. The processor (140) may control one or any combination of other components of the electronic device (100) and may perform operations or data processing related to communication. The processor (140) may execute one or more programs or instructions stored in the memory (130) of the electronic device (100). For example, the processor (140) may perform a method according to one embodiment of the present disclosure by executing one or more instructions stored in the memory (130).

[0061] When a method according to one embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by a single processor or by a plurality of processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to one embodiment, the first operation, the second operation, and the third operation may all be performed by a first processor, or the first operation and the second operation may be performed by a first processor (e.g., a general-purpose processor) and the third operation may be performed by a second processor (e.g., an artificial intelligence dedicated processor).

[0062] The processor (140) may be implemented as a single-core processor including one core, or as one or more multicore processors including multiple cores (e.g., homogeneous multicore or heterogeneous multicore). When the processor (140) is implemented as a multicore processor, each of the multiple cores included in the multicore processor may include internal processor memory such as cache memory or on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. Additionally, each of the multiple cores included in the multicore processor (or some of the multiple cores) may independently read and execute program instructions for implementing a method according to one embodiment of the present disclosure, or all (or some) of the multiple cores may be linked together to read and execute program instructions for implementing a method according to one embodiment of the present disclosure.

[0063] When a method according to one embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one of the plurality of cores included in a multi-core processor, or may be performed by a plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to one embodiment, the first operation, the second operation, and the third operation may all be performed by a first core included in a multi-core processor, or the first operation and the second operation may be performed by a first core included in a multi-core processor and the third operation may be performed by a second core included in a multi-core processor.

[0064] In the embodiments of the present disclosure, a processor may mean a system-on-chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, GPU, APU, MIC, DSP, NPU, hardware accelerator, or machine learning accelerator, but the embodiments of the present disclosure are not limited thereto.

[0065] FIG. 3 is a drawing for explaining the detailed configuration of an electronic device according to one embodiment of the present disclosure.

[0066] Referring to FIG. 3, the electronic device (100) may include a camera (110), a display (120), a memory (130), a processor (140), a communication circuit (150), an input interface (160), and an output interface (170). However, such configurations are exemplary, and it is understood that new configurations may be added or some configurations omitted in addition to such configurations when implementing the present disclosure. Meanwhile, detailed descriptions of configurations shown in FIG. 3 that overlap with configurations shown in FIG. 2 will be omitted.

[0067] The communication circuit (150) can perform data communication with an external electronic device under the control of the processor (140). The external electronic device may include a server, a home appliance, a mobile device (e.g., a smartphone, a tablet PC, a wearable device, etc.).

[0068] The communication circuit (150) can perform data communication with an external electronic device (e.g., server, home appliance, mobile device, etc.) under the control of the processor (140). For example, the communication circuit (150) can communicate with an external electronic device through a network. For example, the processor (140) can receive data from an external electronic device through the communication circuit (150) and transmit data to an external electronic device through the communication circuit (150).

[0069] The communication circuit (150) may include hardware components to support the transmission and / or reception of electrical signals between the electronic device (100) and an external electronic device. For example, the communication circuit (150) may perform data communication between the electronic device (100) and the electronic device using at least one of a data communication method including wired LAN, wireless LAN, Wi-Fi, Wi-Fi Direct, Bluetooth, ZigBee, WFD (Wi-Fi Direct), infrared communication (IrDA, infrared Data Association), BLE (Bluetooth Low Energy), NFC (Near Field Communication), Wibro (Wireless Broadband Internet), WiMAX (World Interoperability for Microwave Access), SWAP (Shared Wireless Access Protocol), WiGig (Wireless Gigabit Alliances), and RF communication. The communication circuit (140) may also be expressed as a communication interface.

[0070] The input interface (160) includes circuitry. The input interface (160) can receive user input and transmit the user input to the processor (140). For example, the input interface (160) can receive various user inputs for setting or selecting various functions supported by the electronic device (100).

[0071] The input interface (160) may include various types of input devices.

[0072] According to one example, the input interface (160) may include a physical button. The physical button may include a function key or a dial button. The physical button may be implemented as one or more keys.

[0073] According to one example, the input interface (160) can receive user input using a touch method. For example, the input interface (160) can be implemented as a touch screen capable of performing the function of a display (120).

[0074] According to one example, the input interface (160) can receive user voice using a microphone. The processor (140) can perform a function corresponding to the user voice using voice recognition. For example, the processor (140) can convert the user voice into text data using a Speech To Text (STT) function, obtain control command data based on the text data, and perform a function corresponding to the user voice based on the control command data. According to an embodiment, the STT function may be performed on an external server.

[0075] The output interface (170) may include a display (120) and a speaker. The display (120) is as described above. The speaker may output an audio signal. The processor (140) may output warning sounds, notification messages, response messages corresponding to user input, etc., related to the operation of the electronic device (100) through the speaker.

[0076] FIG. 4 is a flowchart for explaining the operation of an electronic device according to one embodiment of the present disclosure. A processor (140) may perform at least one of the operations of FIG. 4. When instructions stored in memory (130) are executed individually or collectively by the processor (140), the electronic device (100) may be made to perform the operations of FIG. 4.

[0077] In operation 405, the electronic device (100) can acquire an image through the camera (110) and identify the user's gesture input based on the acquired image.

[0078] In one embodiment, the electronic device (100) can enter a gesture mode when a trigger gesture is detected in an image acquired through a camera (110).

[0079] A trigger gesture may be a gesture command to activate a gesture mode. The trigger gesture may be set at the time of manufacture or initial use of the electronic device (100), or may be set and changed by user input. A memory (130) may store information about the trigger gesture. For example, the memory (130) may store the type (e.g., class) of the trigger gesture and / or a feature vector for the trigger gesture. The feature vector may include information about the position of the hand, the angle of the fingers, the shape of the hand, etc.

[0080] The electronic device (100) can identify the user's gesture in an image obtained through the camera (110).

[0081] For example, the electronic device (100) can detect a hand in an image by detecting an outline in the image, and can identify the type of gesture (e.g., class of gesture) by analyzing the position, shape, and angle of the fingers of the detected hand. For example, the electronic device (100) can obtain a feature vector for the gesture in the image. The feature vector may include the position of the hand, the angle of the fingers, the shape of the hand, etc. For example, the electronic device (100) can identify the gesture using an artificial intelligence model. The artificial intelligence model may be a neural network model trained to recognize gestures. The electronic device (100) can input an image obtained through the camera (110) into the artificial intelligence model to obtain the type of gesture recognized in the image and / or a feature vector for the gesture from the artificial intelligence model. The artificial intelligence model may be implemented in the form of an on-device included in the electronic device (100). The electronic device (100) can input the image into the artificial intelligence model stored in memory (130). Alternatively, the artificial intelligence model may be stored in a server connected to the electronic device (100). If the artificial intelligence model is stored in the server, the electronic device (100) can transmit an image to the server through a communication circuit (e.g., the communication circuit (150) of FIG. 3) to obtain information about the gesture from the server.

[0082] The electronic device (100) can identify whether a gesture detected in an image acquired through the camera (110) is a trigger gesture. For example, if the electronic device (100) identifies that the type of gesture detected in the image and the type of trigger gesture are the same, it can identify that the gesture detected in the image is a trigger gesture. For example, the electronic device (100) can identify the similarity between a feature vector for a gesture detected in the image and a feature vector for a trigger gesture, and if the similarity is identified as being greater than or equal to a threshold value, it can identify that the gesture detected in the image is a trigger gesture. Methods for calculating similarity may include cosine similarity, Euclidean similarity, or dynamic time warping (DTW), but are not limited thereto.

[0083] For example, the electronic device (100) can identify that a trigger gesture has been input when a gesture in which a clenched fist is placed in front of the user's face is identified.

[0084] In one embodiment, the electronic device (100) can enter a gesture mode when a trigger gesture is identified.

[0085] Gesture mode may refer to a mode in which a user can control an electronic device (100) using gestures. When gesture mode is activated, the electronic device (100) can identify the user's gesture input based on an image acquired through a camera (110). Gesture input may refer to a gesture entered by the user into the electronic device (100) to control the electronic device (100).

[0086] For example, the electronic device (100) can acquire an image through a camera (110), identify what kind of gesture is recognized in the image, or acquire a feature vector for the gesture based on the image. Meanwhile, since the method by which the electronic device (100) identifies a gesture based on an image has been described above, a repetitive description is omitted for brevity.

[0087] Meanwhile, in the example described above, the electronic device (100) is described as entering a gesture mode based on a trigger gesture, but the present disclosure is not limited thereto. For example, when a user is recognized using a camera (110), the electronic device (100) may output a voice guidance requesting the utterance of a trigger word through a speaker, or display an interface screen requesting the utterance of a trigger word on a display (120). The trigger word may be a voice command to activate the gesture mode. The trigger word may be set at the time of manufacturing or initial use of the electronic device (100), or may be set and changed by user input. For example, when the user (1) is recognized, the electronic device (100) may output a voice signal such as "Try saying hello" through a speaker. When a user voice such as "hello" is received through an input interface (e.g., the input interface (160) of FIG. 3), the electronic device (100) may identify that a trigger word has been entered and activate the gesture mode.

[0088] In operation 410, when a gesture input is identified in an image acquired through a camera (110), the electronic device (100) can identify whether the gesture input is a gesture set for the execution of a function based on a plurality of gestures set for the execution of a plurality of functions.

[0089] In one embodiment, the memory (130) may store information about a plurality of gestures set for the execution of a plurality of functions of the electronic device (100). For example, information about each gesture may include a type of gesture and / or a feature vector for the gesture. The memory (130) may include a database containing information about a plurality of gestures. The database may store information about functions and gestures by mapping them using a key-value data structure. However, the present disclosure is not limited thereto, and the electronic device (100) may store information about functions and gestures by mapping them using various data structures.

[0090] FIG. 5 illustrates an example of a plurality of gestures set for the execution of a plurality of functions of an electronic device according to one embodiment of the present disclosure.

[0091] For example, referring to FIG. 5, a gesture of folding all fingers except the index finger and raising the index finger upward can be mapped to a volume up function. Additionally, a gesture of folding all fingers except the index finger and lowering the index finger downward can be mapped to a volume down function. A gesture of folding all fingers except the index finger and moving the hand to the right can be mapped to a function of changing the channel forward. Additionally, a gesture of folding all fingers except the index finger and moving the hand to the left can be mapped to a function of changing the channel backward. A gesture of moving the hand up or down with the thumb and the rest of the fingers touching can be mapped to a function of switching the screen. Additionally, a gesture of closing the hand from an open state can be mapped to a function of selecting a function. However, the present disclosure is not limited thereto, and various functions may be set for various gestures.

[0092] In one embodiment, the electronic device (100) may receive a gesture to be mapped to a function and store information about the received gesture by mapping it to a function. For example, the electronic device (100) may acquire an image through a camera (110) while the gesture setting mode is activated, identify a gesture in the image, acquire information about the gesture, and store information about the gesture in memory (130) by mapping it to a function. For example, the electronic device (100) may identify a gesture of closing a hand from an open state in an image acquired through the camera (110), and store the identified gesture in memory (130) by mapping it to a function selection function.

[0093] In one embodiment, the electronic device (100) can identify whether the gesture input is a gesture set for the execution of a function.

[0094] For example, if the electronic device (100) identifies that there is a gesture of the same type as the gesture input among a plurality of gestures set for the execution of a plurality of functions, the gesture input can be identified as the gesture set for the execution of a function. As another example, the electronic device (100) identifies the similarity between a feature vector for the gesture input and a feature vector for each of the plurality of gestures set for the execution of a plurality of functions, and if it identifies that there is a gesture whose similarity to the gesture input is greater than or equal to a threshold, the gesture input can be identified as the gesture set for the execution of a function.

[0095] For example, as in the example described above, it is assumed that the electronic device (100) receives a gesture of closing a hand while it is open. The electronic device (100) acquires a feature vector of the gesture and can compare the similarity between the acquired feature vector and the feature vectors for each of the multiple gestures set for the execution of multiple functions. The multiple gestures may include a gesture of closing a hand while it is open. As a result of comparing the similarity between the received gesture and the set gesture, the electronic device (100) can identify that the similarity between the received gesture and the gesture set for selecting a function among the multiple functions is greater than or equal to a threshold value. Therefore, the electronic device (100) can identify that the received gesture is a gesture set for selecting a function.

[0096] In operation 410-Y, 415, the electronic device (100) can identify whether the gesture input is a gesture embodied by the user when the gesture input is identified as a gesture set for the execution of a function.

[0097] The fact that a gesture is embodied in the user may include the user recognizing that a function is executed in the electronic device (100) according to the gesture the user has taken.

[0098] In one embodiment, the electronic device (100) can identify whether the gesture input is a gesture embodied by the user based on the number of times the gesture input is input.

[0099] The memory (130) can store information about the number of times a gesture input has been entered. For example, the memory (130) can store information about the number of times a positive response to an execution inquiry for a function corresponding to a gesture has been entered consecutively for each of the multiple gestures. Being entered consecutively may mean that no negative response was entered while positive responses were entered consecutively.

[0100] The electronic device (100) can identify the number of times a positive response to an execution inquiry for a function set in the gesture input is entered consecutively based on information stored in the memory (130), and can identify whether the gesture input is a gesture embodied by the user based on the identified number. For example, the electronic device (100) can identify that the gesture input is a gesture embodied by the user if the number of times a positive response to an execution inquiry for a function set in the gesture input is entered consecutively is greater than or equal to a preset number. Additionally, the electronic device (100) can identify that the gesture input is a gesture not embodied by the user if the number of times a positive response to an execution inquiry for a function set in the gesture input is entered consecutively is less than a preset number. The preset number may be set at the time of manufacturing or initial use of the electronic device (100), or may be set and changed by user input.

[0101] Below, we will describe the operation of the electronic device (100) storing information about the number of times a gesture input has been received in the memory (130).

[0102] In one embodiment, the electronic device (100) can identify a gesture input in an image acquired through a camera (110) and identify whether the gesture input is a gesture set for the execution of a function. If the electronic device (100) identifies that the gesture input is a gesture set for the execution of a function, it can display a first interface screen for inquiring about the execution of a function on a display (120). For example, referring to FIG. 6, it is assumed that a user's gesture of closing a hand from an open position is a gesture set for selecting a function. When the electronic device (100) receives a gesture of closing a hand from an open position, it can display a first interface screen for inquiring about the execution of a function selection according to the received gesture. The first interface screen can display text (610) such as “Would you like to select a function?”.

[0103] When the electronic device (100) receives a positive response to a request for the execution of a function set in a gesture input while a first interface screen for requesting the execution of a function is displayed on the display (130), it can update the number of positive responses corresponding to the gesture input stored in the memory (130). The update may include adding 1 to the number of positive responses corresponding to the gesture input.

[0104] For example, regarding a gesture of closing the hand from an open position, it is assumed that the number of positive responses stored in memory (130) is 1. When the electronic device (100) additionally receives a gesture of closing the hand from an open position and receives a positive response in response to a query for function execution, it can update the number of positive responses stored in memory (130) to 2. When the electronic device (100) receives a negative response in response to a query for function execution, it can update the number of positive responses stored in memory (130) to 0.

[0105] In this manner, the electronic device (100) can store information regarding the number of times a positive response to an execution inquiry for a function corresponding to a gesture is continuously entered for each of the multiple gestures. As described above, this information can be used to identify whether the gesture input is a gesture embodied by the user. Thus, even if the number of positive responses entered is high, if the positive responses are entered discontinuously, the user may not have fully embodied the gesture. Therefore, the electronic device (100) can identify whether the gesture input is a gesture embodied by the user based on the number of times a positive response is continuously entered.

[0106] In operation 410-N, 435, if the electronic device (100) does not identify that the gesture input is a gesture set for the execution of a function, it may display a second interface screen for inquiring about the function to be set for the gesture input.

[0107] The second interface screen may be an interface screen for inquiring about a function that the user wants to execute. The specific operation of the electronic device (100) displaying the second interface screen and inquiring about a function to be set for gesture input is described later in operations 430-Y and 435.

[0108] In operation 415-Y, 420, if the electronic device (100) identifies that the gesture input is a gesture embodied by the user, it can execute a function set for the gesture input among a plurality of functions.

[0109] As described above, the memory (130) can store information regarding multiple gestures for executing multiple functions. The electronic device (100) can use the information stored in the memory (130) to identify a function set for gesture input among multiple functions and execute the identified function.

[0110] For example, as described above, when the electronic device (100) receives a gesture of closing the hand while it is open, it can identify the received gesture as a gesture set for selecting a function among a plurality of functions. Accordingly, the electronic device (100) can execute the function of selecting a function.

[0111] In one embodiment, if the electronic device (100) identifies that the gesture input is a gesture embodied by the user, it may execute the function set in the gesture input without displaying a first interface screen for inquiring about the execution of the function set in the gesture input. If the user has already embodied the gesture input, it can be seen as the user's intention to execute the function they desire through the gesture. Therefore, if the electronic device (100) identifies that the gesture input is a gesture embodied by the user, it may not display a first interface screen for inquiring about the execution of the function set in the gesture input.

[0112] In operation 415-N, 425, if the electronic device (100) identifies that the gesture input is a gesture not embodied by the user, it may display a first interface screen on the display (120) to inquire about the execution of a function set for the gesture input. Specific details regarding the first interface screen are as described above.

[0113] In operation 430, the electronic device (100) can receive a response to a request to execute a function set in a gesture input while the first interface screen is displayed.

[0114] The electronic device (100) can receive gesture input from the user indicating a positive or negative.

[0115] For example, when the electronic device (100) receives a gesture input while the first interface screen is displayed, it can identify whether the received gesture is a gesture indicating a positive intention or a gesture indicating a negative intention, and obtain a response to an execution inquiry. A gesture indicating a positive or negative intention may also be expressed as a symbolic gesture.

[0116] For example, an electronic device (100) can use an artificial intelligence model to identify whether a received gesture indicates a positive or negative gesture. The artificial intelligence model may be stored in memory (130). The artificial intelligence model may be trained to identify gestures. The electronic device (100) can input a received gesture into the artificial intelligence model to identify whether the received gesture indicates a positive gesture or a negative gesture.

[0117] If the electronic device (100) identifies that the received gesture is a gesture indicating a positive, it can identify that a positive response has been received to the execution inquiry of the function set in the gesture input. On the other hand, if the electronic device (100) identifies that the received gesture is a gesture indicating a negative, it can identify that a negative response has been received to the execution inquiry of the function set in the gesture input.

[0118] For example, assume that while the first interface screen is displayed, the electronic device (100) receives a gesture input in the form of a rounded thumb and index finger touching each other. At this time, a gesture in the form of a rounded thumb and index finger touching each other can generally signify a positive response. The electronic device (100) can identify that the received gesture is a gesture indicating a positive response if the received gesture is a gesture in the form of a rounded thumb and index finger touching each other. Therefore, the electronic device (100) can identify that a positive response has been received to the inquiry regarding the execution of a function set in the gesture input.

[0119] Additionally, it is assumed that the electronic device (100) receives a gesture input in which the index fingers of both hands are spread and crossed while the first interface screen is displayed. At this time, the gesture of spreading and crossing the index fingers of both hands may generally signify a negative response. If the received gesture is a gesture of spreading and crossing the index fingers of both hands, the electronic device (100) can identify that the received gesture is a gesture indicating a negative response. Therefore, the electronic device (100) can identify that a negative response to the inquiry regarding the execution of a function set in the gesture input has been received.

[0120] Additionally, the electronic device (100) may receive voice input indicating a positive or negative response from the user. For example, if the electronic device (100) receives a voice input of “Yes” from the user while the first interface screen is displayed, it may identify that a positive response has been received to the inquiry regarding the execution of a function set in the gesture input. On the other hand, if the electronic device (100) receives a voice input of “No” from the user while the first interface screen is displayed, it may identify that a negative response has been received to the inquiry regarding the execution of a function set in the gesture input.

[0121] In operation 430-N, 420, when the electronic device (100) receives a positive response to a request to execute a function set in the gesture input, it can execute a function set in the gesture input among a plurality of functions.

[0122]

[0123] When the electronic device (100) receives a positive response to a request to execute a function set for a gesture input, it can identify a function corresponding to the gesture input using information stored in memory (130). When the electronic device (100) identifies a function corresponding to the gesture input, it can execute the identified function.

[0124] For example, the electronic device (100) can receive a positive response to a request to execute a volume up function in response to a gesture of moving an index finger upward. When the electronic device (100) receives a positive response to a request to execute a volume up function, it can execute a volume up function.

[0125] In operation 430-Y, 435, when the electronic device (100) receives a negative response, it may display a second interface screen on the display (120) to inquire about a function to be set for gesture input. The second interface screen may be an interface screen for inquiring about a function that the user wants to execute, as described above.

[0126] For example, referring to FIG. 7, there may be cases where a user inputs a gesture to increase the volume of the electronic device (100), but the gesture set in the electronic device (100) for the execution of the volume up function is different from the gesture taken by the user. For example, the gesture (710) set in the volume up function is a gesture of moving the index finger upward, but the user may input a gesture input (720) of moving the index finger to the right to the electronic device (100). In this case, if the function set in response to the gesture input of moving the index finger to the right is a function of changing the channel forward, the electronic device (100) may display a first interface screen on the display (110). The first interface screen may include text such as “Would you like to change the channel forward?”. Since the function the user wants is the function of executing volume up, the user may input a negative response to the inquiry such as “Would you like to change the channel forward?” to the electronic device (100). When the electronic device (100) receives a negative response such as “No,” it may display a second interface screen as shown in FIG. 8. The second interface screen may include text (810) such as “What function do you want to execute?”

[0127] In operation 440, the electronic device (100) can receive user input for a function to be set to gesture input.

[0128] For example, the electronic device (100) can receive a response to an inquiry about a function that the user wants to execute while the second interface screen is displayed. The electronic device (100) can receive a response to an inquiry about a function that the user wants to execute as voice input. Additionally, the electronic device (100) can receive a response to an inquiry about a function that the user wants to execute in the form of text entered through an input interface such as a keyboard or a remote control. As an example, the electronic device (100) can receive a response such as “Volume Up” while the second interface screen is displayed.

[0129] In operation 445, the electronic device (100) may display a third interface screen containing information related to a function to be set for gesture input on the display (120).

[0130] When the electronic device (100) receives a response to an inquiry about a function that the user wants to execute, it can display a third interface on the display (120) for resetting a gesture corresponding to the function that the user wants to execute.

[0131] The electronic device (100) may display a phrase (1010) requesting a user to input a gesture to reset the gesture on the third interface screen (1000). For example, referring to FIG. 10, the electronic device (100) may display text (1010) such as “Please perform the desired gesture” on the third interface screen (1000).

[0132] The third interface screen (1000) may include information related to the function set for gesture input.

[0133] The third interface screen (1000) may include gesture information (1020) set in correspondence with the function that the user wants to execute.

[0134] The electronic device (100) can identify a gesture corresponding to a function that the user wants to execute from data stored in memory (130). The electronic device (100) can identify the identified gesture as a pre-set gesture. The electronic device (100) can display the pre-set gesture on a third interface screen.

[0135] For example, referring to FIG. 10, the electronic device (100) can display a gesture (1020) that the user has already set to perform a volume up on a third interface screen (1000).

[0136] The electronic device (100) can display the gesture (1030) actually entered by the user on the third interface screen (1000).

[0137] The third interface screen (1000) may include input gesture information. When the third interface (1000) for resetting the gesture is displayed on the display (120), the electronic device (100) may display the input gesture (1030) on the third interface (1000).

[0138] For example, referring to FIG. 10, the electronic device (100) can display the gesture input (1030) actually received to execute the volume up function on the third interface screen (1000).

[0139] The third interface screen (1000) may include gesture information set by multiple users for a function to be set for gesture input.

[0140] Referring to FIG. 9, the electronic device (100) communicates with the server (200) to receive information regarding gestures set by multiple users and functions set by multiple users for gesture input. Specifically, the server (200) can acquire information about multiple users using gesture functions and store it in a database. The information about multiple users may include the model of the device used by the user, the nationality of the user, a list of functions that the user can control the device through gestures, and gestures used by the user for each function. The server (200) can use an artificial intelligence model to classify the gestures used by multiple users into symbolic gestures (e.g., gestures indicating affirmation and gestures indicating negation) and action gestures for performing functions. The server (200) can store the symbolic gestures and action gestures used by multiple users in a database.

[0141] For example, the server (200) can distinguish between symbolic gestures and motion gestures by device model and store them in a database. Additionally, the server (200) can distinguish between symbolic gestures and motion gestures by country and store them in a database.

[0142] The electronic device (100) may request the server (200) to transmit information regarding gestures set by multiple users for a function to be set for gesture input among the information stored in the database. The electronic device (100) may receive the information transmitted by the server (200) based on the request of the electronic device (100).

[0143] The electronic device (100) can receive information from the server (200) regarding the motion gestures of a user who has the same nationality as the user of the electronic device (100) and uses a device of the same model name. The information regarding the motion gestures of a user who uses a device of the same model name may include information regarding the gestures used by multiple users for a function that the user wishes to execute.

[0144] For example, referring to FIG. 10, the electronic device (100) can display a gesture (1040) used by multiple users to execute a volume up function on a third interface screen (1000) based on information received from a server (200).

[0145] By displaying another user's gesture on a third interface screen, the electronic device (100) can set a gesture by referring to the gesture used by another user.

[0146] The third interface screen (1000) may include functions set by multiple users for gesture input.

[0147] The electronic device (100) can communicate with the server (200) to receive functions set by multiple users for gesture input. Specifically, as in the example described above, the electronic device (100) can receive information from the server (200) regarding the motion gestures of users who have the same nationality as the user of the electronic device (100) and use a device of the same model name. The information regarding the motion gestures of users who use a device of the same model name may include information regarding functions used by multiple users for the gestures input by the user.

[0148] For example, referring to FIG. 10, the electronic device (100) may display information (1050) on a third interface screen (1000) that, in relation to a gesture (1030) input by a user, a plurality of users use the gesture (1030) input by the user primarily to “change channels forward” or “a function to move forward a scene of streaming content being played.”

[0149] In operation 450, the electronic device (100) can receive user input for a new gesture.

[0150] Specifically, the electronic device (100) can receive user input for a new gesture while the third interface screen is displayed. The method by which the electronic device (100) receives the gesture input is the same as the method described in operation 405.

[0151] In operation 455, the electronic device (100) can store a new gesture in memory (130).

[0152] When the electronic device (100) receives a gesture input, it can store the received gesture input in memory (130) by mapping it to a function that the user wants to execute. For example, when the electronic device (100) receives a gesture input of raising the hand from bottom to top with the index and middle fingers extended while the third interface (1000) is displayed on the display (120), it can update the gesture for performing the volume up function to a gesture of raising the hand from bottom to top with the index and middle fingers extended.

[0153] By the aforementioned operation, the electronic device (100) identifies whether the received gesture input has been embodied by the user, and even if the user forgets the gesture set while using the electronic device, the gesture can be easily reset.

[0154] If the electronic device (100) identifies that the received gesture input is similar to another gesture that has already been set, it can identify the received gesture and the other gesture that has already been set as a group and store them in memory (130).

[0155] For example, referring to FIG. 9, the electronic device (100) can compare the similarity of gestures stored in memory (130) and identify gestures whose similarity is greater than or equal to a preset value as a group. The specific method for comparing the similarity of gestures is the same as described in operation 405. For each gesture belonging to a group, the electronic device (100) can store the number of times a function corresponding to the gesture is executed in memory (130).

[0156] The electronic device (100) can identify that the most frequently used gesture input among the gestures in the group has been received if the received gesture input is similar to a gesture that has already been set.

[0157] When the electronic device (100) receives input for a reset gesture, it can identify the number of times a positive response is received consecutively among the user's responses to a query for the execution of a function corresponding to the gesture. Based on the number of times a positive response is received consecutively, the electronic device (100) can identify whether the reset gesture has been embodied by the user.

[0158] The electronic device (100) can store in memory (130) the number of times a positive response to an inquiry to execute a function corresponding to a gesture is received from the time the input of the reset gesture is received after a preset time has elapsed since the gesture was reset. For example, the preset time may be 24 hours. Accordingly, the electronic device (100) can store in memory (130) the number of times a user's positive response to an inquiry to execute a function of the reset gesture is received after 24 hours have elapsed since the gesture was reset.

[0159] The method for identifying whether a gesture input received by the electronic device (100) is embodied by the user is as described above in operations 405 to 415. Additionally, referring to FIG. 9, the electronic device (100) can identify whether the gesture is embodied by the user and store it in memory (130).

[0160] Additionally, as described above, the electronic device (100) can identify whether a gesture has been embodied by the user based on the number of consecutive positive responses received among the user's responses to a query for the execution of a function corresponding to the gesture. Specifically, the electronic device (100) can identify that the input gesture has been embodied by the user if positive responses are received more than a preset number of times among the user's responses to a query for the execution of a function corresponding to the input gesture. The electronic device (100) may also adjust the preset number of times to identify whether the gesture input is a gesture embodied by the user.

[0161] For example, the electronic device (100) may identify a different number of preset times based on the similarity between a gesture already set for the same function and a gesture preferred by another user. The gesture preferred by another user may be the gesture that is set most frequently among the gestures set by another user for the function set for gesture input.

[0162] Specifically, the electronic device (100) may request the server (200) to transmit gesture information set by another user. The electronic device (100) may receive gesture information set by another user from the server (200). The electronic device (100) may identify the gesture set most frequently among the gestures set by another user for the function set for gesture input as the gesture preferred by the other user.

[0163] The electronic device (100) can identify the similarity between gestures preferred by other users for gesture input and functions set in gesture input based on received information.

[0164] Additionally, the electronic device (100) can identify a different preset number of times to identify whether the gesture input is a gesture embodied by the user based on the identified similarity.

[0165] The method by which the electronic device (100) identifies the similarity of gestures is as described above in operation 405.

[0166] The electronic device (100) can reduce the number of times it identifies whether a gesture input is a gesture embodied by the user if the identified similarity is greater than or equal to a preset threshold.

[0167] Specifically, the electronic device (100) can identify the similarity between a gesture set by a user and a gesture preferred by another user. If the identified similarity is greater than or equal to a preset threshold, the electronic device (100) can identify that the gesture set by the user is similar to the gesture preferred by another user. If the gesture set by the user is similar to the gesture preferred by another user that corresponds to the function of the gesture set by the user, the electronic device (100) can identify that the gesture set by the user is a general gesture.

[0168] For example, referring to FIG. 11, it is assumed that a user has set a gesture for the volume up function in which all fingers except the index finger are folded and the index finger is raised. The electronic device (100) can identify that the similarity between the gesture set by the user and the other user's preferred gesture is greater than or equal to a preset threshold value if the other user's preferred gesture for the volume up function is also the gesture of folding all fingers except the index finger and raising the index finger. The electronic device (100) can identify that the gesture set by the user in response to the volume up function is similar to the other user's preferred gesture. Therefore, the electronic device (100) can identify that the gesture set by the user in response to the volume up function is a general gesture.

[0169] The electronic device (100) can reduce the preset number of times for identifying whether the gesture input is a gesture embodied by the user if the gesture set by the user is a general gesture. That is, the electronic device (100) can identify the preset number of times for identifying whether the gesture input is a gesture embodied by the user as being smaller than the initial value of the preset number of times if the gesture set by the user is a general gesture.

[0170] For example, referring to FIG. 11, the electronic device (100) assumes that for the volume up function, the initial value of the preset number of times for identifying whether the gesture input is a gesture embodied by the user is 4 times. Since the gesture set by the user in response to the volume up function is a general gesture, the electronic device (100) can identify that the preset number of times for identifying whether the gesture input is a gesture embodied by the user is 2 times.

[0171] If the identified similarity is less than a preset threshold value, the electronic device (100) may increase the preset number of times to identify whether the gesture input is a gesture embodied by the user.

[0172] The electronic device (100) can identify that the preferred gesture of another user and the gesture set by the user are dissimilar if the identified similarity is less than a preset threshold. The electronic device (100) can identify that the gesture set by the user is a special gesture if the gesture set by the user and the preferred gesture of another user corresponding to the function of the gesture set by the user are dissimilar.

[0173] For example, referring to FIG. 11, it is assumed that the user has set a gesture for the screen switching function in which all fingers except the index finger are folded and the index finger is moved in a circular motion from left to right. The electronic device (100) can identify that the similarity between the gesture set by the user and the preferred gesture of another user is less than a preset threshold value if the preferred gesture of another user for the screen switching function is a gesture of moving the hand upward with the thumb and the rest of the fingers touching. The electronic device (100) can identify that the gesture set by the user in response to the screen switching function is dissimilar to the preferred gesture of another user. Therefore, the electronic device (100) can identify that the gesture set by the user in response to the screen switching function is a special gesture.

[0174] The electronic device (100) can increase the preset number of times for identifying whether the gesture input is a gesture embodied by the user if the gesture set by the user is a special gesture. That is, the electronic device (100) can identify the preset number of times for identifying whether the gesture input is a gesture embodied by the user as being greater than the initial value of the preset number of times if the gesture set by the user is a special gesture.

[0175] For example, referring to FIG. 11, the electronic device (100) assumes that for a screen switching function, the initial value of the preset number of times for identifying whether a gesture input is a gesture embodied by the user is 4 times. The electronic device (100) can identify that the preset number of times for identifying whether a gesture input is a gesture embodied by the user is 6 times because the gesture set by the user in response to the screen switching function is a special gesture.

[0176] If the electronic device (100) has a ratio of general gestures among the gestures set by the user corresponding to multiple functions that is greater than the ratio of special gestures, the preset number of times for each of the multiple gestures to identify whether the gesture input is a gesture embodied by the user can be adjusted significantly compared to the initial value.

[0177] On the other hand, if the electronic device (100) has a ratio of general gestures among the gestures set by the user corresponding to multiple functions that is smaller than the ratio of special gestures, the preset number of times for each of the multiple gestures to identify whether the gesture input is a gesture embodied by the user can be adjusted to a smaller range than the initial value.

[0178] For example, assume the initial value is 4 times.

[0179] Referring to FIG. 11, the electronic device (100) can identify that the proportion of general gestures among the gestures set by the user is greater than the proportion of special gestures. The electronic device (100) can identify that gestures corresponding to the volume up function, the channel forward change function, and the selection function are general gestures and that the preset number of times is 2. Additionally, the electronic device (100) can identify that gestures corresponding to the screen switching function are special gestures and that the preset number of times is 6. The electronic device (100) can identify that the preset number of times for the set gestures is ±2 times compared to the initial value.

[0180] On the other hand, referring to FIG. 12, the electronic device (100) can identify that the proportion of special gestures among the gestures set by the user is greater than the proportion of general gestures. The electronic device (100) can identify gestures for the volume up function, the channel forward change function, and the screen switching function as special gestures, and identify that the preset number of times is 5.

[0181] Additionally, the electronic device (100) can identify a gesture corresponding to a function selection function as a general gesture and identify that the preset number of times is 3. The electronic device (100) can identify that the preset number of times for the set gestures is ±1 time compared to the initial value.

[0182] For example, the electronic device (100) may identify the same number of times for all set gestures based on the similarity between a gesture already set in correspondence with the same function and a gesture preferred by another user.

[0183] Specifically, the electronic device (100) can compare the similarity between a gesture already set in correspondence with the same function and a gesture preferred by another user. The electronic device (100) can identify each gesture as a general gesture or a special gesture based on the similarity. This has been described above. If the number of general gestures among the set gestures is greater than the number of special gestures, the electronic device (100) can identify the user as a general user.

[0184] For example, referring to FIG. 13, the electronic device (100) can identify gestures corresponding to a volume up function, a channel forward change function, and a function selection function as general gestures, and gestures corresponding to a screen switching function as special gestures. Since the number of general gestures is greater than the number of special gestures, the electronic device (100) can identify the user as a general user.

[0185] On the other hand, the electronic device (100) can identify the user as a special user if the number of special gestures among the set gestures is greater than the number of general gestures.

[0186] For example, referring to FIG. 14, the electronic device (100) can identify a gesture corresponding to a volume up function, a channel forward change function, and a screen switching function as a special gesture, and identify a gesture corresponding to a function selection function as a general gesture. Since the number of special gestures is greater than the number of general gestures, the electronic device (100) can identify the user as a special user.

[0187] When the electronic device (100) identifies the user as a general user, it can identify the preset number of times as being less than the initial value.

[0188] Additionally, if the electronic device (100) identifies that the user is a special user, it can identify that the preset number of times is greater than the initial value.

[0189] For example, assume that the initial value of the preset number of times is 4 times. Referring to FIG. 13, if the electronic device (100) identifies that the user is a general user, it can identify that the preset number of times is 2 times.

[0190] On the other hand, referring to FIG. 14, if the electronic device (100) identifies that the user is a special user, it can identify that the preset number of times is 5 times.

[0191] In addition, as an example, the electronic device (100) may identify a preset number of times based on whether the first received gesture input after the gesture is set matches the set gesture. The first received gesture after the gesture is set may include the first received gesture after a first preset time has elapsed after the gesture is set.

[0192] Specifically, the electronic device (100) can set a gesture in response to a specific function and identify whether the first received gesture after a first preset time has elapsed matches the set gesture. The method by which the electronic device (100) determines whether the first received gesture input after a first preset time has elapsed matches the set gesture is the same as the similarity determination method described above.

[0193] The electronic device (100) can identify the ratio of positive responses from the user to the inquiry for function execution from the time the gesture input is first received until a second preset time has elapsed. Specifically, the electronic device (100) can display the inquiry for function execution to the user on the display (120) when the gesture input is received again after the gesture input is first received. The electronic device (100) can receive the response to the inquiry for function execution. The electronic device (100) can identify the ratio of positive responses from the user to the second preset time after the gesture input is first received.

[0194] The electronic device (100) can identify that the set gesture is a gesture with a high matching rate if the ratio of positive responses is greater than or equal to a preset ratio. On the other hand, the electronic device (100) can identify that the set gesture is a gesture with a low matching rate if the ratio of positive responses is less than a preset ratio.

[0195] If the electronic device (100) identifies that the first received gesture input matches the set gesture and is a gesture with a high matching rate, it can adjust the preset number of times to be smaller than the initial value by a first number of times.

[0196] If the electronic device (100) identifies that the first received gesture input matches a set gesture and is a gesture with a low matching rate, it can adjust the preset number of times to be higher than the initial value by a second number of times. The first number of times may be a value greater than the second number of times.

[0197] Additionally, if the electronic device (100) identifies that the first received gesture input is inconsistent with the set gesture but is a gesture with a high matching rate, it can identify the preset number as being smaller by a second number compared to the initial value.

[0198] Additionally, if the electronic device (100) identifies that the first received gesture input is inconsistent with the set gesture and is a gesture with a low matching rate, it can identify that the preset number is greater than the initial value by a first number.

[0199] Meanwhile, various embodiments of the present disclosure may be implemented in a recording medium readable by a computer or a similar device using software, hardware, or a combination thereof. In some cases, the embodiments described herein may be implemented as the processor itself. According to software implementation, embodiments such as the procedures and functions described herein may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein.

[0200] Meanwhile, computer instructions for performing processing operations of an electronic device (100) according to various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium. When computer instructions stored in such a non-transitory computer-readable medium are executed by a processor of a specific device, they cause the specific device to perform processing operations in a washer / dryer (100) according to various embodiments described above.

[0201] A non-transient computer-readable medium refers to a medium that stores data semi-permanently and can be read by a device, unlike media that store data for a short period of time such as registers, caches, and memory. Specific examples of non-transient computer-readable media include CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, and ROMs.

[0202] Although the present invention has been described above with reference to the attached drawings, the scope of the present invention is determined by the claims set forth below and should not be interpreted as being limited to the aforementioned embodiments and / or drawings. Furthermore, it should be clearly understood that improvements, changes, and modifications to the invention described in the claims that are obvious to those skilled in the art are also included within the scope of the present invention.

Claims

In electronic devices, camera; display; Memory for storing instructions; and at least one processor including processing circuitry; and When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Identifying user gesture input based on images acquired through the above camera, and Based on a plurality of gestures set for the execution of a plurality of functions of the electronic device, identifying whether the gesture input is a gesture set for the execution of a function, and If the above gesture input is identified as a gesture set for the execution of a function, it is determined whether the gesture input is a gesture embodied by the user based on the number of times the gesture input has been entered, and If the above gesture input is identified as a gesture embodied by the user, the function set in the gesture input among the plurality of functions is executed, and An electronic device that, when the above-mentioned identified gesture is identified as a gesture not embodied by the user, displays a first interface screen on the display for inquiring about the execution of a function set in the gesture input. In paragraph 1, The above memory stores information regarding the number of times a positive response to an execution inquiry for a function corresponding to a gesture is continuously entered for each of the plurality of gestures, and When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Identify the number of times a positive response to an inquiry regarding the execution of a function set in the gesture input is continuously entered based on the information stored in the memory above, and An electronic device that identifies whether the gesture input is a gesture embodied by the user based on the number of times identified above. In paragraph 2, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, If the number of consecutive positive responses to a query for the execution of a function set in the above gesture input is greater than or equal to a preset number, the above gesture input is identified as a gesture embodied by the user, and An electronic device that identifies the gesture input as a gesture not embodied by the user if the number of consecutive positive responses to a query for the execution of a function set in the gesture input is less than the number of preset times. In paragraph 2, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, An electronic device that updates the number of positive responses corresponding to the gesture input stored in the memory when a positive response is received to a request to execute a function set in the gesture input while the first interface screen is displayed. In paragraph 1, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, When a negative response is received to an inquiry regarding the execution of a function set in the gesture input while the first interface screen is displayed, a second interface screen for inquiring about a function to be set in the gesture input is displayed on the display, and An electronic device that, when user input regarding a function to be set for the gesture input is received, displays a third interface screen containing information related to the function to be set for the gesture input on the display. In paragraph 5, The above third interface screen is, An electronic device comprising at least one of a gesture already set on the electronic device for a function to be set on the gesture input, a gesture set by another user for a function to be set on the gesture input, the gesture input, and a function set by another user for the gesture input. In paragraph 3, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, Identifying the similarity between the gesture input and the gesture set by another user for the function set in the gesture input, and An electronic device that adjusts the preset number of times to identify whether the gesture input is a gesture embodied by the user based on the similarity identified above. In Paragraph 7, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, If the identified similarity is greater than or equal to a preset threshold, the preset number of times for identifying whether the gesture input is a gesture embodied by the user is reduced, and An electronic device that, if the identified similarity is less than a preset threshold, increases the preset number of times to identify whether the gesture input is a gesture embodied by the user. In paragraph 1, When the above instructions are executed individually or collectively by the at least one processor, the electronic device, If the above gesture input is identified as not being a gesture set for the execution of a function, a second interface screen for inquiring about a function to be set for the above gesture input is displayed on the display, and An electronic device that, when user input regarding a function to be set for the gesture input is received, displays a third interface screen containing information related to the function to be set for the gesture input on the display. In a method for controlling an electronic device, A step of identifying user gesture input based on an image acquired through a camera; A step of identifying whether the gesture input is a gesture set for the execution of a function based on a plurality of gestures set for the execution of a plurality of functions of the electronic device; If the above gesture input is identified as a gesture set for the execution of a function, a step of identifying whether the gesture input is a gesture embodied by the user based on the number of times the gesture input has been input; If the gesture input is identified as a gesture embodied by the user, the step of executing a function set in the gesture input among the plurality of functions; and A control method comprising: a step of displaying a first interface screen on a display for inquiring about the execution of a function set in the gesture input when the identified gesture is identified as a gesture not embodied by the user. In Paragraph 10, The step of identifying whether the above-mentioned embodied gesture is, A step of identifying the number of times a positive response to an inquiry regarding the execution of a function set in the gesture input is continuously entered based on stored information; and A control method comprising the step of identifying whether the gesture input is a gesture embodied by the user based on the number of times identified above. In Paragraph 11, A step of identifying the gesture input as a gesture embodied by the user if the number of consecutive positive responses to a query for the execution of a function set in the gesture input is greater than or equal to a preset number; and A control method further comprising the step of identifying the gesture input as a gesture not embodied by the user if the number of consecutive positive responses to a query for the execution of a function set in the gesture input is less than the preset number. In Paragraph 11, A control method further comprising: a step of updating the number of positive responses corresponding to the stored gesture input when a positive response is received to a request to execute a function set in the gesture input while the first interface screen is displayed. In Paragraph 10, When a negative response to an inquiry regarding the execution of a function set in the gesture input is received while the first interface screen is displayed, a step of displaying a second interface screen on the display for inquiring about a function to be set in the gesture input; and A control method further comprising the step of displaying a third interface screen containing information related to the function to be set for the gesture input on the display when user input regarding the function to be set for the gesture input is received. In Paragraph 14, The above third interface screen is, A control method comprising at least one of a gesture already set in the electronic device for a function to be set in the gesture input, a gesture set by another user for a function to be set in the gesture input, the gesture input, and a function set by another user for the gesture input.

Images