Recognition method and recognition device for gestures and electronic equipment

Abstract

The invention relates to the technical field of gesture recognition, and discloses a recognition method and a recognition device for gestures, and electronic equipment. The recognition method comprises the steps of determining a first center coordinate of a hand in a starting action image under the condition that a plurality of infrared images detected continuously for a first set time length are starting action images; under the condition that the change rate of the first center coordinate of the hand in the multiple starting action images is smaller than a first set threshold value, multiple infrared images within a subsequent second set duration are continuously obtained; under the condition that the proportion of the judgment action image in the plurality of infrared images is greater than a first set proportion, determining a second center coordinate of the hand in the judgment action image; and determining the gesture action of the user according to the second center coordinates of the hand in the plurality of action judgment images. According to the method, the gesture action recognition efficiency and accuracy can be improved.

Description

Technical Field

[0001] This application relates to the field of gesture recognition technology, such as a gesture recognition method and device, and an electronic device. Background Technology

[0002] Currently, gesture recognition technology is considered an important component of human-computer interaction and is widely used in electronic devices in various fields such as smart homes and automotive electronics.

[0003] In related technologies, a common approach is to set up a high-resolution camera on an electronic device, and then use the high-resolution camera to capture and process subtle changes in the hand image to recognize the user's gestures.

[0004] In the process of implementing the embodiments of this disclosure, at least the following problems were found in the related art:

[0005] High-resolution cameras place high demands on both ambient lighting and the computing power of electronic devices. Poor ambient lighting leads to lower accuracy in recognizing user gestures. Similarly, low computing power or insufficient computing resources on the electronic device, or its being consumed by other applications, results in low efficiency in recognizing user gestures. Therefore, developing an efficient and accurate gesture recognition method has become a pressing technical challenge.

[0006] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0007] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0008] This disclosure provides a method, device, and electronic device for gesture recognition, which can improve the efficiency and accuracy of gesture recognition.

[0009] In some embodiments, a gesture recognition method includes: determining a first center coordinate of a hand in a startup action image when multiple infrared images detected for a first predetermined duration are all startup action images; continuously acquiring multiple infrared images within a subsequent second predetermined duration when the rate of change of the first center coordinate of the hand in the multiple startup action images is less than a first predetermined threshold; determining a second center coordinate of the hand in a judgment action image when the proportion of judgment action images in the multiple infrared images is greater than a first predetermined proportion; and determining a user's gesture based on the second center coordinate of the hand in the multiple judgment action images.

[0010] Optionally, the infrared image is confirmed as a start-up action image by determining the third center coordinates of the hand in the infrared image; if the hand in the infrared image is a first set gesture and the third center coordinates are located in a set area of ​​the infrared image, the first infrared image is confirmed as a start-up action image.

[0011] Optionally, the infrared image is confirmed as a judgment action image in the following manner: determine the fourth center coordinate of the hand in the infrared image; if the hand in the infrared image is a second set gesture and the fourth center coordinate is not located in the set area of ​​the infrared image, the infrared image is confirmed as a judgment action image.

[0012] Optionally, if the infrared image includes multiple hands, only the hand with the highest detection score is identified.

[0013] Optionally, when the infrared image includes multiple hands, the detection score of each hand in the infrared image is determined as follows: the number of pixels and the contour area of ​​each hand in the infrared image are determined; the product of the number of pixels of the current hand and a first weighting coefficient is used as the first score of the current hand, and the product of the contour area of ​​the current hand and a second weighting coefficient is used as the second score of the current hand; the sum of the first score and the second score of the current hand is used as the detection score of the current hand.

[0014] Optionally, the rate of change of the first center coordinates includes a horizontal rate of change and a vertical rate of change; the rate of change of the first center coordinates of the hand in the multiple start-up action images is determined as follows: the start image and the end image in the multiple start-up action images are determined; the ratio of the difference between the horizontal coordinates of the first center coordinates of the hand in the end image and the start image to a first set duration is calculated as the horizontal rate of change; the ratio of the difference between the vertical coordinates of the first center coordinates of the hand in the end image and the start image to a first set duration is calculated as the vertical rate of change.

[0015] Optionally, the user's gesture is determined based on the second center coordinates of the hand in multiple action images, including: if the hand in the action image is a first-category activation action, and the proportion of the horizontal coordinate change values ​​of two adjacent action images being greater than zero is greater than a second preset proportion, then the user's gesture is determined to be a right wave; if the hand in the action image is a first-category activation action, and the proportion of the horizontal coordinate change values ​​of two adjacent action images being less than zero is greater than a second preset proportion, then the user's gesture is determined to be a left wave; if the hand in the action image is a second-category activation action, and the proportion of the vertical coordinate change values ​​of two adjacent action images being greater than zero is greater than a second preset proportion, then the user's gesture is determined to be an upward wave; if the hand in the action image is a second-category activation action, and the proportion of the vertical coordinate change values ​​of two adjacent action images being less than zero is greater than a second preset proportion, then the user's gesture is determined to be a downward wave.

[0016] In some embodiments, a gesture recognition device includes: a first detection module configured to determine a first center coordinate of a hand in a startup action image when multiple infrared images detected for a first predetermined duration are all startup action images; an acquisition module configured to continuously acquire multiple infrared images within a subsequent second predetermined duration when the rate of change of the first center coordinate of the hand in the multiple startup action images is less than a first predetermined threshold; a second detection module configured to determine a second center coordinate of a hand in a judgment action image when the proportion of judgment action images in the multiple infrared images is greater than a first predetermined proportion; and a determination module configured to determine a user's gesture based on the second center coordinate of the hand in the multiple judgment action images.

[0017] In some embodiments, a gesture recognition device includes a processor and a memory storing program instructions, the processor being configured to execute the gesture recognition method as described above when the program instructions are executed.

[0018] In some embodiments, the electronic device includes: a device body; an infrared sensor disposed on the device body for detecting infrared images of a user's hand; and a gesture recognition device as described above disposed on the device body and electrically connected to the infrared sensor.

[0019] The gesture recognition method, recognition device, and electronic device provided in this disclosure can achieve the following technical effects:

[0020] In this embodiment, user gestures are recognized using infrared images detected by an infrared sensor mounted on an electronic device. Compared to methods that use high-resolution cameras to detect hand images, this approach has lower requirements for ambient light and the computing power of the electronic device. Even in poor lighting conditions or when the electronic device's computing power is limited, it can quickly and accurately recognize user gestures. In this embodiment, when recognizing user gestures based on infrared images, it is first necessary to determine whether the infrared image is a starting action image. If multiple infrared images detected over a first set duration are all starting action images, and the rate of change of the first center coordinate of the hand in these multiple starting action images is less than a first set threshold, then multiple infrared images within a second set duration will be acquired. Next, the proportion of the judgment action image in the multiple infrared images is confirmed. If the proportion of the judgment action image in the multiple infrared images is greater than a first set proportion, then the user's specific gesture is recognized based on the second center coordinate of the hand in the judgment action image. This reduces the risk of misjudgment and missed detection when recognizing gestures. Therefore, this embodiment can improve the efficiency and accuracy of gesture recognition.

[0021] The above general description and the description below are exemplary and illustrative only and are not intended to limit this application. Attached Figure Description

[0022] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are shown as similar elements. The drawings are not to be scaled. And wherein:

[0023] Figure 1 This is a schematic diagram of an electronic device provided in an embodiment of this disclosure;

[0024] Figure 2 This is a schematic diagram of a gesture recognition method provided in an embodiment of this disclosure;

[0025] Figure 3 This is a schematic diagram of a startup action image provided in an embodiment of this disclosure;

[0026] Figure 4 This is a schematic diagram of a judgment action image provided in an embodiment of this disclosure;

[0027] Figure 5 This is a schematic diagram of a gesture recognition device provided in an embodiment of this disclosure;

[0028] Figure 6 This is a schematic diagram of another gesture recognition device provided in an embodiment of this disclosure. Detailed Implementation

[0029] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0030] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0031] Unless otherwise stated, the term "multiple" means two or more.

[0032] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0033] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0034] The term "correspondence" can refer to an association or binding relationship. The correspondence between A and B means that there is an association or binding relationship between A and B.

[0035] It should be noted that, unless otherwise specified, the embodiments and features described in the present disclosure can be combined with each other.

[0036] In this embodiment of the disclosure, the electronic device may be a home appliance, a terminal, or an in-vehicle computer, but is not limited thereto.

[0037] For example, the electronic device can be an air conditioner. By setting a gesture recognition device on the air conditioner, users can adjust the air conditioner's fan speed, operating mode, etc., by gesture. The following describes the solution proposed in the embodiments of this disclosure by taking an electronic device as an air conditioner.

[0038] like Figure 1 As shown, the electronic device 100 includes a device body 110, an infrared sensor 120, and a gesture recognition device 500 (600).

[0039] Specifically, the infrared sensor 120 and the gesture recognition device 500 (600) are disposed on the device body 110, and the infrared sensor 120 and the gesture recognition device 500 (600) are electrically connected.

[0040] Optionally, the gesture recognition device 600 includes a processor that can identify whether a hand movement in an infrared image detected by an infrared sensor is a starting action or a judgment action, and confirm whether the infrared image is a starting action image or a judgment action image. If multiple infrared images detected by the processor for a continuous first set duration are all starting action images, and the rate of change of the first center coordinate of the hand in the multiple starting action images is less than a first set threshold, multiple infrared images within a subsequent second set duration can be acquired. If the proportion of judgment action images in the multiple infrared images is greater than a first set proportion, the processor can recognize the user's gesture based on the second center coordinate of the hand in the judgment action image.

[0041] In conjunction with the aforementioned electronic devices, embodiments of this disclosure provide a method for gesture recognition, such as... Figure 2 As shown, the identification method includes:

[0042] S201, if the processor detects multiple infrared images for a first set duration and all of them are startup action images, it determines the first center coordinates of the hand in the startup action image.

[0043] Specifically, a startup action image refers to an image in the detected infrared image in which the user's hand makes a startup gesture, and the center coordinates of the hand are located within a set area in the infrared image.

[0044] Specifically, if multiple infrared images detected over a first set duration are all startup action images, it indicates that the user is highly likely to make a gesture to control the electronic device to perform a certain function. Based on the first center coordinates of the hand in the startup action image, it can be further determined whether the user intends to make a gesture to control the electronic device to perform a certain function. Therefore, in this case, the processor needs to determine the first center coordinates of the hand in the startup action image.

[0045] Optionally, the first set duration is 0.5s to 1.5s.

[0046] S202, if the rate of change of the first center coordinate of the hand in multiple startup action images is less than a first set threshold, the processor continuously acquires multiple infrared images within a subsequent second set time period.

[0047] Specifically, if the hand makes large movements in multiple startup action images, it indicates that the user is not about to perform a gesture to control the electronic device to perform a specific function. Conversely, if the rate of change of the first center coordinate of the hand in multiple startup action images is less than a first preset threshold, it indicates that the hand does not make large movements in the multiple startup action images. Therefore, in this case, it can be determined that the user intends to perform a gesture to control the electronic device to perform a specific function. Thus, the processor needs to continuously acquire multiple infrared images within a subsequent second preset time period.

[0048] Optionally, the second set duration is 2 to 4 seconds.

[0049] Optionally, the rate of change of the first center coordinates includes a horizontal rate of change and a vertical rate of change; the rate of change of the first center coordinates of the hand in the multiple start-up action images is determined as follows: the start image and the end image in the multiple start-up action images are determined; the ratio of the difference between the horizontal coordinates of the first center coordinates of the hand in the end image and the start image to a first set duration is calculated as the horizontal rate of change; the ratio of the difference between the vertical coordinates of the first center coordinates of the hand in the end image and the start image to a first set duration is calculated as the vertical rate of change.

[0050] Specifically, taking the first center coordinates of the hand in the starting image as (x1, y1) and the first center coordinates of the hand in the ending image as (x2, y2) as an example, the horizontal and vertical rates of change can be calculated respectively according to the following formulas.

[0051]

[0052] Among them, P x P represents the rate of change at the horizontal level. y T represents the vertical rate of change, and T represents the first set duration.

[0053] S203, if the proportion of the action image in the multiple infrared images is greater than the first set proportion, the processor determines the second center coordinates of the hand in the action image.

[0054] Specifically, a judgment action image refers to an infrared image in which the user's hand displays a judgment action gesture, and the center coordinates of the hand are not located within a defined area in the infrared image. The judgment action gesture is a gesture used to control an electronic device to perform a certain function.

[0055] Specifically, if the proportion of action images among the multiple infrared images acquired within the second set time period is greater than the first set proportion, it indicates that most of the user's hand movements within the second set time period are gestures controlling the electronic device. Therefore, in this case, the processor needs to further determine the second center coordinates of the hand in the action images to further determine what the specific gesture is.

[0056] Optionally, the first set ratio is greater than or equal to 80%.

[0057] S204, the processor determines the user's gesture based on the second center coordinates of the hand in multiple motion images.

[0058] Specifically, by using the second center coordinates of the hand in multiple motion images, the trend of the user's hand movements can be determined, and based on this trend, the trajectory of the user's hand movement can be identified. Therefore, the processor can determine the user's hand gesture based on the second center coordinates of the hand in multiple motion images.

[0059] Optionally, the user's gesture is determined based on the second center coordinates of the hand in multiple action images, including: if the hand in the action image is a first type of start action, and the proportion of the change in the horizontal coordinate of two adjacent action images being greater than zero is greater than a second preset proportion, then the user's gesture is determined to be a right wave; if the hand in the action image is a first type of start action, and the proportion of the change in the horizontal coordinate of two adjacent action images being less than zero is greater than a second preset proportion, then the user's gesture is determined to be a left wave.

[0060] Specifically, the first category of launch actions refers to launch actions performed when the user's hand is positioned on the left or right side of the launch action image.

[0061] Specifically, if the hand gesture in the startup animation is classified as a first-category startup action, and the change in the horizontal coordinate of two adjacent animation images is greater than zero, it indicates that the user's hand is moving to the right. If the proportion of horizontal coordinate changes greater than zero exceeds a second predetermined proportion, it indicates that the user's gesture trajectory is from left to right. Therefore, in this case, the user's gesture can be determined to be a rightward wave.

[0062] Specifically, if the hand gesture in the startup animation is classified as a first-category startup action, and the change in the horizontal coordinate of two adjacent animation images is less than zero, it indicates that the user's hand is moving to the left. If the proportion of horizontal coordinate changes less than zero is greater than a second predetermined proportion, it indicates that the user's gesture trajectory is from right to left. Therefore, in this case, the user's gesture can be determined to be a leftward wave.

[0063] Optionally, the change in the horizontal coordinate of two adjacent action images can be calculated using the following formula:

[0064] judgex i =x i+1 -x i ;

[0065] Among them, judgex i The x-axis represents the change in value. i+1 To obtain the x-coordinate value of the center coordinate of the hand in the action image of the later time, x i To obtain the x-coordinate value of the center coordinate of the hand in the action image that is earlier in the time frame.

[0066] Optionally, the user's gesture is determined based on the second center coordinates of the hand in multiple action images, including: if the hand in the action image is a second type of activation action, and the proportion of the vertical coordinate change values ​​of two adjacent action images being greater than zero is greater than a second preset proportion, then the user's gesture is determined to be an upward wave; if the hand in the action image is a second type of activation action, and the proportion of the vertical coordinate change values ​​of two adjacent action images being less than zero is greater than a second preset proportion, then the user's gesture is determined to be a downward wave.

[0067] Specifically, the second category of launch actions refers to launch actions performed by the user with their hand positioned above or below the launch action image.

[0068] Specifically, if the hand gesture in the startup animation is classified as a second type of startup action, and the change in the vertical coordinate of two adjacent animation images is greater than zero, it indicates that the user's hand is moving upwards. If the proportion of vertical coordinate changes greater than zero exceeds a second predetermined proportion, it indicates that the user's gesture trajectory is from bottom to top. Therefore, in this case, the user's gesture can be determined to be an upward wave.

[0069] Specifically, if the hand gesture in the startup animation is classified as a second type of startup action, and the change in the vertical coordinate of two adjacent animation images is less than zero, it indicates that the user's hand is moving downwards. If the proportion of vertical coordinate changes less than zero is greater than a second predetermined proportion, it indicates that the user's gesture trajectory is from top to bottom. Therefore, in this case, the user's gesture can be determined to be a downward wave.

[0070] Optionally, the change in the vertical coordinate of two adjacent action images can be calculated using the following formula:

[0071] judgey i =y i+1 -y i ;

[0072] Among them, judgey i The change in the vertical axis, y i+1To obtain the y-coordinate value of the center coordinate of the hand in the action image that is later in time, i To obtain the x-coordinate value of the center coordinate of the hand in the action image that is earlier in the time frame.

[0073] Optionally, the second set ratio is greater than or equal to 90%.

[0074] In this embodiment, user gestures are recognized using infrared images detected by an infrared sensor mounted on an electronic device. Compared to methods that use high-resolution cameras to detect hand images, this approach has lower requirements for ambient light and the computing power of the electronic device. Even in poor lighting conditions or when the electronic device's computing power is limited, it can quickly and accurately recognize user gestures. In this embodiment, when recognizing user gestures based on infrared images, it is first necessary to determine whether the infrared image is a starting action image. If multiple infrared images detected over a first set duration are all starting action images, and the rate of change of the first center coordinate of the hand in these multiple starting action images is less than a first set threshold, then multiple infrared images within a second set duration will be acquired. Next, the proportion of the judgment action image in the multiple infrared images is confirmed. If the proportion of the judgment action image in the multiple infrared images is greater than a first set proportion, then the user's specific gesture is recognized based on the second center coordinate of the hand in the judgment action image. This reduces the risk of misjudgment and missed detection when recognizing gestures. Therefore, this embodiment can improve the efficiency and accuracy of gesture recognition.

[0075] In some embodiments, it is determined whether an infrared image is a start-up action image by: determining the third center coordinates of the hand in the infrared image; if the hand in the infrared image is a first set gesture and the third center coordinates are located in a set area of ​​the infrared image, the first infrared image is determined to be a start-up action image.

[0076] Specifically, the first preset gesture represents the gesture for initiating the action. This first preset gesture can vary depending on the electronic device. Taking an air conditioner as an example, the first preset gesture could be as follows: Figure 3 The "clenched fist" gesture shown.

[0077] Specifically, the designated area in the infrared image refers to the area used by the user to perform the activation action.

[0078] Specifically, if the hand in the infrared image is in the first predetermined gesture, and the third center coordinate is located within the predetermined area of ​​the infrared image, it indicates that the hand in the area used for the activation action in this infrared image is performing an activation action. Therefore, in this case, the infrared image can be determined as an activation action image.

[0079] In some embodiments, the infrared image is determined to be a judgment action image by: determining the fourth center coordinate of the hand in the infrared image; if the hand in the infrared image is a second preset gesture and the fourth center coordinate is not located in the preset area of ​​the infrared image, the infrared image is determined to be a judgment action image.

[0080] Specifically, the second preset gesture represents the gesture used to determine the action. This second preset gesture can vary depending on the electronic device. Taking an air conditioner as an example, the second preset gesture could be as follows: Figure 4 The gesture shown is "open palm".

[0081] Specifically, the designated area in the infrared image refers to the area used by the user to perform the activation action.

[0082] Specifically, if the hand in the infrared image is in the second predetermined gesture, and the fourth center coordinate is not located within the predetermined area of ​​the infrared image, it indicates that in this infrared image, outside the area used for the initiation action, the hand is in a judgment action. Therefore, in this case, this infrared image can be determined as a judgment action image.

[0083] In some embodiments, a model for identifying infrared images as initiating or determining action images can first be constructed based on the AlexNet algorithm. Then, the category of the infrared image is identified based on this model. This improves the efficiency and accuracy of identifying infrared images as initiating or determining action images.

[0084] In some embodiments, before identifying the infrared image as a start-up action image or determining the action, the identification method further includes: preprocessing the infrared image.

[0085] Specifically, by preprocessing infrared images, blurry images, invalid images, and images that remain noisy even after processing can be removed. This improves the accuracy of subsequent recognition.

[0086] Optionally, the preprocessing steps for the infrared image include: using a filtering algorithm to separate useful signals and noise in the infrared image, improving the image's anti-interference capability and signal-to-noise ratio, filtering out uninteresting parts, and improving image accuracy; and using a bilinear interpolation algorithm to improve the resolution of the retained infrared image.

[0087] In some embodiments, where the infrared image includes multiple hands, only the hand with the highest detection score is identified.

[0088] Specifically, the detection score is used to determine which of several hands is most likely the target hand used to identify the gesture. The higher the detection score, the higher the probability that the hand is the target hand. Therefore, when an infrared image includes multiple hands, only the hand with the highest detection score is identified.

[0089] It is understandable that there may be situations where multiple hands are detected in the infrared image by the infrared sensor. In such cases, if gesture recognition is performed on all hands, the recognized control commands will be inconsistent, leading to abnormal control of the electronic device. Therefore, in this situation, the embodiments of this disclosure only recognize the hand with the highest detection score. This helps to improve the accuracy of gesture recognition.

[0090] In some embodiments, when the infrared image includes multiple hands, the detection score of each hand in the infrared image is determined as follows: the number of pixels and the contour area of ​​each hand in the infrared image are determined; the product of the number of pixels of the current hand and a first weighting coefficient is used as the first score of the current hand; the product of the contour area of ​​the current hand and a second weighting coefficient is used as the second score of the current hand; and the sum of the first score and the second score of the current hand is used as the detection score of the current hand.

[0091] It is understandable that the hand used to identify gestures in an infrared image is closer to the infrared sensor than other hands, and this hand is typically an adult's hand. Therefore, this hand should be the largest hand in the infrared image. Thus, in this embodiment, each hand is scored based on the number of pixels and the outline area of ​​each hand in the infrared image.

[0092] Specifically, the number of pixels reflects the overall pixel scale of the hand in the image, while the contour area reflects the image area occupied by the hand shape. Evaluating these two dimensions provides a more comprehensive understanding of the hand's state in the infrared image. Therefore, a first score for the current hand needs to be calculated based on the number of pixels and a first weighting coefficient, and a second score needs to be calculated based on the contour area and a second weighting coefficient. The sum of the first and second scores is then used as the detection score for the current hand.

[0093] Specifically, the first weighting coefficient and the second weighting coefficient can be adjusted according to the environmental parameters of the current operation of the electronic device.

[0094] In this embodiment of the disclosure, the accuracy of calculating the hand detection score is improved by calculating the detection score of each hand in the infrared image from two dimensions: the number of pixels and the contour area of ​​the hand.

[0095] Combination Figure 5As shown, this embodiment of the present disclosure provides a gesture recognition device 500, including: a first detection module 501, an acquisition module 502, a second detection module 503, and a determination module 504. The first detection module 501 is configured to determine the first center coordinates of the hand in the activation action image when multiple infrared images detected for a first set duration are all activation action images. The acquisition module 502 is configured to continuously acquire multiple infrared images within a subsequent second set duration when the rate of change of the first center coordinates of the hand in the multiple activation action images is less than a first set threshold. The second detection module 503 determines the second center coordinates of the hand in the judgment action image when the proportion of judgment action images in the multiple infrared images is greater than a first set proportion. The determination module 504 is configured to determine the user's gesture action based on the second center coordinates of the hand in the multiple judgment action images.

[0096] Combination Figure 6 As shown, this disclosure provides a gesture recognition device 600, which includes a processor 601 and a memory 602. Optionally, the device may further include a communication interface 603 and a bus 604. The processor 601, communication interface 603, and memory 602 can communicate with each other via the bus 604. The communication interface 603 can be used for information transmission. The processor 601 can call logical instructions in the memory 602 to execute the gesture recognition method of the above embodiment.

[0097] Furthermore, the logic instructions in the aforementioned memory 602 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0098] The memory 602, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor 601 executes functional applications and data processing by running the program instructions / modules stored in the memory 602, that is, it implements the gesture recognition method in the above embodiments.

[0099] The memory 602 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 602 may include high-speed random access memory and may also include non-volatile memory.

[0100] This disclosure provides a computer-readable storage medium storing computer-executable instructions configured to perform the above-described gesture recognition method.

[0101] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this disclosure. The aforementioned storage medium can be a non-transitory storage medium, such as a USB flash drive, external hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., and other media capable of storing program code.

[0102] The foregoing description and accompanying drawings fully illustrate embodiments of this disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or” as used in this application means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0103] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0104] The methods and products (including but not limited to devices and equipment) disclosed in the embodiments herein can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed units may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to implement this embodiment according to actual needs. Furthermore, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0105] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. A method for recognizing a gesture, characterized in that, The method comprises the following steps: In the case that all the infrared images detected in the first set time period are start action images, the first center coordinates of the hand in the start action images are determined; In the case that the change rate of the first center coordinates of the hand in the plurality of start action images is less than the first set threshold, a plurality of infrared images in the subsequent second set time period are continuously acquired; In the case that the proportion of the judgment action images in the plurality of infrared images is greater than the first set proportion, the second center coordinates of the hand in the judgment action images are determined; According to the second center coordinates of the hand in the plurality of judgment action images, the gesture action of the user is determined.

2. The identification method according to claim 1, characterized in that, The infrared image is determined as a start action image in the following manner: The third center coordinates of the hand in the infrared image are determined; In the case that the hand in the infrared image is a first set gesture, and the third center coordinates are located in a set region of the infrared image, the first infrared image is determined as a start action image.

3. The identification method according to claim 1, characterized in that, The infrared image is determined as a judgment action image in the following manner: The fourth center coordinates of the hand in the infrared image are determined; In the case that the hand in the infrared image is a second set gesture, and the fourth center coordinates are not located in a set region of the infrared image, the infrared image is determined as a judgment action image.

4. The recognition method according to claim 2 or 3, characterized in that, In the case that the infrared image comprises a plurality of hands, only the hand with the highest detection score is recognized.

5. The identification method according to claim 4, characterized in that, In the case that the infrared image comprises a plurality of hands, the detection score of each hand in the infrared image is determined in the following manner: The pixel point number and the contour area of each hand in the infrared image are determined; The product of the pixel point number of the current hand and the first weight coefficient is taken as the first score of the current hand; The product of the contour area of the current hand and the second weight coefficient is taken as the second score of the current hand; The sum of the first score and the second score of the current hand is taken as the detection score of the current hand.

6. The identification method according to any one of claims 1 to 3, characterized in that, The change rate of the first center coordinates comprises a horizontal change rate and a vertical change rate; the change rate of the first center coordinates of the hand in the plurality of start action images is determined in the following manner: The start image and the end image in the plurality of start action images are determined; The difference value of the horizontal coordinate in the first center coordinates of the hand between the end image and the start image is divided by the first set time period to obtain the horizontal change rate; The difference value of the vertical coordinate in the first center coordinates of the hand between the end image and the start image is divided by the first set time period to obtain the vertical change rate.

7. The identification method according to any one of claims 1 to 3, characterized in that, According to the second center coordinates of the hand in the plurality of judgment action images, the gesture action of the user is determined, comprising: In the case that the hand in the start action image is a first category of start action, if the proportion of the horizontal coordinate change value greater than zero between two adjacent judgment action images in the plurality of judgment action images is greater than a second set proportion, the gesture action of the user is determined as right swing hand; In the case that the hand in the start action image is a first category of start action, if the proportion of the horizontal coordinate change value less than zero between two adjacent judgment action images in the plurality of judgment action images is greater than a second set proportion, the gesture action of the user is determined as left swing hand; In a case where the hand in the starting action image is of the second category of starting action, if the proportion of the adjacent two judgment action images in the multiple judgment action images whose longitudinal coordinate change values are greater than zero is greater than a second set proportion, it is determined that the user's gesture action is waving up; In a case where the hand in the starting action image is of the second category of starting action, if the proportion of the adjacent two judgment action images in the multiple judgment action images whose longitudinal coordinate change values are less than zero is greater than a second set proportion, it is determined that the user's gesture action is waving down.

8. An apparatus for recognizing a gesture, characterized by Comprise: The first detection module is configured to determine the first center coordinate of the hand in the starting action image in a case where the multiple infrared images detected for a first set duration are all starting action images; The acquisition module is configured to continuously acquire multiple infrared images within a subsequent second set duration in a case where the change rate of the first center coordinate of the hand in the multiple starting action images is less than a first set threshold; The second detection module determines the second center coordinate of the hand in the judgment action image in a case where the proportion of the judgment action images in the multiple infrared images is greater than a first set proportion; The determination module is configured to determine the user's gesture action according to the second center coordinate of the hand in the multiple judgment action images.

9. An apparatus for recognizing a gesture, comprising a processor and a memory having stored program instructions, characterized in that, The processor is configured to execute the gesture recognition method according to any one of claims 1 to 7 when running the program instructions.

10. An electronic device, comprising: Comprise: The device body; The infrared sensor is arranged on the device body and is used for detecting the infrared image of the user's hand; The gesture recognition device according to claim 8 or 9 is arranged on the device body and is electrically connected with the infrared sensor.

Images