A gesture recognition method and related device

Abstract

The embodiments of the present application disclose a method for gesture recognition and related equipment, which can effectively reduce the interference of objects similar to gesture movements on gesture recognition and improve the accuracy of gesture recognition. The present application can optimize the human-computer interaction mode based on gesture recognition through artificial intelligence. The method of the present application includes: obtaining a differential image frame between a current image frame and a previous image frame, wherein the current image frame includes a target human body; obtaining a momentum accumulation value of the current image frame, wherein the momentum accumulation value is obtained based on the momentum of the current image frame and the momentum of multiple consecutive image frames before the current image frame, wherein the momentum of the current image frame is obtained based on the differential image frame; based on the momentum accumulation value of the current image frame, obtaining the average momentum accumulation value of each pixel point in the target area of the current image frame, wherein the target area includes the hand of the target human body; if it is determined that the average value is greater than or equal to a preset threshold, performing gesture recognition on the target area to obtain the target gesture movement.

Description

Technical Field

[0001] The present application relates to the field of human-computer interaction technology, and in particular to a gesture recognition method and related equipment. Background Art

[0002] With the development of information technology, human-computer interaction has become an increasingly important part of daily life. Traditional human-computer interaction devices, such as mice, keyboards, and remote controls, lack naturalness and user-friendliness. Consequently, users are eager to replace traditional input-based human-computer interaction models with a natural and intuitive approach.

[0003] The human-computer interaction model based on gesture recognition is being used more and more widely due to its naturalness, intuitiveness, simplicity and other characteristics. Specifically, when a user remotely inputs a specific gesture to a device, the device can recognize the gesture based on the human-computer interaction model of gesture recognition, thereby determining the control command input by the user. However, when performing gesture recognition, if there are some objects similar to the specific gesture next to the user (for example, a paper bag next to a user with a certain gesture printed on the surface), it will usually cause the device to misrecognize the gesture, thereby affecting the accuracy of gesture recognition. Summary of the Invention

[0004] The embodiments of the present application provide a method for gesture recognition and related devices, which can effectively reduce the interference of objects similar to gesture movements on gesture recognition and improve the accuracy of gesture recognition.

[0005] A first aspect of an embodiment of the present application provides a method for gesture recognition, the method comprising:

[0006] When gesture recognition is required, a current image frame containing the target person can be obtained first. After obtaining the current image frame, inter-frame difference processing can be performed based on the current image frame and the previously obtained previous image frame to obtain a differential image frame between the current image frame and the previous image frame.

[0007] After obtaining a differential image frame between the current image frame and the previous image frame, the momentum of the current image frame can be calculated based on the differential image frame. Then, the momentum accumulation value of the current image frame is calculated based on the momentum of the current image frame and the momentum of multiple consecutive image frames before the current image frame.

[0008] A target area is set in the current image frame, the target area including the hands of the target human body. Then, based on the momentum accumulation value of the current image frame, an average value of the momentum accumulation value of each pixel point in the target area of the current image frame is calculated.

[0009] If it is determined that the average value is greater than or equal to a preset threshold, gesture recognition is performed on the target area to obtain a target gesture action, thereby determining a control instruction input by the user.

[0010] From the above method, it can be seen that after obtaining the differential image frame between the current image frame and the previous image frame, the momentum of the current image frame can be calculated based on the differential image frame, and then the momentum accumulation value of the current image frame can be calculated. Then, based on the momentum accumulation value of the current image frame, the average momentum accumulation value of each pixel point in the target area can be determined. If this average value is greater than or equal to a preset threshold, the hand of the target person in the target area is determined to be a moving hand. Finally, gesture recognition is performed on the target area to obtain the target gesture action, thereby determining the control command input by the user. Since the target person's hand is usually in motion when performing human-computer interaction through gestures, after determining that the target person's hand is in motion in the current image frame, the result obtained by gesture recognition is basically the target person's gesture action, which can effectively reduce the interference caused by objects similar to gesture actions on gesture recognition and improve the accuracy of gesture recognition.

[0011] In a possible implementation, if it is determined that the average value is greater than or equal to a preset threshold value includes: if it is determined that the average value is greater than or equal to the preset threshold value, and if it is determined that the hand of the target person in the target area does not hold an object.

[0012] In one possible implementation, performing gesture recognition on the target area to obtain the target gesture action includes: performing gesture recognition on the target area to obtain probabilities of multiple candidate gesture actions, and determining the target gesture action from the multiple candidate gesture actions based on the probabilities of the multiple candidate gesture actions.

[0013] In a possible implementation, the distance between the target human body and the camera is within a preset range, and the camera is used to acquire image frames.

[0014] In a possible implementation, the momentum of the current image frame is the product of the differential image frame and a preset momentum coefficient.

[0015] In a possible implementation, the momentum accumulation value is obtained based on a preset momentum coefficient, the momentum of the current image frame, and the momentum of a plurality of consecutive image frames before the current image frame.

[0016] A second aspect of an embodiment of the present application provides a gesture recognition device, the device comprising:

[0017] The first acquisition module is used to acquire a differential image frame between a current image frame and a previous image frame, where the current image frame includes a target human body.

[0018] The second acquisition module is used to obtain the momentum accumulation value of the current image frame. The momentum accumulation value is obtained based on the momentum of the current image frame and the momentum of multiple consecutive image frames before the current image frame. The momentum of the current image frame is obtained based on the differential image frame.

[0019] The third acquisition module is used to obtain the average value of the momentum accumulation value of each pixel point in the target area of the current image frame based on the momentum accumulation value of the current image frame, and the target area includes the hand of the target human body.

[0020] The recognition module is configured to perform gesture recognition on the target area to obtain a target gesture action if it is determined that the average value is greater than or equal to a preset threshold.

[0021] It can be seen from the above device that after obtaining the differential image frame between the current image frame and the previous image frame, the momentum of the current image frame can be calculated based on the differential image frame, and then the momentum accumulation value of the current image frame can be calculated. Then, based on the momentum accumulation value of the current image frame, the average momentum accumulation value of each pixel point in the target area can be determined. If the average value is greater than or equal to a preset threshold, it is determined that the hand of the target person in the target area is a hand in motion. Finally, gesture recognition is performed on the target area to obtain the target gesture action, thereby determining the control command input by the user. Since the target person's hand is usually in motion when performing human-computer interaction through gestures, after determining that the target person's hand in the current image frame is in motion, the result obtained by gesture recognition is basically the gesture action of the target person, which can effectively reduce the interference of objects similar to gesture actions on gesture recognition and improve the accuracy of gesture recognition.

[0022] In a possible implementation, the recognition module is further configured to perform gesture recognition on the target area to obtain a target gesture action if it is determined that the average value is greater than or equal to a preset threshold and it is determined that the hand of the target person in the target area does not hold an object.

[0023] In one possible implementation, the recognition module is further configured to, if the average value is determined to be greater than or equal to a preset threshold and it is determined that the hand of the target person in the target area is not holding an object, perform gesture recognition on the target area to obtain probabilities of multiple candidate gesture actions. Based on the probabilities of the multiple candidate gesture actions, a target gesture action is determined from the multiple candidate gesture actions.

[0024] In a possible implementation, the device further includes a camera, which is used to acquire image frames, and the distance between the target human body and the camera is within a preset range.

[0025] In a possible implementation, the momentum of the current image frame is the product of the differential image frame and a preset momentum coefficient.

[0026] In a possible implementation, the momentum accumulation value is obtained based on a preset momentum coefficient, the momentum of the current image frame, and the momentum of a plurality of consecutive image frames before the current image frame.

[0027] A third aspect of the embodiments of the present application provides a gesture recognition device, the device comprising:

[0028] One or more processors, memory, input and output interfaces, wired or wireless network interfaces, and power supply;

[0029] The memory is either transient or persistent storage;

[0030] The processor is configured to communicate with the memory and execute instructions in the memory on the gesture recognition device to perform any one of the methods described in the first aspect.

[0031] A fourth aspect of an embodiment of the present application provides a terminal device, which includes the apparatus according to the third aspect.

[0032] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enable the computer to execute the method described in any one of the first aspects.

[0033] A sixth aspect of an embodiment of the present application provides a computer program product comprising instructions, wherein the computer program product includes program instructions. When the program instructions are executed on a computer or a processor, the computer or the processor executes the method as described in any one of the first aspects.

[0034] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0035] In an embodiment of the present application, after obtaining a differential image frame between the current image frame and the previous image frame, the momentum of the current image frame can be calculated based on the differential image frame, and then the momentum accumulation value of the current image frame can be calculated. Then, based on the momentum accumulation value of the current image frame, the average value of the momentum accumulation value of each pixel point in the target area can be determined. If the average value is greater than or equal to a preset threshold value, it is determined that the hand of the target human body in the target area is a hand in motion. Finally, gesture recognition is performed on the target area to obtain the target gesture action, thereby determining the control instruction input by the user. Since the hand of the target human body is usually in motion when performing human-computer interaction through gesture actions. After determining that the hand of the target human body in the current image frame is in motion, the result obtained by gesture recognition is basically the gesture action of the target human body, which can effectively reduce the interference of objects similar to gesture actions on gesture recognition and improve the accuracy of gesture recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present application;

[0037] Figure 2 A flowchart of a gesture recognition method provided in an embodiment of the present application;

[0038] Figure 3 A schematic diagram of a target area provided in an embodiment of the present application;

[0039] Figure 4 A schematic diagram of a gesture action provided in an embodiment of the present application;

[0040] Figure 5 A schematic diagram of the structure of a gesture recognition device provided in an embodiment of the present application;

[0041] Figure 6 Another structural diagram of the gesture recognition device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] The embodiments of the present application provide a method for gesture recognition and related devices, which can effectively reduce the interference of objects similar to gesture movements on gesture recognition and improve the accuracy of gesture recognition.

[0043] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0044] Artificial Intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a branch of computer science that seeks to understand the essence of intelligence and develop new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making. Research in the field of AI includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, and basic AI theory. Therefore, AI can be used to optimize human-computer interaction models based on gesture recognition.

[0045] This example uses a user's gesture control to illustrate an application scenario. In this scenario, the device is typically a stationary terminal device, such as a television or monitor equipped with a camera that can capture image frames. Within a certain distance, the user can make specific gestures toward the device, allowing the device to recognize the gesture and determine the control command entered by the user. Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present application. Figure 1 As shown in the figure, when a user appears in front of the TV, the TV's camera can capture the user's image frames in real time. Once the user makes a specific gesture toward the TV, the TV's built-in processor can recognize the user's hand gesture based on the image frames captured by the camera, thereby determining the control command input by the user and responding to the control command.

[0046] However, if an object similar to a specific gesture appears next to the user, the device may misrecognize the gesture, thereby affecting the accuracy of gesture recognition. In order to improve the accuracy of gesture recognition by the device, an embodiment of the present application provides a method for gesture recognition. Figure 2 This is a flow chart of the gesture recognition method provided in the embodiment of the present application. Figure 2 The execution subject of the method shown is a gesture recognition device, which is usually built into a terminal device and includes a camera and a processor, wherein the camera is used to collect image frames including human bodies, and the processor can perform operations such as target detection and gesture recognition on the image frames from the camera. Figure 2 As shown, the method includes:

[0047] 201. Obtain a differential image frame between a current image frame and a previous image frame, where the current image frame includes a target human body.

[0048] After the terminal device is activated, the camera in the gesture recognition device can capture image frames in real time in front of the terminal device. The camera can send the captured image frames to the processor, which then performs target detection on the image frames. It should be noted that the processor is loaded with a trained neural network. Therefore, the processor can input the image frames into the neural network, causing the neural network to perform target detection on the image frames. If the neural network determines that multiple people appear in the image frame, and the distance between at least one person and the device is within a preset range, the at least one person within the preset range is determined as a target person. It is understood that the preset range is a pre-set range, such as 3m, 4m, etc. Since multiple people may appear in front of the terminal device, the neural network can filter the distances, eliminating people farther from the device and identifying people closer to the device as target people. The target person's hand gestures can then be considered valid gestures (i.e., gestures that may issue control commands to the device). This reduces the processor's workload and power consumption, thereby achieving energy conservation. After the processor determines the target person based on the neural network, it can control the camera to lock onto the target person, causing the camera to capture multiple consecutive image frames containing the target person.

[0049] It should be understood that the above-mentioned neural network can be any one of the models such as multi-layer perceptron (MLP), convolutional neural networks (CNN), recursive neural network (recursive neural network), recurrent neural network (RNN), etc., and is not limited here.

[0050] It should also be understood that the neural network training process in this embodiment is a common neural network training process in AI technology and will not be described in detail here.

[0051] In this embodiment, the image frame containing the target human body captured by the camera at the current moment is referred to as the current image frame. After the processor obtains the current image frame, it can obtain the previous image frame obtained at the previous moment. Then, an inter-frame difference operation is performed on the current image frame and the previous image frame to obtain a differential image frame between the current image frame and the previous image frame. Specifically, the calculation formula for obtaining the differential image frame is as follows:

[0052] D m =F m -F m-1

[0053] In the above formula, D m is the differential image frame of the current image frame, F m is the pixel value of the current image frame, Fm-1 is the pixel value of the previous image frame, m is the frame number, and m ≥ 2. For example, when m is 2, it indicates that the current image frame F2 is the second image frame, the previous image frame F1 is the first image frame, and D2 is the difference image frame between F2 and F1. For another example, when m is 3, it indicates that the current image frame F3 is the third image frame, the previous image frame F2 is the second image frame, and D3 is the difference image frame between F3 and F2, and so on.

[0054] 202. Obtain a momentum accumulation value of the current image frame, where the momentum accumulation value is obtained based on the momentum of the current image frame and the momentum of a plurality of consecutive image frames before the current image frame, and the momentum of the current image frame is obtained based on the differential image frame.

[0055] After obtaining the differential image frame between the current image frame and the previous image frame, the processor can calculate the momentum of the current image frame based on the differential image frame. Specifically, the momentum of the current image frame is the product of the differential image frame and a preset momentum coefficient. The calculation formula for the momentum of the current image frame is as follows:

[0056] A m =αD m

[0057] In the above formula, A m is the momentum of the current image frame, and α is the preset momentum coefficient.

[0058] After obtaining the momentum of the current image frame, the processor may calculate the accumulated momentum value of the current image frame based on the momentum of the current image frame and the momentum of multiple consecutive image frames before the current image frame. Specifically, the accumulated momentum value of the current image frame is obtained based on a preset momentum coefficient, the momentum of the current image frame, and the momentum of multiple consecutive image frames before the current image frame (i.e., the momentum of all image frames before the current image frame). The calculation formula for the accumulated momentum value of the current image frame is as follows:

[0059] DIF1=A1

[0060] DIF2=A2+(1-α)DIF1 ......

[0062] DIF m =A m +(1-α)DIF m-1

[0063] In the above formula, A1 is the momentum of the first image frame, DIF1 is the momentum accumulation value of the first image frame, A2 is the momentum of the second image frame, DIF2 is the momentum accumulation value of the second image frame, and DIF m-1 The momentum accumulation value of the previous image frame of the current image frame, DIF mis the momentum accumulation value of the current image frame. It is understood that, as the frame number of the current image frame increases, the momentum accumulation value of the current image frame becomes more correlated with the momentum of the current image frame and less correlated with the momentum accumulation value of the previous image frame (i.e., the momentum accumulation value of the current image frame is less affected by the momentum of multiple consecutive image frames preceding the current image frame).

[0064] 203. Based on the momentum accumulation value of the current image frame, obtain an average value of the momentum accumulation value of each pixel point in the target area of the current image frame, where the target area includes the hand of the target person.

[0065] After obtaining the momentum accumulation value of the current image frame, it is equivalent to obtaining the momentum accumulation value of each pixel in the current image frame. In order to further reduce the workload of the processor, a target area can be set in the current image frame. The target area includes the hands of the target human body, so that the processor only performs subsequent processing on the target area in the current image frame. The processor does not need to process the remaining areas in the current image frame except the target area, so as to further reduce the power consumption of the processor and achieve energy saving. For further understanding, the following is combined with Figure 3 Provide further introduction to the target area. Figure 3 A schematic diagram of the target area provided in the embodiment of the present application, such as Figure 3 As shown, the processor sets a detection frame (ie, target area) in the current image frame, and the detection frame can be used to capture the hand of the target human body in the current image frame.

[0066] After determining the target area of the current image frame, the processor may obtain an average of the momentum accumulation values of each pixel in the target area of the current image frame. Specifically, after demarcating the target area from the current image frame, the momentum accumulation value of each pixel in the target area may be determined and averaged (i.e., the momentum accumulation values of all pixels in the target area are summed and then divided by the number of pixels in the target area) to obtain the average of the momentum accumulation values of each pixel in the target area.

[0067] 204. Determine whether the average value is greater than or equal to a preset threshold. If so, execute step 205; if not, stop the operation.

[0068] After obtaining the average value of the momentum accumulation value of each pixel point in the target area of the current image frame, the processor can compare the size between the average value and the preset threshold value. If the average value is greater than or equal to the preset threshold value, it is determined that the object in the target area is a moving object, that is, the hand of the target body (because when the user sends a control instruction to the device through a gesture, the user's hand needs to move), and the hand of the target body is in motion, rather than a stationary object, so as to avoid the influence of objects similar to specific gestures. If the average value is less than the preset threshold value, the object in the target area may be a stationary object (for example, an object located next to the user that is similar to a specific gesture), and there is a possibility of gesture misrecognition, and subsequent operations will not continue.

[0069] 205. Determine whether the hand of the target person in the target area holds an object. If not, execute step 206. If yes, stop the operation.

[0070] After determining that the average value is greater than or equal to the preset threshold, the processor can detect whether the hand of the target person in the target area is holding an object. Specifically, the processor can perform an intersection of union (IOU) detection on the objects in the target area. After obtaining the IOU between the hand of the target person and the remaining objects, if the IOU is less than the preset IOU, it is determined that the hand of the target person is not holding an object. If the IOU is greater than or equal to the preset IOU threshold, it is determined that the hand of the target person is holding an object. There may be gesture misrecognition (for example, the user is drinking water, his hand is holding a water cup, and the hand is in motion, but the user does not want to issue control instructions to the device), and the subsequent operation will not continue.

[0071] 206. Perform gesture recognition on the target area to obtain a target gesture action.

[0072] After determining that the target person's hand is not holding an object, the processor performs gesture recognition on the target area to obtain the target gesture action. Specifically, the processor first performs gesture recognition on the target area to obtain the probabilities of multiple candidate gesture actions. Then, based on the probabilities of multiple candidate gesture actions, the processor determines the target gesture action from multiple candidate gesture actions. When selecting a final gesture action as the target gesture action from multiple candidate gesture actions, various situations may occur, which will be introduced below:

[0073] (1) After obtaining the probabilities of the first number of candidate gesture actions, the probability of each candidate gesture action can be compared with the corresponding probability threshold. If the probabilities of the second number (less than or equal to the first number) of candidate gesture actions are greater than the corresponding probability threshold, and these candidate gesture actions are different from each other, then according to the preset priority, the gesture action with the highest priority among these candidate gesture actions is selected as the target gesture action. For ease of understanding, the following text combines Figure 4 The aforementioned process is introduced. Figure 4 A schematic diagram of the gesture action provided in the embodiment of the present application, such as Figure 4 As shown, the processor obtains the probabilities of four gesture actions, namely the probability of gesture action 1, the probability of gesture action 2, the probability of gesture action 3 and the probability of gesture action 4. Then, the probability of gesture action 1 is compared with the preset probability threshold A (set to 0.95), the probability of gesture action 2 is compared with the preset probability threshold B (set to 0.9), the probability of gesture action 3 is compared with the preset probability threshold C (set to 0.85), and the probability of gesture action 4 is compared with the preset probability threshold D (set to 0.8). After determining that only the probability of gesture action 1, the probability of gesture action 2 and the probability of gesture action 3 are greater than the corresponding probability thresholds, gesture action 1 is selected as the target gesture action according to the priority (gesture action 1 is better than gesture action 2, gesture action 2 is better than gesture action 3). It should be noted that the priority of gesture actions is pre-set, and the priority of gesture actions can be set according to the false touch rate of gesture actions.

[0074] (2) After obtaining the probabilities of the first number of candidate gesture actions, the probability of each candidate gesture action can be compared with the corresponding probability threshold. If the probabilities of the second number (less than or equal to the first number) of candidate gesture actions are greater than the corresponding probability threshold, and these candidate gesture actions are the same gesture actions, then a gesture action closest to the camera is selected from them as the target gesture action. For example, the processor obtains the probabilities of four gesture actions, namely, the probability a of gesture action 1, the probability b of gesture action 1, the probability of gesture action 2, and the probability of gesture action 3. Then, the probability a of gesture action 1 is compared with the preset probability threshold A, the probability b of gesture action 1 is compared with the preset probability threshold A, the probability of gesture action 2 is compared with the preset probability threshold B, and the probability of gesture action 3 is compared with the preset probability threshold C. After determining that only the probability a of gesture action 1 and the probability b of gesture action 1 are greater than the corresponding probability threshold, assuming that the gesture action corresponding to the probability a of gesture action 1 is closer to the camera (displayed larger in the image frame), then this gesture action can be used as the target gesture action.

[0075] After determining the target gesture action, the processor may determine the instruction corresponding to the target gesture action, that is, determine the control instruction issued by the user, and respond to the control instruction.

[0076] In this embodiment, after obtaining a differential image frame between the current image frame and the previous image frame, the momentum of the current image frame can be calculated based on the differential image frame, and then the accumulated momentum value of the current image frame can be calculated. Then, based on the accumulated momentum value of the current image frame, the average accumulated momentum value of each pixel in the target area can be determined. If this average value is greater than or equal to a preset threshold, the hand of the target person in the target area is determined to be in motion. Finally, gesture recognition is performed on the target area to obtain the target gesture action, thereby determining the control command input by the user. Since the target person's hand is typically in motion when performing human-computer interaction through gestures, after determining that the target person's hand is in motion in the current image frame, the result obtained by gesture recognition is essentially the target person's gesture action, which can effectively reduce interference caused by objects similar to the gesture action on gesture recognition and improve the accuracy of gesture recognition.

[0077] The above is a detailed description of the gesture recognition method provided in the embodiment of the present application. The following will introduce the gesture recognition device provided in the embodiment of the present application. Figure 5 A structural diagram of a gesture recognition device provided in an embodiment of the present application is shown as follows: Figure 5 As shown, the device includes:

[0078] The first acquisition module 501 is configured to acquire a differential image frame between a current image frame and a previous image frame, wherein the current image frame includes a target human body.

[0079] The second acquisition module 502 is used to obtain the momentum accumulation value of the current image frame. The momentum accumulation value is obtained based on the momentum of the current image frame and the momentum of multiple consecutive image frames before the current image frame. The momentum of the current image frame is obtained based on the differential image frame.

[0080] The third acquisition module 503 is configured to acquire an average value of the momentum accumulation values of each pixel point in a target area of the current image frame based on the momentum accumulation value of the current image frame, where the target area includes the hands of the target person.

[0081] The recognition module 504 is configured to perform gesture recognition on the target area to obtain a target gesture action if it is determined that the average value is greater than or equal to a preset threshold.

[0082] In a possible implementation, the recognition module is further configured to perform gesture recognition on the target area to obtain a target gesture action if it is determined that the average value is greater than or equal to a preset threshold and it is determined that the hand of the target person in the target area does not hold an object.

[0083] In one possible implementation, the recognition module is further configured to, if the average value is determined to be greater than or equal to a preset threshold and it is determined that the hand of the target person in the target area is not holding an object, perform gesture recognition on the target area to obtain probabilities of multiple candidate gesture actions. Based on the probabilities of the multiple candidate gesture actions, a target gesture action is determined from the multiple candidate gesture actions.

[0084] In a possible implementation, the device further includes a camera, which is used to acquire image frames, and the distance between the target human body and the camera is within a preset range.

[0085] In a possible implementation, the momentum of the current image frame is the product of the differential image frame and a preset momentum coefficient.

[0086] In a possible implementation, the momentum accumulation value is obtained based on a preset momentum coefficient, the momentum of the current image frame, and the momentum of a plurality of consecutive image frames before the current image frame.

[0087] It should be noted that the information interaction, execution process, etc. between the modules / units of the above-mentioned device are based on the same concept as the method embodiment of the present application, and the technical effects they bring are the same as those of the method embodiment of the present application. For specific contents, please refer to the description in the method embodiment shown above in the present application, and no further details will be given here.

[0088] Figure 6 This is another structural diagram of the gesture recognition device provided in the embodiment of the present application. Figure 6 As shown, the device may include one or more processors 601 , a memory 602 , an input / output interface 603 , a wired or wireless network interface 604 , and a power supply 605 .

[0089] The memory 602 may be a temporary storage or a permanent storage. Furthermore, the processor 601 may be configured to communicate with the memory 602 and execute a series of instruction operations in the memory 602 on the gesture recognition device.

[0090] In this embodiment, the processor 601 may execute the aforementioned Figure 2 The method steps in the illustrated embodiment will not be described in detail here.

[0091] In this embodiment, the specific functional module division in the processor 601 can be the same as the above Figure 5 The functional module division method of the first acquisition module, second acquisition module, third acquisition module, identification module and other modules described in are similar and will not be repeated here.

[0092] The embodiment of the present application also provides a terminal device, which can be a device such as a television with a camera, a monitor, etc. that can capture image frames. The terminal device includes: Figure 6 The device shown.

[0093] The embodiment of the present application also provides a computer-readable storage medium, including instructions, which, when executed on a computer, enable the computer to execute the following Figure 2 The method shown.

[0094] The embodiment of the present application further provides a computer program product comprising instructions, wherein the computer program product comprises program instructions, and when the program instructions are executed on a computer or a processor, the computer or the processor executes the following Figure 2 The method shown.

[0095] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0096] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0097] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0098] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0099] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

Claims

1. A method for gesture recognition, characterized in that: The method comprises: Acquire a differential image frame between a current image frame and a previous image frame, wherein the current image frame includes a target human body; Obtaining a momentum accumulation value of the current image frame, wherein the momentum accumulation value is obtained based on the momentum of the current image frame and the momentum of a plurality of consecutive image frames before the current image frame, and the momentum of the current image frame is obtained based on the differential image frame; wherein, The calculation formula for the momentum of the current image frame includes: ,in, is the momentum of the current image frame, is the preset momentum coefficient, is the differential image frame of the current image frame; The calculation formula of the momentum accumulation value of the current image frame includes: ,in, The momentum accumulation value of the previous image frame of the current image frame, is the momentum accumulation value of the current image frame; Based on the momentum accumulation value of the current image frame, obtaining an average value of the momentum accumulation value of each pixel point in a target area of the current image frame, where the target area includes a hand of the target person; If it is determined that the average value is greater than or equal to a preset threshold, gesture recognition is performed on the target area to obtain a target gesture action.

2. The method according to claim 1, characterized in that Determining that the average value is greater than or equal to a preset threshold includes: If it is determined that the average value is greater than or equal to the preset threshold, and it is determined that the hand of the target person in the target area does not hold an object.

3. The method according to claim 2, characterized in that The performing gesture recognition on the target area to obtain a target gesture action includes: Performing gesture recognition on the target area to obtain probabilities of multiple candidate gesture actions; Based on the probabilities of the multiple candidate gesture actions, a target gesture action is determined from the multiple candidate gesture actions.

4. The method according to any one of claims 1 to 3, characterized in that The distance between the target human body and the camera is within a preset range, and the camera is used to acquire image frames.

5. The method according to any one of claims 1 to 3, characterized in that The momentum of the current image frame is the product of the differential image frame and a preset momentum coefficient.

6. The method according to claim 5, characterized in that The momentum accumulation value is obtained based on the preset momentum coefficient, the momentum of the current image frame, and the momentum of a plurality of consecutive image frames before the current image frame.

7. A gesture recognition device, characterized in that: The device comprises: A first acquisition module is configured to acquire a differential image frame between a current image frame and a previous image frame, wherein the current image frame includes a target human body; The second acquisition module is configured to acquire a momentum accumulation value of the current image frame, wherein the momentum accumulation value is obtained based on the momentum of the current image frame and the momentum of a plurality of consecutive image frames before the current image frame, and the momentum of the current image frame is obtained based on the differential image frame; wherein, The calculation formula for the momentum of the current image frame includes: ,in, is the momentum of the current image frame, is the preset momentum coefficient, is the differential image frame of the current image frame; The calculation formula of the momentum accumulation value of the current image frame includes: ,in, The momentum accumulation value of the previous image frame of the current image frame, is the momentum accumulation value of the current image frame; a third acquisition module, configured to acquire, based on the momentum accumulation value of the current image frame, an average value of the momentum accumulation values of each pixel point in a target area of the current image frame, wherein the target area includes a hand of the target person; The recognition module is configured to perform gesture recognition on the target area to obtain a target gesture action if it is determined that the average value is greater than or equal to a preset threshold.

8. A gesture recognition device, characterized in that: include: One or more processors, memory, input and output interfaces, wired or wireless network interfaces, and power supply; The memory is a transient storage memory or a persistent storage memory; The processor is configured to communicate with the memory and execute instructions in the memory on the gesture recognition device to perform the method according to any one of claims 1 to 4.

9. A terminal device, characterized in that: The terminal device comprises the apparatus according to claim 8.

10. A computer-readable storage medium comprising instructions, which, when executed on a computer, enable the computer to perform the method according to any one of claims 1 to 4.

Images