Continuous frame correction method, device, electronic device and medium for gesture control

Abstract

The present invention discloses a continuous frame correction method, device, electronic device and medium for gesture control, wherein the correction method comprises the following steps: obtaining continuous n-frame images from the acquired original video; performing gesture recognition on each frame and obtaining the gesture type of the frame; performing gesture type correction on the last frame and the gesture type correction on the middle frame for the continuous n-frame images with the current frame as the last frame. The present invention can correct the gesture type of the misdetected frames and lost frames in the continuous multi-frame images, which helps to improve the accuracy and effectiveness of the gesture control in the air.

Description

Technical Field

[0001] The present invention relates to air gesture control technology, and in particular to a continuous frame correction method, device, electronic device and computer-readable storage medium for gesture control. Background Art

[0002] With the continuous development of electronic technology, users' use of various electronic devices is no longer limited to operating by touching the touch screen. At present, more and more electronic devices can be equipped with video acquisition devices such as cameras to recognize the gestures performed by users in three-dimensional space (i.e., air gestures), thereby realizing air operation of some functions in electronic devices. Air gesture control (abbreviated as gesture control) technology allows users to get rid of traditional external devices or remote control devices when controlling electronic devices, and there is no need to use close-range touch methods such as touch screens. The control methods of electronic devices are becoming more and more diversified, simple and convenient, and more humanized. Therefore, air gesture control technology is also widely used in various application scenarios, such as smart homes, vehicle control systems, virtual reality interactions, etc.

[0003] Videos are composed of a series of still images, which are called frames. Gesture control is to determine the hand area, gesture type or audio command contained in each frame in the continuous frames, and calculate and process the image according to the gesture type and audio command. In the conventional calculation strategy, the air gesture control system needs to correctly detect the gesture state of each frame in the entire hand movement in order to respond to the user accurately and promptly. However, due to the rapid changes in hand movement speed and the variety of gesture types, coupled with the lighting characteristics of the surrounding environment and various interference factors, it is difficult for the gesture control system to avoid false detection and frame loss. If there is a frame in the hand movement sequence that has a gesture recognition detection error or does not detect the user's gesture at all, this will cause the air gesture control system to be unable to correctly understand the user's gesture, and thus cannot execute the correct command, seriously affecting the user experience. Summary of the invention

[0004] Based on the above situation, the main purpose of the present invention is to provide a continuous frame correction method, device, electronic device and computer-readable storage medium for gesture control, which can judge the end frame and intermediate frame with abnormal and sudden changes in gesture type in continuous multi-frame images, and correct the gesture type of these misdetected frames according to the gesture type and control attributes of the previous and next continuous image frames to ensure that the correct control instructions are obtained.

[0005] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0006] According to a first aspect of the present invention, a continuous frame correction method for gesture control comprises the following steps:

[0007] Obtain n consecutive frames of images from the acquired original video;

[0008] Performing gesture recognition on each frame of image and acquiring the gesture type of the frame, wherein the gesture type includes an unrecognizable gesture type and a plurality of recognizable gesture types having different control attributes;

[0009] The gesture type correction of the last frame and the gesture type correction of the middle frame are performed on the continuous n-frame images with the current frame as the last frame, wherein:

[0010] The last frame gesture type correction includes: if the gesture types of the first n-1 frames are all the first gesture type, and the gesture type of the last n-th frame is the second gesture type, then

[0011] If the first gesture type is an unrecognizable gesture type, and the second gesture type is a recognizable gesture type, the gesture type of the nth frame is corrected to an unrecognizable gesture type;

[0012] If the first gesture type is a recognizable gesture type, correcting the gesture type of the nth frame according to the control attribute of the first gesture type;

[0013] The intermediate frame gesture type correction includes: if the gestures of the first k-1 frames and the last nk frames are both of the third gesture type, and the gesture type of the intermediate kth frame is of the fourth gesture type, then correcting the gesture type of the kth frame according to the third gesture type.

[0014] Furthermore, the control properties include switch type and continuous type;

[0015] The correcting the gesture type of the nth frame according to the control attribute of the first gesture type includes:

[0016] If the control attribute of the first gesture type is a switch type, no correction is performed on the gesture type of the nth frame;

[0017] If the control attribute of the first gesture type is continuous, the gesture type of the nth frame is corrected to the first gesture type.

[0018] Furthermore, the performing gesture recognition on each frame of image further includes obtaining position information of the gesture in the frame;

[0019] The correcting the gesture type of the nth frame according to the control attribute of the first gesture type further includes:

[0020] If the control attribute of the first gesture type is continuous, the gesture position of the nth frame remains unchanged relative to the gesture position of the (n-1)th frame.

[0021] Furthermore, the intermediate frame gesture type correction includes:

[0022] If the third gesture type is an unrecognizable gesture type and the fourth gesture type is a recognizable gesture type, then the gesture type of the kth frame is not corrected;

[0023] If the third gesture type is a recognizable gesture type, the gesture type of the kth frame is corrected to the third gesture type.

[0024] Furthermore, the correction method further includes:

[0025] If the gesture type is not recognized in the middle frame of the continuous n frames of images, the gesture type of the middle frame is set to an unrecognizable gesture type, and the continuous n frames of images are corrected according to the gesture type correction of the middle frame.

[0026] Furthermore, the correction method further includes:

[0027] If the gesture type is not recognized in the last frame of the continuous n frames of images, the gesture type of the last frame is set to an unrecognizable gesture type, and the continuous n frames of images are corrected according to the gesture type correction of the last frame.

[0028] Furthermore, the gesture type after correction of the last frame and the gesture type after correction of the middle frame are only used to calculate the control instruction within the calculation period of the last frame.

[0029] According to a second aspect of the present invention, a continuous frame correction device for gesture control comprises:

[0030] An acquisition module, used to acquire video and obtain continuous multi-frame images;

[0031] A recognition module, used to perform gesture recognition on each frame of image and obtain the gesture type of the frame; the gesture type includes an unrecognizable gesture type and a plurality of recognizable gesture types having different control attributes;

[0032] an end frame judgment module, used to judge whether the gesture type of the previous n-1 frames of the continuous n-frame image with the current frame as the end frame is the first gesture type, and the gesture type of the last n-th frame is the second gesture type, and used to judge whether the first gesture type and the second gesture type are recognizable gesture types and / or unrecognizable gesture types;

[0033] a last frame correction module, configured to correct the gesture type of the nth frame to an unrecognizable gesture type when the first gesture type is an unrecognizable gesture type and the second gesture type is a recognizable gesture type; and to correct the gesture type of the nth frame according to a control attribute of the first gesture type when the first gesture type is a recognizable gesture type;

[0034] An intermediate frame judgment module, used to judge whether the gesture type of the continuous n-frame image with the current frame as the last frame is the third gesture type for the first k-1 frames and the last nk frames, and the gesture type of the intermediate k-th frame is the fourth gesture type;

[0035] The intermediate frame correction module is used to correct the gesture type of the kth frame according to the third gesture type.

[0036] Furthermore, the control properties include switch type and continuous type;

[0037] A first processing unit is used to correct the gesture type of the nth frame according to the control attribute of the first gesture type, and to not correct the gesture type of the nth frame when the control attribute of the first gesture type is a switch type, and to correct the gesture type of the nth frame to the first gesture type when the control attribute of the first gesture type is a continuous type.

[0038] Furthermore, the recognition module is also used to obtain the position information of the gesture in each frame of the image;

[0039] The end frame correction module also includes:

[0040] The second processing unit is configured to set the gesture position of the nth frame relative to the gesture position of the (n-1)th frame to remain unchanged when the control attribute of the first gesture type is continuous.

[0041] Furthermore, the intermediate frame correction module includes:

[0042] The third processing unit is used for not correcting the gesture type of the kth frame when the third gesture type is an unrecognizable gesture type and the fourth gesture type is a recognizable gesture type, and for correcting the gesture type of the kth frame to the third gesture type when the third gesture type is a recognizable gesture type.

[0043] Furthermore, the intermediate frame correction module further includes:

[0044] The fourth processing unit is used to set the gesture type of the intermediate frame to an unrecognizable gesture type when the gesture type is not recognized in the intermediate frame of the continuous n frames of images, and to correct the continuous n frames of images according to the gesture type correction of the intermediate frame.

[0045] Furthermore, the end frame correction module further includes:

[0046] The fifth processing unit is used to set the gesture type of the last frame to an unrecognizable gesture type if the gesture type is not recognized in the last frame of the continuous n frames of images, and to correct the continuous n frames of images according to the gesture type correction of the last frame.

[0047] According to a third aspect of the present invention, an electronic device comprises a processor, a memory and a computer program stored in the memory and running on the processor, wherein when the computer program is executed by the processor, the continuous frame correction method for gesture control as described in the first aspect above is implemented.

[0048] According to a fourth aspect of the present invention, a computer-readable storage medium stores a computer program thereon, wherein the computer program is used to run to implement the continuous frame correction method for gesture control as described in the first aspect above.

[0049] The continuous frame correction method for gesture control of the present invention can effectively judge the end frame and the middle frame (i.e., the false detection frame) where abnormal mutations occur in continuous multi-frame images through preset judgment conditions, and can correct the gesture type of the false detection frame according to the gesture type and / or control attributes of the previous and next continuous frames, so that the gesture type changes continuously and there are no abnormal mutations, and the control instructions are calculated according to the corrected gesture type within the calculation cycle of each frame. Even if there are false detection frames in the continuous multi-frame images, it can be ensured that the correct control instructions are obtained, which can improve the accuracy of gesture control, enhance the effectiveness of gesture control, reduce the situation where the control gesture recognition is interfered with by the false detection frames during the video acquisition process, and help to improve the user experience during the remote gesture control process.

[0050] The continuous frame correction device for gesture control of the present invention adopts the above-mentioned continuous frame correction method for gesture control and is applied to a remote gesture control system. By judging the gestures of multiple consecutive frames of images, the last frame and the middle frame in which the gesture type undergoes abnormal mutation can be corrected, which can significantly improve the accuracy and effectiveness of gesture control and help improve the user experience.

[0051] The electronic device and computer-readable storage medium of the present invention, and the computer programs stored thereon are used to run to implement the above-mentioned continuous frame correction method for gesture control. When applied to the scenario of air gesture control, it can reduce the interference of false detection, frame loss and other problems in multiple frame images on gesture control, and can effectively improve the accuracy and effectiveness of gesture control.

[0052] Other beneficial effects of the present invention will be explained in the specific implementation manner through the introduction of specific technical features and technical solutions. Through the introduction of these technical features and technical solutions, those skilled in the art should be able to understand the beneficial technical effects brought about by the technical features and technical solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The following will describe a preferred embodiment of the continuous frame correction method for gesture control according to the present invention with reference to the accompanying drawings.

[0054] Figure 1 is a flow chart of a continuous frame correction method for gesture control of the present invention;

[0055] Figure 2 It is a schematic diagram of different gesture types between the last frame and the preceding multiple frames of the present invention;

[0056] Figure 3 It is a schematic diagram of different gesture types in the middle frame and the preceding and following frames of the present invention;

[0057] Figure 4 This is a schematic diagram of the present invention in which there is no gesture recognized in the middle frame. DETAILED DESCRIPTION

[0058] See also Figure 1 The present invention provides a continuous frame correction method for gesture control, which is used in an air gesture control system. The correction method comprises the following steps:

[0059] S100, obtaining n consecutive frames of images from the acquired original video. Among them, the acquisition module is used to shoot a video within a preset range and obtain consecutive multiple frames of images. In the air gesture control system, the control instructions are generally calculated comprehensively through multiple consecutive frames of gesture data. For example, in the calculation cycle of the mth frame, the gesture types of the consecutive n frames preceding the mth frame (i.e., the m-n+1th frame to the mth frame) are required to determine the gesture changes and calculate the control instructions. It should be noted that in the calculation cycle of each frame, the gesture types of n consecutive frames are all from the gesture data in the original image.

[0060] S200, performing gesture recognition on each frame of image and acquiring a gesture type of the frame, wherein the gesture type includes an unrecognizable gesture type and a plurality of recognizable gesture types having different control attributes.

[0061] Among them, gesture recognition is performed on each frame of the image through the recognition module. If the gesture can be recognized, the corresponding gesture type is obtained. Generally, the recognizable gesture type is generally used for gestures that need to control the system (i.e., control gestures). There are many recognizable gesture types and they can be assigned different control attributes. Unrecognizable gesture types are used for non-control gestures.

[0062] S300, performing gesture type correction for the last frame and gesture type correction for the middle frame on the continuous n-frame image with the current frame as the last frame, wherein:

[0063] The final frame gesture type correction includes: Figure 2 , if the gesture type of the first n-1 frames is the first gesture type, and the gesture type of the last n-th frame is the second gesture type, then

[0064] If the first gesture type is an unrecognizable gesture type, and the second gesture type is a recognizable gesture type, the gesture type of the nth frame is corrected to an unrecognizable gesture type;

[0065] If the first gesture type is a recognizable gesture type, the gesture type of the nth frame is corrected according to the control attribute of the first gesture type.

[0066] The intermediate frame gesture type correction includes: Figure 3 If the gestures of the first k-1 frames and the last nk frames are both of the third gesture type, and the gesture type of the middle k-th frame is of the fourth gesture type, the gesture type of the k-th frame is corrected according to the third gesture type.

[0067] It should be noted that the first gesture type and the second gesture type here are used to illustrate two different gesture types during the end frame correction process. Similarly, the third gesture type and the fourth gesture type are used to illustrate two different gesture types during the intermediate frame correction process. However, it is not limited to the first gesture type and the third gesture type must be different, nor is it limited to the second gesture type and the fourth gesture type must be different.

[0068] Through the above steps, gesture recognition is performed on continuous multiple-frame images and corresponding gesture types are obtained, and when the change in gesture type meets the preset conditions, the false detection frames in the continuous multiple-frame images are judged. If it is found that the gesture type of a certain image frame has an abnormal mutation, the gesture type of the image frame is corrected according to the gesture types of the continuous image frames before and after the image frame and / or the control attributes of the gesture type, which can reduce the adverse effects of gesture control caused by the existence of false detection frames in the multiple-frame images. The corrected gesture type is used to calculate the control instruction to ensure the accuracy of the control instruction, and also significantly improves the effectiveness of gesture control.

[0069] As an optional embodiment, the control attribute includes a switching type and a continuous type.

[0070] S310, correcting the gesture type of the nth frame according to the control attribute of the first gesture type includes:

[0071] If the control attribute of the first gesture type is a switch type, the gesture type of the nth frame is not corrected;

[0072] If the control attribute of the first gesture type is continuous, the gesture type of the nth frame is corrected to the first gesture type.

[0073] Among them, the switch type is used to execute the switch type instruction operation when the gesture meets certain trigger conditions. For example, the trigger condition can be to keep the fist state and be still for 3 seconds, move 10cm to the right in the fist state, change from the fist state to the palm state, etc. The switch type instruction can be to turn on / off the screen, mute / unmute, start recording / turn off recording, etc. The continuous type is used to execute the continuous type instruction operation through the position information of the gesture when the gesture maintains a specific gesture type. For example, the trigger condition can be to move while keeping the fist state, to rotate while keeping the thumb up, etc. The position information of the gesture can be the movement amount of the hand or the rotation amount of the hand. The continuous type instruction can be to adjust the backlight brightness, adjust the volume, move the cursor, etc.

[0074] If the control attribute of the first gesture type is a switch type, the gesture type of the nth frame is temporarily ignored, and the judgment is made when the n+1th frame is the current frame. This is equivalent to taking the nth frame as the intermediate frame for correction processing within the calculation cycle of the n+1th frame. Specifically, if the gesture type of the n+1th frame (as the last frame) is also the first gesture type, the gesture type of the nth frame is corrected to the first gesture type; if the gesture type of the n+1th frame is another recognizable gesture type or an unrecognizable gesture type, it is considered that the gesture used for control has made an active change, and therefore the gesture type of the nth frame (as the intermediate frame) is not corrected.

[0075] If the control attribute of the first gesture type is continuous, the gesture type of the nth frame is corrected to the first gesture type.

[0076] Through the above steps, the last false detection frame where an abnormal gesture type mutation occurs can be identified in multiple consecutive image frames, and the gesture type of the last false detection frame can be corrected according to the control attribute of the gesture type of the preceding image frame of the last false detection frame. Thus, the corrected gesture type of the last false detection frame is used for the calculation of control instructions, which can ensure that the correct control instructions are obtained and can effectively reduce the interference of the last false detection frame on the control gesture processing, which helps to improve the effectiveness of gesture control.

[0077] As an optional embodiment, the performing gesture recognition on each frame of image further includes obtaining position information of the gesture in the frame; and the correcting the gesture type of the nth frame according to the control attribute of the first gesture type further includes:

[0078] S320: If the control attribute of the first gesture type is continuous, the gesture position of the nth frame remains unchanged relative to the gesture position of the (n-1)th frame.

[0079] When the gesture type of the nth frame is corrected to the first gesture type, considering that the first gesture type is continuous, the movement or rotation amount of the position information of the gesture of the nth frame relative to the position information of the gesture of the n-1th frame is set to 0.

[0080] Through the above steps, when judging the misdetected frame at the end and making corresponding correction processing, the influence of possible miscontrol in the correction processing can be effectively limited through timely correction of position information.

[0081] As an optional embodiment, the intermediate frame gesture type correction includes:

[0082] S330: If the third gesture type is an unrecognizable gesture type and the fourth gesture type is a recognizable gesture type, the gesture type of the kth frame is not corrected. If the third gesture type is a recognizable gesture type, regardless of whether the fourth gesture type is a recognizable gesture type or an unrecognizable gesture type, the gesture type of the kth frame is corrected to the third gesture type.

[0083] It should be noted that, whether it is switch type control or continuous type control, a specific gesture sequence must be met to trigger the corresponding operation, that is, the gestures in multiple consecutive frames of images must meet certain shapes and sequences. The presence of a recognizable gesture type in only one frame of the image will not trigger the control instruction. Therefore, in the above case, the gesture type of the kth frame is not corrected. If forced correction is performed, it will affect the calculation of the control instructions in the subsequent image frame calculation cycle.

[0084] Through the above steps, the intermediate false detection frame in which the gesture type undergoes an abnormal mutation can be identified in multiple consecutive image frames, and the gesture type of the intermediate false detection frame can be corrected according to the gesture types of the consecutive image frames before and after the intermediate false detection frame. The gesture type of the intermediate false detection frame after correction is used for the calculation of control instructions, which can ensure that the correct control instructions are obtained and can effectively reduce the interference of the intermediate false detection frame on the control gesture processing, which helps to improve the effectiveness of gesture control.

[0085] As an optional embodiment, the correction method further includes:

[0086] S340, see Figure 4 If the gesture type is not recognized in the middle frame of the continuous n-frame image, it is considered that the middle frame has been lost, and the gesture type of the middle frame is set to an unrecognizable gesture type, and the continuous n-frame image is corrected according to the above-mentioned middle frame gesture type correction.

[0087] Among them, since no gesture is detected in the middle k-th frame, the gesture type in the frame cannot be obtained, which affects the processing of other image frames. Therefore, in this case, the gesture type of the k-th frame is first set to an unrecognizable gesture type, that is, the k-th frame is used as the middle false detection frame, and the gesture type of the k-th frame is corrected according to the above-mentioned method of correcting the gesture type of the middle frame.

[0088] Through the above steps, it is possible to determine that an intermediate frame in a continuous plurality of image frames has been lost, and the intermediate frame can be processed as a false detection frame. The specific correction processing steps are the same as those for the intermediate false detection frame, and the processing logic remains consistent, which is easy to implement and can effectively reduce the interference of frame loss on control gesture processing, thereby helping to improve the effectiveness of gesture control.

[0089] As an optional embodiment, the correction method further includes:

[0090] S350: If the gesture type is not recognized in the last frame of the continuous n-frame images, the gesture type of the last frame is set to an unrecognizable gesture type, and the continuous n-frame images are corrected according to the gesture type correction of the last frame.

[0091] Through the above steps, the frame loss in the last frame can be discovered in time, the gesture type of the last frame can be corrected to an unrecognizable gesture type in time within the calculation cycle of the nth frame, and the last frame gesture type correction method can be used to perform corresponding correction on the last frame, which can improve the timeliness of the last frame correction and reduce the probability of control gesture recognition errors caused by last frame loss.

[0092] As an optional embodiment, the gesture type after correction of the last frame and the gesture type after correction of the middle frame are only used to calculate the control instruction within the calculation period of the last frame.

[0093] Therefore, the gesture type after correction of the last frame and the gesture type after correction of the middle frame obtained through the above steps are only used in the calculation cycle of the last frame, which can effectively control the influence on the gesture judgment and calculation results of the subsequent image frames, avoid the accumulation and amplification of correction errors, and even if correction errors occur, their influence on the entire processing process of gesture control is relatively small.

[0094] The present invention also provides a continuous frame correction device for gesture control, which is applied to an air gesture control system. The device includes an acquisition module, a recognition module, a last frame judgment module, a last frame correction module, an intermediate frame judgment module and an intermediate frame correction module.

[0095] The acquisition module is used to capture video and obtain continuous multi-frame images.

[0096] The recognition module is used to perform gesture recognition on each frame of image and obtain the gesture type of the frame, wherein the gesture type includes an unrecognizable gesture type and a plurality of recognizable gesture types having different control properties.

[0097] The end frame judgment module is used to judge whether the gesture type of the continuous n frames of images with the current frame as the end frame is the first gesture type, and the gesture type of the last n frame is the second gesture type, and is used to judge whether the first gesture type and the second gesture type are recognizable gesture types and / or unrecognizable gesture types.

[0098] The end frame correction module is used to correct the gesture type of the nth frame to an unrecognizable gesture type when the first gesture type is an unrecognizable gesture type and the second gesture type is a recognizable gesture type; and to correct the gesture type of the nth frame according to the control attributes of the first gesture type when the first gesture type is a recognizable gesture type.

[0099] The intermediate frame judgment module is used to judge whether the gesture type of the continuous n frames of images with the current frame as the last frame is the third gesture type for the first k-1 frames and the last nk frames, and the gesture type of the intermediate kth frame is the fourth gesture type.

[0100] The intermediate frame correction module is used to correct the gesture type of the kth frame according to the third gesture type.

[0101] It should be noted that the end frame determination module and the middle frame determination module can be relatively independent modules or one module. Similarly, the end frame correction module and the middle frame correction module can be relatively independent modules or one module.

[0102] Therefore, the continuous frame correction device is applied to the air gesture control system, which can determine the last false detection frame and the middle false detection frame in multiple continuous frames of images, and can correct the gesture type of the last false detection frame and the middle false detection frame according to the gesture type and / or control attributes of the continuous image frames before and after the false detection frame, thereby improving the accuracy and effectiveness of gesture control.

[0103] As an optional embodiment, the control attribute includes a switch type and a continuous type. The switch type is used to execute the operation of the switch type instruction when the gesture meets certain trigger conditions. The continuous type is used to execute the operation of the continuous type instruction through the position information of the gesture when the gesture maintains a specific gesture type.

[0104] The end frame correction module includes a first processing unit.

[0105] The first processing unit is used to correct the gesture type of the nth frame according to the control attributes of the first gesture type, and to not correct the gesture type of the nth frame when the control attributes of the first gesture type are switch type, and to correct the gesture type of the nth frame to the first gesture type when the control attributes of the first gesture type are continuous type.

[0106] Therefore, the correction module in the device can determine the last false detection frame in which the gesture type undergoes an abnormal mutation in multiple consecutive frames of images, and correct the gesture type of the last false detection frame according to the control attribute of the gesture type of the preceding image frame of the last false detection frame. The corrected gesture type of the last false detection frame is used for the calculation of control instructions, which can ensure that the correct control instructions are obtained and can reduce the interference of the last false detection frame on the control gesture processing, which helps to improve the effectiveness of gesture control.

[0107] As an optional embodiment, the recognition module is further used to obtain the position information of the gesture of each frame image. The end frame correction module also includes a second processing unit.

[0108] The second processing unit is used for setting the gesture position of the nth frame to remain unchanged relative to the gesture position of the (n-1)th frame when the control attribute of the first gesture type is continuous.

[0109] As an optional embodiment, the intermediate frame correction module includes a third processing unit.

[0110] The third processing unit is used for not correcting the gesture type of the kth frame when the third gesture type is an unrecognizable gesture type and the fourth gesture type is a recognizable gesture type, and for correcting the gesture type of the kth frame to the third gesture type when the third gesture type is a recognizable gesture type.

[0111] Therefore, the intermediate frame correction module can determine the intermediate false detection frame in which the gesture type undergoes an abnormal mutation in the continuous multiple frame images, and correct the gesture type of the intermediate false detection frame according to the gesture types of the continuous image frames before and after the intermediate false detection frame. The gesture type of the intermediate false detection frame after correction is used for the calculation of control instructions, which can ensure that the correct control instructions are obtained, and can effectively reduce the interference of the intermediate false detection frame on the control gesture processing, which helps to improve the effectiveness of gesture control.

[0112] As an optional embodiment, the intermediate frame correction module further includes a fourth processing unit.

[0113] The fourth processing unit is used to set the gesture type of the intermediate frame to an unrecognizable gesture type when the gesture type is not recognized in the intermediate frame of the continuous n frames of images, and to correct the continuous n frames of images according to the gesture type correction of the intermediate frame.

[0114] Therefore, the intermediate frame correction module can determine that the intermediate frame has been lost in multiple consecutive frames of images, and process the intermediate frame as a false detection frame. The specific correction processing steps are the same as those for the intermediate false detection frame, and the processing logic remains consistent, which is easy to implement. It can also effectively reduce the interference of frame loss on control gesture processing, which helps to improve the effectiveness of gesture control.

[0115] As an optional embodiment, the last frame correction module further includes a fifth processing unit.

[0116] The fifth processing unit is used to set the gesture type of the last frame as an unrecognizable gesture type if the gesture type is not recognized in the last frame of the continuous n frames of images, and to correct the continuous n frames of images according to the gesture type correction of the last frame.

[0117] Therefore, the last frame correction module can promptly detect the frame loss in the last frame, promptly correct the gesture type of the last frame to an unrecognizable gesture type within the calculation cycle of the nth frame, and use the last frame gesture type correction method to make corresponding corrections to the last frame, which can improve the timeliness of the last frame correction and reduce the probability of control gesture recognition errors caused by last frame loss.

[0118] The present invention also provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and running on the processor, wherein the computer program, when executed by the processor, implements the continuous frame correction method for gesture control described in the above embodiment.

[0119] The present invention further provides a computer-readable storage medium on which a computer program is stored. The computer program is used to run to implement the continuous frame correction method for gesture control described in the above embodiment.

[0120] Those skilled in the art will appreciate that, without conflict, the above-mentioned preferred solutions can be freely combined and superimposed.

[0121] It should be understood that the above-mentioned embodiments are merely illustrative and not restrictive. Without departing from the basic principles of the present invention, various obvious or equivalent modifications or substitutions that can be made by those skilled in the art to the above-mentioned details will all be included in the scope of the claims of the present invention.

Claims

1. A continuous frame correction method for gesture control, characterized in that: The calibration method comprises the following steps: Obtain n consecutive frames of images from the acquired original video; Performing gesture recognition on each frame of image and acquiring the gesture type of the frame, wherein the gesture type includes an unrecognizable gesture type and a plurality of recognizable gesture types having different control attributes; The gesture type correction of the last frame and the gesture type correction of the middle frame are performed on the continuous n-frame images with the current frame as the last frame, wherein: The last frame gesture type correction includes: if the gesture types of the first n-1 frames are all the first gesture type, and the gesture type of the last n-th frame is the second gesture type, then If the first gesture type is an unrecognizable gesture type, and the second gesture type is a recognizable gesture type, the gesture type of the nth frame is corrected to an unrecognizable gesture type; If the first gesture type is a recognizable gesture type, correcting the gesture type of the nth frame according to the control attribute of the first gesture type; The intermediate frame gesture type correction includes: if the gesture types of the first k-1 frames and the last nk frames are both the third gesture type, and the gesture type of the intermediate kth frame is the fourth gesture type, then correcting the gesture type of the kth frame according to the third gesture type.

2. The continuous frame correction method for gesture control according to claim 1, characterized in that: The control properties include switch type and continuous type; The correcting the gesture type of the nth frame according to the control attribute of the first gesture type includes: If the control attribute of the first gesture type is a switch type, no correction is performed on the gesture type of the nth frame; If the control attribute of the first gesture type is continuous, the gesture type of the nth frame is corrected to the first gesture type.

3. The continuous frame correction method for gesture control according to claim 2, characterized in that: The performing gesture recognition on each frame of image further includes obtaining position information of the gesture in the frame; The correcting the gesture type of the nth frame according to the control attribute of the first gesture type further includes: If the control attribute of the first gesture type is continuous, the gesture position of the nth frame remains unchanged relative to the gesture position of the (n-1)th frame.

4. The continuous frame correction method for gesture control according to claim 1, characterized in that: The intermediate frame gesture type correction includes: If the third gesture type is an unrecognizable gesture type and the fourth gesture type is a recognizable gesture type, then the gesture type of the kth frame is not corrected; If the third gesture type is a recognizable gesture type, the gesture type of the kth frame is corrected to the third gesture type.

5. The continuous frame correction method for gesture control according to claim 1, characterized in that: The correction method further comprises: If the gesture type is not recognized in the middle frame of the continuous n frames of images, the gesture type of the middle frame is set to an unrecognizable gesture type, and the continuous n frames of images are corrected according to the gesture type correction of the middle frame.

6. The continuous frame correction method for gesture control according to claim 1, characterized in that: The correction method further comprises: If the gesture type is not recognized in the last frame of the continuous n frames of images, the gesture type of the last frame is set to an unrecognizable gesture type, and the continuous n frames of images are corrected according to the gesture type correction of the last frame.

7. The continuous frame correction method for gesture control according to any one of claims 1 to 6, characterized in that: The gesture type after correction of the last frame and the gesture type after correction of the middle frame are only used to calculate the control instruction in the calculation period of the last frame.

8. A continuous frame correction device for gesture control, characterized in that: The correction device comprises: An acquisition module, used to acquire video and obtain continuous multi-frame images; A recognition module, used to perform gesture recognition on each frame of image and obtain the gesture type of the frame; the gesture type includes an unrecognizable gesture type and a plurality of recognizable gesture types having different control attributes; an end frame judgment module, used to judge whether the gesture type of the previous n-1 frames of the continuous n-frame image with the current frame as the end frame is the first gesture type, and the gesture type of the last n-th frame is the second gesture type, and used to judge whether the first gesture type and the second gesture type are recognizable gesture types and / or unrecognizable gesture types; a last frame correction module, configured to correct the gesture type of the nth frame to an unrecognizable gesture type when the first gesture type is an unrecognizable gesture type and the second gesture type is a recognizable gesture type; and to correct the gesture type of the nth frame according to a control attribute of the first gesture type when the first gesture type is a recognizable gesture type; An intermediate frame judgment module, used to judge whether the gesture type of the continuous n-frame image with the current frame as the last frame is the third gesture type for the first k-1 frames and the last nk frames, and the gesture type of the intermediate k-th frame is the fourth gesture type; The intermediate frame correction module is used to correct the gesture type of the kth frame according to the third gesture type.

9. An electronic device comprising a processor, a memory, and a computer program stored in the memory and running on the processor, characterized in that: When the computer program is executed by the processor, the continuous frame correction method for gesture control according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is used to run to implement the continuous frame correction method for gesture control according to any one of claims 1 to 7.

Images