A device control method and system based on gesture recognition

Abstract

The application relates to a gesture recognition-based device control method and system, which comprises the following steps: defining a gesture continuous loss image frame number and a motion category variable corresponding to different gestures; obtaining a kth image from a real-time video stream and comparing the kth image with a standard gesture image database to obtain a gesture recognition result; and performing a corresponding device control motion process according to the gesture recognition result, the gesture continuous loss image frame number and the motion category variable corresponding to different gestures. The application can avoid system misjudgment caused by different gestures or no gesture, and can ensure that even if a small number of frame images cannot detect gestures, the gesture motion recognition is not affected, so that the device control sensitivity can be improved, and incomplete gesture motions can be filtered out, and mistriggering can be reduced.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a device control method and system based on gesture recognition. Background Technology

[0002] Currently, in technical solutions for controlling related devices (such as controlling in-vehicle equipment like air conditioners and audio systems) through gesture recognition, misjudgments of gestures are prone to occur. For example, during the process of gesture recognition using infrared imaging, a human head may be misjudged as a fist. Furthermore, due to the existence of gesture misjudgments, it is impossible to accurately control the devices, resulting in a poor user experience and, in severe cases, safety hazards. For instance, if a misjudgment of a gesture leads to the accidental touch of certain devices while the vehicle is in motion, it may cause the driver's attention to be diverted from driving, creating a driving safety hazard. Summary of the Invention

[0003] The purpose of this invention is to provide a device control method and system based on gesture recognition, which can avoid system misjudgment caused by different gestures or no gestures, and can ensure that even if a small number of frames do not detect gestures, it will not affect gesture action recognition. This can improve the sensitivity of device control, and can also filter out incomplete gesture actions and reduce false triggering.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] On the one hand, a device control method based on gesture recognition is provided, which includes the following steps:

[0006] The k-th frame image is acquired from the real-time video stream and compared with the standard gesture image database to obtain the gesture recognition result;

[0007] Based on the gesture recognition result, the number of consecutive lost image frames for the current gesture (GestureLoss) and / or the action category variable (GestureX) corresponding to different gestures are updated, and the corresponding device control action flow is executed; wherein, the number of consecutive lost image frames for the gesture (GestureLoss) is the number of consecutive image frames for which the gesture is not recognized or the number of consecutive image frames for which the same gesture category is not recognized.

[0008] Preferably, based on the gesture recognition result, the number of consecutively lost image frames (GestureLoss) for the current gesture and / or the action category variable (GestureX) corresponding to different gestures are updated, and the corresponding device control action flow is executed, including the following steps:

[0009] If the gesture recognition result indicates that the k-th frame contains different gesture actions or does not contain any gesture actions, then determine whether the number of consecutive lost image frames for the current gesture, GestureLoss, is greater than or equal to the first threshold T1.

[0010] If the number of consecutive lost image frames for the current gesture is greater than or equal to the first threshold T1, then the number of consecutive lost image frames for the gesture, GestureLoss, and all action category variables, GestureX, are cleared to zero, and the device's working state remains unchanged.

[0011] If the number of consecutive image frames lost during the current gesture is less than the first threshold T1, then the number of consecutive image frames lost during the current gesture is incremented by 1, and the device's operating state remains unchanged.

[0012] Preferably, based on the gesture recognition result, the number of consecutively lost image frames (GestureLoss) for the current gesture and / or the action category variable (GestureX) corresponding to different gestures are updated, and the corresponding device control action flow is executed, including the following steps:

[0013] If the gesture recognition result indicates that the k-th frame contains a single gesture, the number of consecutive lost image frames for the current gesture, GestureLoss, is reset to zero, and the corresponding device control action process is executed according to the category of the single gesture.

[0014] Preferably, the corresponding device control action flow is executed according to the category of the single gesture action, including the following steps:

[0015] If the action category variable GestureX corresponding to the current gesture is 0, then record the center position of the image where the current gesture is located, Gesture_location, increment the action category variable GestureX by 1, and do not generate a device control signal.

[0016] Preferably, the corresponding device control action flow is executed according to the category of the single gesture action, including the following steps:

[0017] If 0 < GestureX, the action category variable corresponding to the current gesture, < the second threshold T2, then determine whether the current gesture has moved relative to the previous frame image.

[0018] If it is assumed that no movement has occurred during the current gesture, GestureX is incremented by 1, and no device control signal is generated;

[0019] If it is assumed that the current gesture has moved, and the occurrence of the current gesture is not part of the device function trigger action, then GestureX will be cleared and no device control signal will be generated.

[0020] Preferably, the corresponding device control action flow is executed according to the category of the single gesture action, including the following steps:

[0021] If the action category variable GestureX corresponding to the current gesture is greater than or equal to the second threshold T2, then the number of consecutive lost image frames GestureLoss and all action category variables GestureX are cleared to zero, and a device control signal is generated, so that the device can switch its working state according to the device control signal.

[0022] Preferably, the switching of the working state is achieved by controlling discrete variables.

[0023] Preferably, the corresponding device control action flow is executed according to the category of the single gesture action, including the following steps:

[0024] Once the device is turned on, the device's operating status is switched based on whether a movement gesture is detected.

[0025] Preferably, after the device has entered the powered-on state, the process of switching the device's operating state based on whether a movement gesture is observed includes the following steps:

[0026] If the action category variable GestureMove corresponding to the current gesture is 0, then record the starting position of the current gesture GestureMove_location, increment GestureMove by 1, and do not generate a device control signal.

[0027] Preferably, after the device has entered the powered-on state, the process of switching the device's operating state based on whether a movement gesture is observed includes the following steps:

[0028] If the action category variable GestureMove corresponding to the current gesture is 1, then the center position of the image where the current gesture is located is obtained, and the first movement direction Move_Direction_first of the current gesture is determined by comparing it with the center position of the image where the gesture is located when the action category variable GestureMove is 0, GestureMove_location. Then, GestureMove is incremented by 1, and no device control signal is generated.

[0029] Preferably, after the device has entered the powered-on state, the process of switching the device's operating state based on whether a movement gesture is observed includes the following steps:

[0030] If 1 < GestureMove, the action category variable corresponding to the current gesture, < the third threshold T3, then determine whether the current gesture has moved relative to the previous frame image;

[0031] If it is assumed that no movement has occurred during the current gesture, no device control signal will be generated;

[0032] If it is believed that the current gesture has moved, then the direction of the next movement of the current gesture, Move_Direction, is compared with the direction of the first movement, Move_Direction_first.

[0033] If the angle between the current gesture's subsequent movement path and the initial movement path is 0±20°, then GestureMove is incremented by 1, and no device control signal is generated; if the current gesture's subsequent movement direction is inconsistent with the reference movement direction Move_Direction, then GestureMove is cleared to zero, and no device control signal is generated.

[0034] Preferably, after the device has entered the powered-on state, the process of switching the device's operating state based on whether a movement gesture is observed includes the following steps:

[0035] If the action category variable GestureMove corresponding to the current gesture is greater than or equal to the third threshold T3, then the action category variable GestureMove is cleared to zero, and a device control signal is generated, so that the device can switch working states according to the device control signal.

[0036] Preferably, the switching of the working state is achieved by controlling continuous variables.

[0037] On the other hand, a device control system for implementing the above-described device control method is also provided, comprising:

[0038] A database used to store images of different standard gestures;

[0039] Imaging equipment used to acquire real-time video streams;

[0040] A gesture recognition unit is used to acquire a frame image from a real-time video stream and compare it with the standard gesture image database to obtain a gesture recognition result;

[0041] And a device control unit, which is used to execute the corresponding device control action process based on the gesture recognition result.

[0042] In summary, the present invention has the following advantages compared with the prior art:

[0043] This invention can execute the corresponding device action control process by changing the number of consecutive lost image frames (GestureLoss) and the value of the action category variable (GestureX) based on the gesture. This can avoid system misjudgment caused by different gestures or no gestures, and ensure that even if a small number of frames do not detect gestures, it will not affect gesture recognition. This can improve the device control sensitivity and filter out incomplete gestures, reducing false triggers. Attached Figure Description

[0044] Figure 1 This is a flowchart of the steps of the device control method based on gesture recognition disclosed in this invention.

[0045] Figure 2 Infrared images for recognizing different hand gestures.

[0046] Figure 3 Infrared images that did not identify any hand gestures.

[0047] Figure 4 Infrared images for recognizing single hand gestures.

[0048] Figure 5 This is a schematic diagram of the device control system based on gesture recognition disclosed in this invention. Detailed Implementation

[0049] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0050] Example 1

[0051] like Figure 1 As shown, this embodiment provides a device control method based on gesture recognition. In this embodiment, the device can be installed on a mobile vehicle (such as a vehicle) or on a stationary device. The device can adjust and control its own functions through signal interaction, such as starting and stopping the air conditioner, adjusting the air conditioning output, and adjusting the volume of the audio equipment. Specifically, the device control method includes the following steps:

[0052] S1. Under different angles, backgrounds, and regions, images of different standard gestures are captured by an imaging device, and a standard gesture image database is established as the basis for subsequent gesture recognition. The imaging device includes one or more of visible light imaging devices and infrared imaging devices.

[0053] Meanwhile, a gesture loss image frame count (GestureLoss) is defined, which is the number of consecutively unrecognized image frames or the number of consecutively unrecognized image frames of the same gesture category. Additionally, a motion category variable (GestureX) is defined for each gesture. X is assigned a different name based on the gesture category, and different motion category variables (GestureX) correspond to different device control motion flows. For example, the motion category variable for a turn-off gesture is GestureStop, corresponding to the device turn-off motion flow; the motion category variable for a turn-on gesture is GestureStart, corresponding to the device turn-on motion flow; and the motion category variable for a movement gesture is GestureMove, etc., corresponding to the device control flow completed through the movement motion.

[0054] Furthermore, in the initial state, the number of consecutive image frames lost due to gestures is GestureLoss=0, and the variable GestureX for any action category is 0;

[0055] Furthermore, to avoid misrecognition of gestures caused by other objects, such as the fact that a human head is often identified as a fist gesture in infrared thermal imaging, and both will only appear as a patch of the same color in infrared thermal imaging, causing inaccurate action recognition, this embodiment needs to collect corresponding gesture images under different angles, different backgrounds, and different areas to reduce gesture recognition errors.

[0056] S2. Obtain the k-th frame image from the real-time video stream and compare it with the standard gesture image database to obtain the gesture recognition result, where k is a positive integer;

[0057] The gesture results in the same frame image can contain one or more detection result boxes; the information of each detection result box includes gesture action category, confidence level, coordinates of the top left point of the box, and width and height, etc.

[0058] S3. Based on the gesture recognition result, update the number of consecutively lost image frames (GestureLoss) for the current gesture and / or the action category variable (GestureX) corresponding to different gestures, and execute the corresponding device control action flow, which specifically includes the following steps:

[0059] 3.1 If the gesture recognition result indicates that the k-th frame image contains different gestures (e.g., ... Figure 2 (as shown), or does not include any hand gestures (such as...) Figure 3 If the current gesture continuously loses image frames (as shown), then it is further determined whether the number of consecutive lost image frames GestureLoss is greater than or equal to the first threshold T1 (the unit is the number of image frames, such as 5 images, etc.).

[0060] If the number of consecutive lost image frames for the current gesture is greater than or equal to the first threshold T1, it means that the target gesture (i.e. the gesture used for device control) has not been detected for a number of consecutive frames, and the duration of the target gesture loss exceeds the tolerable duration. At this time, the number of consecutive lost image frames for the gesture, GestureLoss, and all action category variables, GestureX, are cleared to zero, and the device working state does not change.

[0061] If the number of consecutive image frames lost by the current gesture is less than the first threshold T1, then the number of consecutive image frames lost by the current gesture is incremented by 1, and the device working state does not change. Then the current device control action process is completed, and steps S2-S3 are repeated.

[0062] 3.2 If the gesture recognition result indicates that the k-th frame image contains a single gesture (e.g., ... Figure 4 If the gesture (as shown) is continuously lost (the number of image frames lost) is reset to zero (GestureLoss), the corresponding device control action flow is executed according to the category of the single gesture action. The specific steps include the following:

[0063] 3.2.1 If the action category variable GestureX corresponding to the current gesture is 0, it means that the current gesture is appearing for the first time and it is uncertain whether the gesture is a device control action. At this time, record the center position Gesture_location of the image where the current gesture is located (i.e. the k-th frame image), increment the action category variable GestureX by 1, and do not generate a device control signal. Then the current device control action process is completed, and repeat steps S2-S3.

[0064] 3.2.2 If 0 < GestureX, the action category variable corresponding to the current gesture, < the second threshold T2, it means that the current gesture has lasted for a certain period of time, but the corresponding function of the device has not yet been triggered. Further determine whether the current gesture has moved relative to the previous frame image, and execute the corresponding device control process according to the determination result. The determination of whether the gesture has moved includes the following steps: obtain the distance between the center position of the image where the current gesture is located (i.e., the k+n frame image, where n is a positive integer greater than or equal to 2) and the center position Gesture_location of the image where the gesture is located when the action category variable GestureX = 0. If the distance meets the first condition (e.g., less than or equal to a preset value), it is considered that the current gesture has not moved and the appearance of the current gesture is part of the device function control action. At this time, GestureX is incremented by 1, no device control signal is generated, and the current device control action process is completed. Steps S2-S3 are repeated. If the distance meets the second condition (e.g., greater than or equal to a preset value), it is considered that the current gesture has moved and the appearance of the current gesture is not part of the device function trigger action. At this time, GestureX is cleared to zero, no device control signal is generated, and the current device control action process is completed. Steps S2-S3 are repeated again.

[0065] 3.2.3 If the action category variable GestureX corresponding to the current gesture is greater than or equal to the second threshold T2 (the unit is the number of frames of the image, such as 5 frames), it means that the current gesture has a long duration and the gesture is the corresponding device control gesture. At this time, the number of consecutive lost image frames GestureLoss and all action category variables GestureX are cleared to zero, and a device control signal is generated so that the device can switch the working state according to the device control signal. Then the current device control action process is completed. The switching of the working state can be achieved by controlling discrete variables, such as the device switching from the closed state (0) to the open state (1), or from the open state (1) to the closed state (0), etc.

[0066] For example, if the current gesture is a palm, define its corresponding action category variable as GestureStart. When the palm gesture appears and does not move within a predetermined time, then execute the device turn-on function (such as turning on the car air conditioner).

[0067] Based on this, when the hand gesture is detected for the first time and it is determined that the device is not in the powered-on state, GestureStart = 0. However, at this time it is impossible to determine whether the gesture is a device function trigger action (it may be a misidentification). Therefore, at this time, the center position of the image where the hand is located, Gesture_location, is recorded, and GestureStart is incremented by 1, that is, GestureStart = 1 at this time, and no device control signal is generated.

[0068] Repeat step S2. If a palm is detected again, and 0 < GestureStart (GestureStart = 1) < the second threshold T2 (e.g., 5 images), it means that the palm gesture has appeared and lasted for a certain period of time, but its duration is insufficient to turn on the device. At this time, obtain the distance between the center position of the image where the palm is located and the center position of the image where the gesture is located when the action category variable GestureStart = 0. If the distance meets the first condition (e.g., less than or equal to the preset value), it is considered that the palm has not moved and the appearance of the palm is part of the device opening function triggering action. At this time, increment GestureStart by 1 and do not generate a device control signal. If the distance meets the second condition (e.g., greater than or equal to the preset value), it is considered that the palm has moved and the appearance of the palm is not part of the device opening action. At this time, clear GestureStart to zero and do not generate a device control signal.

[0069] Repeat step S2 and the above operation of incrementing GestureStart by 1 until a palm is detected again. If GestureStart is greater than or equal to the second threshold T2 (e.g., 5 palms), it indicates that the palm action has lasted for a long time and has not moved within the predetermined time. This can trigger the device to turn on. At this time, GestureStart is cleared to zero and a device turn-on signal is generated. The device switches its working state according to the device control signal, such as switching a car air conditioner from the off state to the on state.

[0070] For example, if the current gesture is a fist, and its corresponding action category variable is defined as GestureStop, when the fist gesture appears and no movement occurs within a predetermined time, the device will be turned off (such as turning off the car air conditioner). Similarly, the device can be turned off by changing the value assigned to GestureStop. The process is the same as the process of turning on the device by using a palm gesture, and will not be repeated here.

[0071] In this embodiment, when different gestures or no gestures appear in the same frame, the occurrence of multiple gestures may be due to false detection, multiple detection, or improper user operation. If multiple gestures appear, the control command cannot be clear. The absence of gestures may be due to the failure to detect the gesture target or the user interrupting the gesture control behavior. In all of the above situations, it is impossible to determine whether a single target gesture exists. Therefore, this embodiment considers all of them as gesture loss and counts the number of consecutively lost image frames (GestureLoss). If the number of consecutively lost image frames (GestureLoss) reaches a set threshold, which means that the gesture has been continuously lost for a period of time, the number of consecutively lost image frames (GestureLoss) and all action category variables (GestureX) are cleared to zero, and gesture action recognition restarts. Otherwise, the number of consecutively lost image frames (GestureLoss) is incremented by 1, and gesture action recognition continues in the next frame. This avoids system misjudgment caused by different gestures or the absence of gestures.

[0072] When a single gesture action target is present, the condition of continuous gesture loss is not met. Therefore, the number of consecutively lost image frames (GestureLoss) is reset to zero to eliminate its impact on action recognition.

[0073] Missed or false recognition of gesture targets can affect the recognition of device actions. Deep learning cannot achieve 100% accuracy. However, by setting the number of consecutive lost image frames for gestures as mentioned above, it can be ensured that even if a small number of frames do not detect the gesture, it will not affect the recognition of the gesture action. This can improve the sensitivity of device control and filter out incomplete gesture actions, reducing false triggers.

[0074] Furthermore, the holding time of a gesture is determined by setting a first threshold T1 and a second threshold T2. For example, when the frame rate is 10 frames per second, the first threshold T1 is set to 20. If the number of consecutive lost image frames (GestureLoss) is greater than or equal to the first threshold T1, it means that the target gesture has not been detected for 2 consecutive seconds. At this time, the target gesture can be restarted. Also, if the action category variable (GestureX) corresponding to the current gesture is greater than or equal to the second threshold T2, it means that the current gesture has been held still for 2 consecutive seconds. The corresponding device control signal can be generated, which further reduces the misrecognition of other gestures and improves the recognition efficiency.

[0075] Example 2

[0076] The above-mentioned device operating state switching process only applies to the case where discrete variable control of the device is completed by the appearance or absence of a gesture and the duration of the gesture. For example, the device (such as an air conditioner) can be started or stopped by a hand gesture that remains still for N seconds. However, the control of some device functions also requires the control and adjustment of continuous variables. This requires not only the appearance of the corresponding gesture, but also the movement of the gesture. For example, the volume control of an audio device can be completed by sliding the palm (e.g., sliding the palm to the left to turn down the volume and sliding it to the right to turn up the volume).

[0077] Therefore, in this embodiment, the process of executing the corresponding device control action according to the category of the single gesture action further includes the following steps:

[0078] After the device has entered the on state, the device's working state is switched according to whether a movement gesture occurs. The device can enter the on state through the content of section 3.2.3 in step S3 of embodiment 1, or by directly triggering the device's key, such as by directly pressing the device's power switch to turn on the device. The movement gesture can be the same as or different from the current gesture, and the switching of the device's working state is achieved by controlling continuous variables (such as volume, airflow, etc.).

[0079] Specifically, once the device is in the powered-on state, the process of switching the device's operating state based on whether a movement gesture is detected includes the following steps:

[0080] 3.3.1 If the action category variable GestureMove corresponding to the current gesture is 0, it means that the current gesture is appearing for the first time and has not yet moved. It is impossible to determine whether the current gesture is a moving gesture. At this time, only the starting position of the current gesture GestureMove_location is recorded, and GestureMove is incremented by 1. No device control signal is generated. The current device control action process is completed. Then repeat steps S2-S3.

[0081] 3.3.2 If the action category variable GestureMove = 1 corresponding to the current gesture, it means that the current gesture has appeared for the second time. At this time, the center position of the image where the current gesture is located is obtained, and the first movement direction Move_Direction_first of the current gesture is determined by comparing it with the center position of the image where the gesture is located when the action category variable GestureMove = 0, GestureMove_location. This serves as the basis for judging the movement direction of subsequent gestures. GestureMove is incremented by 1, and no device control signal is generated. Then the current device control action process is completed, and steps S2-S3 are repeated.

[0082] 3.3.3 If 1 < the action category variable GestureMove corresponding to the current gesture < the third threshold T3 (unit is the number of frames of the image, such as 5 frames), it means that the gesture may be a moving gesture and has lasted for a certain period of time, but the corresponding function of the device has not yet been triggered. Further determine whether the current gesture has moved relative to the previous frame image, and execute the corresponding device control process according to the determination result. The determination of whether the current gesture has moved relative to the previous frame image includes the following steps: obtain the distance between the center position of the current gesture image and the center position GestureMove_location of the gesture image when the action category variable GestureMove = 0. If the distance meets the first condition (such as less than or equal to the preset value), it is considered that the current gesture has not moved, the current gesture is not a moving gesture, and the device cannot be controlled. At this time, no device control signal is generated, and the current device control action process is completed. Then repeat steps S2-S3.

[0083] If the distance meets the second condition (such as being greater than or equal to a preset value), it is considered that the current gesture has moved. At this time, the next movement direction of the current gesture is obtained (the method for determining the next movement direction of the current gesture can be the same as the method for determining the first movement direction Move_Direction_first), and it is compared with the first movement direction Move_Direction_first.

[0084] If the current gesture's next movement direction (Move_Direction) is consistent with the first movement direction (Move_Direction), then the current gesture's movement is part of the device control action. In this case, GestureMove is incremented by 1, no device control signal is generated, and the current device control action flow is completed. Then, steps S2-S3 are repeated. Here, "consistent" means that the angle between the current gesture's next movement path and the first movement path is 0±20°, preferably 0±10°, and particularly preferably 0±5°. If the current gesture's next movement direction is inconsistent with the reference movement direction (Move_Direction), it means that the current gesture is not moving continuously in the same direction, and the current gesture is not a movement gesture. In this case, GestureMove is cleared to zero, no device control signal is generated, and the current device control action flow is completed. Then, steps S2-S3 are repeated.

[0085] 3.3.4 If the action category variable GestureMove corresponding to the current gesture is greater than or equal to the third threshold T3, it means that the current gesture is moving continuously in the same direction. The current gesture is determined to be a moving gesture and can be controlled by the device. At this time, the action category variable GestureMove is cleared to zero, and a device control signal is generated according to the movement direction Move_Direction of the current gesture. The device completes the switching of working state according to the device control signal, and the current device control action process is completed.

[0086] For example, first define the action category variable corresponding to the movement gesture as GestureMove. After the car air conditioner is turned on (the car air conditioner can be turned on by recognizing the palm gesture or by directly triggering the corresponding function button), the air conditioner fan speed is adjusted by moving the movement gesture (which can still be the palm, or it can be a different gesture, such as extending the index finger) in the same direction. For example, swiping left will reduce the fan speed, and swiping right will increase the fan speed.

[0087] When the index finger is first detected to be extended, the corresponding action category variable GestureMove = 0, indicating that the index finger has not moved yet. At this time, the starting position of the index finger gesture GestureMove_location is recorded, and GestureMove is incremented by 1. No air conditioning fan speed adjustment signal is generated.

[0088] Repeat step S2. When the index finger gesture is detected for the second time, GestureMove=1 and the movement direction is not determined. Then, obtain the center position of the image where the current index finger gesture is located, and determine the first movement direction Move_Direction_first by comparing it with the center position of the image where the gesture is located when the action category variable GestureMove=0, GestureMove_location. This serves as the basis for judging the movement direction of subsequent gestures. Also, increment GestureMove by 1 and do not generate an air conditioning fan speed adjustment signal.

[0089] Repeat step S2. If a finger gesture is detected again, and 0 < GestureMove < the third threshold T3, it indicates that the index finger gesture may be a movement gesture and has lasted for a certain period of time, but the corresponding function of the device has not yet been triggered. At this time, first determine whether the index finger gesture has indeed moved. The specific process is as follows:

[0090] Get the distance between the center position of the current index finger gesture in the image and the center position of the gesture in the image when the action category variable GestureMove=0, GestureMove_location. If the distance meets the first condition (such as being less than or equal to a preset value), it is considered that the current index finger gesture has not moved and the movement of the current index finger gesture is not part of the air conditioning fan speed adjustment action. At this time, no air conditioning fan speed adjustment signal is generated. If the distance meets the second condition (such as being greater than or equal to a preset value), it is considered that the current index finger gesture has moved.

[0091] Furthermore, it is determined whether the direction of the current index finger gesture is consistent with the first movement direction Move_Direction_first. If they are consistent, it means that the index finger gesture is moving in the same direction, which is part of the air conditioning fan speed adjustment action. At this time, GestureMove is incremented by 1 and no air conditioning fan speed adjustment signal is generated. If they are inconsistent, it means that the current index finger gesture is not moving continuously in the same direction and is not a movement gesture. At this time, GestureMove is cleared to zero.

[0092] Repeat step S2. If GestureMove ≥ the third threshold T3, it means that the index finger gesture has been moving continuously in the same direction, and the air conditioning fan speed adjustment function can be activated. At this time, GestureMove is reset to zero, and an air conditioning fan speed adjustment signal is generated so that the air conditioner can complete the fan speed adjustment.

[0093] Therefore, this embodiment can further control and adjust the continuous variables of the device through gesture recognition, and the accuracy of gesture recognition and action control can also be improved by changing the values ​​of the number of consecutive lost image frames (GestureLoss), the action category variable (GestureMove), and the third threshold (T3).

[0094] Example 3

[0095] This embodiment provides a device control system based on gesture recognition, which can implement the device control method in Embodiment 1 or 2, such as... Figure 5 The equipment control system includes:

[0096] Database 1 is used to store images of different standard gestures;

[0097] Imaging device 2, which is used to acquire real-time video streams, wherein the imaging device includes one or more of visible light imaging devices and infrared imaging devices;

[0098] The gesture recognition unit 3 is used to acquire a frame image from a real-time video stream and compare it with the standard gesture image database to obtain a gesture recognition result;

[0099] The device control unit 4 is used to execute the corresponding device control action process according to the gesture recognition result. The specific content is the same as step S3 in embodiment 1 or 2, and will not be repeated here.

[0100] In summary, this invention defines the number of consecutively lost image frames for a gesture (GestureLoss) and the action category variable (GestureX) corresponding to different gestures. Furthermore, during gesture recognition, the values ​​of GestureLoss and GestureX are updated by comparing them with a set threshold. Then, the corresponding device action control process is executed, such as device startup or airflow adjustment. This avoids system misjudgments caused by different or no gestures, and ensures that even if a small number of frames fail to detect a gesture, it does not affect gesture recognition. This improves device control sensitivity and filters out incomplete gestures, reducing false triggers.

[0101] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A device control method based on gesture recognition, characterized in that, Includes the following steps: S1. Establish a standard gesture image database; S2. Obtain the k-th frame image from the real-time video stream and compare it with the standard gesture image database to obtain the gesture recognition result; S3. Based on the gesture recognition result, update the number of consecutively lost image frames (GestureLoss) for the current gesture and / or the action category variable (GestureX) corresponding to different gestures, and execute the corresponding device control action flow, which includes the following steps: 3.1 If the gesture recognition result indicates that the k-th frame contains different gesture actions or does not contain any gesture actions, then determine whether the number of consecutive lost image frames for the current gesture, GestureLoss, is greater than or equal to the first threshold T1. If the number of consecutive lost image frames for the current gesture is greater than or equal to the first threshold T1, then the number of consecutive lost image frames for the gesture, GestureLoss, and all action category variables, GestureX, are cleared to zero, and the device's working state remains unchanged. If the number of consecutive image frames lost during the current gesture is less than the first threshold T1, then the number of consecutive image frames lost during the current gesture is incremented by 1, and the device's working state remains unchanged. 3.2 If the gesture recognition result indicates that the k-th frame contains a single gesture action, then the number of consecutive lost image frames for the current gesture is reset to zero, and the corresponding device control action process is executed according to the category of the single gesture action. The gesture loss image frame count (GestureLoss) refers to the number of consecutively unrecognized gestures or the number of consecutively unrecognized gestures of the same type. The X in the action category variable (GestureX) is assigned a different name according to the different gesture categories, and different action category variables (GestureX) correspond to different device control action processes.

2. The equipment control method as described in claim 1, characterized in that, The corresponding device control action flow is executed according to the category of the single gesture, including the following steps: If the action category variable GestureX corresponding to the current gesture is 0, then record the center position of the image where the current gesture is located, Gesture_location, increment the action category variable GestureX by 1, and do not generate a device control signal.

3. The equipment control method as described in claim 1, characterized in that, The corresponding device control action flow is executed according to the category of the single gesture, including the following steps: If 0 < GestureX, the action category variable corresponding to the current gesture, < the second threshold T2, then determine whether the current gesture has moved relative to the previous frame image. If it is assumed that no movement has occurred during the current gesture, GestureX is incremented by 1, and no device control signal is generated; If it is assumed that the current gesture has moved, and the occurrence of the current gesture is not part of the device function trigger action, then GestureX will be cleared and no device control signal will be generated.

4. The equipment control method as described in claim 1, characterized in that, The corresponding device control action flow is executed according to the category of the single gesture, including the following steps: If the action category variable GestureX corresponding to the current gesture is greater than or equal to the second threshold T2, then the number of consecutive lost image frames GestureLoss and all action category variables GestureX are cleared to zero, and a device control signal is generated, so that the device can switch its working state according to the device control signal.

5. The equipment control method as described in claim 4, characterized in that, The switching of the working state is achieved by controlling discrete variables.

6. The equipment control method as described in claim 1, characterized in that, The corresponding device control action flow is executed according to the category of the single gesture, including the following steps: Once the device is turned on, the device's operating status is switched based on whether a movement gesture is detected.

7. The equipment control method as described in claim 6, characterized in that, Once the device is powered on, the process of switching the device's operating state based on whether a movement gesture is detected includes the following steps: If the action category variable GestureMove corresponding to the current gesture is 0, then record the starting position of the current gesture GestureMove_location, increment GestureMove by 1, and do not generate a device control signal.

8. The equipment control method as described in claim 7, characterized in that, Once the device is powered on, the process of switching the device's operating state based on whether a movement gesture is detected includes the following steps: If the action category variable GestureMove corresponding to the current gesture is 1, then the center position of the image where the current gesture is located is obtained, and the first movement direction Move_Direction_first of the current gesture is determined by comparing it with the center position of the image where the gesture is located when the action category variable GestureMove=0, GestureMove_location. Then, GestureMove is incremented by 1, and no device control signal is generated.

9. The equipment control method as described in claim 7, characterized in that, Once the device is powered on, the process of switching the device's operating state based on whether a movement gesture is detected includes the following steps: If 1 < the action category variable GestureMove corresponding to the current gesture < the third threshold T3, then determine whether the current gesture has moved relative to the previous frame image; If it is assumed that no movement has occurred during the current gesture, no device control signal will be generated; If it is believed that the current gesture has moved, then the direction of the next movement of the current gesture, Move_Direction, is compared with the direction of the first movement, Move_Direction_first. If the angle between the current gesture's subsequent movement path and the initial movement path is 0±20°, then GestureMove is incremented by 1, and no device control signal is generated; if the current gesture's subsequent movement direction is inconsistent with the reference movement direction Move_Direction, then GestureMove is cleared to zero, and no device control signal is generated.

10. The equipment control method as described in claim 7, characterized in that, Once the device is powered on, the process of switching the device's operating state based on whether a movement gesture is detected includes the following steps: If the action category variable GestureMove corresponding to the current gesture is greater than or equal to the third threshold T3, then the action category variable GestureMove is cleared to zero, and a device control signal is generated, so that the device can switch working states according to the device control signal.

11. The equipment control method as described in claim 7, characterized in that, The switching of the working state is achieved by controlling continuous variables.

12. A device control system for implementing the device control method according to any one of claims 1-11, characterized in that, include: A database used to store images of different standard gestures; Imaging equipment used to acquire real-time video streams; A gesture recognition unit is used to acquire a frame image from a real-time video stream and compare it with the standard gesture image database to obtain a gesture recognition result; And a device control unit, which is used to execute the corresponding device control action process based on the gesture recognition result.

Images