Control method based on gesture recognition

Abstract

The invention relates to a control method based on gesture recognition. Target data formed by a single or multiple continuous images is obtained through a camera device on a terminal. And then, processing and identifying the obtained target data to obtain gesture data in the target data. Corresponding to a single or a plurality of continuous images in the target data, the gesture data also comprises static gesture data and dynamic gesture data. And generating a corresponding control or operation instruction based on the static gesture data and the dynamic gesture data, wherein the generated instruction is executed by the terminal. By introducing continuous dynamic recognition, gesture recognition is not limited to a specific action corresponding to a specific instruction any more, and adaptive fusion is carried out by understanding the gesture action and combining with an instruction execution end of the terminal, so that the control instruction is accurate, excessive complex gesture control actions do not need to be remembered, and the operation convenience is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of gesture recognition, in particular to a control method based on gesture recognition. BACKGROUND

[0002] Smart home appliances are household appliances formed by introducing microprocessors, sensor technology, and network communication technology into household appliances. They have the ability to automatically sense the state of the residence space and the state of the appliances themselves and the service state of the appliances. They can automatically control and receive control instructions from users in the residence or remotely, and share information between devices through a home local area network, the Internet of Things, and the Internet. As a component of smart homes, smart home appliances can interconnect with other appliances, furniture, and facilities in the residence to form a system and achieve smart home functions. The core features include interconnection, intelligent control, and energy management.

[0003] Currently, smart home appliances are mostly controlled by voice, but the recognition accuracy of voice control is low when the environment becomes noisy. Therefore, some smart home appliances also have gesture recognition control. However, the gesture recognition of existing smart home appliances mostly relies on basic gestures or custom gesture recognition, which limits their convenience and recognition accuracy to some extent. SUMMARY

[0004] Therefore, it is necessary to provide a control method based on gesture recognition to solve the problem that the gesture recognition of traditional smart home appliances mostly relies on basic gestures or custom gesture recognition, resulting in low convenience and accuracy.

[0005] The present application provides a control method based on gesture recognition, which includes: Obtaining target data, which includes a single target image or multiple continuous target images; Preprocessing the target data to obtain standard target data; Performing target gesture recognition based on the standard target data to obtain target gesture data; Generating corresponding control instructions based on the target gesture data and combining terminal self-information; Executing the control instructions.

[0006] Further, the target gesture recognition based on the standard target data to obtain target gesture data includes: Performing feature recognition on the standard target data to obtain one or more features; Identifying the type of the features, assigning different feature labels to the features according to the feature types, and including all features in a feature set; Filtering the features in the feature set to obtain effective features and including the effective features in an effective feature set.

[0007] Further, the type of the feature is identified, and different feature labels are assigned to the features according to the feature type, and all features are included in the feature set, including: determine whether the features obtained after feature recognition are all static features; If the features obtained after feature recognition are all static features, a static feature label is assigned to each feature; If the features obtained after feature recognition are all dynamic features, a dynamic feature label is assigned to each feature; All features assigned with feature labels are placed in the feature set.

[0008] Further, the features in the feature set are screened to obtain effective features, and the effective features are included in the effective feature set, including: select a feature in the feature set; read the feature label of the feature, and determine whether the feature is a static feature; If the feature is a static feature, the feature is verified as an effective feature by a static gesture template comparison verification method; If the feature is not a static feature, determine whether the feature is a dynamic feature; If the feature is a dynamic feature, the feature is verified as an effective feature by a dynamic gesture continuity verification method; return to selecting a feature in the feature set until all features in the feature set are selected; All effective features screened are included in the effective feature set.

[0009] Further, the feature is verified as an effective feature by a static gesture template comparison verification method, including: Compare the feature with all static gesture templates in the preset feature library one by one, and determine whether there is any static gesture template with a similarity greater than a similarity threshold to the feature; If there is any static gesture template with a similarity greater than a similarity threshold to the feature, the feature is determined to be an effective feature.

[0010] Further, the preset feature library includes a basic feature library and a custom feature library.

[0011] Further, the feature is verified as an effective feature by a dynamic gesture continuity verification method, including: Fuse the feature in time sequence, space and trajectory to generate a transition feature; determine whether the transition feature has continuity; If the transition feature has continuity, the feature is determined as a valid feature; If the transition feature does not have continuity, the feature is determined as an invalid feature.

[0012] Further, the target gesture data is used to generate a corresponding control instruction in combination with terminal information. An effective feature is selected from the set of effective features; The feature label of the effective feature is read to determine whether the effective feature is a static feature; If the effective feature is a static feature, a static gesture template matching sequence of the effective feature is obtained; The selection of an effective feature from the set of effective features is returned until all effective features are selected, and a static gesture template matching sequence of each effective feature is obtained; The static gesture template matching sequences of the effective features are aggregated, a final static gesture template matched by each effective feature is determined according to a preset decision rule, and the static gesture template is used as a target static gesture template; A control instruction corresponding to the target static gesture template is obtained.

[0013] Further, after the reading of the feature label of the effective feature and the determination of whether the effective feature is a static feature, the method further includes: If the effective feature is not a static feature, it is determined whether the effective feature is a dynamic feature; If the effective feature is a dynamic feature, a gesture action parameter corresponding to the effective feature is obtained; The selection of an effective feature from the set of effective features is returned until all effective features are selected, and a gesture action parameter of each effective feature is obtained; The gesture action parameters of all effective features are matched with terminal information to obtain a matching result; A corresponding control instruction is generated based on the matching result.

[0014] Further, the gesture action parameter includes one or more of a gesture action speed, a gesture action direction, and a repetition number.

[0015] This application relates to a control method based on gesture recognition. It acquires target data consisting of one or more consecutive images using a camera device on a terminal. The acquired target data is then processed and recognized to obtain gesture data. Corresponding to the single or multiple consecutive images in the target data, the gesture data also includes static and dynamic gesture data. Based on the static and dynamic gesture data, corresponding control or operation commands are generated, which are then executed by the terminal. By introducing continuous dynamic recognition, gesture recognition is no longer limited to specific actions corresponding to specific commands. Instead, it adaptively integrates the understanding of gesture actions with the terminal's execution command end, achieving precise control commands without requiring the memorization of numerous complex gesture control actions, thus improving operational convenience. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a gesture recognition-based control method provided in an embodiment of this application.

[0017] Figure 2 This is a schematic diagram showing the feature comparison between a feature provided in an embodiment of this application and a static gesture template. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] like Figure 1 As shown, in one embodiment of this application, the control method based on gesture recognition includes the following steps S100 to S500.

[0020] S100, acquire target data, the target data including a single target image or multiple consecutive target images.

[0021] Specifically, target data is acquired through a camera configured on the terminal, where a single image is used for static feature recognition, while multiple consecutive images are used for dynamic gesture recognition.

[0022] The threshold includes a rotation angle threshold and a plurality of direction displacement thresholds. When the features of the hand in the images obtained in the period T1 change by more than the threshold, the camera configured on the terminal is controlled to continuously obtain images until the features of the hand in the obtained images do not change by more than the threshold. When the features of the hand in the images obtained in the period T1 do not change by more than the threshold, the image with the most complete and clearest features is obtained from the images obtained in the period T1 as the target image.

[0023] S200, pre-processing the target data to obtain standard target data.

[0024] Specifically, the pre-processing of the target data includes image denoising, brightness correction, background segmentation and the like, so as to eliminate the interference in the environment and improve the quality of the data.

[0025] S300, performing target gesture recognition based on the standard target data to obtain target gesture data.

[0026] Specifically, the target gesture data refers to a set of gesture features recognized in a single or multiple images with clear and complete gesture features, for example, the set of gesture features can include the spread state of the fingers and the relative position relationship between the fingers.

[0027] S400, generating corresponding control instructions based on the target gesture data and combining the terminal self information.

[0028] S500, executing the control instructions.

[0029] Specifically, the control instructions are executed by the terminal performing gesture feature recognition, which can be a washing machine, a television, a refrigerator or the like.

[0030] In the embodiment, the target data composed of a single or multiple continuous images is obtained by the camera device on the terminal. Then, the obtained target data is processed and recognized to obtain gesture data in the target data. The single or multiple continuous images in the target data also have static gesture data and dynamic gesture data. Corresponding control or operation instructions are generated based on the static gesture data and the dynamic gesture data, and the generated instructions are executed by the terminal. By introducing continuous dynamic recognition, the gesture recognition is no longer limited to specific actions corresponding to specific instructions, but through the understanding of gesture actions and the adaptive fusion of the terminal execution instruction end, the control instructions are accurate and do not need to remember too many complex gesture control actions, so that the operation convenience is improved.

[0031] In an embodiment of the present application, the target gesture recognition based on the standard target data obtains target gesture data, and includes the following S301-S303.

[0032] S301, feature recognition is performed on the standard target data to obtain one or more features.

[0033] S302, the type of the features is recognized, different feature labels are assigned to the features according to the feature types, and all the features are included in a feature set.

[0034] Specifically, the feature labels include static features and dynamic features.

[0035] S303, the features in the feature set are screened to obtain effective features, and the effective features are included in an effective feature set.

[0036] The type of the features is recognized, different feature labels are assigned to the features according to the feature types, and all the features are included in a feature set, and includes the following S302a-S302d.

[0037] S302a, it is judged whether the features obtained after the feature recognition are all static features.

[0038] Specifically, the static features refer to fixed postures of the gestures, such as clenched fists, open palms, etc.

[0039] S302b, if the features obtained after the feature recognition are all static features, a static feature label is assigned to each feature.

[0040] S302c, if the features obtained after the feature recognition are all dynamic features, a dynamic feature label is assigned to each feature.

[0041] Specifically, the dynamic features refer to motion processes of the gestures, such as sliding of the whole hand, rotation of the hand, and swinging of the hand, etc.

[0042] S302d, all the features assigned with the feature labels are placed in the feature set.

[0043] In the embodiment, the features are classified according to the space-time characteristics of the gesture features, different resources are allocated to understand and analyze the static features or the dynamic features, the processing efficiency of the data is improved, and the adaptability of the system to complex gestures is enhanced.

[0044] In an embodiment of the present application, the features in the feature set are screened to obtain effective features, and the effective features are included in an effective feature set, and includes the following S303a-S303h.

[0045] S303a, one feature is selected from the feature set.

[0046] S303b, read the feature label of the feature, and determine whether the feature is a static feature.

[0047] S303c, if the feature is a static feature, use a static gesture template comparison verification method to verify the feature.

[0048] S303d, if the feature is not a static feature, determine whether the feature is a dynamic feature.

[0049] S303e, if the feature is a dynamic feature, use a dynamic gesture continuity verification method to verify the feature.

[0050] S303f, return to selecting a feature from the feature set until all features in the feature set are selected after verification.

[0051] S303g, include all filtered effective features in the effective feature set.

[0052] S303h, if the feature set is not a dynamic feature, the feature set is invalid.

[0053] In this embodiment, the feature filtering process is achieved by iterating through the feature set, the static feature verification uses the template matching method, the dynamic feature verification uses the motion model to check the continuity of the trajectory, and the invalid feature is discarded. The effective features are retained to facilitate subsequent instruction generation to improve the reliability of the features.

[0054] In an embodiment of the present application, the effective feature verification of the feature using the static gesture template comparison verification method includes S313c to S323c.

[0055] S313c, compare the feature with all static gesture templates in the preset feature library one by one, and determine whether there is any static gesture template with a similarity greater than a similarity threshold.

[0056] Specifically, for each static gesture template in the preset feature library, the feature is disassembled and decomposed into a main feature and a secondary feature.

[0057] The main feature is the most representative and highest distinguishing feature in the gesture, such as the thumb spread, the index finger bending, etc.

[0058] The secondary feature is an auxiliary feature, such as the palm inclination angle, the finger spacing, etc.

[0059] In the comparison, the main feature comparison is performed first: Determine whether the feature matches the main feature of any template, i.e., the similarity between the feature and the main feature is greater than the similarity threshold.

[0060] If the matching is successful, the feature is a valid feature.

[0061] If the matching of the feature with the main feature is not successful, it is determined whether the feature matches any sub-feature of any template, i.e. the similarity between the feature and the sub-feature is greater than a similarity threshold.

[0062] If the matching is successful, the feature is a valid feature.

[0063] Further, the similarity between the feature and the static gesture template is embodied by a structural similarity index.

[0064] The formula for calculating the structural similarity index is as follows:

[0065] wherein x is the image of the feature; y is the image of the feature compared with the static gesture template; is the brightness similarity between the image of the feature and the image of the feature in the static gesture template; is the contrast between the image of the feature and the image of the feature in the static gesture template; is the structural similarity between the image of the feature and the image of the feature in the static gesture template; are weight coefficients, and are all set to 1.

[0066] Usually, the image of the feature and the image of the feature compared with the static gesture template can also be taken as a window size of 11 11, and the average of the SIMM values of the plurality of windows is taken as the SIMM value of the image of the feature.

[0067] The obtained SSIM value is compared with a preset similarity threshold to obtain a comparison result.

[0068] Further, the matching of the distance between the key points and the matching of the contour profile can also be introduced, and the similarity is comprehensively evaluated in combination with the SIMM value.

[0069] S323c, if there is any static gesture template with a similarity greater than a similarity threshold with the feature, the feature is determined to be a valid feature.

[0070] Specifically, there is a static gesture template for the victory gesture in the static gesture template. The main feature of the static gesture template is: 1. The index finger is straight, and the area A is straight. 2. The middle finger is straight, and the area B is straight. The sub-feature of the static gesture template is: 1. The thumb is bent, and the area C is bent. ​​2. palm facing forward, region D; If the feature is the index finger straight, and is located in region A.

[0071] The feature is compared with the main feature of the static gesture template of the victory gesture, and the feature hits the main feature of the static gesture template of the victory gesture, the index finger straight, region A, and the similarity is greater than the similarity threshold. The feature is an effective feature.

[0072] If the feature is the thumb bent, and is located in region C.

[0073] The feature is compared with the main feature of the static gesture template of the victory gesture, and the feature does not hit the main feature matching in the static gesture template of the victory gesture. The feature is compared with the secondary feature of the static gesture template of the victory gesture, and the feature hits the secondary feature of the static gesture template of the victory gesture, the thumb bent, region C, and the similarity is greater than the similarity threshold. The feature is an effective feature.

[0074] If the feature is the little finger straight, and is located in region E.

[0075] The feature is compared with the main feature of the static gesture template of the victory gesture, and the feature does not match the main feature and the secondary feature of the static gesture template of the victory gesture, and the feature does not match other static gesture templates. The feature is an invalid feature.

[0076] Specifically, the preset feature library includes a basic feature library and a self-defined feature library.

[0077] In this embodiment, when comparing a feature in a static gesture, the static gesture needs to be adjusted first, for example, rotating and scaling the static gesture, so that the overall orientation and proportion of the static gesture are consistent with the gesture in the template, facilitating more accurate feature comparison in the subsequent process.

[0078] In this embodiment, the template matching-based method is used to identify features to achieve fast static gesture recognition. Meanwhile, users can also input specified gestures and define corresponding control instructions to provide customized self-definition.

[0079] In an embodiment of the present application, the dynamic gesture continuity verification method is used to verify the feature, including the following S313e to S343e.

[0080] S313e, the time sequence, space and trajectory of the feature are fused to generate a transition feature.

[0081] Specifically, the time sequence dimension fusion is: A plurality of continuous gesture feature sequences are obtained.

[0082] Time resample to fixed length N frames, where N is 30; when the sequence length is greater than N: select N frames by equal interval downsampling method; when the sequence length is less than N: complete to N frames by linear interpolation method.

[0083] Normalize the timestamp, map the time axis to the [0, 1] interval.

[0084] Use sliding window technology to traverse the entire sequence, and the window size is recommended to be 3 frames.

[0085] Calculate the statistics of the features in each window.

[0086] Specifically, the statistics of the features in each window include: calculating the mean vector of the features in the window. Calculate the standard deviation vector of the features in the window. Calculate the feature change speed, that is, the difference value of adjacent frame features.

[0087] Calculate the mean and standard deviation of the speed.

[0088] Organize the time sequence statistics of all windows in chronological order.

[0089] The fusion of spatial dimensions is: Take the wrist as the key point as the origin of the coordinate system.

[0090] Convert all hand key point coordinates to coordinates relative to the wrist.

[0091] Calculate the palm plane normal vector.

[0092] Specifically, calculating the palm plane normal vector includes: selecting the wrist, the base of the index finger, and the base of the little finger. Calculate the palm plane normal vector by vector cross product. Normalize the normal vector.

[0093] Calculate the relative angle features of each finger.

[0094] Specifically, calculating the relative angle features of each finger includes: processing the thumb, index finger, middle finger, ring finger, and little finger respectively. Calculate the angle between the fingertip-to-finger-root vector and the palm normal vector.

[0095] Extract the overall direction feature of the hand.

[0096] Specifically, extracting the overall direction feature of the hand includes taking the middle finger root key point as the overall direction of the palm.

[0097] Construct the inter-finger distance matrix.

[0098] Specifically, constructing the inter-finger distance matrix includes: first selecting five fingertip key points. Calculate the Euclidean distance between all fingertip points. Form a 5x5 symmetric distance matrix.

[0099] Fusion of trajectory dimensions: Select key trajectory points: wrist, index fingertip, thumb tip.

[0100] Extract position coordinates of each point in time series.

[0101] Apply Kalman filter for trajectory smoothing.

[0102] Specifically, applying Kalman filter for trajectory smoothing is achieved by initializing a 6-state filter (position + velocity). Perform a prediction-update cycle for each trajectory point. Extract smoothed position estimates.

[0103] Calculate trajectory for each key point.

[0104] Specifically, calculating trajectory for each key point includes trajectory length: cumulative path length. Trajectory curvature: mean and maximum of point curvature values. Motion velocity: statistics of the modulus length of the first-order derivative of the trajectory. Motion acceleration: statistics of the modulus length of the second-order derivative of the trajectory.

[0105] Calculate motion smoothness index.

[0106] Specifically, calculating motion smoothness index is achieved by evaluating motion fluency through jerk. Smoothness score = 1 / (1+average jerk modulus).

[0107] Fusion process of three dimensions: fusion of time series and space.

[0108] Combine time series rate of change with spatial structure.

[0109] Specifically, combining time series rate of change with spatial structure includes combining shape velocity with spatial configuration. Combining shape stability with hand direction.

[0110] Generate spatio-temporal joint feature vector.

[0111] Fusion process of three dimensions: trajectory continuity evaluation.

[0112] Calculate the continuity score of each key point trajectory: where, smoothness weight: 50%; velocity consistency weight: 30%; curvature continuity weight: 20%.

[0113] Integrate the scores of each key point to obtain the overall trajectory continuity.

[0114] Fusion process of three dimensions: overall motion consistency.

[0115] Use weighted fusion method: where, trajectory smoothness contribution: 40%; time series stability contribution: 30%; spatial consistency contribution: 30%.

[0116] Calculate the final overall consistency score.

[0117] S323e, determining whether the transition feature has continuity.

[0118] S333e, if the transition feature has continuity, determining that the feature is a valid feature.

[0119] S343e, if the transition feature does not have continuity, determining that the feature is an invalid feature.

[0120] Specifically, the lack of continuity means that the feature has jitter or interruption.

[0121] In this embodiment, the continuity is determined by setting specific thresholds. For example, the overall consistency threshold is greater than or equal to 0.75; the average trajectory continuity threshold is greater than or equal to 0.70; and the spatiotemporal feature matching degree threshold is greater than or equal to 0.65.

[0122] It is determined to be a valid feature if the following conditions are met: the overall consistency score exceeds the threshold; the average trajectory continuity exceeds the threshold; and the trajectory continuity of at least two key points exceeds the threshold.

[0123] In an embodiment of the present application, the control instruction corresponding to the target gesture data is generated based on the terminal itself information, including the following S401 to S406.

[0124] S401, selecting one valid feature in the valid feature set.

[0125] S402, reading the feature label of the valid feature, and determining whether the valid feature is a static feature.

[0126] S403, if the valid feature is a static feature, obtaining the static gesture template matching sequence of the valid feature.

[0127] S404, returning to selecting one valid feature in the valid feature set until all valid features are selected, and obtaining the static gesture template matching sequence of each valid feature.

[0128] S405, aggregating the static gesture template matching sequences of each valid feature, determining the static gesture template matched by the overall of each valid feature according to a preset decision rule, and taking the static gesture template as a target static gesture template.

[0129] Specifically, the preset decision rule can be a voting mechanism, which determines the final matching static gesture template by voting on the matching results of the final each valid feature.

[0130] For example, a palm open static gesture feature contains five open finger effective features, wherein the static gesture template matching the thumb has A, B, C; the static gesture template matching the index finger has A, C; the static gesture template matching the middle finger has D, B, A; the static gesture template matching the ring finger has A, B; and the static gesture template matching the little finger has B, A, C. The above template matching order is the order of similarity, and the static gesture template in the first position has the highest similarity. Voting is performed on the above matching results, and the voting result order is A (5 votes), B (4 votes), C (3 votes), and D (1 vote), which are sorted in descending order.

[0131] Therefore, the static gesture template A is the final matching static gesture template.

[0132] S406, obtaining a control instruction corresponding to the target static gesture template.

[0133] In the embodiment, the static gesture matching sequence is gradually narrowed by performing static gesture template matching on each static effective feature, and the static gesture template matching sequence of the whole effective feature is obtained after all the static effective features are matched. Then, voting is performed on the matching in the static gesture template matching sequence to obtain the final target static gesture template.

[0134] In an embodiment of the present application, after the feature tag of the effective feature is read and it is judged whether the effective feature is a static feature, the following S407 to S411 are further included.

[0135] S407, if the effective feature is not a static feature, it is judged whether the effective feature is a dynamic feature.

[0136] S408, if the effective feature is a dynamic feature, a gesture action parameter corresponding to the effective feature is obtained.

[0137] Specifically, the gesture action speed calculation includes: based on the displacement change of the hand key point in the continuous frame sequence; calculating the instantaneous speed as v=Δs / Δt, wherein Δs is the key point displacement between adjacent frames, and Δt is the frame interval time; calculating the average speed as the displacement integral average of the whole gesture duration.

[0138] For example, in a palm horizontal motion action, the palm center key point is taken as the reference, and it is calculated that the palm center key point moves 30 centimeters in 0.5 seconds, and the average speed is 0.6 meters / second.

[0139] The gesture action direction recognition includes: establishing a three-dimensional space coordinate system, wherein X: horizontal direction, Y: vertical direction, and Z: front-back direction; determining the main direction of the gesture motion by principal component analysis; projecting the continuous motion trajectory onto the coordinate axis to determine the dominant motion direction;

[0140] For example, leftward sliding is recognized as the negative direction of X axis, and upward sliding is recognized as the positive direction of Y axis.

[0141] Calculation of repetition times: detecting periodic pattern of gesture actions; analyzing repetition of motion trajectory by autocorrelation function; setting minimum repetition interval threshold, such as 0.5 seconds, to avoid false detection.

[0142] For example, three consecutive fist actions are counted by detecting the complete cycle of hand closing-opening-closing.

[0143] S409, returning to selecting one effective feature from the set of effective features until all effective features are selected to obtain gesture action parameters corresponding to each effective feature.

[0144] S410, matching gesture action parameters of all effective features with terminal information to obtain matching results.

[0145] Specifically, terminal state is perceived based on terminal information, which specifically includes obtaining current running state of the terminal, such as booting state, standby state, and specific application running state; identifying current interaction context of the terminal, such as menu navigation mode, volume adjustment interface, and channel selection state; detecting terminal environment parameters, such as ambient light intensity and approximate distance of the user from the terminal.

[0146] Further, a conversion matrix of gesture parameters to control instructions is established; different mapping rule sets are enabled for different terminal states.

[0147] S411, generating corresponding control instructions based on the matching results.

[0148] The gesture action parameters include one or more of gesture action speed, gesture action direction, and repetition times.

[0149] Specifically, priority of specific control instructions can be set, such as priority of static gesture instructions being higher than that of dynamic gesture instructions; continuous dynamic gestures can override single static gesture instructions; for conflicting instructions, time proximity principle or context relevance principle is adopted.

[0150] Further, gesture action parameters are converted into specific control values, for example, volume adjustment amount = gesture speed × sensitivity coefficient × direction sign; channel switching step = gesture repetition times × basic step; brightness adjustment amplitude = gesture movement distance × proportion factor.

[0151] For example, the terminal is a TV, and its state is a movie on-demand interface, and the cursor is navigating among categories. User gesture: palm continuously slides left twice with medium speed. Parameter extraction: direction = left, repeat times = 2, speed = 0.7 m / s. Matching result: switch category left, fast slide skips intermediate categories.

[0152] For example, the terminal is a drum washing machine, and its state is in a washing mode selection state. User gesture: hand rotates clockwise about 180 degrees with slow speed. Parameter extraction: direction = clockwise, rotation angle = 180°, speed = 30° / s. Matching result: rotate to select washing mode, slow rotation selects accurately.

[0153] In the embodiment, the dynamic gesture features are recognized, and the recognition result is matched to the information of the terminal, and then a control instruction for the terminal is generated.

[0154] For example, a user uses a hand to make a continuous twisting action in front of a drum washing machine, and at this time, the drum washing machine is just in a mode selection condition. The twisting angle, twisting speed and twisting direction of the user are understood, and the mode is switched among multiple modes according to the twisting direction and speed.

[0155] For example, a user makes a continuous one-way palm translation action in front of a TV, and at this time, the TV is in a movie channel type selection in a power-on mode. It is determined that the purpose of the user is to control in this page. The palm translation direction, translation distance and translation speed are controlled to control the position of a cursor or a selection box on the screen of the TV, for example, up and down movement is used to switch movie types, and left and right movement is used to enter or exit a movie type database.

[0156] The technical features of the above-described embodiments can be combined in any manner, and the execution order of the method steps is not limited. To make the description simple, all possible combinations of the technical features in the above-described embodiments are not described, but as long as the combinations of the technical features do not contradict, they should be considered as the scope of the description.

[0157] The above-described embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the patent scope of the present application. It should be noted that for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A control method based on gesture recognition, characterized in that, The gesture recognition-based control method includes: Acquire target data, which includes a single target image or multiple consecutive target images; Preprocess the target data to obtain standard target data; Target gesture data is obtained by recognizing target gestures based on standard target data. Based on the target gesture data, combined with the terminal's own information, corresponding control commands are generated. Execute control commands.

2. The control method based on gesture recognition according to claim 1, characterized in that, The target gesture recognition based on standard target data, to obtain target gesture data, includes: Perform feature recognition on standard target data to obtain one or more features; The feature types are identified, different feature labels are assigned to the features according to the feature types, and all features are included in the feature set; The features in the feature set are filtered to obtain the effective features, and the effective features are added to the effective feature set.

3. The control method based on gesture recognition according to claim 2, characterized in that, The process of identifying the type of features, assigning different feature labels to features based on their type, and including all features in a feature set includes: Determine whether all features obtained after feature recognition are static features; If all the features obtained after feature recognition are static features, then assign a static feature label to each feature. If all the features obtained after feature recognition are dynamic features, then assign a dynamic feature label to each feature. All features that have been assigned feature labels are placed into the feature set.

4. The control method based on gesture recognition according to claim 3, characterized in that, The process of filtering features in the feature set to obtain effective features and incorporating these effective features into the effective feature set includes: Select a feature from the feature set; Read the feature label of the feature and determine whether the feature is a static feature; If the feature is a static feature, then the feature is verified by comparing it with a static gesture template. If the feature is not a static feature, then determine whether the feature is a dynamic feature; If the feature is dynamic, the coherence test method of dynamic gestures is used to verify the feature as a valid feature. After verification, return to the previous step of selecting a feature from the feature set until all features in the feature set have been selected. All the selected valid features are included in the valid feature set.

5. The control method based on gesture recognition according to claim 4, characterized in that, The method of comparing and verifying the feature with a static gesture template is used to perform effective feature verification, including: The feature is compared one by one with all static gesture templates in the preset feature library to determine whether there is any static gesture template whose similarity to the feature is greater than the similarity threshold. If there exists any static gesture template whose similarity to the feature is greater than the similarity threshold, then the feature is determined to be a valid feature.

6. The control method based on gesture recognition according to claim 5, characterized in that, The preset feature library includes a basic feature library and a custom feature library.

7. The control method based on gesture recognition according to claim 4, characterized in that, The method for verifying the coherence of dynamic gestures to validate the feature effectively includes: The feature is fused with temporal, spatial, and trajectory data to generate transitional features; Determine whether the transition feature is coherent; If the transition feature is consistent, then the feature is considered a valid feature. If the transition feature is not coherent, then the feature is determined to be invalid.

8. The control method based on gesture recognition according to claim 4, characterized in that, The process of generating corresponding control commands based on target gesture data and the terminal's own information includes: Select one effective feature from the set of effective features; Read the feature label of the valid feature and determine whether the valid feature is a static feature; If the valid feature is a static feature, then obtain the static gesture template matching sequence of the valid feature; Return to the set of valid features and select one valid feature until all valid features have been selected, to obtain a static gesture template matching sequence for each valid feature; The static gesture template matching sequence of each effective feature is aggregated, and the static gesture template that is finally matched by each effective feature as a whole is determined according to the preset decision rules, and the static gesture template is used as the target static gesture template. Obtain the control commands corresponding to the target static gesture template.

9. The control method based on gesture recognition according to claim 8, characterized in that, After reading the feature label of the valid feature and determining whether the valid feature is a static feature, the process further includes: If the effective feature is not a static feature, then determine whether the effective feature is a dynamic feature; If the valid feature is a dynamic feature, then obtain the gesture action parameters corresponding to the valid feature; Return to the step of selecting a valid feature from the set of valid features, and continue until all valid features have been selected, to obtain the gesture action parameters for each valid feature; Match all valid gesture parameters with the terminal information to obtain the matching result; The corresponding control commands are generated based on the matching results.

10. The control method based on gesture recognition according to claim 9, characterized in that, The gesture parameters include one or more of the following: gesture speed, gesture direction, and number of repetitions.

Images