Real-time gesture recognition method based on adaptive gesture frame sequence extraction algorithm

Abstract

The present application relates to a kind of real-time gesture recognition method based on adaptive gesture frame sequence extraction algorithm, belong to human-computer interaction field.It includes constructing an adaptive network to select the frame sequence with discriminative in video, so that algorithm can automatically adjust the video frame number according to the category of gesture in video, and the feature of selected frame sequence is input into real-time gesture recognition network to carry out discrimination task, and return value is fed back to original algorithm network, and original algorithm network is jointly trained and promotes each other.The advantage is that the recognition time is shortened, in application, the problem that tester needs to keep gesture static for a period of time to obtain gesture recognition result in static gesture recognition process can be effectively avoided, system recognition speed is improved, and system delay is reduced.

Description

Technical Field

[0001] This invention belongs to the field of human-computer interaction, specifically relating to a real-time recognition method based on adaptive gesture frame sequence extraction. Background Technology

[0002] With the rapid development of computer technology, information interaction between humans and computers is indispensable. Using gestures to execute commands and interact with computers is more intelligent and natural than using hardware controls such as a mouse. Gesture recognition interaction methods mainly include those based on external devices and those based on computer vision. The computer vision-based method is more flexible and has lower equipment costs. The main challenges of this method are selecting features that are indestructible in scaling, lighting, and rotation; accurately segmenting gestures in complex backgrounds; and achieving high-accuracy real-time recognition.

[0003] With the development of machine vision technology, there are more and more applications using depth sensors for object recognition, behavior recognition, etc. Unlike ordinary cameras, depth cameras can detect the distance between the gesture and the camera, and can better restore the spatial position of the gesture.

[0004] The main reasons why computer vision-based gesture recognition is difficult to achieve high accuracy in real time are as follows:

[0005] 1. Dynamic gesture videos typically contain hundreds of video frames, which is very time-consuming when processing large amounts of image data in a dataset.

[0006] 2. In actual gesture recognition application systems, when using static gestures acquired in real time for recognition, the tester needs to keep the gesture still for a period of time in order to obtain the gesture recognition result. This will reduce the system recognition speed and cause system delay. Summary of the Invention

[0007] This invention provides a real-time gesture recognition method based on an adaptive gesture frame sequence extraction algorithm, which improves the real-time performance of gesture recognition and solves the recognition delay problem when performing static gesture recognition in real-time gestures. This method is applicable to real-time gesture recognition methods that extract gesture features frame by frame.

[0008] The technical solution adopted by this invention includes the following steps:

[0009] Step 1: Load and read the pre-generated M random line segment data in the gesture recognition program and use a depth camera to acquire real-time depth images;

[0010] Step 2: Use a computer to input classification labels and record multiple depth video frame sequences corresponding to each classification label as a training set, where each depth video frame sequence contains N frames, which is one sample;

[0011] Step 3: Calculate the gesture feature value for each sample;

[0012] Step 4: Calculate the dynamic classification threshold for samples of the same type;

[0013] Step 5: Simultaneously, by calculating the current frame image data acquired in real time and the sequence of the most recent N frames of depth video frames, two classification results are obtained;

[0014] Step 6: Compare the two classification results and output the classification result that differs less from the sample as the final gesture classification result.

[0015] Step 1 of the present invention specifically involves: based on experimental observations, in order to ensure that random line segments are distributed across the image as much as possible so as to fully describe the gesture, the value of M is selected to be no less than 1000.

[0016] Step 2 of this invention specifically involves: each sample has a motion trajectory, which is defined as: in a depth video frame sequence of N frames, the feature vector value corresponding to the m-th random line segment in the n-th frame is calculated to be f. m (n) There are M random line segments in total, and the feature vector p of the nth frame is... (n) =[f1 (n) f2 (n) ,...,f M (n) By concatenating N feature vectors in the order of the depth video frame sequence, a trajectory T = [p] is formed. (1) ,p (2) ,...,p (N) The frame number N is the motion trajectory. According to experimental observations, in order to take into account both static and dynamic gestures with a large amount of data, the frame number N is selected to be no less than 30, and samples with the same classification label are considered to be of the same type.

[0017] Step 3 of this invention includes the following steps:

[0018] Step 3.1: Obtain the hand region after segmenting the depth video frame sequence;

[0019] Step 3.2: Extract the feature vector values ​​of M random line segments;

[0020] M randomly generated line segments are scattered across the hand region of each frame in the sequence. Each random line segment corresponds to a feature vector, which consists of the following five features:

[0021] Feature 1: Defined as the proportion of pixels in the hand region containing random line segments to the total number of pixels in the entire image;

[0022] Feature 2: Defined as the average depth value of pixels in the hand area covered by random line segments;

[0023] Feature 3: Represents the position of the pixel with the minimum depth value on a random line segment;

[0024] Feature 4: Represents the position of the pixel with the maximum depth value on a random line segment;

[0025] Feature 5: Represents the variation characteristics of pixel depth values ​​in the hand region, estimated by summing the absolute values ​​of the depth differences between adjacent pixels in the hand region on random line segments;

[0026] Finally, an eigenvector value is calculated for each random line segment;

[0027] Step 3.3: Based on the random line segment feature vector values ​​obtained in Step 3.2, calculate the three features of the motion trajectory;

[0028] The first feature is the three-dimensional distance feature, which describes the distance of feature points on the motion trajectory in the depth video frame sequence in three-dimensional space, and can represent the appearance of gestures;

[0029] The second feature is the motion length feature, which can represent the position of a gesture in space as time changes. The formula for calculating the motion length feature is as follows:

[0030]

[0031] The third characteristic is the speed characteristic, which is obtained by calculating the speed of the gesture movement.

[0032] Step 4 of this invention includes the following steps:

[0033] Step 4.1: Calculate the three-dimensional distance feature difference, motion length feature difference, and velocity feature difference;

[0034] Formula for calculating the feature difference of three-dimensional distance:

[0035] The two feature vectors p in the nth frame a (n) and p b (n) The distance is defined as:

[0036]

[0037] Where f m,a (n) and f m,b (n) Representing the eigenvectors p a (n) and p b (n)The m-th value out of M values ​​represents the Euclidean distance between two feature vectors, specifically the 3D distance feature difference.

[0038]

[0039] Where Sa and Sb represent two depth video frame sequences for which feature difference calculation is required;

[0040] Formula for calculating the difference in motion length characteristics:

[0041]

[0042] Among them, T l and T s These represent the motion trajectories T corresponding to depth video frame sequences Sa and Sb, respectively. a and T b Trajectories with relatively large and small eigenvalues ​​in medium length;

[0043] Formula for calculating velocity characteristic difference:

[0044]

[0045] Where v a (n) and v b (n) These are respectively represented as the depth video frame sequence S a and S b In the middle, the speed at which the hand moves in the nth frame;

[0046] Step 4.2: Calculate the dynamic threshold of the sample;

[0047] Each gesture label corresponds to a threshold Th(Label(S)). The threshold is calculated by taking the weighted average of the median differences of the three features between all pairs of samples with the same label in the training set, i.e.:

[0048] Th(Label(S))=w1·medianD1(S a ,S b )+w2·medianD2(S a ,S b )+w3·medianD3(S a ,S b )

[0049] Where w1, w2, and w3 represent the weights of the three feature differences. Through multiple experiments, the w1, w2, and w3 values ​​corresponding to the case with the highest average recognition rate are taken as the weights of the dynamic threshold.

[0050] Two depth video frame sequences S a and Sb Differences are represented by D(S) a ,S b The formula for its calculation is as follows:

[0051] D(S a ,S b )=w1·medianD1(S a ,S b )+w2·medianD2(S a ,S b )+w3·medianD3(S a ,S b )

[0052] Where w1, w2, and w3 represent the weights of the three feature differences. Through multiple experiments, the w1, w2, and w3 values ​​corresponding to the case with the highest average recognition rate are taken as the weights of the dynamic threshold.

[0053] Step 5 of this invention includes the following steps:

[0054] Step 5.1: Process the data of the current frame depth image acquired in real time to obtain a classification result 1;

[0055] The gesture features of the most recent frame of the real-time depth video are extended by copying the feature vector of the current frame to obtain the same number of frames as the sample, i.e., N frames of feature vectors. These vectors are used to form a motion trajectory, and the feature value of the processed frame sequence is calculated. The difference between this value and the sample feature is calculated and compared with the sample threshold to output a classification result 1.

[0056] The gesture threshold comparison and classification methods are as follows:

[0057] The sequence obtained by copying the feature vector of the current frame is denoted by S0. The sequence S1 is the depth video frame sequence of the sample acquired before real-time testing. The sequence S0 matches the depth video frame sequence S1 of the sample when the following conditions are met:

[0058] D(S0,S1) <Th(label(S1))

[0059] Where label(S1) is the classification label, and the threshold Th(label(S1)) corresponding to each gesture label is obtained by dynamic threshold calculation of the samples under the corresponding label;

[0060] During real-time recognition, after each time the depth camera captures the latest frame, the sequence of motion trajectories is compared with the sequence of depth video frames of all samples in the training set. If no match is found, the algorithm will not output anything. If there are one or more matches, the corresponding 1 / D(S0,S1) is used as the score of each matched sample, and the KNN algorithm is used to output the label with the largest cumulative score as the classification result 1.

[0061] Step 5.2: Generate the motion trajectory composed of the most recent N frames in the real-time acquired depth video frame sequence, calculate the feature value of the depth video frame sequence, and compare it with the threshold of different types of samples to obtain another classification result 2;

[0062] The gesture threshold comparison and classification methods are as follows:

[0063] The most recent N-frame depth video frame sequence is used with S 00 This indicates that sequence S2 is a sequence of sample depth video frames acquired before real-time testing. Sequence S2 is considered valid when the following conditions are met. 00 Matches with sample sequence S2:

[0064] D(S 00 S2) <Th(label(S2))

[0065] Where label(S2) is the classification label, and the threshold Th(label(S2)) corresponding to each gesture label is obtained by dynamically calculating the threshold of the samples under the corresponding label;

[0066] During real-time recognition, the most recent N-frame depth video sequence is compared with all sample depth video frame sequences in the training set. If no match is found, the algorithm does not output anything; if one or more matches are found, the corresponding 1 / D(S) is output. 00 S2) is the score of each matched sample. The KNN algorithm is used to output the label with the largest cumulative score as the classification result 2.

[0067] Step 6 of this invention includes the following steps:

[0068] Step 6.1: Use DIF1 to represent the degree of difference between the generated gesture trajectory and the gesture trajectory corresponding to classification result 1:

[0069] DIF1=|D(S0,S1)-Th(label(S1))|

[0070] Let DIF2 represent the degree of difference between the real-time gesture trajectory and the gesture trajectory corresponding to classification result 2:

[0071] DIF2=|D(S 00 ,S2)-Th(label(S2))|

[0072] Step 6.2: When DIF1 <= DIF2, the program determines the classification as T and outputs classification result 1; otherwise, the program determines the classification as F and outputs classification result 2.

[0073] The advantages of this invention are: it uses an adaptive frame sequence extraction method, taking advantage of the characteristic that static gestures are identical in every frame of a depth video frame sequence. By processing the most recent frame of depth image data in the depth video frame sequence and combining it with the feature values ​​of the most recent N frames of depth video frame sequence already acquired, a classification result is obtained, instead of waiting to extract N more frames of depth image data starting from the current frame to obtain a classification result, thus shortening the recognition time. In application, it effectively avoids the problem of requiring the tester to keep their gesture still for a period of time during static gesture recognition to obtain a gesture recognition result, improving the system's recognition speed and reducing system latency. Attached Figure Description

[0074] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0075] This invention relates to an adaptive gesture frame sequence extraction algorithm, comprising the following steps:

[0076] 1. Load and read pre-generated M=1000 random line segment data in the gesture recognition program and use a depth camera to acquire real-time depth images;

[0077] 2. Use a computer to input classification labels and record multiple depth video frame sequences corresponding to each classification label as the training set, where each depth video frame sequence contains N=30 frames, which is one sample;

[0078] Each sample has a motion trajectory. The motion trajectory is defined as follows: in a depth video frame sequence of N = 30 frames, the feature vector value f corresponding to the m-th random line segment in the n-th frame is calculated. m (n) There are a total of M = 1000 random line segments, and the feature vector p of the nth frame is... (n) =[f1 (n) f2 (n) ,...,f 1000 (n) By concatenating N feature vectors in the order of the depth video frame sequence, a trajectory T = [p] is formed. (1) ,p (2) ,...,p (N) ], which is the trajectory of the action;

[0079] Samples with the same category label are considered to be of the same category.

[0080] 3. Calculate the gesture feature value for each sample.

[0081] (1) Obtain the hand region after segmentation of the depth video frame sequence;

[0082] (2) Extract 1000 random line segment features to form a 1000-dimensional feature vector. Distribute the 1000 random line segments on the hand region of each frame of the sequence. Each random line segment corresponds to a feature vector, which consists of the following five features:

[0083] Feature 1: Defined as the proportion of pixels in the hand region containing random line segments to the total number of pixels in the entire image;

[0084] Feature 2: Defined as the average depth value of pixels in the hand area covered by random line segments;

[0085] Feature 3: Represents the position of the pixel with the minimum depth value on a random line segment;

[0086] Feature 4: Represents the position of the pixel with the maximum depth value on a random line segment;

[0087] Feature 5: Represents the variation characteristics of pixel depth values ​​in the hand region, estimated by summing the absolute values ​​of the depth differences between adjacent pixels in the hand region on random line segments;

[0088] Finally, an eigenvector value is calculated for each random line segment;

[0089] (1) Based on the obtained random line segment feature vector values, calculate the three features of the motion trajectory;

[0090] The first feature is the three-dimensional distance feature, which describes the distance of feature points on the motion trajectory in the depth video frame sequence in three-dimensional space, and can represent the appearance of gestures;

[0091] The second feature is the motion length feature, which can represent the position of a gesture in space as time changes. The formula for calculating the motion length feature is as follows:

[0092]

[0093] The third feature is the speed feature, which is obtained by calculating the speed of the hand gesture.

[0094] 4. Calculate the dynamic classification threshold for samples of the same type.

[0095] (1) Calculate the three-dimensional distance feature difference, motion length feature difference, and velocity feature difference;

[0096] Formula for calculating the feature difference of three-dimensional distance:

[0097] The two feature vectors p in the nth frame a (n) and p b(n) The distance is defined as

[0098]

[0099] Where f m,a (n) and f m,b (n) Representing the eigenvectors p a (n) and p b (n) For the m-th value out of 1000 values, the trajectory feature difference is the Euclidean distance between the two feature vectors, i.e.:

[0100]

[0101] Where Sa and Sb represent two depth video frame sequences for which feature difference calculation is required;

[0102] Formula for calculating the difference in motion length characteristics:

[0103]

[0104] Among them, T l and T s These represent the motion trajectories T corresponding to depth video frame sequences Sa and Sb, respectively. a and T b Trajectories with relatively large and small eigenvalues ​​in medium length;

[0105] Formula for calculating characteristic velocity difference:

[0106]

[0107] Where v a (n) and v b (n) These are respectively represented as the depth video frame sequence S a and S b In the middle, the speed at which the hand moves in the nth frame;

[0108] (2) Calculate the dynamic threshold of the sample

[0109] Each gesture label corresponds to a threshold Th(Label(S)). The threshold is calculated by taking the weighted average of the median differences of the three features between all pairs of samples with the same label in the training set, i.e.:

[0110] Th(Label(S))=w1·medianD1(S a ,S b )+w2·medianD2(S a ,Sb )+w3·medianD3(S a ,S b (6)

[0111] Two depth video frame sequences S a and S b Differences are represented by D(S) a ,S b The formula for its calculation is as follows:

[0112] D(S a ,S b )=w1·medianD1(S a ,S b )+w2·medianD2(S a ,S b )+w3·medianD3(S a ,S b (7)

[0113] Where w1, w2, and w3 represent the weights of the three feature differences. Through multiple experiments, the w1, w2, and w3 values ​​corresponding to the case with the highest average recognition rate are taken as the weights of the dynamic threshold.

[0114] 5. Simultaneously, by calculating the current frame image data and the most recent thirty frames image data acquired in real time, and comparing them with different label sample thresholds, two classification results are obtained;

[0115] (1) Process the real-time acquired image data of the current frame to obtain a classification result 1;

[0116] The gesture features of the most recent frame of the real-time depth video are extended by copying the feature vector of the current frame to obtain the same number of frames as the sample, i.e., the number of feature vectors of thirty frames. These vectors are used to form a motion trajectory, and the feature value of the processed frame sequence is calculated. The difference between this value and the sample feature is calculated and compared with the sample threshold to output a classification result 1.

[0117] The gesture threshold comparison and classification methods are as follows:

[0118] The sequence obtained by copying the feature vector of the current frame is denoted as S0, and the sequence S1 is the depth video frame sequence of the sample acquired before real-time testing. The sequence S0 matches the depth video frame sequence S1 of the sample when the following conditions are met:

[0119] D(S0,S1) <Th(label(S1))(8)

[0120] Where label(S1) is the classification label. The threshold Th(label(S1)) corresponding to each gesture label is obtained by dynamically calculating the threshold for the samples under the corresponding label;

[0121] During real-time recognition, after the depth camera captures the latest frame, the sequence of motion trajectories is compared with the sequence of depth video frames of all samples in the training set according to the above formula (8). If no matching item is found, the algorithm will not output; if there is one or more matching items, the corresponding 1 / D(S0,S1) is used as the score of each matching sample, and the KNN algorithm is used to output the label with the largest cumulative score as the classification result 1.

[0122] (2) Generate the motion trajectory of the most recent 30 frames in the real-time acquired depth video frame sequence, calculate the feature value of the depth video frame sequence, and compare it with the threshold of different types of samples to obtain another classification result 2.

[0123] The gesture threshold comparison and classification methods are as follows:

[0124] The most recent thirty-frame depth video frame sequence is used with S 00 This indicates that sequence S2 is a sequence of sample depth video frames acquired before real-time testing. Sequence S2 is considered valid when the following conditions are met. 00 Matches with sample sequence S2:

[0125] D(S 00 S2) <Th(label(S2))(9)

[0126] Where label(S2) is the classification label. The threshold Th(label(S2)) corresponding to each gesture label is obtained by dynamically calculating the threshold for the samples under the corresponding label;

[0127] During real-time recognition, the most recent thirty-frame depth video sequence is compared with all sample depth video frame sequences in the training set according to the above formula (9). If no match is found, the algorithm will not output; if one or more match items are found, the corresponding 1 / D(S 00 S2) is the score of each matched sample. The KNN algorithm is used to output the label with the largest cumulative score as the classification result 2.

[0128] 6. Compare the two classification results and output the classification result that differs less from the sample as the final gesture classification result;

[0129] Let DIF1 represent the degree of difference between the generated gesture trajectory and the gesture trajectory corresponding to classification result 1:

[0130] DIF1=|D(S0,S1)-Th(label(S1))|(9)

[0131] Let DIF2 represent the degree of difference between the real-time gesture trajectory and the gesture trajectory corresponding to classification result 2:

[0132] DIF2=|D(S 00 ,S2)-Th(label(S2))|(10)

[0133] When DIF1 <= DIF2, the program determines the classification as T and outputs classification result 1; otherwise, the program determines the classification as F and outputs classification result 2.

Claims

1. A real-time gesture recognition method based on an adaptive gesture frame sequence extraction algorithm, characterized in that: Includes the following steps: Step 1: Load and read the pre-generated M random line segment data in the gesture recognition program and use a depth camera to acquire real-time depth images; Step 2: Use a computer to input classification labels and record multiple depth video frame sequences corresponding to each classification label as a training set, where each depth video frame sequence contains N frames, which is one sample; Step 3: Calculate the gesture feature value for each sample; Step 4: Calculate the dynamic classification threshold for samples of the same type, including the following steps: Step 4.1: Calculate the three-dimensional distance feature difference, motion length feature difference, and velocity feature difference; Formula for calculating the feature difference of three-dimensional distance: The two feature vectors of the nth frame and The distance is defined as: ； in and They represent the eigenvectors respectively. and The m-th value out of M values ​​represents the Euclidean distance between two feature vectors, specifically the 3D distance feature difference. ； Where Sa and Sb represent two depth video frame sequences for which feature difference calculation is required; Formula for calculating the difference in motion length characteristics: ； in, and These represent the motion trajectories corresponding to depth video frame sequences Sa and Sb, respectively. and Trajectories with relatively large and small eigenvalues ​​in medium length; Formula for calculating velocity characteristic difference: ； Where v a (n) and v b (n) These are respectively represented as the depth video frame sequence S a and S b In the middle, the speed at which the hand moves in the nth frame; Step 4.2: Calculate the dynamic threshold of the sample; Each gesture label corresponds to a threshold Th(Label(S)). The threshold is calculated by taking the weighted average of the median differences of the three features between all pairs of samples with the same label in the training set, i.e.: ; in , , This represents the weights assigned to the differences among the three features. Through multiple experiments, the value corresponding to the highest average recognition rate was selected. , , The value serves as the weight for the dynamic threshold; Two depth video frame sequences S a and S b Differences The calculation formula is as follows: ； in , , This represents the weights assigned to the differences among the three features. Through multiple experiments, the value corresponding to the highest average recognition rate was selected. , , The value serves as the weight for the dynamic threshold; Step 5: Simultaneously, by calculating the current frame of image data acquired in real time and the sequence of the most recent N frames of depth video frames, two classification results are obtained, including the following steps: Step 5.1: Process the data of the current frame depth image acquired in real time to obtain a classification result 1; The gesture features of the most recent frame of the real-time depth video are extended by copying the feature vector of the current frame to obtain the same number of frames as the sample, i.e., N frames of feature vectors. These vectors are used to form a motion trajectory, and the feature value of the processed frame sequence is calculated. The difference between this value and the sample feature is calculated and compared with the sample threshold to output a classification result 1. The gesture threshold comparison and classification methods are as follows: The sequence obtained by copying the feature vector of the current frame is denoted by S0. The sequence S1 is the depth video frame sequence of the sample acquired before real-time testing. The sequence S0 matches the depth video frame sequence S1 of the sample when the following conditions are met: ； Where label(S1) is the classification label, and the threshold Th(label(S1)) corresponding to each gesture label is obtained by dynamic threshold calculation of the samples under the corresponding label; During real-time recognition, after each frame captured by the depth camera, the sequence of motion trajectories is compared with all sample depth video frame sequences in the training set. If no match is found, the algorithm does not output anything; if one or more matches are found, the corresponding... As the score for each matched sample, the KNN algorithm is used to output the label with the largest cumulative score as the classification result 1; Step 5.2: Generate the motion trajectory composed of the most recent N frames in the real-time acquired depth video frame sequence, calculate the feature value of the depth video frame sequence, and compare it with the threshold of different types of samples to obtain another classification result 2; The gesture threshold comparison and classification methods are as follows: The most recent N-frame depth video frame sequence is used with S 00 This indicates that sequence S2 is a sequence of sample depth video frames acquired before real-time testing. Sequence S2 is considered valid when the following conditions are met. 00 Matches with sample sequence S2: ； Where label(S2) is the classification label, and the threshold Th(label(S2)) corresponding to each gesture label is obtained by dynamically calculating the threshold of the samples under the corresponding label; During real-time recognition, the most recent N-frame depth video sequence is compared with all sample depth video frame sequences in the training set. If no match is found, the algorithm does not output anything; if one or more matches are found, the corresponding... As the score for each matched sample, the KNN algorithm is used to output the label with the largest cumulative score as the classification result 2; Step 6: Compare the two classification results and output the classification result that differs less from the sample as the final gesture classification result.

2. The real-time gesture recognition method based on an adaptive gesture frame sequence extraction algorithm according to claim 1, characterized in that: Step 1 specifically involves the following: Based on experimental observations, in order to ensure that random line segments are distributed across the image as much as possible so as to fully describe the gesture, the value of M should be no less than 1000.

3. The real-time gesture recognition method based on an adaptive gesture frame sequence extraction algorithm according to claim 1, characterized in that: Step 2 specifically involves: Each sample has a motion trajectory, which is defined as follows: In a depth video frame sequence of N frames, the feature vector value corresponding to the m-th random line segment in the n-th frame is calculated to be f. m (n) There are M random line segments in total, and the feature vector p of the nth frame is... (n) = [f1 (n) f2 (n) , ... , f M (n) By concatenating N feature vectors in the order of the depth video frame sequence, a trajectory T = [p] is formed. (1) , p (2) , ..., p (N) The frame number N is the motion trajectory. According to experimental observations, in order to take into account both static and dynamic gestures with a large amount of data, the frame number N is selected to be no less than 30, and samples with the same classification label are considered to be of the same type.

4. The real-time gesture recognition method based on an adaptive gesture frame sequence extraction algorithm according to claim 1, characterized in that: Step 3 includes the following steps: Step 3.1: Obtain the hand region after segmenting the depth video frame sequence; Step 3.2: Extract the feature vector values ​​of M random line segments; M randomly generated line segments are scattered across the hand region of each frame in the sequence. Each random line segment corresponds to a feature vector, which consists of the following five features: Feature 1: Defined as the proportion of pixels in the hand region containing random line segments to the total number of pixels in the entire image; Feature 2: Defined as the average depth value of pixels in the hand area covered by random line segments; Feature 3: Represents the position of the pixel with the minimum depth value on a random line segment; Feature 4: Represents the position of the pixel with the maximum depth value on a random line segment; Feature 5: Represents the variation characteristics of pixel depth values ​​in the hand region, estimated by summing the absolute values ​​of the depth differences between adjacent pixels in the hand region on random line segments; Finally, an eigenvector value is calculated for each random line segment; Step 3.3: Based on the random line segment feature vector values ​​obtained in Step 3.2, calculate the three features of the motion trajectory; The first feature is the three-dimensional distance feature, which describes the distance of feature points on the motion trajectory in the depth video frame sequence in three-dimensional space, and can represent the appearance of gestures; The second feature is the motion length feature, which can represent the position of a gesture in space as time changes. The formula for calculating the motion length feature is as follows: ； The third characteristic is the speed characteristic, which is obtained by calculating the speed of the gesture movement.

5. The real-time gesture recognition method based on an adaptive gesture frame sequence extraction algorithm according to claim 1, characterized in that: Step 6 includes the following steps: Step 6.1: Use DIF1 to represent the degree of difference between the generated gesture trajectory and the gesture trajectory corresponding to classification result 1: ； Let DIF2 represent the degree of difference between the real-time gesture trajectory and the gesture trajectory corresponding to classification result 2: ； Step 6.2: When DIF1 <= DIF2, the program determines the classification as T and outputs classification result 1; otherwise, the program determines the classification as F and outputs classification result 2.

Images