Adaptive kalman-based dynamic gesture recognition and intent prediction method

Abstract

This invention discloses a dynamic gesture recognition and intent prediction method based on adaptive Kalman filtering. By employing multi-channel spatiotemporal feature encoding, adaptive noise estimation Kalman filtering, intent prediction branching, decision fusion and instruction caching, and a lightweight deployment strategy, it successfully resolves the triangular contradiction of "poor robustness, high latency, and high computing power" in dynamic gesture recognition. This improves the robustness of gesture recognition, reduces latency, and enables advance prediction of user intent, thereby providing a more natural, smooth, and efficient interactive experience for intelligent interactive devices.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and human-computer interaction technology, specifically to a dynamic gesture recognition and intent prediction method based on adaptive Kalman blotting. Background Technology

[0002] In today's digital age, human-computer interaction technology is developing towards a more natural and intuitive direction, and dynamic gesture recognition, as an important branch of this, has received widespread attention. Dynamic gesture recognition technology allows users to interact with devices through natural gestures, without the need for traditional physical buttons or touchscreen operations, bringing a brand-new interactive experience to various application scenarios such as smart TVs, augmented reality (AR) devices, and in-vehicle systems.

[0003] However, most existing dynamic gesture recognition systems have some limitations. On the one hand, many systems use deep neural networks to directly regress gesture categories. These models typically have large parameters, resulting in high computational latency and making them difficult to deploy in real time on low-cost devices such as mobile phones and embedded gateways. On the other hand, although some systems introduce Kalman filtering (KF) to denoise key points, the process noise and observation noise are fixed and cannot adapt to uncertainties such as sudden changes in lighting, motion blur, and hand self-occlusion, which leads to a decrease in recognition accuracy in practical applications. In addition, most existing methods only stay at the "recognition" level and lack the ability to predict users' future actions in advance, resulting in interaction lag and affecting user experience. Summary of the Invention

[0004] The present invention aims to solve the problems of poor robustness, high latency, high computing power, and lag in existing technologies, and provides a dynamic gesture recognition and intent prediction method based on adaptive Kalman.

[0005] To solve the above problems, the present invention is achieved through the following technical solution:

[0006] The method for dynamic gesture recognition and intent prediction based on adaptive Kalman blotting includes the following steps:

[0007] Step 1: Acquire real-time dynamic video of the hand, and process the real-time dynamic video of the hand using the adaptive noise estimation Kalman filter method to obtain the filtered real-time dynamic video of the hand.

[0008] Step 2: Extract composite features from the filtered real-time hand video, which contain geometric and motion information of the hand. The composite features include the coordinates of key points of the hand, the speed of movement of key points of the hand, the angle of finger joints, and the distance between fingertips.

[0009] Step 3: Construct the intent prediction branch network, and train the intent prediction branch network by jointly optimizing the position regression loss and the category cross-entropy loss to obtain the intent prediction model;

[0010] Step 4: Based on the composite features obtained in Step 2, directly obtain the gesture recognition result at the current moment. At the same time, feed the composite features obtained in Step 2 into the intent prediction model in Step 3 to predict the gesture prediction result at the next moment.

[0011] Step 5: Perform Bayesian fusion on the current gesture recognition result and the next gesture prediction result to obtain the final gesture decision result for the next moment; when the final gesture decision results for the next moment are consistent at least twice in a row, trigger the execution of the gesture instruction corresponding to the final gesture decision result.

[0012] In step 1 above, the adaptive noise estimation Kalman filter method is based on the Kalman filter method. First, a noise estimator is added after the prediction step of the Kalman filter. This noise estimator uses the residuals within the sliding window to calculate the covariance matching statistic. Then, the process noise covariance matrix and the measurement noise covariance matrix are updated online based on the covariance matching statistic using the Sage-Husa suboptimal unbiased estimation method.

[0013] The above-mentioned Covariance matching statistic at time step for:

[0014] ,

[0015] In the formula, Indicates the first One observation value, (i) indicates the first There are several predicted state values, where H represents the observation matrix. Indicates the size of the sliding window. Indicates the current moment. This indicates transpose.

[0016] In the adaptive noise estimation Kalman filter method, an adaptive adjustment factor is introduced for fast-moving frames. And amplify the process noise covariance matrix. times;

[0017] The above adaptive adjustment factor for:

[0018] ,

[0019] In the formula, Indicates the speed of the key point. This indicates the preset speed threshold. This represents the function to be minimized.

[0020] In step 2 above,

[0021] Key points of the hand coordinate The coordinates of key hand points output by the MediaPipe gesture recognition model;

[0022] Key points of the hand Speed ​​of movement for:

[0023] ,

[0024] ,

[0025] finger joints and angle for:

[0026] ,

[0027] fingertips and distance for:

[0028] ,

[0029] in, and These represent the key points of the hand. In the current frame of coordinates and coordinate, and These represent the key points of the hand. In the previous frame of coordinates and coordinate; ; and Representing joints and joints ; and Representing the fingertips of coordinates and coordinate, and Representing the fingertips of coordinates and coordinate.

[0030] In step 3 above, the intention prediction branch network consists of a one-dimensional convolutional neural network, a bidirectional long short-term memory network, and a fully connected layer; the input of the one-dimensional convolutional neural network forms the input of the intention prediction branch network, the output of the one-dimensional convolutional neural network is connected to the input of the bidirectional long short-term memory network, and the output of the bidirectional long short-term memory network forms the output of the intention prediction branch network.

[0031] In step 3 above, the total loss function for training the intention prediction branch network is:

[0032] ,

[0033] ,

[0034] ,

[0035] in, Indicates the weighting coefficient. This represents the position regression loss. Represents the category cross-entropy loss; It is the actual offset of the key point position. It is the predicted key point position offset. It is the total number of key points; These are actual gesture category labels. is the predicted probability of the gesture category, and M is the total number of gesture categories.

[0036] In step 5 above, the calculation formula for Bayesian fusion is as follows:

[0037]

[0038] in, This represents the probability of the final decision outcome of the gesture at the next moment. This represents the probability of the gesture recognition result at the current moment. This represents the probability of the gesture prediction result at the next moment. This represents the measured delay of the system.

[0039] Compared with the prior art, the present invention has the following characteristics:

[0040] (1) Noise Adaptive Mechanism: In existing Kalman filter gesture recognition systems, the process noise covariance matrix Q and observation noise covariance matrix R are usually determined offline and kept fixed during operation. This method can lead to filter divergence when faced with uncertainties such as sudden changes in illumination, motion blur, or hand self-occlusion, making it impossible to accurately track the movement of key points. This invention introduces the Sage-Husa online unbiased estimator into 21-dimensional hand key point filtering for the first time and designs a velocity-adaptive amplification factor α(k) to achieve frame-level updates of Q and R. This noise adaptive mechanism does not require additional sensors and can dynamically correct noise statistical characteristics using only visual residuals, which is significantly different from the traditional fixed noise model. In this way, this invention can dynamically adjust the noise estimation in different environments, improve the robustness and accuracy of filtering, and provide more stable and smooth key point data for subsequent intention prediction.

[0041] (2) Parallel Architecture for Intent Prediction and Recognition: Traditional gesture recognition methods typically employ a "smoothing before classification" approach, where the recognition result only reflects the action of the current frame and cannot predict the user's intent in advance, leading to interaction lag. This invention proposes a parallel branch of IntentNet, using an ANE-KF-smoothed trajectory as input to directly learn joint offsets and gesture probabilities for the next 150ms. This "prediction-recognition" dual-task joint training strategy overcomes the causal limitations of traditional pipelines, enabling early response. Through this parallel architecture, this invention can not only accurately recognize the current gesture but also predict the user's intent in advance, providing the possibility for early response to user actions and thus achieving more natural and fluid human-computer interaction.

[0042] (3) Decision-level Bayesian fusion caching: To address the uncertainty brought about by prediction, this invention introduces a latency-weighted Bayesian fusion strategy, dynamically weighting the current observation and prediction results according to system latency, and setting a three-frame consistent cache. This strategy differs from simple threshold voting and a single probability threshold, effectively suppressing occasional jitter and improving the stability of instructions. In this way, this invention can obtain stable and reliable decision results based on prediction, thereby achieving smooth and natural human-computer interaction. This decision-level fusion caching mechanism provides an effective solution for handling prediction uncertainty, improving the overall system performance and user experience.

[0043] Through the above three innovations, this invention solves the triangular contradiction of "poor robustness, high latency, and high computing power" in dynamic gesture recognition, and provides a brand-new technical path for low-cost, low-latency, and predictable contactless interaction, which has important theoretical and practical significance. Attached Figure Description

[0044] Figure 1This is a diagram of 21 key nodes of the hand. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific examples and the accompanying drawings.

[0046] The method for dynamic gesture recognition and intent prediction based on adaptive Kalman blotting includes the following steps:

[0047] Step 1, Adaptive Noise Estimation Kalman Filter (ANE-KF): Acquire real-time dynamic video of the hand, and process the real-time dynamic video of the hand using the adaptive noise estimation Kalman filter method to obtain the filtered real-time dynamic video of the hand.

[0048] Kalman filtering (KF) is a commonly used recursive filtering algorithm for estimating the state of dynamic systems. However, traditional Kalman filtering has a limitation: its process noise covariance matrix Q and observation noise covariance matrix R are usually fixed. This makes it unable to adapt well to uncertainties such as sudden changes in illumination, motion blur, and hand self-occlusion, resulting in poor filtering performance. To address this problem, this invention proposes an adaptive noise estimation Kalman filtering method.

[0049] (1) Design of noise estimator

[0050] First, after the prediction step of the Kalman filter, we add a noise estimator. Specifically, we use the residuals within the sliding window to calculate the covariance matching statistic.

[0051] Let the size of the sliding window be N, and the first... Time residuals for:

[0052] ,

[0053] Where z(k) are the observed values, and H is the observation matrix. (k) is the predicted state value.

[0054] Covariance matching statistic for:

[0055] ,

[0056] Then, the process noise covariance matrix Q(k) and the measurement noise covariance matrix R(k) are updated online based on the covariance matching statistic using the Sage-Husa suboptimal unbiased estimation method.

[0057] Sage-Husa adaptive filtering is an adaptive filtering method based on maximum a posteriori (MAP) estimation. Its core idea is to utilize the innovation residuals during the filtering process to estimate the process noise covariance matrix Q and the measurement noise covariance matrix R in real time using recursive formulas, thus maintaining the optimality of the filter even with inaccurate prior noise information. The Sage-Husa suboptimal unbiased maximum a posteriori estimator is the core mechanism of Sage-Husa adaptive filtering. It estimates the statistical characteristics of the noise online through a recursive method, thereby solving the accuracy degradation problem caused by inaccurate noise parameters in traditional Kalman filtering.

[0058] (2) Processing of fast motion frames

[0059] For fast-moving frames, motion blur can cause filter drift. To suppress this drift, we introduce an adaptive adjustment factor. :

[0060] ,

[0061] in, It is the speed at the key point. It is a preset speed threshold.

[0062] In fast-moving frames, Q(k) is amplified. times, that is

[0063] ,

[0064] In this way, by increasing the process noise covariance matrix Q, the uncertainty of filtering can be increased, thereby reducing the reliance on model prediction during rapid motion and decreasing the impact of motion blur on the filtering results.

[0065] Through the aforementioned adaptive noise estimation and fast motion frame processing mechanisms, ANE-KF can dynamically adjust noise estimation under different environments, improve the robustness and accuracy of filtering, and provide more stable and smooth key point data for subsequent intent prediction.

[0066] Step 2, Multi-channel spatiotemporal feature encoding: Extract composite features containing the geometric shape information and motion dynamic information of the hand from the filtered real-time dynamic video of the hand. The composite features include the coordinates of the hand key points, the motion speed of the hand key points, the finger joint angles, and the fingertip distance.

[0067] Before performing gesture recognition, the input hand data needs to be feature-encoded to extract composite features that contain both geometric and dynamic information.

[0068] (1) Coordinates of key points of the hand

[0069] This invention is based on 21 key hand points output by the MediaPipe gesture recognition model. These key points can comprehensively describe the hand's posture and shape, such as... Figure 1 As shown. Each keypoint has a coordinate position in the two-dimensional image coordinate system. , Therefore, initially we have 42 dimensions of raw data (21 key points × 2 coordinate dimensions).

[0070] (2) Speed ​​of movement of key points of the hand

[0071] To capture the dynamic information of the hand, we introduce a motion velocity vector. For each keypoint, its motion velocity vector... This can be obtained by calculating the coordinate difference of the key point in two consecutive frames, i.e.:

[0072] ,

[0073] ,

[0074] in, and They represent key points respectively. In the current frame of coordinates and coordinate, and They represent key points respectively. In the previous frame of coordinates and coordinate, .

[0075] This adds 2 dimensions of speed information to each key point, for a total of 42 dimensions, bringing the total to 84 dimensions.

[0076] (3) Finger joint angle

[0077] The joint angles of the hand can effectively reflect the hand's posture characteristics. For adjacent joints on the fingers... and The angle between The angle value of each joint can be calculated using the formula for the angle between vectors, i.e.:

[0078] ,

[0079] in, and These represent two joint vectors respectively.

[0080] Each finger has 3 joints. The joint angle between any two adjacent joints is calculated. Each finger gets an angle sequence containing 3 angle values. In this way, 5 fingers can get 5 angle sequences, each containing 3 angle values, for a total of 15 dimensions.

[0081] (4) Distance between fingertips

[0082] The distance between fingertips can also provide important clues for gesture recognition. and fingertips Euclidean distance between The calculation formula is:

[0083] ,

[0084] in, and Representing the fingertips of coordinates and coordinate, and Representing the fingertips of coordinates and coordinate.

[0085] 5×5 fingertip distance matrix There are 25 elements in total, but due to symmetry, there are actually 15 independent elements, thus adding 15 dimensions.

[0086] By introducing motion velocity vectors (motion velocity of hand key points), joint angle sequences (finger joint angles), and fingertip distance matrices (finger-to-finger distances), we expand the original key point coordinate data into composite features. These composite features contain both the geometric shape information of the hand and the dynamic motion information of the hand, which can more comprehensively describe the features of the gesture and provide an information foundation for subsequent processing.

[0087] Step 3: Intent Prediction Branch Network: Construct the intent prediction branch network and train it by jointly optimizing the position regression loss and the category cross-entropy loss to obtain the intent prediction model.

[0088] In practical applications, simply recognizing the current category of a gesture is insufficient. To achieve more natural and fluid human-computer interaction, it is also necessary to predict the user's intent in advance. Therefore, this invention designs an intent prediction branch network, IntentNet, to predict the positional shift of key points and the probability of gesture categories over a future period.

[0089] (1) Network structure

[0090] The IntentNet network architecture comprises a one-dimensional convolutional neural network (CNN), a bidirectional long short-term memory (LSTM) network, and fully connected layers. The input to the CNN forms the input to the IntentNet, and the output of the CNN is connected to the input of the LSTM network, with the output of the LSTM network forming the output of the IntentNet. A 1D-CNN (One-Dimensional Convolutional Neural Network) is a convolutional neural network specifically designed for processing sequential data. 1D-CNNs have 32-64 channels and are used to extract local features of keypoint sequences over time, capturing local patterns of keypoint motion through convolutional operations. A bidirectional LSTM is a variant of LSTM, containing 32 hidden units. It considers both past and future information, demonstrating excellent performance in modeling sequential data and learning long-term dependencies in keypoint sequences. Fully connected layers map the LSTM output to the prediction target space, obtaining the future keypoint position offset and gesture category probability.

[0091] (2) Training objectives

[0092] The training objective of the IntentNet network is to jointly optimize the position regression loss and the class cross-entropy loss.

[0093] Location regression loss Used to measure the difference between the predicted keypoint location offset and the actual location offset, i.e.:

[0094] ,

[0095] in, It is the actual offset of the key point position. It is the predicted key point position offset. It is the total number of key points.

[0096] Category cross-entropy loss This is used to measure the difference between the predicted gesture category probability and the true category, i.e.:

[0097] ,

[0098] in, These are actual gesture category labels (using one-hot encoding). is the predicted probability of the gesture category, and M is the total number of gesture categories.

[0099] The total loss function is

[0100] ,

[0101] in, It is a weighting coefficient that balances the two losses. In this invention, it is set to 0.7, which allows for greater emphasis on the accuracy of location regression while not neglecting the accuracy of category recognition.

[0102] By using the IntentNet network, this invention can predict the positional shift of key points and the probability of gesture categories approximately 150ms in the future, thereby predicting the user's intentions in advance and making it possible to respond to user actions in advance.

[0103] Step 4: Obtain the gesture recognition result: Based on the composite features obtained in Step 2, directly obtain the gesture recognition result at the current moment, and then feed the composite features obtained in Step 2 into the intention prediction model in Step 3 to predict the gesture prediction result at the next moment.

[0104] Since the composite features for the current moment are obtained from the current video frame, including the coordinates of hand key points, the speed of hand key point movement, finger joint angles, and fingertip distance, the gesture recognition result for the current moment can be directly determined based on the composite features. However, the composite features for the next moment cannot be directly obtained from the current video frame; therefore, an intent prediction model is needed to predict the hand intent for the next moment.

[0105] Step 5, Decision Fusion: Perform Bayesian fusion on the gesture recognition result at the current moment and the gesture prediction result at the next moment to obtain the final decision result of the gesture at the next moment; when the final decision result of the gesture at the next moment is consistent for at least two consecutive times, trigger the execution of the gesture instruction corresponding to the final decision result of the gesture.

[0106] (1) Bayesian fusion

[0107] The current gesture recognition result And the next moment's gesture prediction results Perform Bayesian fusion. Specifically, calculate the final probability. :

[0108]

[0109] in, This represents the probability of the final decision outcome of the gesture at the next moment. This represents the probability of the gesture recognition result at the current moment. This represents the probability of the gesture prediction result at the next moment. This represents the measured delay of the system.

[0110] This fusion method takes into account the impact of system latency on the prediction results, and makes the fusion results more reasonable and reliable by using latency weighting.

[0111] (2) Instruction cache

[0112] To avoid jitter caused by the uncertainty of prediction, this invention establishes an instruction buffer. The gesture instruction corresponding to the final gesture decision result is only triggered when the final gesture decision results are consistent across three consecutive frames. This effectively suppresses occasional jitter and improves the stability and reliability of the instructions.

[0113] The decision fusion and instruction caching mechanism can obtain stable and reliable decision results based on prediction, thereby achieving smooth and natural human-computer interaction.

[0114] In order to run in real time on resource-constrained terminal devices, this invention adopts a lightweight deployment strategy.

[0115] (1) Lightweight implementation of ANE-KF

[0116] ANE-KF is implemented entirely based on NumPy, resulting in very low computational cost; at 720p resolution, the computation time is less than 0.5ms. NumPy is a commonly used Python scientific computing library with efficient matrix operation capabilities. Through proper algorithm design and optimization, ANE-KF can be ensured to run quickly on resource-constrained devices.

[0117] (2) Lightweighting of IntentNet

[0118] IntentNet has approximately 42k parameters, which are quantized to INT8 format using TensorFlow-Lite, further reducing the model's computational and storage requirements. The quantized model maintains high accuracy while achieving an inference latency of only 2.3ms. TensorFlow-Lite is a lightweight deep learning framework specifically designed for deploying deep learning models on mobile and embedded devices. Through techniques such as model quantization, it significantly improves the model's runtime efficiency.

[0119] (3) System resource usage

[0120] The entire system has a total memory footprint of less than 30MB, enabling it to run in real time on low-cost devices such as the Raspberry Pi 4. This makes the method of this invention applicable not only to high-performance computer devices but also to various resource-constrained terminal devices, demonstrating excellent versatility and practicality.

[0121] The lightweight deployment strategy can ensure the effectiveness of identification and prediction while meeting the requirements for real-time operation on resource-constrained devices, making it possible for widespread application in various practical application scenarios.

[0122] The dynamic gesture recognition and intent prediction method of the present invention can be implemented through various specific embodiments. The following are some typical application scenarios:

[0123] Smart TV Interaction: In smart TV interaction scenarios, this invention can be integrated into the TV's system-on-chip (SoC). Users can adjust the volume using simple gestures, such as drawing a circle with their index finger. Utilizing the dynamic gesture recognition and intent prediction method of this invention, the system can predict the user's gesture intent 150 milliseconds in advance and adjust the volume bar accordingly, thus providing a smooth and lag-free interactive experience. This ability to predict in advance makes the user feel more natural and fluid during operation, as if the TV can "understand" their intentions.

[0124] Air Input for AR Glasses: This invention also plays a crucial role in augmented reality (AR) glasses applications. In this scenario, the AR glasses primarily handle image acquisition, while ANE-KF (Adaptive Noise Estimation Kalman Filter) and IntentNet (Intent Prediction Network) run on the mobile device's coprocessor. Through Bluetooth technology, predicted commands can be quickly transmitted back to the AR glasses, enabling hands-free air typing. Experiments show that using this method significantly improves character input speed, providing users with an efficient and convenient interaction method.

[0125] In-vehicle gesture control: In in-vehicle systems, driver safety is paramount. This invention can be deployed on the Kirin A55 chip in the cockpit, allowing drivers to switch songs or adjust other in-vehicle functions using simple gestures such as waving. The system can complete interface switching half a beat in advance, thereby reducing the driver's eye-off time and improving driving safety. This rapid response capability is achieved through the intent prediction and decision fusion mechanism of this invention, making the in-vehicle gesture control system more intelligent and reliable.

[0126] This invention successfully resolves the triangular contradiction of "poor robustness, high latency, and high computational cost" in dynamic gesture recognition by employing multi-channel spatiotemporal feature encoding, adaptive noise estimation Kalman filtering (ANE-KF), IntentNet with intent prediction branch, decision fusion and instruction caching, and a lightweight deployment strategy. It improves the robustness of gesture recognition, reduces latency, and enables advance prediction of user intent, thereby providing a more natural, smooth, and efficient interactive experience for intelligent interactive devices. This invention is not only theoretically innovative but also demonstrates significant advantages in practical applications. Whether in smart TV interaction, AR glasses air input, or in-vehicle gesture control scenarios, this invention provides a low-cost, low-latency, and predictable contactless interaction solution, paving a new path for the future development of human-computer interaction technology. With continuous technological advancements and the expansion of application scenarios, this invention is expected to be widely applied in more fields, bringing greater convenience and innovative experiences to people's lives and work.

[0127] It should be noted that although the embodiments described above are illustrative, they are not intended to limit the invention. Therefore, the invention is not limited to the specific embodiments described above. Any other embodiments obtained by those skilled in the art under the guidance of this invention without departing from its principles are considered to be within the protection scope of this invention.

Claims

1. A dynamic gesture recognition and intent prediction method based on adaptive Kalman blotting, characterized by: The steps include the following: Step 1: Acquire real-time dynamic video of the hand, and process the real-time dynamic video of the hand using the adaptive noise estimation Kalman filter method to obtain the filtered real-time dynamic video of the hand. Step 2: Extract composite features from the filtered real-time hand video, which contain geometric and motion information of the hand. The composite features include the coordinates of key points of the hand, the speed of movement of key points of the hand, the angle of finger joints, and the distance between fingertips. Step 3: Construct the intent prediction branch network, and train the intent prediction branch network by jointly optimizing the position regression loss and the category cross-entropy loss to obtain the intent prediction model; Step 4: Based on the composite features obtained in Step 2, directly obtain the gesture recognition result at the current moment. At the same time, feed the composite features obtained in Step 2 into the intent prediction model in Step 3 to predict the gesture prediction result at the next moment. Step 5: Perform Bayesian fusion on the current gesture recognition result and the next gesture prediction result to obtain the final gesture decision result for the next moment; When the final decision results of the gesture obtained at least twice in a row are consistent, the gesture instruction corresponding to the final decision result of the gesture is triggered.

2. The dynamic gesture recognition and intent prediction method based on adaptive Kalman blotting as described in claim 1, characterized in that, In step 1, the adaptive noise estimation Kalman filter method is based on the Kalman filter method. First, a noise estimator is added after the prediction step of the Kalman filter. This noise estimator uses the residuals within the sliding window to calculate the covariance matching statistic. Then, the process noise covariance matrix and the measurement noise covariance matrix are updated online based on the covariance matching statistic using the Sage-Husa suboptimal unbiased estimation method.

3. The dynamic gesture recognition and intent prediction method based on adaptive Kalman blotting as described in claim 2, characterized in that, No. Covariance matching statistic at time step for: ， In the formula, Indicates the first One observation value, (i) indicates the first There are several predicted state values, where H represents the observation matrix. Indicates the size of the sliding window. Indicates the current moment. This indicates transpose.

4. The dynamic gesture recognition and intent prediction method based on adaptive Kalman blotting as described in claim 2, characterized in that, In the adaptive noise estimation Kalman filter method, an adaptive adjustment factor is introduced for fast-moving frames. And amplify the process noise covariance matrix. times; The above adaptive adjustment factor for: ， In the formula, Indicates the speed of the key point. This indicates the preset speed threshold. This represents the function to be minimized.

5. The dynamic gesture recognition and intent prediction method based on adaptive Kalman blotting as described in claim 1, characterized in that, In step 2, Key points of the hand coordinate The coordinates of key hand points output by the MediaPipe gesture recognition model; Key points of the hand Speed ​​of movement for: ， ， finger joints and angle for: ， fingertips and distance for: ， in, and These represent the key points of the hand. In the current frame of coordinates and coordinate, and These represent the key points of the hand. In the previous frame of coordinates and coordinate; ; and Representing joints and joints ; and Representing the fingertips of coordinates and coordinate, and Representing the fingertips of coordinates and coordinate.

6. The dynamic gesture recognition and intent prediction method based on adaptive Kalman blotting as described in claim 1, characterized in that, In step 3, the intention prediction branch network consists of a one-dimensional convolutional neural network, a bidirectional long short-term memory network, and a fully connected layer. The input of the one-dimensional convolutional neural network forms the input of the intention prediction branch network, the output of the one-dimensional convolutional neural network is connected to the input of the bidirectional long short-term memory network, and the output of the bidirectional long short-term memory network forms the output of the intention prediction branch network.

7. The dynamic gesture recognition and intent prediction method based on adaptive Kalman blotting as described in claim 1, characterized in that, In step 3, the total loss function for training the intention prediction branch network is: ， ， ， in, Indicates the weighting coefficient. This represents the position regression loss. Represents the category cross-entropy loss; It is the actual offset of the key point position. It is the predicted key point position offset. It is the total number of key points; These are actual gesture category labels. is the predicted probability of the gesture category, and M is the total number of gesture categories.

8. In the adaptive Kalman-based dynamic gesture recognition and intent prediction method according to claim 1, the Bayesian fusion calculation formula in step 5 is as follows: in, This represents the probability of the final decision outcome of the gesture at the next moment. This represents the probability of the gesture recognition result at the current moment. This represents the probability of the gesture prediction result at the next moment. This represents the measured delay of the system.

Images