Human-computer interaction optimization system and method based on deep learning

Abstract

The invention discloses a human-computer interaction optimization system and method based on deep learning, and relates to the technical field of human-computer interaction, and the method comprises the following steps: firstly, a camera captures a hand image of a user according to a preset frame rate; the captured hand image frames are stored and organized into an analysis set, and key features reflecting rapid changes of gestures are extracted from the analysis set; and carrying out detailed analysis on the extracted key features in the detection window. Through a deep learning intelligent evaluation mechanism, the rapid change of the user gesture is accurately recognized. The system analyzes the skipping change and action alternating frequency of key points of the hand in real time, generates key factors and evaluates a gesture mode. According to the evaluation result, the frame rate of the camera is dynamically adjusted, more dynamic details are captured, and identification errors or delay caused by insufficient frame rate is avoided. According to the mechanism, the accuracy and interaction response of gesture recognition are improved, smoother and more accurate user experience is provided, and overall interaction perception is optimized.

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction technology, and in particular to a human-computer interaction optimization system and method based on deep learning. Background Art

[0002] Human-computer interaction optimization based on deep learning refers to the use of deep learning algorithms to improve the quality and efficiency of interaction between humans and computer systems. Traditional human-computer interaction mainly relies on rules and manually designed systems, while optimization methods based on deep learning automatically learn more intelligent interaction methods by analyzing and understanding input data such as user behavior, language, and actions. For example, through models such as deep neural networks (DNN) or convolutional neural networks (CNN), users' voices, gestures, expressions, etc. are recognized and predicted in real time, thereby achieving a more natural and personalized interactive experience. By continuously learning and adapting to user input, deep learning can optimize the interactive interface, improve the accuracy and speed of system response, and dynamically adjust the interaction mode according to user needs and preferences, achieving more efficient and intelligent human-computer interaction.

[0003] The prior art has the following deficiencies:

[0004] In the prior art, gesture change information is usually obtained by a camera at a fixed frame rate in user gesture recognition. The camera captures hand images at a certain frame rate and transmits the image data to a computer or processing system in real time. Each frame of the image records the state or action information of the hand, and the system analyzes the changes in gestures based on these continuous image frames. However, the fixed frame rate means that the camera can only capture a limited number of image frames per second. If the user's gesture changes rapidly in a short period of time, the low frame rate camera may not be able to capture enough details, resulting in serious consequences. When the gesture changes very quickly, the low frame rate camera may not be able to capture all the key details of the action. For example, when the user waves quickly or makes complex gestures, the long inter-frame time may cause some important dynamics to be missed. This will not only lead to incomplete gesture recognition, but also may make the system unable to accurately identify the changes in details in fast movements, especially at the start, end or turning point of the gesture. As a result, this inaccuracy in recognition may lead to misclassification, the system misunderstands the user's intention, and then produces incorrect interactive responses, affecting the user experience.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0006] The purpose of the present invention is to provide a human-computer interaction optimization system and method based on deep learning, which can realize accurate recognition of rapid changes in user gestures by introducing an intelligent evaluation mechanism based on deep learning. When gestures change rapidly, the system analyzes the jumping changes of key points of the hand and the frequency of alternating movements in real time, generates key factors and evaluates gesture patterns. Based on the evaluation results, the system dynamically adjusts the frame rate of the camera to capture more dynamic details and avoid recognition errors or delays caused by insufficient frame rate. This mechanism improves the accuracy of gesture recognition and the real-time nature of interactive response, provides users with a smoother and more accurate experience, and optimizes the overall user interaction perception to solve the problems in the above-mentioned background technology.

[0007] In order to achieve the above object, the present invention provides the following technical solution: a human-computer interaction optimization method based on deep learning, comprising the following steps:

[0008] First, the camera captures the user's hand image at a preset frame rate;

[0009] The captured hand image frames are stored and organized into an analysis set, from which key features reflecting the rapid changes of gestures are extracted;

[0010] The extracted key features are analyzed in detail within the detection window, and the analyzed key features are input into the pre-trained machine learning model, which is used to intelligently evaluate abnormal changes in user gestures.

[0011] Based on the evaluation results of the model, the user's gesture changes are divided into two categories: normal gesture changes and fast gesture changes;

[0012] For normal gesture changes, continue to capture hand images at a preset fixed frame rate and use gesture recognition algorithms to identify actions and intentions;

[0013] For rapid gesture changes, the camera frame rate is dynamically adjusted based on the evaluation results of the machine learning model to reduce the time interval between frames and capture more dynamic details.

[0014] Preferably, key features reflecting rapid changes in gestures are extracted from the analysis set, the extracted key features including the jumping changes in hand key points and the alternating frequency of actions in the gestures. Under the detection window, the extracted jumping changes in hand key points and the alternating frequency of actions in the gestures are analyzed to generate key point jumping factors and alternating frequency factors, respectively. The key point jumping factors quantify the drastic degree of position changes of hand key points between adjacent frames, reflecting the mutation and discontinuity of spatial positions during gesture movement; the alternating frequency factors quantify the frequent switching of actions in the gestures, reflecting the intensity of alternating changes in the dynamic pattern of the gestures.

[0015] Preferably, after obtaining the key point jump factor and alternation frequency factor generated after analyzing the extracted key features, the key point jump factor and alternation frequency factor are input into a pre-learned machine learning model, and a gesture change coefficient is generated by the machine learning model. The gesture change coefficient is used to perform an intelligent evaluation of abnormal changes in user gestures.

[0016] Preferably, the gesture change coefficient generated after analyzing the extracted key features is compared and analyzed with a preset gesture change coefficient reference threshold, and the user gesture changes are divided, and the division steps are as follows:

[0017] If the gesture change coefficient is greater than or equal to a preset gesture change coefficient reference threshold, the user gesture change is classified as a rapid gesture change;

[0018] If the gesture change coefficient is less than a preset gesture change coefficient reference threshold, the user gesture change is classified as a normal gesture change.

[0019] Preferably, for fast gesture changes, according to the evaluation results of the machine learning model, based on particle swarm optimization, the frame rate of the camera is dynamically adjusted to reduce the time interval between frames and capture more dynamic details. The specific steps are as follows:

[0020] When it is detected that the gesture is in a fast-changing state, the particle swarm optimization algorithm is started to initialize the particle frame rate increment. The initialization particle frame rate increment not only depends on the gesture change coefficient GVC, but also integrates the historical gesture change coefficient GVC value to make subsequent frame rate adjustments more accurately. The initialization frame rate increment formula is as follows:

[0021] ΔFPS b (t) = (GVC(t) - GVC ref )+θ·GVC(t-1)

[0022] , where ΔFPS b (t) is the frame rate increment of the bth particle at time point t, GVC(t) is the current gesture change coefficient, θ is the historical weight factor, which reflects the impact of historical changes, and GVC ref is the reference threshold of the gesture change coefficient, GVC(t-1) is the gesture change coefficient at the previous moment;

[0023] Particle swarm optimization algorithm based on frame rate increment ΔFPS b (t) changes, and optimizes the optimal frame rate increment for each particle. The calculation expression is as follows:

[0024] ΔFPS opt (t)=|GVC(t)-GVC ref |+τ·|ΔFPS b(t)-ΔFPS b (t-1)|

[0025] Where, ΔFPS opt (t) is the optimal frame rate increment, τ is the penalty coefficient, ΔFPS b (t-1) is the frame rate increment of the previous moment, that is, the frame rate increment of the bth particle at time point t-1;

[0026] Get the optimal frame rate increment ΔFPS opt (t), the optimal frame rate increment ΔFPS opt (t) is combined with the initial preset frame rate to dynamically adjust the frame rate of the camera. The optimized camera frame rate calculation expression is as follows:

[0027]

[0028] Where, FPS new (t) is the optimized camera frame rate, FPS0 is the preset camera frame rate, δ is the historical smoothing factor, which controls the influence of historical frame rate changes on the current frame rate adjustment, and B is the total number of particles.

[0029] Preferably, in the detection window, the specific steps of analyzing the jump changes of the key points of the hand and generating the key point jump factor are as follows:

[0030] In the detection window, for each frame, the spatial coordinate data of the key points are extracted using the hand tracking algorithm and marked as P i (t), represents the coordinate position of the i-th key point at time t, P i (t) = {x i (t), y i (t), z i (t)},x i (t), y i (t), z i (t) are the positions of the i-th key point on the x-axis, y-axis and z-axis at time point t;

[0031] Calculate the position change of the key points between each pair of consecutive frames, that is, the displacement of the i-th key point between two adjacent frames. The expression is as follows:

[0032]

[0033] Where Δt is the time interval, ΔP i (t) is the change in the position of the key point, that is, the spatial change of the i-th key point within the time interval Δt, x i (t+Δt) is the x-axis position of the ith key point at time t+Δt, yi (t+Δt) is the position of the ith key point on the y axis at time t+Δt, z i (t+Δt) is the z-axis position of the i-th key point at time t+Δt;

[0034] In order to quantify the degree of mutation of each hand key point, a nonlinear weighted method is introduced to define the local key point jump factor. The calculation expression is as follows:

[0035] J i (t)=(ΔP i (t)) α ·e -β·Δt

[0036] In the formula, J i (t) is the local key point jump factor, α is the nonlinear weighting index, e -β·Δt is the time decay factor, e is the natural base, and β is the time decay constant;

[0037] The jump factor J for all local key points i (t) is integrated to obtain the final key point jump factor, and the calculation expression is as follows:

[0038]

[0039] Where J(t) is the key point jump factor, w i is the weight of the i-th key point, and N is the total number of key points.

[0040] Preferably, in the detection window, the alternating frequency of the actions in the gesture is analyzed, and the specific steps of generating the alternating frequency factor are as follows:

[0041] Under the detection window, the captured continuous gesture image frames are divided into multiple small segments. Each segment represents the duration of a gesture action. Each segment of the action is divided according to the change in the position of the key points of the hand between consecutive frames. The expression is as follows:

[0042]

[0043] Where A(t) is the change in gesture action, which means the change in gesture action at time point t, and P i (t) is the coordinate position of the i-th key point at time t, P i (t-1) is the coordinate position of the i-th key point at time t-1, and N is the total number of key points;

[0044] When each segment of the action is divided, the alternation frequency between each action segment is calculated. The alternation frequency measures the rate of gesture switching in unit time and reflects the dynamic changes of the gesture. The calculation expression is as follows:

[0045]

[0046] Where ω(t) is the alternating frequency, action j (t) is the characteristic value of the jth action segment at time point t, action j (t-1) is the characteristic value of the jth action segment at time point t-1, and Δt is the time interval;

[0047] In order to consider the influence of different action segments, the alternation frequency needs to be weighted to adjust the intensity of different action segments. Based on the complexity and intensity of each action segment, the weight is used to adjust the calculation of the alternation frequency. The calculation expression of the alternation frequency after weighted adjustment is as follows:

[0048]

[0049] In the formula, is the weighted alternation frequency, w j is the complexity weight of the jth action segment, ω j (t) is the alternation frequency of the jth action segment;

[0050] Finally, all weighted alternating frequencies are further processed by a nonlinear adjustment function to generate the final alternating frequency factor, which is calculated as follows:

[0051]

[0052] Where γ(t) is the alternating frequency factor, is the weighted alternation frequency of the jth action segment, λ j is the weighting factor of the jth action segment.

[0053] A human-computer interaction optimization system based on deep learning, including a gesture image capture module, an image data storage and analysis set construction module, a feature extraction and analysis module, an intelligent evaluation and classification module, a normal gesture recognition module and a dynamic frame rate adjustment module:

[0054] In the gesture image capture module, first, the camera captures the user's hand image at a preset frame rate;

[0055] Image data storage and analysis set building module, the captured hand image frames are stored and organized into an analysis set, from which key features reflecting the rapid changes of gestures are extracted;

[0056] The feature extraction and analysis module performs detailed analysis on the extracted key features within the detection window and inputs the analyzed key features into a pre-trained machine learning model, which then performs intelligent evaluation on abnormal changes in user gestures.

[0057] Intelligent evaluation and classification module, based on the evaluation results of the model, divides the user's gesture changes into two categories: normal gesture changes and fast gesture changes;

[0058] The normal gesture recognition module continues to capture hand images at a preset fixed frame rate for normal gesture changes, and uses gesture recognition algorithms to identify actions and intentions;

[0059] The dynamic frame rate adjustment module dynamically adjusts the frame rate of the camera for rapid gesture changes based on the evaluation results of the machine learning model, reduces the time interval between frames, and captures more dynamic details.

[0060] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0061] The present invention can accurately identify the rapid changes of user gestures by introducing an intelligent evaluation mechanism based on deep learning. When the user's gesture changes rapidly in a short period of time, the traditional fixed frame rate camera may miss key dynamic details, resulting in incomplete or incorrect gesture recognition. By analyzing the jumping changes of the key points of the hand and the alternating frequency of the actions in the gesture in real time, and generating the key point jumping factor and the alternating frequency factor, the machine learning model can intelligently evaluate the changing pattern of the gesture. When the user's gesture is detected to change rapidly, the system can dynamically adjust the frame rate of the camera to reduce the time interval between frames, thereby capturing more dynamic details. This real-time and sensitive adjustment ensures that the system can more accurately identify rapidly changing gestures, avoid missing or misclassifying gestures, and thus improve the accuracy of gesture recognition. Users can get a more accurate and smooth interactive experience, reducing misoperations caused by inaccurate recognition.

[0062] The dynamic frame rate adjustment mechanism of the present invention is combined with the real-time analysis of gesture changes, and can flexibly adjust the working mode of the camera according to the changes in gestures. When the user performs a quick gesture, the system can adjust the frame rate in time to capture the key details of the gesture, avoiding interaction delays or missed information due to insufficient frame rate. This mechanism ensures the real-time response of the interactive system to the user's intentions and avoids the system misunderstanding the user's behavior due to recognition errors. Furthermore, the combination of intelligent evaluation and dynamic frame rate adjustment not only improves the system's sensitivity to quick gestures, but also makes the system's recognition of different types of gestures more natural and smooth. During use, users can obtain a smoother, delay-free interactive experience. This precise response and high sensitivity will greatly improve user satisfaction and interactive perception, and optimize the overall user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0064] Figure 1 This is a method flow chart of a human-computer interaction optimization method based on deep learning of the present invention.

[0065] Figure 2 This is a module schematic diagram of a human-computer interaction optimization system based on deep learning in the present invention. DETAILED DESCRIPTION

[0066] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of the present disclosure will be more comprehensive and complete, and the concept of the example embodiments will be fully conveyed to those skilled in the art.

[0067] The present invention provides Figure 1 A human-computer interaction optimization method based on deep learning is shown, comprising the following steps:

[0068] First, the camera captures the user's hand image at a preset frame rate;

[0069] The preset frame rate refers to the number of image frames captured by the camera within a certain period of time (usually per second) in the gesture recognition system. The setting of this frame rate is usually determined based on historical data or empirical data. For example, in past gesture recognition applications, through dynamic analysis of user gesture changes, researchers or engineers may find that the recognition accuracy at a specific frame rate is higher and can capture most of the key gestures. Therefore, based on these historical data, the frame rate is set to a fixed value in advance to ensure efficient recognition of gestures during the system's operation.

[0070] When setting the frame rate, you need to ensure that the value can take into account the consumption of computing resources and the detail requirements of image capture. Generally, the set frame rate should be sufficient to capture the key details of the user's gesture movements, while avoiding excessive frame rates that lead to excessive data volume, which in turn affects the processing speed and the real-time responsiveness of the system. Therefore, the frame rate setting must balance the camera's resolution, computing power, and the speed of gesture changes to ensure that images can be captured efficiently without wasting resources during common gesture executions.

[0071] The captured hand image frames are stored and organized into an analysis set, from which key features reflecting the rapid changes of gestures are extracted;

[0072] First, the camera continuously captures image frames of the user's hand at a set frame rate, and each frame contains information about the state and movement of the hand. These image frames are then transmitted to a computer or processing system, which stores and organizes these images in chronological order into an analysis set. The analysis set is a data container that contains all hand image frames captured within a certain time window and is arranged according to the chronological order of gesture movements. This set provides basic data for subsequent gesture analysis.

[0073] The role of the analysis set is to provide a complete and systematic data foundation for subsequent feature extraction, processing and gesture recognition. By organizing the image frames in chronological order, the analysis set can provide the system with continuous gesture change information, allowing the algorithm to identify dynamic changes in the image sequence, especially the turning points, rapid changes and key actions of the gesture. At the same time, it also facilitates the subsequent extraction of features that reflect the rapid changes in gestures, helping the system to more accurately understand the intention and details of the gesture, and providing high-quality input data for the machine learning model, thereby improving the accuracy and real-time performance of recognition.

[0074] The extracted key features are analyzed in detail within the detection window, and the analyzed key features are input into the pre-trained machine learning model, which is used to intelligently evaluate abnormal changes in user gestures.

[0075] Key features reflecting the rapid changes of gestures are extracted from the analysis set. The extracted key features include the jumping changes of hand key points and the alternating frequency of actions in gestures. Under the detection window, the extracted jumping changes of hand key points and the alternating frequency of actions in gestures are analyzed to generate key point jumping factors and alternating frequency factors respectively. The key point jumping factors quantify the drastic degree of position changes of hand key points between adjacent frames, reflecting the mutation and discontinuity of spatial position during gesture movement; the alternating frequency factor quantifies the frequent switching of actions in gestures, reflecting the intensity of alternating changes in the dynamic pattern of gestures.

[0076] After obtaining the key point jump factor and alternation frequency factor generated by analyzing the extracted key features, the key point jump factor and alternation frequency factor are input into the pre-learned machine learning model, and the gesture change coefficient is generated by the machine learning model. The abnormal changes of the user's gestures are intelligently evaluated through the gesture change coefficient.

[0077] A pre-learned machine learning model refers to a model that is trained using a large amount of historical data and label information of gesture changes before the system is deployed. This model has learned different types of gesture data through certain algorithms (such as deep learning, support vector machines, random forests, etc.), so that it can recognize and predict different gesture patterns and their changing patterns. Through training, the model can grasp the spatiotemporal characteristics of gestures, understand which gesture changes are normal and which are abnormal, and even learn some subtle change patterns. During the training process, the dynamic features of gestures (such as key point jump factors and alternation frequency factors) will be associated with corresponding labels (such as whether the gesture is abnormal, gesture type, etc.) so that the model can predict the results of unknown inputs through the patterns in the data.

[0078] This pre-learned machine learning model enables intelligent evaluation in real-time applications. Specifically, once the key point jump factor and alternation frequency factor are extracted from the analysis set and input into the model, the model will evaluate the input data according to the previously learned pattern and output the gesture change coefficient. The gesture change coefficient reflects the degree of abnormality of the user's current gesture. If the coefficient exceeds a certain threshold, the model can determine that the gesture may be in a rapidly changing or abnormal state, and thus issue corresponding warnings or adjust the recognition strategy. Through continuous learning and adjustment, the pre-trained model can continuously optimize its judgment criteria to ensure that accurate recognition and evaluation can still be made when faced with different scenarios and new gesture changes.

[0079] The sudden change of the hand key points indicates that the current user gesture is in a state of rapid change. When the hand key points undergo drastic jump changes between adjacent frames, it usually means that the spatial position of the gesture has changed significantly, and it may be a rapid action transition or movement completed in a short period of time. For example, when the user waves quickly or makes complex gestures, the position of the hand key points may change greatly between each frame. This mutation reflects the rapid change of gestures. This mutation is usually related to a sharp change in user intention, such as suddenly starting or ending an action, or quickly switching gesture types. Compared with smooth movements, jump changes usually occur during dynamic acceleration or high-intensity changes. Therefore, when the hand key points have obvious jump changes, the system can infer that the user's gesture is undergoing a rapid and drastic change state.

[0080] Under the detection window, the jump changes of the key points of the hand are analyzed, and the specific steps for generating the key point jump factor are as follows:

[0081] In the detection window, for each frame, the spatial coordinate data of the key points are extracted using the hand tracking algorithm and marked as P i (t), represents the coordinate position of the i-th key point at time t, P i (t) = {x i (t), y i (t), z i (t)},x i (t), y i (t), z i (t) are the positions of the i-th key point on the x-axis, y-axis and z-axis at time point t;

[0082] Hand tracking algorithm is a computer vision technology used to track and identify the position, posture and movement of hands in images or videos in real time. It determines the position and movement trajectory of hands in three-dimensional space by analyzing feature points, contours or depth information in images or video sequences. Hand tracking algorithms usually combine machine learning, deep learning or traditional computer vision methods, using cameras to capture continuous image frames of hands and infer the position and movement of hands based on the dynamic changes between these frames.

[0083] The reason for using hand tracking algorithms is that they can accurately capture the subtle movements and changes of the hands, thus providing a basis for gesture recognition and interaction. Through hand tracking, the user's intention, posture, and movement amplitude can be perceived in real time when performing various actions, thereby providing the system with a more efficient and intuitive interaction method. For example, in applications such as virtual reality, augmented reality, and gesture control, hand tracking algorithms can achieve a smooth and accurate user interaction experience, avoiding the limitations of traditional input devices, and improving the user's immersion and convenience of operation.

[0084] Calculate the position change of the key points between each pair of consecutive frames, that is, the displacement of the i-th key point between two adjacent frames. The expression is as follows:

[0085]

[0086] Where Δt is the time interval, ΔP i (t) is the change in the position of the key point, that is, the spatial change of the i-th key point within the time interval Δt, x i (t+Δt) is the x-axis position of the ith key point at time t+Δt, y i (t+Δt) is the position of the ith key point on the y axis at time t+Δt, z i (t+Δt) is the z-axis position of the i-th key point at time t+Δt;

[0087] The role of the change in the position of key points is to quantify the dynamics of hand movements, especially to evaluate the rate of change and suddenness of gestures. By analyzing the change in the position of key points between adjacent frames, we can determine whether the hand movement is smooth or has undergone drastic fluctuations. A large change usually means that the hand has experienced a large displacement in a short period of time, which may indicate that the gesture is changing rapidly or a sudden change has occurred. This information is crucial for identifying fast gestures or abnormal changes, and can help the system accurately capture key details in fast movements and avoid missing important movement information due to frame rate limitations. Therefore, calculating the change in the position of key points provides basic data support for further gesture classification, anomaly detection, and intelligent recognition, and improves the responsiveness of gesture recognition systems in dynamic environments.

[0088] In order to quantify the degree of mutation of each hand key point, a nonlinear weighted method is introduced to define the local key point jump factor. The calculation expression is as follows:

[0089] J i (t)=(ΔP i (t)) α ·e -β·Δt

[0090] In the formula, J i(t) is the local key point jump factor, α is the nonlinear weighting index, e -β·Δt is the time decay factor, e is the natural base, and β is the time decay constant;

[0091] The time decay factor is a mathematical term introduced when calculating dynamic changes. Its main function is to adjust the impact of a certain change on the final result according to the length of the time interval. Specifically, the time decay factor is usually expressed by an exponential decay function (such as e -β·Δt ), where β is the decay constant and Δt is the time interval. Its function is that when the time interval is long, the decay factor will reduce the impact of the change, and conversely, when the time interval is short, the change has a greater impact on the result. In this way, the time decay factor can effectively handle dynamic changes with long time intervals, so that the system pays more attention to rapid changes that occur in a short time span, while ignoring slow changes with a long time span, thereby more accurately reflecting the rapid changes in gestures or actions, and avoiding the unnecessary impact of slow changes caused by long time intervals on the results.

[0092] The jump factor J for all local key points i (t) is integrated to obtain the final key point jump factor, and the calculation expression is as follows:

[0093]

[0094] Where J(t) is the key point jump factor, w i is the weight of the i-th key point, N is the total number of key points;

[0095] Under the detection window, the larger the key point jump factor expression value generated after analyzing the jump changes of the hand key points, the more drastic the spatial position change of the hand key points between adjacent frames is, which reflects that the user's gesture is undergoing rapid and sudden changes. Therefore, a larger key point jump factor expression value indicates that the user's gesture is in a state of rapid change. On the contrary, if the jump factor expression value is small, it means that the change of the hand key points is relatively stable or continuous, indicating that the user's gesture changes are relatively slow or stable, which is a normal gesture change state.

[0096] A rapid increase in the frequency of alternating actions in a gesture usually indicates that the current user's gesture is in a rapidly changing state, because the alternating frequency reflects the rate of switching or repetition between gesture actions. When a user makes rapid gesture changes, the transitions between each action become more frequent and rapid, resulting in a significant increase in the alternating frequency. This frequent switching or repeated action is usually a sign of a drastic change in the gesture state, which may be caused by actions such as rapid waving, shaking, and rapid clicking. Compared with smooth or slowly changing gestures, rapidly changing gestures have higher dynamic characteristics, and the transition of actions is faster and more obvious. Therefore, the rapid increase in the alternating frequency directly indicates that the gesture has undergone more action switching or changes in a shorter period of time, reflecting the increased intensity of the gesture change. At this time, the gesture pattern becomes more complex and dynamic, requiring the system to capture more details to ensure the accuracy of recognition.

[0097] Under the detection window, the alternation frequency of the actions in the gesture is analyzed, and the specific steps to generate the alternation frequency factor are as follows:

[0098] Under the detection window, the captured continuous gesture image frames are divided into multiple small segments. Each segment represents the duration of a gesture action. Each segment of the action is divided according to the change in the position of the key points of the hand between consecutive frames. The expression is as follows:

[0099]

[0100] Where A(t) is the change in gesture action, which means the change in gesture action at time point t, and P i (t) is the coordinate position of the i-th key point at time t, P i (t-1) is the coordinate position of the i-th key point at time t-1, that is, the coordinate of the key point at the previous moment, and N is the total number of key points;

[0101] The captured continuous gesture image frames are divided into multiple small segments, each of which represents the duration of a gesture action, in order to accurately identify and analyze the time span and motion characteristics of each action. This division method helps the system understand the start and end time of each action and clarify the various dynamic stages of the gesture, thereby providing a more detailed reference for subsequent action analysis. For example, when the key points of the hand change greatly between consecutive frames, the system can identify the change of an action and start it as a new action segment; when the change is small or remains stable, the system can continue to analyze in the current segment. This division can help accurately capture the details of mutations, pauses, and smoothness in gesture actions, providing more accurate input for subsequent action recognition, pattern analysis, and anomaly detection, thereby improving the accuracy and real-time response capabilities of the overall recognition system.

[0102] When each segment of the action is divided, the alternation frequency between each action segment is calculated. The alternation frequency measures the rate of gesture switching in unit time and reflects the dynamic changes of the gesture. The calculation expression is as follows:

[0103]

[0104] Where ω(t) is the alternating frequency, action j (t) is the characteristic value of the jth action segment at time point t, action j (t-1) is the characteristic value of the jth action segment at time point t-1, that is, the characteristic value of the action segment at the previous moment, and Δt is the time interval;

[0105] In order to consider the influence of different action segments, the alternation frequency needs to be weighted to adjust the intensity of different action segments. Based on the complexity and intensity of each action segment, the weight is used to adjust the calculation of the alternation frequency. The calculation expression of the alternation frequency after weighted adjustment is as follows:

[0106]

[0107] In the formula, is the weighted alternation frequency, w j is the complexity weight of the jth action segment, ω j (t) is the alternation frequency of the jth action segment;

[0108] The significance of weighting the alternation frequency is that it can more accurately reflect the relative importance and change intensity of each action segment in the gesture. Different action segments may have different time lengths, dynamic change characteristics and frequencies during the gesture process, so their contributions to the overall gesture change are uneven. Through weighting, higher weights can be given to key action segments that play a leading role in the rapid change of gestures, so that the final calculated alternation frequency is more sensitive and representative, and can accurately capture the important moments of change in the gesture. Weighted processing helps to reduce the interference of unimportant or short-term changes, ensuring that the system can give priority to those action segments that have a greater impact on gesture recognition and classification, thereby improving the accuracy of gesture recognition and real-time responsiveness.

[0109] Finally, all weighted alternating frequencies are further processed by a nonlinear adjustment function to generate the final alternating frequency factor, which is calculated as follows:

[0110]

[0111] Where γ(t) is the alternating frequency factor, is the weighted alternation frequency of the jth action segment, λ jis the weighting factor of the jth action segment;

[0112] The weighted alternation frequency is further processed by a nonlinear adjustment function in order to enhance the system's sensitivity to the dynamic changes of gestures and to capture complex change patterns more accurately. The alternation frequency of gestures usually exhibits nonlinear characteristics when changing rapidly, and simple linear weighting may not be able to fully reflect these complex changes. The nonlinear adjustment function can amplify the rapidly changing parts and reduce the impact of relatively stable or repetitive movements, thereby more accurately reflecting the dynamic characteristics of gestures. In this way, gestures in rapidly changing and stable states can be better identified and distinguished, improving the system's responsiveness to rapidly changing gestures and avoiding misjudgments or omissions caused by insufficient linear processing. In addition, the nonlinear adjustment function can also enhance the robustness of the model, adapt it to different gesture change scenarios, and improve the system's performance in diverse and dynamic environments.

[0113] Under the detection window, the larger the value of the alternation frequency factor generated after analyzing the alternation frequency of the actions in the gesture, the faster the user's gesture is changing. The alternation frequency factor quantifies the rate at which the actions in the gesture are frequently switched or repeated. When the user performs fast gestures, the transitions between actions become more frequent, resulting in an increase in the alternation frequency, which is manifested as a higher alternation frequency factor value. In this case, a larger value of the alternation frequency factor reflects a dramatic fluctuation in the gesture state, meaning that the gesture changes rapidly and is unstable, usually a manifestation of rapid changes. On the contrary, a smaller alternation frequency factor indicates that the gesture is relatively stable, the switching is not frequent, and the gesture is in a normal or slowly changing state, indicating that there is no significant acceleration or fluctuation in the current gesture change.

[0114] The machine learning model is not limited here, and any machine learning model that can perform a comprehensive analysis of the key point jump factor J(t) and the alternation frequency factor γ(t) to generate the gesture change coefficient GVC is acceptable. To implement the technical solution of the present invention, the present invention provides a specific implementation method:

[0115] The gesture change coefficient GVC generation formula is as follows:

[0116]

[0117] Wherein, k1 and k2 are preset proportional coefficients of the key point jump factor J(t) and the alternation frequency factor γ(t), respectively, and both k1 and k2 are greater than 0.

[0118] It can be seen from the calculation expression of the gesture change coefficient that, under the detection window, the larger the key point jump factor expression value generated after analyzing the jump changes of the hand key points, and the larger the alternation frequency factor expression value generated after analyzing the alternation frequency of the actions in the gesture, the larger the gesture change coefficient expression value generated after analyzing the extracted key features under the detection window, indicating that the current user gesture is in a rapidly changing state. Otherwise, it indicates that the current user gesture is changing rapidly and is in a normal change.

[0119] The preset proportionality coefficients (k1 and k2) are important weight coefficients used to adjust the key point jump factor J(t) and the alternation frequency factor γ(t) when calculating the gesture change coefficient GVC. Their role is to reasonably adjust and optimize the contribution of the two factors according to the actual application requirements. Specifically, k1 and k2 are pre-set values ​​used to indicate the relative importance or influence of the two features (jump change and alternation frequency). By adjusting these two coefficients, the influence of the two features on the final gesture change coefficient GVC can be flexibly controlled, so as to better adapt to different gesture recognition scenarios or specific task requirements. The preset proportionality coefficients are usually determined based on previous experimental data or domain knowledge, with the aim of achieving optimal recognition accuracy and stability in comprehensive analysis.

[0120] Based on the evaluation results of the model, the user's gesture changes are divided into two categories: normal gesture changes and fast gesture changes;

[0121] The gesture change coefficient generated after analyzing the extracted key features is compared with the preset gesture change coefficient reference threshold to divide the user gesture changes. The division steps are as follows:

[0122] If the gesture change coefficient is greater than or equal to a preset gesture change coefficient reference threshold, the user gesture change is classified as a rapid gesture change;

[0123] If the gesture change coefficient is less than a preset gesture change coefficient reference threshold, the user gesture change is classified as a normal gesture change;

[0124] Normal gesture changes refer to the smooth and gradual changes in the user's gestures over a certain period of time. The speed and amplitude are usually relatively uniform and stable. In this case, the gesture movement transitions are relatively slow, and there will be no drastic jumps or sudden changes. Rapid gesture changes refer to the rapid and drastic changes in the user's gestures over a short period of time, usually accompanied by a large amplitude or frequent dynamic switching. Such changes are manifested as rapid changes in hand movements, such as rapid waving, sudden swings, or rapid gesture alternations.

[0125] For normal gesture changes, continue to capture hand images at a preset fixed frame rate and use gesture recognition algorithms to identify actions and intentions;

[0126] For situations classified as normal gesture changes, the system continues to capture hand images at a preset fixed frame rate and uses existing gesture recognition algorithms to identify actions and intentions. The purpose is to ensure the efficiency and stability of the system in regular gesture operations. Normal gesture changes usually refer to relatively smooth changes in user movements without drastic changes or irregular fluctuations. In this case, maintaining a fixed frame rate to capture images can fully meet the requirements of gesture recognition while avoiding excessive consumption of computing resources. By using existing gesture recognition algorithms, the system can efficiently classify and parse common gestures without additional computing burden or frequent adjustments. This not only ensures real-time and smoothness, but also ensures the reliability and user experience of the system in daily operations, and is suitable for most regular gesture input scenarios.

[0127] For fast gesture changes, the frame rate of the camera is dynamically adjusted based on the evaluation results of the machine learning model and particle swarm optimization to reduce the time interval between frames and capture more dynamic details.

[0128] For fast gesture changes, according to the evaluation results of the machine learning model, based on particle swarm optimization, the frame rate of the camera is dynamically adjusted to reduce the time interval between frames and capture more dynamic details. The specific steps are as follows:

[0129] When the gesture is detected to be in a fast-changing state, the particle swarm optimization (PSO) algorithm is started to initialize the particle frame rate increment. The initialization particle frame rate increment not only depends on the gesture change coefficient GVC, but also integrates the historical gesture change coefficient GVC value to make subsequent frame rate adjustments more accurately. The initialization frame rate increment formula is as follows:

[0130] ΔFPS b (t) = (GVC(t) - GVC ref )+θ·GVC(t-1)

[0131] Where, ΔFPS b (t) is the frame rate increment of the bth particle at time point t, GVC(t) is the current gesture change coefficient, θ is the historical weight factor, which reflects the impact of historical changes, and GVC ref is the reference threshold of the gesture change coefficient, GVC(t-1) is the gesture change coefficient at the previous moment;

[0132] Particle swarm optimization algorithm based on frame rate increment ΔFPS b (t) changes, and optimizes the optimal frame rate increment for each particle. The calculation expression is as follows:

[0133] ΔFPS opt (t)=|GVC(t)-GVC ref |+τ·|ΔFPS b (t)-ΔFPS b (t-1)|

[0134] Where, ΔFPS opt (t) is the optimal frame rate increment, τ is the penalty coefficient, ΔFPS b (t-1) is the frame rate increment of the previous moment, that is, the frame rate increment of the bth particle at time point t-1;

[0135] The penalty coefficient τ is a parameter used in the particle swarm optimization (PSO) algorithm to adjust and control the influence of past frame rate increment changes on the particle optimization process. Its main function is to prevent particles from over-adjusting the frame rate and avoid unnecessary fluctuations in the system. Specifically, the penalty coefficient imposes a certain penalty on the changes in the historical frame rate adjustment of each particle in the particle swarm, so that particles pay more attention to smooth and gradual frame rate adjustments rather than drastic changes during the optimization process. This helps to balance the dynamic adjustment of the frame rate with the stability of the system when the gestures change rapidly, ensuring that the system will not be unstable or waste resources due to rapidly changing gestures. By adjusting the penalty coefficient, the system can maintain sufficient smoothness when responding to the needs of rapid gesture changes and avoid the side effects of over-adjustment.

[0136] Get the optimal frame rate increment ΔFPS opt (t), the optimal frame rate increment ΔFPS opt (t) is combined with the initial preset frame rate to dynamically adjust the frame rate of the camera. The optimized camera frame rate calculation expression is as follows:

[0137]

[0138] Where, FPS new (t) is the optimized camera frame rate, FPS0 is the preset camera frame rate, δ is the historical smoothing factor, which controls the influence of historical frame rate changes on the current frame rate adjustment, and B is the total number of particles.

[0139] In the case of rapid gesture changes, the system needs to be able to respond and capture gesture details in time to ensure accurate recognition of the user's actions and intentions. The role of this step is to dynamically adjust the camera frame rate based on the evaluation results of the machine learning model using the particle swarm optimization algorithm (PSO), thereby reducing the time interval between frames and increasing the system's capture frequency and accuracy of gesture changes. Specifically, when the gesture change coefficient GVC is higher than the set reference threshold, it indicates that the gesture is undergoing rapid changes, and a higher frame rate is required to capture gesture details in detail. The particle swarm optimization algorithm can efficiently explore and optimize the frame rate adjustment strategy by simulating the search and aggregation process of particles in nature. Through particle swarm optimization, the system can find the optimal solution among multiple possible frame rate increments and dynamically adjust the camera frame rate to adapt to the rapid changes in gestures. Reducing the time interval between frames can help the system obtain more dynamic images in a shorter time, thereby improving the ability to capture rapid gesture changes, enabling the system to achieve higher recognition accuracy in a highly dynamic gesture environment. This adjustment strategy not only ensures that the system can cope with changes in different gestures, but also avoids the omission or blurring of actions caused by too low a frame rate, thereby enhancing the system's ability to respond quickly and accurately to user intentions.

[0140] The present invention can accurately identify the rapid changes of user gestures by introducing an intelligent evaluation mechanism based on deep learning. When the user's gesture changes rapidly in a short period of time, the traditional fixed frame rate camera may miss key dynamic details, resulting in incomplete or incorrect gesture recognition. By analyzing the jumping changes of the key points of the hand and the alternating frequency of the actions in the gesture in real time, and generating the key point jumping factor and the alternating frequency factor, the machine learning model can intelligently evaluate the changing pattern of the gesture. When the user's gesture is detected to change rapidly, the system can dynamically adjust the frame rate of the camera to reduce the time interval between frames, thereby capturing more dynamic details. This real-time and sensitive adjustment ensures that the system can more accurately identify rapidly changing gestures, avoid missing or misclassifying gestures, and thus improve the accuracy of gesture recognition. Users can get a more accurate and smooth interactive experience, reducing misoperations caused by inaccurate recognition.

[0141] The dynamic frame rate adjustment mechanism of the present invention is combined with the real-time analysis of gesture changes, and can flexibly adjust the working mode of the camera according to the changes in gestures. When the user performs a quick gesture, the system can adjust the frame rate in time to capture the key details of the gesture, avoiding interaction delays or missed information due to insufficient frame rate. This mechanism ensures the real-time response of the interactive system to the user's intentions and avoids the system misunderstanding the user's behavior due to recognition errors. Furthermore, the combination of intelligent evaluation and dynamic frame rate adjustment not only improves the system's sensitivity to quick gestures, but also makes the system's recognition of different types of gestures more natural and smooth. During use, users can obtain a smoother, delay-free interactive experience. This precise response and high sensitivity will greatly improve user satisfaction and interactive perception, and optimize the overall user experience.

[0142] The present invention provides Figure 2 A human-computer interaction optimization system based on deep learning is shown, including a gesture image capture module, an image data storage and analysis set construction module, a feature extraction and analysis module, an intelligent evaluation and classification module, a normal gesture recognition module and a dynamic frame rate adjustment module:

[0143] In the gesture image capture module, first, the camera captures the user's hand image at a preset frame rate;

[0144] Image data storage and analysis set building module, the captured hand image frames are stored and organized into an analysis set, from which key features reflecting the rapid changes of gestures are extracted;

[0145] The feature extraction and analysis module performs detailed analysis on the extracted key features within the detection window and inputs the analyzed key features into a pre-trained machine learning model, which then performs intelligent evaluation on abnormal changes in user gestures.

[0146] Intelligent evaluation and classification module, based on the evaluation results of the model, divides the user's gesture changes into two categories: normal gesture changes and fast gesture changes;

[0147] The normal gesture recognition module continues to capture hand images at a preset fixed frame rate for normal gesture changes, and uses gesture recognition algorithms to identify actions and intentions;

[0148] The dynamic frame rate adjustment module dynamically adjusts the frame rate of the camera for rapid gesture changes based on the evaluation results of the machine learning model, reduces the time interval between frames, and captures more dynamic details.

[0149] A human-computer interaction optimization method based on deep learning provided in an embodiment of the present invention is implemented by the above-mentioned human-computer interaction optimization system based on deep learning. The specific method and process of a human-computer interaction optimization system based on deep learning are detailed in the embodiment of the above-mentioned human-computer interaction optimization method based on deep learning, which will not be repeated here.

[0150] The above description is only by way of illustration of certain exemplary embodiments of the present invention. It is undoubted that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A human-computer interaction optimization method based on deep learning, characterized in that: The following steps are involved: First, the camera captures the user's hand image at a preset frame rate; The captured hand image frames are stored and organized into an analysis set, from which key features reflecting the rapid changes of gestures are extracted; The extracted key features are analyzed in detail within the detection window, and the analyzed key features are input into the pre-trained machine learning model, which is used to intelligently evaluate abnormal changes in user gestures. Based on the evaluation results of the model, the user's gesture changes are divided into two categories: normal gesture changes and fast gesture changes; For normal gesture changes, continue to capture hand images at a preset fixed frame rate and use gesture recognition algorithms to identify actions and intentions; For rapid gesture changes, the camera frame rate is dynamically adjusted based on the evaluation results of the machine learning model to reduce the time interval between frames and capture more dynamic details.

2. The human-computer interaction optimization method based on deep learning according to claim 1, characterized in that: Key features reflecting the rapid changes of gestures are extracted from the analysis set. The extracted key features include the jumping changes of hand key points and the alternating frequency of actions in gestures. Under the detection window, the extracted jumping changes of hand key points and the alternating frequency of actions in gestures are analyzed to generate key point jumping factors and alternating frequency factors respectively. The key point jumping factors quantify the drastic degree of position changes of hand key points between adjacent frames, reflecting the mutation and discontinuity of spatial position during gesture movement; the alternating frequency factor quantifies the frequent switching of actions in gestures, reflecting the intensity of alternating changes in the dynamic pattern of gestures.

3. The human-computer interaction optimization method based on deep learning according to claim 2, characterized in that: After obtaining the key point jump factor and alternation frequency factor generated by analyzing the extracted key features, the key point jump factor and alternation frequency factor are input into the pre-learned machine learning model, and the gesture change coefficient is generated by the machine learning model. The abnormal changes of the user's gestures are intelligently evaluated through the gesture change coefficient.

4. The human-computer interaction optimization method based on deep learning according to claim 3 is characterized in that: The gesture change coefficient generated after analyzing the extracted key features is compared with the preset gesture change coefficient reference threshold to divide the user gesture changes. The division steps are as follows: If the gesture change coefficient is greater than or equal to a preset gesture change coefficient reference threshold, the user gesture change is classified as a rapid gesture change; If the gesture change coefficient is less than a preset gesture change coefficient reference threshold, the user gesture change is classified as a normal gesture change.

5. The method for human-computer interaction optimization based on deep learning according to claim 4, characterized in that: For fast gesture changes, according to the evaluation results of the machine learning model, based on particle swarm optimization, the frame rate of the camera is dynamically adjusted to reduce the time interval between frames and capture more dynamic details. The specific steps are as follows: When it is detected that the gesture is in a fast-changing state, the particle swarm optimization algorithm is started to initialize the particle frame rate increment. The initialization particle frame rate increment not only depends on the gesture change coefficient GVC, but also integrates the historical gesture change coefficient GVC value to make subsequent frame rate adjustments more accurately. The initialization frame rate increment formula is as follows: ΔFPS b (t)=(GVC(t)-GVC ref )+θ·GVC(t-1) Where, ΔFPS b (t) is the frame rate increment of the bth particle at time point t, GVC(t) is the current gesture change coefficient, θ is the historical weight factor, which reflects the impact of historical changes, and GVC ref is the reference threshold of the gesture change coefficient, GVC(t-1) is the gesture change coefficient at the previous moment; Particle swarm optimization algorithm based on frame rate increment ΔFPS b (t) changes, and optimizes the optimal frame rate increment for each particle. The calculation expression is as follows: ΔFPS opt (t)=|GVC(t)-GVC ref |+τ·|ΔFPS b (t)-ΔFPS b (t-1)| Where, ΔFPS opt (t) is the optimal frame rate increment, τ is the penalty coefficient, ΔFPS b (t-1) is the frame rate increment of the previous moment, that is, the frame rate increment of the bth particle at time point t-1; Get the optimal frame rate increment ΔFPS opt (t), the optimal frame rate increment ΔFPS opt (t) is combined with the initial preset frame rate to dynamically adjust the frame rate of the camera. The optimized camera frame rate calculation expression is as follows: Where, FPS new (t) is the optimized camera frame rate, FPS0 is the preset camera frame rate, δ is the historical smoothing factor, which controls the influence of historical frame rate changes on the current frame rate adjustment, and B is the total number of particles.

6. The method for optimizing human-computer interaction based on deep learning according to claim 2, characterized in that: Under the detection window, the jump changes of the key points of the hand are analyzed, and the specific steps for generating the key point jump factor are as follows: In the detection window, for each frame, the spatial coordinate data of the key points are extracted using the hand tracking algorithm and marked as P i (t), represents the coordinate position of the i-th key point at time t, P i (t) = {x i (t), y i (t), z i (t)},x i (t), y i (t), z i (t) are the positions of the i-th key point on the x-axis, y-axis and z-axis at time point t; Calculate the position change of the key points between each pair of consecutive frames, that is, the displacement of the i-th key point between two adjacent frames. The expression is as follows: Where Δt is the time interval, ΔP i (t) is the change in the position of the key point, that is, the spatial change of the i-th key point within the time interval Δt, x i (t+Δt) is the x-axis position of the ith key point at time t+Δt, y i (t+Δt) is the position of the ith key point on the y axis at time t+Δt, z i (t+Δt) is the z-axis position of the i-th key point at time t+Δt; In order to quantify the degree of mutation of each hand key point, a nonlinear weighted method is introduced to define the local key point jump factor. The calculation expression is as follows: J i (t)=(ΔP i (t)) α ·e -β·Δt In the formula, J i (t) is the local key point jump factor, α is the nonlinear weighting index, e -β·Δt is the time decay factor, e is the natural base, and β is the time decay constant; The jump factor J for all local key points i (t) is integrated to obtain the final key point jump factor, and the calculation expression is as follows: Where J(t) is the key point jump factor, w i is the weight of the i-th key point, and N is the total number of key points.

7. The human-computer interaction optimization method based on deep learning according to claim 2, characterized in that: Under the detection window, the alternation frequency of the actions in the gesture is analyzed, and the specific steps to generate the alternation frequency factor are as follows: Under the detection window, the captured continuous gesture image frames are divided into multiple small segments. Each segment represents the duration of a gesture action. Each segment of the action is divided according to the change in the position of the key points of the hand between consecutive frames. The expression is as follows: Where A(t) is the change in gesture action, which means the change in gesture action at time point t, and P i (t) is the coordinate position of the i-th key point at time t, P i (t-1) is the coordinate position of the i-th key point at time t-1, and N is the total number of key points; After each segment of the action is divided, the alternation frequency between each action segment is calculated. The alternation frequency measures the rate of gesture switching in unit time and reflects the dynamic changes of the gesture. The calculation expression is as follows: Where ω(t) is the alternating frequency, action j (t) is the characteristic value of the jth action segment at time point t, action j (t-1) is the characteristic value of the jth action segment at time point t-1, and Δt is the time interval; In order to consider the influence of different action segments, the alternation frequency needs to be weighted to adjust the intensity of different action segments. Based on the complexity and intensity of each action segment, the weight is used to adjust the calculation of the alternation frequency. The calculation expression of the alternation frequency after weighted adjustment is as follows: In the formula, is the weighted alternation frequency, w j is the complexity weight of the jth action segment, ω j (t) is the alternation frequency of the jth action segment; Finally, all weighted alternating frequencies are further processed by a nonlinear adjustment function to generate the final alternating frequency factor, which is calculated as follows: Where γ(t) is the alternating frequency factor, is the weighted alternation frequency of the jth action segment, λ j is the weighting factor of the jth action segment.

8. A human-computer interaction optimization system based on deep learning, used to implement a human-computer interaction optimization method based on deep learning as described in any one of claims 1 to 7, characterized in that: It includes gesture image capture module, image data storage and analysis set construction module, feature extraction and analysis module, intelligent evaluation and classification module, normal gesture recognition module and dynamic frame rate adjustment module: In the gesture image capture module, first, the camera captures the user's hand image at a preset frame rate; Image data storage and analysis set building module, the captured hand image frames are stored and organized into an analysis set, from which key features reflecting the rapid changes of gestures are extracted; The feature extraction and analysis module performs detailed analysis on the extracted key features within the detection window and inputs the analyzed key features into a pre-trained machine learning model, which then performs intelligent evaluation on abnormal changes in user gestures. Intelligent evaluation and classification module, based on the evaluation results of the model, divides the user's gesture changes into two categories: normal gesture changes and fast gesture changes; The normal gesture recognition module continues to capture hand images at a preset fixed frame rate for normal gesture changes, and uses gesture recognition algorithms to identify actions and intentions; The dynamic frame rate adjustment module dynamically adjusts the frame rate of the camera for rapid gesture changes based on the evaluation results of the machine learning model, reduces the time interval between frames, and captures more dynamic details.

Images