A gesture control optimization method based on spatial depth and dynamic threshold grading

Abstract

The application discloses a gesture control optimization method based on spatial depth and dynamic threshold grading, and relates to the field of gesture recognition control. The method comprises the following steps: collecting multi-source data; pre-processing the multi-source data; classifying gesture states by combining a dynamic threshold grading algorithm, and calculating a spatial depth value according to a classification result; and predicting and smoothing gesture trajectories under a high-speed motion state by using a Kalman filtering algorithm, and eliminating slight jitter of gesture trajectories under a low-speed static state by using a high-pass filtering algorithm. The application provides a gesture control optimization method based on spatial depth and dynamic threshold grading, corrects depth map data by using an error compensation model of machine learning, effectively improves the measurement accuracy of spatial depth information, and realizes accurate tracking and smoothing processing of gesture trajectories by combining the Kalman filtering algorithm and the high-pass filtering algorithm, thereby significantly improving the stability and fluency of cursor control.

Description

A gesture control optimization method based on spatial depth and dynamic threshold grading Technical Field

[0001] This invention relates to the field of gesture recognition and control, and in particular to a gesture control optimization method based on spatial depth and dynamic threshold grading. Background Technology

[0002] With the rapid development of gesture recognition technology, gesture control technology based on camera devices (such as large-screen displays, televisions, smartphones, and tablets) has gradually become a research hotspot in the field of human-computer interaction. However, existing gesture control technologies still have room for improvement in terms of dynamic threshold fluctuations, spatial depth calculation accuracy, and real-time filtering optimization. Users' needs for precise and smooth control have not been fully met, especially during the switching between high-speed movement and low-speed stationary states, where obvious jitter or delay may occur, thus affecting the user experience.

[0003] A search revealed a Chinese invention patent, CN112507918B, which discloses a gesture recognition method. This patent acquires frame images from video files and uses a trained gesture recognition model to classify and recognize static and dynamic gestures, ultimately generating corresponding gesture commands. However, this technical solution relies primarily on a single frame image processing method, which has limitations in terms of accurate spatial depth measurement and hierarchical optimization of dynamic thresholds. Under high-speed motion, the system may exhibit significant jitter; under low-speed stationary conditions, the suppression of minor jitter is insufficient, resulting in less smooth and stable cursor control. Furthermore, this method does not adequately consider errors caused by different gesture shapes, potentially leading to decreased recognition accuracy and impacting the user experience.

[0004] A search revealed Chinese invention patent CN110442242B, which discloses a smart mirror system and control method based on binocular spatial gesture interaction. This patent utilizes a binocular camera for hand region extraction and key point depth calculation, and implements gesture recognition and control functions through optimized algorithms, supporting various gesture operations (such as movement, dragging, play / pause, etc.). However, this technical solution still has room for improvement in dynamic threshold adjustment and real-time filtering optimization, failing to design different filtering strategies for high-speed movement and low-speed stationary states. During rapid movement, the cursor trajectory may exhibit unevenness; while in a stationary state, the elimination of subtle jitter is not thorough enough. Furthermore, the spatial depth calculation of this method relies on the calibration and correction process of binocular vision, requiring high precision from the hardware, which may increase the system's cost and complexity.

[0005] The aforementioned materials indicate that existing gesture control technologies still have room for improvement in spatial depth measurement, dynamic threshold grading optimization, and real-time filtering. Therefore, this invention provides a gesture control optimization method based on spatial depth and dynamic threshold grading. This method aims to achieve high-precision tracking of gestures in both dynamic and static states by integrating dynamic threshold calculation, spatial coordinate reference definition, Kalman filtering, and high-pass filtering algorithms. Simultaneously, it optimizes computational performance, significantly reducing the resource consumption of real-time computation, thereby achieving smooth and efficient dynamic cursor control to meet the demands for accuracy, stability, and fluency in modern human-computer interaction. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a gesture control optimization method based on spatial depth and dynamic threshold grading. By using a machine learning error compensation model to correct depth map data, the measurement accuracy of spatial depth information is effectively improved. At the same time, by combining Kalman filtering and high-pass filtering algorithms, the precise tracking and smoothing of gesture trajectories are achieved, significantly improving the stability and smoothness of cursor control.

[0007] In a first aspect, the present invention provides a gesture control optimization method based on spatial depth and dynamic threshold grading, comprising the following steps:

[0008] S1. Multi-source data acquisition: The multi-source data includes gesture image data, depth map data, and ambient light intensity data;

[0009] S2. Multi-source data preprocessing: Extract key feature points from gesture image data in multi-source data, correct depth map data errors, and adjust the brightness and contrast of gesture image data according to ambient light intensity data.

[0010] S3. Based on the preprocessed data, the gesture state is classified using a dynamic threshold grading algorithm, and the spatial depth value is calculated based on the classification results.

[0011] S4. In high-speed motion, the Kalman filter algorithm is used to predict and smooth the gesture trajectory; in low-speed stationary state, the high-pass filter algorithm is used to eliminate the slight jitter of the gesture trajectory.

[0012] S5. Map the filtered gesture trajectory to the screen coordinate system to generate cursor control commands;

[0013] S6. Adjust the filtering coefficient and dynamic threshold in real time according to the characteristics of gesture state switching to ensure smooth transition between different states.

[0014] In the above technical solution, the multi-source data acquisition includes the following steps:

[0015] Capture continuous gesture image data through a camera;

[0016] Depth map data is generated using a binocular vision system to reflect the positional information of gestures in three-dimensional space;

[0017] Real-time monitoring of ambient light intensity data in interactive scenarios using an ambient light sensor;

[0018] The screen resolution parameters and interactive area range are provided by external input devices or preset configuration files and are temporarily stored through the data storage unit.

[0019] In the above technical solution, the multi-source data preprocessing includes the following steps:

[0020] Twenty-one gesture feature points were extracted from the gesture image data, and a gesture skeleton model was defined based on the 21 gesture feature points. The 21 gesture feature points include 5 fingertip feature points, 5 proximal interphalangeal feature points, 5 distal interphalangeal feature points, 5 palm-finger feature points, and 1 wrist center feature point.

[0021] An error compensation model based on machine learning is used to correct errors in depth map data;

[0022] The brightness and contrast of the gesture image data are adjusted based on the ambient light intensity data, and the histogram equalization method is used to enhance the gesture recognition effect.

[0023] In the above technical solution, the step of using a machine learning-based error compensation model to correct depth map data errors includes the following steps:

[0024] S21. Record the pixel depth values ​​of the depth map data as... The actual depth values ​​collected at the same time are recorded as The depth map data error consists of systematic error and random noise error;

[0025] The pixel depth value is the raw depth data directly measured by the binocular vision system; the actual depth value is the real physical distance from the object in the scene to the camera.

[0026] The formula for calculating the depth map data error is as follows:

[0027] ;

[0028] ;

[0029] In the formula, For the first The pixel depth values ​​of the depth map data. for Data error, The mean of the system error. This represents the amplitude of random noise. It is the standard normal distribution function;

[0030] S22. Calculate the actual error based on the pixel depth values ​​and actual depth values ​​of the depth map data, and determine the correction value for each pixel depth based on the actual error. The correction formula is as follows:

[0031] ;

[0032] In the formula, for The correction value, for The error compensation value, and , This represents the statistical suppression of random noise.

[0033] In the above technical solution, the step of classifying gesture states using a dynamic threshold grading algorithm and calculating spatial depth values ​​based on the classification results includes the following steps:

[0034] S31. Define the set of gesture feature points based on the gesture skeleton model. Each parameter corresponds to a feature parameter of a gesture feature point, and the feature parameter includes velocity, acceleration and spatial depth value;

[0035] S32. Set several gesture motion state intervals, and each gesture motion state interval is set with a corresponding speed threshold and acceleration threshold.

[0036] S33. Based on the feature parameters of each gesture feature point in the current frame of the gesture skeleton model, calculate the similarity between the gesture and each gesture motion state using the Euclidean distance formula, and select the gesture motion state corresponding to the maximum similarity as the current gesture state.

[0037] S34. Based on the feature parameters of the current gesture state, an interpolation method is used to calculate the spatial depth value of each gesture feature point to compensate for the problem of discontinuous acquisition of pixel depth values ​​of gesture feature points by the binocular vision system when the gesture feature points are moving at high speed. The calculation formula of the interpolation method includes:

[0038] ;

[0039] In the formula, For the first Smooth spatial depth values ​​of gesture feature points This represents the spatial depth value of the previous frame. This represents the spatial depth value of the current frame. The interpolation weight coefficient has a value range of [0, 1]; the interpolation method is used to smooth the changes in spatial depth values ​​in a continuous representation.

[0040] In the above technical solution, the step of using a Kalman filter algorithm to predict and smooth the gesture trajectory during high-speed motion and using a high-pass filter algorithm to eliminate subtle jitter in the gesture trajectory during low-speed stationary motion includes the following steps:

[0041] S41. Set a state threshold to distinguish between low-speed stationary state and high-speed moving state;

[0042] S42. For the hand gesture trajectory in a high-speed motion state, initialize the hand gesture state vector and state estimation covariance of the Kalman filter, and define the state transition matrix and observation matrix.

[0043] S43. Based on the gesture observations of the current frame, update the gesture state vector and state estimation covariance of the Kalman filter. The update calculation formula is as follows:

[0044] ;

[0045] ;

[0046] In the formula, The current frame gesture state vector includes the spatial depth value, motion state, and position coordinates of each gesture feature point. The state transition matrix is ​​pre-defined based on the physical model of the gesture motion and is used to predict the state at the next moment. The Kalman gain matrix; The gesture observations for the current frame include corrected depth map data, histogram-enhanced gesture feature point coordinates, and ambient light compensation parameters. The observation matrix; This is the gesture state vector from the previous frame; To observe the residuals, which are used to reflect sudden gesture changes and trigger adjustments to the interpolation weight coefficients; Estimate the covariance for the current frame state; It is the identity matrix, used to maintain the integrity of matrix operations; Estimate the covariance of the state in the previous frame;

[0047] S44. Use the smooth trajectory output by the Kalman filter as the gesture trajectory in a high-speed motion state;

[0048] S45. For gesture trajectories in a low-speed, stationary state, a high-pass filtering algorithm is used to eliminate subtle jitter in the gesture trajectory. The filtering formula is as follows:

[0049] ;

[0050] In the formula, These are the trajectory values ​​after high-pass filtering. These are the filter coefficients, with values ​​ranging from [0,1].

[0051] In the above technical solution, mapping the filtered gesture trajectory to the screen coordinate system includes the following steps:

[0052] S51. Define the origin of the screen coordinate system and import the screen resolution parameters to establish the mapping relationship between gesture trajectory and screen coordinates.

[0053] S52. Based on the trajectory coordinates of the gesture feature points in the current frame, calculate their mapped position in the screen coordinate system. The mapping formula is as follows:

[0054] ;

[0055] ;

[0056] In the formula, , and These represent the horizontal, vertical, and axial coordinates of the gesture feature points in the current frame, respectively. and These represent the horizontal and vertical coordinates of the mapped position in the screen coordinate system, respectively. and These represent the horizontal and vertical coordinates of the origin of the screen coordinate system, respectively. and These represent the screen resolution parameters;

[0057] S53. Generate cursor control commands based on the mapped position and send them to the target device to achieve real-time response to gesture operations.

[0058] In the above technical solution, the real-time adjustment of filtering parameters and dynamic thresholds based on gesture state switching includes:

[0059] The trigger condition for the gesture state switching is that the speed and acceleration of the gesture feature points switch to different gesture state ranges;

[0060] When a gesture state change is detected, the state estimation covariance of the Kalman filter and the filtering coefficients of the high-pass filter algorithm are adjusted according to the current state.

[0061] The interpolation weight coefficients in the dynamic threshold grading algorithm are recalculated to adapt to the new gesture state.

[0062] Secondly, the present invention provides a gesture control optimization system based on spatial depth and dynamic threshold grading, which applies the gesture control optimization method based on spatial depth and dynamic threshold grading as described above. The system includes a data acquisition module, a data preprocessing module, a depth calculation module, a trajectory filtering module, a coordinate mapping module, and a dynamic parameter tuning module.

[0063] The data acquisition module is used to acquire gesture image data, depth map data, and ambient light intensity data;

[0064] The data preprocessing module is used to extract key feature points from gesture image data in multi-source data and correct depth map data errors.

[0065] The depth calculation module is used to classify gesture states by combining a dynamic threshold grading algorithm and calculate spatial depth values ​​based on the classification results.

[0066] The trajectory filtering module is used to optimize the gesture trajectory. For gesture trajectories in high-speed motion, a Kalman filter algorithm is used to smooth the trajectory; for gesture trajectories in low-speed stationary state, a high-pass filter algorithm is used to eliminate minor jitter.

[0067] The coordinate mapping module is used to establish a mapping relationship between gesture trajectory and screen coordinate system based on screen resolution parameters, and to generate cursor control commands.

[0068] The dynamic parameter adjustment module adjusts the filter coefficient and dynamic threshold in real time according to the characteristics of gesture state switching to ensure a smooth transition between different states.

[0069] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0070] In terms of spatial depth calculation, this invention uses a machine learning-based error compensation model to correct depth map data, which effectively improves the measurement accuracy of spatial depth information and thus enhances the accuracy of gesture recognition.

[0071] Regarding gesture trajectory switching, this invention adopts a threshold grading strategy, combined with Kalman filtering and high-pass filtering algorithms, to achieve accurate tracking and smooth processing of gesture trajectories, significantly improving the stability and fluency of cursor control.

[0072] Furthermore, by adjusting the filtering parameters and dynamic thresholds in real time, this invention meets the response sensitivity requirements under different gesture states, enabling the system to better adapt to complex user interaction scenarios and providing more efficient and accurate technical support for the modern human-computer interaction field. Attached Figure Description

[0073] Figure 1 is a schematic diagram of the first process of the gesture control optimization method based on spatial depth and dynamic threshold grading of the present invention.

[0074] Figure 2 is a schematic diagram of the second process of the gesture control optimization method based on spatial depth and dynamic threshold grading of the present invention.

[0075] Figure 3 is a schematic diagram of the gesture control optimization system based on spatial depth and dynamic threshold grading of the present invention. Detailed Implementation

[0076] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention. It should be noted that relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0077] Example 1

[0078] Referring to Figures 1-2, this invention provides a gesture control optimization method based on spatial depth and dynamic threshold grading. In practical applications, this invention uses a multi-source data acquisition module composed of a camera, a binocular vision system, and an ambient light sensor to acquire gesture image data, depth map data, and ambient light intensity data. The camera captures continuous gesture image data, the binocular vision system generates depth map data reflecting the gesture's position in three-dimensional space, and the ambient light sensor monitors real-time changes in illumination within the interactive scene. The camera, binocular vision system, and ambient light sensor communicate synchronously via a data transmission bus, ensuring timely processing of the acquired data. User interaction scene parameters, such as screen resolution, interaction area range, gesture operation type, and response sensitivity requirements, are provided by external input devices or preset configuration files and temporarily stored in a data storage unit for later processing.

[0079] In the multi-source data preprocessing stage, key feature points are extracted and depth map data errors are corrected on the acquired gesture image data. Specifically, 21 gesture feature points are extracted from the gesture image data, and a gesture skeleton model is defined based on these 21 gesture feature points as a benchmark. The 21 gesture feature points include 5 fingertip feature points, 5 proximal interdigital feature points, 5 distal interdigital feature points, 5 palmodigital feature points, and 1 wrist center feature point.

[0080] Error correction methods for depth map data include geometric calibration, stereo matching optimization, and machine learning-based error compensation models. This embodiment uses a machine learning-based error compensation model to correct errors in depth map data, including the following steps:

[0081] S21. Record the pixel depth values ​​of the depth map data as... The actual depth values ​​collected at the same time are recorded as The depth map data error consists of systematic error and random noise error; the pixel depth value is the raw depth data directly measured by the binocular vision system; the actual depth value is the real physical distance from the object in the scene to the camera.

[0082] The formula for calculating the depth map data error is as follows:

[0083] ;

[0084] ;

[0085] In the formula, For the first The pixel depth values ​​of the depth map data. for Data error, The mean of the system error. This represents the amplitude of random noise. It is the standard normal distribution function.

[0086] S22. Calculate the actual error based on the pixel depth values ​​and actual depth values ​​of the depth map data, and determine the correction value for each pixel depth based on the actual error. The correction formula is as follows:

[0087] ;

[0088] In the formula, for The correction value, for The error compensation value, and , This is a statistical suppression term for random noise; this process can significantly reduce depth map data errors and improve the accuracy of depth map data.

[0089] Specifically, the machine learning-based error compensation model is a pre-trained multiple linear regression model. Its training process includes: using a high-precision laser rangefinder as the ground truth device, simultaneously collecting the actual depth values ​​of the same target at different distances and angles, and the pixel depth values ​​measured by the system's binocular vision system, forming training sample pairs; using these sample pairs, the error compensation value and random noise statistical suppression amount in the correction formula are obtained through least squares fitting, thereby constructing an error compensation model capable of compensating for system errors and suppressing random noise.

[0090] In addition, the brightness and contrast of the gesture image data can be adjusted based on the ambient light intensity data, and the histogram equalization method can be used to enhance the recognition effect of the gesture, making the subsequent feature point extraction more accurate.

[0091] After completing the multi-source data preprocessing, the system proceeds to the gesture state classification and spatial depth information calculation stage. In practical applications, when the gesture state changes, the system needs to accurately identify the changes in gesture state, including the following steps:

[0092] S31. Define the set of gesture feature points based on the gesture skeleton model. Each parameter corresponds to a feature parameter of a gesture feature point, and the feature parameter includes velocity, acceleration and spatial depth value;

[0093] S32. Set several gesture motion state intervals, and each gesture motion state interval is set with a corresponding speed threshold and acceleration threshold.

[0094] S33. Based on the feature parameters of each gesture feature point in the current frame of the gesture skeleton model, calculate the similarity between each gesture motion state using the Euclidean distance formula, and select the gesture motion state corresponding to the maximum similarity as the current gesture state.

[0095] S34. Based on the feature parameters of the current gesture state, an interpolation method is used to calculate the spatial depth value of each gesture feature point to compensate for the problem of discontinuous acquisition of pixel depth values ​​of gesture feature points by the binocular vision system when the gesture feature points are moving at high speed. The calculation formula of the interpolation method includes:

[0096] ;

[0097] In the formula, For the first Smooth spatial depth values ​​of gesture feature points This represents the spatial depth value of the previous frame. This represents the spatial depth value of the current frame. The interpolation weight coefficient has a value range of [0, 1]. The interpolation method can effectively smooth the changes in spatial depth values, obtain the gesture trajectory of each gesture feature point, and avoid gesture trajectory recognition errors caused by sudden changes in depth values.

[0098] In the gesture trajectory filtering stage, Kalman filtering and high-pass filtering algorithms are used for high-speed motion and low-speed stationary states, respectively. When the user quickly moves their hand to select menu options, the system uses the Kalman filtering algorithm to predict and smooth the gesture trajectory in the high-speed motion state. The steps include:

[0099] S41. Set velocity state threshold and acceleration state threshold to distinguish between low-speed stationary state and high-speed motion state; when the velocity and acceleration of the gesture feature point are both less than the velocity state threshold and acceleration state threshold, it is judged as low-speed stationary state; when the velocity and acceleration of the gesture feature point are greater than or equal to the velocity state threshold and acceleration state threshold, it is judged as low-speed stationary state.

[0100] S42. For the hand gesture trajectory in a high-speed motion state, initialize the hand gesture state vector and state estimation covariance of the Kalman filter, and define the state transition matrix and observation matrix.

[0101] S43. Based on the gesture observations of the current frame, update the gesture state vector and state estimation covariance of the Kalman filter. The update calculation formula is as follows:

[0102] ;

[0103] ;

[0104] In the formula, The current frame gesture state vector includes the spatial depth value, motion state, and position coordinates of each gesture feature point. The state transition matrix is ​​pre-defined based on the physical model of the gesture motion and is used to predict the state at the next moment. The Kalman gain matrix; The gesture observations for the current frame include corrected depth map data, histogram-enhanced gesture feature point coordinates, and ambient light compensation parameters. The observation matrix; This is the gesture state vector from the previous frame; To observe the residuals, which are used to reflect sudden gesture changes and trigger adjustments to the interpolation weight coefficients; Estimate the covariance for the current frame state; It is the identity matrix, used to maintain the integrity of matrix operations; Estimate the covariance of the state in the previous frame;

[0105] S44. Use the smooth trajectory output by the Kalman filter as the gesture trajectory in a high-speed motion state;

[0106] S45. For hand gesture trajectories in a low-speed, stationary state, a high-pass filtering algorithm is used to eliminate subtle jitter. The filtering formula is as follows:

[0107] ;

[0108] In the formula, These are the trajectory values ​​after high-pass filtering. These are the filter coefficients, with values ​​ranging from [0,1].

[0109] The combined use of the two filtering algorithms enables precise tracking and smoothing of gesture trajectories under different motion states. For example, when a user remains stationary after selecting a menu option, the high-pass filtering algorithm can effectively eliminate subtle jitter, ensuring the cursor remains stable and preventing accidental operations.

[0110] The filtered gesture trajectory needs to be mapped to the screen coordinate system to generate cursor control commands, including the following steps:

[0111] S51. Define the origin of the screen coordinate system and import the resolution parameters to establish the mapping relationship between gesture trajectory and screen coordinates.

[0112] S52. Based on the trajectory coordinates of the gesture feature points in the current frame, calculate their mapped position in the screen coordinate system. The mapping formula is as follows:

[0113] ;

[0114] ;

[0115] In the formula, , and These represent the horizontal, vertical, and axial coordinates of the gesture feature points in the current frame, respectively. and These represent the horizontal and vertical coordinates of the mapped position in the screen coordinate system, respectively. and These represent the horizontal and vertical coordinates of the origin in the screen coordinate system, respectively. and These represent the screen resolution parameters;

[0116] S53. Generate cursor control commands based on the mapped position and send them to the target device to achieve real-time response to gesture operations.

[0117] To ensure a smooth transition between different gesture states, this invention also designs filtering parameters and a dynamic threshold adjustment mechanism based on the gesture state switching characteristics. The triggering conditions for gesture state switching include the rate of change of velocity and the rate of change of acceleration. When a gesture state switch is detected, the covariance matrix of the Kalman filter and the filtering coefficients of the high-pass filter algorithm are adjusted according to the current state. Simultaneously, the interpolation weight coefficients in the dynamic threshold grading algorithm are recalculated to adapt to the new gesture state. This dynamic adjustment mechanism can effectively address state switching problems in complex user interaction scenarios, improving the system's adaptability and stability.

[0118] The above detailed embodiments describe the technical solution and operating principle of the present invention. By employing a multi-level dynamic threshold grading algorithm, the fusion of Kalman filtering and high-pass filtering algorithms, and a real-time filtering strategy based on gesture state switching, the shortcomings of existing technologies in dynamic threshold fluctuation, spatial depth calculation accuracy, and real-time filtering optimization are addressed. Simultaneously, the introduction of a depth map data error correction method significantly improves the measurement accuracy of spatial depth information, ensuring the accuracy of gesture recognition.

[0119] Example 2

[0120] Please refer to Figure 3. This invention provides a gesture control optimization system based on spatial depth and dynamic threshold grading. It applies the gesture control optimization method based on spatial depth and dynamic threshold grading as described above. The system includes a data acquisition module, a data preprocessing module, a depth calculation module, a trajectory filtering module, a coordinate mapping module, and a dynamic parameter tuning module.

[0121] The data acquisition module is used to acquire gesture image data, depth map data, and ambient light intensity data;

[0122] The data preprocessing module is used to extract key feature points from gesture image data in multi-source data and correct depth map data errors.

[0123] The depth calculation module is used to classify gesture states by combining a dynamic threshold grading algorithm and calculate spatial depth values ​​based on the classification results.

[0124] The trajectory filtering module is used to optimize the gesture trajectory. For gesture trajectories in high-speed motion, a Kalman filter algorithm is used to smooth the trajectory; for gesture trajectories in low-speed stationary state, a high-pass filter algorithm is used to eliminate minor jitter.

[0125] The coordinate mapping module is used to establish a mapping relationship between gesture trajectory and screen coordinate system based on screen resolution parameters, and to generate cursor control commands.

[0126] The dynamic parameter adjustment module adjusts the filter coefficient and dynamic threshold in real time according to the characteristics of gesture state switching to ensure a smooth transition between different states.

[0127] Example 3

[0128] Please refer to Figures 1-3. In order to enable those skilled in the art to fully understand and implement the present invention, the specific implementation principle of the present invention will be further explained below in conjunction with a specific application scenario.

[0129] In the application scenarios of gesture control optimization methods, data acquisition is first completed through the collaborative work of a camera, a binocular vision system, and an ambient light sensor. Taking the interactive interface of a smart TV as an example, users can control the cursor movement to select menu options using gestures. The camera is responsible for capturing a continuous sequence of images of the hand area, the binocular vision system generates a depth map to reflect the hand's position information in three-dimensional space, and the ambient light sensor monitors changes in lighting conditions in the interactive scene in real time. These modules achieve synchronous information transmission through a data transmission bus, ensuring that the collected data can be processed in a timely manner. User interaction scene parameters such as screen resolution parameters, interactive area range, and response sensitivity requirements are provided by external input devices or preset configuration files and temporarily stored in a data storage unit for later processing. The core of this stage lies in the efficient acquisition and synchronous transmission of multi-source data, laying the foundation for subsequent data preprocessing.

[0130] After entering the data preprocessing stage, key feature points are first extracted and depth map data errors are corrected. In the depth map data correction process, the pixel depth values ​​of the depth map data are denoted as... The actual depth values ​​collected at the same time are recorded as The depth map data error consists of systematic error and random noise. The pixel depth value is the raw depth data directly measured by the binocular vision system. The actual depth value is the real physical distance from the object in the scene to the camera.

[0131] The formula for calculating the depth map data error is as follows:

[0132] ;

[0133] ;

[0134] In the formula, For the first The pixel depth values ​​of the depth map data. for Data error, The mean of the system error. This represents the amplitude of random noise. It is the standard normal distribution function.

[0135] The actual error is calculated based on the pixel depth values ​​and actual depth values ​​from the depth map data, and a correction value for each pixel depth is determined based on the actual error. The correction formula is as follows:

[0136] ;

[0137] In the formula, for The correction value, for The error compensation value, and , This represents the statistical suppression of random noise. The corrected depth map result is denoted as... The error distribution is significantly reduced, thereby improving the accuracy of depth information.

[0138] Specifically, the machine learning-based error compensation model is a pre-trained multiple linear regression model. Its training process includes: in a controlled environment, using a high-precision laser rangefinder as the ground truth device, simultaneously collecting the actual depth values ​​of the same target at different distances and angles, and the pixel depth values ​​measured by the system's binocular vision system, forming training sample pairs; using these sample pairs, the error compensation value and random noise statistical suppression amount in the correction formula are obtained through least squares fitting, thereby constructing an error compensation model capable of compensating for system errors and suppressing random noise.

[0139] Furthermore, image brightness and contrast are adjusted based on ambient light intensity distribution data, and histogram equalization is used to enhance the recognition effect of the hand region, making subsequent feature point extraction more accurate. The key to this stage lies in the introduction of a depth map error correction method, which significantly improves the measurement accuracy of spatial depth information and provides reliable data support for subsequent gesture state classification and trajectory filtering.

[0140] After data preprocessing, the system proceeds to the gesture state classification and spatial depth value calculation stage. In practical applications, the system needs to accurately identify changes in gesture state. First, a set of gesture states is defined. Each state parameter corresponds to a feature parameter of a gesture feature point, including velocity, acceleration, and spatial depth. Several gesture motion state intervals are then defined, each with corresponding velocity and acceleration thresholds. Based on the feature parameters of each gesture feature point in the current frame of the gesture skeleton model, the similarity between the gesture and each gesture motion state is calculated using the Euclidean distance formula, and the gesture motion state with the highest similarity is selected as the current gesture state. Finally, based on the feature parameters of the current gesture state, an interpolation method is used to calculate the spatial depth value of each gesture feature point. This compensates for the discontinuous acquisition of pixel depth values ​​of gesture feature points by the binocular vision system during high-speed movement. The calculation formula for the interpolation method includes:

[0141] ;

[0142] In the formula, For the first Smooth spatial depth values ​​of gesture feature points This represents the spatial depth value of the previous frame. This represents the spatial depth value of the current frame. The interpolation weight coefficient has a value range of [0, 1]. The interpolation method is used to smooth the changes in spatial depth values ​​in a continuous representation. For example, when a user switches from a low-speed stationary state to a high-speed moving state, the interpolation method can smoothly transition, reduce the occurrence of jitter, and thus improve the user experience.

[0143] In the gesture trajectory filtering stage, Kalman filtering and high-pass filtering algorithms are used for high-speed motion and low-speed stationary states, respectively. For gesture trajectories in high-speed motion states, the gesture state vector and state estimation covariance of the Kalman filter are first initialized, and the state transition matrix and observation matrix are defined. Then, based on the gesture observations of the current frame, the gesture state vector and state estimation covariance of the Kalman filter are updated. The update calculation formula is as follows:

[0144] ;

[0145] ;

[0146] In the formula, The current frame gesture state vector includes the spatial depth value, motion state, and position coordinates of each gesture feature point. The state transition matrix is ​​pre-defined based on the physical model of the gesture motion and is used to predict the state at the next moment. The Kalman gain matrix; The gesture observations for the current frame include corrected depth map data, histogram-enhanced gesture feature point coordinates, and ambient light compensation parameters. The observation matrix; This is the gesture state vector from the previous frame; To observe the residuals, which are used to reflect sudden gesture changes and trigger adjustments to the interpolation weight coefficients; Estimate the covariance for the current frame state; It is the identity matrix, used to maintain the integrity of matrix operations; Estimate the covariance of the state in the previous frame;

[0147] For hand gesture trajectories in a low-speed, stationary state, a high-pass filtering algorithm is used to eliminate subtle jitter. The filtering formula is as follows:

[0148] ;

[0149] In the formula, These are the trajectory values ​​after high-pass filtering. These are the filter coefficients, with values ​​ranging from [0,1].

[0150] The combined use of the two filtering algorithms enables precise tracking and smoothing of gesture trajectories under different motion states. For example, when a user remains stationary after selecting a menu option, the high-pass filtering algorithm can effectively eliminate subtle jitter, ensuring the cursor remains stable and preventing accidental operations.

[0151] The filtered gesture trajectory needs to be mapped to the screen coordinate system to generate cursor control commands, which are then sent to the target device to achieve real-time response to gesture operations. In the above application scenario, the origin of the screen coordinate system needs to be defined first, and resolution parameters need to be imported to establish the mapping relationship between the gesture trajectory and the screen coordinates. Then, based on the trajectory coordinates of the gesture feature points in the current frame, their mapped positions in the screen coordinate system are calculated, thus obtaining the mapped position in the screen coordinate system. The mapping formula is as follows:

[0152] ;

[0153] ;

[0154] In the formula, , and These represent the horizontal, vertical, and axial coordinates of the gesture feature points in the current frame, respectively. and These represent the horizontal and vertical coordinates of the mapped position in the screen coordinate system, respectively. and These represent the horizontal and vertical coordinates of the origin in the screen coordinate system, respectively. and These represent screen resolution parameters. For example, when a user moves a gesture to a menu option, the system can accurately locate the cursor based on the mapped position of the gesture trajectory and generate corresponding control commands to ensure that the cursor matches the user's intention.

[0155] To ensure a smooth transition between different gesture states, this invention designs a filtering parameter and dynamic threshold adjustment mechanism based on the characteristics of gesture state switching. In the aforementioned application scenario, when a user's gesture is detected to switch from a high-speed motion state to a low-speed stationary state, the system triggers the gesture state switching condition based on changes in speed and acceleration. Subsequently, the state estimation covariance of the Kalman filter and the filtering coefficients of the high-pass filter algorithm are adjusted according to the current state. Simultaneously, the interpolation weight coefficients in the dynamic threshold grading algorithm are recalculated to adapt to the new gesture state. This dynamic adjustment mechanism can effectively address state switching problems in complex user interaction scenarios, improving the system's adaptability and stability. For example, when a user suddenly stops after a rapid gesture, the system can quickly adjust the filtering parameters and dynamic threshold to ensure a smooth transition of the cursor to the target position, avoiding noticeable jitter or delay.

[0156] In summary, this invention addresses the shortcomings of existing technologies in terms of dynamic threshold fluctuation, spatial depth calculation accuracy, and real-time filtering optimization by employing a multi-level dynamic threshold grading algorithm, the fusion of Kalman filtering and high-pass filtering algorithms, and a real-time filtering strategy based on gesture state switching. Simultaneously, the introduction of a depth map data error correction method significantly improves the measurement accuracy of spatial depth information, ensuring the accuracy of gesture recognition. Combined with the specific application scenarios described above, the technical solution of this invention effectively meets the demands for accuracy, stability, and smoothness in modern human-computer interaction, providing users with a superior interactive experience.

Claims

1. A gesture control optimization method based on spatial depth and dynamic threshold grading, characterized in that, The process includes the following steps: S1, Multi-source data acquisition: The multi-source data includes gesture image data, depth map data, and ambient light intensity data; S2, Multi-source data preprocessing: Key feature points of the gesture image data are extracted from the multi-source data, and errors in the depth map data are corrected. Simultaneously, the brightness and contrast of the gesture image data are adjusted based on the ambient light intensity data; S3, Based on the preprocessed data, the gesture state is classified using a dynamic threshold grading algorithm, and the spatial depth value is calculated based on the classification results; S4, In high-speed motion, a Kalman filter algorithm is used to predict and smooth the gesture trajectory; in low-speed stationary state, a high-pass filter algorithm is used to eliminate subtle jitter in the gesture trajectory; S5, The filtered gesture trajectory is mapped to the screen coordinate system to generate cursor control commands; S6, Based on the gesture state switching characteristics, the filter coefficient and dynamic threshold are adjusted in real time to ensure smooth transitions between different states; The step of classifying the gesture state using the dynamic threshold grading algorithm and calculating the spatial depth value based on the classification results includes: S31, Defining a set of gesture feature points based on the gesture skeleton model. Each parameter corresponds to a feature parameter of a gesture feature point, and the feature parameter includes velocity, acceleration, and spatial depth value; S32, set several gesture motion state intervals, and each gesture motion state interval has a corresponding velocity threshold and acceleration threshold; S33, according to the feature parameters of each gesture feature point in the current frame in the gesture skeleton model, calculate the similarity between it and each gesture motion state using the Euclidean distance formula, and select the gesture motion state corresponding to the maximum similarity as the current gesture state; S34, based on the feature parameters of the current gesture state, use interpolation to calculate the spatial depth value of each gesture feature point to make up for the problem of discontinuous acquisition of pixel depth values ​​of gesture feature points by the binocular vision system when the gesture feature points move at high speed; The step of adjusting the filter coefficients and dynamic thresholds in real time based on the characteristics of gesture state switching includes: the trigger condition for gesture state switching is that the velocity and acceleration of the gesture feature points switch to different gesture state intervals; when a gesture state switch is detected, the state estimation covariance of the Kalman filter and the filter coefficients of the high-pass filter algorithm are adjusted according to the current state; the interpolation weight coefficients in the dynamic threshold grading algorithm are recalculated to adapt to the new gesture state.

2. The gesture control optimization method based on spatial depth and dynamic threshold grading according to claim 1, characterized in that: The multi-source data acquisition includes the following steps: capturing continuous gesture image data through a camera; generating depth map data through a binocular vision system to reflect the position information of the gesture in three-dimensional space; The ambient light intensity data in the interactive scene is monitored in real time by an ambient light sensor; the screen resolution parameters and interactive area range are provided by an external input device or a preset configuration file and temporarily stored through a data storage unit.

3. The gesture control optimization method based on spatial depth and dynamic threshold grading according to claim 1, characterized in that: The multi-source data preprocessing includes the following steps: extracting 21 gesture feature points from the gesture image data, and defining a gesture skeleton model based on the 21 gesture feature points, wherein the 21 gesture feature points include 5 fingertip feature points, 5 proximal interdigital feature points, 5 distal interdigital feature points, 5 palmar-finger feature points, and 1 wrist center feature point; correcting the depth map data error using a machine learning-based error compensation model; adjusting the brightness and contrast of the gesture image data according to the ambient light intensity data, and using a histogram equalization method to enhance the gesture recognition effect.

4. The gesture control optimization method based on spatial depth and dynamic threshold grading according to claim 3, characterized in that: The method of using a machine learning-based error compensation model to correct depth map data errors includes the following steps: S21, recording the pixel depth values ​​of the depth map data as... The actual depth values ​​collected at the same time are recorded as The depth map data error consists of systematic error and random noise error; the pixel depth value is the raw depth data directly measured by the binocular vision system; the actual depth value is the real physical distance from the object in the scene to the camera; S22, calculate the actual error based on the pixel depth value and the actual depth value of the depth map data, and determine the correction value of each pixel depth based on the actual error.

5. The gesture control optimization method based on spatial depth and dynamic threshold grading according to claim 1, characterized in that: The process involves using a Kalman filter algorithm to predict and smooth the gesture trajectory during high-speed motion, and a high-pass filter algorithm to eliminate subtle jitter in the gesture trajectory during low-speed stationary motion. The steps include: S41, setting a state threshold to distinguish between low-speed stationary and high-speed motion states; S42, initializing the gesture state vector and state estimation covariance of the Kalman filter for the gesture trajectory during high-speed motion, and defining the state transition matrix and observation matrix; S43, updating the gesture state vector and state estimation covariance of the Kalman filter based on the gesture observations of the current frame; S44, using the smoothed trajectory output by the Kalman filter as the gesture trajectory during high-speed motion; and S45, using a high-pass filter algorithm to eliminate subtle jitter in the gesture trajectory during low-speed stationary motion.

6. The gesture control optimization method based on spatial depth and dynamic threshold grading according to claim 1, characterized in that: The step of mapping the filtered gesture trajectory to the screen coordinate system includes the following steps: S51, defining the origin of the screen coordinate system and importing the screen resolution parameters to establish a mapping relationship between the gesture trajectory and the screen coordinates; S52, calculating the mapping position of the gesture feature point trajectory coordinates in the current frame in the screen coordinate system; S53, generating cursor control commands according to the mapping position and sending them to the target device to achieve real-time response to gesture operations.

7. A gesture control optimization system based on spatial depth and dynamic threshold hierarchy, employing the gesture control optimization method based on spatial depth and dynamic threshold hierarchy as described in any one of claims 1-6, characterized in that, The system includes a data acquisition module, a data preprocessing module, a depth calculation module, a trajectory filtering module, a coordinate mapping module, and a dynamic parameter tuning module. The data acquisition module acquires gesture image data, depth map data, and ambient light intensity data. The data preprocessing module extracts key feature points from the gesture image data from multiple sources and corrects errors in the depth map data. The depth calculation module classifies gesture states using a dynamic threshold grading algorithm and calculates spatial depth values ​​based on the classification results. The trajectory filtering module optimizes gesture trajectories; for high-speed motion gesture trajectories, a Kalman filter is used to smooth the trajectory; for low-speed, stationary gesture trajectories, a high-pass filter is used to eliminate minor jitter. The coordinate mapping module establishes a mapping relationship between the gesture trajectory and the screen coordinate system based on screen resolution parameters and generates cursor control commands. The dynamic parameter adjustment module adjusts the filter coefficient and dynamic threshold in real time according to the characteristics of gesture state switching to ensure a smooth transition between different states.

Images