A method for processing gesture data in motion, electronic device and storage medium

Abstract

A method for processing gesture data in motion, an electronic device and a storage medium, belonging to the field of user interaction technology. In order to solve the problem of accurate data processing of gestures in motion, the present invention includes using an intelligent data glove to collect three-axis accelerometer data and three-axis gyroscope data in motion; performing comparative analysis on the collected three-axis accelerometer data and three-axis gyroscope data in motion, and extracting gesture recognition data used in motion; constructing an arm state segmentation algorithm module based on arm motion state detection, and inputting the gesture recognition data used in motion into the arm state segmentation algorithm module based on arm motion state detection to obtain filtered dynamic gesture data; then inputting the filtered dynamic gesture data into the dynamic gesture segmentation algorithm module based on line segment fitting model comparison to perform gesture segmentation, and then performing data noise reduction to obtain processed gesture data in motion.

Description

Technical Field

[0001] The present invention belongs to the technical field of user interaction, and in particular relates to a method for processing gesture data in motion, an electronic device and a storage medium. Background Art

[0002] With the continuous development of science and technology, human-computer interaction has gradually become an indispensable part of people's lives. As an important branch of human-computer interaction, gesture recognition technology is now used in various fields such as smart home, autonomous driving, virtual reality, etc. With the continuous advancement of gesture recognition technology, there are higher recognition requirements for gesture recognition accuracy.

[0003] For gesture recognition under the interference of motion environment, during the use of smart data gloves, when the user is in motion and the arm swings naturally, the data change trend of the motion sensor is similar to the motion trajectory of the real dynamic gesture, which can easily misidentify the arm swing as a real dynamic gesture. Real-time gesture segmentation is required in the interference environment. Summary of the invention

[0004] The problem to be solved by the present invention is to perform accurate data processing on gestures in motion, and propose a method for processing gesture data in motion, an electronic device and a storage medium.

[0005] To achieve the above object, the present invention is implemented through the following technical solutions:

[0006] A method for processing gesture data in motion comprises the following steps:

[0007] S1. Use smart data gloves to collect experimental data during exercise, including three-axis accelerometer data and three-axis gyroscope data during exercise;

[0008] S2. Compare and analyze the three-axis accelerometer data and the three-axis gyroscope data collected during the movement in step S1, and extract gesture recognition data for the movement;

[0009] S3. Construct an arm state segmentation algorithm module based on arm motion state detection, and input the gesture recognition data for motion obtained in step S2 into the arm state segmentation algorithm module based on arm motion state detection to obtain filtered dynamic gesture data;

[0010] S4. Then, the filtered dynamic gesture data obtained in step S3 is input into the dynamic gesture segmentation algorithm module based on line fitting model comparison for gesture segmentation, and then data noise reduction is performed to obtain the processed gesture data in motion.

[0011] Furthermore, step S1 collects experimental data during the movement process. The movement process is controlled to jog at a speed of 5 km / h or less for more than 1.5 seconds before making a gesture. The arms are naturally swung during jogging. Before making a gesture, the arms are raised to the chest and kept stable for 0.05 seconds. Then, the same dynamic gesture is made twice. After that, the arms are kept raised and stable for 0.05 seconds, and then jogging for more than 1.5 seconds is continued before making a gesture.

[0012] Furthermore, step S2 plots the collected 6-axis sensor data of the state switching between natural arm swinging during movement, raising the arm horizontally during movement, and then to natural arm swinging during movement according to time and dimension, and performs analysis as follows: based on the influence of gravity, the movement data of the three-axis accelerometer data in different arm states is not much different, and the data features of natural arm swinging and raising the arm horizontally cannot be extracted, and the movement state of the arm cannot be distinguished; the data features of the three-axis gyroscope data switching between the arm states during movement are obvious, and the three-axis gyroscope data is extracted for gesture recognition data in movement.

[0013] Furthermore, the specific implementation method of step S3 to construct an arm state segmentation algorithm module based on arm motion state detection includes the following steps:

[0014] S3.1. Use the energy function of the gyroscope signal to describe the fluctuation of the gyroscope signal. The expression of the energy of the gyroscope signal of the i-th frame is E i for:

[0015]

[0016] in, Indicates the jth component value of the gyroscope in the i-th frame;

[0017] S3.2. Set the gyroscope threshold Th gro , steady-state threshold Th static , motion window threshold Th sports_window ;

[0018] S3.3. Initialize the static fluctuation window and the dynamic fluctuation window to 10, and then use the gesture recognition data for motion obtained in step S2 to calculate the sliding window differential energy value Energy gro , the expression is:

[0019]

[0020] Among them, X i [k] is the k-th dimension data of the i-th frame, k is one of the 11 dimensions, and abs is the absolute value function;

[0021] S3.4. Determine whether it is an arm swing fluctuation by comparing the sliding window differential energy value obtained in step S3.3 with the gyroscope threshold:

[0022] If the sliding window differential energy value is less than the gyroscope threshold, and if the stable state threshold is TRUE, the arm is currently in a stable state, and the current frame effectively enters the next step of the dynamic gesture segmentation algorithm module based on line segment fitting model comparison. The arm goes from swinging to stability, and the motion window is cleared;

[0023] If the sliding window differential energy value is greater than or equal to the gyroscope threshold, the current state is the arm swing state, and the size relationship between the motion window and the motion window threshold is continued to be determined. If the motion window is smaller than the motion window threshold, it cannot be distinguished whether it is an arm swing or a dynamic gesture action. Assuming that the current frame is valid, the current frame is valid and enters the next step of the dynamic gesture segmentation algorithm module based on the line segment fitting model comparison, and the motion window is increased by 1;

[0024] If the number of motion windows exceeds the motion window threshold, the motion state is identified as an invalid gesture of natural arm swinging, the collected dynamic data corresponding to the invalid gesture is cleared, and the motion window is increased by 1.

[0025] Furthermore, the specific implementation method of step S4 for constructing a dynamic gesture segmentation algorithm module based on line segment fitting model comparison includes the following steps:

[0026] S4.1. Set the gyroscope threshold Th gro , steady-state threshold Th static , similarity threshold Th Similarity , motion window threshold Th sports_window , the slope of the straight line when the gesture is raised flat K 抬平 ;

[0027] S4.2. Initialize the static fluctuation window and the dynamic fluctuation window to 10, and then use the filtered dynamic gesture data obtained in step S3 to calculate the sliding window differential energy value Energy gro Then, use the five frames of data to perform linear fitting in a time domain space to obtain the slope K of the current fitting line. i , the expression is:

[0028] K i =(g i -g i-1 ) / (g i-1 -g i-2 )=(g i-2 -g i-3 ) / (g i-3 -g i-4 )

[0029] Among them, gi represents the total acceleration of the three axes x and zy of the i-th frame;

[0030] S4.3. Determine the slope K of the straight line when the gesture is lifted in step S4.1 抬平 state, if K 抬平 If the value of is not empty, the Euclidean distance between the straight line when the arm is raised and the current fitting straight line is calculated. The expression is:

[0031] distance=(K 抬平 -K i ) 2

[0032] In order to increase the gap between different straight lines of linear fitting and make them easier to distinguish, distance' is used to represent the exponential power of the Euclidean distance between two states. The expression is:

[0033]

[0034] If K 抬平 The value of is empty, and Th static ==TRUE, then it is the first time that the arm is raised horizontally, let K 抬平 =K i ;

[0035] If the sliding window differential energy value is less than the gyroscope threshold, and the Euclidean distance between the straight line when the arm is raised and the current fitting line is less than the similarity threshold, it is in a relatively stable state and the straight line when the arm is raised is similar to the current fitting line. Dynamic gesture segmentation information is added and recorded as the end of gesture segmentation;

[0036] Otherwise, it means that the current gesture is dynamic, and the dynamic gesture segmentation information is added, recorded as the end of gesture segmentation, and then the gesture segmentation index is returned;

[0037] S4.4. Perform data denoising on the dynamic gesture data after gesture segmentation in step S4.3 to obtain processed gesture data in motion. The method of data denoising is to first perform 5-point 3-times filtering on the data using the five-point cubic filtering method, and then use arithmetic mean filtering for each point in the data, taking the average value of a certain number of sampling points around it as the output value of the point. The overall expression is:

[0038]

[0039] Among them, y(i) is the output value of the five-point cubic arithmetic mean filtering method of the original data of the i-th frame of the current calculation dimension, a i The dynamic gesture data of the i-th frame of the current calculation dimension.

[0040] An electronic device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps of the method for processing gesture data in motion are implemented.

[0041] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for processing gesture data in motion is implemented.

[0042] Beneficial effects of the present invention:

[0043] The method for processing gesture data in motion described in the present invention, the five-point cubic arithmetic mean filtering method neutralizes the five-point cubic filtering method's ability to retain the overall trend and detail features of the data sequence, but the data may produce data that is not true value; and the arithmetic mean filtering method can greatly reduce the noise in the signal, but it will also destroy certain features of the signal during processing, and may cause certain changes in the peak and valley values ​​of the signal. Compared with the differential energy numerical segmentation algorithm that simply uses the accelerometer gyroscope, the dynamic gesture segmentation algorithm based on line segment fitting model comparison has the advantage of being able to accurately segment dynamic gestures when the user is in motion or non-motion. The differential energy numerical segmentation algorithm that simply uses the accelerometer and gyroscope will first set a fixed energy threshold. When the differential energy of the accelerometer and gyroscope exceeds the energy threshold, it is considered a dynamic gesture. This algorithm can achieve good results when the user is in a non-motion state, with a recognition accuracy of 99%. However, the disadvantages are obvious. First, when the user is in motion, the recognition accuracy is only 0%, because the differential energy of the accelerometer and gyroscope in motion will exceed the energy threshold even if no dynamic gestures are made, and the dynamic gestures cannot be segmented. Second, if the energy threshold is increased, some dynamic gestures in the non-motion state will not be correctly segmented because the energy threshold is too high. For dynamic gestures in the motion state, the energy threshold is too high, which requires users to make dynamic gestures quickly and accurately, which is not conducive to new users who have never used it. The advantage of the dynamic gesture segmentation algorithm based on line fitting model comparison is that there is no fixed threshold when segmenting dynamic gestures. Instead, the fitting straight line when the arm is raised flat is used as the basis for segmentation. In this way, even if the fitting straight line when the arm is raised flat is different each time, the algorithm does not need to change the threshold, and can perform segmentation in real time according to the current state. Moreover, this segmentation method can perform accurate segmentation in both non-motion and motion states.

[0044] The method for processing gesture data in motion described in the present invention has an average recognition accuracy that is 7%, 3% and 2% higher than that of the classic classification algorithms SVM, Knn and spectral clustering, respectively. At the same time, the prediction time is shorter than that of other algorithms. Therefore, the recognition effect of the adopted algorithm is verified, indicating the advancement and superiority of the proposed static gesture recognition method in a moving environment, and at the same time meets the research content requirements.

[0045] The method for processing gesture data in motion described in the present invention has an average recognition accuracy of 83.32% for dynamic gestures under limited motion conditions, which is about 10% higher than the recognition accuracy of 70% of the existing algorithm. In addition, gesture recognition can be performed without collecting a large amount of dynamic gesture data in motion, saving a lot of manpower and time costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A flowchart of a method for processing gesture data in motion according to the present invention;

[0047] Figure 2 The following are comparison diagrams of the raw data of the continuous gesture sensor of the present invention, wherein group (a) is a three-axis accelerometer data curve during the motion process, and group (b) is a three-axis gyroscope data curve during the motion process;

[0048] Figure 3 This is a comparison diagram of the five-point cubic smoothing filter processing of the present invention;

[0049] Figure 4 This is a diagram of the arm state detection result of the method of the present invention;

[0050] Figure 5 is the X-axis gyroscope data curve obtained by the method of the present invention in the jogging experiment;

[0051] Figure 6 This is a Z-axis gyroscope data curve obtained by the method of the present invention in a jogging experiment. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention more clear, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention, that is, the specific embodiments described are only part of the embodiments of the present invention, rather than all of the specific embodiments. The components of the specific embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations, and the present invention can also have other embodiments.

[0053] Therefore, the following detailed description of the specific embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents the selected specific embodiments of the present invention. Based on the specific embodiments of the present invention, all other specific embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0054] In order to further understand the content, features and effects of the present invention, the following specific implementation methods are given as examples, and the attached Figure 1 -Attached Figure 6 The detailed instructions are as follows:

[0055] Embodiment 1:

[0056] A method for processing gesture data in motion comprises the following steps:

[0057] S1. Use smart data gloves to collect experimental data during exercise, including three-axis accelerometer data and three-axis gyroscope data during exercise;

[0058] Further, step S1 collects experimental data during the exercise process, and the exercise process is controlled to jog at a speed of 5 km / h or less for more than 1.5 seconds before making a gesture, and the arms are naturally swung during jogging. Before making a gesture, the arms are raised flat to the chest, and the arms are kept stable for 0.05 seconds, and then the same dynamic gesture is made twice. After the gesture, the arms are kept raised flat and stable for 0.05 seconds, and then jogging for more than 1.5 seconds is continued before making a gesture;

[0059] Furthermore, the smart data gloves adopt the smart data gloves disclosed in the patent number CN117290773A and the invention name “A method and system for identifying amphibious and land-based personalized gestures based on smart data gloves”;

[0060] S2. Compare and analyze the three-axis accelerometer data and the three-axis gyroscope data collected during the movement in step S1, and extract gesture recognition data for the movement;

[0061] Further, step S2 plots the collected 6-axis sensor data of the state switching between the natural arm swing during the movement, the arm raised during the movement, and the natural arm swing during the movement according to time and dimension, and analyzes it as follows: based on the influence of gravity, the movement data of the three-axis accelerometer data in different arm states is not much different, and the data features of the natural arm swing and the arm raised cannot be extracted, and the movement state of the arm cannot be distinguished. The data features of the three-axis gyroscope data switching between the arm states during the movement are obvious, and the three-axis gyroscope data is extracted for gesture recognition data in the movement;

[0062] like Figure 2 As shown, Figure 2 The four data segments marked in red boxes are (a) the data of three-axis acceleration. The differential acceleration values ​​fluctuate greatly, and it is impossible to determine whether the arm is in a stable or swinging state; (b) the three pictures are the data of three-axis gyroscopes. The differential gyroscope values ​​of the gyroscopes are basically near the same horizontal line. It is easy to judge from the pictures that the arm is in a stable state. After analyzing the data features of the arm in different motion states, an algorithm is designed to detect the state change of the arm from natural swing to raising the arm, and filter out invalid dynamic data such as arm swing.

[0063] S3. Construct an arm state segmentation algorithm module based on arm motion state detection, and input the gesture recognition data for motion obtained in step S2 into the arm state segmentation algorithm module based on arm motion state detection to obtain filtered dynamic gesture data;

[0064] Furthermore, the specific implementation method of step S3 to construct an arm state segmentation algorithm module based on arm motion state detection includes the following steps:

[0065] S3.1. Use the energy function of the gyroscope signal to describe the fluctuation of the gyroscope signal. The expression of the energy of the gyroscope signal of the i-th frame is E i for:

[0066]

[0067] in, Indicates the jth component value of the gyroscope in the i-th frame;

[0068] S3.2. Set the gyroscope threshold Th gro , steady-state threshold Th static , motion window threshold Th sports_window ;

[0069] S3.3. Initialize the static fluctuation window and the dynamic fluctuation window to 10, and then use the gesture recognition data for motion obtained in step S2 to calculate the sliding window differential energy value Energy gro , the expression is:

[0070]

[0071] Among them, X i [k] is the k-th dimension data of the i-th frame, k is one of the 11 dimensions, and abs is the absolute value function;

[0072] S3.4. Determine whether it is an arm swing fluctuation by comparing the sliding window differential energy value obtained in step S3.3 with the gyroscope threshold:

[0073] If the sliding window differential energy value is less than the gyroscope threshold, and if the stable state threshold is TRUE, the arm is currently in a stable state, and the current frame effectively enters the next step of the dynamic gesture segmentation algorithm module based on line segment fitting model comparison. The arm goes from swinging to stability, and the motion window is cleared;

[0074] If the sliding window differential energy value is greater than or equal to the gyroscope threshold, the current state is the arm swing state, and the size relationship between the motion window and the motion window threshold is continued to be determined. If the motion window is smaller than the motion window threshold, it cannot be distinguished whether it is an arm swing or a dynamic gesture action. Assuming that the current frame is valid, the current frame is valid and enters the next step of the dynamic gesture segmentation algorithm module based on the line segment fitting model comparison, and the motion window is increased by 1;

[0075] If the number of motion windows exceeds the motion window threshold, the motion state is identified as an invalid gesture of natural arm swing, the collected dynamic data corresponding to the invalid gesture is cleared, and the motion window is increased by 1;

[0076] Furthermore, when the arm is raised flat, a stable window of 3 frames will be collected. If all 3 frames remain stable, it means that the arm is in the raised state, the motion window is cleared, and dynamic gestures can be started. At the same time, after segmenting the state change between the natural swing of the arm and the raised arm, the segmentation of dynamic gestures continues. Since raising the arm flat and then making a dynamic gesture to raising the arm flat again is actually equivalent to two state changes, these two changes can be used as the starting point for gesture segmentation. However, since the time of raising the arm flat is uncertain, direct segmentation will result in the collection of more redundant data. Therefore, a dynamic gesture segmentation algorithm based on line segment fitting model comparison is proposed to perform dynamic gesture segmentation in motion to ensure that the dynamic gestures in motion are neither missing nor redundant.

[0077] S4. Then, the filtered dynamic gesture data obtained in step S3 is input into the dynamic gesture segmentation algorithm module based on line fitting model comparison for gesture segmentation, and then data noise reduction is performed to obtain the processed gesture data in motion.

[0078] Furthermore, the specific implementation method of step S4 for constructing a dynamic gesture segmentation algorithm module based on line segment fitting model comparison includes the following steps:

[0079] S4.1. Set the gyroscope threshold Th gro , steady-state threshold Th static , similarity threshold Th Similarity , motion window threshold Th sports_window , the slope of the straight line when the gesture is raised flat K 抬平 ;

[0080] S4.2. Initialize the static fluctuation window and the dynamic fluctuation window to 10, and then use the filtered dynamic gesture data obtained in step S3 to calculate the sliding window differential energy value Energy gro Then, use the five frames of data to perform linear fitting in a time domain space to obtain the slope K of the current fitting line. i , the expression is:

[0081] K i =(g i -g i-1 ) / (g i-1 -g i-2 )=(g i-2 -g i-3 ) / (g i-3 -g i-4 )

[0082] Among them, g i represents the total acceleration of the three axes x and zy of the i-th frame;

[0083] S4.3. Determine the slope K of the straight line when the gesture is lifted in step S4.1 抬平 state, if K 抬平 If the value of is not empty, the Euclidean distance between the straight line when the arm is raised and the current fitting straight line is calculated. The expression is:

[0084] distance=(K 抬平 -K i ) 2

[0085] In order to increase the gap between different straight lines of linear fitting and make them easier to distinguish, distance' is used to represent the exponential power of the Euclidean distance between two states. The expression is:

[0086]

[0087] If K 抬平 The value of is empty, and Th static ==TRUE, then it is the first time that the arm is raised horizontally, let K 抬平 =K i ;

[0088] If the sliding window differential energy value is less than the gyroscope threshold, and the Euclidean distance between the straight line when the arm is raised and the current fitting line is less than the similarity threshold, it is in a relatively stable state and the straight line when the arm is raised is similar to the current fitting line. Dynamic gesture segmentation information is added and recorded as the end of gesture segmentation;

[0089] Otherwise, it means that the current gesture is dynamic, and the dynamic gesture segmentation information is added, recorded as the end of gesture segmentation, and then the gesture segmentation index is returned;

[0090] S4.4. Perform data denoising on the dynamic gesture data after gesture segmentation in step S4.3 to obtain processed gesture data in motion. The method of data denoising is to first perform 5-point 3-times filtering on the data using the five-point cubic filtering method, and then use arithmetic mean filtering for each point in the data, taking the average value of a certain number of sampling points around it as the output value of the point. The overall expression is:

[0091]

[0092] Among them, y(i) is the output value of the five-point cubic arithmetic mean filtering method of the original data of the i-th frame of the current calculation dimension, a i The dynamic gesture data of the i-th frame of the current calculation dimension.

[0093] Furthermore, during the application process in complex environments, it may be affected by strong interference environments, and the sensor will generate large jitter noise during movement. In such a strong interference environment during movement, although the use of the extreme value average filtering algorithm has a better smoothing effect, it may not be able to retain the detailed information of the data series during movement because it will remove extreme points. In order to neutralize the five-point cubic filtering method, which can retain the overall trend and detailed characteristics of the data series, but the data may produce data that is not the true value; and the arithmetic mean filtering, which can greatly reduce the noise in the signal, but will also destroy certain characteristics of the signal during processing, and may cause certain changes in the peak and valley values ​​of the signal, the data is processed by combining the two. In this way, the data of the principle true value generated after the 5-point 3-times filtering can be effectively removed, making the entire segment of data closer to the real data. Figure 3 As shown, blue is the original data, red is the five-point cubic filter, yellow is the five-point cubic filter combined with the arithmetic mean filter, and the yellow line segment shows that the data that deviates from the true value of the red line is changed to data closer to the true value. The yellow line segment is the data noise reduction process obtained by combining the five-point cubic filter algorithm with the arithmetic mean filter algorithm. After the original data is filtered three times at five points, the overall trend and detailed features of the data sequence are retained, but this data may produce data that is not true, especially at the very beginning. So the arithmetic mean filter is used again.

[0094] In this embodiment, a method for processing gesture data in motion is described, which determines the arm state according to an algorithm, uses a differential gyroscope as a detection condition, and the arm state detection result is as follows: Figure 4 As shown, the green line is the differential gyroscope threshold, and the red line is the stable state of the arm segmented by the dynamic gesture segmentation algorithm based on line segment fitting model comparison.

[0095] The method for processing gesture data in motion described in this embodiment is to test the dynamic gesture recognition algorithm in motion: collect data of jogging first, then raising the arm flat, then making a dynamic gesture, then raising the arm flat, and then jogging again, and use the algorithm to automatically segment the data. Take the X and Z axis gyroscope data as an example, as shown in Figure 2 and Figure 3. Figure 6 As shown in the figure, the green dotted line is the state of the arm being raised flat as segmented by the algorithm, and the red dotted line is the beginning and end of the real dynamic gesture segmented by the algorithm. It can be seen that the natural arm swing during running is not mistakenly recognized as a real dynamic gesture.

[0096] For the segmented dynamic gesture sequence, there will be a large amount of jitter noise generated during the movement. In this strong interference environment during movement, although the de-extreme average filtering algorithm has a good smoothing effect, it may not be able to retain the detailed information of the data sequence during movement because it removes the extreme points. Therefore, the five-point cubic arithmetic average smoothing filtering method mentioned above is used. After comparing with the data without any processing, it can be seen that the data has a good smoothing effect after the filtering algorithm and the original trend is maintained. Finally, the filtered data is sent to the model for recognition and the results are obtained.

[0097] Dynamic gesture recognition in motion. Since the noise of inertial sensors in motion is unavoidable, the conditions for limiting the motion state are as follows: the motion scene is a treadmill. The user needs to jog at a speed of 5 km / h or less for more than 1.5 seconds before making gestures. The user needs to swing the arms naturally during jogging. Before making gestures, the user needs to raise the arms to the chest and keep the arms as stable as possible for 0.05 seconds. Then, the user needs to make the same dynamic gesture twice. After the gestures are made, the user needs to keep the arms raised and stable for 0.05 seconds. The user needs to continue jogging for more than 1.5 seconds before making gestures. The algorithm proposed in the project is as follows: under limited motion conditions, the dynamic gesture data in motion is collected, and the dynamic gesture segmentation algorithm in motion based on arm motion state detection is used to segment the dynamic gestures. The segmented dynamic gestures are subjected to five-point cubic smoothing filtering to obtain smooth low-noise data, and the filtered data is then sent to the model for recognition.

[0098] In existing papers, the highest accuracy of dynamic gesture recognition in motion can only reach 70%, and there are many motion restrictions (walking, arms raised and placed flat on the chest). Through experiments, this implementation method collects dynamic gesture data of 5 people in motion under limited motion conditions, uses the dynamic gesture segmentation algorithm in motion based on arm motion state detection proposed in the project to segment the data, sends the segmented gesture sequence to the hierarchical large model for recognition, and compares the recognition results with the calibrated data. The recognition accuracy is shown in Table 1 below.

[0099] Table 1. Accuracy of dynamic gesture recognition in motion

[0100]

[0101] As shown in Table 1, under limited motion conditions, the average recognition accuracy of dynamic gestures reaches 83.32%, which is about 10% higher than the recognition accuracy of 70% of the existing algorithm. In addition, the proposed dynamic gesture recognition method in a motion environment does not need to collect a large amount of dynamic gesture data in motion like some existing studies, saving a lot of manpower and time costs. Therefore, the proposed dynamic gesture recognition method in a motion environment is advanced and superior, and meets the requirements of the research content.

[0102] Embodiment 2:

[0103] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of a method for processing gesture data in motion described in Example 1 when executing the computer program.

[0104] The computer device of the present invention may be a device including a processor and a memory, such as a single chip microcomputer including a central processing unit, etc. Moreover, the processor is used to implement the steps of the above-mentioned method for processing gesture data in motion when executing a computer program stored in the memory.

[0105] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf programmable gate arrays (FPGAs), or a processor.

[0106] Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0107] The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0108] Embodiment 3:

[0109] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements a method for processing gesture data in motion as described in Example 1.

[0110] The computer-readable storage medium of the present invention can be any form of storage medium that can be read by the processor of a computer device, including but not limited to non-volatile memory, volatile memory, ferroelectric memory, etc. A computer program is stored on the computer-readable storage medium. When the processor of the computer device reads and executes the computer program stored in the memory, the steps of the above-mentioned method for processing gesture data in motion can be implemented.

[0111] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer readable media do not include electric carrier signals and telecommunication signals.

[0112] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0113] Although the present application has been described above with reference to specific embodiments, various modifications may be made thereto and parts thereof may be replaced with equivalents without departing from the scope of the present application. In particular, as long as there is no structural conflict, the various features in the specific embodiments disclosed in the present application may be used in combination with each other in any manner, and the fact that these combinations are not exhaustively described in this specification is only for the sake of omitting space and saving resources. Therefore, the present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for processing gesture data in motion, characterized in that: The steps include: S1. Use smart data gloves to collect experimental data during exercise, including three-axis accelerometer data and three-axis gyroscope data during exercise; S2. Compare and analyze the three-axis accelerometer data and the three-axis gyroscope data collected during the movement in step S1, and extract gesture recognition data for the movement; S3. Construct an arm state segmentation algorithm module based on arm motion state detection, and input the gesture recognition data for motion obtained in step S2 into the arm state segmentation algorithm module based on arm motion state detection to obtain filtered dynamic gesture data; The specific implementation method of step S3 constructing an arm state segmentation algorithm module based on arm motion state detection includes the following steps: S3.

1. Use the energy function of the gyroscope signal to describe the fluctuation of the gyroscope signal. The expression of the energy of the gyroscope signal of the i-th frame is E i for: in, Indicates the jth component value of the gyroscope in the i-th frame; S3.

2. Set the gyroscope threshold Th gro , steady-state threshold Th static , motion window threshold Th sports_window ; S3.

3. Initialize the static fluctuation window and the dynamic fluctuation window to 10, and then use the gesture recognition data for motion obtained in step S2 to calculate the sliding window differential energy value Energy gro , the expression is: Among them, X i [k] is the k-th dimension data of the i-th frame, k is one of the 11 dimensions, and abs is the absolute value function; S3.

4. Determine whether it is an arm swing fluctuation by comparing the sliding window differential energy value obtained in step S3.3 with the gyroscope threshold: If the sliding window differential energy value is less than the gyroscope threshold, and if the stable state threshold is TRUE, the arm is currently in a stable state, and the current frame effectively enters the next step of the dynamic gesture segmentation algorithm module based on line segment fitting model comparison. The arm goes from swinging to stability, and the motion window is cleared; If the sliding window differential energy value is greater than or equal to the gyroscope threshold, the current state is the arm swing state, and the size relationship between the motion window and the motion window threshold is continued to be determined. If the motion window is smaller than the motion window threshold, it cannot be distinguished whether it is an arm swing or a dynamic gesture action. Assuming that the current frame is valid, the current frame is valid and enters the next step of the dynamic gesture segmentation algorithm module based on the line segment fitting model comparison, and the motion window is increased by 1; If the number of motion windows exceeds the motion window threshold, the motion state is identified as an invalid gesture of natural arm swing, the collected dynamic data corresponding to the invalid gesture is cleared, and the motion window is increased by 1; S4. Then, the filtered dynamic gesture data obtained in step S3 is input into the dynamic gesture segmentation algorithm module based on line segment fitting model comparison to perform gesture segmentation, and then data noise reduction is performed to obtain processed gesture data in motion; The specific implementation method of step S4 for constructing a dynamic gesture segmentation algorithm module based on line segment fitting model comparison includes the following steps: S4.

1. Set the gyroscope threshold Th gro , steady-state threshold Th static , similarity threshold Th Similarity , motion window threshold Th sports_window , the slope of the straight line when the gesture is raised flat K 抬平 ; S4.

2. Initialize the static fluctuation window and the dynamic fluctuation window to 10, and then use the filtered dynamic gesture data obtained in step S3 to calculate the sliding window differential energy value Energy gro Then, use the five frames of data to perform linear fitting in a time domain space to obtain the slope K of the current fitting line. i , the expression is: K i =(g i -g i-1 ) / (g i-1 -g i-2 )=(g i-2 -g i-3 ) / (g i-3 -g i-4 ) Among them, g i represents the total acceleration of the three axes x and zy of the i-th frame; S4.

3. Determine the slope K of the straight line when the gesture is lifted in step S4.1 抬平 state, if K 抬平 If the value of is not empty, the Euclidean distance between the straight line when the arm is raised and the current fitting straight line is calculated. The expression is: distance=(K 抬平 -K i ) 2 In order to increase the gap between different straight lines of linear fitting and make them easier to distinguish, distance' is used to represent the exponential power of the Euclidean distance between two states. The expression is: If K 抬平 The value of is empty, and Th static =TRUE, it is the first time that the arm is raised horizontally, let K 抬平 =K i ; If the sliding window differential energy value is less than the gyroscope threshold, and the Euclidean distance between the straight line when the arm is raised and the current fitting line is less than the similarity threshold, it is in a relatively stable state and the straight line when the arm is raised is similar to the current fitting line. Dynamic gesture segmentation information is added and recorded as the end of gesture segmentation; Otherwise, it means that the current gesture is dynamic, and the dynamic gesture segmentation information is added, recorded as the end of gesture segmentation, and then the gesture segmentation index is returned; S4.

4. Perform data denoising on the dynamic gesture data after gesture segmentation in step S4.3 to obtain processed gesture data in motion. The method of data denoising is to first filter the data using the five-point cubic filtering method, and then use arithmetic mean filtering for each point in the data, taking the average value of a certain number of sampling points around it as the output value of the point. The overall expression is: Among them, y(i) is the output value of the five-point cubic arithmetic mean filtering method of the original data of the i-th frame of the current calculation dimension, a i The dynamic gesture data of the i-th frame of the current calculation dimension.

2. The method for processing gesture data in motion according to claim 1, characterized in that: Step S1 collects experimental data during the exercise process. The exercise process is controlled to jog at a speed of 5 km / h for 1.5 seconds and then make a gesture. During the jogging process, the arms are naturally swung. Before making a gesture, the arms are raised flat to the chest, and the arms are kept stable for 0.05 seconds. Then, the same dynamic gesture is made twice. After finishing, the arms are kept raised flat and stable for 0.05 seconds, and then jogging for 1.5 seconds is continued before making a gesture.

3. The method for processing gesture data in motion according to claim 2, characterized in that: Step S2 plots the collected 6-axis sensor data of the state switching between natural arm swinging during movement, raising the arm horizontally during movement, and then to natural arm swinging during movement according to time and dimension, and performs analysis as follows: based on the influence of gravity, the movement data of the three-axis accelerometer data in different arm states is not much different, and the data features of natural arm swinging and raising the arm horizontally cannot be extracted, and the movement state of the arm cannot be distinguished. The data features of the three-axis gyroscope data switching between the arm states during movement are obvious, and the three-axis gyroscope data is extracted for gesture recognition data in movement.

4. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of a method for processing gesture data in motion as described in any one of claims 1 to 3 when executing the computer program.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for processing gesture data in motion as described in any one of claims 1 to 3 is implemented.

Images