A method and system for guiding device training operations based on action recognition

Abstract

This invention belongs to the field of image processing technology, specifically relating to a method and system for guiding device training operations based on action recognition. The method includes: collecting hand movement data of the trainee, including spatial coordinates and velocity at each moment; calculating the curvature at each moment; determining the hysteresis of the operation for any given moment by combining the curvature distribution entropy within its window with the proportion of low-speed movements; performing frequency domain analysis on the velocity within its window to obtain high-frequency energy; calculating the predicted spatial coordinates at any given moment; determining the jitter of the operation based on the Euclidean distance between the spatial coordinates at any given moment and the predicted spatial coordinates, as well as the high-frequency energy; determining a penalty coefficient based on the hysteresis and jitter of the operation, and calculating a weighted distance; and providing guidance to the trainee in response to the weighted distance exceeding a distance threshold. This invention solves the problem of prior art neglecting the quality of micro-movements, effectively identifying and guiding hesitation and tremor behaviors in precision operations.

Description

Technical Field

[0001] This invention relates to the field of image processing technology. More specifically, this invention relates to a method and system for guiding device training operations based on action recognition. Background Technology

[0002] In high-end equipment maintenance training in fields such as aerospace and precision manufacturing, motion capture-based simulation systems have become mainstream. Trainees practice disassembling and assembling precision components in a virtual environment using wearable devices to improve their operational proficiency. However, most existing training systems focus on assessing the compliance of operational procedures, neglecting the refined evaluation of the quality of the actions themselves. In precision operation scenarios, the stability and decisiveness of operations are crucial, and these microscopic bad habits can easily lead to component damage in actual operation. Therefore, how to identify and correct these high-risk microscopic actions has become a technical problem that current virtual training systems urgently need to solve.

[0003] To evaluate motion trajectories, existing technologies typically employ dynamic time warping algorithms, which assess the standardization of motion by calculating the geometric similarity between the trainee's actual trajectory and the standard expert trajectory.

[0004] However, in precision operation scenarios, dynamic time warping algorithms mainly focus on the matching degree of macroscopic paths and are not sensitive to microscopic dynamic features in the time dimension. For example, when a trainee experiences high-frequency micro-tremors in their hands due to nervousness, or hesitates repeatedly near the target point due to lack of proficiency, their final movement trajectory may closely match the standard trajectory. However, these actions involve extremely high operational risks, and the algorithm has difficulty effectively distinguishing between skilled and smooth movements and movements with micro-tremors or lag. This causes the system to fail to identify and warn of these critical non-standard actions, thus missing the opportunity for corrective guidance. Summary of the Invention

[0005] To address the aforementioned technical problem of insensitivity to micro-motion characteristics in precision operation assessment, which leads to the inability to effectively identify high-risk behaviors such as hysteresis and micro-tremors, the present invention provides solutions in the following aspects.

[0006] In a first aspect, the present invention provides a device training operation guidance method based on action recognition, comprising: acquiring a video of a trainee's operation, processing the video frame by frame to obtain hand motion data, including spatial coordinates and velocity at each moment; calculating the angle between two vectors formed by the spatial coordinates of each moment and its adjacent moments, as the curvature at each moment; determining the hysteresis of the operation based on the distribution entropy of the curvature of all moments within any given moment window and the proportion of moments with velocities less than a velocity threshold; performing frequency domain analysis on the velocities of all moments within any given moment window to obtain high-frequency energy; obtaining the predicted spatial coordinates at any given moment, wherein the predicted spatial coordinates are obtained by polynomial fitting of the spatial coordinates of all historical moments prior to that moment; determining the jitter of the operation based on the Euclidean distance between the spatial coordinates of any given moment and the predicted spatial coordinates, and the high-frequency energy; determining a penalty coefficient based on the hysteresis and jitter of the operation; using the product of the penalty coefficient and the Euclidean distance as a weighted distance; and providing guidance information to the trainee in response to a weighted distance greater than a distance threshold.

[0007] This invention utilizes curvature distribution entropy and low-speed proportion to assess operational lag, combines high-frequency energy and trajectory deviation to determine operational jitter, and constructs a dynamic penalty coefficient to weighted correct spatial distance. This mechanism can artificially amplify the trainee's lag, hesitation, or high-frequency micro-tremors, making microscopic defects that are not significant in physical space but extremely harmful in precision operations explicit. This not only sensitively identifies the trainee's deep-seated psychological uncertainty or physiological instability but also triggers timely guidance to avoid damage to precision components, significantly improving the safety and precision of training.

[0008] Preferably, the step of calculating the angle between two vectors formed by the spatial coordinates of each moment and its adjacent moments includes: for each moment, calculating the difference between the spatial coordinates of that moment and the moment before that moment to obtain a first vector; calculating the difference between the spatial coordinates of the moment after that moment and the moment before that moment to obtain a second vector; and calculating the angle between the two vectors.

[0009] This invention calculates curvature by using the angle between vectors at adjacent time points, which can capture subtle changes in the direction of hand movement, providing accurate geometric data for curvature entropy analysis and ensuring the accuracy of the analysis of the degree of disorder in the direction of movement.

[0010] Preferably, determining the hysteresis of the operation includes: treating the value of each curvature within the window at any given time as a category, and calculating the probability of each type of curvature value occurring; calculating the hysteresis of the operation at any given time. , In the formula, For the first The hand movement trajectory within the window at the moment of the moment The probability of a curvature value; It is a logarithmic function; For the first The index value and total number of curvature values ​​of each category on the hand movement trajectory within a window at each moment; For the first The number of times within a time window where the velocity is less than the velocity threshold; This represents the total number of moments within the window.

[0011] This invention combines curvature distribution entropy and low-speed proportion to construct a hysteresis model of operation, which comprehensively reflects the disorder of actions and unexpected pauses. Based on this, it assesses the psychological hesitation state of trainees when they lack confidence, distinguishes between normal slow operation and ineffective hesitant and tentative behavior, and thus achieves an effective assessment of trainees' cognitive proficiency.

[0012] Preferably, the step of performing frequency domain analysis on the velocity at all times within any time window to obtain high-frequency energy includes: performing a fast Fourier transform on the velocity at all times within any time window to obtain the amplitude of each frequency component, and calculating the sum of the squares of the amplitudes of each frequency component corresponding to the high-frequency band as the high-frequency energy; wherein, the high-frequency band is 5Hz-20Hz.

[0013] Preferably, the jitter of the operation satisfies the expression: In the formula, For the first The degree of jitter during operation at any given moment; For the first Spatial coordinates at time 1 Rather than predicting spatial coordinates The Euclidean distance; For the first High-frequency energy at a given moment; This represents the maximum value of high-frequency energy across all historical moments.

[0014] This invention combines trajectory deviation and high-frequency energy to construct a jitter model for operation. It uses energy values ​​to dynamically amplify spatial errors. Even if the physical distance deviation is very small, the accompanying micro-vibration will be identified as high risk. This effectively solves the problem that small-amplitude high-frequency oscillations are easily misjudged as normal operation, accurately identifies physiological jitter, and prevents precision components from being damaged by micro-vibration.

[0015] Preferably, determining the penalty coefficient includes: calculating the sum of the normalized value of the hysteresis of the operation and the normalized value of the jitter of the operation, and adding the sum of the normalized values ​​to 1 to obtain the penalty coefficient.

[0016] This invention utilizes normalized hysteresis and jitter to generate a penalty coefficient, transforming abstract action quality into a distance amplification factor. When hesitation or hand tremor risk is detected, the difference between the value and the standard path is artificially amplified, thereby improving the sensitivity to identify potential operational hazards.

[0017] Preferably, the distance threshold acquisition method includes: calculating the Euclidean distance between the spatial coordinates and the corresponding predicted spatial coordinates at each time point in the stable training trajectory, and using the minimum value of the distance as the distance threshold.

[0018] Preferably, providing guidance information to the trainee includes: when the normalized value of the hysteresis of the operation is greater than the normalized value of the jitter of the operation, sending the predicted spatial coordinates at the current moment as guidance information to the training device; otherwise, controlling the training device to generate pulse vibration to prompt the trainee to stabilize their hand.

[0019] Preferably, the method further includes using a data glove sensor to transmit hand movement data from the image in real time.

[0020] Secondly, the present invention provides a device training operation guidance system based on action recognition, including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the above-mentioned device training operation guidance method based on action recognition is implemented.

[0021] By adopting the above technical solution, a computer program is generated from the above-mentioned method for training and guiding devices based on action recognition, and stored in a memory so that it can be loaded and executed by a processor. In this way, a terminal device can be made based on the memory and the processor, which is convenient to use.

[0022] The beneficial effects of this invention are as follows:

[0023] (1) The present invention constructs a curvature entropy and high frequency energy model, which effectively identifies actions with high path fit but implicit risks of hesitation or micro-tremor, and provides data basis for subsequent action guidance;

[0024] (2) The present invention introduces a penalty coefficient, which transforms hysteresis and jitter into a distance amplification factor, breaking the limitation of single spatial distance assessment and enabling the capture of actions with potential operational hazards;

[0025] (3) By comparing the hysteresis and jitter of the normalized operation, the present invention locates the action category of the current operation hazard, and triggers visual guidance or tactile vibration accordingly, thereby realizing a closed-loop feedback from discovering micro defects to targeted correction of the cause. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating a device training operation guidance method based on action recognition according to the present invention;

[0027] Figure 2 It schematically illustrates the trainee's original operational trajectory;

[0028] Figure 3It schematically illustrates the trainee's operational trajectory after receiving guidance. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0031] This invention discloses a device training operation guidance method based on action recognition, referring to... Figure 1 This includes steps S1-S5:

[0032] S1. Collect video of the trainee's operation, process it frame by frame to obtain hand movement data, including the spatial coordinates and velocity at each moment.

[0033] It should be noted that precision operations involve not only positional movement but also posture adjustment, and the operations often occur on a time scale of milliseconds and a spatial scale of millimeters. In order to fully capture the details of the movements, video data is collected at high frequency through motion capture equipment. Since video data is essentially composed of continuous static images, each frame represents a moment in time. Acquiring the hand end-effector motion data in each frame is the basis for subsequent analysis of the continuity and stability of the movements.

[0034] Specifically, a high-speed industrial camera is set up in the training environment of the trainee and connected to a data glove sensor. The video stream of the trainee's operation process is collected at a preset frequency. The video stream is processed by frame segmentation. The data glove sensor is used to transmit the hand movement data in the image in real time, that is, the spatial coordinates and speed at each moment. In this embodiment of the invention, the sampling frequency is set to 100Hz, which can be adjusted by the implementer as needed.

[0035] At this point, the spatial coordinates and velocity at each moment have been obtained.

[0036] S2. Calculate the angle between the two vectors formed by the spatial coordinates of each time point and its adjacent time points, and use it as the curvature of each time point; determine the hysteresis of the operation based on the distribution entropy of the curvature of all time points within any time window and the proportion of time points where the velocity is less than the velocity threshold.

[0037] It should be noted that in precision equipment maintenance training, decisiveness of operation is the core indicator for measuring the trainee's cognitive proficiency. When the trainee has a vague memory of the target position or lacks confidence, their hand movements will degenerate from a definite approach to a tentative search movement. Their hand movements are usually characterized by constant fine-tuning of direction and intermittent speed. This lag characteristic is the key to judging cognitive proficiency. If it is judged only by the slow speed, it may misjudge normal alignment movements. Therefore, it is necessary to combine the changes in the geometric curvature of the trajectory and the proportion of invalid pauses to comprehensively assess the trainee's lag status.

[0038] Specifically, the curvature at each moment is obtained by: for each moment, calculating the difference between the spatial coordinates of that moment and the previous moment to obtain a first vector; calculating the difference between the spatial coordinates of the next moment and that moment to obtain a second vector; and calculating the angle between the two vectors as the curvature at that moment. The calculation of the angle between the vectors is a well-known technique and will not be elaborated here.

[0039] Set a length of The window, The preset length; where the preset length is... The time scale for recent motion change analysis is defined; the window length cannot be too short to avoid failing to capture recent motion changes. The value should be greater than 10. This invention will preset the length. The value is set to 20; it should be added that, for the initial motion data, due to insufficient historical data, it is impossible to obtain a window of length 20. Therefore, for the current moment... The situation where the window is incomplete, i.e. And the window start time When using the current time The data is used to complete the missing historical time data within the window; specifically, all data points with a time value less than 0 within the window are assigned the current time value. The motion data is used to ensure that the window length meets the preset requirements.

[0040] For any given moment, acquire the motion data and curvature data for all moments within its window. For example, the current moment is... Its time window refers to Interval.

[0041] Furthermore, the value of each curvature within the window at that moment is regarded as a category, and the probability of each type of curvature value occurring is calculated.

[0042] The hysteresis of the operation at that moment is determined based on the curvature distribution entropy of all moments within the window at that moment and the proportion of moments where the velocity is less than the velocity threshold; the hysteresis satisfies the expression:

[0043]

[0044] In the formula, For the first The lag in operation at any given moment; For the first The hand movement trajectory within the window at the moment of the moment The probability of a curvature value; It is a logarithmic function; For the first The index value and total number of curvature values ​​of each category on the hand movement trajectory within a window at each moment; For the first The number of times within a time window where the velocity is less than the velocity threshold; This represents the total number of moments within the window.

[0045] in, Reflecting the The curvature entropy of the hand movement trajectory within a window of time. The larger the value, the more disordered the curvature distribution of the local movement trajectory at that time. That is, the trainee frequently changes the direction of movement during the operation, which means that the trainee may be uncertain about the target position at that time and is making repeated ineffective attempts. Reflecting the The percentage of invalid pauses within a given time window indicates that the trainee has a large number of unexpected pauses during the operation, implying disjointed movements and a lack of confidence. If the curvature entropy of the hand movement trajectory within that time window is greater and the percentage of invalid pauses is higher, it indicates that the trainee may have a more severe psychological hesitation when operating at that time, meaning that the perceived lag in the operation at that time is greater. The speed threshold is the minimum speed required for the equipment training operation.

[0046] At this point, the lag of the operation at each moment has been obtained.

[0047] S3. Perform frequency domain analysis on the velocity at all times within any given time window to obtain high-frequency energy; obtain the predicted spatial coordinates at any given time; determine the jitter of the operation based on the Euclidean distance between the spatial coordinates at any given time and the predicted spatial coordinates, as well as the high-frequency energy.

[0048] It should be noted that in scenarios such as precision component docking or the tightening of tiny screws, even if trainees have a clear understanding of the operating path, they often experience high-frequency micro-tremors due to excessive tension or muscle fatigue. Unlike lag, the hand attempts to maintain the ideal path, but the actual trajectory oscillates rapidly and with small amplitudes around that path. Although the deviation from the target distance may be small, the extremely high frequency of these oscillations can cause scratches on the contact surfaces of components or breakage of pins. If only spatial distance is calculated, the small distance of these high-risk oscillations can easily be misjudged as normal high-precision operations. If only frequency is analyzed, normal rapid movements can be easily confused. Therefore, it is necessary to acquire the spatial distance to the predicted position and the high-frequency energy in the frequency domain in real time to assess the trainee's physiological jitter state.

[0049] Specifically, for any given moment, the least squares method is used to perform polynomial fitting on the spatial coordinates of all historical moments prior to that moment to obtain the predicted spatial coordinates for that moment.

[0050] Simultaneously, the velocity of all moments within the window at that moment is extracted, and a fast Fourier transform is performed to obtain the amplitude of each frequency component; the sum of squares of the amplitudes of each frequency component corresponding to the high-frequency band is calculated as the high-frequency energy at that moment; the high-frequency band is 5Hz-20Hz, which usually corresponds to physiological tremors or shaking caused by muscle tension in the human body; and the fast Fourier transform is a well-known technique, which will not be elaborated here.

[0051] Based on the Euclidean distance between the spatial coordinates at that moment and the predicted spatial coordinates, as well as the high-frequency energy, the jitter of the operation at that moment is determined; the jitter satisfies the expression:

[0052]

[0053] In the formula, For the first The degree of jitter during operation at any given moment; For the first Spatial coordinates at time 1 Rather than predicting spatial coordinates The Euclidean distance; For the first High-frequency energy at a given moment; This represents the maximum value of high-frequency energy across all historical moments.

[0054] in, Reflecting the The larger the deviation of the spatial coordinates at time t from the predicted spatial coordinates of the ideal fitted trajectory, the better. The greater the deviation of the hand movement from the expected trajectory at any given moment, the worse the spatial precision of the operation. Reflecting the The penalty factor for the deviation distance at time n, the larger the value, the more significant the penalty factor. At any given moment, hand movements may contain high-frequency shaking components; if the first... The greater the deviation of the hand movement trajectory at each moment and the more high-frequency oscillation components, the more it indicates that the trainee... The worse the hand stability at a given moment, the more easily it can damage precision components; that is, the greater the degree of tremor during the operation. It should be noted that when there is... When it is 0, let A value of 0 indicates jitter. equal .

[0055] At this point, the jitter level of the operation at each moment has been obtained.

[0056] S4. Determine the penalty coefficient based on the hysteresis and jitter of the operation; use the product of the penalty coefficient and the Euclidean distance as the weighted distance.

[0057] It should be noted that existing motion assessments often suffer from blind spots where small positional deviations result in poor motion quality, neglecting potential motion risks. To overcome this limitation of a single-dimensional assessment, a dynamic penalty coefficient is introduced, converting the acquired hysteresis and jitter into amplification factors of the original distance. When poor operating habits are detected, the difference between the trainee and the standard path is artificially amplified, making errors that are originally negligible in physical space significant in the assessment indicators. This allows the system to more sensitively capture micro-level operational defects and trigger timely guidance.

[0058] Specifically, based on the hysteresis and jitter of the operation at each time step, a penalty coefficient is determined for each time step. The penalty coefficient satisfies the following expression:

[0059]

[0060] In the formula, For the first The penalty coefficient at each moment; For the first The lag in operation at any given moment; For the first The degree of jitter during operation at any given moment; This is the standard normalization function.

[0061] If the trainee exhibits sluggishness or trembling, that is... or When it is larger, If the value is significantly greater than 1, even if the trainee is not far from the predicted target location in space, the final distance will increase sharply due to the penalty coefficient; conversely, when the trainee moves normally and smoothly, Approaching 1, maintaining the original distance.

[0062] Furthermore, the product of the penalty coefficient at each time step and the Euclidean distance between its spatial coordinates and its predicted spatial coordinates is calculated as the weighted distance at each time step.

[0063] At this point, the weighted distance at each moment has been obtained.

[0064] S5. In response to a weighted distance greater than a distance threshold, provide guidance information to the trainee.

[0065] It should be noted that, considering normal, risk-free, and skilled operation, the trainee's hand movements should be smooth and continuous. Under this ideal state, the sequence of actions performed by the trainee within this window should match the predicted path, without high-frequency oscillations or directional confusion. That is, there may be slight spatial errors, but the weighted matching distance will still remain at a low level. Conversely, if the weighted matching distance increases significantly, it indicates that the action has lost its original smoothness and deviated from the normal trajectory. Therefore, the system needs to use this as a boundary. When it detects that the action is no longer smooth, i.e., the weighted distance exceeds the standard, it should further analyze whether the main cause of the disruption of smoothness is hysteresis or hand tremor, so as to provide targeted correction.

[0066] Specifically, before guiding the operation, the Euclidean distance between the spatial coordinates and the corresponding predicted spatial coordinates at each moment in the training trajectory during stable training is calculated, and the minimum value of the distance is used as the distance threshold; if the weighted distance at each moment in the test training is less than the distance threshold... A multiple of 1 indicates that the trainee is currently in a normal state and does not require guidance; otherwise, a defect type judgment is performed: if This indicates that the anomaly at the current moment is mainly caused by hysteresis, meaning the trainee is uncertain about the target and sends the predicted spatial coordinates at the current moment as a guidance signal to the training device; if This indicates that the current anomaly is mainly caused by shaking, that is, the trainee's hand is unsteady. The system sends a tactile feedback control signal to the data glove, generating pulse vibrations to prompt the trainee to stabilize their hand.

[0067] For example, Figure 2 The trainee's original operating trajectory shows that after starting from the starting point, the trajectory experienced multiple abrupt changes in direction and local entanglement as the trainee approached the target point, indicating that the trainee was uncertain about the path and hesitated. At the same time, the trajectory line showed high-frequency sawtooth fluctuations in some areas, indicating that the trainee's hand stability was poor.

[0068] Figure 3 Compare the trainee's operational trajectory after receiving instruction. Figure 2After visual path guidance and tactile hand stabilization feedback were triggered based on hysteresis and jitter respectively, the smoothness of the trajectory was greatly improved, the path from the starting point to the end point was straighter, and the phenomenon of hesitation and knotting in the middle was eliminated; at the same time, high-frequency oscillation disappeared and the lines were smooth; this verifies that the guidance method proposed in this invention can effectively identify micro-movement defects and assist trainees in completing high-quality operation training.

[0069] This invention also discloses a device training operation guidance system based on action recognition, including a processor and a memory. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, a device training operation guidance method based on action recognition according to the present invention is implemented.

[0070] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.

Claims

1. A device training operation guidance method based on action recognition, characterized in that, include: Collect training videos of the trainees' actions, process them frame by frame to obtain hand movement data, including spatial coordinates and velocity at each moment; Calculate the angle between the two vectors formed by the spatial coordinates of each time step and its adjacent time steps, and use it as the curvature of each time step; determine the hysteresis of the operation based on the distribution entropy of the curvature of all time steps within any time window and the proportion of time steps with a velocity less than the velocity threshold. The determination of the hysteresis of the operation includes: treating the value of each curvature within the window at any given time as a category, and calculating the probability of each type of curvature value occurring; Calculate the hysteresis of the operation at any given time. , In the formula, For the first The hand movement trajectory within the window at the moment of the moment The probability of a curvature value; It is a logarithmic function; For the first The index value and total number of curvature values ​​of each category on the hand movement trajectory within a window at each moment; For the first The number of times within a time window where the velocity is less than the velocity threshold; This represents the total number of moments within the window. Frequency domain analysis is performed on the velocity at all times within any given time window to obtain high-frequency energy; the predicted spatial coordinates at any given time are obtained, which are obtained by polynomial fitting of the spatial coordinates of all historical times prior to that time. The jitter of the operation is determined based on the Euclidean distance between the spatial coordinates at any given time and the predicted spatial coordinates, as well as the high-frequency energy. A penalty coefficient is determined based on the hysteresis and jitter of the operation; the product of the penalty coefficient and the Euclidean distance is used as the weighted distance; in response to the weighted distance being greater than a distance threshold, guidance information is provided to the trainee; if the normalized value of the hysteresis of the operation is greater than the normalized value of the jitter of the operation, it indicates that the anomaly at the current moment is mainly caused by hysteresis, and the trainee is uncertain about the target, so the predicted spatial coordinates at the current moment are sent as a guidance signal to the training device; if the normalized value of the hysteresis of the operation is less than the normalized value of the jitter of the operation, it indicates that the current anomaly is mainly caused by jitter, and the trainee's hand is unsteady, so a tactile feedback control signal is sent to the data glove to generate pulse vibration, prompting the trainee to stabilize their hand.

2. The device training operation guidance method based on action recognition according to claim 1, characterized in that, The calculation of the angle between two vectors formed by the spatial coordinates of each moment and its adjacent moments includes: for each moment, calculating the difference between the spatial coordinates of that moment and the moment before that moment to obtain a first vector; calculating the difference between the spatial coordinates of the moment after that moment and the moment before that moment to obtain a second vector; and calculating the angle between the two vectors.

3. The device training operation guidance method based on action recognition according to claim 1, characterized in that, The step of performing frequency domain analysis on the velocity at all times within any given time window to obtain high-frequency energy includes: Perform a Fast Fourier Transform on the velocity at all times within any given time window to obtain the amplitude of each frequency component. Calculate the sum of the squares of the amplitudes of each frequency component corresponding to the high-frequency band as the high-frequency energy; where the high-frequency band ranges from 5Hz to 20Hz.

4. The device training operation guidance method based on action recognition according to claim 1, characterized in that, The jitter of the operation satisfies the expression: ； In the formula, For the first The degree of jitter during operation at any given moment; For the first Spatial coordinates at time 1 Rather than predicting spatial coordinates The Euclidean distance; For the first High-frequency energy at a given moment; This represents the maximum value of high-frequency energy across all historical moments.

5. The device training operation guidance method based on action recognition according to claim 1, characterized in that, The determination of the penalty coefficient includes: The sum of the normalized value of the operation's hysteresis and the normalized value of the operation's jitter is calculated, and the sum of the normalized values ​​is added to 1 to obtain the penalty coefficient.

6. The device training operation guidance method based on action recognition according to claim 1, characterized in that, The distance threshold acquisition method includes: calculating the Euclidean distance between the spatial coordinates and the corresponding predicted spatial coordinates at each moment in the stable training trajectory, and using the minimum value of the distance as the distance threshold.

7. The device training operation guidance method based on action recognition according to claim 1, characterized in that, The provision of guidance information to trainees includes: When the normalized value of the hysteresis of the operation is greater than the normalized value of the jitter of the operation, the predicted spatial coordinates at the current moment are sent to the training device as guidance information; otherwise, the training device is controlled to generate pulse vibration to prompt the trainee to stabilize their hand.

8. The device training operation guidance method based on action recognition according to claim 1, characterized in that, The method also includes using a data glove sensor to transmit hand movement data from images in real time.

9. A device training operation guidance system based on action recognition, characterized in that, include: A processor and a memory, wherein the memory stores computer program instructions that, when executed by the processor, implement a device training operation guidance method based on action recognition according to any one of claims 1-8.

Images