Sign language translation training system based on gesture tracking and sign language tracking device

Abstract

The invention relates to the technical field of sign language translation, in particular to a sign language translation training system based on gesture tracking and a sign language tracking device, and the system comprises a gesture tracking module, a skeleton structure completion module, a gesture standard comparison module, a gesture dynamic correction module and a training effect analysis module. According to the method, through screening visible skeleton joint points, detecting a space displacement trend and calculating a motion rate and a direction change rate, key frame data meeting a stability standard can be effectively extracted, the accuracy of gesture track analysis is improved, missing key points are complemented by using skeleton structure constraints, interference on an identification result caused by loss of the key points is avoided, and the accuracy of gesture track analysis is improved. Dynamic posture correction is performed by calculating the joint angle change rate and analyzing the deviation range and combining the finger opening and closing angle and the wrist rotation angle, adjustment suggestions are provided according to the correction angle, sign language training optimization guidance is achieved, and the pertinence and the refinement level of sign language training are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of sign language translation, and particularly to a sign language translation training system and a sign language tracking device based on gesture tracking. Background Art

[0002] The technical field of sign language translation involves technologies such as computer vision, gesture recognition, natural language processing, and human-computer interaction, aiming to convert sign language into text or speech to achieve barrier-free communication between deaf and hearing people. This technology relies on core links such as gesture detection, gesture classification, semantic parsing, and language modeling, and is widely used in sign language translation devices, education and training, intelligent interaction systems, and auxiliary communication platforms. The sign language translation system not only supports real-time translation but also combines context understanding to optimize translation accuracy, enabling sign language expressions to be recognized and understood more naturally.

[0003] A sign language translation training system based on gesture tracking is an auxiliary system specifically used to train an individual's sign language translation ability. Based on gesture tracking technology, this system can capture and analyze the user's gesture movements in real time, analyze whether the user's sign language movements conform to the standards, and provide immediate feedback. Through visual gesture comparison, translation accuracy evaluation, and real-time correction, it helps users improve the proficiency and accuracy of sign language translation, and is applicable to the training of sign language translators, the language learning of deaf people, and the teaching support of sign language education institutions.

[0004] Traditional training systems only rely on basic key point detection and trajectory analysis, and do not fully consider the problem of repairing key point loss due to bone topology. As a result, in the case of complex gesture transformations or occlusions, the lack of key points affects the recognition accuracy. The gesture matching calculation method is limited to static comparison and does not refine the time weight of key frames, resulting in a decrease in the accuracy of gesture dynamic matching and making it difficult to adapt to the natural and fluent characteristics of sign language expressions. The feedback during the training process mainly relies on overall matching and does not conduct a detailed analysis of the specific joint angle errors. It is difficult for trainers to accurately adjust their gesture movements, increasing the difficulty of standardizing gesture expressions. In the training evaluation link, traditional systems mainly rely on static error comparison and do not provide targeted suggestions in combination with the actual adjustment trends of trainers, affecting the efficiency and accuracy of gesture learning. Summary of the Invention

[0005] The purpose of the present invention is to solve the drawbacks existing in the prior art, and to propose a sign language translation training system based on gesture tracking.

[0006] To achieve the above purpose, the present invention adopts the following technical solution: A sign language translation training system based on gesture tracking, the system includes: The gesture tracking module obtains the three-dimensional coordinate data of the hand bone key points, detects the spatial displacement trend of the key points during the gesture movement, calculates the movement speed and the direction change rate of the gesture trajectory, screens the key frame data that meets the gesture stability standard, and obtains the bone topology feature set; Based on the bone topology feature set, the bone structure completion module analyzes the missing key points in the current frame, calculates the reasonable positions of the key points under the bone structure constraints, screens the candidate positions of the completion points, detects the spatial consistency and motion continuity of the candidate points, filters out the completion points that do not meet the constraint conditions, and establishes the completed key point coordinate set; Based on the completed key point coordinate set, the gesture standard comparison module calls the joint coordinate data of the current frame, extracts the standard gesture data according to the current sign language training content, calculates the gesture feature deviation value, and calculates the gesture matching confidence according to the feature deviation value of the key frame; Based on the gesture matching confidence, the gesture dynamic correction module analyzes the key joints with deviations exceeding the range, combines the finger opening and closing angles, the wrist rotation angle and the motion trend of adjacent frames, calculates the target angle correction value of the joints, and generates the dynamic posture correction parameters.

[0007] The improvement of the present invention is that the bone topology feature set includes the wrist spatial coordinates, the position of the palm center point, the coordinates of the five finger tips, the relative angles of the finger joints and the ratio of the lengths of adjacent joints. The completed key point coordinate set includes the spatial coordinates of the finger joint points, the position of the wrist key points, the bone connection relationship and the joint rotation completion angle. The gesture matching confidence specifically refers to the gesture feature deviation value, the key frame matching error, the time weight comparison score and the gesture standard similarity evaluation coefficient. The dynamic posture correction parameters include the corrected joint angle value, the target posture adjustment vector, the gesture adjustment reference benchmark and the correction step sequence.

[0008] The improvement of the present invention is that the gesture tracking module includes: The bone key point acquisition sub-module obtains the three-dimensional coordinate data of the hand bone key points, screens the visible bone joint points according to the hand topology structure, judges the visibility based on the coordinate values of each bone joint point in space, stores the three-dimensional coordinate data of the visible joint points, and generates the visible bone joint point coordinate set; Based on the visible bone joint point coordinate set, the spatial displacement trend calculation sub-module calculates the spatial displacement of the key points during the gesture movement, obtains the displacement vector of the key points in each frame, and uses the formula: ; and ; The operation obtains the rate of change of the movement speed, and filters out the frames with the rate of change of the speed fluctuating below the change threshold according to the gesture stability standard, and generates the key frame data set; Among them, represents the three-dimensional coordinates of the joint points of the th frame, is the motion rate of the th frame, is the change rate of the motion rate of the th frame, is the time interval between adjacent frames; The bone topology feature extraction sub-module calculates the change of joint angles between key points according to the key frame data set, obtains the change trend of frame joint angles, establishes a time series of joint angles, filters the bone topology structures with stability exceeding the stability threshold, obtains the set of joint points in the bone topology structure, and obtains the bone topology feature set.

[0009] The improvement of the present invention is that the bone structure completion module includes: The missing key point analysis sub-module analyzes the missing key points of the current frame based on the bone topology feature set, judges the existence state of each joint point in the bone topology structure, locates the position of the missing joint point in the bone chain according to the topological relationship, calls the relative position relationship of known adjacent key points, obtains the three-dimensional coordinate set of adjacent key points, and generates the adjacent coordinate set of missing key points; The completion point position calculation sub-module calls the bone length ratio of the joint according to the adjacent coordinate set of missing key points, and calculates the reasonable position of the missing key point under the constraint of the bone structure according to the relative position relationship of adjacent key points, using the formula: ; Performs operations to obtain the set of candidate positions of completion points; Among them, is the three-dimensional coordinate of the completion point, is the three-dimensional coordinate of the adjacent known key point, is the three-dimensional coordinate of another known key point, is the bone length between the completion point and the adjacent key point, is the bone length ratio between known key points; The completion key point screening sub-module detects the spatial consistency and motion continuity of candidate points based on the set of candidate positions of completion points, calculates the spatial position deviation and motion trajectory deviation of candidate points between adjacent frames, screens out candidate points with spatial deviation and trajectory deviation exceeding the deviation threshold, obtains the completion points meeting the bone structure constraint conditions, and obtains the coordinate set of completion key points.

[0010] The improvement of the present invention is that the gesture standard comparison module includes: The gesture feature extraction sub-module, based on the completed key point coordinate set, calls the joint coordinate data of the current frame, extracts the finger bending angle, wrist rotation angle, and the spatial positions of key joints, obtains the standard gesture data according to the current sign language training content, and extracts the standard gesture features corresponding to the joints, including the finger bending angle, wrist rotation angle, and spatial position, to obtain the current frame gesture feature set and the standard gesture feature set; The feature deviation calculation sub-module calculates the deviation value of each feature item according to the current frame gesture feature set and the standard gesture feature set, using the formula: ; Performs operations to obtain the gesture feature deviation value set; where, is the total gesture feature deviation value, is the th feature value in the current frame, is the th feature value in the standard gesture, is the weight of the th feature, is the total number of features; The gesture matching confidence calculation sub-module screens the key frames of the gesture according to the gesture feature deviation value set, and extracts the time weights of the key frames. Based on the feature deviation values and time weights of the key frames, it uses the formula: ; Performs operations to obtain the gesture matching confidence; where, is the gesture matching confidence, is the total feature deviation value of the th key frame, is the time weight of the th key frame, is the total number of standard key frames.

[0011] The improvement of the present invention is that the gesture dynamic correction module includes: The joint angle change calculation sub-module, based on the gesture matching confidence, extracts the deviation degree of the current gesture posture, calculates the angle change rate of each joint, calculates the joint angle change between adjacent frames according to the finger bending angle, wrist rotation angle, and the spatial positions of key joints, and arranges the angle change rates of each joint to obtain the joint angle change rate data set; The key joint deviation judgment sub-module analyzes the key joints with deviations exceeding the range according to the joint angle change rate data set, compares the angle change rate of each joint with the set angle change rate threshold, screens the key joints with change rates exceeding the threshold, and generates the key joint deviation set; The dynamic posture correction parameter generation sub-module calculates the target angle correction value for each joint according to the set of key joint deviations, in combination with the finger opening / closing angle, the wrist rotation angle, and the motion trend of adjacent frames, using the formula: ; Performs operations to obtain a set of target angle correction values and generates dynamic posture correction parameters; where, is the correction angle of the target joint, is the current joint angle, is the reference angle of the joint in the adjacent frame , is the time weight of the adjacent frame , is the total number of adjacent frames, is the deviation sign indicator.

[0012] An improvement of the present invention is that the system further includes a training effect analysis module; The training effect analysis module calls the dynamic posture correction parameters to remind the trainer, detects the current gesture adjustment trend of the trainer, analyzes whether the gesture correction path conforms to the target correction angle, determines whether additional posture adjustment guidance is required, outputs adjustment suggestions for the training process, and obtains sign language training adjustment suggestions; The sign language training adjustment suggestions are specifically gesture correction prompts, adjustment path guidance, posture matching degree analysis, and training progress feedback.

[0013] An improvement of the present invention is that the training effect analysis module includes: The gesture adjustment trend detection sub-module calls the dynamic posture correction parameters to remind the current gesture of the trainer, detects the change in the joint angle of the trainer after posture adjustment, calculates the angle change rate and spatial displacement between adjacent frames, and obtains a gesture adjustment trend data set according to the gesture correction direction and speed; The correction path compliance judgment sub-module analyzes whether the gesture correction path conforms to the target correction angle according to the gesture adjustment trend data set, compares the current gesture adjustment trend with the target correction path, calculates the angle deviation and displacement error, using the formula: ; Performs operations to obtain the correction path error value, determines whether the correction path error value is within the set threshold range, and screens the key joints that require additional adjustment to obtain a set of additional posture adjustment requirements; where, is the correction path error value, is the th joint angle in the current frame, is the target correction angle, is the weight of the th joint, is the total number of joints; The training adjustment suggestion generation sub-module determines whether additional gesture adjustment guidelines are needed according to the set of additional pose adjustment requirements, combines the current gesture adjustment trend, outputs the adjustment direction and amplitude of each joint, and generates sign language training adjustment suggestions.

[0014] A sign language tracking device includes a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, it implements the sign language translation training system according to gesture tracking as described above.

[0015] Compared with the prior art, the advantages and positive effects of the present invention are as follows: In the present invention, by screening visible skeletal joint points, detecting the spatial displacement trend, calculating the movement speed and the direction change rate, key frame data meeting the stability standard can be effectively extracted, improving the accuracy of gesture trajectory analysis. Using the skeletal structure constraint to complement missing key points, combining the skeletal length ratio and the relative position of adjacent key points to ensure the spatial consistency and movement continuity of the complemented points, avoiding interference with the recognition result caused by the loss of key points. For the gesture training process, calculating the gesture feature deviation value and screening key frames in combination with time weights, making the matching calculation more conform to the gesture semantic features and optimizing the accuracy of the matching confidence calculation. For the error in the training process, by calculating the joint angle change rate, analyzing the deviation range, and combining the finger opening and closing angle and the wrist rotation angle for dynamic posture correction, enabling the trainer to accurately adjust the gesture action, improving the standardization degree of gesture expression, combining the posture adjustment trend analysis to correct the path, and providing adjustment suggestions based on the correction angle to achieve sign language training optimization guidance and improve the pertinence and refinement level of sign language training. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is the system flowchart of the present invention; Figure 2 is the flowchart of the gesture tracking module of the present invention; Figure 3 is the flowchart of the skeletal structure complementation module of the present invention; Figure 4 is the flowchart of the gesture standard comparison module of the present invention; Figure 5 is the flowchart of the gesture dynamic correction module of the present invention; Figure 6 is the flowchart of the training effect analysis module of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] In order to make the objectives, technical solutions and advantages of the present invention more clear and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0018] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the accompanying drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. In addition, in the description of the present invention, the meaning of "a plurality of" is two or more unless otherwise specifically defined.

[0019] Please refer to Figure 1 , the present invention provides a technical solution: a sign language translation training system based on gesture tracking, the system includes: The gesture tracking module obtains the three-dimensional coordinate data of the hand bone key points, filters the visible bone joint points based on the hand topology structure, detects the spatial displacement trend of the key points during the gesture movement process, calculates the movement speed and direction change speed of the gesture trajectory, filters the key frame data that meets the gesture stability standard, extracts the joint angle change trend, and obtains the bone topology feature set; The bone structure completion module analyzes the missing key points in the current frame based on the bone topology feature set, calls the bone length ratio of the joints, calculates the reasonable positions of the key points under the bone structure constraint according to the relative position relationship of the known adjacent key points, filters the candidate positions of the completion points, detects the spatial consistency and motion continuity of the candidate points, screens out the completion points that do not meet the constraint conditions, and establishes the completed key point coordinate set; The gesture standard comparison module, based on the completed key point coordinate set, calls the joint coordinate data of the current frame, including extracting the finger bending angle, wrist rotation angle and the spatial position of the key joints, extracts the standard gesture data according to the current sign language training content, calculates the gesture feature deviation value, screens the gesture key frames and extracts the time weights, and calculates the gesture matching confidence according to the feature deviation value of the key frames; The gesture dynamic correction module, based on the gesture matching confidence, extracts the deviation degree of the current gesture posture, calculates the change rate of each joint angle, analyzes the key joints whose deviation exceeds the range, and combines the finger opening and closing angle, wrist rotation angle and the motion trend of adjacent frames to calculate the target angle correction value of the joint, and generates the dynamic posture correction parameter; The training effect analysis module calls dynamic gesture correction parameters to remind the trainer, detects the current gesture adjustment trend of the trainer, analyzes whether the gesture correction path conforms to the target correction angle, determines whether additional posture adjustment guidance is needed, outputs adjustment suggestions for the training process, and obtains sign language training adjustment suggestions; The skeletal topology feature set includes wrist spatial coordinates, palm center point position, five-finger fingertip coordinates, relative finger joint angles, and adjacent joint length ratios. The complete key point coordinate set includes finger joint point spatial coordinates, wrist key point position, bone connection relationship, and joint rotation completion angle. The gesture matching confidence specifically refers to the gesture feature deviation value, key frame matching error, time weight comparison score, and gesture standard similarity evaluation coefficient. The dynamic gesture correction parameters include corrected joint angle values, target posture adjustment vectors, gesture adjustment reference benchmarks, and correction step sequences. The sign language training adjustment suggestions are specifically gesture correction prompts, adjustment path guidance, posture matching degree analysis, and training progress feedback; Please refer to Figure 2 , the gesture tracking module includes: The skeletal key point acquisition sub-module acquires three-dimensional coordinate data of the hand's skeletal key points, filters visible skeletal joint points according to the hand's topology structure, determines the visibility based on the coordinate values of each skeletal joint point in space, stores the three-dimensional coordinate data of the visible joint points, and generates a set of visible skeletal joint point coordinates; The skeletal key point acquisition sub-module acquires three-dimensional coordinate data of the hand's skeletal key points. By calling a depth camera and computer vision algorithms, it first collects multiple frame image data of the hand and performs three-dimensional reconstruction on each frame of the image to obtain the spatial coordinates of each skeletal joint point. During the specific execution process, according to the hand's topology structure, visible skeletal joint points are filtered. This filtering process calculates the angle between each joint point and the camera and the occlusion relationship. If the angle is greater than the set threshold or is occluded by other parts, it is judged as invisible and removed from the joint point set. The three-dimensional coordinate data of the remaining visible joint points is stored to form a set of visible skeletal joint point coordinates. Among them, the threshold is set based on the fact that in the normal activity range of human hand joint points, the angle with the camera is usually between and . When the angle exceeds , the joint point is extremely likely to be occluded by the hand, and it is found in the experiment that after exceeding , the three-dimensional reconstruction error increases significantly. Therefore, is selected as the threshold to ensure the visibility of the joint point in the field of view and the reconstruction accuracy. For example, when the hand extends perpendicular to the camera direction, since the palm occludes the finger roots, the root joint points are removed, while the fingertip joint points are retained because they are exposed in the camera's field of view, as shown in Table 1.

[0020] Example of the visible skeletal joint point coordinate set in Table 1 As shown in Table 1, the three-dimensional coordinates of the fingertip joint points are retained after screening, and the invisible joint points are removed and stored to generate the visible skeletal joint point coordinate set.

[0021] Based on the visible skeletal joint point coordinate set, the spatial displacement trend calculation sub-module calculates the spatial displacement of the key points during the gesture movement, obtains the displacement vector of the key points in each frame, and uses the formula: ; and ; Performs operations to obtain the rate of change of the movement rate, screens the frames with the rate of change of the rate fluctuating below the change threshold according to the gesture stability standard, and generates the key frame data set; Among them, represents the three-dimensional coordinates of the joint point in the th frame, is the movement rate in the th frame, is the rate of change of the movement rate in the th frame, is the time interval between adjacent frames; Based on the visible skeletal joint point coordinate set, the spatial displacement trend calculation sub-module calculates the spatial displacement of the key points during the gesture movement, obtains the displacement vector of the key points in each frame by taking the difference of the three-dimensional coordinates of the joint points in each frame , and combines with the time interval to calculate the movement rate . For example, for the time interval between frames seconds (30 frames per second), if the change amount of the three-dimensional coordinates of a certain joint point is mm, then its movement rate is mm / s. Further calculate the rate of change of the movement rate . If the movement rate of the previous frame is mm / s, then mm / s2. Among them, the rate of change threshold The setting of 100 mm / s2 is based on the statistical analysis of the rate of change of the hand's smooth movements (such as slight fingertip movements) and violent movements (such as rapid waving). It is found that the rate of change of smooth movements is usually lower than 100 mm / s2, while violent movements exceed 150 mm / s2. Therefore, 100 mm / s2 is set as the threshold to distinguish between stable and unstable states, so as to accurately identify key frames and ensure the reliability of the key frame data set. According to the gesture stability standard, frames with a rate of change fluctuation lower than the set threshold are screened to generate the key frame data set, as shown in Table 2.

[0022] Table 2 Example of Key Frame Data Set As shown in Table 2, frames with a rate of change lower than the threshold are screened to form the key frame data set.

[0023] The bone topology feature extraction sub-module calculates the change in joint angles between key points according to the key frame data set, obtains the change trend of the frame joint angles, establishes a time series of joint angles, screens the bone topologies with stability exceeding the stability threshold, obtains the set of joint points in the bone topology, and gets the bone topology feature set; The bone topology feature extraction sub-module calculates the change in joint angles between key points according to the key frame data set. In the specific process, based on the bone vectors formed by adjacent joint points, the joint angles are calculated through the vector included angle formula , taking finger joints as an example, assuming that bone vectors are formed by joint points 1 and 2, and joint points 2 and 3 respectively and , then the joint angle is . Calculate the change in joint angles for each frame and establish a time series, and screen the bone topologies with stability exceeding the stable threshold . Among them, the setting of the stable threshold is based on the fact that through statistical data, it can be found that during normal gesture transformations, when the joint angle fluctuation is lower than , the continuity and stability of the gesture movements are relatively high, while when it exceeds , there are obvious fluctuations. Therefore, is set as the stable threshold to ensure the stability and coherence of the topology, obtain the set of stable joint points, and get the bone topology feature set, as shown in Table 3.

[0024] Table 3 Example of Bone Topology Feature Set As shown in Table 3, the bone topologies with joint angle stability exceeding the set threshold are screened and stored to form the bone topology feature set.

[0025] Please refer to Figure 3 , the bone structure completion module includes: The missing key point analysis sub-module analyzes the missing key points in the current frame based on the bone topology feature set, determines the existence status of each joint point in the bone topology structure, locates the position of the missing joint point in the bone chain according to the topological relationship, calls the relative position relationship of known adjacent key points, obtains the three-dimensional coordinate set of adjacent key points, and generates the adjacent coordinate set of missing key points; The missing key point analysis sub-module analyzes the missing key points in the current frame based on the bone topology feature set. First, it obtains the bone joint point coordinate set of the current frame and compares it one by one with all the joint points defined in the topology feature set to determine the existence status of each joint point. By comparing whether the three-dimensional coordinates of the joint point exist, if the three-dimensional coordinates of a certain joint point are empty, it is determined that the joint point is in a missing state. Taking the hand bone as an example, if the fingertip joint point of the little finger is not detected in the current frame, it is determined that it is a missing joint point. Further, according to the topological relationship, the position of the missing joint point in the bone chain is located, and the predecessor joint point and successor joint point of the missing joint point are determined through the upstream and downstream relationship of the joint points. For example, if the fingertip joint point of the little finger is missing, the predecessor joint point is the middle phalanx joint point of the little finger, and the successor joint point is empty (there is no successor node at the fingertip). Then, it calls the relative position relationship of known adjacent key points to obtain the three-dimensional coordinates of the middle phalanx joint point of the little finger, which is the predecessor joint point and the three-dimensional coordinates of the root joint point of the little finger , and generates the adjacent coordinate set of missing key points, as shown in Table 4.

[0026] Table 4 Example of adjacent coordinate set of missing key points As shown in Table 4, the three-dimensional coordinates of the middle phalanx and root joint points adjacent to the missing fingertip joint point of the little finger are extracted and stored as the adjacent coordinate set of missing key points.

[0027] The completion point position calculation sub-module calculates the reasonable position of the missing key point under the bone structure constraint according to the adjacent coordinate set of missing key points, calls the bone length ratio of the joint, and based on the relative position relationship of adjacent key points, using the formula: ; Performs operations to obtain the candidate position set of completion points; wherein, is the three-dimensional coordinate of the completion point, is the three-dimensional coordinate of the adjacent known key point, is the three-dimensional coordinate of another known key point, is the bone length between the completion point and the adjacent key point, is the bone length ratio between the known key points; The missing key point position calculation sub-module calls the bone length ratio of the joint according to the set of adjacent coordinates of the missing key points , and calculates the reasonable position of the missing key point under the constraint of the bone structure according to the relative position relationship of the adjacent key points. In the specific execution process, first calculate the bone length between the known adjacent key points , for example, the distance between the middle joint point and the root joint point of the little finger is mm, and call the pre-set bone length ratio , this ratio is set according to the hand anatomy data. The length from the tip of the little finger to the middle section accounts for about 75% of the length from the middle section to the root. This value is derived from the actual measured hand joint ratio data and fluctuates with the hand size. Usually, this ratio fluctuates between 0.7 and 0.8 for adult hands. Here, 0.75 is taken as the middle value to ensure applicability and reasonableness, and calculate the distance between the missing key point and the middle joint point mm, then, according to the formula , calculate the three-dimensional coordinates of the missing key point, where the adjacent known key points and another known key point , substitute into the calculation to get , and finally obtain the three-dimensional coordinates of the complement point , generate a set of candidate positions for the complement point, as shown in Table 5

[0028] Table 5 Example of the set of candidate positions for the complement point As shown in Table 5, according to the relative position of the adjacent joint points and the bone length ratio, calculate the three-dimensional coordinates of the complement point, and generate a set of candidate positions for the complement point

[0029] The missing key point screening sub-module is based on the set of candidate positions for the complement point, detects the spatial consistency and motion continuity of the candidate points, calculates the spatial position deviation and motion trajectory deviation of the candidate points between adjacent frames, screens out the candidate points whose spatial deviation and trajectory deviation exceed the deviation threshold, obtains the complement points that meet the bone structure constraint conditions, and gets the set of coordinates of the missing key points The missing key point screening sub-module is based on the set of candidate positions for the complement point, detects the spatial consistency and motion continuity of the candidate points. First, judge its spatial consistency by calculating the spatial position deviation between the candidate point and the corresponding joint point in the previous frame, specifically, calculate the modulus of the three-dimensional coordinate difference , where and the coordinate of this joint point in the previous frame , then the spatial deviation mm, the spatial deviation threshold The value of mm is set according to the normal fluctuation range of joint points during the gesture movement. This value refers to the tolerance range of noise interference in the field of gesture recognition and is obtained through verification with multiple sets of experimental data. This threshold usually fluctuates between 0.8 and 1.2 mm. In this case, it is set to 1.0 mm to balance accuracy and stability. If is less than the deviation threshold mm, it is determined that the spatial consistency is qualified; otherwise, it is excluded. Next, calculate the motion trajectory deviation of the candidate points by comparing the motion vector differences of the candidate points in adjacent frames , where mm / s and mm / s, then mm / s, the motion trajectory deviation threshold mm / s is set according to the acceleration fluctuation range of normal gesture movements. Through comparative analysis of normal gesture trajectories and abnormal jitter data, it is set to 0.5 mm / s to effectively filter out abnormal motion points. Finally, filter out the candidate points whose spatial deviation and trajectory deviation exceed the threshold, obtain the complement points that meet the bone structure constraint conditions, and get the complement key point coordinate set to complete the complement of the missing key points.

[0030] Please refer to Figure 4 , the gesture standard comparison module includes: Based on the complement key point coordinate set, the gesture feature extraction sub-module calls the joint coordinate data of the current frame, extracts the finger bending angles, wrist rotation angles, and spatial positions of key joints, and obtains the standard gesture data according to the current sign language training content. Extract the standard gesture features corresponding to the joints, including finger bending angles, wrist rotation angles, and spatial positions, to obtain the current frame gesture feature set and the standard gesture feature set; Based on the complement key point coordinate set, the gesture feature extraction sub-module calls the joint coordinate data of the current frame, analyzes the three-dimensional coordinates of each joint point in each frame, calculates the bending angles of each finger, the rotation angle of the wrist, and the spatial positions of key joint points respectively. The calculation of the finger bending angle is based on the joint vectors formed by adjacent joint points. Through the vector included angle formula obtain the bending angles of each joint. For example, for the vector and formed between the second joint (proximal phalanx) and the third joint (middle phalanx) of the index finger, if and , then the bending angle is . Similarly, calculate the bending angles of each joint of each finger and store them as a bending angle array. The wrist rotation angle is obtained by calculating the included angle of the plane vector formed by the wrist joint and the arm joint. For example, if the vector between the wrist joint and the elbow joint is , and the vector with the arm joint point is , the wrist rotation angle is , the spatial positions of the key joint points are recorded in the form of three-dimensional coordinates, as shown in Table 6. Subsequently, according to the current sign language training content, the standard gesture data is obtained, the standard gesture database is called, and the standard gesture features corresponding to the joints are extracted, including the finger bending angle, the wrist rotation angle, and the spatial position. The gesture features extracted in the current frame are stored corresponding to the standard gesture features to obtain the current frame gesture feature set and the standard gesture feature set.

[0031] Table 6 Example of gesture feature set As shown in Table 6, the bending angles and spatial positions of the joint points in the current frame are extracted and stored as a gesture feature set, and are corresponding to the standard gesture feature set.

[0032] The feature deviation calculation sub-module calculates the deviation value of each feature item according to the current frame gesture feature set and the standard gesture feature set, using the formula: ; The operation obtains the gesture feature deviation value set; Among them, is the total gesture feature deviation value, is the th feature value in the current frame, is the th feature value in the standard gesture, is the weight of the th feature, is the total number of features; The feature deviation calculation sub-module calculates the deviation value of each feature item according to the current frame gesture feature set and the standard gesture feature set. First, each feature value in the current frame gesture feature set is compared one by one with the corresponding standard gesture feature value , and the square of the difference between the two is calculated. Then, according to the importance of each feature item, the corresponding weight is called and weighted to obtain the weighted deviation . Among them, the weight is set according to the importance of each feature item in gesture recognition, and is determined by analyzing a large number of sign language actions. The finger bending angle has a greater impact on gesture classification, so a higher weight is set, while the wrist rotation angle has a relatively smaller impact on gesture classification, and a lower weight is set. This setting range is fine-tuned according to the recognition accuracy of different sign language letters or words to ensure the rationality of the weight setting. The specific adjustment basis is that the error fluctuation range does not exceed 0.05. For example, the bending angle of the second joint of the index finger in the current frame is , while the bending angle of the corresponding joint in the standard gesture is , and the bending angle deviation is . Similarly, the weighted deviations of all feature items are calculated and accumulated, and finally the total gesture feature deviation value is obtained . As shown in Table 7, the deviation value sets of all feature items are calculated and stored.

[0033] Table 7 Example of Gesture Feature Deviation Value Set As shown in Table 7, the deviation values of each feature item in the current frame are calculated and weighted and stored, and finally the total gesture feature deviation value is obtained.

[0034] The gesture matching confidence calculation sub-module screens the key frames of the gesture according to the gesture feature deviation value set, extracts the time weights of the key frames, and based on the feature deviation values and time weights of the key frames, uses the formula: ; Performs operations to obtain the gesture matching confidence; where, is the gesture matching confidence, is the total feature deviation value of the th key frame, is the time weight of the th key frame, is the total number of standard key frames; The gesture matching confidence calculation sub-module screens the key frames of the gesture according to the gesture feature deviation value set, extracts the time weights of the key frames. First, based on the key nodes of the gesture action, the frames with significant changes in the motion trajectory or low feature deviation are screened out as key frames. For example, if the deviation values of all feature items in a certain frame are less than the set threshold , then this frame is marked as a key frame. Then, the time weight corresponding to each key frame is called, and the gesture matching confidence is calculated according to the formula . Taking three key frames in the current gesture as an example, their total feature deviation values are respectively , , , and the corresponding time weights are respectively , , . The time weight is set as the relative timing weight of this frame in the entire gesture action to ensure the consistency of the connection before and after the gesture action. Usually, it is between 0.2 and 0.4, and the specific value is set according to the amplitude of the motion change between frames. Then the gesture matching confidence is , the calculation results show that the matching confidence of the current gesture and the standard gesture is 67%, and finally the gesture matching confidence is obtained and stored.

[0035] Please refer to Figure 5 , the gesture dynamic correction module includes: Based on the gesture matching confidence, the joint angle change calculation sub-module extracts the deviation degree of the current gesture posture, calculates the angle change rate of each joint, calculates the joint angle change between adjacent frames according to the finger bending angle, wrist rotation angle and the spatial position of key joints, and arranges the angle change rate of each joint to obtain the joint angle change rate data set; Based on the gesture matching confidence, the joint angle change calculation sub-module extracts the deviation degree of the current gesture posture. First, it obtains the gesture feature sets of the current frame and the adjacent previous frame, including the bending angles of each joint point, the wrist rotation angle, and the three-dimensional spatial positions of the key joint points. By comparing the joint angles between adjacent frames one by one, it calculates the angle change rate of each joint. In the specific calculation process, based on the vector change between joint points, it obtains the angle change of each joint. For example, for the angle between the second joint and the third joint of the index finger , in the current frame, this angle is , in the previous frame it is , then the angle change of this joint , and the angle change rate is , where the time interval between frames seconds (30 frames per second). Similarly, calculate the angle change rate of each joint, arrange it into an angle change rate array, and then store it as the joint angle change rate data set. As shown in Table 8, the angle change rate threshold is set to , this value is set according to the upper limit of the angle change rate of the human finger and wrist during normal gesture changes, referring to the maximum angle change rate of normal gesture transformations in multiple gesture experiments, and according to the fluctuation range of the gesture operation speed in the experiments, finally select as the threshold to ensure that it can capture faster gesture transformations and will not misjudge natural fluctuations as abnormalities. At the same time, this value fluctuates with the change of gesture types (such as static gestures and dynamic gestures) and operation speeds (such as normal gestures and fast gestures), usually between and to adapt to the gesture recognition requirements of different scenarios, and finally obtain the angle change rate data set of each joint.

[0036] Table 8 Example of joint angle change rate data set As shown in Table 8, calculate the angle change and its change rate of each joint point between adjacent frames, and store it as the joint angle change rate data set.

[0037] The key joint deviation judgment sub-module analyzes the key joints with deviations exceeding the range according to the joint angle change rate data set, compares the angle change rate of each joint with the set angle change rate threshold, screens out the key joints with change rates exceeding the threshold, and generates a key joint deviation set; The key joint deviation judgment sub-module analyzes the key joints with deviations exceeding the range according to the joint angle change rate data set. First, it obtains the angle change rate of each joint and compares it with the set angle change rate threshold. The set angle change rate threshold is determined according to the gesture stability requirement. Taking the second joint of the index finger as an example, if its angle change rate is while the set angle change rate threshold is then , it is judged to exceed the threshold. Similarly, the angle change rates of all joints are compared one by one. If the angle change rate of a certain joint exceeds the threshold, the joint point is marked as a key joint with deviation exceeding the range, and the joint point number, angle change rate, and threshold are stored to generate a key joint deviation set, as shown in Table 9.

[0038] Table 9 Example of key joint deviation set As shown in Table 9, the angle change rate of the second joint of the index finger exceeds the threshold, so it is marked as a key joint with deviation exceeding the range and is stored in the key joint deviation set.

[0039] The dynamic posture correction parameter generation sub-module calculates the target angle correction value of each joint according to the key joint deviation set, combined with the finger opening and closing angle, wrist rotation angle, and motion trend of adjacent frames, using the formula: ; Calculates to obtain a set of target angle correction values and generates dynamic posture correction parameters; Among them, is the correction angle of the target joint, is the current joint angle, is the reference angle of the joint in the adjacent frame , is the time weight of the adjacent frame , is the total number of adjacent frames, is the deviation sign indicator, -1 indicates decreasing the angle, and 1 indicates increasing the angle; The dynamic posture correction parameter generation sub-module calculates the target angle correction value of each joint according to the key joint deviation set, combined with the finger opening and closing angle, wrist rotation angle, and motion trend of adjacent frames. First, it calls the joint points marked as exceeding the threshold in the key joint deviation set and obtains their current angles and the reference angles of adjacent frames , then, calculate the average deviation of the joint angles in adjacent frames. Taking the second joint of the index finger in the current frame as an example, assume the reference angles in three adjacent frames are respectively , , , and assume the time weights are , , . The time weights are set according to the temporal proximity of each frame to the current frame. The closer the distance, the greater the weight, to reflect the influence of time proximity on pose correction. The time weight values are adjusted according to the interval between the front and rear frames and the gesture speed, and are usually set between and , and the sum is 1. Then, calculate the target angle correction value according to the formula , where and (decrease the angle), then , and finally obtain the target angle correction value , store it as a set of target angle correction values, and generate dynamic pose correction parameters, as shown in Table 10.

[0040] Table 10 Example of the set of dynamic pose correction parameters As shown in Table 10, calculate the target angle correction value according to the reference angles of adjacent frames and time weights, and generate dynamic pose correction parameters.

[0041] Please refer to Figure 6 , the training effect analysis module includes: The gesture adjustment trend detection sub-module calls the dynamic pose correction parameters to remind the trainer of the current gesture, detects the change in the joint angles of the trainer after pose adjustment, calculates the angle change rate and spatial displacement between adjacent frames, and obtains the gesture adjustment trend data set according to the gesture correction direction and speed; The gesture adjustment trend detection sub-module calls the dynamic pose correction parameters to remind the trainer of the current gesture. First, according to the obtained dynamic pose correction parameters, extract the joint point numbers, target correction angles, and correction directions that need to be adjusted. Taking the second joint of the index finger as an example, if the dynamic pose correction parameter is , and the current angle is , and the correction direction is to decrease, then the system prompts the trainer to bend this joint, detects the change in the joint angles of the trainer after pose adjustment, calculates the angle change rate of each joint point through the three-dimensional coordinates of consecutive frames, and combines the spatial displacement to obtain the angle change and spatial movement during the gesture adjustment. For example, in subsequent frames, if the angles of the second joint of the index finger change sequentially to , , , the corresponding angular rate of change is respectively , and . At the same time, the three-dimensional space displacement is calculated. For example, if the three-dimensional coordinates of the joint point move from to , the space displacement is mm. Further combining the correction direction and speed, the angular rate of change and space displacement between adjacent frames are organized into a gesture adjustment trend data set, as shown in Table 11, and finally the gesture adjustment trend data set is obtained.

[0042] Table 11 Example of gesture adjustment trend data set As shown in Table 11, the joint angle, angular rate of change, space displacement, and correction direction in each frame are recorded to form a gesture adjustment trend data set.

[0043] According to the gesture adjustment trend data set, the correction path compliance judgment sub-module analyzes whether the gesture correction path conforms to the target correction angle, compares the current gesture adjustment trend with the target correction path, calculates the angle deviation and displacement error, and uses the formula: ; Calculate the correction path error value through operation, judge whether the correction path error value is within the set threshold range, screen the key joints that need additional adjustment, and obtain the set of additional posture adjustment requirements; Among them, is the correction path error value, is the th joint angle in the current frame, is the target correction angle, is the weight of the th joint, is the total number of joints; According to the gesture adjustment trend data set, the correction path compliance judgment sub-module analyzes whether the gesture correction path conforms to the target correction angle. First, obtain the joint angle of each frame in the gesture adjustment trend data set and compare it with the target correction angle to calculate the angle deviation , and call the corresponding weight according to the importance of each joint, and use the formula to calculate the correction path error value. Taking the second joint of the index finger as an example, if the angles of the current three frames are respectively , , , the target angle is , and the weights are respectively , , , the corrected path error value is , then, compare the calculated corrected path error value with the set threshold . If , it is determined that the current corrected path meets the requirements; otherwise, it is determined that it does not meet the requirements. Furthermore, key joints that need additional adjustment are screened, and a set of additional pose adjustment requirements is generated, as shown in Table 12.

[0044] Regarding the setting of the threshold and weight: The basis for setting the threshold : This threshold is set to 1.0 based on the accuracy requirements for fine adjustment of gestures. For the natural fluctuation range of human joints during gesture training, it is obtained by actually measuring the stability of finger bending and rotation. This value usually varies between and . The specific value is related to the complexity of the gesture and the learning stage. For example, it can be set to 2.0 in the beginner stage, and to 1.0 or lower in the advanced stage to improve gesture accuracy. The basis for setting the weight : The setting of the weight is based on the importance of the joint points and their influence degree in the gesture. For example, the index finger and thumb have higher recognition in sign language, so their weights are larger, usually set to or , while the wrist rotation has less influence on gesture recognition, so it is set to or . In this example, the weights of each joint of the index finger are set to , and . This is related to their components and angle change sensitivity in the gesture. The specific weight values are adjusted according to the actual recognition effects of different gestures to ensure accuracy and robustness. Table 12 Example of corrected path error value and set of additional adjustment requirements As shown in Table 12, the corrected path error value of the second joint of the index finger is less than the threshold, so no additional adjustment is required. However, the error value of the wrist joint exceeds the threshold, so it is determined that additional adjustment is needed and stored as a set of additional pose adjustment requirements.

[0045] The training adjustment suggestion generation sub-module determines whether additional pose adjustment guidance is needed based on the set of additional pose adjustment requirements, combines the current gesture adjustment trend, outputs the adjustment direction and amplitude of each joint, and generates sign language training adjustment suggestions; The training adjustment suggestion generation sub-module determines whether additional gesture adjustment guidance is needed according to the set of additional gesture adjustment requirements. First, it calls the joint point numbers marked as requiring additional adjustment in the set of additional gesture adjustment requirements, and combines the current gesture adjustment trend to extract the angular change rate and spatial displacement of the joint point. Taking the wrist joint as an example, if its angular change rate is , and the spatial displacement is mm, and the correction direction is to decrease, then according to the angular change rate being lower than the set correction speed threshold , it is determined that the adjustment speed is too slow, and a prompt "Increase the rotation speed" is output, and the required additional adjustment amplitude is calculated. If the current angle is , and the target angle is , then the additional adjustment amplitude is . Furthermore, training adjustment suggestions are generated, including the adjustment direction and adjustment amplitude, as shown in Table 3.

[0046] A sign language tracking device includes a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the sign language translation training system for gesture tracking as described above is implemented.

[0047] The above are only the preferred embodiments of the present invention, and do not limit the present invention in other forms. Any person skilled in the art may use the disclosed technical content to make changes or modifications into equivalent embodiments with equivalent changes and apply them to other fields. However, as long as it does not depart from the technical solution content of the present invention, any simple modification, equivalent change, and modification made to the above embodiments based on the technical essence of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. A sign language translation training system based on gesture tracking, characterized in that: The system comprises: The gesture tracking module obtains the three-dimensional coordinate data of the key points of the hand skeleton, detects the spatial displacement trend of the key points during the gesture movement, calculates the movement rate and direction change rate of the gesture trajectory, selects the key frame data that meets the gesture stability standard, and obtains the skeleton topology feature set; The skeleton structure completion module analyzes the missing key points of the current frame based on the skeleton topology feature set, calculates the reasonable positions of the key points under the constraints of the skeleton structure, screens the candidate positions of the completion points, detects the spatial consistency and motion continuity of the candidate points, screens out the completion points that do not meet the constraints, and establishes a coordinate set of the completion key points; The gesture standard comparison module calls the joint coordinate data of the current frame based on the completed key point coordinate set, extracts the standard gesture data according to the current sign language training content, calculates the gesture feature deviation value, and calculates the gesture matching confidence according to the feature deviation value of the key frame; The gesture dynamic correction module analyzes the key joints whose deviations exceed the range based on the gesture matching confidence, calculates the target angle correction values ​​of the joints based on the finger opening and closing angles, wrist rotation angles and motion trends of adjacent frames, and generates dynamic posture correction parameters.

2. The sign language translation training system based on gesture tracking according to claim 1, characterized in that: The skeleton topology feature set includes the wrist spatial coordinates, the palm center point position, the five fingertip coordinates, the relative angles of the finger joints and the ratio of the adjacent joint lengths. The completed key point coordinate set includes the finger joint point spatial coordinates, the wrist key point position, the skeleton connection relationship and the joint rotation completion angle. The gesture matching confidence specifically refers to the gesture feature deviation value, the key frame matching error, the time weighted comparison score and the gesture standard similarity evaluation coefficient. The dynamic gesture correction parameters include the corrected joint angle value, the target gesture adjustment vector, the gesture adjustment reference benchmark and the correction step sequence.

3. The sign language translation training system based on gesture tracking according to claim 1, characterized in that: The gesture tracking module includes: The skeleton key point acquisition submodule obtains the three-dimensional coordinate data of the skeleton key points of the hand, screens the visible skeleton joint points according to the hand topological structure, judges the visibility according to the coordinate value of each skeleton joint point in space, stores the three-dimensional coordinate data of the visible joint points, and generates a visible skeleton joint point coordinate set; The spatial displacement trend calculation submodule calculates the spatial displacement of key points during gesture movement based on the visible skeleton joint point coordinate set, obtains the displacement vector of the key points in each frame, and uses the formula: ； and ； The motion rate change rate is obtained by calculation, and frames whose rate change rate fluctuation is lower than the change threshold are selected according to the gesture stability standard to generate a key frame data set; in, Representative The three-dimensional coordinates of the frame joint points, For the Frame rate, For the Frame rate change rate, is the time interval between adjacent frames; The skeletal topology feature extraction submodule calculates the joint angle changes between key points based on the key frame data set, obtains the change trend of the frame joint angles, establishes a time series of the joint angles, screens the skeletal topology structures whose stability exceeds the stability threshold, obtains the joint point set in the skeletal topology structure, and obtains the skeletal topology feature set.

4. The sign language translation training system based on gesture tracking according to claim 1, characterized in that: The skeleton structure completion module includes: The missing key point analysis submodule analyzes the missing key points of the current frame based on the skeleton topology feature set, determines the existence status of each joint point in the skeleton topology structure, locates the position of the missing joint point in the skeleton chain according to the topological relationship, calls the relative position relationship of the known adjacent key points, obtains the three-dimensional coordinate set of the adjacent key points, and generates the adjacent coordinate set of the missing key points; The completion point position calculation submodule calls the bone length ratio of the joint according to the adjacent coordinate set of the missing key points, and calculates the reasonable position of the missing key points under the constraints of the bone structure based on the relative position relationship of the adjacent key points, using the formula: ； Calculate and obtain a set of candidate positions for completion points; in, is the three-dimensional coordinate of the completed point, are the three-dimensional coordinates of nearby known key points, is the three-dimensional coordinate of another known key point, To complete the bone length between the point and the adjacent key point, is the ratio of bone lengths between known key points; The completion key point screening submodule detects the spatial consistency and motion continuity of the candidate points based on the completion point candidate position set, calculates the spatial position deviation and motion trajectory deviation of the candidate points between adjacent frames, screens out candidate points whose spatial deviation and trajectory deviation exceed the deviation threshold, obtains the completion points that meet the bone structure constraints, and obtains the completion key point coordinate set.

5. The sign language translation training system based on gesture tracking according to claim 1, characterized in that: The gesture standard comparison module includes: The gesture feature extraction submodule calls the joint coordinate data of the current frame based on the completed key point coordinate set, extracts the finger bending angle, wrist rotation angle and spatial position of the key joints, obtains the standard gesture data according to the current sign language training content, extracts the standard gesture features corresponding to the joints, including the finger bending angle, wrist rotation angle and spatial position, and obtains the current frame gesture feature set and the standard gesture feature set; The feature deviation calculation submodule calculates the deviation value of each feature item according to the current frame gesture feature set and the standard gesture feature set, using the formula: ； Calculate and obtain a set of gesture feature deviation values; in, is the total deviation of gesture features, The current frame eigenvalues, The standard gesture eigenvalues, For the The weight of the feature, is the total number of features; The gesture matching confidence calculation submodule screens gesture key frames according to the gesture feature deviation value set, and extracts the time weight of the key frames. According to the feature deviation value and time weight of the key frames, the formula is adopted: ； Calculate and obtain the gesture matching confidence; in, is the gesture matching confidence, For the The total feature deviation of key frames, For the The time weight of the keyframes, is the total number of standard keyframes.

6. The sign language translation training system based on gesture tracking according to claim 1, characterized in that: The gesture dynamic correction module comprises: The joint angle change calculation submodule extracts the deviation degree of the current gesture posture based on the gesture matching confidence, calculates the angle change rate of each joint, calculates the joint angle change between adjacent frames according to the finger bending angle, wrist rotation angle and spatial position of the key joints, and sorts out the angle change rate of each joint to obtain a joint angle change rate data set; The key joint deviation judgment submodule analyzes the key joints whose deviation exceeds the range according to the joint angle change rate data set, compares the angle change rate of each joint with the set angle change rate threshold, screens the key joints whose change rate exceeds the threshold, and generates a key joint deviation set; The dynamic posture correction parameter generation submodule calculates the target angle correction value of each joint based on the key joint deviation set, combined with the finger opening and closing angle, wrist rotation angle and the motion trend of adjacent frames, using the formula: ； Calculate and obtain the target angle correction value set to generate dynamic posture correction parameters; in, is the correction angle of the target joint, is the current joint angle, For adjacent frames The reference angle of the mid joint, For adjacent frames The time weight, is the total number of adjacent frames, Indicates the deviation sign.

7. The sign language translation training system based on gesture tracking according to claim 1, characterized in that: The system also includes a training effect analysis module; The training effect analysis module calls the dynamic posture correction parameters to remind the trainee, detects the trainee's current gesture adjustment trend, analyzes whether the gesture correction path meets the target correction angle, determines whether additional posture adjustment guidance is needed, outputs adjustment suggestions for the training process, and obtains sign language training adjustment suggestions; The sign language training adjustment suggestions specifically include gesture correction prompts, adjustment path guidance, posture matching analysis and training progress feedback.

8. The sign language translation training system based on gesture tracking according to claim 7, characterized in that: The training effect analysis module includes: The gesture adjustment trend detection submodule calls the dynamic gesture correction parameters, reminds the trainee of the current gesture, detects the change of the trainee's joint angle after the gesture adjustment, calculates the angle change rate and spatial displacement between adjacent frames, and obtains the gesture adjustment trend data set according to the gesture correction direction and speed; The correction path conformity judgment submodule analyzes whether the gesture correction path conforms to the target correction angle according to the gesture adjustment trend data set, compares the current gesture adjustment trend with the target correction path, calculates the angle deviation and displacement error, and uses the formula: ； Obtain the correction path error value through calculation, determine whether the correction path error value is within the set threshold range, select key joints that require additional adjustment, and obtain a set of additional posture adjustment requirements; in, To correct the path error value, The current frame joint angles, Correct the angle for the target, For the The weight of the joints, is the total number of joints; The training adjustment suggestion generation submodule determines whether additional posture adjustment guidance is needed based on the additional posture adjustment requirement set, outputs the adjustment direction and amplitude of each joint in combination with the current gesture adjustment trend, and generates sign language training adjustment suggestions.

9. A sign language tracking device, comprising a memory and a processor, characterized in that: The memory stores a computer program, and when the processor executes the computer program, the sign language translation training system based on gesture tracking according to any one of claims 1 to 8 is implemented.

Images