Improved gesture action scoring method and system based on DTW intelligent algorithm, and medium

Abstract

The invention discloses an improved gesture action scoring method based on a DTW (Dynamic Time Warning) intelligent algorithm, and discloses a system and a medium with the improved gesture action scoring method based on the DTW intelligent algorithm, and the improved gesture action scoring based on the DTW intelligent algorithm uses a Gaussian distance as a distance measurement function in the DTW. According to the method, the relationship between different features can be considered in the time sequence, the similarity in the time sequence can be better captured, and the scoring accuracy in the electric power operation situation can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of dynamic uncertainty measurement, and in particular to a gesture scoring method, system and medium based on an improved DTW intelligent algorithm. Background Art

[0002] This paper proposes an action evaluation method based on a dynamic time warping (DTW) algorithm improved by Gaussian distance, which is used to evaluate fine movements during power grid operations. The DTW algorithm is an algorithm that is very suitable for processing time series data. In action evaluation, an action sequence can be regarded as a time series, in which the state at each moment (such as body posture, joint angle, etc.) can be used as an element of the sequence. By performing DTW matching on two action sequences, the similarity between them can be measured and a score can be calculated.

[0003] In power operations, workers' hand gestures require an effective scoring system. Traditional DTW algorithms only consider the shape information of time series, ignoring the differences between different features within the time series. In power operations, the data of workers' hand gestures varies little, and features are not obvious. Due to interference from factors such as the environment and equipment, the sequences of workers' hand gestures may contain noise and outliers. Traditional DTW cannot effectively handle these issues in power operation gesture scoring systems, resulting in inaccurate scoring. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes an improved gesture scoring method based on the DTW intelligent algorithm. This method defines a distance metric function in the DTW algorithm to calculate the distance between two gesture sequences, and incorporates Gaussian distance into the algorithm to improve the accuracy of scoring in specific application scenarios.

[0005] The present invention also provides a system and medium having the above-mentioned gesture action scoring method improved based on the DTW intelligent algorithm.

[0006] According to the first embodiment of the present invention, the gesture scoring method based on the improved DTW intelligent algorithm is characterized by comprising the following steps:

[0007] Professionals perform several groups of operations, and the most standard one is selected from the groups of operations as the evaluation standard; the standard movements are simulated in a virtual environment, and the movement images are analyzed and processed to obtain hand movement information, and the hand movement information is extracted and sample preprocessed to obtain movement features of the hand movement information;

[0008] Decomposing the operation of the evaluation object into different action groups, extracting and reducing the dimension of each action group to obtain a one-dimensional vector corresponding to each action group, inputting the one-dimensional vector into the DTW algorithm and comparing it with the evaluation criteria to obtain the similarity between each action group and the evaluation criteria; wherein, in the DTW algorithm, Gaussian distance is used as the distance metric function in DTW to evaluate the similarity between the action group and the evaluation criteria;

[0009] The influencing factors of each joint point in different actions are set, and a scoring formula for the operation is constructed. Based on the scoring formula and the gap between each action group and the evaluation criteria, each joint point is scored to obtain the final total score.

[0010] According to an embodiment of the present invention, the improved gesture action scoring method based on the DTW intelligent algorithm has at least the following beneficial effects: the present invention improves the DTW algorithm using Gaussian distance, and uses Gaussian distance as the distance metric function in DTW, which can consider the relationship between different features in the time series and better capture the similarity in the time series, ultimately improving the accuracy of the scoring.

[0011] According to some embodiments of the present invention, the operating operation has 24 action groups, including wearing a safety helmet, wearing insulating gloves, testing electricity, opening the electrical box door, and screwing screws; wearing a safety helmet has 90 action information; wearing insulating gloves has 70 action information, testing electricity has 67 action information, opening the electrical box door has 98 action information, and screwing screws has 67 action information.

[0012] According to some embodiments of the present invention, the operation has 27 joint points in the virtual environment, including 24 finger joint points, 1 wrist joint point, 1 hand center node, and 1 node representing the entire hand model.

[0013] According to some embodiments of the present invention, the method uses Gaussian distance as the distance metric function in DTW, and its formula is:

[0014]

[0015] Where x=(x1, x2, .., x n ) and y=(y1,y2,..,y n ) represent two n-dimensional vectors, n represents the dimension of the vector; ω i Is a weight parameter used to adjust the importance of different features. When ωi takes a smaller value, it means that the corresponding feature contributes less to the distance. i A larger value indicates that the corresponding feature contributes more to the distance.

[0016] According to some embodiments of the present invention, in the step of setting the influence factor of each joint point in different actions, constructing a scoring formula for the operation, and scoring each joint point based on the scoring formula and the gap between each action group and the evaluation criteria to obtain the final total score, the scoring formula can be described as follows:

[0017] Es k =(1-dist*F k )*s / 27

[0018] score=Es1+Es2+Es3+......+Es 27

[0019] Where k represents the number of joints (1, 2, 3, ..., 27); s represents the full score of the action (100 points), and there are 27 joints in total, so the full score of each joint is s / 27; dist is the cumulative distance value of the shortest path calculated by the DTW algorithm; F k is the impact factor of each joint point; Es k is the score of each joint point; score is the final score of the test action, which is the sum of the scores of 27 joint angles.

[0020] According to the second embodiment of the present invention, the gesture scoring system based on the improved DTW intelligent algorithm is characterized by including:

[0021] The standard setting module can allow professionals to perform several groups of operations and select the most standard one from the groups of operations as the evaluation standard; simulate the actions of the evaluation standard in a virtual environment, analyze and process the action pictures to obtain hand movement information, extract the hand information movements and perform sample preprocessing to obtain the movement features of the hand movement information;

[0022] The action comparison module can decompose the operation of the evaluation object into different action groups, extract and reduce the dimension of each action group to obtain a one-dimensional vector corresponding to each action group, input the one-dimensional vector into the DTW algorithm to compare with the evaluation criteria, and obtain the similarity between each action group and the evaluation criteria; wherein, the DTW algorithm uses Gaussian distance as the distance metric function in DTW to evaluate the similarity between the action group and the evaluation criteria;

[0023] The action scoring module can set the influencing factors of each joint point in different actions, construct a scoring formula for the operation, and score each joint point based on the scoring formula and the gap between each action group and the evaluation criteria to obtain the final total score.

[0024] According to some embodiments of the present invention, the operating operation has 24 action groups, including wearing a safety helmet, wearing insulating gloves, testing electricity, opening the electrical box door, and screwing screws; wearing a safety helmet has 90 action information; wearing insulating gloves has 70 action information, testing electricity has 67 action information, opening the electrical box door has 98 action information, and screwing screws has 67 action information.

[0025] According to some embodiments of the present invention, the operation has 27 joint points in the virtual environment, including 24 finger joint points, 1 wrist joint point, 1 hand center node, and 1 node representing the entire hand model.

[0026] According to some embodiments of the present invention, the system uses Gaussian distance as the distance metric function in DTW, and its formula is:

[0027]

[0028] Where x=(x1, x2, .., x n ) and y=(y1,y2,..,y n ) represent two n-dimensional vectors, n represents the dimension of the vector; ω i Is a weight parameter used to adjust the importance of different features. i When the value is smaller, it means that the corresponding feature contributes less to the distance. i A larger value indicates that the corresponding feature contributes more to the distance.

[0029] According to some embodiments of the present invention, in the step of setting the influence factor of each joint point in different actions, constructing a scoring formula for the operation, and scoring each joint point based on the scoring formula and the gap between each action group and the evaluation criteria to obtain the final total score, the scoring formula can be described as follows:

[0030] Es k =(1-dist*F k )*s / 27

[0031] score=Es1+Es2+Es3+......+Es 27

[0032] Where k represents the number of joints (1, 2, 3, ..., 27); s represents the full score of the action (100 points), and there are 27 joints in total, so the full score of each joint is s / 27; dist is the cumulative distance value of the shortest path calculated by the DTW algorithm; F k is the impact factor of each joint point; Es kis the score of each joint point; score is the final score of the test action, which is the sum of the scores of 27 joint angles.

[0033] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which:

[0035] Figure 1 Schematic diagram of the steps of the gesture scoring method based on the improved DTW intelligent algorithm according to an embodiment of the present invention;

[0036] Figure 2 This is a structural diagram of the gesture scoring system improved based on the DTW intelligent algorithm provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0038] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limitations on the present invention.

[0039] In the description of the present invention, "several" means one or more, "many" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The use of "first" and "second" in the description is solely for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, implicitly specifying the number of the indicated technical features, or implicitly specifying the order of the indicated technical features.

[0040] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, and connecting should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0041] During power plant operations, workers' hand motion data has minimal variation and unclear features. Due to interference from environmental and equipment factors, the sequences of workers' hand motions may contain noise and outliers. Traditional DTW (Delayed Wisdom Warping) cannot effectively handle these issues in power grid gesture scoring systems, resulting in inaccurate scoring.

[0042] Based on the characteristics of power operations, this application uses Gaussian distance instead of the distance measurement function in the DTW method, which can better capture the similarity in the time series and thus improve the accuracy of the scoring.

[0043] Example 1

[0044] The embodiment of the present application provides a gesture scoring method based on the improved DTW intelligent algorithm, such as Figure 1 As shown, the method includes the following steps:

[0045] Step S100: Professionals perform several groups of operations, and select the most standard one from the several groups of operations as the evaluation standard; simulate the actions of the evaluation standard in a virtual environment, analyze and process the action pictures to obtain hand action information, extract the hand information actions, and perform sample preprocessing to obtain the action features of the hand action information.

[0046] The motion data is obtained by professional power grid workers who perform operations in the training system, from which a set of relatively standard actions are selected as the evaluation criteria. There are 24 scenarios in the safety operation training, with many actions. In this embodiment, five actions are selected for evaluation.

[0047] To obtain a standard multi-scale hand motion dataset, 20 experienced operators simulated a series of grid operation-related actions in a virtual environment, including donning a hard hat, wearing insulating gloves, testing electricity, opening a box door, and tightening screws. Using OpenCV technology, the action images displayed in UE4 were analyzed and processed to obtain hand motion information. The samples were then preprocessed to extract motion features. The specific distribution of each action in the dataset is shown in Table 1.

[0048] Table 1. Movement distribution of multi-scale hand dataset

[0049]

[0050]

[0051] The virtual glove has 27 nodes, including 24 finger joints, 1 wrist joint, 1 hand center node, and 1 node representing the entire hand model. Each frame of data contains three values ​​for each hand joint in virtual space: coordinate offset, rotation angle, and scaling factor. This paper preprocesses the data. When scoring actions, the entire system only uses coordinate offset data, so the rotation angle and scaling factor data are eliminated during data preprocessing. The coordinate offset data is three-dimensional (x, y, z). The DTW used for action evaluation requires one-dimensional data for evaluation, so the three-dimensional coordinate offset data is reduced in dimensionality using a summation method.

[0052] Step S200: Decompose the operation of the evaluation object into different action groups, extract and reduce the dimension of each action group to obtain a one-dimensional vector corresponding to each action group, input the one-dimensional vector into the DTW algorithm and compare it with the evaluation criteria to obtain the similarity between each action group and the evaluation criteria; wherein, in the DTW algorithm, Gaussian distance is used as the distance metric function in DTW to evaluate the similarity between the action group and the evaluation criteria.

[0053] Based on the actual action data extraction and dimensionality reduction, a one-dimensional vector is generated and input into the improved DTW algorithm to determine the Gaussian distance between the operation of the evaluation object and the standard operation as their similarity. The specific calculation formula is as follows:

[0054]

[0055] Where x=(x1, x2, ..., x n ) and y=(y1,y2,...,y n ) represent two n-dimensional vectors, n represents the dimension of the vector; ω i Is a weight parameter used to adjust the importance of different features. i When the value is smaller, it means that the corresponding feature contributes less to the distance. i A larger value indicates that the corresponding feature contributes more to the distance.

[0056] Step S300: Set the influence factor of each joint point in different actions, construct a scoring formula for the operation, and score each joint point based on the scoring formula and the gap between each action group and the evaluation criteria to obtain the final total score.

[0057] After obtaining the distance between the standard action and the action to be evaluated, it is necessary to construct a suitable action evaluation formula to realize the function of scoring the action standard.

[0058] It can be imagined that the distribution of influencing factors affects the rationality of the action evaluation formula. The influencing factors of different joints need to be set according to different actions. For example, when tightening a screw, a larger influencing factor needs to be set for the joint of the thumb. When wearing a hat, a larger influencing factor needs to be set for the wrist joint.

[0059] After determining the impact factor, you can construct the scoring formula:

[0060] Es k =(1-dist*F k )*s / 27 (2)

[0061] score=Es1+Es2+Es3+……+Es 27 (3)

[0062] Where k represents the number of joints (1, 2, 3, ..., 27); s represents the full score of the action (100 points), and there are 27 joints in total, so the full score of each joint is s / 27; dist is the cumulative distance value of the shortest path calculated by the DTW algorithm; F k is the impact factor of each joint point; Es k is the score of each joint point; score is the final score of the test action, which is the sum of the scores of 27 joint angles.

[0063] Example 2

[0064] In order to verify the effectiveness of the gesture action scoring method of the Gaussian distance improved DTW algorithm provided by the present invention, a control experiment was conducted by setting the evaluation score of experts and the action evaluation score based on DTW (Dynamic Time Warping) for verification.

[0065] The conventional DTW algorithm, based on dynamic programming, is also used to determine the similarity between time series data. It has been widely used in audio processing and is also applied to motion evaluation. It performs dimensionality reduction on the coordinate offsets of the wrist nodes of the standard and test movements, calculates the shortest distance between the two sequences, determines the similarity between the standard and test movements, and completes the evaluation of the movement.

[0066] Expert scoring, DTW-based action evaluation, and the grid-related action evaluation method based on the Gaussian-modified DTW system model were used to evaluate five types of actions: wearing a hard hat, wearing insulating gloves, testing electricity, opening the electrical box door, and tightening screws. Each type of action was evaluated three times. The evaluation results are shown in Table 2.

[0067] Table 2: Distribution of multi-scale hand data set actions

[0068]

[0069]

[0070] As Table 2 shows, expert scoring is somewhat subjective. Simple, non-threatening maneuvers are scored relatively uniformly, with scores above 90. However, expert scoring for dangerous maneuvers uses more detailed criteria and stricter requirements. Mistakes or improper execution of dangerous maneuvers result in scores below 60, indicating a failure. However, relying solely on one-on-one instruction and scoring by experts would be a significant drain on human resources and material resources, severely impacting training efficiency.

[0071] While DTW-based motion evaluation is effective for scoring large-scale movements, it fails to effectively demonstrate the accuracy of fine-scale movements. Minor errors in electrical testing and screw tightening go undetected, resulting in roughly the same score. Furthermore, it is affected by invalid movements and cannot score sequential movements. For example, the standard procedure for putting on insulating gloves is to put on the left hand first, while the "Insulating Gloves 2" procedure involves putting on the right hand first. No points are deducted for this subtle error.

[0072] The grid-related motion evaluation method based on the Gaussian-modified DTW model can provide a more detailed evaluation of motions of any scale and better detect errors in fine motions. The results demonstrate that the grid-related motion evaluation method based on the Gaussian-modified DTW model can be applied to a virtual reality safety training system for power grids, and that the evaluation scores produced by this method are effective and objective.

[0073] Example 3:

[0074] Another embodiment of the present application provides a gesture scoring system based on an improved DTW intelligent algorithm, such as Figure 2 As shown, the system 20 includes:

[0075] The standard setting module 201 can be used by professionals to perform several groups of operations and select the most standard one from the groups of operations as the evaluation standard; simulate the actions of the evaluation standard in a virtual environment, analyze and process the action images to obtain hand movement information, extract the hand movement information, and perform sample preprocessing to obtain the movement features of the hand movement information;

[0076] The action comparison module 202 can decompose the operation of the evaluation object into different action groups, extract and reduce the dimension of each action group to obtain a one-dimensional vector corresponding to each action group, input the one-dimensional vector into the DTW algorithm to compare with the evaluation criteria, and obtain the similarity between each action group and the evaluation criteria; wherein, the DTW algorithm uses Gaussian distance as the distance metric function in DTW to evaluate the similarity between the action group and the evaluation criteria;

[0077] The action scoring module 203 can set the influencing factors of each joint point in different actions, construct a scoring formula for the operation, and score each joint point based on the scoring formula and the gap between each action group and the evaluation criteria to obtain the final total score.

[0078] Furthermore, the above-mentioned operation has 24 action groups, including wearing a safety helmet, wearing insulating gloves, testing electricity, opening the electrical box door, and tightening screws; the wearing of a safety helmet has 90 action information; wearing insulating gloves has 70 action information, testing electricity has 67 action information, opening the electrical box door has 98 action information, and tightening screws has 67 action information.

[0079] Furthermore, the operation has 27 joint points in the virtual environment, including 24 finger joint points, 1 wrist joint point, 1 hand center node, and 1 node representing the entire hand model.

[0080] Furthermore, the system uses Gaussian distance as the distance metric function in DTW, and its formula is:

[0081]

[0082] Where x=(x1, x2, .., x n ) and y=(y1,y2,..,y n ) represent two n-dimensional vectors, n represents the dimension of the vector; ω i Is a weight parameter used to adjust the importance of different features. i When the value is smaller, it means that the corresponding feature contributes less to the distance. i A larger value indicates that the corresponding feature contributes more to the distance.

[0083] Furthermore, in the action scoring module 203, the scoring formula can be described as:

[0084] Es k =(1-dist*F k )*s / 27

[0085] score=Es1+Es2+Es3+......+Es 27

[0086] Where k represents the number of joints (1, 2, 3, ..., 27); s represents the full score of the action (100 points), and there are 27 joints in total, so the full score of each joint is s / 27; dist is the cumulative distance value of the shortest path calculated by the DTW algorithm; F kis the impact factor of each joint point; Es k is the score of each joint point; score is the final score of the test action, which is the sum of the scores of 27 joint angles.

[0087] Another aspect of the present application provides a computer-readable storage medium storing computer-executable instructions for executing the above-mentioned Figure 1 The gesture action scoring method shown is based on the improved DTW intelligent algorithm.

[0088] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0089] Those skilled in the art will appreciate that all or some of the steps and systems in the method disclosed above can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, and the computer-readable medium can include computer storage media (or non-transitory media) and communication media (or temporary media). As known to those skilled in the art, the term computer storage media is included in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data) and is volatile and non-volatile, removable, and non-removable. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory, or other memory technology, CD-ROM, digital versatile disks (DVD), or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage, or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0090] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the above implementation mode. Technical personnel familiar with the field can also make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A gesture scoring method based on the improved DTW intelligent algorithm, characterized in that: The following steps are involved: Professionals perform several groups of operation operations, and select the most standard one from the several groups of operation operations as the evaluation standard; simulate the action of the evaluation standard in a virtual environment, analyze and process the action picture, obtain hand action information, extract the hand information action and perform sample preprocessing to obtain the action features of the hand action information; Decomposing the operation of the evaluation object into different action groups, extracting and reducing the dimension of each action group to obtain a one-dimensional vector corresponding to each action group, inputting the one-dimensional vector into the DTW algorithm to compare with the evaluation criteria, and obtaining the similarity between each action group and the evaluation criteria; wherein, in the DTW algorithm, Gaussian distance is used as the distance metric function in DTW to judge the similarity between the action group and the evaluation criteria; The influencing factors of each joint point in different actions are set, and a scoring formula for the operation is constructed. Based on the scoring formula and the gap between each action group and the evaluation criteria, each joint point is scored to obtain the final total score.

2. The method according to claim 1, characterized in that The operation has 24 action groups, including wearing a safety helmet, wearing insulating gloves, testing electricity, opening the electrical box door, and tightening screws; wearing a safety helmet has 90 action information; wearing insulating gloves has 70 action information, testing electricity has 67 action information, opening the electrical box door has 98 action information, and tightening screws has 67 action information.

3. The method according to claim 2, characterized in that The operation has 27 joint points in the virtual environment, including 24 finger joint points, 1 wrist joint point, 1 hand center node, and 1 node representing the entire hand model.

4. The method according to claim 1, characterized in that: This method uses Gaussian distance as the distance metric function in DTW, and its formula is: Where x=(x1,x2,...,x n ) and y=(y1,y2,...,y n ) represent two n-dimensional vectors, n represents the dimension of the vector; ω i is a weight parameter used to adjust the importance of different features. i When the value is smaller, it means that the corresponding feature contributes less to the distance. i When a larger value is taken, it means that the corresponding feature contributes more to the distance.

5. The method according to claim 3, characterized in that: In the step of setting the influencing factor of each joint point in different actions, constructing a scoring formula for the operation, and scoring each joint point based on the scoring formula and the gap between each action group and the judging standard to obtain the final total score, the scoring formula can be described as: Es k =(1-dist*F k )*s / 27 score=Es1+Es2+Es3+……+Es 27 Among them, k represents the number of joints 1, 2, 3...27; s means the full score of the action is 100 points, there are 27 joints in total, so the full score of each joint is s / 27; dist is the cumulative distance value of the shortest path calculated by the DTW algorithm; F k is the impact factor of each joint point; Es k is the score for each joint point; score is the final score of the test action, which is the sum of the scores of the 27 joint angles.

6. A gesture scoring system based on the improved DTW intelligent algorithm, characterized in that: include: The standard setting module can be used by professionals to perform several groups of operations and select the most standard one from the several groups of operations as the evaluation standard; The action of the judging standard is simulated in a virtual environment, and the action picture is analyzed and processed to obtain hand action information, and the hand information action is extracted and sample preprocessing is performed to obtain the action features of the hand action information; The action comparison module can decompose the operation of the evaluation object into different action groups, extract and reduce the dimension of each action group, obtain the one-dimensional vector corresponding to each action group, input the one-dimensional vector into the DTW algorithm to compare with the evaluation standard, and obtain the similarity between each action group and the evaluation standard; wherein, in the DTW algorithm, Gaussian distance is used as the distance measurement function in DTW to judge the similarity between the action group and the evaluation standard; The action scoring module can set the influencing factors of each joint point in different actions, construct a scoring formula for the operation, and score each joint point based on the scoring formula and the gap between each action group and the evaluation criteria to obtain the final total score.

7. The system according to claim 6, characterized in that The operation has 24 action groups, including wearing a safety helmet, wearing insulating gloves, testing electricity, opening the electrical box door, and tightening screws; wearing a safety helmet has 90 action information; wearing insulating gloves has 70 action information, testing electricity has 67 action information, opening the electrical box door has 98 action information, and tightening screws has 67 action information.

8. The system according to claim 7, characterized in that The operation has 27 joint points in the virtual environment, including 24 finger joint points, 1 wrist joint point, 1 hand center node, and 1 node representing the entire hand model.

9. The system according to claim 6, characterized in that The system uses Gaussian distance as the distance metric function in DTW, and its formula is: Where x=(x1,x2,...,x n ) and y=(y1,y2,...,y n ) represent two n-dimensional vectors, n represents the dimension of the vector; ω i is a weight parameter used to adjust the importance of different features. i When the value is smaller, it means that the corresponding feature contributes less to the distance. i When a larger value is taken, it means that the corresponding feature contributes more to the distance.

10. The system according to claim 8, characterized in that In the step of setting the influencing factor of each joint point in different actions, constructing a scoring formula for the operation, and scoring each joint point based on the scoring formula and the gap between each action group and the judging standard to obtain the final total score, the scoring formula can be described as: Es k =(1-dist*F k )*s / 27 score=Es1+Es2+Es3+……+Es 27 Among them, k represents the number of joints 1, 2, 3...27; s means the full score of the action is 100 points, there are 27 joints in total, so the full score of each joint is s / 27; dist is the cumulative distance value of the shortest path calculated by the DTW algorithm; F k is the impact factor of each joint point; Es k is the score for each joint point; score is the final score of the test action, which is the sum of the scores of the 27 joint angles.

11. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the method according to any one of claims 1 to 5.

Images