A method for identifying and scoring queue-style skydiving training actions

Abstract

The present invention belongs to the field of video processing technology, and specifically relates to a method for motion recognition and scoring for parachuting training in formation. The present invention makes the training process traceable by recording videos, which facilitates repeated viewing and analysis by coaches, and allows trainees to see their own training movements through the videos. The video recorded by a depth camera is combined with image depth information to analyze the posture of the training subject in the video and assess the standardization of their movements, which helps avoid discrepancies caused by subjective factors of the coach. The training target is locked using target tracking, motion recognition, and training space calibration methods to reduce interference from irrelevant personnel in video analysis in multi-person scenarios. Based on the target locking method described above, real-time analysis of the locked trainees is also possible, ultimately segmenting the video into individual video segments for each student and generating their own evaluation reports, which helps reduce the time spent waiting for analysis results and improves training efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of video processing, and in particular relates to a method for recognizing and scoring queue-type skydiving training actions. Background Art

[0002] The skydiving process is divided into three stages: exiting the cabin, descending, and landing. Each stage places high demands on the skydiver's basic body control ability. Based on these three stages, skydiving training is broken down into three ground training programs.

[0003] Skydiving analysis techniques can be categorized into traditional expert experience and computer-based analysis. Currently, relying on expert observation to assess students remains the mainstream approach. However, this approach can lead to subjective variations in individual expert guidance, and the rapid nature of the process can hinder clear and accurate evaluation. Furthermore, students may be limited in their understanding and lack intuitive and accurate feedback.

[0004] In real-world skydiving training, it's often not individual training, but rather multiple personnel training in formation. In formation-based training, traditional expert experience assessment methods can reduce the efficiency of the entire training process by blocking the entire formation while the instructor corrects individual students or preventing them from observing the next student. Summary of the Invention

[0005] To address the above issues, the present invention proposes a method for identifying and scoring parachuting training movements, which specifically includes:

[0006] S1. Setting up a depth camera to collect videos and point cloud images of skydiving training;

[0007] S2. Turn on the depth camera at the beginning of training and calibrate the camera parameters for the training area to obtain the smallest possible calibration space;

[0008] S3. After the training starts, the depth camera is used to collect the video stream and the point cloud information corresponding to each frame to obtain continuous time series data;

[0009] S4, according to the posture action, find the training target and lock it. If the target is locked, enter S5, otherwise record the stage as irrelevant stage 0 in the data sequence synchronized with the video, the key point data is a 25*3 0 matrix and repeat S4;

[0010] S5. Continuously track the locked target and record the time when the lock starts. s Frame, record the three-dimensional coordinates corresponding to the coordinates of the target's posture key points in each frame, and record the stage of the target's action recognized by the action recognition model in each frame, and determine whether the target leaves the calibration space. If so, record the time at which the target leaves the calibration space at the same time. eAfter the frame, enter S6, otherwise repeat S5;

[0011] S6, score the target and determine whether the training is completed. If so, go to S7, otherwise return to S4;

[0012] S7. Divide the obtained video into training segments for each student and associate them with their respective scoring reports before outputting them.

[0013] Furthermore, in S1, the parachuting process is decomposed into three parts: exiting the cabin, descending, and landing, and the position of the depth camera is set according to the corresponding parts.

[0014] Furthermore, the specific method of obtaining the calibration space in S2 is: calibrating the area where the initial position of the trainee is located in the two-dimensional image through the depth camera, and at the same time making the calibration area just cover the trainee during the training process.

[0015] Furthermore, the specific method for finding the training target and locking it according to the posture action in S4 is:

[0016] Use the target tracking model to track all people appearing in the current screen and give each person a unique ID j , and obtain a mask that only covers the person; the mask is a binary matrix with the same resolution as the image captured by the camera, which is used to code the area outside the target, and at the same time obtain the spatial position P of the person based on the point cloud information j The spatial position is based on the three-dimensional coordinates (x j ,y j ,z j );

[0017] Select the people who appear in the calibration space, use the mask to mark the non-personnel area, and then use the posture key point recognition model to identify the 25 key points of each person's two-dimensional coordinates (x p ',y p '), and finally the posture of each person is represented by a 25*2 matrix;

[0018] The resulting 25*2 matrix is ​​converted into a vector, and the training phase of the vector is determined by the action phase recognition model. The classification labels of the action phase recognition model are: classification 1 is the preparation phase, classification 2 is the take-off phase, classification 3 is the landing phase, and classification 0 is irrelevant action;

[0019] Find the person in the calibration space who is classified as the preparation stage (i.e., classification 1) by the action stage recognition model and has the highest confidence level;

[0020] If a person in the preparation stage is selected, he / she will be designated as a training target and his / her ID will be recorded. t And the Nth video captured at this time s If there is no person in the preparation stage, a 25*2 zero matrix, a 25*3 zero matrix, and classification 0 are recorded in the time series data, indicating that the target is not locked.

[0021] Furthermore, the specific method for continuously tracking the locked target in S5 is:

[0022] According to ID t Get the target mask Mask and spatial position P t (x t ,y t ,z t ), only use the mask to encode the image;

[0023] Obtain the target's 25*2 key point coordinate matrix through the posture key point recognition model;

[0024] Map the two-dimensional key point coordinates to the point cloud information to obtain the corresponding 25*3 three-dimensional key point coordinate matrix;

[0025] Identify the training stage through the action stage recognition model;

[0026] The obtained two-dimensional coordinate matrix, three-dimensional coordinate matrix, and the stage are recorded in the time series data.

[0027] Furthermore, the specific method for scoring the target in S6 is:

[0028] The interval of the data sequence collected when the locked training target leaves the calibration space [N s ,N e ]The data in the stage series smoothing process:

[0029] Set up a new sequence S' to store the stage smoothing results; use a sliding window of size 5 to slide from the beginning of the stage sequence bit by bit, count the number of classification labels in the sliding window, and record the stage with the largest number in the third position in the middle of the sliding window in S'. At this time, the sliding window takes the stage of the interval [N1, N5] in S. The position in the middle of the sliding window actually corresponds to N3, so the smoothing result is recorded in the N3 position of S'; finally, the new stage sequence S' is obtained;

[0030] S' is smoothed again to obtain S";

[0031] Take out the longest subsequence in which the stage categories in S are arranged from small to large and the first number is 1 and the last number is 3, and record the actual position N corresponding to the first and last positions of the subsequence. is 'Frame and Nth ie 'frame, then the interval [N is ',N ie The video and data sequences of '] are the actual segments of the target being trained; if there is no regular subsequence, it is regarded as an irrelevant process that has been misidentified and the interval is not recorded;

[0032] Extract the time series data interval [N s ',N e '] data sequence S i , including 25*2 two-dimensional key point coordinate matrix, 25*3 three-dimensional key point coordinate matrix and the stage; according to the index requirements, the data is processed and the score is calculated; the indicators include overall body stability, leg stability, and standard degree of movement; finally, the score and report recommendation R are generated based on the index calculation results i ; The method for calculating the overall body stability index is: traverse the entire sequence to calculate the correlation between the adjacent data before and after. The correlation calculation is based on the vector composed of the angles between the limbs calculated using the three-dimensional key point coordinates, and then accumulate each correlation value as the evaluation parameter; the method for calculating the leg stability index is: calculate the ratio of the distance between the two hips to the distance between the two knees, and the ratio of the distance between the two ankles to the distance between the two knees in each frame, and use the two as evaluation parameters; the method for calculating the standard degree calculation index is: use the DWT algorithm to calculate the similarity between the standard action sequence and the sequence to be evaluated, an N*M matrix A whose elements are the cosine similarities between the vectors transformed from the three-dimensional key point coordinates of the corresponding positions in the two sequences, and find a path from (1,1) to (N,M) with the smallest sum of cosine values ​​from the matrix A, and the sum of the cosine values ​​on the final path is used as the evaluation parameter; the scoring parameters are combined with their corresponding weights to calculate the final training score, and the weights of each scoring parameter are trained using the historical scores through a machine learning algorithm, and the problem and degree of the problem corresponding to each scoring parameter value are determined based on the coach's experience and opinions.

[0033] The beneficial effects of the present invention are as follows: the present invention makes the training process traceable by recording videos, which is beneficial for coaches to watch and analyze repeatedly, and students can see their own training movements through videos; the video recorded by the depth camera is combined with the image depth information to analyze the posture of the training object in the video and evaluate the standard degree of its movement, which is beneficial to avoid differences caused by subjective factors of the coach; the training target is locked by using target tracking, action recognition, and training space calibration methods to reduce the interference factors of irrelevant personnel in video analysis in multi-person scenarios; based on the above-mentioned target locking method, real-time analysis of the locked students can also be achieved, and finally the video is divided into single-person video clips of each student and respective evaluation reports are generated, which is beneficial to reducing the time waiting for analysis results and improving training efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of the main process of the present invention.

[0035] Figure 2 Flowchart for finding a target before locking onto the training target.

[0036] Figure 3 is the distribution map of posture key points.

[0037] Figure 4 Flowchart for tracking a target after locking onto the target in training.

[0038] Figure 5 The figure shows the process and principle diagram for smoothing the stage sequence. DETAILED DESCRIPTION

[0039] The present invention will be described in detail below with reference to the accompanying drawings.

[0040] like Figure 1 As shown, the main process of the present invention includes:

[0041] 1. Point the camera directly at the training facility and the first student preparing. The three training programs have different facilities, so the camera placement distances are different. For extravehicular training, the camera is placed 5-8 meters away from the facility. For landing training, the camera is placed 4-6 meters away from the facility. For landing training, the camera is placed 18-20 meters away from the facility.

[0042] 2. After i trainees (i is a positive integer) have lined up, the cameras are turned on and the training space calibration parameters are configured to limit the most suitable and minimum training activity range;

[0043] 3. After training begins, the depth camera collects the video stream and the point cloud information corresponding to each frame, and the data constitutes continuous time series data;

[0044] 4. Such as Figure 2 As shown in the figure, when the training target is not locked, the camera captures each frame:

[0045] 1) Use the target tracking model to track all people (students and unrelated people) who appear in the current screen and assign each person a unique ID j , and obtain a mask that only covers the person. The Mask is a binary matrix with the same resolution as the image captured by the camera, which is used to code the area outside the target. At the same time, the spatial position P of the person is obtained according to the point cloud information. j The spatial position is based on the three-dimensional coordinates (x j ,y j ,z j );

[0046] 2) Select the people who appear in the calibration space, use the Mask to mark the non-personnel area, and then use the posture key point recognition model to identify the two-dimensional coordinates (x p ',y p '),like Figure 3 The key points include 25 points (p∈[0,24]), including the top of the head, eyes, ears, nose, neck, shoulders, elbows, wrists, hips, knees, ankles, heels, little toes and big toes. The posture of each person is represented by a 25*2 matrix.

[0047] 3) Convert the 25*2 matrix into a vector, and determine the training stage of the vector using the action stage recognition model. The classification labels of the action stage recognition model are: preparation stage is 1, take-off stage is 2, landing stage is 3, and irrelevant action is 0.

[0048] 4) searching for the persons in the calibration space who are determined by the action stage recognition model to be in the preparation stage, i.e., classified as 1 and have the highest confidence;

[0049] 5) If a person in the preparation stage is selected, he / she will be treated as a training target and his / her ID will be recorded. t At this time, the video captures the Nth s If the personnel in the preparation stage are not satisfied, a 25*2 zero matrix, a 25*3 zero matrix, and the irrelevant action classification 0 are recorded in the time series data, and the above 1) to 4) are continued in each subsequent frame;

[0050] 5. Such as Figure 4 As shown in the figure, when the training target is locked, the camera captures each frame:

[0051] 1) Based on ID t Get the target's Mask and spatial position P t (x t ,y t ,zt ), only use Mask to encode the image;

[0052] 2) Obtaining the 25*2 key point coordinate matrix of the target through the posture key point recognition model;

[0053] 3) Mapping the two-dimensional key point coordinates to the point cloud information to obtain the corresponding 25*3 three-dimensional key point coordinate matrix;

[0054] 4) identifying the training stage of the action using the action stage recognition model;

[0055] 5) The two-dimensional coordinate matrix, three-dimensional coordinate matrix, and the stage in which they are located are recorded in the time series data;

[0056] 6) If the target does not leave the calibration space, continue to execute 1) to 5) for each frame; if the target leaves the calibration space, stop tracking and record the Nth frame captured by the video. e Frame and cancel recording ID t At the same time, a subtask is started to score the training process of the target. The subtask will not block other processes. After that, the target under training will continue to be searched in each frame, that is, the above 4 is executed;

[0057] 6. Steps for the above subtask of scoring the target:

[0058] 1) Take out the time series data in the interval [N s ,N e ] subsequence S;

[0059] 2) If Figure 5 As shown, the stage sequence in S is smoothed:

[0060] Set up a new sequence S' to store the results of the stage smoothing process; use a sliding window of size 5 to slide from the beginning to the beginning of the stage sequence, and count the number of each stage in the sliding window, such as 2 irrelevant actions, 2 initial stages, and 1 take-off stage in [0,0,1,1,2]; if there are 2 or 3 zeros in the sliding window, remove 1 zero and only count the remaining four stages, such as [0,1,0,0,1] becomes [1,0,0,1] after removal; the stage with the largest number is recorded in the corresponding position of the third position in the middle of the sliding window in S', such as the largest number in [0,1,2,1,1] If it is 1, record 1 in S'. At this time, the sliding window takes the stage of the interval [N1, N5] in S. The middle position of the sliding window actually corresponds to N3, so the smoothing result is recorded in the N3 position of S'; if there are two stages with the same number of stages but the above-mentioned "removal of 1" is not performed, the stage with the larger category is recorded. For example, if the number of 0s and 1s in [0,1,0,0,1] is the same after removing 1 0, then record 1; if "removal of 1 0" is not performed, then record the stage with the smaller category. For example, if the number of 1s and 2s in [1,1,0,2,2] is the same, then record 1; finally, we get the new stage sequence S';

[0061] 3) Repeat step 2) on S' to obtain S";

[0062] 4) Take out the longest subsequence in S" whose stage categories are arranged in ascending order and whose first number is 1 and whose last number is 3. For example, from [0,0,1,1,1,2,2,3,3,3,0,0,0,0], take out the subsequence [1,1,1,2,2,3,3,3] according to the rule, and record the actual position N corresponding to the first and last positions of the subsequence. is 'Frame and Nth ie 'frame, then the interval [N is ',N ie The video and data sequences are the actual segments of the training target being tracked. If there is no subsequence of the rule, such as [0,0,3,2,0,3,1,0], it is regarded as an irrelevant process that has been misidentified, and the interval is not recorded.

[0063] 5) Take out the time series data interval [N s ',N e '] data sequence S i , including the 25*2 two-dimensional key point coordinate matrix, the 25*3 three-dimensional key point coordinate matrix and the stage; the above data are processed and the scores are calculated according to the index requirements; the indicators mainly include the overall stability of the body, the stability of the legs, and the standard degree of the movements. The indicators can be dynamically adjusted based on experience; finally, the scores and report recommendations are generated based on the index calculation results. iThe method for calculating the overall body stability index is as follows: the correlation between the adjacent data before and after is calculated by traversing the entire sequence. The correlation calculation is based on the vector formed by the angle between each limb calculated using the three-dimensional key point coordinates, and then each correlation value is accumulated as the evaluation parameter; the method for calculating the leg stability index is as follows: the ratio of the distance between the two hips to the distance between the two knees, and the ratio of the distance between the two ankles to the distance between the two knees in each frame are calculated, and the two are used as evaluation parameters; the method for calculating the standard degree calculation index is as follows: the DWT (Dynamic Time A warping algorithm is used to calculate the similarity between the standard action sequence and the sequence to be evaluated. An N*M matrix A (N and M are the lengths of the standard action sequence and the sequence to be evaluated, respectively) whose elements are the cosine similarities between the vectors of the three-dimensional key point coordinate transformations at corresponding positions in the two sequences is used. A path from (1,1) to (N,M) with the minimum sum of cosine values ​​is found in matrix A. The sum of the cosine values ​​on the final path is used as the evaluation parameter. The scoring parameters are combined with their corresponding weights to calculate the final training score. The weights of each scoring parameter can be trained using historical scores through a machine learning algorithm. The problem location and degree of the problem corresponding to each scoring parameter value are determined based on the coach's experience and opinions.

[0064] 7. After the training, according to the interval [N is ',N ie '] Split the video into training segments for each student i and associate their rating reports R i .

[0065] In order to facilitate the overall understanding of the method of the present invention, the method of the present invention is broken down into the following more detailed steps for explanation:

[0066] 1. Use a depth camera to collect training videos and point cloud images, while keeping the camera facing the training facilities and the trainees waiting in line for training at a relatively close distance;

[0067] 2. Calibrate the approximate spatial position (distance from the camera) of the trainee during initial preparation and the area in the 2D image. The calibration area should be as small as possible while ensuring that the entire training process covers the trainee.

[0068] 3. The target tracking model tracks all people who appear in front of the camera. The model generates a unique ID for each person and a mask matrix that can be used to isolate the person from the image. The model also obtains the person's position based on the point cloud image. The position is a three-dimensional coordinate with the camera as the origin.

[0069] 4. Using the mask provided by the target tracking algorithm, the image is processed so that all areas except the person in the calibration area are black. The pose keypoint recognition model is then used to identify the people in the processed image and extract the 2D image coordinates of 25 keypoints for each person.

[0070] 5. The vector converted from the 25*2 key point matrix is ​​identified through the action stage recognition model to obtain the action stage of the posture. If the recognition result is most likely the preparation stage and is in the marked area, the person is locked and the person's ID is recorded for subsequent tracking. Otherwise, it is regarded as an unrelated person who is not in training. The above-mentioned action stage recognition model is a classifier trained by a machine learning algorithm using key point data with action stage labels. The action stage label categories are set as positive integers from small to large in the order of preparation, take-off and landing, and the remaining unrelated action labels are set to 0. This action stage recognition model can not only be used to identify the stage, but also to distinguish between unrelated people in training and those who are not in training;

[0071] 6. If locked on the training target, record the time when lock starts at Nth s Frame, record the three-dimensional coordinates corresponding to the coordinates of the target's posture key points in each frame, and record the stage of the target's action recognized by the action recognition model in each frame until the target leaves the calibration area, and record the time at which the target leaves the calibration area at the same time. e frame;

[0072] 7. If there is no locked target, the phase is recorded as irrelevant phase 0 in the data sequence synchronized with the video, and the key point data is a 25*3 0 matrix;

[0073] 8. When the locked training target leaves the calibration area, the collected data sequence interval [N s ,N e ]The data within the stage sequence smoothing process is performed;

[0074] 9. The above-mentioned steps for smoothing the stage sequence are as follows:

[0075] 1) Count the stages of the current position and the stages of the two positions before and after it (a total of 5 positions), and get the number of preparation, take-off, landing, and irrelevant stages respectively.

[0076] 2) If there are 2 or 3 irrelevant stages among the 5 stages, i.e., they are classified as 0, then one 0 is removed and only the stages at the remaining four positions are counted;

[0077] 3) The stage with the largest number is taken as the stage of the current position;

[0078] 4) If the statistical results show that there are two stages with the same number of stages and the above 2) is performed, the stage with the larger classification label is taken as the current stage; if there are only two stages with the same number of stages and the above 2) is not performed, the stage with the smaller classification label is taken as the current stage;

[0079] 5) Repeat 1) to 4) until all N s to N e The sequence of intervals is used to obtain a new equal-length sequence S'. If there are less than 5 statistical positions at the beginning or end of the sequence, the stage at that position is directly regarded as irrelevant;

[0080] 10. Perform the above stage sequence smoothing process on S' again to obtain the final stage sequence S";

[0081] 11. Find the longest subsequence from S that changes in order from small to large, that is, the sequence interval from the first appearance of the preparation stage to the last appearance of the landing stage, and the stages between the two are arranged in order as the training process interval of the training target [N s ',N e '], if not found, it is considered that the interval [N s ,N e ] is a non-training process that is misidentified;

[0082] 12. Regarding the above [N s ',N e The three-dimensional key point coordinates, two-dimensional key point coordinates and the stage recorded in each frame within the interval are processed and calculated according to the quantitative indicators of the corresponding training project to finally obtain the score and evaluation report.

Claims

1. A method for identifying and scoring parachuting training movements, characterized in that: include: S1. Setting up a depth camera to collect videos and point cloud images of skydiving training; S2. Turn on the depth camera at the beginning of training and calibrate the camera parameters for the training area to obtain the smallest possible calibration space; S3. After the training starts, the depth camera is used to collect the video stream and the point cloud information corresponding to each frame to obtain continuous time series data; S4. Find the target under training according to the posture and lock it. If the target is locked, enter S5. Otherwise, record the stage as an irrelevant stage in the data sequence synchronized with the video, and the key point data is a 25*3 0 matrix and repeat S4. S5. Continuously track the locked target and record the time when the lock starts. s Frame, record the three-dimensional coordinates corresponding to the coordinates of the target's posture key points in each frame, and record the stage of the target's action recognized by the action recognition model in each frame, and determine whether the target leaves the calibration space. If so, record the time at which the target leaves the calibration space at the same time. e After the frame, enter S6, otherwise repeat S5; S6: Score the target and determine whether the training is finished. If so, go to S7; otherwise, go back to S4. The specific method is: The interval of the data sequence collected when the locked training target leaves the calibration space [N s ,N e ]The data in the stage series smoothing process: Set up a new sequence S' to store the stage smoothing results; use a sliding window of size 5 to slide through the stage sequence from the beginning, count the number of categories in the sliding window, and record the stage with the largest number in the third position from the middle of the sliding window in S'. At this time, the sliding window takes the stage of the interval [N1, N5] in S. The middle position of the sliding window actually corresponds to N3, so the smoothing result is recorded in the N3 position of S'; finally, the new stage sequence S' is obtained; S' is smoothed again to obtain S''; Take out the longest subsequence in S'' whose stage categories are arranged from small to large and whose first number is 1 and whose last number is 3, and record the actual position N corresponding to the first and last positions of the subsequence. is 'Frame and Nth ie 'frame, then the interval [N is ',N ie The video and data sequences of '] are the actual segments of the target being trained; if there is no regular subsequence, it is regarded as an irrelevant process that has been misidentified and the interval is not recorded; Extract the time series data interval [N s ',N e '] data sequence S i , including 25*2 two-dimensional key point coordinate matrix, 25*3 three-dimensional key point coordinate matrix and the stage; according to the index requirements, the data is processed and the score is calculated; the indicators include overall body stability, leg stability, and standard degree of movement; finally, the score and report recommendation R are generated based on the index calculation results i The calculation method of the overall body stability index is as follows: traverse the entire sequence to calculate the correlation of the adjacent data before and after. The correlation calculation is based on the vector composed of the angles between the limbs calculated using the three-dimensional key point coordinates, and then accumulate each correlation value as the evaluation parameter; the calculation method of the leg stability index is as follows: calculate the ratio of the distance between the two hips to the distance between the two knees, and the ratio of the distance between the two ankles to the distance between the two knees in each frame, and use the two as evaluation parameters; the calculation method of the standard degree calculation index is as follows: use the DWT algorithm to calculate the similarity between the standard action sequence and the sequence to be evaluated, an N*M matrix A whose elements are the cosine similarities between the vectors transformed from the three-dimensional key point coordinates of the corresponding positions in the two sequences, and find a path from (1,1) to (N,M) with the smallest sum of cosine values ​​from the matrix A. The sum of the cosine values ​​on the final path is used as the evaluation parameter; Scoring parameters are combined with their corresponding weights to calculate the final training score. The weights of each scoring parameter are trained using historical scores through a machine learning algorithm. The problem and degree of each scoring parameter value are determined based on the coach's experience and opinions. S7. Divide the obtained video into training segments for each student and associate them with their respective scoring reports before outputting them.

2. The method for identifying and scoring parachuting training movements according to claim 1, wherein: In S1, the parachuting process is decomposed into three parts: exiting the cabin, descending, and landing. The position of the depth camera is set according to the corresponding parts.

3. The method for identifying and scoring parachuting training movements according to claim 1, wherein: The specific method of obtaining the calibration space in S2 is: calibrating the area where the initial position of the trainee is located in the two-dimensional image through the depth camera, and at the same time making the calibration area just cover the trainee during the training process.

4. The method for identifying and scoring parachuting training movements according to claim 1, wherein: The specific method of finding the training target and locking it according to the posture and action in S4 is as follows: Use the target tracking model to track all people appearing in the current screen and give each person a unique ID j , and obtain a mask that only covers the person; The mask is a binary matrix with the same resolution as the image captured by the camera, which is used to mark the area outside the target and obtain the spatial position P of the person based on the point cloud information. j The spatial position is based on the three-dimensional coordinates (x j ,y j ,z j ); Select the people who appear in the calibration space, use the mask to mark the non-personnel area, and then use the posture key point recognition model to identify the 25 key points of each person's two-dimensional coordinates (x p ',y p '), and finally the posture of each person is represented by a 25*2 matrix; The resulting 25*2 matrix is ​​converted into a vector, and the training phase of the vector is determined by the action phase recognition model. The classification labels of the action phase recognition model are: classification 1 is the preparation phase, classification 2 is the take-off phase, classification 3 is the landing phase, and classification 0 is irrelevant action; Find the person in the calibration space who is classified as the preparation stage (i.e., classification 1) by the action stage recognition model and has the highest confidence level; If a person in the preparation stage is selected, he / she will be designated as a training target and his / her ID will be recorded. t And the Nth video captured at this time s If there is no person in the preparation stage, a 25*2 zero matrix, a 25*3 zero matrix, and classification 0 are recorded in the time series data, indicating that the target is not locked.

5. The method for identifying and scoring parachuting training movements according to claim 4, wherein: The specific method for continuously tracking the locked target in S5 is as follows: According to ID t Get the target mask Mask and spatial position P t (x t ,y t ,z t ), only use the mask to encode the image; Obtain the target's 25*2 key point coordinate matrix through the posture key point recognition model; Map the two-dimensional key point coordinates to the point cloud information to obtain the corresponding 25*3 three-dimensional key point coordinate matrix; Identify the training stage through the action stage recognition model; The obtained two-dimensional coordinate matrix, three-dimensional coordinate matrix, and the stage are recorded in the time series data.

Images