A shot-shooting action scoring method based on multivariate time series similarity

Abstract

The embodiment of the present invention discloses a method for scoring a shot put action based on the similarity of a multivariate time series, comprising the steps of: field device and video acquisition, obtaining the original information of the thrower's posture, extracting the part of the shot put action from the original information of the thrower's posture, performing advection filtering, and calculating the similarity between the multivariate time series representing the coach's shot put action and the multivariate time series representing the thrower's shot put action to be scored using an improved time dynamic regularization algorithm, as the thrower's score. The present invention has low requirements for the field, simple deployment, and low cost, and takes into account the whip-like force characteristics of the shot put action. Through the dynamic time regularization of the multivariate time series, it can reflect the relationship between joint movements in time, and can be better applied to the thrower tracking and the shot put action extraction, and can realize automatic identity tracking and recognition by expanding the face recognition technology.

Description

Technical Field

[0001] The present invention relates to the field of sports information technology processing, and in particular to a shot put action scoring method based on multivariate time series similarity. Background Art

[0002] At present, shot put has been included in the physical examination items of the senior high school entrance examination in many cities in my country as an optional or compulsory examination item. The scientific training method of shot put is becoming increasingly important. Scientific training requires a set of quantifiable methods, so the scoring method of shot put action is an indispensable part of assisting the scientific training of shot put. Existing methods for judging shot put throwing posture, such as a method for correcting shot put throwing posture based on intelligent data collection disclosed in Chinese patent CN109146960A, disclose the following steps: collecting the body model parameters and corresponding throwing posture data of a group of students who have achieved full marks in the physical education shot put throwing project in the senior high school entrance examination, and uploading them to the cloud server as standard throwing posture data, the body model parameters and throwing posture data constitute the influencing factor matrix X, wherein the body model parameters are environmental variables and the throwing posture data are decision variables; collecting data samples of the user's corresponding throwing posture to constitute the indicator matrix Y, and using the Elman neural network to learn and train the indicator matrix Y , test, and establish an Elman neural network model of throwing posture for the user's body model parameters; use the Elman neural network model established in S2 to predict the throwing posture data, obtain the recommended decision variable X, and send the recommended decision variable X to the user terminal. The user corrects his throwing posture according to the recommended decision variable X. It collects the throwing posture data through the intelligent data acquisition device and compares it with the standard throwing posture data to obtain the throwing posture score. However, it is based on the intelligent acquisition device to collect the throwing posture data. The wearing and removal of the intelligent acquisition device takes time. At the same time, the purchase of hardware is also a considerable cost. It is difficult to apply it in a middle school physical education class with only 45 minutes and dozens of students at the same time. In addition, the whip-like force characteristics of the solid ball throwing action are not considered in its steps, that is, there is a process of force center transfer in the process of solid ball force. It starts from the ankle joint, then the knee joint, then the hip joint, then the waist force, then the shoulder joint, and finally the arm to throw the ball. Only comparing various collected data without considering the relationship between joint movements in time, the scientificity of the data obtained is lacking. Summary of the invention

[0003] The technical problem to be solved by the embodiment of the present invention is to provide a shot put action scoring method based on multivariate time series similarity, which can improve the reliability and simplicity of the shot put action skill evaluation.

[0004] In order to solve the above technical problems, an embodiment of the present invention provides a shot put action scoring method based on multivariate time series similarity, comprising the following steps:

[0005] S1: Setting up a camera on the field to collect video of a shot-throwing action, and obtaining a sample video of a coach throwing a shot-throwing action;

[0006] S2: Obtaining the original information of the thrower’s posture from the collected video;

[0007] S3: extracting a part having a shot throwing action from the original information;

[0008] S4: Remove irrelevant factors from the multivariate time series representing the action of throwing a solid ball obtained by S3;

[0009] S5: Calculate S4 to get the multivariate time series representing the coach's shot throwing action and a multivariate time series representing the thrower's shot-throwing actions to be scored The improved time dynamic warping algorithm is used to calculate the similarity between the two and serve as the thrower's score.

[0010] Wherein, the S3 comprises the steps of:

[0011] S3-1: Find the key frame and take the frame where the shot put occurs as the key frame;

[0012] S3-2: Take the body trunk and the horizontal unit vector Angle As a characteristic of the shot-throwing action

[0013] S3-3: Using keyframe m as anchor, track forward and backward respectively The value of , and the interval [a, b] is the effective part of throwing a solid ball, take the original information Part as a multivariate time series of the action of throwing a medicine ball.

[0014] Wherein, the S4 comprises the steps of:

[0015] S4-1: Use SG filter data for smoothing;

[0016] S4-2: Use min-max normalization to perform data normalization on smoothed data.

[0017] Wherein, the S5 comprises the steps of:

[0018] S5-1: Define the distance calculation method between two key points, take They represent the i-th key point in posture p and the j-th key point in posture q respectively, and the cosine distance is used to calculate the distance between the key points;

[0019] S5-2: Use weighted Euclidean distance to calculate the distance between postures (p, q);

[0020] S5-3: Design The distance matrix To calculate the sequence with sequence Similarity between sequences of unequal length;

[0021] S5-4: Through the distance matrix Calculate the cost matrix ;

[0022] S5-5: Starting from i=n, ​​j=m, trace back to the cost matrix obtained in S5-4 and select The minimum subscript in , construct arrive Minimum cost path cost, representing the sequence Transform to sequence Specific path of

[0023] Among them, the cost matrix obtained from S5-4 is Get The value of represents the sequence Complete transformation to sequence The cost, yes Normalize to get the desired cost, and calculate the similarity s in the following way: s=1-cost / 17.

[0024] The implementation of the embodiments of the present invention has the following beneficial effects: the present invention has low requirements on the venue, simple deployment, and low cost, takes into account the whip-like force characteristics of the solid ball throwing action, and can reflect the temporal relationship of joint movements through dynamic time regularization of multivariate time series. It can be better suitable for tracking the thrower and extracting the action of throwing a solid ball, and can realize automatic identity tracking and recognition by expanding face recognition technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram of site deployment of the present invention. DETAILED DESCRIPTION

[0026] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings.

[0027] A shot put action scoring method based on multivariate time series similarity according to an embodiment of the present invention is implemented by the following method.

[0028] S1: Site installation and video acquisition.

[0029] S1-1: Figure 1 As shown, a camera is set up on one side of the throwing position, L meters away from the person, and the camera projection is H meters from the ground. The camera projection is parallel to the throwing trajectory, and at the same time, ensure that the person stands in the hot spot area of ​​the picture. This distance mainly depends on the focal length of the lens. Taking the equivalent focal length of a general mobile phone of 22 mm as an example, L is about 5 meters. It is necessary to ensure that the throwing trajectory can be captured. H is about 1.5 meters. The camera equipment is set up on the right side of the person by default. If the camera equipment is set up on the left side of the person, the video needs to be mirrored to ensure that the solid ball is thrown from left to right in the picture.

[0030] S1-2: Collect examples of coaches throwing solid balls. The coaches throw the solid balls according to the standard sequence of bending forward, bending backward, throwing, leaning forward, and returning to the center. This sequence of movements is used as the *standard movement*.

[0031] S1-3: Start collecting after the thrower enters the throwing position. Stop collecting after the student completes the shot put and before leaving the throwing position.

[0032] S2: Get the original information of the thrower’s posture from the video collected by S1

[0033] S2-1: Based on the Faster R-CNN pre-trained model, from the frame Identify the human bounding box set and solid sphere bounding box For all , calculate its intersection over Union ratio with the hotspot area A, that is, the area of ​​intersection between A and s divided by the area of ​​union between A and s:

[0034]

[0035] Take the s with the largest intersection ratio as the rough target human bounding box , thereby realizing the tracking of the thrower.

[0036] S2-2: Passed right The sub-image containing the target human body is cropped and passed to HRNet for posture estimation. Convert back Human body posture data Here we use the human body posture annotation of the COCO dataset. The human body annotated in the COCO dataset is composed of 17 key points, each of which is Represents the position coordinates in the input image. = [p0,p1,…,p 16 ].in Indicates five senses, Represents the joints of the human body.

[0037] S2-3: From Take out , as The human body bounding box ,in for The upper left corner, for .

[0038] S2-4: Repeat the operations from S2-1 to S2-3 for each frame of the video captured by S1 to obtain the original information of the entire thrower's posture.

[0039] S3: Since the thrower's action is random, it is necessary to extract the part of the shot-throwing action from the original information of the thrower's posture obtained in S-2.

[0040] S3-1: Find the key frame, take the frame where the shot is thrown as the key frame, and start calculating from frame 0 ,until , indicating that the ball has left the human body, and then go back one frame, and call this frame m as the key frame, where and The current frame The bounding box of the human body and the bounding box of the solid sphere.

[0041] S3-2: Take the body trunk and the horizontal unit vector Angle As a characteristic of the action of throwing a solid ball. It can be calculated by the following formula:

[0042]

[0043] in, is the vector formed by the key point 5 of the left shoulder and the key point 11 of the left hip, It is the vector formed by key point 6 of the right shoulder and key point 12 of the right hip.

[0044] S3-3: Using keyframe m as anchor, track forward The value of the swing back to the maximum amplitude and then the return to the correct part is the back bow action, and then The forward swing and then return to the normal position is the forward bending movement, and the current frame is recorded as a. The interval [a, m] is obtained. Then return to key frame m, and key frame m is tracked backward Swing forward and then return to the normal position. This part is the forward leaning action. The current frame at this time is recorded as b, and the interval [b, m] is obtained. The interval [a, m] [m, b] is combined to obtain the interval [a, b]. The video segment surrounded by the interval [a, b] is the effective part of the shot throwing action. Take the posture data part, since the posture data is a 17-dimensional vector , so the data can be recorded as , where t represents the tth frame from a and i represents the i-th key point. represents the value of the i-th key point in the t-th frame, so the sequence also meets the definition of a multivariate time series.

[0045] S4: Preprocess the multivariate time series representing the shot-throwing action obtained by S3, including filtering random interference and removing irrelevant factors such as body shape and position.

[0046] S4-1: The movement of the human body is continuous. Due to the influence of factors such as the camera equipment, environment, motion blur, and the error of the model itself, the data will be randomly disturbed, resulting in the actual data being not smooth. Therefore, the data needs to be smoothed. Here, the SG filter (Savitzky-Golay Filter) is used to filter the data. It can better retain the trend of the curve change. The multivariate time series representing the action of throwing a solid ball, that is, By separating key points and dimensions, we can get a set of time series , and then apply the SG filter respectively, and finally the filtered and smoothed posture data can be obtained.

[0047] S4-2: Perform data normalization on the smoothed data obtained from S4-1 to remove irrelevant factors such as body shape and position. Here, the minimum-maximum normalization is used to perform a linear transformation on the posture data and map the result to [0 - 1], where the maximum value of the posture data is 1 and the minimum value is 0. The specific function is as follows:

[0048]

[0049] Where min is the minimum value in the posture data, and max is the maximum value in the posture data.

[0050] S5: The multivariate time series representing the coach's shot throwing action calculated by S4 Recorded as Similarly, the thrower's shot-throwing action to be scored calculated by S4 is recorded as , using the improved time dynamic warping algorithm to calculate and , as the thrower's score.

[0051] S5-1: Define the distance calculation method between two key points, take They represent the i-th key point in posture p and the j-th key point in posture q respectively. Considering that the key point is a two-dimensional vector and the biggest influencing factor of the solid ball action is the angle of the action, the cosine distance is used to calculate the distance between the key points. The formula is as follows:

[0052]

[0053] S5-2: Define the distance calculation method between two postures (p, q). Since the posture is a vector composed of 17 key points, and the contribution of each key point to the shot throwing action is not equal, the weighted Euclidean distance is used here to calculate the distance between postures:

[0054]

[0055] in This is the key point distance formula defined in S5-1, which can be considered . refers to the weight of key point i.

[0056] Finally, we select the appropriate weights. First, considering that the facial key points do not participate in the action, the weight of the facial key points should be 0. Since the ball is thrown from left to right, the right side of the body is blocked, and the corresponding weight should also be reduced. Combined with the actual participation of each joint in the actual shot blasting action and the actual adjustment, we finally get the following weight vector:

[0057]

[0058] S5-3: Sequence with sequence The time for different people to throw a shot is not consistent, so n and m should be considered unequal. To calculate the similarity between sequences of unequal lengths, we need to design a The distance matrix is:

[0059]

[0060] The values ​​of the matrix elements are defined as follows:

[0061] in The distance between the pose of the i-th frame of sequence P and the pose of the j-th frame of sequence Q is the distance formula between poses defined in S5-2.

[0062] S5-4: Distance matrix from S5-3 Calculate the cost matrix .

[0063] First, let The n = 0 row and m = 0 column in Then calculate the value of the corresponding position in The value of . The value of satisfies the following definition:

[0064]

[0065] by Let's start the calculation as an example. , from the definition of distance matrix, we know so , and so on, until finally calculating value to complete the calculation of the cost matrix.

[0066] S5-5: Cost matrix obtained from S5-4 Perform the following operations, set the coordinates i=n, ​​j=m, and select The index of the minimum value in the minimum cost path is saved Then update the values ​​of i and j to the minimum index, and repeat this operation until i=0, j=0. Finally, we construct arrive Least Cost Path , which represents the sequence Transform to sequence specific path.

[0067] S5-6: Calculate similarity, cost matrix obtained from S5-4 Get The value of represents the sequence Complete transformation to sequence Due to the limitation of the definition of cost matrix C, the alignment cost of Q's subsequence is always less than the alignment cost of Q, which is not in line with the actual situation. Standardization is required to get the desired cost:

[0068]

[0069] The cost metric is the distance between Q and P, which is inversely proportional to the score. The similarity s needs to be calculated to serve as the score of the action:

[0070] .

[0071] The above disclosure is only a preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A shot-shot action scoring method based on multivariate time series similarity, characterized in that: The following steps are involved: S1: Setting up a camera on the field to collect video of a shot-throwing action, and obtaining a sample video of a coach throwing a shot-throwing action; S2: Obtaining the original information of the thrower’s posture from the collected video; S3: extracting the part having the action of throwing a solid ball from the original information, comprising the steps of: S3-1: Find the key frame and take the frame where the shot put occurs as the key frame; S3-2: Take the body trunk and the horizontal unit vector Angle As a characteristic of the shot-throwing action S3-3: Using keyframe m as anchor, track forward and backward respectively The value of , and the interval [a, b] is the effective part of throwing a solid ball, take the original information Partially as a multivariate time series of the action of throwing a medicine ball; S4: Remove irrelevant factors from the multivariate time series representing the action of throwing a solid ball obtained by S3; S5: Calculate S4 to get the multivariate time series representing the coach's shot throwing action and a multivariate time series representing the thrower's shot-throwing actions to be scored The improved time dynamic warping algorithm is used to calculate the similarity between the two and serve as the thrower's score.

2. The solid ball action scoring method based on multivariate time series similarity according to claim 1 is characterized in that: The S4 comprises the steps of: S4-1: Use SG filter data for smoothing; S4-2: Use min-max normalization to perform data normalization on smoothed data.

3. The solid ball action scoring method based on multivariate time series similarity according to claim 2 is characterized in that: The S5 comprises the steps of: S5-1: Define the distance calculation method between two key points, take They represent the i-th key point in posture p and the j-th key point in posture q respectively, and the cosine distance is used to calculate the distance between the key points; S5-2: Use weighted Euclidean distance to calculate the distance between postures (p, q); S5-3: Design The distance matrix To calculate the sequence with sequence Similarity between sequences of unequal length; S5-4: Through the distance matrix Calculate the cost matrix ; S5-5: Start from i=n,j=m and trace back to the cost matrix obtained in S5-4, select The minimum subscript in , construct arrive Minimum cost path cost, representing the sequence Transform to sequence Specific path of S5-6: Cost matrix obtained from S5-4 Get The value of represents the sequence Complete transformation to sequence The cost, yes Normalize to get the desired cost, and calculate the similarity s in the following way: s=1-cost / 17.

Images