Ball movement recognition system and method based on time-series pose coding

Abstract

This invention belongs to the field of action recognition, and particularly relates to a ball action recognition system and method based on temporal pose coding. The system includes: a video stream acquisition module that acquires RGB video streams of ball actions of the target object in real time and decomposes them into a continuous video frame sequence in chronological order; a temporal implicit pose coding module that extracts single-frame deep convolutional feature maps, obtains one-dimensional implicit pose feature vectors through channel-dimensional global pooling, and concatenates them sequentially to construct a temporal implicit pose feature matrix; an action cycle detection module that generates motion energy temporal curves based on the Euclidean distance between adjacent feature vectors to complete effective action cycle detection and frame index localization; an action frequency statistics module that outputs action frequency; and a standardization evaluation module that achieves standardized evaluation of action standardization through dynamic time warping alignment. This invention does not require wearable devices, can robustly identify high-frequency ball action cycles and simultaneously output frequency and standardization, significantly improving the generalization ability in daily campus training scenarios.

Description

Technical Field

[0001] This invention belongs to the field of action recognition, and particularly relates to a ball action recognition system and method based on temporal pose coding. Background Technology

[0002] In daily training scenarios for ball games on campus, ordinary RGB cameras are typically used to capture video streams of students' movements to achieve real-time motion recognition and accuracy assessment without wearable devices. However, this approach faces a fundamental underlying technical problem: RGB video streams only contain two-dimensional appearance information and lack direct measurement of three-dimensional spatial posture. Furthermore, ball games (such as basketball dribbling and volleyball passing) are characterized by high-frequency periodicity, rapid self-occlusion of limbs, and local fusion of the ball and limbs, making it extremely difficult to stably extract temporal motion features related to the phase, frequency, and accuracy of the movements from continuous frames. Specifically, traditional methods based on explicit keypoint detection are prone to keypoint jitter or loss due to clothing texture, lighting changes, or self-occlusion, leading to phase misalignment and counting errors in subsequent temporal modeling. While methods based on optical flow or frame difference are acceptable for slow movements, motion blur and background interference severely contaminate motion features when dealing with high-speed reciprocating movements like dribbling and passing (2-4 times per second), making it difficult to accurately define the start and end times of a single movement. Furthermore, significant differences in height, clothing, and movement habits among students amplify these interferences, resulting in a severe deficiency in the generalization ability of existing models in cross-individual scenarios. Therefore, the core technical bottleneck restricting the application of this technology in daily campus training and testing scenarios lies in how to robustly extract temporal implicit pose features that are strongly correlated with the cycle of ball movements and are insensitive to changes in individual appearance, without the need for depth sensors or wearable devices and relying solely on monocular RGB video streams. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention proposes a ball-playing motion recognition system and method based on temporal pose coding. The system includes: a video stream acquisition module for real-time acquisition of RGB video streams and decomposition into a continuous sequence of video frames; a temporal implicit pose coding module for extracting deep convolutional feature maps from each frame and performing global pooling along the channels to obtain a one-dimensional implicit pose feature vector, which is then concatenated temporally to construct a temporal implicit pose feature matrix; and a motion cycle detection module for calculating the Euclidean distance between feature vectors of adjacent frames to generate a motion energy temporal curve, detecting local maxima points where the peak value exceeds a preset threshold, using the interval between adjacent maxima points as candidate cycles, and determining a valid motion cycle if the cycle duration falls within a preset interval, and recording the start and end frame indices. Furthermore, the system also includes a motion frequency statistics module and a standardization evaluation module, used to count the total number of valid cycles and, after aligning the feature submatrices within a cycle with a standard template, evaluate the average feature distance to determine the motion standardization. This invention eliminates the need for wearable devices, enabling real-time recognition of ball-playing motion frequency and standardization based on RGB video streams, making it suitable for daily training scenarios on campuses.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A ball-playing motion recognition system based on temporal pose coding includes:

[0006] The video stream acquisition module is configured to: acquire an RGB video stream containing the ball-like actions of the target object in real time, and decompose the RGB video stream into a continuous sequence of video frames in chronological order;

[0007] The temporal implicit pose coding module is configured to: extract a deep convolutional feature map for each frame in the video frame sequence, perform a global pooling operation on the deep convolutional feature map along the channel dimension to obtain a one-dimensional implicit pose feature vector for the corresponding single frame image, and concatenate the one-dimensional implicit pose feature vectors corresponding to each frame image in chronological order to construct a temporal implicit pose feature matrix; the rows of the temporal implicit pose feature matrix correspond to the time step, and the columns correspond to the implicit pose feature dimensions;

[0008] The motion cycle detection module is configured to: calculate the Euclidean distance between the feature vectors corresponding to adjacent time steps in the temporal implicit attitude feature matrix, generate a motion energy temporal curve, and simultaneously detect local maxima points in the motion energy temporal curve where the peak value is greater than a preset energy threshold, and determine the time interval between two adjacent local maxima points as candidate motion cycles; calculate the duration of the candidate motion cycle, and when the duration falls within a preset cycle duration interval, determine the candidate motion cycle as a valid motion cycle, and record the start frame index and end frame index corresponding to the valid motion cycle.

[0009] Specifically, the ball motion recognition system also includes:

[0010] The action frequency statistics module is configured to: count the total number of valid action cycles within a preset statistical time window, and output the total number as the ball action frequency of the target object;

[0011] The standardization evaluation module is configured to: for each valid action cycle, extract the sub-temporal implicit posture feature matrix within the range from the start frame index to the end frame index corresponding to the cycle; dynamically time-align the sub-temporal implicit posture feature matrix with the pre-stored standard action template feature matrix; calculate the average feature distance after alignment; when the average feature distance is less than the preset standardization threshold, determine that the corresponding ball action meets the standard; otherwise, determine that the corresponding ball action deviates from the standard, and accumulate the number of action deviations or output the action deviation ratio.

[0012] Specifically, a deep convolutional feature map is extracted from each frame of the video frame sequence, and a global pooling operation is performed on the deep convolutional feature map along the channel dimension, including:

[0013] Get the RGB image I of the current frame;

[0014] The current frame RGB image I is input into a pre-trained convolutional neural network, which contains N convolutional layers; the deep convolutional feature map F output by the last convolutional layer in the convolutional neural network is extracted. The deep convolutional feature map F has a spatial height dimension H, a spatial width dimension W, and a channel dimension C, and the feature value at any position in the deep convolutional feature map F is denoted as F(i, j, c), i∈[1, H], j∈[1, W], c∈[1, C];

[0015] For each channel index c in the channel dimension C, calculate the global average of the feature values ​​at all spatial locations (i, j) within the channel, and use this average as the scalar pooling value A for the corresponding channel. c ;

[0016] The scalar pooling values ​​A corresponding to each of the C channels are... c Arranged in ascending order according to channel index c, they form a one-dimensional implicit pose feature vector v for the current frame, where v = [A1, A2, ..., A...]. c …, A C The length of vector v is equal to C.

[0017] Specifically, the generation of motion energy time-series curves includes:

[0018] Obtain the temporal implicit pose feature matrix M, where the number of rows in the temporal implicit pose feature matrix M is the total number of frames T in the video frame sequence, and the number of columns is the dimension C of the implicit pose feature vectors; the vector in the t-th row of matrix M is denoted as v. tWhere t ranges from 1 to T, and each v t The length of is equal to C;

[0019] For adjacent time steps t and t+1, calculate the t-th row vector v. t With the (t+1)th row vector v t+1 The Euclidean distance d between them t This constitutes the original distance sequence;

[0020] Perform sliding window mean filtering on the original distance sequence, wherein the width of the sliding window is set to Q consecutive elements;

[0021] For the k-th starting position in the original distance sequence, take Q elements from the k-th element to the k-th plus Q minus one element, calculate the arithmetic mean of these Q elements, and obtain the k-th smoothed motion energy value; the value of k ranges from 1 to T minus Q, and the smoothed motion energy sequence is obtained.

[0022] Specifically, the time interval between two adjacent local maxima is determined as the candidate action period, including:

[0023] Obtain the smoothed motion energy time-series curve, denoted as E. The curve E contains M1 discrete energy values, where M1 is equal to the total number of frames T in the video frame sequence minus the sliding window width Q. Each energy value corresponds to a time position index m, and the value of m ranges from 1 to M1.

[0024] Local maximum point detection is performed on the motion energy time series curve E to obtain the initial peak sequence P. init Each peak point is denoted as p, which includes the peak position index pos(p) and the peak amplitude amp(p). The detection method is as follows: for each time position index m, if E(m) is greater than its left adjacent value E(m-1) and greater than its right adjacent value E(m+1), then m is determined as a local maximum point. E(m) represents the energy value of the motion energy time series curve at the time position index m.

[0025] Based on the preset action type recognition results, determine the ball action type corresponding to the current video stream;

[0026] When the action type is basketball dribbling, perform the following operations: Calculate the initial peak sequence P. init The average amplitude of all peak values, denoted as amp. mean ;Amplitude greater than or equal to amp mean Peak values ​​multiplied by the first scaling factor are retained;

[0027] For the retained peak sequence, the time interval between two adjacent peaks is calculated. If the time interval is less than the preset minimum interval threshold, the two peaks are merged, the peak with the larger amplitude is retained, and the peak with the smaller amplitude is removed. The minimum interval threshold is 1 / 3 of the lower limit of the cycle duration of the corresponding action type. The time interval between adjacent peaks in the processed peak sequence is determined as the candidate action cycle.

[0028] Specifically, determining the time interval between two adjacent local maxima as the candidate action period also includes:

[0029] When the action type is volleyball passing, perform the following operations: Set the initial peak sequence P... init The peaks are sorted from largest to smallest amplitude, and the three peaks with the largest amplitudes are selected and designated as the main peak, the first peak, and the second peak, respectively. The first amplitude ratio between the main peak and the first peak, and the second amplitude ratio between the main peak and the second peak are calculated. If the first amplitude ratio is greater than the second preset ratio threshold, only the main peak is retained as a valid peak.

[0030] If the first amplitude ratio is less than or equal to the second preset ratio threshold, then both the main peak and the first peak are retained. The retained peaks are arranged in ascending order of time position, and the time interval between two adjacent valid peaks is determined as the candidate action cycle.

[0031] Specifically, the sub-temporal implicit pose feature matrix is ​​dynamically time-warped and aligned with the pre-stored standard action template feature matrix, including:

[0032] For each effective action cycle, extract the sub-temporal implicit pose feature matrix within the range from the start frame index to the end frame index corresponding to the effective action cycle. The sub-temporal implicit pose feature matrix is ​​denoted as the matrix to be evaluated. The number of rows in the matrix to be evaluated is equal to the total number of video frames contained in the effective action cycle, which is denoted as the first row number. The number of columns in the matrix to be evaluated is equal to the implicit pose feature dimension.

[0033] Simultaneously, a standard action template feature matrix corresponding to the current ball action type is obtained from the pre-stored standard action template library, denoted as the standard template matrix. The number of rows in the standard template matrix is ​​equal to the number of reference frames of the standard action, denoted as the second row number. The number of columns in the standard template matrix is ​​also equal to the implicit pose feature dimension.

[0034] Obtain the normalized matrix to be evaluated and the standard template matrix.

[0035] Specifically, the dynamic time warping and alignment of the sub-temporal implicit pose feature matrix with the pre-stored standard action template feature matrix further includes:

[0036] Construct a dynamically time-warped distance matrix, wherein the number of rows in the distance matrix is ​​equal to the number of the first row of the normalized matrix to be evaluated, and the number of columns in the distance matrix is ​​equal to the number of the second row of the normalized standard template matrix; the element in the i-th row and j-th column of the distance matrix has a value equal to the Euclidean distance between the i-th row vector of the normalized matrix to be evaluated and the j-th row vector of the normalized standard template matrix.

[0037] Perform a dynamic time warping path search with boundary constraints and window constraints on the distance matrix to obtain the optimal warping path;

[0038] Based on the optimal regularization path obtained from the search, the normalized matrix to be evaluated is aligned with the normalized standard template matrix to form an aligned pairing sequence.

[0039] Calculate the arithmetic mean of the Euclidean distances between all paired points in the aligned pairing sequence.

[0040] Specifically, performing a dynamic time-warped path search with boundary constraints and window constraints on the distance matrix includes:

[0041] Construct a cumulative distance matrix, wherein the number of rows in the cumulative distance matrix is ​​the same as the number of rows in the distance matrix, and the number of columns in the cumulative distance matrix is ​​the same as the number of columns in the distance matrix; set the element of the first row and first column of the cumulative distance matrix to the element value of the first row and first column of the distance matrix;

[0042] For other positions in the cumulative distance matrix, the calculation is performed sequentially in ascending order of row index and column index. The cumulative distance value of each position is equal to the value of the distance matrix element corresponding to that position plus the minimum cumulative distance value of the predecessor position of that position. The predecessor position includes the left adjacent position, the upper adjacent position, and the upper left adjacent position of that position.

[0043] A window constraint is set, which limits the range of predecessor positions that can be calculated: for a position with row index u and column index q in the cumulative distance matrix, the position will only participate in the path search calculation if the absolute value of the difference between the row index u divided by the number of the first row and the column index q divided by the number of the second row does not exceed the preset window width parameter.

[0044] Specifically, performing a dynamic time-warped path search with boundary constraints and window constraints on the distance matrix further includes:

[0045] Under the window constraint, starting from the first row and first column of the cumulative distance matrix, the cumulative distance value at each row and column is calculated until the position where the row index is equal to the number of the first row and the column index is equal to the number of the second row is obtained as the minimum normalization cost.

[0046] Starting from the position in the cumulative distance matrix where the row index equals the first row number and the column index equals the second row number, backtrack to the first row and first column in the direction of decreasing cumulative distance value, record the position traversed each time backtracking, and construct the optimal regular path by arranging the positions in order from the starting point to the ending point.

[0047] Ball movement recognition methods based on temporal pose coding include:

[0048] Real-time acquisition of RGB video streams containing the ball-like movements of the target object, and decomposition of the RGB video stream into a continuous sequence of video frames in chronological order;

[0049] For each frame in the video frame sequence, a deep convolutional feature map is extracted. Global pooling is performed on the deep convolutional feature map along the channel dimension to obtain a one-dimensional implicit pose feature vector for the corresponding single frame image. The one-dimensional implicit pose feature vectors corresponding to each frame image are then concatenated in chronological order to construct a temporal implicit pose feature matrix. The rows of the temporal implicit pose feature matrix correspond to the time step, and the columns correspond to the implicit pose feature dimensions.

[0050] The Euclidean distance between the feature vectors corresponding to adjacent time steps in the temporal implicit attitude feature matrix is ​​calculated to generate a motion energy temporal curve. At the same time, local maxima points in the motion energy temporal curve with peak values ​​greater than a preset energy threshold are detected, and the time interval between two adjacent local maxima points is determined as a candidate action cycle. The duration of the candidate action cycle is calculated. When the duration falls within a preset cycle duration interval, the candidate action cycle is determined as a valid action cycle, and the start frame index and end frame index corresponding to the valid action cycle are recorded.

[0051] Within a preset statistical time window, the total number of valid action cycles is counted, and the total number is output as the frequency of ball-related actions of the target object.

[0052] For each valid action cycle, the sub-temporal implicit pose feature matrix within the range from the start frame index to the end frame index corresponding to the cycle is extracted. The sub-temporal implicit pose feature matrix is ​​dynamically time-aligned with the pre-stored standard action template feature matrix, and the average feature distance after alignment is calculated. When the average feature distance is less than the preset standard threshold, the corresponding ball action is determined to meet the standard; otherwise, the corresponding ball action is determined to deviate from the standard, and the number of action deviations is accumulated or the action deviation ratio is output.

[0053] Compared with the prior art, the beneficial effects of the present invention are:

[0054] This invention directly maps RGB video frames into one-dimensional implicit pose feature vectors through a temporal implicit pose coding module, avoiding the jitter and loss risks of traditional explicit keypoint detection under self-occlusion and lighting changes. The motion energy temporal curve constructed by the Euclidean distance of adjacent frame feature vectors can amplify the phase change points of high-frequency periodic movements, making it less sensitive to motion blur compared to optical flow methods. The action period detection module uses local maximum point detection and duration threshold dual screening to accurately define the start and end times of a single action. Even if there are differences in the amplitude and speed of different individuals' actions, it can robustly lock the effective period, thereby accurately counting the frequency of actions. The standardization evaluation module uses dynamic time warping to align the sub-matrices and standard templates within the effective period, eliminating matching errors caused by time axis scaling. This makes the standardization judgment rely only on the shape similarity of the implicit pose trajectory, rather than absolute speed or appearance. In summary, this system achieves real-time period analysis, frequency statistics, and standardization evaluation of high-frequency ball sports actions without the need for wearable devices and relying only on monocular RGB video, significantly improving cross-individual generalization ability and recognition stability in daily campus training scenarios. Attached Figure Description

[0055] Figure 1 This is a block diagram of the ball motion recognition system based on temporal pose coding according to Embodiment 1 of the present invention;

[0056] Figure 2 This is a logic diagram of Embodiment 1 of the present invention, which decomposes the RGB video stream into a continuous sequence of video frames in chronological order.

[0057] Figure 3 This is a flowchart of the ball action recognition method based on temporal pose coding in Embodiment 2 of the present invention. Detailed Implementation

[0058] Example 1

[0059] Please see Figure 1 The present invention provides an embodiment of a ball action recognition system based on temporal pose coding, comprising: a video stream acquisition module, a temporal implicit pose coding module, an action cycle detection module, an action frequency statistics module, and a standardization evaluation module;

[0060] The video stream acquisition module is configured to: acquire RGB video streams containing the ball-like movements of the target object in real time through the camera device of the AI ​​training machine, and decompose the RGB video streams into a continuous video frame sequence in chronological order;

[0061] In one specific embodiment of this example, to address the technical problems of key phases of high-frequency ball sports movements potentially falling into inter-frame gaps due to the limited frame rate of the camera, and the inconsistent index positions of the same movement phase in the frame sequence caused by differences in individual movement speeds, the video stream acquisition module employs the following steps to decompose the RGB video stream into a continuous sequence of video frames in chronological order. Please refer to [link to relevant documentation]. Figure 2 Specifically:

[0062] S1, receive RGB video streams captured by the AI ​​training machine's camera in real time. Each frame in the video stream is accompanied by a timestamp generated by the system clock, with the timestamp in milliseconds. Arrange all raw frames in ascending order of timestamps to form a raw frame sequence. Simultaneously, read the camera's nominal frame rate parameter and record this parameter as the raw frame rate, which is thirty frames per second.

[0063] S2, perform temporal motion compensation interpolation processing on the original frame sequence to increase the effective frame rate to a target frame rate of sixty frames per second. Specific operations include:

[0064] Iterate through each pair of adjacent frames in the original frame sequence, and designate the frame with the smaller timestamp as the forward reference frame and the frame with the larger timestamp as the backward reference frame.

[0065] A dense optical flow algorithm is used to calculate the motion vector field between the forward reference frame and the backward reference frame. The dense optical flow algorithm is either the Farneback algorithm or a dense version of the Lucas-Kanade algorithm. Each vector in the motion vector field corresponds to a pixel position in the forward reference frame, representing the horizontal and vertical displacement of that pixel position from the forward reference frame to the backward reference frame.

[0066] The number of intermediate frames to be inserted is determined based on the ratio of the original frame rate to the target frame rate. Since the original frame rate is 30 frames per second and the target frame rate is 60 frames per second, the ratio is 2 to 1. Therefore, an intermediate frame needs to be inserted between each pair of adjacent original frames.

[0067] The timestamp of the intermediate frame is determined by taking the arithmetic mean of the timestamps of the forward reference frame and the backward reference frame as the timestamp of the intermediate frame.

[0068] The pixel values ​​for generating intermediate frames are as follows:

[0069] For each pixel position in the intermediate frame, the pixel value is calculated as follows: First, based on the motion vector field, determine the position corresponding to the pixel position in the forward reference frame. This corresponding position is obtained by subtracting the displacement from the forward reference frame to the intermediate frame from the pixel position in the intermediate frame, where the displacement from the forward reference frame to the intermediate frame is equal to half of the corresponding vector in the motion vector field. A first pixel value is sampled from this corresponding position in the forward reference frame. Second, based on the motion vector field, determine the position corresponding to the pixel position in the backward reference frame. This corresponding position is obtained by adding the displacement from the backward reference frame to the intermediate frame from the pixel position in the intermediate frame, where the displacement from the backward reference frame to the intermediate frame is also equal to half of the corresponding vector in the motion vector field. A second pixel value is sampled from this corresponding position in the backward reference frame. Finally, the first pixel value and the second pixel value are averaged with equal weights (i.e., each is halved), and the average value is taken as the final pixel value for that pixel position in the intermediate frame. This process is repeated for all pixel positions in the intermediate frames to generate a complete intermediate frame image.

[0070] S3, merge all the original frames in the original frame sequence with all the intermediate frames generated by the above interpolation in ascending order of timestamps to form an extended frame sequence. The frame rate of this extended frame sequence is equal to the target frame rate, which is sixty frames per second. Since the frame rate is doubled, an intermediate frame is inserted exactly between each original frame interval, thus doubling the number of frames contained in each action cycle. This enables the capture of key action phases that might otherwise fall in the middle of the original frame intervals, such as the moment a basketball is hit or a volleyball is touched.

[0071] S4, each frame in the extended frame sequence is output sequentially according to the timestamp in ascending order, as the input video frame sequence for the subsequent temporal implicit pose coding module.

[0072] This embodiment increases the effective frame rate from 30 frames per second to 60 frames per second by performing temporal motion compensation interpolation on the original frame sequence. An intermediate frame, generated by weighting the motion vectors of preceding and following frames, is inserted between each original frame interval. This ensures that critical motion phases, such as the moment a basketball hits the ball or a volleyball touches the ball, which might otherwise fall within the original frame interval, are accurately captured, avoiding the loss of phase information caused by missing keyframes. Secondly, a dense optical flow algorithm is used to calculate the motion vector field, and the intermediate frame pixels are generated based on the equal weighted averaging of preceding and following frames. This ensures the motion continuity and image quality of the interpolated frames, doubling the number of frames per motion cycle and significantly improving the temporal sampling density. Furthermore, the inconsistency in the index position of the same phase in the original frame sequence caused by differences in motion speed among different individuals is addressed through interpolation expansion, resulting in a more uniform temporal representation. This provides the subsequent temporal implicit pose coding module with a consistent frame rate and well-aligned phase input video frame sequence. In summary, this embodiment effectively solves the problem of uneven sampling of key phases caused by insufficient camera frame rate and differences in individual movement speed, significantly enhances the positioning accuracy of the start and end frames of the movement cycle detection, and lays the temporal sampling foundation for the high robustness of the entire ball movement recognition system.

[0073] The temporal implicit pose coding module, which is signal-connected to the video stream acquisition module, is configured to: extract a deep convolutional feature map for each frame in the video frame sequence; perform a global pooling operation on the deep convolutional feature map along the channel dimension to obtain a one-dimensional implicit pose feature vector for the corresponding single frame image; and concatenate the one-dimensional implicit pose feature vectors corresponding to each frame image in temporal order to construct a temporal implicit pose feature matrix; wherein the rows of the temporal implicit pose feature matrix correspond to the time step, and the columns correspond to the implicit pose feature dimensions.

[0074] In this embodiment, for a ball-based motion recognition scenario, the inventors discovered that although the deep convolutional feature map extracted by the convolutional neural network contains rich semantic information, its spatial position dimensions (height and width) are highly sensitive to individual height differences, shooting distance changes, limb self-occlusion, and inter-frame spatial misalignment caused by rapid movement. Specifically, the same implicit pose pattern (such as pressing down an arm) may activate different spatial regions of the feature map in different frames. If the spatial structure is directly preserved for temporal modeling, the response difference of the same channel but different positions between adjacent frames will be misjudged as a pose change, thereby introducing a large number of noise peaks that are not actual motion cycles into the motion energy temporal curve. At the same time, the local fusion of the ball and the limb further leads to sparse and unstable spatial responses in some channels. Therefore, this embodiment extracts a deep convolutional feature map for each frame in the video frame sequence and performs a global pooling operation on the deep convolutional feature map along the channel dimension, including:

[0075] A1. Obtain the RGB image I of the current frame;

[0076] A2. Input the current frame RGB image I into a pre-trained convolutional neural network, which contains N convolutional layers; extract the deep convolutional feature map F output by the last convolutional layer in the convolutional neural network. The deep convolutional feature map F has a spatial height dimension H, a spatial width dimension W, and a channel dimension C, and the feature value at any position in the deep convolutional feature map F is denoted as F(i, j, c), i∈[1, H], j∈[1, W], c∈[1, C]. In this embodiment, to solve the underlying technical problem of unstable temporal motion feature extraction caused by spatial displacement, self-occlusion, and individual appearance differences in RGB video streams, the inventors have specifically constructed and pre-trained the convolutional neural network used to extract the deep convolutional feature map. Specifically, the construction and pre-training process of the convolutional neural network includes the following:

[0077] A21. In this embodiment, a lightweight network with a depthwise separable convolutional structure is selected as the backbone network. The backbone network is either the MobileNetV3-Large network or the EfficientNet-Lite network. The backbone network contains 30 convolutional layers, numbered sequentially from the 1st to the 30th convolutional layer. The 1st convolutional layer is a regular two-dimensional convolutional layer, and the remaining 29 convolutional layers are depthwise separable convolutional layers. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function. The batch normalization layer is used to normalize the convolutional output, and the ReLU activation function is used to introduce a nonlinear transformation.

[0078] The backbone network is used to extract spatial feature maps from the input single-frame RGB image, which has dimensions of 384 pixels in height, 384 pixels in width, and 3 channels. To balance computational efficiency and semantic abstraction capabilities, the backbone network employs the following downsampling strategy: the stride is set to 2 in the 1st, 3rd, 7th, 11th, 16th, 22nd, and 28th convolutional layers, while the stride is set to 1 in the remaining convolutional layers. These seven strides... With a stride of 2, the spatial dimensions of the feature maps decrease sequentially to 1 / 2, 1 / 4, 1 / 8, 1 / 16, 1 / 32, 1 / 64, and 1 / 128 of the input image size. After the 28th convolutional layer, the spatial dimensions of the feature maps are 1 / 128 of the original input image size, i.e., 3 pixels in height and 3 pixels in width. To prevent excessive compression of spatial information, this embodiment does not use convolutions with a stride of 2 in the 29th and 30th convolutional layers, but instead uses depthwise separable convolutions with a stride of 1. Therefore, the spatial dimensions of the feature maps output by the last convolutional layer, the 30th convolutional layer, are 1 / 32 of the original input image size, i.e., 12 pixels in height and 12 pixels in width.

[0079] Simultaneously, the output channel number of the last convolutional layer is set to 256, meaning the 30th convolutional layer uses 256 convolutional kernels, each with a size of 3×3. This results in an output depthwise convolutional feature map with dimensions of 12 (height), 12 (width), and 256 channels. This feature map retains complete information about the channel dimensions, where each channel corresponds to an implicit pose response pattern. These response patterns include, but are not limited to, activation responses to upper limb contours, sphere positions, and limb joint angles.

[0080] The mapping relationship between the input and output of the backbone network in this embodiment is expressed as: F = Φ(I), where I is the input single-frame RGB image with a size of 384×384×3, Φ is the composite function consisting of the above 30 convolutional layers, the subsequent batch normalization layer, and the ReLU activation function, and F is the output deep convolutional feature map with a size of 12×12×256. This feature map will serve as the input to the subsequent spatial attention calibration module.

[0081] A22. Obtain a publicly available large-scale human motion video dataset, either the NTURGB+D dataset or the KineticA-400 dataset. This dataset contains multiple video segments, each segment broken down into single-frame RGB images. Each frame corresponds to a labeling file, which records the coordinates of seventeen two-dimensional joints of the human body in that frame, as well as the motion category label to which that frame belongs.

[0082] The constructed backbone network was used as a pre-trained model. Its input layer received an RGB image of size 224×224×3, and its output layer output a feature map with a spatial size of 7×7 and 256 channels. The backbone network was pre-trained under supervision using the training set in the dataset, as follows:

[0083] A221, randomly sample a batch of images from the training set, with each batch containing 32 images, and associate each image with its corresponding action category label.

[0084] A222: Each image is input into the backbone network, and the output feature map is obtained through forward propagation. Then, global average pooling is performed on the output feature map to obtain a one-dimensional feature vector of length 256. This one-dimensional feature vector is input into a fully connected classification layer. The number of output nodes of the fully connected classification layer is equal to the total number of action categories in the dataset. The Softmax function outputs the predicted probability distribution of the image belonging to each category.

[0085] A223, the cross-entropy loss function is used to calculate the loss value between the predicted probability distribution and the true action category label. The formula for calculating the cross-entropy loss function is L=-∑y_a log(p_a), where y_a is the a-th element in the one-hot encoding of the true label, and p_a is the a-th element in the predicted probability distribution.

[0086] A224 uses a stochastic gradient descent optimizer to update the trainable parameters of the backbone network and the fully connected classification layer. The initial learning rate is set to 0.01, the momentum parameter is set to 0.9, and the weight decay coefficient is set to 0.0001. Each complete traversal of a training set is counted as one cycle, and a total of fifty cycles of iterative training are performed.

[0087] A225 calculates the classification accuracy using the validation set after each cycle. When the validation accuracy no longer increases for five consecutive cycles, training stops and the backbone network parameters for the current cycle are saved.

[0088] A226, after pre-training, the kernel weights of the last three convolutional layers near the output end in the backbone network have converged to a stable state. When any frame of image is input into the pre-trained backbone network, the deep convolutional feature map output by the backbone network has the following characteristics: each channel response map of the feature map presents a spatial activation region corresponding to a specific motion primitive. The motion primitive includes the activation region corresponding to the rotation angle of the human forearm in the image plane, the activation region corresponding to the vertical downward pressing action of the wrist, and the activation region corresponding to the collision position of the ball with the ground or limb. In this embodiment, the position of the activation region in the feature map spatial coordinate system is translated as a whole as the position of the human body in the image changes, but the correspondence between its channel index and the motion primitive type remains fixed.

[0089] A23. Due to the self-occlusion of limbs and the partial fusion of the ball and limbs in ball games, the activation regions of some channels in the depthwise convolutional feature map output by the last convolutional layer exhibit sparse, discrete, or noise responses that deviate from the actual limbs. Therefore, this embodiment adds a spatial attention calibration module after the last convolutional layer of the backbone network. The spatial attention calibration module performs the following operations:

[0090] A231 receives the depthwise convolutional feature map F output from the last convolutional layer, as the initial feature map, denoted as F. init According to the settings in the first step, F init The spatial height H is 12, the spatial width W is 12, and the number of channels C is 256, i.e., F init The dimension is 12×12×256, where F init The eigenvalue located in the i-th row, j-th column, and c-th channel is denoted as F. init (i, j, c), where i and j range from 1 to 12, and c ranges from 1 to 256.

[0091] A232, the spatial attention calibration module calculates F respectively. init The global average and global maximum values ​​along the channel dimension yield two intermediate feature maps, each with a spatial size of 12×12. Specifically:

[0092] For each spatial location (i, j), calculate the average value of that location across all 256 channels to obtain the global average feature map. M avg The dimension is 12×12.

[0093] For each spatial location (i, j), calculate the maximum value at that location across all 256 channels to obtain the global maximum feature map. M max The dimension is 12×12.

[0094] A233, the global average feature map M avg With the global maximum feature map M max The pieces are spliced ​​along the channel direction to form a spliced ​​feature map with dimensions of 12×12×2, denoted as M. concat M concat (i, j, 1) = M avg (i, j), M concat (i, j, 2) = M max (i, j).

[0095] A234, the spliced ​​feature map M concat Input a convolutional layer with a kernel size of 1×1, 2 input channels, and 1 output channel. This convolutional layer contains one learnable convolutional kernel with weight parameters W. att The bias parameter is b_att; after convolution, an intermediate output of size 12×12×1 is obtained, denoted as A. raw (i, j) = W att (1, 1, 1, 1) × M concat (i, j, 1) + W att (1, 1, 1, 2) × M concat (i, j, 2) + b_att, where W att The four indices represent the output channel index, input channel index, kernel height index, and kernel width index, respectively.

[0096] A235, A raw Inputting a sigmoid activation function generates a spatial attention weight map A(i,j), where A(i,j) = 1 / (1+exp(-A)). raw (i, j))). The spatial size of A is 12×12, and the value range of each element in A is an open interval (0, 1).

[0097] A236, the initial feature map F init Element-wise multiplication with the spatial attention weight map A(i,j) yields the calibrated feature map F. calibrated Its dimensions are 12×12×256; the specific calculation formula is: F calibrated (i, j, c) = F init (i, j, c) × A(i, j), where i takes values ​​from 1 to 12, j takes values ​​from 1 to 12, and c takes values ​​from 1 to 256.

[0098] Through the spatial attention calibration module described above, this embodiment achieves adaptive spatial reweighting of the initial depthwise convolutional feature map. Specifically, for the initial feature map F... init For each spatial location (i, j) in the calibration, the spatial attention weight map A(i, j) ranges from 0 to 1. When A(i, j) approaches 0, it indicates that there is false spatial activation at that location due to limb self-occlusion or local fusion of the sphere and limb. In this case, the calibration operation suppresses the feature responses of all channels at that location to a level close to zero, thereby effectively filtering out noise interference. When A(i, j) approaches 1, it indicates that the location corresponds to a critical spatial region related to the stability of the motion cycle. In this case, the calibration operation preserves the feature responses of all channels at that location, thereby enhancing the expression of effective motion information. The feature map F obtained after calibration is... calibrated This approach preserves the channel activation patterns strongly correlated with periodic motion in the original feature map while significantly reducing spatial outlier responses caused by occlusion and fusion. In this embodiment, F calibrated This will serve as the direct input for subsequent global average pooling operations to generate a one-dimensional implicit attitude feature vector that is invariant to spatial displacement, thus laying a reliable feature foundation for accurately constructing motion energy time-series curves.

[0099] A24. To enable the backbone network and spatial attention calibration module of this embodiment to jointly learn implicit features that are insensitive to inter-frame spatial displacement but sensitive to pose pattern changes, this embodiment constructs triplet training samples for secondary pre-training. Specifically, the following operations are performed, and the parameters of the lower-level convolutional layers of the backbone network are frozen during training:

[0100] A241. Training samples are extracted from the campus ball game video data with labeled action cycles. Each training sample consists of three frames, denoted as the first positive sample frame, the second positive sample frame, and the negative sample frame. The first and second positive sample frames are taken from consecutive video frames of the same action cycle, which is a complete cycle of basketball dribbling or volleyball passing as determined in the previous steps. The time interval between the first and second positive sample frames does not exceed 5 frames. The negative sample frames are taken from one of the following two cases: First, from an action cycle different from the positive sample frame, with at least two complete cycles between the two action cycles; second, from any frame in the video frames of different individuals performing the same type of action that is not in the same time window as the positive sample frame.

[0101] A242 inputs the first positive sample frame, the second positive sample frame, and the negative sample frame into a model consisting of a backbone network and a spatial attention calibration module connected in series. The model first passes through a backbone network consisting of 30 convolutional layers, outputting an initial feature map with a dimension of 12×12×256, and then passes through the spatial attention calibration module, outputting a calibrated feature map with a dimension of 12×12×256.

[0102] A243. Perform global average pooling on the calibrated feature maps corresponding to the three frames of images mentioned above. The specific method of global average pooling is as follows: For a calibrated feature map with dimensions of 12×12×256, calculate the arithmetic mean of the feature values ​​at 144 positions in 12 rows and 12 columns in each channel to obtain a scalar; arrange the scalars of the 256 channels in channel index order to form a one-dimensional feature vector of length 256. This yields the first positive sample feature vector, the second positive sample feature vector, and the negative sample feature vector, each with a length of 256.

[0103] A244 defines a triplet constraint condition; the constraint condition requires that: the Euclidean distance between the first positive sample feature vector and the second positive sample feature vector is less than a first preset threshold, the first preset threshold being 0.2; at the same time, the Euclidean distance between the first positive sample feature vector and the negative sample feature vector is greater than a second preset threshold, the second preset threshold being 0.8; and the Euclidean distance between the second positive sample feature vector and the negative sample feature vector is also greater than the second preset threshold of 0.8.

[0104] A245 uses a triplet loss function for optimization. The value of this loss function is determined as follows: First, calculate the Euclidean distance between the first positive sample feature vector and the second positive sample feature vector, denoted as the first distance; then calculate the Euclidean distance between the first positive sample feature vector and the negative sample feature vector, and the Euclidean distance between the second positive sample feature vector and the negative sample feature vector, respectively, and take the smaller value as the second distance; then subtract the second distance from the first distance, and add a preset interval parameter, which is set to 0.5; if the above calculation result is greater than 0, the value of the loss function is equal to the calculation result; if the above calculation result is less than or equal to 0, the value of the loss function is equal to 0. In this embodiment, the interval parameter is set to 0.5 because: the goal of the triplet loss function is to force the Euclidean distance between the first positive sample feature vector and the second positive sample feature vector to be significantly smaller than their respective Euclidean distances with the negative sample feature vectors, thereby forming tight intra-class clustering and clear inter-class separation; the interval parameter, as a forced distance margin, has a larger value, and the stricter the requirement for inter-class separation. However, when the value is too large, it will cause the network to have difficulty converging to a stable state within the preset training period, and the distribution of feature vectors in the embedding space will diverge, thus affecting the consistency of subsequent period detection and standard evaluation; conversely, if the interval parameter is too small, the constraints of intra-class tightness and inter-class separation will tend to weaken, making the inter-frame features within the same action period and different action periods more distinct. The overlapping and confusion of features in distance measurement weakens the discriminative power of implicit pose features for action phase changes. For example, in this embodiment, combined with the distribution characteristics of the dynamic range of Euclidean distance after the normalization of the magnitude of the 256-dimensional implicit pose feature vector between 0 and 2, and the prior observation that the inter-frame distance in the same period of a campus ball game action scene usually converges to below 0.2 and the inter-frame distance across periods usually stretches to above 0.8, after multiple rounds of parameter grid search verification, the interval parameter is set to 0.5. This ensures that the training process converges stably in the 15th to 20th period, and also retains sufficient buffer margin for inter-class distance, so that there is a clear boundary of at least 0.5 between intra-class distance and inter-class distance in the feature space, thereby achieving the optimal balance between training stability and feature discriminative power.

[0105] A246 employs an adaptive moment estimation optimizer to update the parameters of the last 5 convolutional layers in the backbone network and all parameters of the spatial attention calibration module, while keeping the parameters of the first 25 convolutional layers in the backbone network frozen. The initial learning rate is set to 0.001, and the learning rate decays to 0.9 times the current value after every 10 training epochs, for a total of 30 training epochs.

[0106] In the secondary pre-training process of this embodiment, through the constraint optimization of the triplet loss function, the network is forced to learn an implicit feature mapping that is insensitive to spatial displacement but sensitive to pose changes: when any two frames belong to the same action cycle, even if the original image is misaligned due to individual height, shooting distance or limb spatial displacement, the Euclidean distance of its 256-dimensional feature vector is still compressed to below 0.2, achieving high intra-class compactness; conversely, for frames of different cycles or different individuals, even if the overall contours are similar, their feature distance is increased to above 0.8, achieving clear inter-class separation, thereby effectively eliminating feature confusion caused by appearance differences. Based on this, to mitigate the risk of overfitting caused by limited labeled data in campus scenes, this embodiment implements a segmented parameter freezing strategy throughout the pre-training and deployment process: In the secondary pre-training stage, the first 25 layers of the backbone network's 30 convolutional layers are frozen, and only the parameters of the last 5 convolutional layers and the spatial attention calibration module are updated. This allows high-level features to quickly adapt to specific patterns of target actions such as basketball dribbling and volleyball passing while retaining low-level general visual primitives. In the subsequent online fine-tuning stage, this frozen configuration is continued. For example, when the system is deployed on an AI training machine for real-time analysis of students' volleyball passing actions, the weights of the first 25 layers of the backbone network remain unchanged. Only the last 5 layers and the calibration module are specifically fine-tuned, which allows the model to converge quickly in new lighting environments and in groups of students not seen, without disrupting the original compact intra-class and far-reaching feature space structure due to a small number of labeled samples. This ensures that the model maintains high recognition and generalization ability across individuals and scenarios.

[0107] A25. Through the complete process of constructing the backbone network, introducing the spatial attention calibration module, and performing secondary pre-training with temporal consistency of triples, the convolutional neural network obtained in this embodiment possesses the ability to stably extract depth motion features from a single frame of RGB image. Specifically, for any input image frame, the network sequentially performs the following mapping: extracting the initial depth convolutional feature map through a thirty-layer convolutional neural network, then performing adaptive spatial reweighting through the spatial attention calibration module, and outputting the calibrated feature map F. calibratedThe feature map has dimensions of 12×12×256. Subsequently, a global average pooling operation is performed on this feature map, which calculates the arithmetic mean of the feature values ​​at 144 positions (12 rows, 12 columns) in each channel, resulting in a one-dimensional implicit pose feature vector of length 256. Because the spatial attention calibration module effectively suppresses spatial noise caused by limb self-occlusion, sphere fusion, and individual appearance differences, and because triplet pre-training forces the network to learn a mapping relationship that is insensitive to spatial displacement but sensitive to pose cycle changes, the final generated one-dimensional feature vector is naturally invariant to inter-frame spatial displacement, local occlusion, and appearance differences such as height and clothing among individuals. The Euclidean distance between adjacent frames in this feature vector only reflects the intensity change of the overall pose pattern, rather than spatial position offsets or appearance interference, thus laying a solid foundation for subsequently constructing clean, low-noise motion energy time-series curves and ensuring the robustness and generalization ability of motion cycle detection and standardization evaluation.

[0108] A3. For each channel index c in the channel dimension C, calculate the global average of the feature values ​​at all spatial locations (i, j) within the channel, and use this average as the scalar pooling value A for the corresponding channel. c ;

[0109] A4. Convert the scalar pooling value A corresponding to each of the C channels. c Arranged in ascending order according to channel index c, they form a one-dimensional implicit pose feature vector v for the current frame, where v = [A1, A2, ..., A...]. c …, A C The length of vector v is equal to C.

[0110] In this embodiment, for ball-related motion recognition scenarios, a lightweight, deep separable convolutional backbone network adapted to high-frequency ball-related motions is first constructed. This is coupled with a hierarchical, controllable downsampling strategy. While maintaining real-time computational efficiency, this avoids excessive compression of spatial information, ensuring that each channel of the output deep convolutional feature map forms a fixed correspondence with specific motion primitives such as upper limb movements and ball positions. Then, supervised pre-training is performed using a publicly available large-scale human motion dataset. This allows the lower convolutional layers of the backbone network to learn general visual primitives such as edges and contours, while the higher convolutional layers converge to channel response patterns strongly correlated with the motion cycle, guaranteeing the basic generalizability of feature extraction. The system enhances the spatial attention capabilities of the network. Subsequently, a spatial attention calibration module is added at the end of the backbone network. A spatial attention weight map is generated by concatenating the global average and global maximum features of the channel dimension. This adaptively reweights the initial feature map spatially, effectively suppressing spurious spatial activation and outlier noise responses caused by limb self-occlusion and sphere-limb fusion, while fully preserving key spatial region features strongly correlated with periodic motion. This achieves spatial noise filtering and effective motion information enhancement of the feature map. Furthermore, a triplet training sample is constructed, with consecutive frames within the same action period as positive samples and frames across periods / individuals as negative samples. The triplet loss function is then used to freeze the low-level parameters. The high-level convolutional layers of the backbone network and the spatial attention calibration module are pre-trained twice. This process not only preserves general visual primitives by freezing low-level parameters and avoids overfitting in small-labeled samples in campus scenes, but also forces the network to learn feature maps that are insensitive to inter-frame spatial displacement, individual height, clothing, and other appearance differences, but highly sensitive to changes in pose patterns. This ensures that inter-frame features within the same action cycle remain highly compact, and features across cycles and individuals are effectively separated, eliminating feature confusion caused by spatial misalignment and individual differences at the root. Finally, global average pooling is performed along the channel dimension on the calibrated feature maps to generate a one-dimensional implicit pose. The feature vector inherently possesses spatial displacement invariance. The Euclidean distance between adjacent frames can only reflect the intensity change of the true overall posture pattern, rather than noise fluctuations caused by spatial position offset, local occlusion, individual appearance differences, or environmental interference. This provides a solid and reliable feature foundation for the subsequent construction of low-noise, high-recognition motion energy time-series curves. It significantly reduces the introduction of non-motion-related noise at the underlying feature level, effectively improving the accuracy of subsequent motion cycle detection, the robustness of motion standard evaluation, and the real-time recognition performance and engineering adaptability of the entire system across individuals, shooting scenes, and complex motion environments.

[0111] The motion cycle detection module, signal-connected to the temporal implicit attitude encoding module, is configured to: calculate the Euclidean distance between the feature vectors corresponding to adjacent time steps in the temporal implicit attitude feature matrix, and generate a motion energy temporal curve; detect local maxima points in the motion energy temporal curve where the peak value is greater than a preset energy threshold, and determine the time interval between two adjacent local maxima points as candidate motion cycles; calculate the duration of the candidate motion cycle, and when the duration falls within a preset cycle duration interval, determine the candidate motion cycle as a valid motion cycle, and record the start frame index and end frame index corresponding to the valid motion cycle;

[0112] In this embodiment, the generation process of the motion energy temporal curve is as follows: First, the temporal implicit pose feature matrix constructed by the temporal implicit pose encoding module is obtained. The rows of this matrix correspond to the time step, and the columns correspond to the implicit pose feature dimensions. For each pair of adjacent time steps t and t+1 in the temporal implicit pose feature matrix, the Euclidean distance between the feature vector in row t and the feature vector in row t+1 is calculated to obtain an original distance value. The original distance values ​​corresponding to all adjacent time steps are arranged in chronological order to form an original distance sequence. Subsequently, a sliding window mean filtering process is performed on the original distance sequence to suppress high-frequency noise components introduced by individual motion jitter, camera vibration, or changes in illumination. The width of the sliding window is adaptively determined according to the type of ball action to be identified and the spectral characteristics of the video stream. After the sliding window mean filtering process, a smoothed motion energy sequence is obtained. The energy value at each time position index in this sequence constitutes the motion energy temporal curve. Each energy value in the motion energy temporal curve represents the overall change intensity of the implicit pose features between corresponding adjacent frames.

[0113] In this embodiment, the preset cycle duration interval is pre-set according to the ball game action type corresponding to the current video stream. The setting is based on the reasonable fluctuation range of the single action cycle duration of the corresponding action type under standard execution conditions. For example, when the preset action type identification result determines that the current video stream is a basketball dribbling action, given that the single cycle duration of a standard basketball dribbling action is typically between 0.4 seconds and 1.0 seconds, the system sets the preset cycle duration interval to the time length corresponding to this range. When the preset action type identification result determines that the current video stream is a volleyball passing action, given that the single cycle duration of a standard volleyball passing action is typically between 0.5 seconds and 1.2 seconds, the system sets the preset cycle duration interval to the time length corresponding to this range. By pre-setting corresponding cycle duration intervals for different ball game action types, the system determines candidate action cycles whose duration exceeds the interval as invalid cycles caused by noise or false detection and discards them. Only candidate action cycles whose duration falls within the preset cycle duration interval are determined as valid action cycles, thereby improving the accuracy and robustness of action cycle detection.

[0114] In this embodiment, it was further discovered that when calculating the Euclidean distance between the feature vectors corresponding to adjacent time steps in the temporal implicit pose feature matrix and generating the motion energy temporal curve, the following specific technical problems exist: the cycle frequency of ball sports such as basketball dribbling or volleyball passing is relatively high, usually 2 to 4 times per second. The amplitude of the Euclidean distance of the feature vector corresponding to the actual pose change between adjacent frames is small, while the noise introduced by individual motion jitter, slight camera vibration, and lighting changes will also produce distance fluctuations with similar amplitudes. If the original Euclidean distance sequence is directly used as the motion energy temporal curve, it is difficult to distinguish the noise peak from the actual motion cycle peak in terms of amplitude, which makes it easy to produce false peaks or miss the actual peaks when detecting local maxima. In addition, the feature vector change rates corresponding to the compression phase and the rebound phase are inconsistent in different stages within the same motion cycle, causing the distance values ​​of adjacent frames to exhibit an asymmetrical distribution within the cycle, further interfering with the peak discrimination based on a fixed energy threshold. Therefore, how to preserve the abrupt changes in periodic motion energy while suppressing non-motion-related noise distance fluctuations and adapting to the non-uniformity of the rate of change within the period has become a key technical challenge in accurately generating motion energy time-series curves to support subsequent effective period detection. To this end, this embodiment calculates the Euclidean distance between the eigenvectors corresponding to adjacent time steps in the temporal implicit attitude feature matrix to generate motion energy time-series curves, including:

[0115] B1. Obtain the temporal implicit pose feature matrix M, wherein the number of rows in matrix M is the total number of frames T in the video frame sequence, and the number of columns is the dimension C of the implicit pose feature vector; the vector in the t-th row of matrix M is denoted as v. t Where t ranges from 1 to T, and each v tThe length of is equal to C;

[0116] B2. For adjacent time steps t and t+1, calculate the t-th row vector v. t With the (t+1)th row vector v t+1 The Euclidean distance d between them t This constitutes the original distance sequence;

[0117] B3. Perform sliding window mean filtering on the original distance sequence, wherein the width of the sliding window is set to Q consecutive elements. Further, in a specific embodiment, the sliding window width Q is set adaptively, and its specific value is dynamically determined based on the type of ball action to be identified and the spectral characteristics of the real-time acquired video stream. For basketball dribbling actions, the system presets a first window width upper limit of five; for volleyball passing actions, the system presets a second window width upper limit of seven. Before the formal recognition begins, the system acquires the original distance sequence corresponding to two hundred consecutive video images within the initial time period, denoted as the initial original distance sequence. A fast Fourier transform is performed on this initial original distance sequence to obtain the corresponding amplitude spectrum, and the frequency component corresponding to the largest amplitude value is searched from this amplitude spectrum. This frequency component is determined as the dominant frequency, denoted as the dominant frequency value. Simultaneously, the sampling frame rate of the AI ​​training machine camera is read and denoted as the sampling frame rate value. Subsequently, the system calculates the adaptive window width based on the sampled frame rate and the main frequency value. Specifically, it calculates the quotient of the sampled frame rate and twice the main frequency value, rounds the quotient to the nearest integer, and uses the rounded result as the initial value of the adaptive window width. Then, when the identified action type is basketball dribbling, the system limits the final value of the adaptive window width to no more than the upper limit of the first window width; when the identified action type is volleyball passing, the system limits the final value of the adaptive window width to no more than the upper limit of the second window width. Through the above adaptive settings, the passband range of the sliding window mean filter is configured to cover the frequency band where the main frequency component of the action is located, while effectively suppressing noise components with frequencies higher than twice the main frequency. Thus, while preserving the true periodic energy change characteristics, it filters out high-frequency noise interference introduced by individual motion jitter, camera vibration, or changes in lighting to the greatest extent. In this embodiment, for basketball dribbling, the system presets the upper limit of the first window width to be five. The reason for this preset value is that basketball dribbling is a high-frequency periodic motion. A single dribbling cycle includes two stages: pressing down on the ball with the palm and the ball rebounding back to the hand. The frequency of the action is usually two to four times per second, and the Euclidean distance fluctuation of the feature vector corresponding to the actual posture change between adjacent frames is relatively dense. If the sliding window width is set too large, exceeding five consecutive elements, the mean filtering operation will excessively smooth the energy peaks of two adjacent action cycles, causing the peaks that should be separated in the motion energy time sequence curve to stick together, which will lead to missed period detection when detecting local maxima points in the later stage. For example, according to the test of basketball dribbling video samples collected in a campus scene in this embodiment, when the sliding window width exceeds five frames, the peaks of adjacent dribbling cycles begin to show an observable fusion trend, the cycle boundaries are blurred, and the accuracy of action counting decreases significantly. Therefore, in order to effectively suppress the high-frequency noise introduced by individual motion jitter and camera vibration while fully preserving the peak separation characteristics between adjacent cycles in basketball dribbling, the upper limit of the first window width is preset to five.

[0118] In this embodiment, for the volleyball passing action, the system presets the upper limit of the second window width to seven. This preset value is based on the following: a single cycle of a volleyball passing action includes three phases: arm receiving the ball, the moment of impact, and the follow-through. The peak energy amplitude at the moment of impact is significantly higher than the secondary peaks in the receiving and follow-through phases, and the time interval between the secondary and primary peaks exhibits an uneven distribution with shorter intervals at the beginning and longer intervals at the end. The overall cycle length is relatively longer than that of a basketball dribbling action. If the sliding window width is set too small, less than seven consecutive elements, the mean filter's noise suppression capability is insufficient, and the low-amplitude secondary peak areas in the receiving and follow-through phases will still be present. There are many residual high-frequency noise fluctuations, which are easily misjudged as independent valid peaks during subsequent local maximum point detection, causing a single action cycle to be incorrectly segmented into multiple false cycles. For example, based on the volleyball passing video samples collected in a campus scene in this embodiment, when the sliding window width is less than seven frames, the noise spikes in the secondary peak region of the motion energy time sequence curve increase significantly, and the false peak detection rate increases. Therefore, in order to effectively filter out the noise fluctuations in the secondary peak region while retaining the hierarchical time sequence relationship between the main peak and the secondary peak in the volleyball passing action, the upper limit of the second window width is preset to seven.

[0119] B4. For the k-th starting position in the original distance sequence, take Q elements from the k-th element to the k+Q-1 element, and calculate the arithmetic mean of these Q elements to obtain the k-th smoothed motion energy value; the value of k ranges from 1 to T-Q, thus obtaining the smoothed motion energy sequence. Further, in a specific implementation of this embodiment, for the boundary region processing of the original distance sequence, the system uses a mirror-fill method to expand the sequence. Specifically, when the starting position index k is less than the sliding window width Q, or when the starting position index k is greater than the total number of frames T minus the window width Q plus one, the sliding window mean filtering is not directly performed on the original distance sequence. In this case, the system first performs boundary expansion on the original distance sequence: for the Q-1 element at the beginning of the sequence, this Q-1 element is copied in reverse order and added before the starting position of the sequence; for the Q-1 element at the end of the sequence, this Q-1 element is copied in reverse order and added after the end of the sequence. After completing the mirror filling, the expanded original distance sequence is obtained. Then, a sliding window mean filter is applied to this expanded sequence. After filtering, the middle part with the same length as the original distance sequence is truncated as the smoothed motion energy sequence, thus ensuring that the length of the output sequence is consistent with the length of the original distance sequence, effectively avoiding the peak loss problem caused by boundary elements not being able to participate in the filtering.

[0120] B5. Output the smoothed motion energy sequence as the final motion energy time-series curve. The energy value at each moment in the curve represents the overall change intensity of the implicit attitude between adjacent frames. The high-frequency noise components introduced by individual motion jitter, camera vibration and lighting changes have been suppressed by sliding window mean filtering. At the same time, the difference in the rate of change of different stages within the same motion cycle still retains its main peak characteristics after smoothing.

[0121] In this embodiment, for high-frequency ball-related motion recognition, the original distance sequence is obtained by first calculating the Euclidean distance between adjacent time-step feature vectors in the temporal implicit pose feature matrix. Then, a sliding window mean filter, adaptively adjusted by the motion's dominant frequency, is used to process the original distance sequence. The dominant frequency of the motion is obtained by performing a Fast Fourier Transform on the original distance sequence in the initial time period. The appropriate window width is calculated based on the camera's sampling frame rate, and corresponding upper limits for the window width are set for different ball-related motions such as basketball dribbling and volleyball passing. This ensures that the filter passband accurately covers the frequency band where the dominant frequency component of the motion is located, effectively suppressing high-frequency noise components with frequencies higher than twice the dominant frequency. While fully preserving the core features of energy mutations in the real motion cycle, this method filters out noise interference similar to the real motion amplitude introduced by individual motion jitter, minor camera vibrations, and lighting changes to the greatest extent possible. Simultaneously, boundary expansion processing using mirror filling is employed. This invention solves the problem of edge peak loss caused by the incomplete participation of boundary elements at the beginning and end of the sequence in filtering, ensuring that the length of the output motion energy sequence is consistent with that of the original distance sequence. It avoids the loss of key features at the beginning and end of the action cycle. The final output motion energy time-series curve effectively suppresses non-action-related noise and distance fluctuations, while fully preserving the core peak features of the asymmetric distribution of change rates at different stages within the same action cycle. This significantly improves the accuracy of subsequent local maximum point detection, fundamentally reducing the problems of false peak detection and true peak omission. It provides reliable data support for the accurate determination of effective action cycles, thereby comprehensively improving the cycle positioning accuracy, anti-interference ability in complex environments, and recognition robustness of the entire ball action recognition system. At the same time, it can adaptively adapt to the frequency characteristics of different types of high-frequency ball actions, and has stronger scene adaptability and engineering versatility.

[0122] In this scenario, the step of "detecting local maxima in the motion energy time-series curve where the peak value is greater than a preset energy threshold, and determining the time interval between two adjacent local maxima as candidate action cycles" faces the following specific technical problems: The motion energy time-series curves of basketball dribbling and volleyball passing actions are fundamentally different. In basketball dribbling, the peak energy amplitudes of the two phases—the downward pressure on the ball and the ball rebounding back—are similar, and the waveform exhibits a symmetrical double-peak structure with approximately equal amplitude and spacing. However, in volleyball passing, the energy change rates of the three phases—the arm welcoming the ball, the instant of impact, and the follow-through—are different. The peak amplitude at the instant of impact is much higher than the secondary peaks in the welcoming and follow-through phases, and the peak intervals show an uneven distribution with shorter intervals at the beginning and longer intervals at the end. If a single fixed preset energy threshold is used, for basketball dribbling, two equal-amplitude peaks within a cycle may be misjudged as two independent cycles, leading to over-segmentation. For volleyball passing, a threshold that is too high may miss low-amplitude welcoming peaks, or a threshold that is too low may misjudge fluctuations near the main peak as multiple peaks. Furthermore, differences in the amplitude and speed of movements among different individuals within the same movement type, as well as movement deformation due to fatigue, can cause dynamic drift in peak amplitude and interval. Therefore, it should be further explained that this embodiment defines the time interval between two adjacent local maxima as the candidate movement cycle, including:

[0123] C1. Obtain the smoothed motion energy time-series curve, denoted as E. The curve E contains M1 discrete energy values, where M1 is equal to the total number of frames T in the video frame sequence minus the sliding window width Q. Each energy value corresponds to a time position index m, and the value of m ranges from 1 to M1.

[0124] C2. Perform local maximum point detection on the motion energy time series curve E to obtain the initial peak sequence P. init Each peak point is denoted as p, which includes the peak position index pos(p) and the peak amplitude amp(p). The detection method is as follows: for each time position index m, if E(m) is greater than its left adjacent value E(m-1) and greater than its right adjacent value E(m+1), then m is determined as a local maximum point. E(m) represents the energy value of the motion energy time series curve at the time position index m. In a specific embodiment of this example, when performing local maximum point detection on the motion energy time series curve E, it further includes: processing the flat peak region formed by continuous equal energy values. Specifically, when there are multiple consecutive time position indices whose corresponding energy values ​​are equal to each other, and the equal energy values ​​are simultaneously greater than the energy values ​​of their left adjacent position and their right adjacent position, the continuous equal region is regarded as a local maximum region. At this time, the system first determines the starting index of the continuous equal region, denoted as m. start And the ending index, denoted as m end , where m end Greater than mstart Then, calculate the index between the starting index and the ending index. The calculation method is as follows: [Calculate m...] start With m end The sum is divided by two, and the result is rounded down or up. The rounded value is used as the index of the local maximum point. Simultaneously, any energy value within this continuous equal region is used as the amplitude of that local maximum point. This process avoids generating multiple duplicate local maxima in flat peak regions, ensuring that each action energy peak corresponds to only one unique maximum point. This provides reliable peak position and amplitude information for the accurate division of subsequent candidate action cycles.

[0125] C3. Based on the preset action type recognition result, determine the ball action type corresponding to the current video stream. Specifically: The system pre-stores action type identifier parameters in the configuration file. These parameters are selected by the user through a graphical configuration interface before system startup, choosing between the basketball dribbling option and the volleyball passing option. When the system starts, the system initialization program reads the action type identifier parameters from the configuration file and loads their values ​​into a preset global configuration variable in memory, persistently saving them as the preset action type recognition result. When the video stream acquisition module starts real-time acquisition of RGB video streams containing ball actions of the target object, the action period detection module reads the value of the currently effective action type identifier parameter from the global configuration variable and executes the following determination logic based on the value of the action type identifier parameter: When the value of the action type identifier parameter is equal to a first preset value, the ball action type corresponding to the current video stream is determined to be basketball dribbling. When the value of the action type identifier parameter is equal to the second preset value, the ball action type corresponding to the current video stream is determined to be volleyball passing; the first preset value is the system-predefined basketball dribbling type code, and the second preset value is the system-predefined volleyball passing type code. The basketball dribbling type code and the volleyball passing type code are different integer constants. After the action cycle detection module determines the ball action type corresponding to the current video stream, the action cycle detection module retrieves the parameter configuration group bound to the ball action type from the preset parameter mapping table according to the determined ball action type. The parameter configuration group includes the preset energy threshold, preset cycle duration interval, upper limit of sliding window width, and peak filtering strategy corresponding to the ball action type. This enables the subsequent generation of motion energy time series curves, detection of local maxima, and determination of effective action cycles to be performed based on parameters adapted to the current ball action type, thereby achieving differentiated processing of basketball dribbling and volleyball passing actions.

[0126] C4. When the action type is basketball dribbling, perform the following operations: Calculate the initial peak sequence P.init The average amplitude of all peak values, denoted as amp. mean ;Amplitude greater than or equal to amp mean The peak value is retained after multiplying by a first proportional coefficient. In one specific embodiment of this example, the first proportional coefficient is not a fixed constant, but is dynamically and adaptively set according to the fluctuation amplitude of the motion energy time-series curve of the current basketball dribbling action. Specifically, the system first obtains the initial peak sequence detected in step C2, which contains multiple local maxima points and their corresponding amplitudes. Subsequently, the system calculates the standard deviation of all peak amplitudes in the initial peak sequence, denoted as the amplitude standard deviation, and simultaneously calculates the arithmetic mean of all peak amplitudes, denoted as the amplitude mean. Then, the system calculates the ratio of the amplitude standard deviation to the amplitude mean, denoted as the fluctuation ratio. The system presets a first fluctuation threshold, which is 0.3. The system compares the fluctuation ratio with the first fluctuation threshold: when the fluctuation ratio is greater than 0.3, it indicates that the amplitude fluctuation of the current basketball dribbling action is large, and the difference in action amplitude between different cycles is significant. In this case, the system sets the first proportional coefficient to a lower value, i.e., the first low value, which is 0.5; when the fluctuation ratio is less than or equal to 0.3, it indicates that the amplitude fluctuation of the current action is small, and the action is relatively stable. In this case, the system sets the first proportional coefficient to a higher value, i.e., the first high value, which is 0.8. Through the above adaptive setting, when there is a large difference in individual action amplitude or the action is unstable, the system uses a lower proportional coefficient, retaining only reliable peak values ​​with large amplitudes and eliminating false peak values ​​with low amplitudes caused by jitter or deformation, thereby avoiding the introduction of incorrect cycle boundaries; while when the action is stable, a higher proportional coefficient is used to retain more effective peak values, ensuring that no real action cycle is missed. This dynamic adjustment strategy effectively improves the robustness and adaptability of basketball dribbling action cycle detection.

[0127] C5. For the retained peak sequence, calculate the time interval between two adjacent peaks. If the time interval is less than a preset minimum interval threshold, merge the two peaks, retain the peak with the larger amplitude, and remove the peak with the smaller amplitude. The minimum interval threshold is set to 1 / 3 of the lower limit of the cycle duration of the corresponding action type. The time interval between adjacent peaks in the processed peak sequence is determined as a candidate action cycle. In a specific embodiment of this example, the minimum interval threshold is dynamically updated based on the real-time average cycle duration of the current basketball dribbling action. Specifically, when the system has successfully detected and determined at least three consecutive valid action cycles, the system obtains the duration values ​​of these three consecutive valid action cycles, which are respectively denoted as the first cycle duration, the second cycle duration, and the third cycle duration. The arithmetic mean of these three cycle durations is calculated and denoted as the average cycle duration, in seconds. Then, the system sets the minimum interval threshold to the average cycle duration multiplied by a second proportional coefficient, where the second proportional coefficient is 0.3. For example, if the calculated average cycle duration is 0.6 seconds, the minimum interval threshold is set to 0.18 seconds. When the system has not yet detected any valid action cycles, i.e., during the initialization phase where no consecutive valid cycles have been determined, the system adopts a preset initial minimum interval threshold. This initial minimum interval threshold is determined based on the preset lower limit of the cycle length for basketball dribbling actions: assuming the lower limit of the cycle length for basketball dribbling actions is 0.4 seconds, then the initial minimum interval threshold is set to one-third of 0.4 seconds, i.e., 0.133 seconds. Through this dynamic update mechanism, when an individual's dribbling speed changes, the minimum interval threshold can automatically adjust with the real-time average cycle length, ensuring that the peak merging rule always adapts to the current action rhythm. This effectively avoids the erroneous merging of valid peaks or the retention of false peaks due to differences in individual action speeds, thereby improving the accuracy and individual adaptability of basketball dribbling action cycle detection.

[0128] C6. When the action type is volleyball passing, perform the following operations: Set the initial peak sequence P... initThe peak values ​​in the video stream are sorted from largest to smallest amplitude, and the three peak values ​​with the largest amplitudes are selected and designated as the main peak, the first peak, and the second peak, respectively. The first amplitude ratio between the main peak and the first peak, and the second amplitude ratio between the main peak and the second peak are calculated. If the first amplitude ratio is greater than a second preset ratio threshold, only the main peak is retained as a valid peak. In one specific embodiment, the second preset ratio threshold is adaptively adjusted according to different technical stages of the volleyball passing action. Specifically, the system first performs subtype identification on the volleyball passing action in the current video stream to determine whether the action is an underhand pass or an overhand pass. When the identification result is an underhand pass, the system sets the second preset ratio threshold to the first threshold, which has a value of 0.25. When the identification result is an overhand pass, the system sets the second preset ratio threshold to the second threshold, which has a value of 0.4. In this embodiment, when the recognition result is a low-hand pass, the system sets the second preset ratio threshold to the first threshold, which is set to 0.25. The basis for this setting is that the technical characteristics of the low-hand pass are: both arms extend forward, and the ball is struck with the inner plane of the forearm. The main peak value at the moment of impact is generated by the collision between the forearm and the ball. The rate of change in the forward extension of the arm and the follow-through phase is relatively slow, resulting in secondary peak values ​​significantly lower than the main peak value. Statistical analysis of standard low-hand pass motion samples collected in a campus setting in this embodiment shows that, in multiple consecutive effective motion cycles, the ratio between the first peak value generated during the approach phase and the main peak value generated at the moment of impact is typically distributed within the range of 0.25 to 0.45. Meanwhile, the noise peak value caused by minor limb tremors or light fluctuations is significantly lower. The value is usually less than 0.25 times the amplitude of the main peak. If the first threshold is set higher than 0.25, for example, 0.3 or 0.4, the real secondary peaks with amplitude ratios between 0.25 and the set threshold will be incorrectly eliminated, causing the starting boundary positioning of the candidate action cycle to shift, thus causing the missed detection of effective action cycles. If the first threshold is set lower than 0.25, some noise peaks with amplitude ratios between 0.2 and 0.25 will be misjudged as effective secondary peaks, causing a single action cycle to be incorrectly divided into multiple false cycles. For example, in this embodiment, after cross-validation testing of multiple sets of low-hand ball passing video samples, when the first threshold is set to 0.25, the retention rate of real secondary peaks and the suppression rate of noise peaks reach the optimal balance. Therefore, the system presets the first threshold to 0.25.

[0129] In this embodiment, when the recognition result is an overhand pass, the system sets the second preset ratio threshold to a second threshold of 0.4. This second threshold is set based on the following: the technical characteristics of the overhand pass are that both hands form a hemispherical hand shape above the forehead to meet the ball; at the moment of impact, the fingers, wrists, and forearms work together to generate force; the main peak value is generated by multiple parts contacting the ball simultaneously, resulting in concentrated energy and a significantly higher amplitude than the main peak value of a low-hand pass. In contrast, the overhand pass's approach phase only involves a slow adjustment of the arms as they rise, and the difference between the secondary peak amplitude and the main peak amplitude is greater than the corresponding difference in a low-hand pass. Statistical analysis of standard overhand pass motion samples collected in a campus setting in this embodiment shows that the ratio between the first peak amplitude generated during the approach phase and the main peak amplitude generated at the moment of impact is typically distributed in the range of 0.4 to 0.6, which is below 0. The peak value of 4 basically corresponds to the slight pauses during the arm raising process or the noise fluctuations introduced by the automatic exposure adjustment of the camera. If the second threshold is set below 0.4, for example, 0.25 or 0.3, the noise fluctuations during the arm raising process will be misjudged as valid secondary peak values, resulting in redundant candidate cycle boundaries in the motion energy time series curve, causing the motion frequency statistics to be too high. If the second threshold is set above 0.4, the true secondary peak values ​​of some individuals with smaller motion amplitudes may be missed, resulting in the missed detection of valid motion cycles. For example, in this embodiment, after cross-validation testing of multiple sets of overhand pass video samples, when the second threshold is set to 0.4, it can simultaneously take into account the effective suppression of noise fluctuations and the reliable preservation of true secondary peak values ​​in the distinction between the primary and secondary peaks of the overhand pass motion. Therefore, the system presets the second threshold to 0.4.

[0130] Taking the low pass as an example, if the amplitude of the main peak is 1.0 and the amplitude of the first peak is 0.3, then the first amplitude ratio is 0.3, which is greater than 0.25. In this case, the system determines that the first peak is a valid secondary peak and retains it. If the amplitude of the first peak is 0.2, then the first amplitude ratio is 0.2, which is less than 0.25. The system determines that this secondary peak is noise fluctuation and discards it. For the overhand pass, since the energy is more concentrated at the moment of impact, the difference in amplitude between the main peak and the secondary peak is greater. A higher threshold of 0.4 is used, and only when the amplitude of the secondary peak reaches more than 0.4 times the amplitude of the main peak is it retained. This avoids misjudging the small fluctuations in the arm's approach or follow-through phase as independent valid peaks. Through the above differentiated threshold settings for different pass types, the system can more accurately distinguish the hierarchical relationship between the main peak and the secondary peak, avoid misjudging the main and secondary peaks due to different action patterns, and improve the accuracy of volleyball pass action candidate cycle detection.

[0131] C7. If the first amplitude ratio is less than or equal to the second preset ratio threshold, then both the main peak and the first peak are retained. The retained peaks are arranged in ascending order of time position, and the time interval between two adjacent valid peaks is determined as a candidate action cycle. In a specific embodiment of this example, after determining the time interval between two adjacent valid peaks as a candidate action cycle, the system further dynamically corrects the boundary of the candidate cycle to make the cycle boundary more accurately match the actual start and end times of the action. The specific operations include:

[0132] C71. Obtain the starting peak position index of the candidate period, denoted as the starting peak position, and the amplitude corresponding to the starting peak, denoted as the starting peak amplitude; at the same time, obtain the ending peak position index of the candidate period, denoted as the ending peak position, and the amplitude corresponding to the ending peak, denoted as the ending peak amplitude.

[0133] C72. On the motion energy time-series curve E, starting from the initial peak position, search sequentially to the left. For each adjacent left-hand position index, compare its corresponding energy value with a threshold obtained by multiplying the initial peak amplitude by a third proportional coefficient. The third proportional coefficient is set to 0.2, meaning the threshold is 0.2 times the initial peak amplitude. The system continues searching to the left until it finds the first position index with an energy value less than this threshold, and records this position index as the new starting boundary. For example, if the initial peak amplitude is 0.8, then the threshold is 0.16. When the system searches to the left and the energy value at a certain position first falls below 0.16, that position is taken as the corrected starting boundary.

[0134] C73. Starting from the end peak position, search sequentially to the right. For each adjacent position index to the right, compare its corresponding energy value with a threshold obtained by multiplying the end peak amplitude by 0.2. The system continues to search to the right until it finds the first position index with an energy value less than the threshold, and records this position index as the new end boundary.

[0135] Through the correction in this embodiment, the candidate period, which was originally bounded by the peak position, is expanded to include positions where the energy value drops to 20% of the peak amplitude. These positions are closer to the actual start and end points of the action, i.e. the moment when the motion energy begins to rise significantly and recovers to a low level. This correction effectively reduces the boundary offset caused by energy fluctuations near the peak, so that the subsequently extracted sub-temporal implicit attitude feature matrix can more completely and accurately contain the attitude change process of a complete action period, thereby improving the reliability of the standardization assessment.

[0136] This embodiment addresses the fundamental differences in the motion energy time-series curves of two core ball sports: basketball dribbling and volleyball passing. A single fixed energy threshold can easily lead to oversegmentation, causing the symmetrical bi-peak structure of basketball dribbling to be misjudged as an independent period, and the asymmetrical multi-peak structure of volleyball passing to suffer from missed peak detection or misjudgment of noise fluctuations as valid peaks. Simultaneously, it addresses the core technical issues of dynamic drift in peak amplitude and interval caused by differences in motion amplitude and speed among different individuals within the same motion type, as well as motion deformation due to fatigue, and the repeated detection of flat peak regions and the offset between period boundaries and the actual start and end points of the motion. Firstly, when performing local maximum point detection on the motion energy time-series curve, it addresses the issue of continuous equal energy values ​​forming... In flat peak regions, a unique maximum point is selected by using the middle position of the continuous region as the location and the region's energy value as the peak amplitude. This avoids multiple repetitive local maxima in flat peak regions, ensuring that each action energy peak corresponds to only one unique maximum point. This provides a reliable peak position and amplitude basis for the accurate division of subsequent candidate action cycles. Subsequently, based on the preset action type recognition results, a differentiated peak selection and cycle division strategy is executed. For the approximately equal amplitude and spacing symmetrical bimodal structure of basketball dribbling actions, the amplitude fluctuation ratio of the initial peak sequence is first calculated. The first proportional coefficient is dynamically and adaptively adjusted based on the fluctuation magnitude. When the action amplitude fluctuation is large and the stability is poor, a lower proportional coefficient is used. The algorithm eliminates false peaks with low amplitude and uses a higher scaling factor when the action is stable to avoid missing true valid peaks. It then dynamically updates the minimum interval threshold based on the real-time average cycle length of the detected continuous valid action cycles. For adjacent peaks with excessively small intervals, an amplitude-priority merging operation is performed, ensuring that the peak merging rule always adapts to the individual's real-time action rhythm. This fundamentally solves the problem of double equal-amplitude peaks being mistakenly divided into two independent cycles within a basketball dribbling cycle, achieving accurate segmentation of candidate basketball dribbling action cycles. For the asymmetrical waveform characteristics of volleyball passing actions—a prominent main peak at the moment of impact, low amplitude secondary peaks during the receiving and forward passing phases, and uneven peak intervals—the algorithm first distinguishes between low-hand passing and high-hand passing action subtypes and sets differences accordingly. The second preset ratio threshold is used to accurately distinguish between effective secondary peaks and noise fluctuations by the amplitude ratio of the main peak and the secondary peak. This adapts to the energy distribution characteristics of different ball-passing actions and avoids the problem of missing low-amplitude ball-passing peaks or misjudging small fluctuations near the main peak as effective peaks caused by fixed thresholds. Then, the boundary dynamic correction is performed on the determined candidate action cycle. The first position with an energy value lower than the preset ratio of the peak amplitude is searched from the peak position to the left and right as the new cycle start and end boundary. This makes the cycle boundary accurately fit the actual start and end time of the action, effectively reducing the boundary offset caused by energy fluctuations near the peak. This ensures that the sub-time implicit posture feature matrix extracted later can completely and accurately cover the posture change process of the complete action cycle.The entire process, through peak detection-based deduplication preprocessing, differentiated strategy design based on action type and subtype, and full-process adaptive threshold and rule adjustment combined with real-time action fluctuations and rhythm, fully adapts to the drastically different energy curve shapes of basketball and volleyball. It also accommodates individual differences in action, dynamic changes in action speed and amplitude, and action deformation caused by exercise fatigue, significantly improving the accuracy of local maxima detection and the precision of candidate action cycle segmentation. This fundamentally solves the core problems of over-segmentation, missed detections, and false detections caused by a single fixed threshold. Furthermore, cycle boundary correction further aligns with the complete execution process of actual actions, laying a solid and reliable foundation of cycle data for subsequent effective action cycle selection, action frequency statistics, and action standardization evaluation. This comprehensively enhances the robustness, accuracy, and scene adaptability of the entire ball sports action recognition system across different action types, individuals, and motion states.

[0137] The action frequency statistics module, signal-connected to the action cycle detection module, is configured to: count the total number of valid action cycles within a preset statistical time window, and output the total number as the frequency of the target object's ball-like actions; in this embodiment, the preset statistical time window is specifically set as follows: the system pre-stores statistical time window parameters in a configuration file, and the value of these parameters is set by the user through a configuration interface before system startup; when the system starts, the initialization program reads the value of the statistical time window parameters from the configuration file and loads this value into a global timing configuration variable in memory as the preset statistical time window; when the video stream acquisition module begins to acquire RGB video streams containing the target object's ball-like actions in real time, the action... The frequency statistics module reads the value of the preset statistical time window from the global timing configuration variable and starts a timer whose duration is the value of the preset statistical time window. During the timer's timing, the action frequency statistics module receives the start frame index and end frame index of each valid action cycle output by the action cycle detection module in real time. For each valid action cycle determination result received, the internal counter increments by one. When the timer reaches the duration corresponding to the value of the preset statistical time window, the action frequency statistics module stops counting, reads the current count value of the internal counter, outputs this count value as the ball action frequency of the target object, resets the internal counter to zero, and resets the timer to start the next statistical cycle. For example, in this embodiment, the value of the preset statistical time window is set to sixty seconds. This setting is based on the fact that the typical duration of a single set of ball action tests in daily campus training scenarios is usually one minute. Using sixty seconds as the statistical window can not only fully cover the complete process of a set of test training, but also directly correspond to the assessment standards commonly used in physical education teaching in units of minutes. At the same time, it avoids the problem that the frequency statistical value fluctuates too much due to the statistical window being too short, or that the single statistical result cannot be fed back in time due to the statistical window being too long.

[0138] The standardization evaluation module, which is signal-connected to the motion cycle detection module, is configured to: for each valid motion cycle, extract the sub-temporal implicit posture feature matrix within the range from the start frame index to the end frame index of that cycle; dynamically time-align the sub-temporal implicit posture feature matrix with the pre-stored standard motion template feature matrix; calculate the average feature distance after alignment; when the average feature distance is less than a preset standardization threshold, determine that the corresponding ball-type motion conforms to the standard; otherwise, determine that the ball-type motion deviates from the standard, and accumulate the number of motion deviations or output the motion deviation ratio. In this embodiment, the preset standard degree threshold is set as follows: the system stores the standard degree threshold parameter in the configuration file in advance, and the value of the standard degree threshold parameter is set according to the ball action type corresponding to the current video stream; during the offline preparation stage of the system, temporal implicit posture feature extraction is performed on multiple standard action video samples collected from multiple standard demonstrators; for a valid action cycle in each standard action video sample, the sub-temporal implicit posture feature matrix corresponding to the cycle is extracted, and it is dynamically time-normalized and aligned with the pre-stored standard action template feature matrix; the average feature distance after alignment is calculated to obtain a set of standard action alignment distance sample values; the standard action alignment distance sample values ​​are statistically analyzed, and their arithmetic mean and standard deviation are calculated; the sum of the arithmetic mean plus twice the standard deviation is used as the preset standard degree threshold corresponding to the ball action type; during the online operation stage of the system, the standard degree evaluation module reads the preset standard degree threshold bound to the ball action type from the configuration file according to the current ball action type determined by the action cycle detection module, and uses it to compare and judge with the average feature distance after alignment of the action to be evaluated. For example, in this embodiment, when the current ball movement type is basketball dribbling, after statistical analysis of fifty sets of standard dribbling movement samples, the arithmetic mean of the standard movement alignment distance sample values ​​is 0.15 and the standard deviation is 0.05. Following the calculation method of adding twice the standard deviation to the arithmetic mean, the preset standard deviation threshold corresponding to the basketball dribbling movement is set to 0.25. When the current ball movement type is volleyball passing, after statistical analysis of fifty sets of standard passing movement samples, the arithmetic mean of the standard movement alignment distance sample values ​​is 0.18 and the standard deviation is 0.06. Following the same calculation method... The preset standard threshold for the volleyball passing action is set to 0.30. This setting is based on the fact that, according to the statistical characteristics of normal distribution, the alignment distance between the standard action completed by the standard demonstrator and the standard template should be concentrated around the mean. Taking the mean plus twice the standard deviation as the judgment boundary can cover about 95% of the standard action fluctuation range in a statistical sense. This allows individual differences within the normal range to be included, while abnormal cycles that significantly deviate from the standard action trajectory are accurately identified as deviations from the standard. Thus, in actual deployment, the accuracy of the standard assessment and reasonable tolerance for differences in the execution of different individuals are taken into account.

[0139] In this embodiment, it was further discovered that when dynamically aligning the sub-temporal implicit posture feature matrix with the pre-stored standard action template feature matrix using time warping and calculating the average feature distance after alignment, the following underlying technical problems exist: Different individuals performing the same ball-related action exhibit inherent differences in speed and rhythm, and the rate of change in different stages within the same action cycle, such as the downward and rebound stages of basketball dribbling and the receiving and following stages of volleyball passing, is inherently uneven. This results in inconsistencies between the time axis length of the sub-matrix and the time axis length of the standard template matrix, and local phase shifts also exist. Although dynamic time warping can align two time series through nonlinear mapping, in high-frequency ball-related actions, there are transitional segments with low energy values ​​near the start and end points of the action. These transitional segments are easily overstretched or compressed during warping, causing the alignment path to overemphasize the matching of low-energy regions, thereby distorting the correspondence between core action stages. Furthermore, standard templates are typically derived from demonstrated movements. However, differences in body proportions and movement amplitude among different students can lead to systematic deviations in the amplitude of feature vectors. If Euclidean distance is directly used to calculate the average distance between corresponding points after alignment, the amplitude deviation will be coupled with the shape deviation, making it difficult to independently reflect the shape similarity of the movement trajectory, thus affecting the accuracy and fairness of the standardization assessment. Therefore, this embodiment provides a consistent implementation method for dynamically time-warping and aligning the sub-temporal implicit posture feature matrix with the pre-stored standard movement template feature matrix, specifically as follows:

[0140] D1. For each effective action cycle, extract the sub-temporal implicit pose feature matrix within the range from the start frame index to the end frame index corresponding to the effective action cycle. The sub-temporal implicit pose feature matrix is ​​denoted as the matrix to be evaluated. The number of rows of the matrix to be evaluated is equal to the total number of video frames contained in the effective action cycle, which is denoted as the first row number. The number of columns of the matrix to be evaluated is equal to the implicit pose feature dimension.

[0141] D2. Simultaneously, obtain the standard action template feature matrix corresponding to the current ball action type from the pre-stored standard action template library, denoted as the standard template matrix. The number of rows in the standard template matrix is ​​equal to the number of reference frames of the standard action, denoted as the second row number. The number of columns in the standard template matrix is ​​also equal to the implicit posture feature dimension. The standard template matrix is ​​obtained in advance by performing an arithmetic average of the temporal implicit posture feature matrices of multiple standard demonstration actions or by selecting typical samples. In this embodiment, the pre-stored standard action template library is set up as follows: During the system offline preparation phase, for each type of ball action to be identified, multiple RGB video samples are collected from multiple standard demonstrators performing the ball action under standard execution conditions; for example, for basketball dribbling, video samples are collected from ten standard demonstrators each performing twenty consecutive standard dribbling actions; for volleyball passing, video samples are collected from ten standard demonstrators each performing twenty consecutive standard passing actions; each video sample is decomposed into a continuous sequence of video frames in chronological order, and then input into the temporal implicit pose coding module. A depth convolutional feature map is extracted from each frame image, and a global pooling operation is performed to obtain a one-dimensional implicit pose feature vector for the corresponding single frame image. The one-dimensional implicit pose feature vectors corresponding to each frame image are concatenated in chronological order to construct the temporal implicit pose feature matrix of the video sample; the action cycle detection module is used to analyze the temporal implicit pose of each video sample. The posture feature matrix is ​​used for effective motion cycle detection, and the sub-temporal implicit posture feature matrix corresponding to each standard motion is extracted. For each ball game motion type, the sub-temporal implicit posture feature matrices corresponding to all standard motions of all standard demonstrators under that ball game motion type are dynamically time-warped and aligned. The arithmetic mean of the aligned matrix is ​​taken frame by frame to generate a standard motion template feature matrix that uniquely corresponds to that ball game motion type. A mapping relationship is established between the standard motion template feature matrix and the type identifier code of the ball game motion type, and they are stored together in the system storage unit to form the pre-stored standard motion template library. During the online operation of the system, the standardness evaluation module retrieves and reads the corresponding standard motion template feature matrix from the pre-stored standard motion template library according to the current ball game motion type determined by the motion cycle detection module, using its type identifier code as an index. This matrix is ​​used to perform dynamic time warping and alignment and average feature distance calculation with the sub-temporal implicit posture feature matrix corresponding to the period to be evaluated.

[0142] D3. Obtain the normalized matrix to be evaluated and the standard template matrix. Specifically, for each row vector, calculate the sum of squares of each component of the row vector, then take the square root of the sum of squares to obtain the magnitude of the row vector. Then divide each component of the row vector by the magnitude, so that the magnitude of each normalized row vector is equal to one. After normalizing all row vectors, the normalized matrix to be evaluated is obtained. Perform the same L2 norm normalization process on each row vector of the standard template matrix to obtain the normalized standard template matrix. The normalization operation is used to eliminate the systematic bias of the feature vector magnitude caused by differences in body proportions and movement amplitudes among different individuals.

[0143] D4. Constructing a dynamically time-warped distance matrix. In one specific implementation of this embodiment, the standard evaluation module first constructs a distance matrix to measure the similarity of frame vectors between the normalized matrix to be evaluated and the normalized standard template matrix. Taking basketball dribbling as an example, suppose the normalized matrix to be evaluated corresponding to a certain effective action cycle contains 60 row vectors, i.e., the first row has 60 rows; the normalized standard template matrix contains 50 row vectors, i.e., the second row has 50 rows. Therefore, the distance matrix has 60 rows and 50 columns.

[0144] For each element in the distance matrix, the row index u ranges from 1 to 60, and the column index q ranges from 1 to 50. The system calculates the Euclidean distance between the u-th row vector in the normalized matrix to be evaluated and the q-th row vector in the normalized standard template matrix, and uses this Euclidean distance value as the element value of the u-th row and q-th column in the distance matrix. For example, when u equals 10 and q equals 8, the system extracts the 10th row vector of the matrix to be evaluated and the 8th row vector of the standard template matrix, calculates the sum of the squares of the differences between their corresponding components, and then takes the square root to obtain a distance value of 0.35, which is then filled into the 10th row and 8th column of the distance matrix. Similarly, the system iterates through all combinations of u from 1 to 60 and q from 1 to 50, sequentially calculating and filling all elements of the distance matrix, ultimately obtaining a complete distance matrix of 60 rows by 50 columns. This distance matrix provides the basic cost matrix for subsequent dynamic time warping path search, where each element reflects the degree of difference between attitude feature vectors at two different time points. Through the above construction method, this embodiment quantifies the inter-frame correspondence between the action to be evaluated and the standard template into a computable numerical matrix, laying the data foundation for nonlinear alignment.

[0145] D5. Perform dynamic time warping path search with boundary constraints and window constraints on the distance matrix to obtain the optimal warped path. In a specific embodiment of this example, one way to perform dynamic time warping path search with boundary constraints and window constraints on the distance matrix is ​​as follows: Construct a cumulative distance matrix. The number of rows in this matrix is ​​equal to the number of the first row of the matrix to be evaluated, and the number of columns is equal to the number of the second row of the standard template matrix. Taking basketball dribbling as an example, suppose the matrix to be evaluated for a certain effective action cycle contains 60 frames, i.e., the number of the first row is 60; the standard template matrix contains 50 frames, i.e., the number of the second row is 50. Set the element of the first row and first column of the cumulative distance matrix as the distance value of the first row and first column of the distance matrix. This distance value is the Euclidean distance between the first row vector of the normalized matrix to be evaluated and the first row vector of the normalized standard template matrix. Subsequently, for other positions in the cumulative distance matrix, calculate the cumulative distance value of each position in ascending order of row index and ascending order of column index. The cumulative distance value of each position is equal to the value of the distance matrix element corresponding to that position, plus the minimum cumulative distance value among its left neighbor, top neighbor, and top-left neighbor.

[0146] To avoid excessive stretching or compression of low-energy transition segments near the start and end points of actions, this embodiment sets a window constraint. Specifically, for the position with row index u and column index q in the cumulative distance matrix, a first ratio is calculated divided by 60 and a second ratio is calculated divided by 50, and the absolute value of the difference between the two is obtained. Only when this absolute value does not exceed a preset window width parameter is the position allowed to participate in the path search calculation. In this embodiment, the window width parameter is set to 0.25. For example, when u=10 and q=5, the first ratio is 0.1667, the second ratio is 0.1, and the difference is 0.0667, which is less than 0.25, so the position is allowed to be calculated; when u=50 and q=10, the first ratio is 0.8333, the second ratio is 0.2, and the difference is 0.6333, which is greater than 0.25, so the position is excluded. In this embodiment, the window width parameter is set to 0.25 because: the window constraint in the dynamic time warping path search is used to limit the search range of the optimal warping path to a strip-shaped region near the diagonal of the distance matrix, so as to avoid the low-energy transition segments near the start and end points of the action being overstretched or compressed, and to prevent frames that are far apart from the matrix to be evaluated from being incorrectly paired with the standard template matrix; the value of the window width parameter determines the width of the strip-shaped region. When the window width parameter is too large, for example, exceeding 0.5, the strip-shaped region is too wide, and the warping path may deviate from the diagonal and enter the far end region of the distance matrix, causing the low-energy transition segments near the start and end points to mismatch with the core action phase, so that the average feature distance after alignment cannot accurately reflect the similarity of the attitude trajectory of the core action phase; when the window width parameter is too small, for example, less than 0.1, the strip-shaped region is too narrow, and the warping path is forced to be compacted. Placing the image diagonally cannot effectively compensate for the time axis scaling caused by individual movement rhythm differences between the action to be evaluated and the standard template. This results in normal actions with a slightly faster or slower rhythm than the standard template being incorrectly judged as deviations from the standard. For example, in this embodiment, through statistical analysis of basketball dribbling and volleyball passing action samples collected in a campus setting, the ratio of the cycle duration of different students performing the same ball action to the cycle duration of the standard template is usually distributed between 0.75 and 1.25. That is, the ratio of the number of rows in the first row of the matrix to be evaluated to the number of rows in the standard template matrix fluctuates within a range of approximately ±25%. Setting the window width parameter to 0.25 precisely covers this normal rhythm fluctuation range, which not only preserves sufficient time axis scaling tolerance for individual movement rhythm differences but also effectively eliminates the risk of incorrect pairing caused by excessive deviation of the regular path from the diagonal. Therefore, the system presets the window width parameter to 0.25.

[0147] Under the aforementioned window constraint, starting from the first row and first column of the cumulative distance matrix, the cumulative distance is calculated row by row and column by column until the position where the row index equals the first row number 60 and the column index equals the second row number 50. This cumulative distance value is the minimum regularization cost. Then, starting from this endpoint, the system backtracks point by point to the starting point in the direction of decreasing cumulative distance (i.e., selecting the direction with the smallest cumulative distance among left, top, and top-left). The position indices traversed during each backtracking are recorded. These positions are arranged in order from the starting point to the endpoint, thus forming the optimal regularization path. Each point on this path corresponds to a matching relationship between a frame of the matrix to be evaluated and a frame of the standard template matrix, thereby aligning the two matrices. After alignment, a pairing sequence can be generated and the average feature distance calculated for subsequent standardization determination. Through the aforementioned window constraint, the alignment path is confined to a strip-shaped region around the diagonal, effectively suppressing excessive distortion in the transition segment near the start and end points, ensuring that the core action phase (such as pressing down and hitting the ball) receives priority matching.

[0148] D6. Based on the optimal regularization path obtained from the search, the normalized matrix to be evaluated is aligned with the normalized standard template matrix to form an aligned pairing sequence. In a specific implementation of this embodiment, based on the above-mentioned optimal regularization path, the standard evaluation module further performs the operation of generating the aligned pairing sequence. Continuing the previous example, let the first row of the matrix to be evaluated be 60, and the second row of the standard template matrix be 50. The optimal regularization path obtained through dynamic time regularization search with window constraints contains several path points. For example, the path may contain 75 path points, each path point corresponding to a row index and a column index, where the row index ranges from 1 to 60, and the column index ranges from 1 to 50. Specifically, the first path point is row index 1 and column index 1; the second path point may be row index 2 and column index 1; the third path point may be row index 2 and column index 2; and so on, until the last path point is row index 60 and column index 50.

[0149] For each path point in the optimal regularized path, the system extracts the row vector corresponding to the row index of that path point in the normalized evaluation matrix, and simultaneously extracts the row vector corresponding to the column index of that path point in the normalized standard template matrix. These two row vectors are then combined into a pairing point. For example, for a path point with row index 2 and column index 1, the pairing point is (the second row vector of the evaluation matrix and the first row vector of the standard template matrix). All pairing points corresponding to all path points are arranged sequentially from the starting point to the ending point, forming an aligned pairing sequence. The length of this pairing sequence is equal to the number of path points contained in the optimal regularized path, which is 75 pairing points in the previous example. The two row vectors in each pairing point are non-linearly aligned in the time dimension, ensuring that frames corresponding to the same action stage in the evaluation matrix and the standard template matrix are correctly paired, thus providing a one-to-one vector pair for subsequent calculation of the average feature distance. Through this method, this embodiment achieves robust adaptation to individual action rhythm differences while avoiding mismatch problems caused by excessive distortion in transition segments.

[0150] D7. Calculate the arithmetic mean of the Euclidean distances between all paired points in the aligned pairing sequence. Specifically, for each paired point in the pairing sequence, calculate the Euclidean distance between its two row vectors to obtain a set of distance values; sum the set of distance values ​​and divide them by the length of the pairing sequence, and output the quotient as the average feature distance after alignment; the average feature distance is used to compare with a preset standard threshold to determine whether the ball action corresponding to the current effective action cycle meets the standard.

[0151] This implementation eliminates the systematic bias in feature vector amplitude caused by differences in body proportions and movement amplitudes among individuals by performing L2 norm normalization on each row vector of the matrix to be evaluated and the standard template matrix. This ensures that subsequent alignment and distance calculations only reflect the shape similarity of the posture trajectory, resolving the coupling problem between amplitude and shape biases. Secondly, a window constraint is introduced in the dynamic time warping path search, restricting the alignment path to a band-shaped region around the diagonal. This effectively suppresses the overstretching or compression of low-energy transition segments near the start and end points of the movement, preventing the warping path from excessively focusing on low-energy regions and distorting the correspondence of core movement stages. Based on this, the optimal warping path is obtained through the construction and backtracking of the cumulative distance matrix, generating an aligned pairing sequence. This ensures that frames corresponding to the same movement stage in the matrix to be evaluated and the standard template matrix are correctly paired, achieving robust adaptation to differences in speed and rhythm among individuals. Finally, the arithmetic mean of the Euclidean distances of all paired points in the pairing sequence is calculated. This average feature distance is independent of amplitude bias and only reflects the shape similarity of the movement trajectory. In summary, this embodiment significantly improves the accuracy and fairness of ball game movement standard assessment, enabling the system to accurately quantify high-frequency periodic movements, adapt to individual differences among students in daily school training, and enhance the reliability and generalization ability of the assessment results.

[0152] Example 2

[0153] Please see Figure 3 Another embodiment of the present invention provides a ball action recognition method based on temporal pose coding, comprising:

[0154] S1. The AI ​​training machine's camera device collects RGB video streams containing the target object's ball-like movements in real time, and decomposes the RGB video streams into a continuous sequence of video frames in chronological order.

[0155] S2. Extract a deep convolutional feature map from each frame of the video frame sequence, and perform a global pooling operation on the deep convolutional feature map along the channel dimension to obtain a one-dimensional implicit pose feature vector for the corresponding single frame image; concatenate the one-dimensional implicit pose feature vectors corresponding to each frame image in chronological order to construct a temporal implicit pose feature matrix; the rows of the temporal implicit pose feature matrix correspond to the time step, and the columns correspond to the implicit pose feature dimensions.

[0156] S3. Calculate the Euclidean distance between the feature vectors corresponding to adjacent time steps in the temporal implicit attitude feature matrix to generate a motion energy temporal curve; detect local maxima in the motion energy temporal curve where the peak value is greater than a preset energy threshold, and determine the time interval between two adjacent local maxima as candidate action cycles; calculate the duration of the candidate action cycle, and when the duration falls within a preset cycle duration interval, determine the candidate action cycle as a valid action cycle, and record the start frame index and end frame index corresponding to the valid action cycle;

[0157] S4. Within a preset statistical time window, count the total number of valid action cycles and output the total number as the frequency of ball-like actions of the target object.

[0158] S5. For each valid action cycle, extract the sub-temporal implicit posture feature matrix within the range from the start frame index to the end frame index corresponding to that cycle. Perform dynamic time warping and alignment of the sub-temporal implicit posture feature matrix with the pre-stored standard action template feature matrix, and calculate the average feature distance after alignment. When the average feature distance is less than the preset standard threshold, it is determined that the ball action meets the standard; otherwise, it is determined that the ball action deviates from the standard, and the number of action deviations is accumulated or the action deviation ratio is output.

[0159] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments under the guidance of the present invention without departing from the spirit and scope of the present invention. All of these variations are within the protection scope of the present invention.

Claims

1. A ball-playing motion recognition system based on temporal pose coding, characterized in that, include: The video stream acquisition module is configured to: acquire an RGB video stream containing the ball-like actions of the target object in real time, and decompose the RGB video stream into a continuous sequence of video frames in chronological order; The temporal implicit pose coding module is configured to: extract a deep convolutional feature map for each frame in the video frame sequence, perform a global pooling operation on the deep convolutional feature map along the channel dimension to obtain a one-dimensional implicit pose feature vector for the corresponding single frame image, and concatenate the one-dimensional implicit pose feature vectors corresponding to each frame image in chronological order to construct a temporal implicit pose feature matrix; the rows of the temporal implicit pose feature matrix correspond to the time step, and the columns correspond to the implicit pose feature dimensions; The motion cycle detection module is configured to: calculate the Euclidean distance between the feature vectors corresponding to adjacent time steps in the temporal implicit attitude feature matrix, generate a motion energy temporal curve, and simultaneously detect local maxima points in the motion energy temporal curve where the peak value is greater than a preset energy threshold, and determine the time interval between two adjacent local maxima points as candidate motion cycles; calculate the duration of the candidate motion cycle, and when the duration falls within a preset cycle duration interval, determine the candidate motion cycle as a valid motion cycle, and record the start frame index and end frame index corresponding to the valid motion cycle.

2. The ball movement recognition system based on temporal pose coding as described in claim 1, characterized in that, The ball motion recognition system also includes: The action frequency statistics module is configured to: count the total number of valid action cycles within a preset statistical time window, and output the total number as the ball action frequency of the target object; The standardization evaluation module is configured to: for each valid action cycle, extract the sub-temporal implicit posture feature matrix within the range from the start frame index to the end frame index corresponding to the cycle; dynamically time-align the sub-temporal implicit posture feature matrix with the pre-stored standard action template feature matrix; calculate the average feature distance after alignment; when the average feature distance is less than the preset standardization threshold, determine that the corresponding ball action meets the standard; otherwise, determine that the corresponding ball action deviates from the standard, and accumulate the number of action deviations or output the action deviation ratio.

3. The ball movement recognition system based on temporal pose coding as described in claim 2, characterized in that, For each frame in the video frame sequence, a depthwise convolutional feature map is extracted, and a global pooling operation is performed on the depthwise convolutional feature map along the channel dimension, including: Get the RGB image I of the current frame; The current frame RGB image I is input into a pre-trained convolutional neural network, which contains N convolutional layers; the deep convolutional feature map F output by the last convolutional layer in the convolutional neural network is extracted. The deep convolutional feature map F has a spatial height dimension H, a spatial width dimension W, and a channel dimension C, and the feature value at any position in the deep convolutional feature map F is denoted as F(i, j, c), i∈[1, H], j∈[1, W], c∈[1, C]; For each channel index c in the channel dimension C, calculate the global average of the feature values ​​at all spatial locations (i, j) within the channel, and use this average as the scalar pooling value A for the corresponding channel. c ; The scalar pooling values ​​A corresponding to each of the C channels are... c Arranged in ascending order according to channel index c, they form a one-dimensional implicit pose feature vector v for the current frame, where v = [A1, A2, ..., A...]. c …, A C The length of vector v is equal to C.

4. The ball motion recognition system based on temporal pose coding as described in claim 3, characterized in that, The generation of the motion energy time-series curve includes: Obtain the temporal implicit pose feature matrix M, where the number of rows in the temporal implicit pose feature matrix M is the total number of frames T in the video frame sequence, and the number of columns is the dimension C of the implicit pose feature vectors; the vector in the t-th row of matrix M is denoted as v. t Where t ranges from 1 to T, and each v t The length of is equal to C; For adjacent time steps t and t+1, calculate the t-th row vector v. t With the (t+1)th row vector v t+1 The Euclidean distance d between them t This constitutes the original distance sequence; Perform sliding window mean filtering on the original distance sequence, wherein the width of the sliding window is set to Q consecutive elements; For the k-th starting position in the original distance sequence, take Q elements from the k-th element to the k-th plus Q minus one element, calculate the arithmetic mean of these Q elements, and obtain the k-th smoothed motion energy value; the value of k ranges from 1 to T minus Q, and the smoothed motion energy sequence is obtained.

5. The ball movement recognition system based on temporal pose coding as described in claim 4, characterized in that, The step of determining the time interval between two adjacent local maxima points as candidate action cycles includes: Obtain the smoothed motion energy time-series curve, denoted as E. The curve E contains M1 discrete energy values, where M1 is equal to the total number of frames T in the video frame sequence minus the sliding window width Q. Each energy value corresponds to a time position index m, and the value of m ranges from 1 to M1. Local maximum point detection is performed on the motion energy time series curve E to obtain the initial peak sequence P. init Each peak point is denoted as p, which includes the peak position index pos(p) and the peak amplitude amp(p). The detection method is as follows: for each time position index m, if E(m) is greater than its left adjacent value E(m-1) and greater than its right adjacent value E(m+1), then m is determined as a local maximum point. E(m) represents the energy value of the motion energy time series curve at the time position index m.

6. The ball movement recognition system based on temporal pose coding as described in claim 5, characterized in that, The step of determining the time interval between two adjacent local maxima points as candidate action cycles also includes: Based on the preset action type recognition results, determine the ball action type corresponding to the current video stream; When the action type is basketball dribbling, perform the following operations: Calculate the initial peak sequence P. init The average amplitude of all peak values, denoted as amp. mean ;Amplitude greater than or equal to amp mean Peak values ​​multiplied by the first scaling factor are retained; For the retained peak sequence, the time interval between two adjacent peaks is calculated. If the time interval is less than a preset minimum interval threshold, the two peaks are merged. The minimum interval threshold is 1 / 3 of the lower limit of the cycle duration of the corresponding action type. The time interval between adjacent peaks in the processed peak sequence is determined as the candidate action cycle.

7. The ball movement recognition system based on temporal pose coding as described in claim 6, characterized in that, The step of determining the time interval between two adjacent local maxima points as candidate action cycles also includes: When the action type is volleyball passing, perform the following operations: Set the initial peak sequence P... init The peaks are sorted from largest to smallest amplitude, and the three peaks with the largest amplitudes are selected and designated as the main peak, the first peak, and the second peak, respectively. The first amplitude ratio between the main peak and the first peak, and the second amplitude ratio between the main peak and the second peak are calculated. If the first amplitude ratio is greater than the second preset ratio threshold, only the main peak is retained as a valid peak. If the first amplitude ratio is less than or equal to the second preset ratio threshold, then both the main peak and the first peak are retained. The retained peaks are arranged in ascending order of time position, and the time interval between two adjacent valid peaks is determined as the candidate action cycle.

8. The ball movement recognition system based on temporal pose coding as described in claim 7, characterized in that, The sub-temporal implicit pose feature matrix is ​​dynamically time-warped and aligned with the pre-stored standard action template feature matrix, including: For each effective action cycle, extract the sub-temporal implicit pose feature matrix within the range from the start frame index to the end frame index corresponding to the effective action cycle. The sub-temporal implicit pose feature matrix is ​​denoted as the matrix to be evaluated. The number of rows in the matrix to be evaluated is equal to the total number of video frames contained in the effective action cycle, which is denoted as the first row number. The number of columns in the matrix to be evaluated is equal to the implicit pose feature dimension. Simultaneously, a standard action template feature matrix corresponding to the current ball action type is obtained from the pre-stored standard action template library, denoted as the standard template matrix. The number of rows in the standard template matrix is ​​equal to the number of reference frames of the standard action, denoted as the second row number. The number of columns in the standard template matrix is ​​also equal to the implicit pose feature dimension. Obtain the normalized matrix to be evaluated and the standard template matrix.

9. The ball movement recognition system based on temporal pose coding as described in claim 8, characterized in that, The process of dynamically aligning the sub-temporal implicit pose feature matrix with the pre-stored standard motion template feature matrix through temporal warping also includes: A dynamically time-warped distance matrix is ​​constructed, wherein the number of rows in the distance matrix is ​​equal to the number of the first row of the normalized matrix to be evaluated, and the number of columns in the distance matrix is ​​equal to the number of the second row of the normalized standard template matrix; the element located in the u-th row and q-th column of the distance matrix has a value equal to the Euclidean distance between the u-th row vector of the normalized matrix to be evaluated and the q-th row vector of the normalized standard template matrix. Perform a dynamic time warping path search with boundary constraints and window constraints on the distance matrix to obtain the optimal warping path; Based on the optimal regularization path obtained from the search, the normalized matrix to be evaluated is aligned with the normalized standard template matrix to form an aligned pairing sequence. Calculate the arithmetic mean of the Euclidean distances between all paired points in the aligned pairing sequence.

10. The ball motion recognition system based on temporal pose coding as described in claim 9, characterized in that, Performing a dynamic time-warped path search with boundary constraints and window constraints on the distance matrix specifically includes: Construct a cumulative distance matrix, wherein the number of rows in the cumulative distance matrix is ​​the same as the number of rows in the distance matrix, and the number of columns in the cumulative distance matrix is ​​the same as the number of columns in the distance matrix; set the element of the first row and first column of the cumulative distance matrix to the element value of the first row and first column of the distance matrix; For other positions in the cumulative distance matrix, the calculation is performed sequentially in ascending order of row index and column index. The cumulative distance value of each position is equal to the value of the distance matrix element corresponding to that position plus the minimum cumulative distance value of the predecessor position of that position. The predecessor position includes the left adjacent position, the upper adjacent position, and the upper left adjacent position of that position. A window constraint is set, which limits the range of predecessor positions that can be calculated: for a position with row index u and column index q in the cumulative distance matrix, the position participates in the path search calculation only if the absolute value of the difference between the row index u divided by the number of the first row and the column index q divided by the number of the second row does not exceed the preset window width parameter.

11. The ball motion recognition system based on temporal pose coding as described in claim 10, characterized in that, Performing a dynamic time-warped path search with boundary constraints and window constraints on the distance matrix specifically includes: Under the window constraint, starting from the first row and first column of the cumulative distance matrix, the cumulative distance value at each row and column is calculated until the position where the row index is equal to the number of the first row and the column index is equal to the number of the second row is obtained as the minimum normalization cost. Starting from the position in the cumulative distance matrix where the row index equals the first row number and the column index equals the second row number, backtrack to the first row and first column in the direction of decreasing cumulative distance value, record the position traversed each time backtracking, and construct the optimal regular path by arranging the positions in order from the starting point to the ending point.

12. A ball-playing motion recognition method based on temporal pose coding, implemented based on the ball-playing motion recognition system based on temporal pose coding as described in any one of claims 1-11, characterized in that, include: Real-time acquisition of RGB video streams containing the ball-like movements of the target object, and decomposition of the RGB video stream into a continuous sequence of video frames in chronological order; For each frame in the video frame sequence, a deep convolutional feature map is extracted. Global pooling is performed on the deep convolutional feature map along the channel dimension to obtain a one-dimensional implicit pose feature vector for the corresponding single frame image. The one-dimensional implicit pose feature vectors corresponding to each frame image are then concatenated in chronological order to construct a temporal implicit pose feature matrix. The rows of the temporal implicit pose feature matrix correspond to the time step, and the columns correspond to the implicit pose feature dimensions. The Euclidean distance between the feature vectors corresponding to adjacent time steps in the temporal implicit attitude feature matrix is ​​calculated to generate a motion energy temporal curve. At the same time, local maxima points in the motion energy temporal curve with peak values ​​greater than a preset energy threshold are detected, and the time interval between two adjacent local maxima points is determined as a candidate action cycle.

13. The ball action recognition method based on temporal pose coding as described in claim 12, characterized in that, Also includes: Calculate the duration of the candidate action cycle. When the duration falls within a preset cycle duration range, determine the candidate action cycle as a valid action cycle and record the start frame index and end frame index corresponding to the valid action cycle. Within a preset statistical time window, the total number of valid action cycles is counted, and the total number is output as the frequency of ball-related actions of the target object. For each valid action cycle, the sub-temporal implicit pose feature matrix within the range from the start frame index to the end frame index corresponding to the cycle is extracted. The sub-temporal implicit pose feature matrix is ​​dynamically time-aligned with the pre-stored standard action template feature matrix, and the average feature distance after alignment is calculated. When the average feature distance is less than the preset standard threshold, the corresponding ball action is determined to meet the standard; otherwise, the corresponding ball action is determined to deviate from the standard, and the number of action deviations is accumulated or the action deviation ratio is output.

Images