A combined sports action detection method

Abstract

The present invention provides a joint sports action detection method, which relates to the field of video analysis technology. The method combines action classification and boundary positioning targets, and directly maps features into detection results by learning unified action representation and boundary judgment features; it not only improves the performance of fine-grained sports action detection, but also improves the representation ability and flexibility of the model. At the same time, by establishing a grouped feature extraction backbone network and a multi-scale temporal feature extraction neck network module, it can effectively handle fine-grained sports action detection tasks, solving the limitations of traditional sports action detection methods in fine-grained detection tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of video analysis, and in particular to a combined sports action detection method. Background Art

[0002] At present, in the field of sports video analysis, temporal action detection technology plays a vital role. This technology aims to extract the start time, end time and corresponding action category of action clips from untrimmed videos. With the development of computer vision technology, temporal action detection methods have continued to evolve, and most of them adopt a staged approach; among them, the staged approach divides the task into two subtasks: action nomination and action classification. First, candidate regions that may contain actions are generated through nominations, and then these nominations are classified to determine the specific action category. However, this method relies on the accuracy of nominations, and in fine-grained sports action detection, it is difficult to accurately classify sub-actions, resulting in limited overall performance.

[0003] Traditional temporal action detection methods often adopt the strategy of selecting the top-k most likely action categories in the entire video when faced with videos containing multiple action categories, which is difficult to accurately classify sub-actions in fine-grained sports action detection. In addition, these methods are mainly based on RGB feature representation, which makes it difficult to effectively separate action instances and background segments, limiting the accuracy and efficiency of detection.

[0004] In view of this, this application is filed. Summary of the invention

[0005] The present invention provides a combined sports action detection method, which can at least partially improve the above-mentioned problems.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A combined sports action detection method, comprising:

[0008] Acquire a video to be detected, extract the video to be detected, obtain a skeleton feature tensor, and input the skeleton feature tensor into a preset grouping temporal feature extraction backbone network to perform extraction processing to obtain a grouping temporal feature extraction result;

[0009] Preprocessing the grouped temporal feature extraction result using a neck network to obtain a feature feedback result, wherein the neck network is composed of a temporal multi-scale attenuation network and a temporal feature pyramid;

[0010] The feature feedback result is transmitted to the classification detection head and the regression detection head to detect the features of each time t of all L layers on the temporal feature pyramid to obtain the detection result.

[0011] In summary, the core of the joint sports action detection method lies in the processing of three key modules: a grouped feature extraction backbone network, a neck network based on a temporal multiscale decay network (TMDN) and a temporal feature pyramid (TFP), and an action classification and regression detection head. First, the skeleton sequence of the video frame is extracted using the OpenPose technology and used as the input feature tensor; in the feature extraction backbone network, the spatial convolution layer and the temporal convolution layer are constructed by grouping, which enhances the representation ability and flexibility of feature extraction. Secondly, the neck network adopts a method combining TMDN and TFP to enhance the interaction between features of different time ranges and improve the classification accuracy of the network for sports video action frames. Finally, as a lightweight convolutional network, the detection head has classification and regression functions, and predicts the action probability corresponding to each moment and the offset distance of the action boundary. It aims to solve the limitations of traditional sports action detection methods in fine-grained detection tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a flow chart of a combined sports action detection method provided by a first embodiment of the present invention;

[0013] Figure 2 is a schematic diagram of the overall structure of the combined sports action detection method provided by the first embodiment of the present invention;

[0014] Figure 3 It is a structural diagram of a grouped feature extraction backbone network provided by the first embodiment of the present invention;

[0015] Figure 4 is a schematic diagram of the structure of a spatial convolution layer provided by the first embodiment of the present invention;

[0016] Figure 5 is a schematic diagram of the composition of the coefficient matrix A provided by the first embodiment of the present invention;

[0017] Figure 6 is a schematic diagram of the structure of a temporal convolutional layer provided in the first embodiment of the present invention;

[0018] Figure 7 It is a schematic diagram of the structure of the neck network provided by the first embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0020] refer to Figure 1 , Figure 2As shown, the first embodiment of the present invention discloses a combined sports action detection method, which can be executed by a combined sports action detection device (hereinafter referred to as the detection device), and in particular, by one or more processors in the detection device to implement the following method:

[0021] S1, obtaining a video to be detected, extracting the video to be detected to obtain a skeleton feature tensor, and inputting the skeleton feature tensor into a preset grouping temporal feature extraction backbone network to perform extraction processing to obtain a grouping temporal feature extraction result;

[0022] Specifically, step S1 includes: obtaining the video v to be detected i , use the OpenPose algorithm to distort the video v i Perform extraction processing to extract video v i The skeleton sequence of the video frames in the video is obtained to obtain the skeleton feature tensor , where R is a set of real numbers and T is the time channel dimension;

[0023] The skeleton feature tensor X is input into the grouped temporal feature extraction backbone network, and the skeleton feature tensor X is preprocessed using N grouped GCN blocks of the grouped temporal feature extraction backbone network to obtain a grouped temporal feature extraction result, wherein each of the grouped GCN blocks consists of a spatial convolution layer and a temporal convolution layer.

[0024] Preferably, the skeleton feature tensor X is preprocessed using the N grouped GCN blocks of the grouped temporal feature extraction backbone network to obtain a grouped temporal feature extraction result, specifically:

[0025] The skeleton feature tensor X is Convolution processing, get K feature groups, where each feature group contains channels, C is the input channel;

[0026] Based on the coefficient matrix A corresponding to the feature group i , spatial modeling is performed for each feature group;

[0027] All the feature groups after modeling are concatenated in the channel dimension, and the concatenated feature groups are Convolution processing is performed to obtain the spatial convolution processing result.

[0028] Preferably, the coefficient matrix A i The generation steps include:

[0029] Get the original coefficient matrix A for spatial modeling, where V is the joint channel dimension;

[0030] Use normal distribution to initialize the original coefficient matrix A and generate the initial coefficient matrix PA i , where the initial coefficient matrix PA i Including dynamic parameters DA that are independent of the characteristic channel i and the dynamic parameters CA for the feature channel i ;

[0031] For the K input features of the feature group, a time pooling layer is used to eliminate the time channel dimension T of the input features of the feature group, and then two The convolution block performs convolution processing and obtains the convolution results X a and X b ;

[0032] Among them, the dynamic parameter DA is independent of the characteristic channel. i is obtained by convolution result X a and X b The dynamic parameter CA for the feature channel is obtained by multiplying the product and then activating it with the Softmax activation function. i is obtained by convolution result X a and X b The result is obtained by performing subtraction and activating with Tanh activation function.

[0033] The initial coefficient matrix PA i , dynamic parameter DA independent of feature channel i and the dynamic parameters CA for the feature channel i Perform weighted processing to obtain the coefficient matrix A i , the formula is: , ,in, and are all learnable parameters.

[0034] Preferably, it is also specifically:

[0035] In the temporal convolution layer, based on the spatial convolution processing result, average pooling processing is performed on the V joint-level skeleton features to obtain a joint-level skeleton feature;

[0036] The joint-level skeleton features and the joint-level skeleton features are spliced ​​and sampled and grouped, and each grouped feature group is subjected to temporal modeling, and the joint-level skeleton features and their corresponding learnable parameters are dynamically fused. ;

[0037] Adding the joint-level skeleton features and the joint-level skeleton features to obtain a temporal convolution processing result;

[0038] According to the convolution processing result and the spatial convolution processing result, a grouping time series feature extraction result is obtained.

[0039] In this embodiment, the video to be detected is first processed, and the skeleton sequence is extracted from the video using the OpenPose algorithm. These sequences contain the 2D position coordinates and confidence scores of each joint, forming a skeleton feature tensor. , where 3 is the original feature dimension, corresponding to the 2D position coordinates and confidence scores of each joint; the dimension of the joint channel is 25, corresponding to the number of joints, and T is the time channel dimension, corresponding to the time series length of each skeleton sequence. This tensor contains real values ​​and has a time channel dimension, which lays the foundation for subsequent feature extraction.

[0040] Next, the skeleton feature tensor is input into the grouped temporal feature extraction backbone network, which consists of multiple grouped graph convolutional network blocks, each of which contains a spatial convolution layer and a temporal convolution layer. In the spatial convolution layer, the input feature tensor is convolved to divide it into multiple feature groups, each of which contains a certain number of channels. Then, spatial modeling is performed using the coefficient matrices corresponding to these feature groups. These coefficient matrices are obtained through learning and can capture the spatial relationship between features. The generation of the coefficient matrix involves the initialization of the original coefficient matrix, as well as the temporal pooling and convolution of the input features. The dynamic parameters are obtained through the Softmax and Tanh activation functions, and finally the coefficient matrix for spatial modeling is formed.

[0041] For details, please refer to Figure 3 , Figure 3 In represents the GCN block, g, They represent the spatial convolution module and the temporal convolution module respectively. Figure 4 The main steps of the grouped temporal feature extraction backbone network are: Convolutional processing generates K feature groups, each of which contains C / K channels. Subsequently, each feature group is independently spatially modeled and the corresponding coefficient matrix A is used. i Finally, these K feature groups are concatenated in the channel dimension and then passed through a Convolution is performed to generate the final output.

[0042] In this embodiment, the coefficient matrix A i It consists of three learnable parts. First, the initial coefficient matrix is ​​generated by PA based on normal distribution initialization. iThis step ensures that the initial parameters have a certain degree of randomness and diversity so that they can be effectively optimized and learned during the training process. Secondly, the coefficient matrix includes the dynamic parameters DA related to the learning data i and CA i , these dynamic parameters can capture the specific dynamic changes of input features by analyzing and processing the characteristics of the input data. Finally, the weighted sum of its three components is used as the final coefficient matrix. Among them, Figure 5 Demonstrates the acquisition of dynamic parameters DA i and CA i steps.

[0043] In this embodiment, in the temporal convolution layer, the present invention averages the joint-level skeleton features to obtain the joint-level skeleton features, and concatenates and groups the joint-level and joint-level features. Each grouped feature group is temporally modeled, dynamically fused with the joint-level skeleton features and learnable parameters, and then the joint-level and joint-level features are added to obtain the temporal convolution processing result. In this way, the grouped temporal feature extraction results are obtained, which integrate the spatial and temporal information and provide rich feature representation for subsequent action classification and boundary positioning.

[0044] Specifically, in the temporal convolution layer, the V joint-level features are first average-pooled to obtain the joint-level skeleton features, and then the joint-level skeleton features and the joint-level skeleton features are concatenated, and the sampling grouping strategy is also adopted, such as Figure 6 As shown in the figure, a grouped temporal convolution layer consists of multiple branches with different receptive fields, each of which independently models a feature group in time. In addition, D-JF (Dynamic-Joint Feature) contains a learnable parameter about the joint-level skeleton feature. , dynamic fusion And the joint-level skeleton features, and finally the joint-level skeleton features are added to the joint-level skeleton features to get the final output.

[0045] S2, using a neck network to preprocess the grouped temporal feature extraction result to obtain a feature feedback result, wherein the neck network is composed of a temporal multi-scale attenuation network and a temporal feature pyramid;

[0046] Specifically, step S2 includes: using the temporal multi-scale attenuation network to perform a downsampling operation on the features in the group temporal feature extraction result, and inputting the operated data into the context perception layer TCA to obtain a fused branch feature, wherein the context perception layer TCA includes a first branch and a second branch;

[0047] The fused branch features are input into the temporal feature pyramid, and the high-level features are interpolated in the time dimension to align the high-level features with the low-level features, and feature fusion processing is performed to obtain feature feedback results.

[0048] Preferably, the first branch of the context perception layer TCA performs average pooling processing, full connection layer processing and ReLU activation function processing in the time dimension, specifically:

[0049] Extracting posture features from the grouped temporal feature extraction results Perform average pooling on the joint V channel to obtain the aggregated temporal information features ;

[0050] Based on time series information features , performing a downsampling operation in each layer of the temporal multi-scale attenuation network, and inputting the operated data into the context perception layer TCA;

[0051] Time series information features Perform average pooling in the time dimension, and input the processed features into the fully connected layer and ReLU activation function for calculation;

[0052] Multiply the calculated features with the 1D convolution on the time series to get the features .

[0053] Preferably, the second branch of the context perception layer TCA convolutionally parses the action details in time sequence and is processed by the Conv1D convolution layer and the GELU activation function, specifically:

[0054] Two independent one-dimensional convolutions are used to perform the temporal information feature Multiply them to get the features ;

[0055] The characteristics With features Fusion is performed to obtain fusion branch features , the calculation formula is: , , where GELU is a nonlinear activation function, Conv is a deep one-dimensional convolution in the time dimension, ReLU represents the ReLU activation function, FC represents a deep one-dimensional convolution on the fully connected layer, AvgPool represents aggregation in the time dimension, and d represents downsampling. Represents a one-dimensional temporal convolution operation with a window size of wv, Represents a one-dimensional temporal convolution operation with a window size of wk.

[0056] In this embodiment, the grouped temporal feature extraction results are further processed by the neck network to obtain feature feedback results. The neck network consists of a temporal multi-scale attenuation network and a temporal feature pyramid. This design enhances the model's ability to model temporal information and improves the classification accuracy of sports video action frames.

[0057] First, the features in the grouped temporal feature extraction results are downsampled using a temporal multi-scale attenuation network. This step helps capture action features at different time scales and enhances the model's understanding of the long-term dependencies and contextual relationships of actions. The downsampled data is input into the context-aware layer TCA, which includes two branches, responsible for processing long-term dependencies and action details respectively.

[0058] In the first branch of the context-aware layer TCA, the posture features are average-pooled on the joint V channels to aggregate the temporal information features. This aggregation operation not only improves the discriminability of temporal information, but also significantly reduces the number of learning parameters. Next, downsampling operations are performed in each layer of the temporal multi-scale attenuation network, and the results are input into the fully connected layer and the ReLU activation function for calculation. This process further strengthens the model's understanding of the overall action context and improves the distinguishability of action categories. Finally, the calculated features are multiplied by the 1D convolution on the temporal sequence to obtain a richer feature representation.

[0059] In the second branch of the context-aware layer TCA, two independent one-dimensional convolutions are used to multiply the temporal information features to accurately understand the details of the action. This design improves the separability between different moments, allowing the model to more accurately capture the detailed changes in the action, and to more accurately understand the action details in the posture features, so as to improve the separability between different moments. Subsequently, the output results of the two branches are fused to obtain the fused branch features. This fusion not only improves the performance of the model, but also makes the model more adaptable to complex action scenes; in addition, it can also enable the model to better understand the hierarchical structure and intrinsic relationships of the action. This fusion method from features of different scales helps the network to more comprehensively understand high-level semantic feature information. Its architecture is as follows: Figure 7 shown.

[0060] The fused branch features are input into the temporal feature pyramid. Here, the high-level features are interpolated in the time dimension to align the high-level features with the low-level features, and feature fusion is performed. This step helps the network to understand the high-level semantic feature information more comprehensively, and through the fusion of features at different scales, the model's understanding of the hierarchical structure and internal relationship of the action is enhanced. Finally, feature feedback results are obtained, which will be used for subsequent action classification and boundary location tasks.

[0061] S3, transmitting the feature feedback result to the classification detection head and the regression detection head to detect the features of each time t of all L layers on the temporal feature pyramid to obtain the detection result.

[0062] Specifically, step S3 includes: based on the feature feedback result, using the three-layer 1D convolution of the classification detection head to detect the features of each time t of all L layers on the temporal feature pyramid, and using the ReLU activation function to predict the action probability corresponding to each time t;

[0063] Based on the feature feedback result, the 1D convolution of the regression detection head is used to detect the features of each time t of all L layers on the temporal feature pyramid, and the ReLU activation function is used for distance estimation.

[0064] Preferably, the loss function of the classification detection head is focal loss, the loss function of the regression detection head is regression loss, and the formula of the overall loss function is: , , where T1 is the length of the input sequence, is the indicator function, T1+ is the total number of positive samples, is the closeness to the true label category, is a learnable parameter, B is the closeness to the true label category, is the action boundary of the true label, To balance the coefficients of classification and regression loss, For B and The square of the distance between the center points of B and The coverage length between.

[0065] In this embodiment, the feature feedback results processed by the neck network are transmitted to the classification detection head and the regression detection head to detect the features of each time t of all L layers on the temporal feature pyramid, and finally obtain the detection results. This step is the core of the whole method, which converts the previously extracted and processed features into specific detection outputs, including the classification and boundary location of the action.

[0066] Specifically, based on the feature feedback results, the three-layer 1D convolution of the classification detection head is used to detect and process the features of each moment t of all L layers on the temporal feature pyramid. In this process, the ReLU activation function is used to predict the action probability corresponding to each moment t. This lightweight 1D convolutional network structure is not only parameter-efficient, but also can capture local dependencies in temporal features, improving the accuracy of action classification. At the same time, the 1D convolution of the regression detection head is used to detect and process the features of each moment t of all L layers on the temporal feature pyramid, and the ReLU activation function is used for distance estimation. The task of the regression head is to predict the start and end boundaries of the action, which is particularly important for fine-grained action detection. By pre-specifying the regression range for each layer and adopting the 1D convolutional network structure, the output regression range requirements of different levels can be met, and the accuracy of boundary positioning is improved.

[0067] In the design of the loss function, a focal loss is selected for the classification detection head. This loss function helps the model pay more attention to samples that are difficult to classify, thereby improving the model's ability to recognize minority categories. For the regression detection head, a regression loss is selected, which focuses on minimizing the difference between the predicted boundary and the true boundary. The overall loss function combines these two losses. By balancing the coefficients of the classification and regression losses, it is able to optimize the accuracy of boundary positioning while maintaining the action classification performance. The design of this loss function not only improves the model's ability to recognize action categories, but also enhances the model's understanding of action timing information through precise boundary positioning. This design enables the model to capture the start and end moments of the action more accurately, providing strong technical support for fine-grained sports action detection.

[0068] In summary, the joint sports action detection method improves the performance of fine-grained sports sub-action detection through the processing of key modules; in terms of feature extraction, a new skeleton feature extraction network is constructed to improve the representation ability and flexibility of feature extraction. At the same time, in the neck module, the method adopts a strategy combining a temporal multi-scale attenuation network and a temporal feature pyramid to improve the accuracy of sub-action instance detection and make the overall design more efficient. These innovations not only improve the performance of the model, but also make the model more adaptable to complex action scenes, providing a new technical solution for sports video analysis.

[0069] In short, the joint sports action detection method aims to improve the detection performance of sub-actions in sports, and achieves efficient detection of sports actions through a carefully designed network structure and loss function. This method not only improves the accuracy and efficiency of action detection, but also provides richer data support for sports analysis through fine-grained boundary positioning. This method has broad application prospects and can play an important role in many fields such as sports event analysis, sports training and rehabilitation, and sports health data collection, providing new possibilities for technological progress and application innovation in related fields.

[0070] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A combined sports action detection method, characterized in that: include: Acquire a video to be detected, extract the video to be detected, obtain a skeleton feature tensor, and input the skeleton feature tensor into a preset grouping temporal feature extraction backbone network to perform extraction processing to obtain a grouping temporal feature extraction result; Preprocessing the grouped temporal feature extraction result using a neck network to obtain a feature feedback result, wherein the neck network is composed of a temporal multi-scale attenuation network and a temporal feature pyramid; The feature feedback result is transmitted to the classification detection head and the regression detection head to detect the features of each time t of all L layers on the temporal feature pyramid to obtain the detection result; The neck network is used to preprocess the grouping time series feature extraction results to obtain feature feedback results, specifically: Using the temporal multi-scale attenuation network to perform a downsampling operation on the features in the group temporal feature extraction result, and inputting the operated data into the context perception layer TCA to obtain a fused branch feature, wherein the context perception layer TCA includes a first branch and a second branch; The fused branch features are input into the temporal feature pyramid, and the high-level features are interpolated in the time dimension to align the high-level features with the low-level features, and feature fusion is performed to obtain feature feedback results; The feature feedback result is transmitted to the classification detection head and the regression detection head to detect the features of each time t of all L layers on the temporal feature pyramid to obtain the detection result, which is specifically: Based on the feature feedback result, the three-layer 1D convolution of the classification detection head is used to detect the features of each time t of all L layers on the temporal feature pyramid, and the ReLU activation function is used to predict the action probability corresponding to each time t; Based on the feature feedback result, the 1D convolution of the regression detection head is used to detect the features of each time t of all L layers on the temporal feature pyramid, and the ReLU activation function is used for distance estimation.

2. The combined sports action detection method according to claim 1, characterized in that: Obtain a video to be detected, extract the video to be detected, obtain a skeleton feature tensor, and input the skeleton feature tensor into a preset grouping temporal feature extraction backbone network for extraction processing to obtain a grouping temporal feature extraction result, specifically: Get the video v to be detected i , use the OpenPose algorithm to distort the video v i Perform extraction processing to extract video v i The skeleton sequence of the video frames in the video is obtained to obtain the skeleton feature tensor , where R is a set of real numbers and T is the time channel dimension; The skeleton feature tensor X is input into the grouped temporal feature extraction backbone network, and the skeleton feature tensor X is preprocessed using N grouped GCN blocks of the grouped temporal feature extraction backbone network to obtain a grouped temporal feature extraction result, wherein each of the grouped GCN blocks consists of a spatial convolution layer and a temporal convolution layer.

3. The combined sports action detection method according to claim 2, characterized in that: The skeleton feature tensor X is preprocessed using the N grouped GCN blocks of the grouped temporal feature extraction backbone network to obtain a grouped temporal feature extraction result, specifically: The skeleton feature tensor X is Convolution processing, get K feature groups, where each feature group contains channels, C is the input channel; Based on the coefficient matrix A corresponding to the feature group i , spatial modeling is performed for each feature group; All the feature groups after modeling are concatenated in the channel dimension, and the concatenated feature groups are Convolution processing is performed to obtain the spatial convolution processing result.

4. The combined sports action detection method according to claim 3, characterized in that: The coefficient matrix A i The generation steps include: Get the original coefficient matrix A for spatial modeling; Use normal distribution to initialize the original coefficient matrix A and generate the initial coefficient matrix PA i , where the initial coefficient matrix PA i Including dynamic parameters DA that are independent of the characteristic channel i and the dynamic parameters CA for the feature channel i ; For the K input features of the feature group, a time pooling layer is used to eliminate the time channel dimension T of the input features of the feature group, and then two The convolution block performs convolution processing and obtains the convolution results X a and X b ; Among them, the dynamic parameter DA is independent of the characteristic channel. i is obtained by convolution result X a and X b The dynamic parameter CA for the feature channel is obtained by multiplying the product and then activating it with the Softmax activation function. i is obtained by convolution result X a and X b The result is obtained by performing subtraction and activating with Tanh activation function. The initial coefficient matrix PA i , dynamic parameter DA independent of feature channel i and the dynamic parameters CA for the feature channel i Perform weighted processing to obtain the coefficient matrix A i , the formula is: , ,in, and These are all learnable parameters.

5. The combined sports action detection method according to claim 3, characterized in that: Specifically: In the temporal convolution layer, based on the spatial convolution processing result, average pooling processing is performed on the V joint-level skeleton features to obtain a joint-level skeleton feature; The joint-level skeleton features and the joint-level skeleton features are spliced ​​and sampled and grouped, and each grouped feature group is subjected to temporal modeling, and the joint-level skeleton features and their corresponding learnable parameters are dynamically fused. , where V is the joint channel dimension; Adding the joint-level skeleton features and the joint-level skeleton features to obtain a temporal convolution processing result; According to the convolution processing result and the spatial convolution processing result, a grouping time series feature extraction result is obtained.

6. The combined sports action detection method according to claim 1, characterized in that: The first branch of the context perception layer TCA performs average pooling processing, full connection layer processing and ReLU activation function processing in the time dimension, specifically: Extracting posture features from the grouped temporal feature extraction results Perform average pooling on the joint V channel to obtain the aggregated temporal information features ; Based on time series information features , performing a downsampling operation in each layer of the temporal multi-scale attenuation network, and inputting the operated data into the context perception layer TCA; Time series information features Perform average pooling in the time dimension, and input the processed features into the fully connected layer and ReLU activation function for calculation; Multiply the calculated features with the 1D convolution on the time series to get the features .

7. The combined sports action detection method according to claim 6, characterized in that: The second branch of the context perception layer TCA convolutionally parses the action details in time sequence and is processed by the Conv1D convolution layer and the GELU activation function, specifically: Two independent one-dimensional convolutions are used to perform the temporal information feature Multiply them to get the features ; The characteristics With features Fusion is performed to obtain fusion branch features , the calculation formula is: , , where GELU is a nonlinear activation function, Conv is a deep one-dimensional convolution in the time dimension, ReLU represents the ReLU activation function, FC represents a deep one-dimensional convolution on the fully connected layer, AvgPool represents aggregation in the time dimension, and d represents downsampling.

8. The combined sports action detection method according to claim 1, characterized in that: The loss function of the classification detection head is focal loss, the loss function of the regression detection head is regression loss, and the formula of the overall loss function is: , , where T1 is the length of the input sequence, is the indicator function, T1+ is the total number of positive samples, B is the closeness to the true label category, is the action boundary of the true label, To balance the coefficients of classification and regression loss, For B and The square of the distance between the center points of B and The coverage length between.

Images