Football video feature extraction method

Abstract

The invention discloses a football video feature extraction method, which comprises the following steps: S10, segmenting a football video into continuous image frames, each frame representing an instantaneous picture in the video, and extracting key features from each frame to obtain structured feature data; s20, combining the structured feature data extracted from each frame of image into a first feature vector matrix; s30, performing dimensionality reduction on the feature matrix by a principal component analysis method to obtain a dimensionality-reduced second feature vector matrix; s40, performing linear discriminant analysis on the second feature vector matrix and the category label of the video data to obtain a low-dimensional feature matrix after linear discriminant analysis projection; and S50, in combination with the low-dimensional feature matrix and the category label, obtaining a classification result of the behavior category to which the football video belongs.

Description

Technical Field

[0001] The invention belongs to the technical field of sports video analysis, and in particular relates to a method for extracting features from a football video. Background Art

[0002] At present, in the field of sports video analysis, especially football video analysis, widely used technologies include computer vision, deep learning, feature extraction, and traditional machine learning algorithms. The following are some mainstream technologies and applications: (1) Deep learning models, especially convolutional neural networks (CNNs) and recurrent neural networks (RNNs), have achieved remarkable success in video classification and action recognition. For example, CNNs are used to extract spatial features of image frames, while RNNs process time series data, such as the movement trajectory of players in videos. These methods rely on a large amount of training data and can capture high-order features in complex scenes. (2) Optical flow is a classic method for extracting motion information from videos. It can capture the speed and direction of objects in the video and has a wide range of applications in sports video analysis (such as the movement of players and balls). Combined with machine learning technology, optical flow can generate dynamic features for analyzing different tactical behaviors in the game. (3) Action recognition and classification: By extracting key point information in the video (such as player joints, ball position, etc.) and combining it with classification algorithms, specific behaviors such as attack, defense, and passing in the game can be identified. Deep learning models usually rely on large-scale labeled data and achieve high recognition accuracy. (4) Spatiotemporal feature extraction: Some methods capture dynamic information in time and space through spatiotemporal convolution (such as 3D CNN) or graph neural network (GNN), helping to better understand the dynamic changes of the game.

[0003] Although current technologies have achieved certain results in sports video analysis, they still have many limitations, especially in the following aspects: (1) Huge data requirements. Deep learning models require a large amount of labeled data to achieve good performance. This is a major challenge in practical applications because labeling various behaviors in football videos (such as passing, shooting, confrontation, etc.) requires a lot of time and human resources. In addition, different game scenes and tactical modes may require different data sets, which increases the complexity of data collection and annotation. (2) High computing resource consumption. Complex deep learning models (such as 3DCNN, two-stream CNN, etc.) have high computational complexity and require powerful computing resources (such as GPU clusters) for training and inference. This makes these models difficult to deploy in real-time applications, especially for real-time analysis and live broadcast of games. (3) Poor model interpretability. The black box nature of deep learning methods makes it difficult to explain how the model makes specific decisions. This is particularly undesirable in scenarios that require interpretability and understanding, such as football tactical analysis. Team coaches and analysts need not only the results, but also know how the model reached these conclusions. (4) Generalization problem. Current models generally perform well on specific datasets, but their generalization ability is poor under different environments such as game conditions, camera angles, weather conditions, etc. This makes it difficult for the model to be generalized to various real-world scenarios. (5) Unable to capture global and local information. Some technologies (such as optical flow and traditional spatiotemporal feature extraction) can only capture local motion information and cannot fully analyze the global tactics of the game. Although action recognition technologies perform well on certain local features, they fail to capture the complex interactions between players and the ball and the overall game dynamics. Summary of the invention

[0004] In view of the above problems, the present invention provides a method for extracting features from football videos, which is used to improve the classification effect while ensuring efficient processing of football video features under the combination of principal component analysis method PCA and linear discriminant analysis LDA. It is very suitable for processing complex dynamic scenes in football match videos and helping to achieve intelligent analysis and understanding of football matches.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] A method for extracting features from a football video comprises the following steps:

[0007] S10, dividing the football video into continuous image frames, each frame represents an instantaneous picture in the video, extracting key features from each frame to obtain structured feature data;

[0008] S20, combining the structured feature data extracted from each frame of the image into a first feature vector matrix;

[0009] S30, performing a principal component analysis method on the feature matrix to reduce the dimension, and obtain a second feature vector matrix after the dimension reduction;

[0010] S40, performing linear discriminant analysis on the second eigenvector matrix and the category label of the video data to obtain a low-dimensional feature matrix after linear discriminant analysis projection;

[0011] S50, combining the low-dimensional feature matrix and the category label to obtain the classification result of the behavior category to which the football video belongs.

[0012] In a possible implementation, in S10, the key feature includes at least one of a player's position, a ball's position, and a field feature.

[0013] In a possible implementation, in S20, the first eigenvector matrix X is an N×d eigenvector matrix, where N is the number of frames and d is the dimension of the eigenvector of each frame.

[0014] In a possible implementation manner, S30 specifically includes:

[0015] S301, performing a mean removal process on the first eigenvector matrix X, that is, subtracting the mean of the corresponding feature from each eigenvector,

[0016] Calculate the mean vector μ: Where X i is the eigenvector of the i-th row, and the matrix X′ after removing the mean is: X′=X-μ;

[0017] S302, calculate the covariance matrix Σ of the feature matrix X′:

[0018] S303, calculate the eigenvalues ​​and eigenvectors of the covariance matrix, and select the eigenvectors corresponding to the largest k eigenvalues ​​to form the eigenspace V after dimensionality reduction k ;

[0019] S304, projecting the original data X′ to the feature subspace V after dimensionality reduction k In the reduced dimension, the second feature matrix X is obtained k .

[0020] In a possible implementation, the S303 calculates the eigenvalues ​​and eigenvectors of the covariance matrix, and selects the eigenvectors corresponding to the largest k eigenvalues ​​to form a eigenspace V after dimensionality reduction. k Specifically include:

[0021] Perform eigendecomposition on the covariance matrix Σ to obtain eigenvalues ​​and eigenvectors;

[0022] The eigenvalues ​​of the covariance matrix Σ represent the variance of the data in the direction of each eigenvector, solving the characteristic equation: Σv i =λ i v i ,

[0023] where λ i is the eigenvalue of the covariance matrix Σ, v i is the corresponding eigenvector;

[0024] Sort by eigenvalue λ from large to small, and select the eigenvectors v1, v2, ..., v corresponding to the first k eigenvalues. k , forming the feature subspace after dimensionality reduction.

[0025] In a possible implementation manner, the S40 specifically includes:

[0026] S401, assuming that the football video data is divided into c categories;

[0027] S402, calculate the intra-class scatter matrix S for each class of data W and the inter-class scatter matrix S B ;

[0028] S403, by maximizing S B and minimize S W The ratio of , find an optimal projection direction;

[0029] S404, further projecting the reduced-dimensional features into a linear space of an optimal projection direction to obtain a projected low-dimensional feature matrix.

[0030] In a possible implementation, in S402, the intra-class scatter matrix S W The calculation methods include:

[0031] The intra-class scatter matrix S W It reflects the compactness of the sample distribution within each class and measures the dispersion between the samples within the class and the mean of the class. There are c classes, and the samples of each class are recorded as X1, X2, ..., X c , each category has N i samples, a total of N samples,

[0032] The mean vector for class i is:

[0033] The population mean vector is: The intra-class scatter matrix S W for: Where S W The dimension of is d×d, where d is the feature dimension.

[0034] In a possible implementation, the inter-class scatter matrix S B The calculation method includes: the inter-class scatter matrix S B It reflects the dispersion between class means and measures the degree of separation between different classes. The inter-class dispersion matrix S B The calculation formula is as follows:

[0035] In a possible implementation manner, the S403 specifically includes:

[0036] Maximize the inter-class scatter matrix S through linear discriminant analysis B and the intra-class scatter matrix S W The ratio of finds the best linear projection direction, and the ratio is expressed by the following objective function:

[0037]

[0038] By solving the Lagrange multiplier method and maximizing J(w), it is finally transformed into a generalized eigenvalue problem:

[0039]

[0040] Among them, λ is the eigenvalue and ω is the corresponding eigenvector, representing the optimal linear projection direction.

[0041] In a possible implementation manner, the S404 specifically includes: selecting eigenvectors ω1, ω2, ...ω corresponding to the first k largest eigenvalues k , forming the projection matrix W k , project the sample into a low-dimensional space, the projection matrix W k The calculation is as follows: X LDA =XW k ;

[0042] Where X is the original data matrix, W k is a d×k matrix, X LDA It is the low-dimensional feature matrix after projection.

[0043] The present invention has the following beneficial effects:

[0044] (1) Improved feature extraction efficiency: PCA is used to reduce the dimensionality of high-dimensional features, effectively removing redundant information and retaining the most representative features in the data. This dimensionality reduction can reduce feature dimensions and computational complexity, making subsequent LDA processing and classification more efficient.

[0045] (2) Improvement of classification accuracy: LDA significantly improves the accuracy of feature classification by increasing the discrimination between classes. It is effective in distinguishing different types of football behaviors, for example, it can clearly identify different tactical actions such as attack, defense, and passing.

[0046] (3) Saving of computing resources: PCA compresses the dimensions of the original data, while LDA reduces the amount of computation required by optimizing the classification process, significantly reducing the consumption of computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 The present invention is a flowchart of a method for extracting football video features according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] See also Figure 1 , which is a flowchart of a method for extracting features from a football video according to an embodiment of the present invention, comprises the following steps:

[0050] S10, dividing the football video into continuous image frames, each frame represents an instantaneous picture in the video, extracting key features from each frame to obtain structured feature data;

[0051] S20, combining the structured feature data extracted from each frame of the image into a first feature vector matrix;

[0052] S30, performing a principal component analysis method on the feature matrix to reduce the dimension, and obtain a second feature vector matrix after the dimension reduction;

[0053] S40, performing linear discriminant analysis on the second eigenvector matrix and the category label of the video data to obtain a low-dimensional feature matrix after linear discriminant analysis projection;

[0054] S50, combining the low-dimensional feature matrix and the category label to obtain the classification result of the behavior category to which the football video belongs.

[0055] A specific application example, in S10, a complete football video 'input_video.mp4' is input, and a continuous sequence of image frames is output, each frame is a static image file frame_i.png, where i represents the frame number. The key features extracted from each frame can be such as the position of the player, the position of the ball, the field features, etc. A convolutional neural network (CNN) can be used to extract dynamic features. The player position is the coordinate of each player on the field. The football position is the specific position of the football in the picture frame. The motion feature is the dynamic information such as the speed and direction of the player and the ball, which can be obtained through continuous calculation of multiple frames. For example, the frame number is 1; the feature data includes:

[0056] Player 1 position: [100,200]; Player 2 position: [300,400]; Ball position: [150,250]; Player 1 speed: 3m / s, direction: 45 degrees; Ball speed: 5m / s, direction: 90 degrees.

[0057] In a specific application example, in S20, the first feature vector matrix X is an N×d feature matrix, where N is the number of frames and d is the dimension of the feature vector of each frame.

[0058] In a specific application example, since the feature dimension extracted from the football video is very high, in order to reduce the amount of calculation and remove redundant information, the principal component analysis method PCA is first used to reduce the dimension. S30 specifically includes:

[0059] S301, performing a mean removal process on the first eigenvector matrix X, that is, subtracting the mean of the corresponding feature from each eigenvector,

[0060] Calculate the mean vector μ: Where X i is the eigenvector of the i-th row, and the matrix X′ after removing the mean is: X′=X-μ;

[0061] S302, calculate the covariance matrix Σ of the feature matrix X′:

[0062] S303, calculate the eigenvalues ​​and eigenvectors of the covariance matrix, and select the eigenvectors corresponding to the largest k eigenvalues ​​to form the eigenspace V after dimensionality reduction k ;

[0063] S304, projecting the original data X′ to the feature subspace V after dimensionality reduction k In the reduced dimension, the second feature matrix X is obtained k .

[0064] Further, S303, the eigenvalues ​​and eigenvectors of the covariance matrix are calculated, and the eigenvectors corresponding to the largest k eigenvalues ​​are selected to form the eigenspace V after dimensionality reduction.k Specifically include:

[0065] Perform eigendecomposition on the covariance matrix Σ to obtain eigenvalues ​​and eigenvectors;

[0066] The eigenvalues ​​of the covariance matrix Σ represent the variance of the data in the direction of each eigenvector, solving the characteristic equation: Σv i =λ i v i ,

[0067] where λ i is the eigenvalue of the covariance matrix Σ, v i is the corresponding eigenvector;

[0068] Sort by eigenvalue λ from large to small, and select the eigenvectors v1, v2, ..., v corresponding to the first k eigenvalues. k , forming the feature subspace after dimensionality reduction.

[0069] Through the above S301 to S304, an N×d feature matrix is ​​input and a reduced-dimensional feature matrix N×k is output, where k is also the number of main components selected by PCA.

[0070] In a possible implementation manner, the S40 specifically includes:

[0071] S401, assuming that the football video data is divided into c categories;

[0072] S402, calculate the intra-class scatter matrix S for each class of data W and the inter-class scatter matrix S B ;

[0073] S403, by maximizing S B and minimize S W The ratio of , find an optimal projection direction;

[0074] S404, further projecting the reduced-dimensional features into a linear space of an optimal projection direction to obtain a projected low-dimensional feature matrix.

[0075] Furthermore, PCA is mainly used for dimensionality reduction, but it may not be able to separate data of different categories (such as offense, defense, etc.) well. In order to further improve the classification effect, LDA is used to maximize the difference between classes while minimizing the difference within classes. In S402, the intra-class scatter matrix S W The calculation method includes: the intra-class scatter matrix S W It reflects the compactness of the sample distribution within each class, and measures the dispersion between the samples within the class and the mean of the class. There are c categories (such as offense, defense, transition, etc.), and the samples of each class are recorded as X1, X2, ..., X c , each category has Ni samples, a total of N samples,

[0076] The mean vector for class i is:

[0077] The population mean vector is: The intra-class scatter matrix S W for:

[0078] Where S W The dimension of is d×d, where d is the feature dimension.

[0079] Furthermore, the inter-class scatter matrix S B The calculation method includes: the inter-class scatter matrix S B It reflects the dispersion between class means and measures the degree of separation between different classes. The inter-class dispersion matrix S B The calculation formula is as follows:

[0080] Furthermore, S403 specifically includes:

[0081] Maximize the inter-class scatter matrix S through linear discriminant analysis B and the intra-class scatter matrix S W The ratio of finds the best linear projection direction, and the ratio is expressed by the following objective function:

[0082]

[0083] By solving the Lagrange multiplier method and maximizing J(w), it is finally transformed into a generalized eigenvalue problem:

[0084]

[0085] Among them, λ is the eigenvalue, w is the corresponding eigenvector, representing the optimal linear projection direction.

[0086] Further, S404 specifically includes: selecting eigenvectors ω1, ω2, ...ω corresponding to the first k largest eigenvalues k , forming the projection matrix W k , project the sample into a low-dimensional space, the projection matrix W k The calculation is as follows: X LDA =XW k ;

[0087] Where X is the original data matrix, W k is a d×k matrix, X LDA It is the low-dimensional feature matrix after projection.

[0088] In a specific application example, in S50, after PCA and LDA processing, a feature vector set in a low-dimensional space is finally obtained, which can well represent the differences between different categories. The classification algorithm is continued to be used to classify the video and analyze which type of behavior it belongs to (attack, shoot, defense, etc.).

[0089] The above-mentioned method for extracting football video features has at least the following improvement effects:

[0090] (1) Improved feature extraction efficiency: PCA is used to reduce the dimensionality of high-dimensional features, effectively removing redundant information and retaining the most representative features in the data. This dimensionality reduction can reduce feature dimensions and computational complexity, making subsequent LDA processing and classification more efficient. Without PCA, the amount of data required to process high-dimensional features is large, which can easily lead to overfitting and poor classification results. After the introduction of PCA, the significant reduction in feature dimensions makes the classification model more generalizable and reduces the risk of overfitting. Taking a 60-second football video clip with a feature dimension of 500 as an example, after PCA dimensionality reduction, the feature dimension is significantly reduced to 50 dimensions, and the computational complexity is greatly reduced.

[0091] (2) Improvement of classification accuracy: LDA significantly improves the accuracy of feature classification by increasing the discrimination between classes. It is effective in distinguishing different types of football behaviors, for example, it can clearly identify different tactical actions such as attack, defense, and passing. Compared with traditional classification methods (such as k-nearest neighbor, support vector machine, etc.), classification algorithms that have not been processed by LDA may find it difficult to obtain high accuracy in complex video scenes. By combining PCA and LDA, the classifier can more accurately separate different categories in the feature space, thereby improving the classification accuracy. Taking the original video clip of 60 seconds with a feature dimension of 500 dimensions as an example, PCA reduces the dimension, which enhances the generalization ability of the classifier, while LDA further optimizes the classification boundary and greatly improves the classification accuracy. The classification accuracy of the original high-dimensional features is 75%, the classification accuracy after PCA dimensionality reduction alone is 82%, and the classification accuracy after PCA dimensionality reduction and LDA classification is 90%.

[0092] (3) Saving of computing resources: The consumption of computing resources is significantly reduced. PCA compresses the dimension of the original data, and LDA reduces the amount of computing required by optimizing the classification process. Therefore, this method can process large-scale video data with lower computing resources and is suitable for real-time analysis or devices with limited resources. If the original high-dimensional data is processed directly, not only will the computing resources required be large, but it will also take a long time to process. The dimension reduction process based on PCA reduces the redundant information in the feature space, making the entire computing process more efficient. Taking a 60-second video clip of a football video with a feature dimension of 500 dimensions (under the same hardware) as an example, after PCA dimension reduction, the processing time is greatly reduced, and LDA classification further optimizes the computing time. The time to directly process the original high-dimensional data is 150 seconds, the processing time after PCA dimension reduction is 60 seconds, and it takes 55 seconds to process the data after LDA classification.

[0093] It should be understood that the exemplary embodiments described herein are illustrative rather than restrictive. Although one or more embodiments of the present invention are described in conjunction with the accompanying drawings, it should be understood by those skilled in the art that various changes in form and detail may be made without departing from the spirit and scope of the present invention as defined by the appended claims.

Claims

1. A method for extracting features from a football video, characterized in that: The following steps are involved: S10, dividing the football video into continuous image frames, each frame represents an instantaneous picture in the video, extracting key features from each frame to obtain structured feature data; S20, combining the structured feature data extracted from each frame of the image into a first feature vector matrix; S30, performing a principal component analysis method on the feature matrix to reduce the dimension, and obtain a second feature vector matrix after the dimension reduction; S40, performing linear discriminant analysis on the second eigenvector matrix and the category label of the video data to obtain a low-dimensional feature matrix after linear discriminant analysis projection; S50, combining the low-dimensional feature matrix and the category label to obtain the classification result of the behavior category to which the football video belongs.

2. The method for extracting football video features as claimed in claim 1, characterized in that: In S10, the key feature includes at least one of a player's position, a ball's position, and a field feature.

3. The method for extracting football video features as claimed in claim 1, characterized in that: In S20, the first feature vector matrix X is an N×d feature matrix, where N is the number of frames and d is the dimension of the feature vector of each frame.

4. The method for extracting football video features as claimed in claim 3, characterized in that: S30 specifically includes: S301, performing a mean removal process on the first eigenvector matrix X, that is, subtracting the mean of the corresponding feature from each eigenvector, Calculate the mean vector μ: Where X i is the eigenvector of the i-th row, and the matrix X′ after removing the mean is: X′=X-μ; S302, calculate the covariance matrix Σ of the feature matrix X′: S303, calculate the eigenvalues ​​and eigenvectors of the covariance matrix, and select the eigenvectors corresponding to the largest k eigenvalues ​​to form the eigenspace V after dimensionality reduction k ; S304, projecting the original data X′ to the feature subspace V after dimensionality reduction k In the reduced dimension, the second feature matrix X is obtained k .

5. The method for extracting football video features as claimed in claim 4, characterized in that: S303, calculates the eigenvalues ​​and eigenvectors of the covariance matrix, and selects the eigenvectors corresponding to the largest k eigenvalues ​​to form the eigenspace V after dimensionality reduction k Specifically include: Perform eigendecomposition on the covariance matrix Σ to obtain eigenvalues ​​and eigenvectors; The eigenvalues ​​of the covariance matrix Σ represent the variance of the data in the direction of each eigenvector, solving the characteristic equation: Σv i =λ i v i , where λ i is the eigenvalue of the covariance matrix Σ, v i is the corresponding eigenvector; Sort by eigenvalue λ from large to small, and select the eigenvectors v1, v2, ..., v corresponding to the first k eigenvalues. k , forming the feature subspace after dimensionality reduction.

6. The method for extracting football video features as claimed in claim 4, characterized in that: The S40 specifically includes: S401, assuming that the football video data is divided into c categories; S402, calculate the intra-class scatter matrix S for each class of data W and the inter-class scatter matrix S B ; S403, by maximizing S B and minimize S W The ratio of , find an optimal projection direction; S404, further projecting the reduced-dimensional features into a linear space of an optimal projection direction to obtain a projected low-dimensional feature matrix.

7. The method for extracting football video features as claimed in claim 6, characterized in that: In S402, the intra-class scatter matrix S W The calculation methods include: The intra-class scatter matrix S W It reflects the compactness of the sample distribution within each class and measures the dispersion between the samples within the class and the mean of the class. There are c classes, and the samples of each class are recorded as X1, X2, ..., X c , each category has N i samples, a total of N samples, The mean vector for class i is: The population mean vector is: The intra-class scatter matrix S W for: Where S W The dimension of is d×d, where d is the feature dimension.

8. The method for extracting football video features as claimed in claim 7, characterized in that: The inter-class scatter matrix S B The calculation method includes: the inter-class scatter matrix S B It reflects the dispersion between class means and measures the degree of separation between different classes. The inter-class dispersion matrix S B The calculation formula is as follows:

9. The method for extracting football video features as claimed in claim 8, characterized in that: The S403 specifically includes: Maximize the inter-class scatter matrix S by linear discriminant analysis B and the intra-class scatter matrix S W The ratio of , find the best linear projection direction, the ratio is expressed by the following objective function: By solving the Lagrange multiplier method and maximizing J(w), it is finally transformed into a generalized eigenvalue problem: Among them, λ is the eigenvalue, w is the corresponding eigenvector, representing the optimal linear projection direction.

10. The method for extracting football video features as claimed in claim 9, characterized in that: The step S404 specifically includes: selecting eigenvectors ω1, ω2, ...ω corresponding to the first k largest eigenvalues k , forming the projection matrix W k , project the sample into a low-dimensional space, the projection matrix W k The calculation is as follows: X LDA =XW k ; Where X is the original data matrix, W k is a d×k matrix, X LDA It is the low-dimensional feature matrix after projection.

Images