A football video cross-camera player re-identification method

Abstract

This invention discloses a method for re-identifying players across shots in football videos, belonging to the field of video analysis. The method includes: for each video segment in a football video containing a long shot-non-long shot-long shot sequence, performing target detection on two long shots respectively; clustering the center positions of the target boxes of each player in the second long shot; and using 1 / M of the maximum number of targets in the largest cluster as a threshold Th. 1 And the target number is less than Th 1 Cluster removal; if the ratio of the sum of the areas of the remaining player target boxes to the area of ​​the smallest bounding box that can contain the remaining player target boxes is greater than Th 2 If the shot type is determined to be a densely positioned shot, then posture-guided cross-border player re-identification is performed; otherwise, if the shot type is determined to be a non-densely positioned shot, position-guided player re-identification is performed. This invention can improve the accuracy of shot type determination and adaptively adopt appropriate player re-identification methods to further improve the accuracy of player re-identification.

Description

A method for re-identifying players across camera angles in football videos Technical Field

[0001] This invention belongs to the field of video analysis, and more specifically, relates to a method for re-identifying players across shots in football videos. Background Technology

[0002] In recent years, with the rapid development of computer vision and deep learning technologies, video analytics has been widely applied in the sports field. Especially in football matches, automated video analytics technology provides crucial support for match data collection, player performance evaluation, and tactical analysis.

[0003] In football videos, shots can be categorized into four types based on field of view and zoom level: long shots, medium shots, close-ups, and exterior shots. Long shots showcase the football field, medium shots focus on a specific area, close-ups display a player's facial expressions and body language, and exterior shots depict the environment and people outside the field. Long shots are the most commonly used shot type in football videos and typically last a long time. Therefore, most research focuses on player tracking within long shots. The main objective of the cross-shot player re-identification task, as shown in Figure 1, is to detect and select players from two long shots within a sequence from long shot to non-long shot to long shot (represented as long shot-non-long shot-long shot), generate player distinguishing features using a player re-identification model, and finally match the players from the two long shots using predefined rules.

[0004] General pedestrian re-identification methods mainly rely on deep learning techniques such as depth metrics, local features, and sequence features. Unlike general pedestrian re-identification methods, cross-camera player re-identification in soccer videos presents the following problems:

[0005] (1) Small differences in appearance among football players: In football matches, players’ appearances and clothing are very similar, especially within a team, which makes player re-identification more difficult.

[0006] (2) Blurred, occluded and disappearing player images: Due to the competitive nature of football, players move quickly during the game, and the players will become blurry when the image frame is obtained; or players collide with each other, causing players to be occluded; or the camera moves, causing players to disappear from the field of view, resulting in the problem of players disappearing.

[0007] (3) Lack of datasets: In player re-identification research in the field of football, the number of samples in the dataset is very limited, and it is difficult to obtain more data; due to the differences in match venues, weather, player clothing, etc., the quality and features of player images will also change, which limits the diversity and quality of the dataset;

[0008] (4) Failure to fully utilize knowledge in the field of football: There are many professional knowledge and rules in the field of football, such as player positioning and field area division, but current player re-identification research rarely utilizes this knowledge.

[0009] To address the aforementioned issues, researchers have proposed an adaptive player re-identification model, comprising posture-guided and position-guided player re-identification algorithm modules. This model dynamically selects different algorithm modules through an adaptive switching strategy to adapt to different shot transition types. However, this approach simply relies on Gaussian kernel density estimation to determine the shot transition type. While Gaussian kernel density estimation is widely used to estimate the probability density function of random variables, in real-world football matches, there is interference from players on the periphery, and the number of players in a match is fixed and finite. This introduces a significant possibility of misjudging the shot transition type, leading to the subsequent player re-identification algorithm module being unable to adapt to the actual shot transition types and further limiting the improvement of player re-identification accuracy. Summary of the Invention

[0010] To address the shortcomings and improvement needs of existing technologies, this invention provides a method for player re-identification across different camera shots in football videos. The aim is to improve the accuracy of shot switching type judgment and adaptively adopt appropriate player re-identification methods under different shot switching types, thereby further improving the accuracy of player re-identification.

[0011] To achieve the above objectives, according to one aspect of the present invention, a method for re-identifying players across camera angles in football videos is provided, comprising:

[0012] The football match video to be analyzed is divided into video segments containing long shots, non-long shots, and long shots. For each video segment, the following steps are performed:

[0013] S1: Perform target detection on the first and second long shots in the video clip respectively to obtain the first player's target box set and the second player's target box set;

[0014] S2: Determine the shot type of the current video clip. If it is a shot of densely packed standing positions, proceed to step S3; otherwise, proceed to step S4. Determining the shot type of the current video clip includes:

[0015] S21: Cluster the center positions of each player's target box in the second player target box set, and take the maximum number of targets in the cluster as 1 / M as the threshold Th1, and remove the clusters with fewer targets than the threshold Th1; M is a preset positive number.

[0016] S22: Find the smallest bounding box that can contain the remaining player target boxes, denote its area as S1, and calculate the sum of the areas of the remaining player target boxes as S2;

[0017] S23: If S2 / S1>Th2, then the shot type is determined to be a densely positioned shot; otherwise, the shot type is determined to be a non-densely positioned shot; Th2 is a preset threshold.

[0018] S3: Obtain the fusion features of the appearance and posture features of each player in the two long shots respectively, and calculate the similarity between the players in the two long shots based on the fusion features to obtain the correlation matrix between the two long shots, and then proceed to step S5.

[0019] S4: Obtain the positional distance similarity matrix, appearance similarity matrix, and posture distance similarity matrix between each player in the two long shots, and fuse them as the correlation matrix between the two long shots;

[0020] S5: Based on the correlation matrix between the two long shots and the bipartite graph matching strategy, perform player matching between the two long shots to complete the cross-shot player re-identification of the current video segment.

[0021] Furthermore, M = 5.

[0022] Furthermore, Th2 = 25%.

[0023] Furthermore, the target detection is achieved using the RT-DETR detection model.

[0024] Furthermore, in step S3, for each long shot, the fusion features of the appearance features and posture features of each player are obtained, including: appearance feature extraction, posture feature extraction and appearance feature-posture feature fusion.

[0025] Appearance feature extraction includes:

[0026] A sliding window is used to divide the image corresponding to the player's target box into N image blocks with overlapping pixels, and a learnable linear projection function is used to map each image block to D dimensions to obtain the image block embedding.

[0027] The positions of the player's target box and each image patch are represented by learnable encoded vectors, which serve as player position embeddings and image patch position embeddings, and together with the image patch embeddings, constitute the input sequence.

[0028] The input sequence is fed into the Transformer encoder to obtain the local features of each image patch, which are used as the appearance features of the corresponding player.

[0029] Pose feature extraction includes:

[0030] The image corresponding to the player's target box is used as the original image, and then scaled at different scales.

[0031] For each image, a heatmap of N key points in the image is estimated using a human pose estimation model, which serves as the pose feature of the corresponding image.

[0032] The pose features of each image are weighted and summed to obtain the pose features of the corresponding player.

[0033] Appearance feature-attitude feature fusion includes:

[0034] For the same player's appearance and posture features, establish a correspondence between N local features and N heatmaps. After unifying the corresponding local features and heatmaps to the same dimension, perform a weighted summation to obtain the fused features at the corresponding key points; N is a preset positive integer.

[0035] Calculate the similarity between the corresponding columns of each local feature and the corresponding fusion feature, and select the column with the highest similarity, index c.

[0036] The c-th column of the fusion feature is added to each column of the local feature to generate the posture guidance feature at the corresponding key point. The posture guidance features at each key point of the same player constitute the fusion feature of the player's appearance feature and posture feature.

[0037] Furthermore, in step S4, the construction of the location distance similarity matrix includes:

[0038] The true position of each player in the two long shots is obtained respectively. For each long shot, if the player appears in the critical frame of the long shot, the true position of the player is obtained by using the homography matrix. Otherwise, the true position of the player is predicted by using Kalman filtering based on state estimation.

[0039] Calculate the Euclidean distance between the players' actual positions in the two long shots to obtain the positional distance similarity matrix.

[0040] The construction of the appearance similarity matrix includes:

[0041] Convolutional neural networks were used to extract the appearance features of each player in two long shots.

[0042] Calculate the similarity between the appearance features of the players in the two long shots to obtain the appearance similarity matrix;

[0043] The construction of the pose distance similarity matrix includes:

[0044] By using the heatmaps of each key point to estimate the position coordinates of the corresponding key points, the human skeleton models of each player in the two long shots are obtained.

[0045] The similarity between the player's posture features in the two long shots is calculated to obtain the posture similarity matrix. The similarity between the player's human skeleton model in the two long shots is also calculated to obtain the skeleton model similarity matrix.

[0046] The pose similarity matrix and the skeleton similarity matrix are taken as reciprocals and then added together to obtain the pose distance matrix;

[0047] The critical frame is the frame where the number of players changes abruptly.

[0048] Furthermore, in step S1, before performing target detection on the first and second distant shots in the video clip respectively, the following steps are also included: For each distant shot, the following operations are performed:

[0049] After converting each frame in the long shot to the HSV color space, the green and yellow areas in each frame are extracted as the court area and the non-court area, respectively.

[0050] Eliminate gaps between the court area and non-court area using image manipulation;

[0051] The Hough transform is used to fit the edge line at the boundary between the court area and the non-court area, and the fitted edge line is extended by a preset distance from the court area to the non-court area to obtain the final court area and non-court area.

[0052] Remove non-court areas from each frame and retain only the court area.

[0053] According to another aspect of the present invention, a computer program product is provided, comprising: a computer program; when the computer program is executed by a processor, it implements the above-described method for re-identifying players across shots in football videos provided by the present invention.

[0054] According to another aspect of the present invention, a computer-readable storage medium is provided, including a stored computer program; when the computer program is executed by a processor, it controls the device where the computer-readable storage medium is located to execute the above-described method for re-identifying players across shots in football videos provided by the present invention.

[0055] According to another aspect of the present invention, an electronic device is provided, comprising:

[0056] A computer-readable storage medium for storing computer programs;

[0057] And a processor, used to read computer programs and execute the above-mentioned method for re-identifying players across shots in football videos provided by the present invention.

[0058] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:

[0059] (1) When determining the shot type for each video segment containing long shot-non-long shot-long shot, this invention first clusters the center positions of each player's target frame and removes clusters with fewer than 1 / M of the target number in the largest cluster. This fully considers the player's position information and eliminates interference from boundary players. Then, it obtains the smallest bounding box containing the remaining player target frames and calculates the ratio of the sum of the areas of the remaining player target frames to the area of ​​this smallest bounding box. If this ratio is greater than a preset threshold, the shot type is determined to be a densely positioned shot; otherwise, it is determined to be a non-densely positioned shot. This fully considers the distribution of players on the field and achieves accurate shot type determination. Overall, this invention comprehensively considers player position information and player distribution on the field, eliminates interference from boundary players, accurately determines shot type, ensures that the selected player re-identification strategy is compatible with the actual shot switching type, and effectively improves the accuracy of cross-shot player re-identification in football videos.

[0060] (2) In a formal football match, each team has 11 players on the field, including 1 goalkeeper and 10 other players in other positions. Players in two distant shots are concentrated in a certain area of ​​the field, and the interaction between players increases, resulting in a large number of clustering and occlusion phenomena. After clustering the center position of the player target frame, the largest cluster is the corresponding player gathering area. Due to technical and tactical considerations and formations, there are often some boundary players. These boundary players will form smaller clusters. Through statistical and experimental verification, this invention sets the threshold for removing the interference of boundary players to 1 / 5 of the number of targets in the largest cluster. This can accurately identify the cluster where the boundary players are located and accurately remove the interference of these boundary players, further improving the accuracy of shot type discrimination.

[0061] (3) After removing the interference of boundary players, the larger the ratio of the sum of the areas of the remaining player target boxes to the area of ​​the smallest bounding box containing the remaining player target boxes, the denser the players are positioned. Conversely, the smaller the ratio, the more dispersed the players are positioned. Through statistical and experimental verification, this invention sets the threshold for distinguishing densely positioned shots and non-densely positioned shots to 25%, which can effectively improve the accuracy of shot type discrimination.

[0062] (4) RT-DETR (Real-time Detection Transformer) is a real-time target detection model based on Transformer, which balances speed and accuracy. This invention uses the RT-DETR detection model to achieve target detection, which can effectively improve real-time performance while ensuring detection accuracy.

[0063] (5) After dividing the player image into blocks, the present invention uses a Transformer encoder to obtain the local features of each image block. These local features can effectively reflect the detailed information of the player's appearance. At the same time, when estimating the player's posture features, the player image is scaled at different scales to obtain the posture features of each image, and then weighted and fused as the player's posture features. The posture features extracted in this way can effectively avoid the failure of detailed information in the appearance features due to changes in posture and occlusion when fused with the appearance features. Furthermore, the posture information can be effectively used to guide the generation of distinguishing features that can accurately identify different players, thereby improving the accuracy and stability of player re-identification.

[0064] (6) In non-dense standing shots, the players are relatively scattered. Existing methods mostly rely on the distance and appearance information of the players to achieve player re-identification. In the scenario of scattered standing, different players have different tasks, which makes the posture of different players also change. In the non-dense standing shots, the present invention simultaneously constructs the position distance similarity matrix, appearance similarity matrix and posture distance similarity matrix of the players. It can fully consider the position, appearance and posture information of the players and effectively improve the accuracy and stability of player re-identification in this scenario.

[0065] (7) Before performing target detection on players in long shots, the present invention first identifies the court area and non-court area in each frame, and removes the non-court area in each frame, retaining only the court area. This can avoid interference from non-players and further improve the accuracy of cross-camera player re-identification. Attached Figure Description

[0066] Figure 1 is a schematic diagram of an existing video clip that includes long shot-non-long shot-long shot;

[0067] Figure 2 is a flowchart of the method for re-identifying players across shots in football videos provided in an embodiment of the present invention;

[0068] Figure 3 is a schematic diagram of the extraction of fusion features of appearance features and posture features provided in an embodiment of the present invention;

[0069] Figure 4 is a schematic diagram of player re-identification based on a bipartite graph matching strategy provided in an embodiment of the present invention. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0071] In this invention, the terms "first," "second," etc. (if present) in the invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0072] To accurately determine the type of shot transition and ensure that the selected cross-shot player re-identification strategy after each shot transition is compatible with the actual shot transition type, thereby improving the accuracy and stability of identification, this invention provides a cross-shot player re-identification method in football videos. The overall concept is to deeply analyze the characteristics of player positioning in different scenarios, improve the shot type judgment method, and eliminate possible interference.

[0073] The following is an example.

[0074] Example 1:

[0075] A method for re-identifying players across camera angles in football videos, comprising:

[0076] The football match video to be analyzed is divided into video segments containing long shots, non-long shots, and long shots. For each video segment, the following steps are performed as shown in Figure 2:

[0077] S1: Perform target detection on the first and second long shots in the video clip respectively to obtain the first player's target box set and the second player's target box set;

[0078] Considering the high requirements for real-time computation in scenarios such as player performance evaluation, as a preferred implementation method, this embodiment uses a pre-trained RT-DETR detection model for target detection, achieving high real-time performance while ensuring detection accuracy. It should be noted that the RT-DETR detection model is only a preferred implementation method of this invention and should not be construed as the only limitation of this invention. Other target detection models such as YOLOv7 can also be applied in other embodiments of this invention.

[0079] S2: Determine the shot type of the current video clip. If it is a shot with dense standing positions, proceed to step S3 to perform posture-guided cross-shot player re-identification; otherwise, proceed to step S4 to perform position-guided player re-identification.

[0080] To accurately determine the shot type of the current video segment and provide a reliable basis for subsequent cross-shot player re-identification strategies, ensuring recognition accuracy, this embodiment fully considers player positioning under various match conditions and potential interference factors on the field when determining the shot type. Specifically, in this embodiment, determining the shot type of the current video segment includes:

[0081] S21: Perform KNN clustering on the center positions of each player's target box in the second player target box set. Take 1 / M of the number of targets in the largest cluster as the threshold Th1, and remove the clusters with fewer than the threshold Th1 to eliminate the interference of boundary players. M is a preset positive number. Taking into account the actual number of players on the field in a football match and the distribution of players in densely positioned scenarios, in this embodiment, M is specifically set to 5. That is to say, the clusters with fewer than 1 / 5 of the number of targets in the largest cluster will be regarded as interference factors and removed.

[0082] S22: Obtain the smallest bounding box that can contain the remaining player target boxes. That is, within the remaining player target boxes, calculate the maximum and minimum values ​​of the four boundaries (top, bottom, left, and right). The bounding box determined by the boundaries corresponding to these four maximum and minimum values ​​is the smallest bounding box that can contain the remaining player target boxes. Record the area of ​​this smallest bounding box as S1, and calculate the sum of the areas of the remaining player target boxes as S2.

[0083] S23: If S2 / S1>Th2, then the shot type is determined to be a densely positioned shot; otherwise, the shot type is determined to be a non-densely positioned shot. Th2 is a preset threshold. Optionally, in this embodiment, Th2 = 25%. The larger the ratio of the sum of the areas of the remaining player target boxes to the area of ​​the smallest bounding box containing the remaining player target boxes, the denser the player positions are. Conversely, the smaller the ratio, the more dispersed the player positions are. In this embodiment, Th2 = 25% is set, which can more accurately distinguish between densely positioned shots and dispersed shots.

[0084] S3: Obtain the fusion features of the appearance and posture features of each player in the two long shots respectively, and calculate the similarity between the players in the two long shots based on the fusion features to obtain the correlation matrix between the two long shots, and then proceed to step S5.

[0085] S4: Obtain the positional distance similarity matrix, appearance similarity matrix, and posture distance similarity matrix between each player in the two long shots, and fuse them as the correlation matrix between the two long shots;

[0086] S5: Based on the correlation matrix between the two long shots and the bipartite graph matching strategy, perform player matching between the two long shots to complete the cross-shot player re-identification of the current video segment.

[0087] This embodiment can comprehensively consider the player's position information and the distribution of players on the field, and eliminate the interference of boundary players, so as to accurately determine the shot type. This provides an accurate and reliable basis for subsequent selection of posture-guided cross-camera player re-identification or position-guided player re-identification, ensuring that the selected player re-identification strategy is compatible with the actual shot switching type, and effectively improving the accuracy of cross-camera player re-identification in football videos.

[0088] To further eliminate interference, as a preferred implementation, in step S1 of this embodiment, before performing target detection on the first and second distant shots in the video clip, the characteristics of the different colors of the court area and the non-court area, and the large area of ​​court color, are used to separate the court area and the non-court area; specifically, in step S1 of this embodiment, before performing target detection on the first and second distant shots in the video clip, the following steps are also performed for each distant shot:

[0089] After converting each frame in the long shot to the HSV color space, the green and yellow areas in each frame are extracted as the court area and the non-court area, respectively.

[0090] Eliminate gaps between the court area and non-court area using image manipulation;

[0091] The Hough transform is used to fit the edge line at the boundary between the court area and the non-court area, and the fitted edge line is extended by a preset distance from the court area to the non-court area to obtain the final court area and non-court area; in the final court area, the players at the boundary can be completely preserved after removing interference factors such as the stands.

[0092] Remove non-court areas from each frame and retain only the court area.

[0093] In close-up shots where players are clustered in a specific area of ​​the field, increased player interaction and the occurrence of crowding and obstruction can lead to player identification. In such situations, player posture information can be effectively used for player differentiation. Therefore, in close-up shots, posture-guided cross-shot player re-identification is employed. Specifically, as shown in Figure 3, in step S3 of this embodiment, for each close-up shot, the fusion features of the player's appearance and posture are obtained, including: appearance feature extraction, posture feature extraction, and appearance-posture feature fusion.

[0094] Appearance feature extraction includes:

[0095] A sliding window is used to divide the image corresponding to the player's target box into N image blocks with overlapping pixels, and a learnable linear projection function is used to map each image block to D dimensions to obtain the image block embedding.

[0096] The positions of the player's target box and each image patch are represented by learnable encoded vectors, which serve as player position embeddings and image patch position embeddings, and together with the image patch embeddings, constitute the input sequence.

[0097] The input sequence is fed into the Transformer encoder to obtain the local features of each image patch, which are used as the appearance features of the corresponding player.

[0098] Pose feature extraction includes:

[0099] The image corresponding to the player's target box is used as the original image, and then scaled at different scales.

[0100] For each image, a heatmap of N key points in the image is estimated using a human pose estimation model, which serves as the pose feature of the corresponding image; N is a preset positive integer; optionally, the human pose estimation model used in this embodiment is ViTPose;

[0101] The pose features of each image are weighted and summed to obtain the pose features of the corresponding player.

[0102] Appearance feature-attitude feature fusion includes:

[0103] For the same player's appearance and posture features, a correspondence is established between N local features and N heatmaps. After unifying the corresponding local features and heatmaps to the same dimension, a weighted sum is performed to obtain the fused features at the corresponding key points. The weight information for the weighted sum can be determined by the statistical characteristics of the appearance features and heatmaps, and the weighted sum process can be implemented through a linear layer. By fusing local features and heatmaps through weighted summation, the information of appearance features at high-probability areas in the heatmap can be enhanced.

[0104] Calculate the similarity between the corresponding columns of each local feature and the corresponding fusion feature, and select the column with the highest similarity, c, to locate the part of the fusion feature that is most similar to the local feature;

[0105] The c-th column of the fusion feature is added to each column of the local feature to generate the posture guidance feature at the corresponding key point. The posture guidance features at each key point of the same player constitute the fusion feature of the player's appearance feature and posture feature.

[0106] In this embodiment, after dividing the player image into blocks, a Transformer encoder is used to obtain the local features of each image block. These local features can effectively reflect the detailed information of the player's appearance. At the same time, when estimating the player's posture features, the player image is scaled at different scales to obtain the posture features of each image, and then weighted and fused as the player's posture features. When the posture features extracted in this way are fused with the appearance features, they can effectively avoid the loss of detailed information in the appearance features due to changes in posture and occlusion, and effectively use the posture information to guide the generation of distinguishing features that can accurately identify different players, thereby improving the accuracy and stability of player re-identification.

[0107] Unlike densely packed shots, in the two distant shots of players in dispersed positions, the players are more widely distributed on the field. In this case, position-guided cross-camera player re-identification is used, taking into account the players' physical features. Considering that different players have different tasks in dispersed positions, their postures will also vary. To ensure re-identification accuracy in non-densely packed positions, this embodiment considers not only the players' position and appearance information but also their posture information when performing position-guided cross-camera player re-identification. Specifically, in step S4 of this embodiment, the construction of the positional distance similarity matrix includes:

[0108] The true position of each player in the two long shots is obtained respectively. For each long shot, if the player appears in the critical frame of the long shot, the true position of the player is obtained by using the homography matrix. Otherwise, the true position of the player is predicted by using Kalman filtering based on state estimation.

[0109] Calculate the Euclidean distance between the real positions of the players in the two long shots, that is, calculate the Euclidean distance between the real positions of each player in the first long shot and the real positions of each player in the second long shot, and obtain the position distance similarity matrix.

[0110] It is easy to understand that in each shot, the critical frame is the frame where the number of players changes abruptly. Specifically, the number of player targets in each frame can be detected by the target detection model, and the difference in the number of targets between the current frame and the previous frame can be determined. When the difference in the number of targets exceeds a certain set threshold, it can be determined that the number of players in the current frame has changed abruptly relative to the previous frame, thus identifying the current frame as the critical frame.

[0111] In step S4 of this embodiment, the construction of the appearance similarity matrix includes:

[0112] Convolutional neural networks were used to extract the appearance features of each player in two long shots.

[0113] The similarity between the appearance features of the players in the two long shots is calculated separately, that is, the similarity between the appearance features of each player in the first long shot and the appearance features of each player in the second long shot is calculated to obtain the appearance similarity matrix; optionally, in this embodiment, cosine similarity is specifically used when calculating the similarity between appearance features.

[0114] In step S4 of this embodiment, the construction of the pose distance similarity matrix includes:

[0115] The position coordinates of the corresponding key points are estimated by using the heat map of each key point, and the human skeleton model of each player in the two long shots is obtained; the heat map of each key point can directly use the estimation results in step S3.

[0116] The similarity between the posture features of the players in the two long shots is calculated to obtain a posture similarity matrix. The similarity between the human skeleton models of the players in the two long shots is also calculated to obtain a skeleton model similarity matrix. Optionally, in this embodiment, the similarity between posture features and the similarity between human skeleton models are both calculated using cosine similarity.

[0117] The pose similarity matrix and the skeleton similarity matrix are taken as reciprocals and then added together to obtain the pose distance matrix.

[0118] After obtaining the correlation matrix between two long shots, a bipartite graph matching strategy is used to match players between the two long shots. The process of cross-shot player re-identification for the current video segment specifically includes:

[0119] K-means clustering was performed on the RGB mean values ​​of non-background colors to divide players into teams, as shown in Figure 4. The number of cluster centers was set to 3.

[0120] Players are treated as nodes, and the values ​​of the association matrix are treated as edge weights between nodes. A bipartite graph is constructed, and the matching of player re-identification across shots is to find the minimum weight allocation of this bipartite graph.

[0121] By adding a source and sink node to the bipartite graph, transforming it into a cost flow model, and solving for the minimum cost maximum flow, player matching can be achieved.

[0122] Overall, this embodiment can comprehensively consider the player's position information and the distribution of players on the field, and eliminate the interference of players on the perimeter, so as to accurately determine the shot type. On this basis, a posture-guided cross-shot player re-identification method is selected for densely positioned shots, and a position-guided cross-shot player re-identification method is selected for non-densely positioned shots, so as to adaptively select the re-identification method that matches the current shot type, and ultimately effectively improve the accuracy and stability of cross-shot player re-identification in football videos.

[0123] Example 2:

[0124] A computer program product includes: a computer program; when the computer program is executed by a processor, it implements the cross-camera player re-identification method in football videos provided in Embodiment 1 above.

[0125] Example 3:

[0126] A computer-readable storage medium includes a stored computer program; when the computer program is executed by a processor, it controls the device where the computer-readable storage medium is located to execute the cross-camera player re-identification method in football video provided in Embodiment 1 above.

[0127] Example 4:

[0128] An electronic device, comprising:

[0129] A computer-readable storage medium for storing computer programs;

[0130] And a processor, used to read computer programs and execute the cross-camera player re-identification method in football videos provided in Embodiment 1 above.

[0131] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for re-identifying players across different camera angles in football videos, characterized in that, include: The football match video to be analyzed is divided into video segments containing long shots-non-long shots-long shots. For each video segment, the following steps are performed: The video segment containing long shots-non-long shots-long shots is a video frame sequence from long shot to non-long shot to long shot; S1: Target detection is performed on the first long shot and the second long shot in the video segment to obtain the first player target box set and the second player target box set. S2: Determine the shot type of the current video clip. If it is a shot with dense standing positions, proceed to step S3. Otherwise, proceed to step S4; wherein, determining the shot type of the current video segment includes: S21: clustering the center positions of each player's target box in the second player target box set, taking 1 / M of the number of targets in the largest cluster as a threshold Th1, and removing clusters with fewer targets than the threshold Th1; M is a preset positive number; S22: obtaining the smallest bounding box that can contain the remaining player target boxes, recording its area as S1, and calculating the sum of the areas of the remaining player target boxes S2; S23: if S2 / S1>Th2, then the shot type is determined to be a densely positioned shot; otherwise, the shot type is determined to be non-dense. S3: Obtain the fusion features of the appearance and posture features of each player in the two long shots, and calculate the similarity between the players in the two long shots based on the fusion features to obtain the correlation matrix between the two long shots, and then proceed to step S5; S4: Obtain the position distance similarity matrix, appearance similarity matrix and posture distance similarity matrix between each player in the two long shots, and fuse them as the correlation matrix between the two long shots; S5: Perform player matching between the two long shots based on the correlation matrix between the two long shots and the bipartite graph matching strategy to complete the cross-shot player re-identification of the current video segment.

2. The method for re-identifying players across shots in football videos as described in claim 1, characterized in that, M=5。 3. The method for re-identifying players across shots in football videos as described in claim 2, characterized in that, Th2=25%.

4. The method for re-identifying players across shots in football videos as described in any one of claims 1-3, characterized in that, The target detection is achieved using the RT-DETR detection model.

5. The method for re-identifying players across shots in football videos as described in any one of claims 1-3, characterized in that, In step S3, for each long shot, the fusion features of the appearance and posture features of each player are obtained, including: appearance feature extraction, posture feature extraction, and appearance-posture feature fusion. The appearance feature extraction includes: dividing the image corresponding to the player's target box into N image patches with overlapping pixels using a sliding window, and mapping each image patch to D dimensions using a learnable linear projection function to obtain image patch embeddings; representing the position of the player's target box and the position of each image patch using learnable encoding vectors, as player position embeddings and image patch position embeddings, together with the image patch embeddings, to form an input sequence; inputting the input sequence into a Transformer encoder to obtain the local features of each image patch, as the appearance features of the corresponding player; the posture feature extraction includes: scaling the image corresponding to the player's target box as the original image at different scales; and... For each image, a heatmap of N key points in the image is estimated using a human pose estimation model, which serves as the pose feature corresponding to the image. The pose features corresponding to each image are weighted and summed to obtain the pose features of the corresponding player. The appearance feature-pose feature fusion includes: for the appearance features and pose features of the same player, establishing a correspondence between N local features and N heatmaps, unifying the corresponding local features and heatmaps to the same dimension, and then performing a weighted summation to obtain the fused features at the corresponding key points; N is a preset positive integer; calculating the similarity between the corresponding columns of each local feature and the corresponding fused features, and selecting the column number c with the highest similarity; adding the c-th column of the fused features to each column of the local features to generate the pose guidance features at the corresponding key points, and the pose guidance features at each key point of the same player constitute the fused features of the appearance features and pose features of the corresponding player.

6. The method for re-identifying players across shots in football videos as described in claim 5, characterized in that, In step S4, the construction of the positional distance similarity matrix includes: obtaining the true position of each player in two long shots; for each long shot, if a player appears in the critical frame of that long shot, the player's true position is obtained using the homography matrix; otherwise, the player's true position is predicted using a Kalman filter based on state estimation; calculating the Euclidean distance between the true positions of the players in the two long shots to obtain the positional distance similarity matrix; the construction of the appearance similarity matrix includes: extracting the appearance features of each player in the two long shots using a convolutional neural network; calculating the similarity between the appearance features of the players in the two long shots to obtain the appearance similarity matrix; the construction of the posture distance similarity matrix includes: estimating the position coordinates of the corresponding key points using the heatmap of each key point to obtain the human skeleton model of each player in the two long shots; calculating the similarity between the posture features of the players in the two long shots to obtain the posture similarity matrix, and calculating the similarity between the human skeleton models of the players in the two long shots to obtain the skeleton model similarity matrix; taking the reciprocal of the posture similarity matrix and the skeleton similarity matrix and adding them together to obtain the posture distance matrix; wherein, the critical frame is the frame where the number of players changes abruptly.

7. The method for re-identifying players across shots in football videos as described in any one of claims 1-3, characterized in that, Before performing target detection on the first and second long shots in the video clip in step S1, the method further includes the following steps for each long shot: after converting each frame in the long shot to the HSV color space, extracting the green and yellow regions in each frame as the court area and non-court area respectively; eliminating gaps in the court area and non-court area through image processing operations; fitting the edge line at the boundary between the court area and non-court area using Hough transform, and expanding the fitted edge line from the court area to the non-court area by a preset distance to obtain the final court area and non-court area; removing the non-court area in each frame and retaining only the court area.

8. A computer program product, characterized in that, include: A computer program; when executed by a processor, the computer program implements the cross-camera player re-identification method in football videos as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer program includes a stored computer program; when executed by a processor, the computer program controls the device containing the computer-readable storage medium to perform the cross-camera player re-identification method in football videos as described in any one of claims 1-7.

10. An electronic device, characterized in that, include: A computer-readable storage medium for storing computer programs; And a processor, configured to read the computer program and execute the cross-camera player re-identification method in football videos as described in any one of claims 1-7.

Images