Multi-target tracking method for football sports scene, medium and product

By introducing the efficient channel attention mechanism ECA and polynomial cross-entropy loss function based on FairMOT, combined with the Kalman filter and data association module, the multi-objective tracking algorithm in football sports scenarios is improved, and the problems of rapid change and complex occlusion are solved. , achieving efficient and accurate multi-objective tracking effect.

CN119006516AActive Publication Date: 2024-11-22HOHAI UNIV

Patent Information

Application Number
CN202410917296.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-10
Publication Date
2024-11-22
Estimated Expiration
2044-07-10

AI Technical Summary

Technical Problem

Existing multi-objective tracking algorithms are difficult to cope with rapidly changing scenarios, complex occlusions and multi-player interactions in football scenarios, resulting in insufficient tracking accuracy and robustness.

Method used

The improved multi-objective tracking method is based on FairMOT, introducing an efficient channel attention mechanism, ECA enhances the depth layer aggregation segmentation network DLAseg, develops a new joint detection head, and uses a polynomial cross-entropy PolyCE loss function in the re-identification module, combined with a Kalman filter Related to the data module to improve detection accuracy and tracking accuracy.

Benefits of technology

Multi-objective standard, efficient and robust tracking in football sports scenarios is achieved in complex environments, significantly improving long-distance detection accuracy and detection performance in complex occlusion situations, ensuring accurate detection and positioning in dense scenes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006516A_ABST
    Figure CN119006516A_ABST
Patent Text Reader

Abstract

The invention provides a multi-target tracking method for a football scene, a medium and a product, and the method comprises the steps: obtaining a video frame, inputting the video frame into an ECADLAseg backbone network, and obtaining a multi-target tracking result; the ECADLAseg backbone network comprises a joint detection head for outputting a feature map detection result containing a thermodynamic diagram, a center offset, a bounding box size and an intersection-to-union ratio, and an ECADLAseg backbone network for outputting a detection result of the feature map containing the thermodynamic diagram, the center offset, the bounding box size and the intersection-to-union ratio according to the feature map detection result containing the thermodynamic diagram, the center offset and the bounding box size. The re-identification module ReID is used for generating a unique identity label of each detection target; the Kalman filter is used for predicting a track of each detection target and obtaining a predicted track; the Kalman filter is used for matching the track predicted by the Kalman filter with the unique identity label of each detection target through data association; and the data association module is used for obtaining a final multi-target tracking result. According to the invention, accurate and efficient tracking of multiple targets in a football scene is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a multi-target tracking method, medium and product for football sports scenes, and belongs to the technical field of computer vision and deep learning. Background Art

[0002] With the rapid development of computer vision and deep learning technologies, multi-target tracking algorithms have been widely used in various application scenarios, such as intelligent monitoring, autonomous driving, motion analysis, etc. However, multi-target tracking still faces many challenges in complex scenarios, especially in dynamic environments such as football games, involving fast movement of multiple targets, complex occlusions, and frequent interactions. Traditional multi-target tracking algorithms mainly rely on the detection-association framework, that is, first detecting the targets in each frame, and then connecting these detection results to form a trajectory through data association. In recent years, end-to-end deep learning methods, such as the real-time multi-target tracking algorithm FairMOT, have gradually become mainstream. The real-time multi-target tracking algorithm FairMOT combines target detection and re-identification tasks in a unified framework to achieve higher tracking accuracy and efficiency.

[0003] However, existing multi-target tracking algorithms still have some problems when applied to football scenes. First, the scene changes rapidly in football games, and the trajectory of the target is complex and changeable. Traditional methods are difficult to cope with such complex dynamic changes. Second, occlusion between players often occurs, especially in dense game areas. Existing methods often perform poorly when dealing with occlusion problems, resulting in tracking failures. Finally, the frequent interactions between players in football games increase the difficulty of tracking, and existing methods find it difficult to accurately distinguish and track each player.

[0004] In order to solve the above problems, the present invention proposes an improved multi-target tracking method, which is specifically used for multi-target tracking in football scenes. This method has made many improvements based on the real-time multi-target tracking algorithm FairMOT, including introducing an efficient channel attention mechanism ECA to enhance the deep layer aggregation segmentation network DLAseg, developing a new joint detection head, and introducing a polynomial cross entropy PolyCE loss function in the re-identification module. The improvements of this method are aimed at improving the detection accuracy of small targets, optimizing the detection effect in dense scenes, and improving the accuracy and robustness of tracking. By training and evaluating on the SoccerNet Tracking dataset, the superior performance of the method proposed in the present invention in fast-changing, complex occlusion and multi-player interaction scenes is verified, demonstrating its potential and effectiveness in actual football game applications.

[0005] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the invention and should not be regarded as an acknowledgement or any form of suggestion that the information constitutes the prior art already known to ordinary technicians in this field. Summary of the invention

[0006] The technical problem to be solved by the present invention is: how to combine target detection and re-identification tasks, solve the multi-target occlusion problem, and track and accurately distinguish the trajectory of each player in real time.

[0007] In order to solve the above technical problems, the present invention is implemented by adopting the following technical solutions.

[0008] In a first aspect, the present invention provides a multi-target tracking method for a football sports scene, comprising: Acquire video frame images of football sports scenes; Input the video frame images of the football scene into the ECA_DLAseg backbone network to obtain the multi-target tracking results; The ECA_DLAseg backbone network includes a joint detection head, a re-identification module ReID, a Kalman filter and a data association module; The joint detection head is used to output the feature map containing the detection target as a feature map detection result including a heat map, a center offset, a bounding box size and an intersection-over-union ratio; The re-identification module ReID uses a polynomial cross entropy PolyCE loss function as the re-identification loss to balance the ratio of positive and negative samples; and is used to generate a unique identity for each detected target based on the feature map detection results including the heat map, center offset, bounding box size, and intersection-over-union ratio; The Kalman filter is used to predict the trajectory of each detected target to obtain a predicted trajectory; The data association module is used to match the trajectory predicted by the Kalman filter with the unique identity of each detected target through data association to obtain the final multi-target tracking result; The training method of the ECA_DLAseg backbone network includes: According to the training set and the total loss function, the ECA_DLAseg backbone network is trained to obtain a trained ECA_DLAseg backbone network.

[0009] Furthermore, the method for constructing the ECA_DLAseg backbone network includes: The upsampling ratio of the input feature map by the deep layer aggregation segmentation network DLAseg is increased from 1 / 4 of the original size of the feature map to 1 / 2 of the original size of the feature map; The efficient channel attention mechanism ECA is embedded in each block of the deep layer aggregation segmentation network DLAseg to obtain the ECA_DLAseg backbone network.

[0010] Furthermore, the method for constructing the joint detection head includes: The generalized intersection-over-union (GIoU) ​​loss function is introduced into the intersection-over-union (IoU) calculation module to obtain an improved IoU calculation module. An improved IoU calculation module is added to the detection head to obtain a joint detection head.

[0011] Furthermore, a method for generating a unique identification of each detected target according to a feature map detection result including a heat map, a center offset, a bounding box size, and an intersection-over-union ratio includes: Based on the detection results of the feature map including the heat map, center offset, bounding box size and intersection-over-union ratio, the bounding box of each detected target is decoded, the bounding box is filtered using non-maximum suppression (NMS), and the appearance feature map of each detected target is extracted from the filtered bounding box; The appearance feature map of each detected target is processed through two layers of convolution and ReLU activation function, and the number of channels of the appearance feature map of each detected target is reduced to 128 to obtain the processed appearance feature map; Through the classifier, the features embedded in the processed appearance feature map are converted into scores corresponding to each detected target as the unique identity of each detected target.

[0012] Furthermore, the total loss obtained according to the total loss function of the ECA_DLAseg backbone network includes the detection loss and re-identification loss of the joint detection head. The total loss of the ECA_DLAseg backbone network is expressed as: ; Where L represents the total loss of the ECA_DLAseg backbone network, represents the weight adjustment term of the detection loss of the joint detection head, represents the weight adjustment term for the re-identification loss, represents the re-identification loss and detection loss, represents the re-identification loss, represents the learnable weight of the detection, Represents the learnable weights for the re-ID loss.

[0013] Furthermore, the detection loss of the joint detection head includes heat map loss, size loss, offset loss and intersection-over-union (IoU) loss; the heat map loss, size loss, offset loss and IoU loss are weighted and summed to obtain the detection loss.

[0014] Furthermore, a Kalman filter is used to predict the trajectory of each detection target to obtain a predicted trajectory, wherein the predicted trajectory includes a confirmed detection target trajectory and an unconfirmed detection target trajectory.

[0015] Furthermore, the step of matching the trajectory predicted by the Kalman filter with the unique identity of each detected target through data association to obtain the final multi-target tracking result includes: Cascade matching is performed on the confirmed detection target trajectory and the unique identity of each detection target to obtain a first unmatched detection result, a first unmatched trajectory, and a first matched trajectory; Perform Hungarian matching on the first mismatch detection result and the first mismatch trajectory and the unconfirmed detection target trajectory to obtain a second mismatch detection result, a second mismatch trajectory and a second matching trajectory; Inputting the second matching trajectory into a Kalman filter to update the parameters of the first matching trajectory, thereby obtaining an updated first matching trajectory; Input the second mismatch detection result into the Kalman filter and initialize it as a new trajectory; If the second unmatched trajectory and the new trajectory can match the confirmed detection target trajectory in three consecutive frames, they are designated as confirmed trajectories and sent to the Kalman filter to update the parameters of the first matching trajectory to obtain the updated first matching trajectory; If the second unmatched trajectory and the new trajectory cannot match the confirmed detection target trajectory in several consecutive frames, and the number of consecutive unmatched frames between the second unmatched trajectory and the new trajectory is less than the set threshold, the second unmatched trajectory and the new trajectory are retained, and the parameters of the first matching trajectory are updated by the Kalman filter to obtain the updated first matching trajectory; If the second unmatched trajectory and the new trajectory cannot match the determined trajectory of the feature map detection result in several consecutive frames, and the number of consecutive unmatched frames of the new trajectory is greater than or equal to the set threshold, delete the new trajectory; Output all updated first matching trajectories as multi-target tracking results.

[0016] In a second aspect, the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon, wherein the computer program / instruction implements the steps of the above method when executed by a processor.

[0017] In a third aspect, the present invention further provides a computer program product, comprising a computer program / instruction, characterized in that the computer program / instruction implements the steps of the above method when executed by a processor.

[0018] Compared with the prior art, the present invention has the following beneficial effects: The invention provides a multi-target tracking method for football sports scenes, which realizes accurate, efficient and robust tracking of multiple targets in football sports scenes. This technology can not only effectively extract and identify multiple targets in video frames, but also predict the movement trajectory of the targets, and assign a unique identity to each target through a re-identification module, ensuring stable tracking effects even in complex environments, and providing strong technical support for real-time analysis and decision-making in football sports scenes.

[0019] The ECA_DLAseg backbone network provided by the present invention enhances the deep layer aggregation segmentation network DLaseg by introducing an efficient channel attention mechanism ECA, thereby significantly improving the detection accuracy of the ECA_DLAseg backbone network for long-distance and detection targets in football sports scenes, so that detail information can be better retained and utilized; secondly, the joint detection head of the ECA_DLAseg backbone network integrates an intersection-over-union (IoU) calculation module, and introduces a generalized intersection-over-union (GIoU) ​​loss function in the intersection-over-union (IoU) calculation module, which effectively improves the detection performance under complex occlusion and multi-player interaction conditions, and ensures accurate detection and positioning in dense scenes; furthermore, the re-identification module ReID of the ECA_DLAseg backbone network introduces a polynomial cross entropy PolyCE loss function to balance the ratio of positive and negative samples, reduce the overfitting of the ECA_DLAseg backbone network for frequently appearing targets, and enhance the ability of the ECA_DLAseg backbone network to distinguish targets with similar appearances, thereby improving the accuracy and robustness of tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Schematic diagram of the overall network architecture of a multi-target tracking method for a football sports scene provided by an embodiment of the present invention; Figure 2 is a schematic diagram of the network architecture of a deep layer aggregation and segmentation network DLAseg provided in an embodiment of the present invention; Figure 3 It is a schematic diagram of the network architecture of the ECA_DLAseg backbone network provided by an embodiment of the present invention; Figure 4 Schematic diagram of the workflow of the efficient channel attention mechanism ECA provided by an embodiment of the present invention; Figure 5 It is a schematic diagram of the ECA_DLAseg backbone network structure provided by an embodiment of the present invention; Figure 6 is a schematic diagram of the structure of a joint detection head provided by an embodiment of the present invention; Figure 7 Schematic diagram of the workflow of the re-identification module ReID provided in an embodiment of the present invention; Figure 8The present invention is a flowchart of a multi-target tracking method for football sports scenes for data association provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. The embodiments of the present invention and the technical features in the embodiments may be combined with each other unless there is a conflict.

[0022] Example 1 This embodiment introduces a multi-target tracking method for a football sports scene, including: Acquire video frame images of football sports scenes; Input the video frame images of the football scene into the ECA_DLAseg backbone network to obtain the multi-target tracking results; The overall network architecture of a multi-target tracking method for football scenes is as follows: Figure 1 shown.

[0023] The ECA_DLAseg backbone network includes a joint detection head, a re-identification module ReID, a Kalman filter and a data association module; The joint detection head is used to output the feature map containing the detection target as a feature map detection result including a heat map, a center offset, a bounding box size and an intersection-over-union ratio; The re-identification module ReID uses a polynomial cross entropy PolyCE loss function as the re-identification loss to balance the ratio of positive and negative samples; and is used to generate a unique identity for each detected target based on the feature map detection results including the heat map, center offset, bounding box size, and intersection-over-union ratio; The Kalman filter is used to predict the trajectory of each detected target to obtain a predicted trajectory; The data association module is used to match the trajectory predicted by the Kalman filter with the unique identity of each detected target through data association to obtain the final multi-target tracking result.

[0024] The present invention introduces an efficient channel attention mechanism ECA to enhance the deep layer aggregation segmentation network DLAseg to obtain the ECA_DLAseg backbone network, thereby significantly improving the detection accuracy of the ECA_DLAseg backbone network for long-distance and detection targets in football scenes, so that detail information can be better retained and utilized. The network architecture of the enhanced deep layer aggregation segmentation network DLAseg is as follows: Figure 2 As shown, the network structure of the ECA_DLAseg backbone network is as follows Figure 3As shown, the workflow diagram of the efficient channel attention mechanism ECA is as follows Figure 4 shown.

[0025] The method for constructing the ECA_DLAseg backbone network comprises: The upsampling ratio of the input feature map by the deep layer aggregation segmentation network DLAseg is increased from 1 / 4 of the original size of the feature map to 1 / 2 of the original size of the feature map; The efficient channel attention mechanism ECA is embedded in each block of the deep layer aggregation segmentation network DLAseg to obtain the ECA_DLAseg backbone network.

[0026] Efficient Channel Attention Mechanism (ECA) is a lightweight channel attention method that enhances the adaptability of convolutional neural networks to features through weighted enhancement, thereby improving the performance of the network on specific tasks.

[0027] For example, the input feature map of the ECA_DLAseg backbone network is ,in, Indicates the number of channels of the feature map, and They represent the height and width of the feature map respectively, and R represents the real number domain.

[0028] First, global average pooling GAP is performed on each channel of the input feature map to obtain a channel descriptor , where C represents the dimension of the channel descriptor, that is, the number of channels, and its cth element is calculated as follows: (1); In the formula, z c represents the average value of the cth channel, c represents a channel of the feature map, and Represent the height and width of the input feature map respectively, i Represents the height dimension index of the feature map, j Represents the width dimension index of the feature map, X cij Represents the pixel value of the feature map X at channel c, height i, and width j.

[0029] After capturing the global information of each channel of the input feature map, the efficient channel attention mechanism ECA uses one-dimensional convolution to process the channel descriptor. According to the number of channels of the input feature map, an adaptive mechanism is used to calculate the size of the convolution kernel to obtain the input feature map. Figure 1 The output after the dimensional convolution enables the efficient channel attention mechanism ECA to dynamically adjust its internal parameters according to different input features.

[0030] The input features Figure 1The output after dimensional convolution is expressed as: (2); In the formula, Represents input features Figure 1 The output after dimensional convolution, represents a one-dimensional convolution operation, and z represents a channel descriptor.

[0031] According to the input characteristics Figure 1 The output after the dimensional convolution calculates the channel weight of each channel of the input feature map, and recalibrates the input feature map by the channel weight of each channel of the input feature map to obtain the output of each channel of the recalibrated input feature map. The Sigmoid function is used to calculate the channel weight to ensure that the value of the channel weight is between 0 and 1.

[0032] The channel weight of each channel of the input feature map is expressed as: (3); In the formula, Represents the channel weight of each channel of the input feature map, Represents input features Figure 1 The output after dimensional convolution, Represents the Sigmoid function, which is used to normalize the output value to between 0 and 1.

[0033] The output of each channel of the recalibrated input feature map is expressed as: (4); In the formula, y c represents the c-th channel output of the recalibrated input feature map, w c represents the channel weight of the cth channel of the input feature map, x c Represents the c-th channel of the input feature map.

[0034] The ECA_DLAseg backbone network constructed above more effectively retains the input detail information by adjusting the upsampling strategy. In addition, the introduced efficient channel attention mechanism ECA significantly enhances the network's sensitivity to global and local details of the image, especially in terms of color and morphology. This mechanism enables the ECA_DLAseg backbone network to focus on processing the feature information of small targets on higher-resolution feature maps, thereby significantly improving the detection accuracy of athletes with small target ranges. These improvements ensure that the ECA_DLAseg backbone network is efficient and accurate in capturing critical sports details.

[0035] The schematic diagram of the ECA_DLAseg backbone network structure is as follows: Figure 5 shown.

[0036] The training method of the ECA_DLAseg backbone network includes: According to the training set and the total loss function, the ECA_DLAseg backbone network is trained to obtain a trained ECA_DLAseg backbone network.

[0037] The training process of the ECA_DLAseg backbone network includes the following steps:

[0038] Step 1: Use the ECA_DLAseg backbone network to extract all feature maps containing the detected targets from the training set. The network architecture of the ECA_DLAseg backbone network is as follows: Figure 3 shown.

[0039] The SoccerNet Tracking dataset is used as the training set of the present invention. The SoccerNet Tracking dataset is a multi-target tracking dataset designed specifically for football game scenes. It provides a wealth of football sports scenes and challenges, such as fast motion, occlusion, and intensive interactions between players, and is very suitable for training multi-target tracking algorithms for football sports scenes. The SoccerNet Tracking dataset contains key moments from 12 important football games shot by the main camera, totaling 200 video clips. The video data is stored in the form of video frame images, each video clip is 30 seconds long and contains 750 frames of images. Each video provides detailed tracking data, including frame ID, tracking ID, bounding box coordinate X, bounding box coordinate Y, bounding box width W, bounding box height H, and bounding box confidence Score.

[0040] The overall annotation format of the SoccerNet Tracking dataset is consistent with that of the MOT20 dataset.

[0041] Step 2: Input the feature map into the joint detection head to obtain the feature map detection result including the heat map, center offset, bounding box size and intersection-over-union ratio. The structural diagram of the joint detection head is shown in Figure 6 As shown, where n represents the number of categories.

[0042] The present invention introduces a generalized intersection-over-union (GIoU) ​​loss function into an intersection-over-union (IoU) calculation module to obtain an improved IoU calculation module, adds the improved IoU calculation module to an existing detection head, and proposes a joint detection head.

[0043] The joint detection head is responsible for performing the target detection task to identify the detection target in the football sports scene.

[0044] The feature map containing the detection target is input into the joint detection head, and through a series of convolution operations and batch normalization operations, a feature map detection result including a heatmap, a center offset, a bounding box size, and an intersection over union (IoU) is obtained.

[0045] Among them, the heat map is used to represent the probability that each position in the feature map containing the detection target is the center of the detection target; the center offset is used to correct the prediction error of the center point of the detection target caused by discretization; the bounding box size is used to predict the width and height of the detection target; the intersection-over-union calculation module is used to evaluate the degree of overlap between the predicted bounding box of the detection target and the true bounding box of the detection target.

[0046] In the intersection-over-union calculation module, the present invention introduces a generalized intersection-over-union (GIoU) ​​loss function to measure the geometric relationship between bounding boxes. The generalized intersection-over-union (GIoU) ​​loss function is expressed as: (5); Where GIoU represents generalized intersection over union, Represents the traditional intersection-over-union ratio, which is calculated as the intersection area between the predicted bounding box of the detected target and the true bounding box of the detected target divided by their union area, and E represents the area of ​​the minimum closed area containing the predicted bounding box of the detected target and the true bounding box of the detected target. Represents the area of ​​the union of the represented regions.

[0047] The introduction of the generalized intersection-over-union (GIoU) ​​loss function helps to more accurately optimize the position and size of the bounding box in the bounding box regression process, especially when there is occlusion or overlap between the detected targets, the generalized intersection-over-union (GIoU) ​​loss function can provide better performance.

[0048] The detection loss of the detection loss function of the joint detection head includes heat map loss, size loss, offset loss and intersection-over-union loss. The heat map loss is calculated using a loss function based on Focal Loss, and the size loss, offset loss and IoU loss are calculated using an L1 loss function. The detection loss is calculated by weighting the heat map loss, size loss, offset loss and intersection-over-union loss by weight. The weight values ​​of the heat map loss and offset loss are set to 1, and the weight values ​​of the size loss and IoU loss are set to 0.1.

[0049] Step 3: According to the feature map detection results including the heat map, center offset, bounding box size and intersection-over-union ratio, decode the bounding box of each detected target, use non-maximum suppression (NMS) to filter the bounding box, and extract the appearance feature map of each detected target from the filtered bounding box.

[0050] Step 4: Input the appearance feature map of each detected target into the re-identification module ReID, use the polynomial cross entropy PolyCE loss function as the re-identification loss of the re-identification module ReID, balance the ratio of positive and negative samples, and generate a unique identity for each detected target. The workflow diagram of the re-identification module ReID is shown in the figure Figure 7 shown.

[0051] The re-identification module ReID is the component responsible for generating re-identification features. In the re-identification module ReID, each detected target is assigned a unique identity.

[0052] The specific steps include: processing the appearance feature map of each detection target through two layers of convolution and ReLU activation function, reducing the number of channels of the appearance feature map of each detection target to 128, and obtaining the processed appearance feature map.

[0053] Through the classifier, the features embedded in the processed appearance feature map are converted into scores corresponding to each detection target as the unique identity of each detection target. The dimension of the score corresponding to each detection target is (B, nID), where B represents the batch size, that is, the number of samples processed at one time, and nID represents the total number of different individuals that the ECA_DLAseg backbone network can identify or track.

[0054] The present invention introduces a polynomial cross entropy PolyCE loss function as a loss function in the ECA_DLAseg backbone network training process. The overall loss of the ECA_DLAseg backbone network includes the detection loss of the joint detection head and the re-identification loss of the re-identification module ReID.

[0055] By imposing additional penalties on samples that are difficult to distinguish, the ECA_DLAseg backbone network can more effectively learn to distinguish detection targets with similar appearances, so that each detection target identity can be accurately identified in a frequently changing environment, thereby enhancing the ability of the ECA_DLAseg backbone network to cope with occlusion, fast movements, and complex scene changes.

[0056] The polynomial cross entropy PolyCE loss function is expressed as: (6); Among them, PolyCE represents the polynomial cross entropy loss function. A vector representing the true labels. A label vector representing the model's predictions. represents the standard cross entropy loss function, which is calculated as Represents a small positive constant used to adjust the balance of the loss function. Represents the first elements, specifically pointing to the wrong prediction items. Represents the power of a polynomial term.

[0057] In summary, the total loss of the ECA_DLAseg backbone network includes the detection loss of the joint detection head and the re-identification loss of the re-identification module ReID. The total loss of the ECA_DLAseg backbone network is expressed as: (7); Where L represents the total loss of the ECA_DLAseg backbone network, represents the weight adjustment term of the detection loss of the joint detection head, represents the weight adjustment term for the re-identification loss, represents the re-identification loss and detection loss, represents the re-identification loss, represents the learnable weight of the detection, Represents the learnable weights for the re-ID loss.

[0058] Step 5: Use the Kalman filter to predict the trajectory of each detected target and obtain the predicted trajectory.

[0059] The Kalman filter is used to predict the trajectory of each detection target to obtain a predicted trajectory, wherein the predicted trajectory includes a confirmed detection target trajectory and an unconfirmed detection target trajectory.

[0060] Step 6: Match the predicted trajectory with the unique identity of each detected target through data association to obtain the final multi-target tracking result. The flowchart of the multi-target tracking method for football sports scene provided by the embodiment of the present invention is as follows: Figure 8 shown.

[0061] Cascade matching is performed on the confirmed detection target trajectory and the unique identity of each detection target to obtain a first unmatched detection result, a first unmatched trajectory, and a first matched trajectory; Perform Hungarian matching on the first mismatch detection result and the first mismatch trajectory and the unconfirmed detection target trajectory to obtain a second mismatch detection result, a second mismatch trajectory and a second matching trajectory; Inputting the second matching trajectory into a Kalman filter to update the parameters of the first matching trajectory, thereby obtaining an updated first matching trajectory; Input the second mismatch detection result into the Kalman filter and initialize it as a new trajectory; If the second unmatched trajectory and the new trajectory can match the confirmed detection target trajectory in three consecutive frames, they are designated as confirmed trajectories and sent to the Kalman filter to update the parameters of the first matching trajectory to obtain the updated first matching trajectory; If the second unmatched trajectory and the new trajectory cannot match the confirmed detection target trajectory in several consecutive frames, and the number of consecutive unmatched frames between the second unmatched trajectory and the new trajectory is less than the set threshold, the second unmatched trajectory and the new trajectory are retained, and the parameters of the first matching trajectory are updated by the Kalman filter to obtain the updated first matching trajectory; If the second unmatched trajectory and the new trajectory cannot match the determined trajectory of the feature map detection result in several consecutive frames, and the number of consecutive unmatched frames of the new trajectory is greater than or equal to the set threshold, delete the new trajectory; Output all updated first matching trajectories as multi-target tracking results.

[0062] Example 2

[0063] A computer-readable storage medium having computer instructions stored thereon, characterized in that the computer instructions implement the steps of the method in the above-mentioned embodiment 1 when executed by a processor.

[0064] Example 3

[0065] The present invention also provides a computer program product, including computer instructions, characterized in that when the computer instructions are executed by a processor, the steps of the method in the above-mentioned embodiment 1 are implemented.

[0066] In summary, the present invention successfully achieves accurate, efficient and robust tracking of multiple targets in football scenes by adopting the ECA_DLAseg backbone network, joint detection head, re-identification module ReID and Kalman filter. This technology can not only effectively extract and identify multiple targets in video frames, but also predict the movement trajectory of the target, and assign a unique identity to each target through the re-identification module, ensuring that a stable tracking effect can be maintained even in complex environments, providing strong technical support for real-time analysis and decision-making in football scenes. It solves the problem that the existing technology cannot solve the problem of multi-target occlusion and real-time tracking and accurate distinction of each player's trajectory.

[0067] The present invention provides a multi-target tracking method for football sports scenes. By introducing an efficient channel attention mechanism ECA to enhance the deep layer aggregation segmentation network DLAseg, the detection accuracy of the ECA_DLAseg backbone network for long-distance and detection targets in football sports scenes is significantly improved, so that detail information can be better retained and utilized; secondly, the joint detection head of the present invention integrates an intersection-over-union (IoU) calculation module, and introduces a generalized intersection-over-union (GIoU) ​​loss function in the intersection-over-union (IoU) calculation module, which effectively improves the detection performance under complex occlusion and multi-player interaction conditions, ensuring accurate detection and positioning in dense scenes; furthermore, the present invention also introduces a polynomial cross entropy PolyCE loss function in the module re-identification module ReID to balance the ratio of positive and negative samples, reduce the overfitting of the ECA_DLAseg backbone network for frequently appearing targets, and enhance the ability of the ECA_DLAseg backbone network to distinguish targets with similar appearances, thereby improving the accuracy and robustness of tracking.

[0068] The present invention is trained and evaluated on the SoccerNet Tracking dataset, verifying the superior performance of a multi-target tracking method for football scenes provided by the present invention in fast-changing, complex occlusion and multi-player interaction scenes, demonstrating its potential and effectiveness in actual football game applications.

[0069] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0070] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0071] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0072] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0073] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A multi-target tracking method for football scenes, It is characterized in that include: Acquire video frame images of football sports scenes; Input the video frame images of the football scene into the ECA_DLAseg backbone network to obtain the multi-target tracking results; The ECA_DLAseg backbone network includes a joint detection head, a re-identification module ReID, a Kalman filter and a data association module; The joint detection head is used to output the feature map containing the detection target as a feature map detection result including a heat map, a center offset, a bounding box size and an intersection-over-union ratio; The re-identification module ReID uses a polynomial cross entropy PolyCE loss function as the re-identification loss to balance the ratio of positive and negative samples; Used to generate a unique identity for each detected target based on the feature map detection results including heat map, center offset, bounding box size and intersection-over-union ratio; The Kalman filter is used to predict the trajectory of each detected target to obtain a predicted trajectory; The data association module is used to match the trajectory predicted by the Kalman filter with the unique identity of each detected target through data association to obtain the final multi-target tracking result; The training method of the ECA_DLAseg backbone network includes: According to the training set and the total loss function, the ECA_DLAseg backbone network is trained to obtain a trained ECA_DLAseg backbone network.

2. The multi-target tracking method for a football scene according to claim 1, It is characterized in that The method for constructing the ECA_DLAseg backbone network comprises: The upsampling ratio of the input feature map by the deep layer aggregation segmentation network DLAseg is increased from 1 / 4 of the original size of the feature map to 1 / 2 of the original size of the feature map; The efficient channel attention mechanism ECA is embedded in each block of the deep layer aggregation segmentation network DLAseg to obtain the ECA_DLAseg backbone network.

3. The multi-target tracking method for football sports scenes according to claim 1, It is characterized in that The method for constructing the joint detection head comprises: The generalized intersection-over-union (GIoU) ​​loss function is introduced into the intersection-over-union (IoU) calculation module to obtain an improved IoU calculation module. An improved IoU calculation module is added to the detection head to obtain a joint detection head.

4. The multi-target tracking method for a football scene according to claim 1, It is characterized in that A method for generating a unique identity for each detected object based on a feature map detection result including a heat map, a center offset, a bounding box size, and an intersection-over-union ratio, including: Based on the detection results of the feature map including the heat map, center offset, bounding box size and intersection-over-union ratio, the bounding box of each detected target is decoded, the bounding box is filtered using non-maximum suppression (NMS), and the appearance feature map of each detected target is extracted from the filtered bounding box; The appearance feature map of each detected target is processed through two layers of convolution and ReLU activation function, and the number of channels of the appearance feature map of each detected target is reduced to 128 to obtain the processed appearance feature map; Through the classifier, the features embedded in the processed appearance feature map are converted into scores corresponding to each detected target as the unique identity of each detected target.

5. The multi-target tracking method for football sports scenes according to claim 1, It is characterized in that The total loss obtained according to the total loss function of the ECA_DLAseg backbone network includes the detection loss and re-identification loss of the joint detection head. The total loss of the ECA_DLAseg backbone network is expressed as: ; Where L represents the total loss of the ECA_DLAseg backbone network, represents the weight adjustment term of the detection loss of the joint detection head, represents the weight adjustment term for the re-identification loss, represents the re-identification loss and detection loss, represents the re-identification loss, represents the learnable weight of the detection, Represents the learnable weights for the re-ID loss.

6. The multi-target tracking method for a football sport scene according to claim 5, It is characterized in that The detection loss of the joint detection head includes heat map loss, size loss, offset loss and intersection over union (IoU) loss; the heat map loss, size loss, offset loss and IoU loss are weighted and summed according to weights to obtain the detection loss.

7. The multi-target tracking method for a football sport scene according to claim 1, It is characterized in that The Kalman filter is used to predict the trajectory of each detection target to obtain a predicted trajectory, wherein the predicted trajectory includes a confirmed detection target trajectory and an unconfirmed detection target trajectory.

8. The multi-target tracking method for a football sport scene according to claim 7, It is characterized in that The steps of matching the trajectory predicted by the Kalman filter with the unique identity of each detected target through data association to obtain the final multi-target tracking result include: Cascade matching is performed on the confirmed detection target trajectory and the unique identity of each detection target to obtain a first unmatched detection result, a first unmatched trajectory, and a first matched trajectory; Perform Hungarian matching on the first mismatch detection result and the first mismatch trajectory and the unconfirmed detection target trajectory to obtain a second mismatch detection result, a second mismatch trajectory and a second matching trajectory; Inputting the second matching trajectory into a Kalman filter to update the parameters of the first matching trajectory, thereby obtaining an updated first matching trajectory; Input the second mismatch detection result into the Kalman filter and initialize it as a new trajectory; If the second unmatched trajectory and the new trajectory can match the confirmed detection target trajectory in three consecutive frames, they are designated as confirmed trajectories and sent to the Kalman filter to update the parameters of the first matching trajectory to obtain the updated first matching trajectory; If the second unmatched trajectory and the new trajectory cannot match the confirmed detection target trajectory in several consecutive frames, and the number of consecutive unmatched frames between the second unmatched trajectory and the new trajectory is less than the set threshold, the second unmatched trajectory and the new trajectory are retained, and the parameters of the first matching trajectory are updated by the Kalman filter to obtain the updated first matching trajectory; If the second unmatched trajectory and the new trajectory cannot match the determined trajectory of the feature map detection result in several consecutive frames, and the number of consecutive unmatched frames of the new trajectory is greater than or equal to the set threshold, delete the new trajectory; Output all updated first matching trajectories as multi-target tracking results.

9. A computer-readable storage medium having a computer program / instructions stored thereon, It is characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer program product comprising a computer program / instructions, It is characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Citation Information

Patent Citations

  • Unmanned aerial vehicle video multi-target tracking method based on attention feature fusion

    CN113807187A

  • Infrared unmanned aerial vehicle small target detection and tracking method in air background

    CN115761534A

  • Ship image trajectory tracking and prediction method based on ship heading recognition

    US20240013402A1

Cited By

  • Detection model training method, tracking method, equipment, storage medium and product

    CN119863688A