Airport scene multi-target tracking method and system

Abstract

The invention discloses an airport scene multi-target tracking method and system, and belongs to the technical field of computer vision, the airport scene multi-target tracking method comprises a training stage and a testing stage, the training stage comprises the following steps: S1, using a backbone network to obtain global apparent characteristics representing video frames; s2, obtaining a target attention feature representing dynamic local fine-grained visual information of the video frame; s3, acquiring local fine-grained visual scale alignment features of the video frames, and performing multi-target tracking in combination with global features; and S4, tracking a to-be-tracked target appearing in the video by using a multi-target tracking network, and calculating error loss to help to train the network. According to the technical scheme, the precision and the stability of multi-target tracking can be remarkably improved.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision, and in particular relates to a method and system for tracking multiple targets at an airport. Background Art

[0002] In airport scene monitoring, multi-target tracking is an important technology to ensure airport safety and efficiency. Currently, the methods of multi-target tracking mainly include methods based on traditional machine learning and methods based on deep learning.

[0003] Traditional machine learning-based methods usually use low-level visual features such as trajectories, color histograms, and motion vectors to represent the appearance and motion characteristics of the target, and then use machine learning algorithms to track the target. However, these methods often have difficulty in achieving satisfactory results when dealing with complex scenes, occlusions, and multi-scale targets. Deep learning-based methods use deep learning models such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs) to directly learn the features of the target from video frames and track it. Although deep learning methods have significant advantages in feature extraction and modeling capabilities, they still face challenges when dealing with mutual occlusion between targets, targets of different scales, and complex motion trajectories.

[0004] Especially in airport scenes, the target may not be fully detected due to occlusion by ground facilities, aircraft and other objects. In addition, the target presents different scales at different viewing angles and distances, which further increases the difficulty of tracking. Existing methods have shortcomings in unifying the feature representation of targets of different scales and improving tracking accuracy and stability.

[0005] In order to solve these problems, the present invention proposes a method and system for multi-target tracking in airport scenes based on dynamic patch and multi-scale feature fusion. By dynamically adjusting the center point and width and height of the patch, each target can be accurately cropped, and the SimCLR self-supervised contrast learning model is used to extract the unified feature representation of similar targets at different scales. Combined with the fusion of global and local features, and the use of Transformer decoder, the accuracy and stability of multi-target tracking are significantly improved. This method can not only effectively track targets in complex scenes, but also maintain the global features of the target in time sequence, meeting the actual needs of multi-target tracking in airport scenes. Summary of the invention

[0006] In view of the defects of the prior art, the present invention provides a method and system for tracking multiple targets at airports. At the level of appearance features, the invention uses global-local modeling to extract and fuse global appearance features and local appearance features; at the level of scale features, the SimCLR self-supervised contrastive learning model is used to extract unified feature representations of similar targets at different scales; at the level of motion features, each target is accurately cut by a patch by dynamically adjusting the center point and width and height of the patch, and finally the target is tracked through the Transformer decoder, and the global features of the target are stored in time series.

[0007] The specific technical solution adopted by the present invention is:

[0008] The first invention purpose of this patent is to provide a method for tracking multiple targets at an airport, including a training phase and a testing phase, wherein the training phase includes:

[0009] S1, using the backbone network to obtain the global appearance features representing the video frame;

[0010] S2. Obtain dynamic target attention features representing local fine-grained visual information of the video frame; specifically including:

[0011] S2-1: First, the attention mechanism is used to learn the fine-grained semantic information of each target in the video frame, and then the video frame with the fine-grained semantic information of each target is divided into static patches to obtain the initial static local visual information;

[0012] S2-2: Using the global appearance features, regress the dynamic center point coordinates and width and height offsets of the static patch, and fuse them with the static local visual information into a local fine-grained dynamic patch;

[0013] S3, obtain the local fine-grained visual scale alignment features of the video frame, and combine the global features to perform multi-target tracking; specifically including:

[0014] S3-1: according to the local fine-grained dynamic patch, a self-supervised contrastive learning encoder SimCLR is used to extract contrastive learning features of the dynamic patch;

[0015] S3-2: The contrastive learning features are sampled through gated sampling to obtain scale-aligned features, and the enhanced global-local fine-grained appearance features are obtained by fusing the global appearance features in S1;

[0016] S3-3: using the scale-aligned features as queries, querying the enhanced global-local fine-grained appearance features for target positions through a query pre-decoder, thereby obtaining target position embedding information;

[0017] S4. Use the transformer-based multi-target tracking network to track the target to be tracked in the video and calculate the error loss to help train the network; specifically, it includes:

[0018] S4-1: The target position embedding information obtained by querying the query decoder is used as the detection query and concatenated with the tracking trajectory features as the tracking query to form a joint tracking query. The tracking result is obtained by querying the enhanced global-local fine-grained appearance features through the Transformer decoder.

[0019] Preferably, S2-1 is specifically as follows: for the video frame feature X extracted using the backbone network, the pooling layer is used to encode it in the horizontal and vertical directions respectively to obtain [Z h ,Z w ]; then the encoded features are subjected to convolution, activation function and sigmoid to obtain the attention score:

[0020] [g h ,g w ]=σ(F 1 (δ(conv([Z h ,Z w ]))))

[0021] Where: h and w represent the height and width of the video frame respectively; F 1 represents the convolutional layer;

[0022] At this point, the attention score is multiplied by the original video frame I to obtain fine-grained semantic information: g h (i)*g w (j)*I(i,j); then the video frame enhanced with fine-grained semantics is resized to 224*224 and then divided into 64 patches, which is the so-called initial static local visual information [P 1 ,P 2 ,...,P 64 ].

[0023] Preferably, S2-2 is specifically as follows: using the video frame feature X extracted by the backbone network, predicting a set of (δ x ,δ y ,δ w ,δ h ):

[0024] δ x ,δ y =Tanh(W offset f(X)

[0025] δ w ,δh =σ(Tanh(W scale f(X)+b scale ))

[0026] Among them: f is the convolution layer, W and b are parameters;

[0027] Based on the original static patch, use (δ x ,δ y ,δ w ,δ h ) offset the original patch to obtain the fused local fine-grained dynamic patch.

[0028] Preferably, S3-1 specifically includes: for each dynamic patch P i and the ground truth target box G j Calculate IoU two by two to get the cost matrix of dynamic patch and ground truth:

[0029] cost(i,j)=IoU(P i ,G j )

[0030] Use the Hungarian matching algorithm to match a unique dynamic patch for each ground truth target box, and record the matched K dynamic patches as: [p 1 ,p 2 ,...,p K ], calculate the patch loss:

[0031] Loss = w l ·L 1 (P k ,G j )+w g ·L GIoU (P k ,G j )+w f ·FL(p,P i )

[0032] Where: w is the loss weight; P i ,G j represents a pair of matched dynamic patches and target boxes, p represents the predicted probability of whether there is a target in each dynamic patch, P i for:

[0033]

[0034] Use a pre-trained self-supervised contrastive learning network to extract K target features that are not affected by scale and angle:

[0035] [A 1 ,A 2 ,...,A K ]=SimCLR([p 1 ,p 2 ,...,p K ])

[0036] In the feature space, the distance between similar targets is shortened, and the distance between different targets is increased. In this stage, the features extracted by SimCLR are used in the feature space that is not affected by scale and angle, and finally the comparative learning features of the dynamic patch are obtained.

[0037] Preferably, S3-2 is specifically: using a gating mechanism to k Through upsampling or downsampling modules respectively: A' k = upsample(A k ) or A' k = downsample(A k ), reduce the size difference of features and improve the overall expressiveness of features to obtain scale-aligned features;

[0038] The scale-aligned features are transformed to the same size as feature X and then concatenated with the global appearance features of the video frames in S1 to obtain the enhanced global-local fine-grained appearance features feature', so as to integrate the multi-scale global expression capabilities of the global and local fine-grained enhanced features.

[0039] Preferably, S3-3 is specifically as follows: using the scale alignment feature as a query to query the target position to the enhanced global-local fine-grained appearance feature through a query pre-decoder; at the same time, using a learnable target query set Q l And the local fine-grained alignment feature query set Q d Composed joint detection query set Q J Query the fused features to get the decoded target position embedding information:

[0040] Decoder query (Q J ,feature').

[0041] Preferably, S4-1 is specifically: using the target position embedding information obtained by decoding in S3-3 as the detection query of this stage and the tracking trajectory feature initialized as the tracking query, splicing them into a joint tracking Q tThe transformer decoder queries the enhanced global-local fine-grained appearance features to decode the tracking results and fine target locations:

[0042] Decoder(Q t ,feature').

[0044] The second invention purpose of this patent is to provide an airport scene multi-target tracking system, including a training module and a test module, the training module includes:

[0045] The global appearance feature acquisition module uses the backbone network to obtain the global appearance features representing the video frame;

[0046] The dynamic target attention feature acquisition module obtains the dynamic target attention features representing the local fine-grained visual information of the video frame; specifically includes:

[0047] First, the attention mechanism is used to learn the fine-grained semantic information of each target in the video frame, and then the video frame with the fine-grained semantic information of each target is divided into static patches to obtain the initial static local visual information;

[0048] Using the global appearance features, regress the dynamic center point coordinates and width and height offsets of the static patch, and fuse them with the static local visual information to form a local fine-grained dynamic patch;

[0049] The multi-target tracking module obtains the local fine-grained visual scale alignment features of the video frame and combines the global features to perform multi-target tracking. Specifically, it includes:

[0050] According to the local fine-grained dynamic patch, a self-supervised contrastive learning encoder SimCLR is used to extract contrastive learning features of the dynamic patch;

[0051] The contrastive learning features are sampled through gated sampling to obtain scale-aligned features, and the enhanced global-local fine-grained appearance features are obtained by fusing the global appearance features in S1;

[0052] Using the scale-aligned features as queries, querying the enhanced global-local fine-grained appearance features for target positions through a query pre-decoder, thereby obtaining target position embedding information;

[0053] The result output module uses a transformer-based multi-target tracking network to track the target to be tracked in the video and calculate the error loss to help train the network; specifically, it includes:

[0054] The target position embedding information obtained by querying the query decoder is used as the detection query and concatenated with the tracking trajectory features as the tracking query to form a joint tracking query. The tracking result is obtained by querying the enhanced global-local fine-grained appearance features through the Transformer decoder.

[0055] The third invention objective of this patent is to provide a computer program product, including a computer program, which, when executed by a processor, is the above-mentioned airport scene multi-target tracking method.

[0056] The fourth invention objective of this patent is to provide an information data processing terminal for implementing the above-mentioned airport scene multi-target tracking method.

[0057] The fifth invention objective of this patent is to provide a computer-readable storage medium, including instructions, which, when run on a computer, enables the computer to execute the above-mentioned airport scene multi-target tracking method.

[0058] The advantages and positive effects of the present invention are:

[0059] By adopting the above technical solution, the present invention has the following technical effects:

[0060] (1) In terms of appearance features, the present invention not only focuses on the global appearance features of the video frame, but also deeply explores the local appearance features. By cleverly extracting and fusing the global and local appearance features, the global and local modeling can make full use of these features. This method can accurately extract the detailed features of the target, thereby significantly improving the accuracy of local minor abnormal behavior detection, making the detection results more accurate and reliable.

[0061] (2) At the scale feature level, the present invention uses the SimCLR self-supervised contrastive learning model to extract unified feature representations at different scales. This method can effectively capture feature information at different scales, thereby providing richer data support at the scale feature level. At the motion feature level, the present invention ensures that each target can be accurately cut by a patch by dynamically adjusting the center point and width and height of the patch. This dynamic adjustment mechanism enables the motion characteristics of the target to be captured and represented more accurately, thereby providing a more reliable data basis for subsequent processing.

[0062] (3) At the motion feature level, the present invention ensures that each target can be accurately cut by a Patch by dynamically adjusting the center point and width and height of the Patch. This dynamic adjustment mechanism enables the motion features of the target to be captured and represented more accurately, thereby providing a more reliable data basis for subsequent processing. Finally, tracking is performed through the Transformer decoder, and the global features of the target are stored in time series. This method not only improves the accuracy of multi-target tracking, but also enhances its stability. In this way, the present invention can track multiple targets more accurately and maintain high tracking performance in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 is a structural block diagram of a preferred embodiment of the present invention;

[0064] Figure 2 Schematic diagram of the working principle of the preferred embodiment of the present invention. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] See also Figure 1 to Figure 2 ;

[0067] The first embodiment is a method for tracking multiple targets at an airport, which mainly includes a training phase and a testing phase, wherein the training phase includes:

[0068] S1. Obtain global appearance features representing the video frame; in this process, it is necessary to extract key features that can fully reflect the visual information of the video frame for subsequent processing and analysis. To achieve this goal, this embodiment will adopt an efficient and widely recognized method, that is, using a powerful backbone network to extract features.

[0069] Specifically, you can choose to use a classic backbone network structure like ResNet, which has shown excellent performance in image recognition and feature extraction. By inputting the video frame into the ResNet network, a set of global appearance features can be obtained. These features not only cover the overall visual content of the video frame, but also capture the key details and texture information in the frame. The ResNet network, through its deep structure and residual connection mechanism, can effectively solve the gradient vanishing problem in deep networks, thereby extracting richer and more discriminative features. These features will provide a solid foundation for subsequent video analysis tasks, allowing us to better understand and process video data.

[0070] S2. Obtaining dynamic target attention characteristics that characterize local detailed visual features of video frames; wherein:

[0071] S2-1: Use the attention mechanism to learn and extract the detailed semantic information of each target in the video frame, and then divide these video frames containing the detailed semantic information of each target into static image patches to obtain preliminary static local visual information; the specific steps are as follows:

[0072] The video frame feature X extracted by the backbone network is encoded in the horizontal and vertical directions using the pooling layer to obtain [Z h ,Z w ]; then the encoded features are subjected to convolution, activation function and sigmoid to obtain the attention score:

[0073] [g h ,g w ]=σ(F 1 (δ(conv([Z h ,Z w ]))))

[0074] Where: h and w represent the height and width of the video frame respectively; F 1 It also represents a convolutional layer.

[0075] At this point, the attention score is multiplied by the original video frame I to obtain fine-grained semantic information: g h (i)*g w The video frame enhanced with fine-grained semantics is then resized to 224*224 and then divided into 64 patches, the so-called initial static local visual information [P 1 ,P 2 ,...,P 64 ].

[0076] S2-2: By using the backbone network, the global appearance features of the video frame are extracted, and then the dynamic center point coordinates and width and height offsets of the static patch are determined through the regression algorithm. Finally, this information is integrated with the static local visual information to form a local fine-grained dynamic patch. Specifically:

[0077] Using the video frame features X extracted by the backbone network, a set of (δ x ,δ y ,δ w ,δ h ):

[0078] δ x ,δ y =Tanh(W offset f(X)

[0079] δ w ,δ h =σ(Tanh(W scale f(X)+b scale ))

[0080] Among them: f is the convolutional layer, W and b are parameters.

[0081] Based on the original static patch, use (δ x ,δ y ,δ w ,δ h ) offset the original patch to obtain the fused local fine-grained dynamic patch.

[0082] S3, obtain the local fine-grained visual scale alignment features of the video frame, and combine the global features to perform multi-target tracking; wherein:

[0083] S3-1: Based on the obtained local fine-grained dynamic patch, the self-supervised contrastive learning encoder SimCLR is used to extract the contrastive learning features of the dynamic patch; specifically:

[0084] For each dynamic patch P i and the ground truth target box G j Calculate IoU two by two to get the cost matrix of dynamic patch and ground truth:

[0085] cost(i,j)=IoU(P i ,G j )

[0086] Use the Hungarian matching algorithm to match a unique dynamic patch for each ground truth target box, and record the matched K dynamic patches as: [p 1 ,p 2 ,...,p K ], calculate the patch loss:

[0087] Loss = w l ·L 1 (P k ,G j )+w g ·L GIoU (P k ,G j )+w f ·FL(p,P i )

[0088] Where: w is the loss weight; P i ,G j represents a pair of matched dynamic patches and target boxes, p represents the predicted probability of whether there is a target in each dynamic patch, P i for:

[0089]

[0090] Use a pre-trained self-supervised contrastive learning network to extract K target features that are not affected by scale and angle:

[0091] [A 1 ,A 2 ,...,A K ]=SimCLR([p 1 ,p 2 ,...,p K ])

[0092] In the feature space, the distance between similar targets is shortened, and the distance between different targets is increased. In this stage, the features extracted by SimCLR are used in the feature space that is not affected by scale and angle, and finally the comparative learning features of the dynamic patch are obtained.

[0093] S3-2: The contrastive learning features are sampled through gated sampling to obtain scale alignment features, and the enhanced global-local fine-grained appearance features are obtained by fusing the global appearance features of the video frame in S1-1; specifically:

[0094] Using the gating mechanism, features A of different sizes are k Through upsampling or downsampling modules respectively: A' k = upsample(A k ) or A'k = downsample(A k ), reduce the size difference of features and improve the overall expressiveness of features to obtain scale-aligned features.

[0095] Finally, the scale-aligned features are transformed to the same size as feature X and then concatenated with the global appearance features of the video frame S1-1 to obtain the enhanced global-local fine-grained appearance feature feature', so as to integrate the multi-scale global expression capabilities of the global and local fine-grained enhanced features.

[0096] S3-3: Use the scale alignment feature of S3-2 as a query to query the target position through a query pre-decoder to the enhanced global-local fine-grained appearance feature, thereby obtaining the target position embedding information; specifically:

[0097] The scale-aligned features are used as queries to query the enhanced global-local fine-grained appearance features through a query pre-decoder to query the target location; at the same time, in order to prevent the scale-aligned features from missing some target features, a learnable target query set Q is used l And the local fine-grained alignment feature query set Q d Composed joint detection query set Q J Query the fused features to get the decoded target position embedding information:

[0098] Decoder query (Q J ,feature').

[0099] S4. Use the transformer-based multi-target tracking network to track the target to be tracked in the video and calculate the error loss to help train the network; where:

[0100] S4-1: The target position embedding information obtained by querying the query decoder is used as the detection query and the tracking trajectory feature as the tracking query is concatenated into a joint tracking query. The tracking result is obtained by querying the enhanced global-local fine-grained appearance feature through the Transformer decoder; specifically:

[0101] The target position embedding information obtained by S3-3 decoding is used as the detection query of this stage and is concatenated with the initial tracking trajectory feature as the tracking query to form the joint tracking Q t The transformer decoder queries the enhanced global-local fine-grained appearance features to decode the tracking results and fine target locations:

[0102] Decoder(Q t,feature').

[0103] At the appearance feature level, the present invention adopts an innovative method to effectively extract and fuse global appearance features and local appearance features. This fusion strategy enables the model to capture both the overall features and detailed features of the target, thereby obtaining more comprehensive and accurate appearance information in multi-target tracking tasks. Specifically, global features provide the overall contour and shape information of the target, while local features focus on the key parts and detailed changes of the target. The combination of the two makes the model more robust when facing complex scenes.

[0104] At the scale feature level, the present invention uses the SimCLR self-supervised contrastive learning model to extract unified feature representations at different scales. The SimCLR model maximizes the similarity between positive sample pairs and minimizes the similarity between negative sample pairs, so that the model can learn discriminative feature representations in an unsupervised manner. The advantage of this method is that it can extract representative scale features from large-scale unlabeled data, thereby coping with targets of different scales in multi-target tracking tasks.

[0105] At the motion feature level, the present invention ensures that each target is accurately cut by a patch by dynamically adjusting the center point and width and height of the patch. The core of this method lies in the dynamic adjustment mechanism, which can adjust the parameters of the patch in real time according to the motion state and shape changes of the target, thereby ensuring that the target is always in the center of the patch and the size of the patch can adapt to the size changes of the target. This dynamic adjustment mechanism significantly improves the accuracy and robustness of target detection, laying a solid foundation for subsequent tracking tasks.

[0106] Finally, the present invention tracks through the Transformer decoder and stores the global features of the target in time series. The Transformer decoder uses the self-attention mechanism to capture the correlation and change trend of the target in time series, thereby achieving continuous tracking of the target. At the same time, by storing the global features of the target in time series, the model can better understand and remember the movement pattern and behavior characteristics of the target, further improving the accuracy and stability of multi-target tracking.

[0107] In summary, the present invention has conducted in-depth research and innovation in three aspects: appearance features, scale features, and motion features. By integrating global and local appearance features, extracting unified feature representations at different scales, and dynamically adjusting Patch parameters, the Transformer decoder is finally used for efficient tracking, and the global features of the target are stored in time series. The comprehensive application of these methods significantly improves the accuracy and stability of multi-target tracking, and provides new ideas and solutions for solving the problem of multi-target tracking in complex scenarios.

[0108] The second embodiment is an airport multi-target tracking system, which is used to execute the method of the first embodiment, and includes a training module and a test module, wherein: the training module includes:

[0109] The global appearance feature acquisition module obtains the global appearance features representing the video frame; wherein:

[0110] Use backbone networks, such as ResNet, to obtain global appearance features of video frames;

[0111] The dynamic target attention feature acquisition module obtains the dynamic target attention features representing the local fine-grained visual information of the video frame; wherein:

[0112] Use the attention mechanism to learn the fine-grained semantic information of each target in the video frame, and then divide the video frame with the fine-grained semantic information of each target into static patches to obtain the initial static local visual information;

[0113] The global appearance features of the video frame extracted by the backbone network are used to regress the dynamic center point coordinates and width and height offsets of the static patch, which are then fused with the static local visual information to form a local fine-grained dynamic patch.

[0114] The multi-target tracking module obtains the local fine-grained visual scale alignment features of the video frame and combines the global features to perform multi-target tracking;

[0115] According to the obtained local fine-grained dynamic patch, the self-supervised contrastive learning encoder SimCLR is used to extract the contrastive learning features of the dynamic patch;

[0116] The contrastive learning features are sampled through gated sampling to obtain scale-aligned features, and the enhanced global-local fine-grained appearance features are obtained by fusing the global appearance features of the video frames.

[0117] Use the scale-aligned features as queries to query the target location through a query pre-decoder to the enhanced global-local fine-grained appearance features, thereby obtaining the target location embedding information;

[0118] The result output module uses a transformer-based multi-target tracking network to track the target to be tracked in the video and calculate the error loss to help train the network;

[0119] The target position embedding information obtained by querying the query decoder is used as the detection query and concatenated with the tracking trajectory features as the tracking query to form a joint tracking query. The tracking result is obtained by querying the enhanced global-local fine-grained appearance features through the Transformer decoder.

[0120] A third embodiment is a computer-readable storage medium storing a computer program, which implements the above-mentioned airport scene multi-target tracking method when executed by a processor.

[0121] A fourth embodiment is a computer program product, comprising a computer program, which implements the above-mentioned airport scene multi-target tracking method when executed by a processor.

[0122] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When the use is implemented in whole or in part in the form of a computer program product, the computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL) or wireless (e.g., infrared, wireless, microwave, etc.) mode) to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk SolidState Disk (SSD)), etc.

[0123] The above is only a preferred embodiment of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for tracking multiple targets at an airport, comprising a training phase and a testing phase, characterized in that: The training phase includes: S1, using the backbone network to obtain the global appearance features representing the video frame; S2. Obtain dynamic target attention features representing local fine-grained visual information of the video frame; specifically including: S2-1: First, the attention mechanism is used to learn the fine-grained semantic information of each target in the video frame, and then the video frame with the fine-grained semantic information of each target is divided into static patches to obtain the initial static local visual information; S2-2: Using the global appearance features, regress the dynamic center point coordinates and width and height offsets of the static patch, and fuse them with the static local visual information into a local fine-grained dynamic patch; S3, obtain the local fine-grained visual scale alignment features of the video frame, and combine the global features to perform multi-target tracking; specifically including: S3-1: according to the local fine-grained dynamic patch, a self-supervised contrastive learning encoder SimCLR is used to extract contrastive learning features of the dynamic patch; S3-2: The contrastive learning features are sampled through gated sampling to obtain scale-aligned features, and the enhanced global-local fine-grained appearance features are obtained by fusing the global appearance features in S1; S3-3: using the scale-aligned features as queries, querying the enhanced global-local fine-grained appearance features for target positions through a query pre-decoder, thereby obtaining target position embedding information; S4. Use the transformer-based multi-target tracking network to track the target to be tracked in the video and calculate the error loss to help train the network; specifically, it includes: S4-1: The target position embedding information obtained by querying the query decoder is used as the detection query and concatenated with the tracking trajectory features as the tracking query to form a joint tracking query. The tracking result is obtained by querying the enhanced global-local fine-grained appearance features through the Transformer decoder.

2. The airport scene multi-target tracking method according to claim 1, characterized in that: S2-1 is specifically: The video frame feature X extracted by the backbone network is encoded in the horizontal and vertical directions using the pooling layer to obtain [Z h ,Z w ]; then the encoded features are subjected to convolution, activation function and sigmoid to obtain the attention score: [g h ,g w ]=σ(F1(δ(conv([Z h ,Z w ])))) Where: h and w represent the height and width of the video frame respectively; F1 represents the convolutional layer; At this point, the attention score is multiplied by the original video frame I to obtain fine-grained semantic information: g h (i)*g w (j)*I(i,j); then the video frame with fine-grained semantic enhancement is resized to 224*224, and then divided into 64 patches, namely the so-called initial static local visual information [P1,P2,...,P 64 ].

3. The airport scene multi-target tracking method according to claim 1, characterized in that: S2-2 is specifically: Using the video frame features X extracted by the backbone network, a set of (δ x ,δ y ,δ w ,δ h ): δ x ,δ y =Tanh(W offset ·f(X)) δ w ,δ h =σ(Tanh(W scale ·f(X)+b scale )) Among them: f is the convolution layer, W and b are parameters; Based on the original static patch, use (δ x ,δ y ,δ w ,δ h ) offset the original patch to obtain the fused local fine-grained dynamic patch.

4. The airport scene multi-target tracking method according to claim 1, characterized in that: S3-1 is as follows: For each dynamic patch P i and the ground truth target box G j Calculate IoU two by two to get the cost matrix of dynamic patch and ground truth: cost(i,j)=IoU(P i ,G j ) Use the Hungarian matching algorithm to match a unique dynamic patch for each ground truth target box, and record the matched K dynamic patches as: [p1, p2, ..., p K ], calculate the patch loss: Loss=w l ·L1(P k ,G j )+w g ·L GIoU (P k ,G j )+w f ·FL(p,P i ) Where: w is the loss weight; P i ,G j represents a pair of matched dynamic patches and target boxes, p represents the predicted probability of whether there is a target in each dynamic patch, P i for: Use a pre-trained self-supervised contrastive learning network to extract K target features that are not affected by scale and angle: [A1,A2,...,A K ]=SimCLR([p1,p2,...,p K ]) In the feature space, the distance between similar targets is shortened, and the distance between different targets is increased. In this stage, the features extracted by SimCLR are used in the feature space that is not affected by scale and angle, and finally the comparative learning features of the dynamic patch are obtained.

5. The airport scene multi-target tracking method according to claim 1, characterized in that: S3-2 is specifically: Using the gating mechanism, features A of different sizes are k Through upsampling or downsampling modules respectively: A' k = upsample(A k ) or A' k = downsample(A k ), reduce the size difference of features and improve the overall expressiveness of features to obtain scale-aligned features; The scale-aligned features are transformed to the same size as feature X and then concatenated with the global appearance features of the video frames in S1 to obtain the enhanced global-local fine-grained appearance features feature', so as to integrate the multi-scale global expression capabilities of the global and local fine-grained enhanced features.

6. The airport scene multi-target tracking method according to claim 1, characterized in that: S3-3 is specifically: The scale-aligned features are used as queries to query the target location through a query pre-decoder to the enhanced global-local fine-grained appearance features; at the same time, a learnable target query set Q is used l And the local fine-grained alignment feature query set Q d Composed joint detection query set Q J Query the fused features to get the decoded target position embedding information: Decoder query (Q J ,feature')。 7. The airport scene multi-target tracking method according to claim 1, characterized in that: S4-1 is as follows: The target position embedding information obtained by S3-3 decoding is used as the detection query of this stage and the tracking trajectory features initialized as the tracking query are concatenated into the joint tracking Q t The transformer decoder queries the enhanced global-local fine-grained appearance features to decode the tracking results and fine target locations: Decoder(Q t ,feature')。 8. An airport scene multi-target tracking system, including a training module and a test module, characterized in that: The training module includes: The global appearance feature acquisition module uses the backbone network to obtain the global appearance features representing the video frame; The dynamic target attention feature acquisition module obtains the dynamic target attention features representing the local fine-grained visual information of the video frame; specifically includes: First, the attention mechanism is used to learn the fine-grained semantic information of each target in the video frame, and then the video frame with the fine-grained semantic information of each target is divided into static patches to obtain the initial static local visual information; Using the global appearance features, regress the dynamic center point coordinates and width and height offsets of the static patch, and fuse them with the static local visual information to form a local fine-grained dynamic patch; The multi-target tracking module obtains the local fine-grained visual scale alignment features of the video frame and combines the global features to perform multi-target tracking. Specifically, it includes: According to the local fine-grained dynamic patch, a self-supervised contrastive learning encoder SimCLR is used to extract contrastive learning features of the dynamic patch; The contrastive learning features are sampled through gated sampling to obtain scale-aligned features, and the enhanced global-local fine-grained appearance features are obtained by fusing the global appearance features in S1; Using the scale-aligned features as queries, querying the enhanced global-local fine-grained appearance features for target positions through a query pre-decoder, thereby obtaining target position embedding information; The result output module uses a transformer-based multi-target tracking network to track the target to be tracked in the video and calculate the error loss to help train the network; specifically, it includes: The target position embedding information obtained by querying the query decoder is used as the detection query and concatenated with the tracking trajectory features as the tracking query to form a joint tracking query. The tracking result is obtained by querying the enhanced global-local fine-grained appearance features through the Transformer decoder.

9. A computer program product, comprising a computer program, characterized in that The computer program, when executed by a processor, is the airport scene multi-target tracking method as described in any one of claims 1-7.

10. A computer-readable storage medium comprising instructions, which, when executed on a computer, enables the computer to execute the airport surface multi-target tracking method according to any one of claims 1 to 7.

Images