Long-time target tracking method and system based on generative adversarial network
By combining generative adversarial networks and convolutional neural networks, and using U-Net and ResNet to extract features, adversarial training, and adaptive template updates, the problems of target appearance changes and background interference in long-term target tracking are solved, achieving efficient and accurate target tracking.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- QINGHAI NORMAL UNIV
- Filing Date
- 2025-12-19
- Publication Date
- 2026-05-15
AI Technical Summary
Existing deep learning target tracking algorithms struggle to effectively handle changes in target appearance, prolonged background interference, and re-detection in long-term tracking tasks, resulting in insufficient tracking performance.
A generative adversarial network-based approach is adopted, using U-Net as the generator network and combining it with ResNet convolutional neural network for feature extraction. Through adversarial training and adaptive template update mechanism, candidate regions are generated and the sample set is enriched. The generator and discriminator networks are then retrained to achieve efficient target tracking.
It improves the accuracy and robustness of target tracking, enables quick and accurate target relocation, avoids tracking failures, and enhances the overall performance of long-term tracking.
Smart Images
Figure CN122048981A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the fields of computer vision and deep learning technology, and in particular relates to a long-term target tracking method and system based on generative adversarial networks. Background Technology
[0002] With the development of computer vision and artificial intelligence technologies, target tracking has become a key technology in many applications, such as video surveillance, autonomous driving, intelligent security, and human-computer interaction. The main task of target tracking is to continuously detect and locate targets in video sequences. However, in practical applications, target tracking faces many challenges, especially during long-term tracking. The appearance of a target may change significantly due to factors such as changes in lighting, occlusion, and deformation, while the scene background may also change dynamically, all of which make accurate and stable target tracking difficult.
[0003] Traditional target tracking methods rely primarily on hand-designed features and simple linear models, making them ill-suited to complex environmental changes. In recent years, the rise of deep learning technology has brought new opportunities to target tracking, particularly methods based on convolutional neural networks (CNNs), which have demonstrated excellent performance in many tracking tasks. However, existing deep learning tracking algorithms still have some shortcomings in long-duration tracking tasks. Specifically, their performance in handling changes in target appearance, long-term background interference, and re-detection needs improvement.
[0004] Generative Adversarial Networks (GANs), with their unique adversarial training mechanism, can effectively improve the performance of generative models. Through adversarial training between the generator and discriminator, GANs enable the generator to produce more realistic images. This mechanism has already achieved significant results in fields such as image generation, image inpainting, and super-resolution reconstruction. Summary of the Invention
[0005] Purpose of the invention: The purpose of this invention is to provide a long-term target tracking method and system based on generative adversarial networks that can improve the performance and robustness of target tracking.
[0006] Technical solution: The present invention provides a long-term target tracking method based on generative adversarial networks, comprising:
[0007] S1: Using a U-Net-based deep neural network as the generator network, the background information of the first frame of the target video is removed, the foreground target is separated, the parameters of the discriminator network are adjusted, and an anti-generator network is trained. During the adversarial training process, the generator network and the discriminator network improve their respective performance by competing with each other.
[0008] S2: Analyze the position of the previous frame of the current frame using the generation strategy and generate multiple candidate regions. Use a convolutional neural network ResNet to extract the features of the candidate regions and input the features of each region into the discriminator network to calculate the probability that each candidate region is the target.
[0009] S3: When the number of tracking frames exceeds a preset threshold, image patches are randomly generated around the target location to enrich the sample set, and positive and negative samples are collected to retrain the generator network and discriminator network to obtain the updated target template.
[0010] S4: During the tracking process, continuously monitor the predicted probability value of the discriminator network. If the preset probability value is not reached within a preset number of consecutive frames, use the target detector to re-detect the target, update the parameters and status, and start a new tracking.
[0011] Further, step S1 includes:
[0012] S1.1: A deep neural network based on the U-Net model is used as the generator network. The U-Net model is trained as a segmenter to segment the foreground object from the background in the first frame of the video to be processed, extract the foreground part and overlay it on a solid color background;
[0013] S1.2: Based on the first part of S1.1 The first frame of the input image for each sample via generator network The generated foreground targets are used to train the discriminator network adversarially. The parameters are fine-tuned, and the loss function for adversarial training is... It can be calculated using the following formula:
[0014] ;
[0015] in, Represents the number of samples. and They represent the first The foreground target of each sample and the first frame of the input image;
[0016] S1.3: Through generator network and discriminator network Adversarial training is performed to minimize the reconstruction error of the generator network and maximize the discriminator network's discriminative power, thereby optimizing the generator network's loss function. This includes the reconstruction error term and the adversarial training term:
[0017] ;
[0018] in, and It is a hyperparameter that balances the two loss terms. It is the reconstruction error term, used to represent the generator. The difference between the generated image and the real image.
[0019] Further, step S2 includes:
[0020] S2.1: Using the target position from the previous frame and the global image of the current frame as input, let the target position be... , and Using the center point coordinates of the target, in the current frame image, a set of candidate region blocks is randomly generated around the target's position from the previous frame. The generation strategy considers the target's motion pattern, calculating a scale factor based on the target's range of change and pose variation. Then, candidate regions are generated based on the scale factor and random offset. The calculation formula is as follows:
[0021] ;
[0022] Where Δx and Δy are random values drawn from a Gaussian probability distribution, representing the horizontal and vertical offsets of the candidate region, respectively. Scale factor;
[0023] S2.2: Use a convolutional neural network ResNet to extract features from each candidate region. For each candidate region, output a feature vector set of dimension C. The formula is expressed as follows:
[0024] ;
[0025] in, Indicates the first Feature vectors of candidate regions;
[0026] S2.3: The discriminator network accepts feature vectors As input, it outputs a probability value representing the probability that the input feature vector corresponds to the target, as shown in the following formula:
[0027] ;
[0028] Where D represents the discriminator network, Indicates candidate region The probability of containing the target;
[0029] The output probability values are sorted. When the highest probability value is greater than or equal to a preset threshold, the candidate region corresponding to the highest probability is taken as the final target position. Otherwise, the target position of the previous frame is maintained.
[0030] Further, step S2.1, which calculates the scale factor based on the target change range and the attitude change range, specifically involves: firstly, based on the previous frame... Historical information calculation of target area in frames, target baseline scale factor The specific calculation formula is as follows:
[0031]
[0032] in, express The scale of the frame target is represented by the target area. ; Indicates the first Target width of the frame Indicates the first Target height of the frame, It is a constant;
[0033] Based on the fundamental scale factor of the target The scale factor is calculated from the attitude change of the ensemble target. The specific calculation formula is as follows:
[0034] ;
[0035] in This is the attitude correction coefficient. The similarity of feature remainder terms can be calculated as follows:
[0036] ;
[0037] in and These are the feature vectors extracted from the current region and the reference template using ResNet in step S2, respectively. This represents the modulo operation of the eigenvector.
[0038] Further, step S3 includes:
[0039] S3.1: When the number of frames tracked reaches or exceeds a preset threshold, the target template update mechanism is triggered; the update mechanism specifically uses the target position tracked in the last frame as the target position estimate for the current frame, and this position is the position confirmed by the discriminator. S3.2: Generate several samples around the target location in the current frame. Specifically, randomly generate image patches around the target location to enrich the sample set, as shown in the following formula:
[0040] ;
[0041] in, Indicates sample , For image cropping operations, For the current frame image, This is an estimate of the target's location. This represents the offset relative to the estimated position;
[0042] The generated samples are labeled, with target regions marked as positive samples and background regions marked as negative samples, as follows:
[0043]
[0044] Create a labeled dataset for retraining the generator and discriminator networks;
[0045] S3.3: Using the expanded sample dataset from S3.2, train the generator network and the discriminator network to minimize the output probability of the discriminator network for the generated samples. Output probabilities of real samples The difference between them is expressed by the following formula:
[0046]
[0047] in, It is random noise. Let be the distribution of noise, representing the probability of random noise occurring;
[0048] To maximize the output probability of the true sample and minimizing the output probability of generated samples The formula for training the discriminator network for the target is as follows:
[0049]
[0050] in, The probability of a real image appearing. The sample is a real sample; then a generator network is used to generate an updated target template.
[0051] Further, step S4 includes:
[0052] S4.1: During the tracking process, continuously monitor the predicted probability value of the discriminator network and record the number of times the predicted probability value of the discriminator network is lower than the set threshold in consecutive frames;
[0053] S4.2: If the predicted probability values obtained by the discriminator network in consecutive frames are all lower than the set threshold, then the tracking is deemed to have failed.
[0054] S4.3: After tracking fails, the target detector is started to re-detect the target in the current frame; specifically, after the target detector re-detects the target, it initializes the tracking target, determines the initial position and size of the target, and updates the parameters and state in the tracking algorithm.
[0055] Based on the same inventive concept, the present invention also provides a long-term target tracking system based on generative adversarial networks, comprising:
[0056] The initialization module is used to use a U-Net-based deep neural network as the generator network to remove the background information of the first frame of the target video, separate the foreground target, adjust the parameters of the discriminator network, and train an anti-generator network. During the adversarial training process, the generator network and the discriminator network improve their respective performance by competing with each other.
[0057] The extraction module is used to analyze the position of the previous frame of the current frame and generate multiple candidate regions through the generation strategy. The features of the candidate regions are extracted using a convolutional neural network ResNet, and the features of each region are input into the discriminator network to calculate the probability that each candidate region is the target.
[0058] The update module is used to enrich the sample set by randomly generating image patches around the target location when the number of tracking frames exceeds a preset threshold, and to collect positive and negative samples for retraining the generator network and discriminator network to obtain the updated target template.
[0059] The tracking module continuously monitors the predicted probability value of the discriminator network during the tracking process. If the preset probability value is not reached within a preset number of consecutive frames, the target detector is used to re-detect the target, update the parameters and status, and start a new tracking process.
[0060] Based on the same inventive concept, the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the long-term target tracking method based on generative adversarial networks as described in any of the preceding claims.
[0061] Based on the same inventive concept, the present invention also provides a computing device, comprising: one or more processors, one or more memories, and one or more programs, the programs being stored in the memory and configured to be executed by the processor, the programs being loaded onto the processor to implement the steps of the long-term target tracking method based on generative adversarial networks according to any of the preceding claims.
[0062] Based on the same inventive concept, the present invention also provides a storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the steps of the long-term target tracking method based on generative adversarial networks as described in any of the preceding claims.
[0063] Beneficial Effects: Compared with existing technologies, this invention has the following significant advantages: 1. By combining generative adversarial networks and convolutional neural networks, this invention can effectively cope with dynamic changes in target appearance and background during long-term tracking; 2. This invention establishes an adaptive template update mechanism. By generating image patches around the target location, enriching the sample set, and collecting positive and negative samples to retrain the generator and discriminator, the model can adapt to changes in the target's appearance, thereby improving tracking accuracy and robustness; 3. This invention employs an efficient re-detection mechanism, ensuring that when the target is lost or its appearance changes significantly, it can quickly and accurately relocate the target, avoiding tracking failure and improving overall tracking performance. Attached Figure Description
[0064] Figure 1 This is a flowchart of a method according to an embodiment of the present invention;
[0065] Figure 2 This is a flowchart illustrating the implementation of an embodiment of the present invention;
[0066] Figure 3 This is a flowchart of the re-detection process according to an embodiment of the present invention. Detailed Implementation
[0067] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.
[0068] As attached Figure 1 As shown, the long-term target tracking method based on generative adversarial networks in this embodiment includes:
[0069] (1) Using a deep neural network based on U-Net as the generator network, the background information of the first frame of the target video is removed, the foreground target is separated, the parameters of the discriminator network are adjusted, and the anti-generator network is trained. During the adversarial training process, the generator network and the discriminator network improve their respective performance by competing with each other.
[0070] (2) Analyze the position of the previous frame of the current frame through the generation strategy and generate multiple candidate regions. Use the convolutional neural network ResNet to extract the features of the candidate regions and input the features of each region into the discriminator network to calculate the probability that each candidate region is the target.
[0071] (3) When the number of tracking frames exceeds the preset threshold, image patches are randomly generated around the target location to enrich the sample set, and positive and negative samples are collected to retrain the generator network and discriminator network to obtain the updated target template.
[0072] (4) During the tracking process, continuously monitor the predicted probability value of the discriminator network. If the preset probability value is not reached within the preset number of consecutive frames, use the target detector to re-detect the target, update the parameters and status, and start a new tracking.
[0073] Specifically, in step (1), firstly, a generator deep neural network G based on U-Net is used to process the first frame image to remove background information and separate the foreground target. Next, the discriminator network D is refined and adjusted according to the samples to better distinguish the target from the background information. Finally, an adversarial generative network is trained. During adversarial training, the generator network G and the discriminator network D compete with each other to improve their respective performance.
[0074] (1.1) The generator network G is used to remove background information, extract samples from the first frame image, and fine-tune the discriminator network D online to distinguish between the target and the background. The generator network adopts a deep neural network based on U-Net, which consists of a symmetrical encoder (downsampling path) and decoder (upsampling path). The encoder is used to extract the features of the image and gradually reduce the spatial resolution; the decoder is responsible for restoring the feature mapping to the size of the original input image while retaining rich spatial information. The U-Net model is trained as a "background-foreground" segmenter, enabling it to segment the foreground target from the background in the input image. Then, by extracting the foreground part and superimposing it on a solid color background, the effect of removing the background from the image is achieved.
[0075] (1.2) Based on the first part of S1.1 The first frame of the input image for each sample via generator network The generated foreground targets are used to train the discriminator network adversarially. The parameters are fine-tuned, and the loss function for adversarial training is... It can be calculated using the following formula:
[0076]
[0077] Where N represents the number of samples, and Let represent the foreground target of the i-th sample and the first frame of the input image, respectively;
[0078] (1.3) Adversarial training is performed between the generator network and the discriminator network to minimize the reconstruction error of the generator network and maximize the discriminator network's discriminative ability. The loss function of the generator network is optimized. This includes the reconstruction error term and the adversarial training term:
[0079]
[0080] in, and It is a hyperparameter that balances the two loss terms. It is the reconstruction error, which is the difference between the image generated by the generator network G and the real image.
[0081] In step (2), to provide a more accurate search area, multiple candidate regions need to be generated by analyzing the position of the target in the previous frame using a custom generation strategy. These candidate regions should cover all possible locations where the target may move, while taking into account possible scale changes of the target and contextual information of the scene. Then, a convolutional neural network ResNet is used to extract the features of these candidate regions, and the feature vector of each region is input into the discriminator network to calculate the probability that each candidate region is the target. In this way, even if the appearance of the target changes significantly, or the target position in the previous frame is inaccurate, the accuracy and continuity of tracking can be maintained.
[0082] (2.1) Using the target location from the previous frame and the global image of the current frame as input, the target location is... , and The coordinates are the center point coordinates of the target. In the current frame image, a set of candidate region blocks is randomly generated around the target's position from the previous frame. The generation strategy needs to consider the target's motion pattern, calculating the scale factor based on the target's typical range of change and pose variation. Then, candidate regions are generated based on the scale factor and random offset, thus covering possible target locations. The candidate region calculation formula is as follows:
[0083]
[0084] Where Δx and Δy are random values drawn from a Gaussian probability distribution, representing the offset of the candidate region in the horizontal and vertical directions, respectively. is the scale factor.
[0085] The calculation of the scale factor based on the target change range and attitude change range specifically involves: firstly, based on the previous frame... Historical information calculation of target area in frames, target baseline scale factor The specific calculation formula is as follows:
[0086]
[0087] in, express The scale of the frame target is represented by the target area. ; Indicates the first Target width of the frame Indicates the first Target height of the frame, Let be a constant, here taken as 10;
[0088] Based on the fundamental scale factor of the target The scale factor is calculated from the attitude change of the ensemble target. The specific calculation formula is as follows:
[0089] ;
[0090] in This is the attitude correction factor, which is set to 0.3 here. The similarity of feature remainder terms can be calculated as follows:
[0091] ;
[0092] in and These are the feature vectors extracted from the current region and the reference template using ResNet in step S2, respectively. This represents the modulo operation of the eigenvector.
[0093] (2.2) Feature extraction is performed on each candidate region using a convolutional neural network (ResNet) to obtain a feature representation with rich semantic information. During feature extraction, ResNet not only captures the local features of the candidate region but also implicitly includes contextual information because the receptive field of the convolution operation covers the pixels surrounding the candidate region. For each candidate region, a feature vector set of dimension C is output. The formula is expressed as follows:
[0094]
[0095] in Let represent the feature vector of the i-th candidate region.
[0096] (2.3) The discriminator network is a classifier that accepts feature vectors. As input, and outputting a probability value representing the probability that the input feature vector corresponds to the target, the formula is as follows:
[0097]
[0098] Where D represents the discriminator network, This represents the probability that candidate region i contains the target. Based on the probability values output by the discriminator network, the candidate region with the highest probability value is selected as the final target location according to the sorted probability values. To prevent excessively low probabilities from being identified as targets, a confidence threshold is set. The discriminator's decision is only accepted when the highest probability exceeds this threshold. If the probability is below this threshold, the target location from the previous frame is maintained.
[0099] (3) To ensure tracking accuracy and robustness, a threshold of 200 tracking frames is set. When the number of tracking frames exceeds the threshold, a template update will be triggered, such as... Figure 2 As shown. This update process enriches the sample set by randomly generating image patches around the target location and collecting positive and negative samples for retraining the generator and discriminator networks.
[0100] (3.1) Set the threshold for the number of tracking frames to 200 frames. When the number of tracking frames reaches or exceeds this threshold, trigger the target template update mechanism (specifically, a historical reference value of the object to be tracked is used to compare with this historical reference value to determine whether the target in the current frame is the target to be tracked). Use the target position tracked in the last frame as the target position estimate for the current frame, and this position is usually the final target position selected in the candidate region evaluation stage (this position is the position confirmed by the discriminator).
[0101] (3.2) Generate a certain number of samples around the target location in the current frame, and randomly generate image patches around the target location to enrich the sample set, as shown in the following formula:
[0102] in, Representative sample, For image cropping operations, For the current frame image, This is an estimate of the target's location. This represents the offset relative to the estimated position.
[0103] The generated samples are labeled, with target regions marked as positive samples and background regions marked as negative samples, as follows:
[0104]
[0105] A labeled dataset is then created to retrain the generator and discriminator networks. During the update process, new positive samples can be introduced, which may come from new perspectives or appearances of the target. Introducing new positive samples helps the network better adapt to changes in the target's appearance, thereby improving tracking accuracy.
[0106] (3.3) Using the expanded labeled sample dataset from (3.2), train the generator network and the discriminator network. The training objective of the generator network is to minimize the difference between the discriminator network's output for generated samples and the output for real samples, i.e., minimize the output probability of the discriminator network for generated samples. Output probabilities of real samples The difference between them is expressed by the following formula:
[0107]
[0108] in, It is random noise. Let be the distribution of noise, i.e., the probability of random noise occurring. The goal of the discriminator network D is to distinguish between real and generated samples, that is, to accurately predict whether a sample is real. Therefore, the training objective of the discriminator network is to maximize the output probability of real samples. and minimizing the output probability of generated samples The formula is as follows:
[0109]
[0110] in, The probability of a real image appearing. The samples used are real samples. Subsequently, a generator network is used to generate updated target templates. The target templates represent the features and appearance of the target and can be updated through training to adapt to possible changes and poses of the target.
[0111] (4) such as Figure 3 As shown, during the tracking process, the predicted probability value of the D network needs to be continuously monitored. If the probability value is not reached within a set number of consecutive frames, a re-detection mechanism is triggered. The target detector is used to re-detect the target, update its parameters and status, and then start a new tracking process.
[0112] (4.1) Each time a target is tracked, the predicted probability values obtained by the D network need to be monitored. These probability values reflect the likelihood that the current candidate region contains the target. The number of times the predicted probability value of the D network is lower than a set threshold in consecutive frames is also recorded.
[0113] (4.2) If the predicted probability values obtained by the D network in consecutive frames are all lower than the set threshold, that is, the set number of consecutive failures is reached (e.g., 3 consecutive frames), then tracking failure is determined. Once tracking failure is confirmed, the tracking failure handling process is triggered immediately.
[0114] (4.3) After tracking fails, the target detector is started to re-detect the target in the current frame. After the target detector re-detects the target, it initializes the target to be tracked. This includes determining the initial position and size of the target and updating the relevant parameters and states in the tracking algorithm in order to start a new tracking process.
[0115] Based on the same inventive concept, this embodiment also provides a long-term target tracking system based on generative adversarial networks, including:
[0116] The initialization module is used to use a U-Net-based deep neural network as the generator network to remove the background information of the first frame of the target video, separate the foreground target, adjust the parameters of the discriminator network, and train an anti-generator network. During the adversarial training process, the generator network and the discriminator network improve their respective performance by competing with each other.
[0117] The extraction module is used to analyze the position of the previous frame of the current frame and generate multiple candidate regions through the generation strategy. The features of the candidate regions are extracted using a convolutional neural network ResNet, and the features of each region are input into the discriminator network to calculate the probability that each candidate region is the target.
[0118] The update module is used to enrich the sample set by randomly generating image patches around the target location when the number of tracking frames exceeds a preset threshold, and to collect positive and negative samples for retraining the generator network and discriminator network to obtain the updated target template.
[0119] The tracking module continuously monitors the predicted probability value of the discriminator network during the tracking process. If the preset probability value is not reached within a preset number of consecutive frames, the target detector is used to re-detect the target, update the parameters and status, and start a new tracking process.
[0120] Based on the same inventive concept, this embodiment also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the long-term target tracking method based on generative adversarial networks as described in any of the preceding claims.
[0121] Based on the same inventive concept, this embodiment also provides a computing device, including: one or more processors, one or more memories, and one or more programs, the programs being stored in the memory and configured to be executed by the processor, the programs being loaded onto the processor to implement the steps of the long-term target tracking method based on generative adversarial networks according to any of the preceding claims.
[0122] Based on the same inventive concept, this embodiment also provides a storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the steps of the long-term target tracking method based on generative adversarial networks as described above.
Claims
1. A long-term target tracking method based on generative adversarial networks, characterized in that, include: S1: Using a U-Net-based deep neural network as the generator network, the background information of the first frame of the target video is removed, the foreground target is separated, the parameters of the discriminator network are adjusted, and an anti-generator network is trained. During the adversarial training process, the generator network and the discriminator network improve their respective performance by competing with each other. S2: Analyze the position of the previous frame of the current frame using the generation strategy and generate multiple candidate regions. Use a convolutional neural network ResNet to extract the features of the candidate regions and input the features of each region into the discriminator network to calculate the probability that each candidate region is the target. S3: When the number of tracking frames exceeds a preset threshold, image patches are randomly generated around the target location to enrich the sample set, and positive and negative samples are collected to retrain the generator network and discriminator network to obtain the updated target template. S4: During the tracking process, continuously monitor the predicted probability value of the discriminator network. If the preset probability value is not reached within a preset number of consecutive frames, use the target detector to re-detect the target, update the parameters and status, and start a new tracking.
2. The long-term target tracking method based on generative adversarial networks according to claim 1, characterized in that, Step S1 includes: S1.1: A deep neural network based on the U-Net model is used as the generator network. The U-Net model is trained as a segmenter to segment the foreground object from the background in the first frame of the video to be processed, extract the foreground part and overlay it on a solid color background; S1.2: Based on the first part of S1.1 The first frame of the input image for each sample via generator network The generated foreground targets are used to train the discriminator network adversarially. The parameters are fine-tuned, and the loss function for adversarial training is... It can be calculated using the following formula: ； in, Represents the number of samples. and They represent the first The foreground target of each sample and the first frame of the input image; S1.3: Through generator network and discriminator network Adversarial training is performed to minimize the reconstruction error of the generator network and maximize the discriminator network's discriminative power, thereby optimizing the generator network's loss function. This includes the reconstruction error term and the adversarial training term: ； in, and It is a hyperparameter that balances the two loss terms. It is the reconstruction error term, used to represent the generator. The difference between the generated image and the real image.
3. The long-term target tracking method based on generative adversarial networks according to claim 1, characterized in that, Step S2 includes: S2.1: Using the target position from the previous frame and the global image of the current frame as input, let the target position be... , and Using the center point coordinates of the target, in the current frame image, a set of candidate region blocks is randomly generated around the target's position from the previous frame. The generation strategy considers the target's motion pattern, calculating a scale factor based on the target's range of change and pose variation. Then, candidate regions are generated based on the scale factor and random offset. The calculation formula is as follows: ； Where Δx and Δy are random values drawn from a Gaussian probability distribution, representing the horizontal and vertical offsets of the candidate region, respectively. Scale factor; S2.2: Use a convolutional neural network ResNet to extract features from each candidate region. For each candidate region, output a feature vector set of dimension C. The formula is expressed as follows: ； in, Indicates the first Feature vectors of candidate regions; S2.3: The discriminator network accepts feature vectors As input, it outputs a probability value representing the probability that the input feature vector corresponds to the target, as shown in the following formula: ； Where D represents the discriminator network, Indicates candidate region The probability of containing the target; The output probability values are sorted. When the highest probability value is greater than or equal to a preset threshold, the candidate region corresponding to the highest probability is taken as the final target position. Otherwise, the target position of the previous frame is maintained.
4. The long-term target tracking method based on generative adversarial networks according to claim 3, characterized in that, Step S2.1, which calculates the scale factor based on the target change range and the attitude change range, specifically involves: first, based on the previous frame... Historical information calculation of target area in frames, target baseline scale factor The specific calculation formula is as follows: ； in, express The scale of the frame target is represented by the target area. ; Indicates the first Target width of the frame Indicates the first Target height of the frame, It is a constant; Based on the fundamental scale factor of the target The scale factor is calculated from the attitude change of the ensemble target. The specific calculation formula is as follows: ； in This is the attitude correction coefficient. The similarity of feature remainder terms can be calculated as follows: ； in and These are the feature vectors extracted from the current region and the reference template using ResNet in step S2, respectively. This represents the modulo operation of the eigenvector.
5. The long-term target tracking method based on generative adversarial networks according to claim 1, characterized in that, Step S3 includes: S3.1: When the number of frames tracked reaches or exceeds a preset threshold, the target template update mechanism is triggered; the update mechanism specifically uses the target position tracked in the last frame as the target position estimate for the current frame, and this position is the position confirmed by the discriminator. S3.2: Generate several samples around the target location in the current frame. Specifically, randomly generate image patches around the target location to enrich the sample set, as shown in the following formula: ； in, Indicates sample , For image cropping operations, For the current frame image, This is an estimate of the target's location. This represents the offset relative to the estimated position; The generated samples are labeled, with target regions marked as positive samples and background regions marked as negative samples, as follows: ； Create a labeled dataset for retraining the generator and discriminator networks; S3.3: Using the expanded sample dataset from S3.2, train the generator network and the discriminator network to minimize the output probability of the discriminator network for the generated samples. Output probabilities of real samples The difference between them is expressed by the following formula: ； in, It is random noise. Let be the distribution of noise, representing the probability of random noise occurring; To maximize the output probability of the true sample and minimizing the output probability of generated samples The formula for training the discriminator network for the target is as follows: ； in, The probability of a real image appearing. The sample is a real sample; then a generator network is used to generate an updated target template.
6. The long-term target tracking method based on generative adversarial networks according to claim 1, characterized in that, Step S4 includes: S4.1: During the tracking process, continuously monitor the predicted probability value of the discriminator network and record the number of times the predicted probability value of the discriminator network is lower than the set threshold in consecutive frames; S4.2: If the predicted probability values obtained by the discriminator network in consecutive frames are all lower than the set threshold, then the tracking is deemed to have failed. S4.3: After tracking fails, the target detector is started to re-detect the target in the current frame; specifically, after the target detector re-detects the target, it initializes the tracking target, determines the initial position and size of the target, and updates the parameters and state in the tracking algorithm.
7. A long-term target tracking system based on generative adversarial networks, characterized in that, include: The initialization module is used to use a U-Net-based deep neural network as the generator network to remove the background information of the first frame of the target video, separate the foreground target, adjust the parameters of the discriminator network, and train an anti-generator network. During the adversarial training process, the generator network and the discriminator network improve their respective performance by competing with each other. The extraction module is used to analyze the position of the previous frame of the current frame and generate multiple candidate regions through the generation strategy. The features of the candidate regions are extracted using a convolutional neural network ResNet, and the features of each region are input into the discriminator network to calculate the probability that each candidate region is the target. The update module is used to enrich the sample set by randomly generating image patches around the target location when the number of tracking frames exceeds a preset threshold, and to collect positive and negative samples for retraining the generator network and discriminator network to obtain the updated target template. The tracking module continuously monitors the predicted probability value of the discriminator network during the tracking process. If the preset probability value is not reached within a preset number of consecutive frames, the target detector is used to re-detect the target, update the parameters and status, and start a new tracking process.
8. A computer program product comprising a computer program / instructions, characterized in that, When executed by a processor, the computer program / instructions implement the steps of the long-term target tracking method based on generative adversarial networks as described in any one of claims 1 to 6.
9. A computing device, characterized in that, include: One or more processors, one or more memories, and one or more programs, said programs being stored in the memory and configured to be executed by the processor, said programs, when loaded onto the processor, implementing the steps of the long-term target tracking method based on any one of claims 1 to 6.
10. A storage medium, characterized in that, The storage medium stores a computer program, which includes program instructions that, when executed by a processor, cause the processor to perform the steps of the long-term target tracking method based on generative adversarial networks according to any one of claims 1 to 6.