Single target tracking method based on deep learning layered video attention

By introducing deep learning hierarchical video attention mechanism and hierarchical inter-sliding window causal attention network model in single-objective tracking technology, combined with coordinate space perturbation and data enhancement technology, the problem of insufficient robustness of the existing technology in complex scenarios and occlusion situations is solved, and efficient and real-time single-objective tracking is achieved.

CN119919457AActive Publication Date: 2025-05-02NANJING UNIV OF POSTS & TELECOMM
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202510412123.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-04-02
Publication Date
2025-05-02
Estimated Expiration
2045-04-02

AI Technical Summary

Technical Problem

When existing single-target tracking technology faces complex scenarios, occlusion, appearance changes and balances real-time and accuracy, it is difficult to meet the needs of diversified applications.

Method used

The hierarchical video attention mechanism based on deep learning is adopted, and the causal attention network model of hierarchical inter-sliding window is trained, combined with coordinate space perturbation and data enhancement technology, an end-to-end training verification closed loop is built to achieve autoregressive target tracking.

Benefits of technology

While maintaining high tracking accuracy, it improves adaptability and real-timeness to complex video scenes, significantly improves anti-interference performance and long-range context reasoning capabilities, and improves training efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119919457A_ABST
    Figure CN119919457A_ABST
Patent Text Reader

Abstract

The invention relates to a single target tracking method based on deep learning layered video attention. The method comprises the steps of reading a data set to obtain original video data and target coordinates and target shielding proportions in each frame; screening out a reference frame by using the target shielding ratio, and constructing an initial video sequence and an initial coordinate truth value sequence which take the reference frame as a first frame; performing coordinate disturbance on the initial coordinate true value sequence, and cutting the initial video sequence by using the disturbed coordinates to obtain a cut video sequence; performing data enhancement on the clipping video sequence and the clipping coordinate truth value sequence to obtain an enhanced video sequence and an enhanced coordinate truth value sequence; training the hierarchical inter-frame sliding window causal attention network model by using the enhanced video sequence and the enhanced coordinate truth value sequence; and performing autoregressive target tracking by using the trained layered inter-frame sliding window causal attention network model. According to the method, the target can be efficiently and accurately tracked under the conditions of video shielding, blurring and jittering.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a single target tracking method based on deep learning hierarchical video attention, belonging to the technical field of computer vision and image processing. Background Art

[0002] With the continuous development of artificial intelligence and computer vision technology, single target tracking has a wide range of application value in security monitoring, unmanned driving, robot navigation, and human-computer interaction. Traditional single target tracking methods are mostly based on correlation filtering or template matching. By giving the target position in the initial frame and extracting its features, the target position is estimated by continuously searching for similar feature areas in subsequent frames. However, these methods face several challenges in practical applications. Occlusion and interference: When the target is partially or completely occluded, traditional methods based on template matching or correlation filtering usually rely on fixed update mechanisms or manually designed occlusion processing strategies, which makes it difficult to achieve accurate and robust tracking in complex scenes; Appearance changes: In real scenes, objects may change in scale, posture, and lighting. Traditional methods often require manual feature design or use models based on limited training data, which is difficult to cope with the diversity of object appearance; Balance between real-time and accuracy: Single target tracking tasks often require real-time processing with limited computing resources, and have high requirements on the efficiency of the algorithm; however, how to ensure the accuracy of tracking while improving the tracking speed has always been a difficult problem to balance.

[0003] In recent years, deep learning has demonstrated strong feature learning capabilities and robustness in computer vision tasks such as target detection, semantic segmentation, and video understanding. Introducing it into single target tracking can effectively improve the ability to adapt to complex scenes. Among them, tracking algorithms based on deep convolutional networks or Siamese structures have made significant progress, but when dealing with long sequence videos, strongly occluded scenes, and drastic changes in target appearance, the robustness and stability of existing technologies still need to be further improved. At the same time, with the successful application of new network structures such as Transformer in the field of images and videos, studying how to combine hierarchical attention mechanisms with single target tracking and use local and global attention to balance speed and accuracy has become a hot spot and difficulty in the current development of technology. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a single target tracking method based on deep learning hierarchical video attention, whose goal is to output the coordinates of the target in all subsequent frames by inputting the coordinates of the target in the video and the first frame. By flexibly using the hierarchical attention mechanism in the network structure, while maintaining high tracking accuracy, it takes into account real-time and adaptability to complex video scenes, thereby meeting diverse application needs.

[0005] In order to achieve the above object, the technical solution of the present invention is as follows: a single target tracking method based on deep learning hierarchical video attention, the method comprising the following steps: Step 1: Read the data set to obtain the original video data and the target coordinates and target occlusion ratio in each frame. Step 2: Use the target occlusion ratio to select the reference frame, and construct the initial video sequence and initial coordinate true value sequence with the reference frame as the first frame. Step 3: Perturb the coordinates of the initial coordinate true value sequence, use the perturbed coordinates to crop the initial video sequence to obtain a cropped video sequence, and use the perturbed coordinates to transform the initial coordinate true value sequence to obtain a cropped coordinate true value sequence. Step 4: Perform data enhancement on the cropped video sequence and the cropped coordinate true value sequence to obtain an enhanced video sequence and an enhanced coordinate true value sequence. Step 5: Use the enhanced video sequence and the enhanced coordinate truth sequence to train the hierarchical inter-frame sliding window causal attention network model. Step 6: Use the trained hierarchical inter-frame sliding window causal attention network model to perform autoregressive target tracking.

[0006] In step 1, the video sampling frequency of the data set is set to 10 frames per second.

[0007] Among them, in step 2, when the occlusion ratio is greater than 50% and there is at least frame, it is determined to be a valid frame. is a natural number greater than or equal to 48. When a valid frame is randomly selected from all valid frames as the reference frame, sampling optimization is performed based on the variance of the occlusion ratio within the interval. The details are as follows: Set the window length to Frame, traverse the occlusion ratio of all frames, calculate the variance in the corresponding window, filter out the variance calculation weights at the valid frame position, and normalize the weights to form a probability distribution: , in, Indicates The probability that a valid start frame is selected is Indicates The variance of the occlusion ratio of the window corresponding to the valid starting frame, Indicates The variance of the occlusion ratio of the window corresponding to the valid starting frame; Calculate the probability distribution and use random weighted sampling to select the final reference frame; The initial video sequence is defined as ,in, Indicates the first frame starting from the base frame. Frame, initial coordinate true value sequence ,in, Indicates The initial coordinate true value of the frame.

[0008] In step 3, the coordinate perturbation uses dynamic noise injection technology, and the initial coordinate true value sequence Middle The initial coordinate value of the frame It is expressed as: , in, For the The horizontal coordinate of the top left corner of the frame. For the The vertical coordinate of the upper left corner of the frame. For the Frame width, For the Frame height, after perturbation The initial coordinate value of the frame Calculated as: , , , , , in, For the The horizontal coordinate of the perturbation of the vertex in the upper left corner of the frame, For the The vertical coordinate of the upper left corner vertex of the frame is disturbed. For the Frame perturbation width, For the Frame perturbation height, represents a natural constant, represents the Gaussian noise sample value with variance 1, Indicates a uniform noise sampling value ranging from 0 to 1; The search area is cropped using an adaptive zoom strategy, with the perturbed coordinates as the center. Frame cropping side length Calculated as: in, The cropping factor to control the cropping scale; Generate cropped video sequence based on cropping edge length and perturbation coordinates , whose expression is: , Among them, each element , By generating , , Represents the image Execute The center point is the center coordinate and the side length is Square cropping operation of ; Clipping coordinate true value sequence The cropped video sequence is generated by the following processing When the initial coordinate true value sequence Perform equivalent mapping processing to finally form a true value sequence of clipping coordinates.

[0009] This method proposes an innovative technical improvement plan to address the inconsistency problem between training and reasoning scenarios in existing target tracking technologies. Different from the traditional single-frame input mode, this method constructs a video sequence as a model input sample for the first time, and achieves consistency optimization between training and reasoning through the following key technical breakthroughs: First, the coordinate space perturbation strategy is innovatively introduced to effectively simulate the predicted coordinate error generated in the actual reasoning process by applying random displacement deviations in the training phase; second, the image preprocessing process in the reasoning phase is accurately reproduced through the search region cropping module to build an end-to-end training and verification closed loop. This dual technical solution successfully bridges the feature distribution differences between the model training environment and the deployment environment, enabling the model to have the ability to train multiple frames in parallel, breaking through the limitations of the traditional autoregressive training model in terms of temporal dependence, and compared with the frame-by-frame processing method, the training efficiency is improved by more than 300%, providing a new model optimization paradigm for the field of video target tracking.

[0010] Among them, in step 4: enhance the video sequence Crop video sequence by pair All frames are synchronously transformed by applying the same rotation angle, flip direction, Gaussian blur and color channel replacement space transformation; Enhanced coordinate true value sequence The true value sequence of the clipping coordinates After cropping the video sequence Transformed into enhanced video sequence The equivalent coordinates are mapped to get .

[0011] Among them, in step 5, the hierarchical feature extraction network adopts a four-level progressive architecture. The first two levels use the local intra-frame window attention mechanism to extract fine-grained features, and the last two levels use the global inter-frame sliding window causal attention to achieve cross-frame information fusion; Among them, the specific calculation steps of the global inter-frame sliding window causal attention module are: First, perform feature combination. Characteristics of frame input , calculated as , in, Indicates The characteristics of the frame image, Indicates The characteristics of the frame coordinates, It means concatenating the vectors along the first dimension, and then inputting the total feature sequence Calculated as: , Inter-frame sliding window causal attention constraint matrix The calculation process is as follows: , , in, Indicates Frame input The feature pair Frame input The attention of features is, Indicates the total number of blocks of features, is less than or equal to A natural constant greater than 0, controlling the Frame features can only focus on the previous frame including itself. Frame features; Finally, enter the total feature sequence , the inter-frame sliding window causal attention constraint matrix , the attention module output features Calculated as: , in, represents the transpose of a matrix, , and are all learnable parameters of the network model. Indicates the length of each feature block; Rotational position encoding is used between frames, and fixed position encoding is used within frames; the prediction confidence uses the intersection of the predicted coordinates and the predicted true value as the true value to indicate the model's confidence in the current predicted coordinates, which is used to correct the tracking state during inference; the loss function is designed to fuse the coordinate regression error and the prediction confidence error, and a weighted summation is used to balance the contribution of different loss items.

[0012] This method proposes a breakthrough time-perception optimization solution to address the weak frame-level context modeling capabilities that are common in existing target tracking technologies. Compared with the twin tracking architecture that relies on fixed template matching or the recursive model that is limited by memory capacity, this method creatively constructs a global temporal attention network and realizes the dynamic correlation modeling of historical features in the entire video domain for the first time: through a differentiable attention weight mechanism, an explicit feature interaction channel between the current frame and any historical frame is established, breaking through the dual limitations of traditional methods in temporal perception range and modeling depth, and enabling the model to have true long-range context reasoning capabilities.

[0013] Among them, in step 6, the reasoning stage adopts an incremental processing flow, and retains the most recent The characteristic state of the frame, in the dynamic adjustment algorithm of the search area, the search area is initialized with the coordinates of the first frame. In the subsequent prediction, the search area is updated by sliding average according to the intersection and union ratio of the predicted coordinates of two adjacent frames as the weight. When the confidence is lower than the threshold, the search area is expanded.

[0014] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the single target tracking method based on deep learning hierarchical video attention is implemented.

[0015] A computer-readable storage medium stores computer instructions, which, when executed by a processor, implement the single target tracking method based on deep learning hierarchical video attention.

[0016] Compared with the prior art, the present invention adopts the above technical solution and has the following beneficial effects: 1. The present invention realizes real-time online tracking based on the hierarchical video attention method. Even if an object has never been seen before, it can be continuously tracked in subsequent frames as long as it is marked in the first frame. 2. The present invention selects more complex, drastically changing and challenging video segments for model training through the target occlusion ratio screening method, which significantly improves the anti-interference performance of the model in the video stream. 3. The method based on layered video attention in the present invention can deal with occlusion well. Compared with the tracking method based on template matching, it avoids the complex template update mechanism, but can pay attention to the information of the long-ago frame through the attention mechanism, so that the model can still achieve robust tracking based on the information of the previous frame when the target is partially or completely occluded. 4. The hierarchical video attention-based method of the present invention uses sliding window attention and autoregressive tracking, which can train the entire video data in parallel during training and efficiently utilize hardware efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is the overall flow chart of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to a program flow chart and specific examples.

[0019] Embodiment: A single target tracking method based on deep learning hierarchical video attention, the method comprising the following steps: Step 1: Read the data set to obtain the original video data and the target coordinates and target occlusion ratio in each frame. Step 2: Use the target occlusion ratio to select the reference frame, and construct the initial video sequence and initial coordinate true value sequence with the reference frame as the first frame. Step 3: Perturb the coordinates of the initial coordinate true value sequence, use the perturbed coordinates to crop the initial video sequence to obtain a cropped video sequence, and use the perturbed coordinates to transform the initial coordinate true value sequence to obtain a cropped coordinate true value sequence. Step 4: Perform data enhancement on the cropped video sequence and the cropped coordinate true value sequence to obtain an enhanced video sequence and an enhanced coordinate true value sequence. Step 5: Use the enhanced video sequence and the enhanced coordinate truth sequence to train the hierarchical inter-frame sliding window causal attention network model. Step 6: Use the trained hierarchical inter-frame sliding window causal attention network model to perform autoregressive target tracking.

[0020] In step 1, the video sampling frequency of the data set is set to 10 frames per second.

[0021] Among them, in step 2, when the occlusion ratio is greater than 50% and there is at least frame, it is determined to be a valid frame. is a natural number greater than or equal to 48. When a valid frame is randomly selected from all valid frames as the reference frame, sampling optimization is performed based on the variance of the occlusion ratio within the interval. The details are as follows: Set the window length to Frame, traverse the occlusion ratio of all frames, calculate the variance in the corresponding window, filter out the variance calculation weights at the valid frame position, and normalize the weights to form a probability distribution: , in, Indicates The probability that a valid start frame is selected is Indicates The variance of the occlusion ratio of the window corresponding to the valid starting frame, Indicates The variance of the occlusion ratio of the window corresponding to the valid starting frame; Calculate the probability distribution and use random weighted sampling to select the final reference frame; The initial video sequence is defined as ,in, Indicates the first frame starting from the base frame. Frame, initial coordinate true value sequence ,in, Indicates The initial coordinate true value of the frame.

[0022] Among them, in step 3: the coordinate perturbation adopts the dynamic noise injection technology, and the initial coordinate true value sequence Middle The initial coordinate value of the frame It is expressed as: , in, For the The horizontal coordinate of the top left corner of the frame. For the The vertical coordinate of the upper left corner of the frame. For the Frame width, For the Frame height, after perturbation The initial coordinate value of the frame Calculated as: , , , , , in, For the The horizontal coordinate of the perturbation of the vertex in the upper left corner of the frame, For the The vertical coordinate of the upper left corner vertex of the frame is disturbed. For the Frame perturbation width, For the Frame perturbation height, represents a natural constant, represents the Gaussian noise sample value with variance 1, Indicates a uniform noise sampling value ranging from 0 to 1; The search area is cropped using an adaptive zoom strategy, with the perturbed coordinates as the center. Frame cropping edge length Calculated as: , in, The cropping factor to control the cropping scale; Generate cropped video sequence based on cropping edge length and perturbation coordinates , whose expression is: , Among them, each element , By generating , , Represents the image Execute The center point is the center coordinate and the side length is Square cropping operation of ; Clipping coordinate true value sequence The cropped video sequence is generated by the following processing When the initial coordinate true value sequence Perform equivalent mapping processing and finally form a true value sequence of clipping coordinates.

[0023] This method proposes an innovative technical improvement plan to address the inconsistency problem between training and reasoning scenarios in existing target tracking technologies. Different from the traditional single-frame input mode, this method constructs a video sequence as a model input sample for the first time, and achieves consistency optimization between training and reasoning through the following key technical breakthroughs: First, the coordinate space perturbation strategy is innovatively introduced to effectively simulate the predicted coordinate error generated in the actual reasoning process by applying random displacement deviations in the training phase; second, the image preprocessing process in the reasoning phase is accurately reproduced through the search area cropping module to build an end-to-end training and verification closed loop. This dual technical solution successfully bridges the feature distribution differences between the model training environment and the deployment environment, enabling the model to have multi-frame parallel training capabilities, breaking through the limitations of the traditional autoregressive training model in terms of temporal dependence, and compared with the frame-by-frame processing method, the training efficiency is improved by more than 300%, providing a new model optimization paradigm for the field of video target tracking.

[0024] Among them, in step 4: enhance the video sequence Crop video sequence by pair All frames are synchronously transformed by applying the same rotation angle, flip direction, Gaussian blur and color channel replacement space transformation; Enhanced coordinate truth sequence The true value sequence of the clipping coordinates After cropping the video sequence Transformed into enhanced video sequence The equivalent coordinates are mapped to get .

[0025] Among them, in step 5, the hierarchical feature extraction network adopts a four-level progressive architecture. The first two levels use the local intra-frame window attention mechanism to extract fine-grained features, and the last two levels use the global inter-frame sliding window causal attention to achieve cross-frame information fusion; Among them, the specific calculation steps of the global inter-frame sliding window causal attention module are: First, perform feature combination. Characteristics of frame input , calculated as , in, Indicates The characteristics of the frame image, Indicates The characteristics of the frame coordinates, Indicates concatenating vectors along the first dimension; Then, enter the total feature sequence Calculated as: , Inter-frame sliding window causal attention constraint matrix The calculation process is as follows: , , in, Indicates Frame input The feature pair Frame input The attention of features is, Indicates the total number of blocks of features, is less than or equal to A natural constant greater than 0, controlling the Frame features can only focus on the previous frame including itself. Frame features; Finally, enter the total feature sequence , the inter-frame sliding window causal attention constraint matrix , the attention module output features Calculated as: , in, represents the transpose of a matrix, , and are all learnable parameters of the network model. Indicates the length of each feature block; Rotational position encoding is used between frames, and fixed position encoding is used within frames; the prediction confidence uses the intersection of the predicted coordinates and the predicted true value as the true value to indicate the model's confidence in the current predicted coordinates, which is used to correct the tracking state during inference; the loss function is designed to fuse the coordinate regression error and the prediction confidence error, and a weighted summation is used to balance the contribution of different loss items.

[0026] Among them, in step 6, the reasoning stage adopts an incremental processing flow, and retains the most recent The characteristic state of the frame, in the dynamic adjustment algorithm of the search area, the search area is initialized with the coordinates of the first frame. In the subsequent prediction, the search area is updated by sliding average according to the intersection and union ratio of the predicted coordinates of two adjacent frames as the weight. When the confidence is lower than the threshold, the search area is expanded.

[0027] The technical solution provided by the present invention is further described below through a specific embodiment. Our solution uses the Ubuntu 22.04.2LTS operating system, the PyTorch deep learning development framework, and Python as the development language. The CPU used in the experiment is Intel Core i7-12800k, and the GPU is NVIDIA GeForce RTX 3090Ti 24G. During the training process, the algorithm runs in the Python 3.11 environment, and the data set parameters are as follows: .

[0028] The above description is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a technician familiar with the technical field within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention.

Claims

1. A single target tracking method based on deep learning hierarchical video attention, characterized in that: The method comprises the following steps: Step 1: Read the data set to obtain the original video data and the target coordinates and target occlusion ratio in each frame. Step 2: Use the target occlusion ratio to select the reference frame, and construct the initial video sequence and initial coordinate true value sequence with the reference frame as the first frame. Step 3: Perturb the coordinates of the initial coordinate true value sequence, use the perturbed coordinates to crop the initial video sequence to obtain a cropped video sequence, and use the perturbed coordinates to transform the initial coordinate true value sequence to obtain a cropped coordinate true value sequence. Step 4: Perform data enhancement on the cropped video sequence and the cropped coordinate true value sequence to obtain an enhanced video sequence and an enhanced coordinate true value sequence. Step 5: Use the enhanced video sequence and the enhanced coordinate truth sequence to train the hierarchical inter-frame sliding window causal attention network model. Step 6: Use the trained hierarchical inter-frame sliding window causal attention network model to perform autoregressive target tracking.

2. The single target tracking method based on deep learning hierarchical video attention according to claim 1 is characterized in that: In step 1, the video sampling frequency of the dataset is set to 10 frames per second.

3. The single target tracking method based on deep learning hierarchical video attention according to claim 1 is characterized in that: In step 2, when the occlusion ratio is greater than 50% and there is at least frame, it is determined to be a valid frame. is a natural number greater than or equal to 48. When a valid frame is randomly selected from the valid frames as the reference frame, sampling optimization is performed based on the variance of the occlusion ratio within the interval. The details are as follows: Set the window length to Frame, traverse the occlusion ratio of all frames, calculate the variance in the corresponding window, filter out the variance calculation weights at the valid frame position, and normalize the weights to form a probability distribution. , in, Indicates The probability that a valid start frame is selected is Indicates The variance of the occlusion ratio of the window corresponding to the valid starting frame, Indicates The variance of the occlusion ratio of the window corresponding to the valid starting frame; Calculate the probability distribution and use random weighted sampling to select the final reference frame. The initial video sequence is defined as ,in, Indicates the first frame starting from the base frame. Frame, initial coordinate true value sequence ,in, Indicates The initial coordinate true value of the frame.

4. The single target tracking method based on deep learning hierarchical video attention according to claim 3 is characterized in that: In step 3, the coordinate perturbation uses dynamic noise injection technology, and the initial coordinate true value sequence Middle The initial coordinate value of the frame It is expressed as: , in, For the The horizontal coordinate of the top left corner of the frame. For the The vertical coordinate of the upper left corner of the frame. For the Frame width, For the Frame height, after perturbation The initial coordinate value of the frame Calculated as: , , , , , in, For the The horizontal coordinate of the perturbation of the vertex in the upper left corner of the frame, For the The vertical coordinate of the upper left corner vertex of the frame is disturbed. For the Frame perturbation width, For the Frame perturbation height, represents a natural constant, represents the Gaussian noise sample value with variance 1, Indicates a uniform noise sampling value ranging from 0 to 1; The search area is cropped using an adaptive zoom strategy, with the perturbed coordinates as the center. Frame cropping side length Calculated as: , in, The cropping factor to control the cropping scale; Generate cropped video sequence based on cropping edge length and perturbation coordinates , whose expression is: , Among them, each element , By generating , , Represents the image Execute The center point is the center coordinate and the side length is Square cropping operation of ; Clipping coordinate true value sequence The cropped video sequence is generated by the following processing When the initial coordinate true value sequence Perform equivalent mapping processing to finally form a true value sequence of clipping coordinates.

5. The single target tracking method based on deep learning hierarchical video attention according to claim 4 is characterized in that: In step 4: Enhanced video sequences Crop video sequence by pair All frames are synchronously transformed by applying the same rotation angle, flip direction, Gaussian blur and color channel replacement space transformation; Enhanced coordinate truth sequence The true value sequence of the clipping coordinates After cropping the video sequence Transformed into enhanced video sequence The equivalent coordinates are mapped to .

6. The single target tracking method based on deep learning hierarchical video attention according to claim 5 is characterized in that: In step 5, the hierarchical feature extraction network adopts a four-level progressive architecture. The first two levels use the local intra-frame window attention mechanism for fine-grained feature extraction, and the last two levels use the global inter-frame sliding window causal attention to achieve cross-frame information fusion; Among them, the specific calculation steps of the global inter-frame sliding window causal attention module are as follows: First, perform feature combination. Characteristics of frame input , calculated as , in, Indicates The characteristics of the frame image, Indicates The characteristics of the frame coordinates, Indicates concatenating the vectors along the first dimension; then, input the total feature sequence Calculated as: , Inter-frame sliding window causal attention constraint matrix The calculation process is as follows: , , in, Indicates Frame input The feature pair Frame input The attention of features is, Indicates the total number of blocks of features, is less than or equal to A natural constant greater than 0, Finally, enter the total feature sequence , the inter-frame sliding window causal attention constraint matrix , the attention module output features Calculated as: , in, represents the transpose of a matrix, , and are all learnable parameters of the network model. Indicates the length of each feature block; Rotational position encoding is used between frames, and fixed position encoding is used within frames; the prediction confidence uses the intersection of the predicted coordinates and the predicted true value as the true value to indicate the model's confidence in the current predicted coordinates, which is used to correct the tracking state during inference; the loss function is designed to fuse the coordinate regression error and the prediction confidence error, and a weighted summation is used to balance the contribution of different loss items.

7. The single target tracking method based on deep learning hierarchical video attention according to claim 6 is characterized in that: In step 6, the inference phase uses an incremental processing flow to retain the most recent The characteristic state of the frame, in the dynamic adjustment algorithm of the search area, the search area is initialized with the coordinates of the first frame. In the subsequent prediction, the search area is updated by sliding average according to the intersection and union ratio of the predicted coordinates of two adjacent frames as the weight. When the confidence is lower than the threshold, the search area is expanded.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the single target tracking method based on deep learning hierarchical video attention as described in any one of claims 1 to 7 above.

9. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instruction is executed by the processor, the single target tracking method based on deep learning hierarchical video attention is implemented as described in any one of claims 1-7.

Citation Information

Patent Citations

  • Scene text recognition method and device

    CN114155527A

  • Multi-template fusion target tracking algorithm based on graph attention network

    CN115409868A

  • Visible light-infrared sequence target tracking method based on residual learning

    CN115690164A

  • Chemical industrial park illegal behavior intervention and tracing system based on edge calculation

    CN119339334A

  • Generating global hierarchical self-attention

    US20240185034A1