Target tracking method, system and device for Internet of Things resource limited terminal device, and medium

Abstract

The invention provides a target tracking method, system and device for an Internet of Things resource limited terminal device, and a medium. The method comprises the following steps: training a translation filter and a scale filter based on an adaptive feature dimension and an adaptive time scale interval to obtain a scale filter based on the adaptive time scale interval and a translation filter with an adaptive dimension feature; predicting a target position through a translation filter with adaptive dimension features and a histogram model; a final target size is then calculated using a scale filter with an adaptive time scale interval. According to the invention, the calculation amount is reduced, and the tracking efficiency is further improved.

Description

Technical Field

[0001] The present invention belongs to the field of Internet of Things terminal devices, and in particular relates to a target tracking method, system, device and medium for resource-constrained Internet of Things terminal devices. Background Art

[0002] Sixth-generation (6G) wireless communication technology has driven the rapid development of the Internet of Things (IoT), enabling the widespread deployment of various terminal devices in numerous applications, including wireless multimedia sensor networks. However, due to limited computing resources, most terminal devices struggle to process the massive amounts of IoT video data for visual object tracking.

[0003] Visual object tracking has widespread applications in IoT fields such as intelligent transportation, wildlife monitoring, and military surveillance. However, it faces challenges in scenarios involving partial occlusion, motion blur, varying illumination, cluttered backgrounds, and scale variations. The most popular visual tracking methods today are primarily based on discriminative correlation filters (DCFs) and deep learning-based trackers. While deep learning-based methods offer excellent performance, they rely on expensive GPUs, making them impractical for most end devices equipped with general-purpose CPUs. Furthermore, the stringent real-time requirements of some IoT applications for end devices exacerbate tracking challenges.

[0004] DCF trackers are suitable for resource-constrained terminal devices due to their high computational efficiency. However, most existing DCF trackers use multiple features and spatiotemporal scale spaces to estimate the target state. The fixed feature dimensions and dense scale intervals lead to high computational costs, which are not optimal when dealing with large-scale changes in the target. It is difficult to achieve a good balance between effectiveness and efficiency. Summary of the Invention

[0005] The purpose of the present invention is to provide a target tracking method for resource-constrained terminal devices of the Internet of Things, so as to solve the problems of high computational cost and low efficiency caused by fixed feature dimensions and dense scale intervals of existing DCF trackers on resource-constrained terminal devices, and to achieve efficient and accurate target tracking.

[0006] The technical solutions of the present invention are as follows:

[0007] A target tracking method for a resource-constrained terminal device of the Internet of Things, the method comprising:

[0008] Based on the adaptive feature dimension and the adaptive time scale interval, the translation filter and the scale filter are trained to obtain the scale filter based on the adaptive time scale interval and the translation filter with the adaptive dimension feature;

[0009] The target position is predicted by a translation filter with adaptive dimensionality features and a histogram model; then a scaling filter with adaptive time scale interval is used to calculate the final target size.

[0010] Furthermore, the translation filter is trained based on the adaptive feature dimension, specifically including:

[0011] Principal component analysis is used to perform feature dimensionality reduction. A projection matrix is ​​constructed by minimizing the reconstruction error, the features are projected into a low-dimensional subspace, the features are reconstructed by Fourier transform, and the features are compressed to train the translation filter.

[0012] Furthermore, the expression of the adaptive time scale interval is:

[0013]

[0014] in, represents the frequency of switching scale filter, is the original time domain scale interval.

[0015] Furthermore, the target position is predicted by the translation filter and histogram model with adaptive dimensionality features as follows:

[0016] A histogram model is constructed based on the color map, and the pixel-by-pixel score map of the target template is calculated. The score map is fused with the Hanning window through the adaptive mapping fusion parameter to generate the adaptive mapping feature.

[0017] Furthermore, the adaptive mapping feature formula is:

[0018] ψ'=ψ⊙{χ(H⊙W)+(1-χ)W},

[0019] Among them, χ is the adaptive mapping fusion parameter, ψ' represents the adaptive mapping feature after dimensionality reduction, H is the adaptive mapping obtained by calculating the pixel-by-pixel score of the target template based on the histogram model, W is the Hanning window, and ψ is the adaptive dimensionality feature.

[0020] A target tracking system for resource-constrained terminal devices in the Internet of Things, the system comprising:

[0021] A filter training module is configured to train a translation filter and a scale filter based on an adaptive feature dimension and an adaptive time scale interval to obtain a scale filter based on an adaptive time scale interval and a translation filter with an adaptive dimension feature;

[0022] The training and target prediction module is configured to predict the target location through a translation filter and a histogram model with adaptive dimensionality features; and then a scaling filter with adaptive time scale interval is used to calculate the final target size.

[0023] An electronic device includes a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the target tracking method described above.

[0024] A computer-readable storage medium stores a computer program, which implements the target tracking method as described above when executed by a processor.

[0025] Compared with the prior art, the present invention has the following advantages:

[0026] Improved tracking efficiency: Through dimensionality reduction processing of adaptive mapping feature responses, the feature dimension is greatly reduced, thereby reducing the amount of computation; adaptive time scale interval estimation reduces the frequency of scale estimation, further improving tracking efficiency.

[0027] Ensuring tracking accuracy: Adaptive mapping of feature responses cleverly combines dimensionality reduction and histogram score maps to generate the final response, significantly enhancing the robustness of state estimation. Adaptive time scale interval estimation effectively adapts to changes in target scale while reducing computational complexity, effectively ensuring tracking accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings illustrate various embodiments generally by way of example and not limitation, and together with the description and claims, serve to explain embodiments of the invention. Where appropriate, the same reference numerals are used throughout the drawings to refer to the same or similar parts. Such embodiments are illustrative and are not intended to be exhaustive or exclusive of the embodiments of the present apparatus or method.

[0029] Figure 1 A schematic flow chart of the method of the present invention is shown. DETAILED DESCRIPTION

[0030] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0031] An embodiment of the present invention provides a target tracking method for a resource-constrained terminal device of the Internet of Things, comprising:

[0032] Based on the adaptive feature dimension and the adaptive time scale interval, the translation filter and the scale filter are trained to obtain the scale filter based on the adaptive time scale interval and the translation filter with the adaptive dimension feature;

[0033] The target position is predicted by a translation filter with adaptive dimensionality features and a histogram model; then a scaling filter with adaptive time scale interval is used to calculate the final target size.

[0034] The present invention introduces a response interference suppression mechanism (RISTrack) to improve temporal consistency and suppress background interference. The modified objective is defined as the energy function:

[0035]

[0036] Where * is the correlation calculation and ⊙ is the element-wise multiplication. To minimize the predicted response map and Gaussian label y c The target-centered difference is obtained by learning the supervised correlation filter. The predicted response map is obtained by adding the c-th channel of the target appearance feature extracted from the current frame t and the filter g to be trained c Here, ω represents the penalty regularization weight. In addition, in order to effectively suppress background interference, a new part is introduced into the objective function. t-1 Represents the response graph of the previous frame, is a binary mask centered at the previous position (i, j) of the target, is the background enhancement feature extracted from the t-th frame, and P is the pixel-wise background weight map (where the weight value of the background area is higher than that of the target area).

[0037] By removing the subscript (·) c , decoupling the solution of (1) into C independent sub-problems. To facilitate efficient optimization, the objective function is reconstructed in the frequency domain by introducing auxiliary variables u and equality constraints. The resulting frequency domain formula is as follows:

[0038]

[0039] Where ^ represents discrete Fourier transform and ˉ represents complex conjugate. Ensure that the filters learned in the frequency domain are consistent with their spatial domain counterparts. To solve this constraint problem, the augmented Lagrangian method is used, and the following objective function is obtained:

[0040]

[0041] The penalty term and the dual variable λ are introduced to softly constrain the mandatory conditions. In the frequency domain, μ is a Lagrange multiplier, which is a penalty parameter that balances constraint strength and optimization stability. Subsequently, the ADMM algorithm is used to iteratively solve the three sub-problems. First, solve The closed-form solution is as follows:

[0042]

[0043] in, Represents training features With label The mutual correlation of and are the autocorrelations of the target feature and background enhancement feature, respectively. The penalty term μ is used to ensure convergence and numerical stability. Secondly, the auxiliary variable u is updated as follows:

[0044]

[0045] in represents the inverse discrete Fourier transform. In the i-th iteration, Update as follows:

[0046]

[0047] Here, the penalty parameter μ i By multiplying the previous value by β and using the maximum value μ max Limit to update, that is, μ i =min{μ max ,βμ i-1}, where μ max An upper limit is set, and β controls the growth rate. Subsequently, in order to cope with the target appearance changing over time, the model template is updated online by exponential moving average:

[0048]

[0049] Here, η∈[0,1] is the learning rate, which is used to control the weight distribution of new observation data and historical information. This update mechanism allows the model to retain the historical template. information, and gradually adapt to the new appearance features

[0050] Finally, to ensure that only the area near the center of the expected target receives high attention, a response auxiliary strategy is used to stabilize the response map. Its formula is as follows:

[0051]

[0052] Here, R t is the value of the previous peak value (i c ,j c ) is a spatial weighted mask centered on The modulated response map. The weight matrix is ​​defined as where δ<0 controls the steepness of the attenuation.

[0053] Thanks to spatiotemporal regularization, adaptive appearance modeling, and spatial response smoothing, the learned filters can become more robust and have good generalization capabilities. At the same time, many effective acceleration techniques are used to reduce the computational effort when solving the response interference suppression correlation filters to achieve real-time tracking. However, because the RISTrack framework uses multiple features and spatiotemporal scale spaces to address target state estimation, direct manipulation of fixed feature dimensions and dense scale intervals can be computationally expensive and suboptimal when dealing with large-scale target variations.

[0054] The present invention proposes an adaptive feature dimension and scale interval method based on histogram score map to reduce the feature dimension and scale estimation frequency when updating the target position and scale.

[0055] Adaptive mapping feature dimensions

[0056] Multi-feature integration is often used to improve tracking performance. However, this simple operation of simply connecting features is not suitable for improving tracking accuracy, but may lead to a decrease in tracking performance. In order to avoid this performance degradation in the multi-feature integration process, an adaptive mapping feature response based on dimensionality reduction and histogram-based score map is proposed to improve tracking performance. Specifically,

[0057] First, the optimal dimension selection of the integrated features is adaptively performed to maintain the real-time performance of RISTrack. Then, the selected dimension features are used to generate the final response using an adaptive color map based on the per-pixel score of the histogram to enhance the robustness of the state estimation. We apply three common features in the adaptive mapping feature response, including histogram of oriented gradients (HOG) and color name (CN) and color map (CM), where CM is applied to HOG+CN features. Therefore, the translation filter and histogram model are trained for the response map of different features, where represents the filter based on HOG+CN, and h represents the histogram model based on CM.

[0058] The dimensionality reduction technique is based on the standard principal component analysis (PCA) applied in the adaptive mapping feature response method. The updated model template of HOG+CN features is defined as u (t) =(1-η)u (t-1) +ηψ (t) , used to adaptively reduce feature dimensions. According to the linear properties of Fourier transform Formula (7) can be reconstructed as Finally, the updated model template u (t) Used to construct the projection matrix P (t) (i.e. low-dimensional subspace). Features can be obtained by The projection matrix P (t)Projected onto this subspace, is the dimension of the compressed feature. Projection matrix P (t) Update the model template u by minimizing the reconstruction (t) The objective function is obtained:

[0059]

[0060] Where m represents the traversal model template u (t) Then, by (t) The solution is obtained by performing eigenvalue decomposition on the autocorrelation matrix of :

[0061]

[0062] P (t) The row is set to C (t) The one with the largest eigenvalue feature vectors. The target block is compressed through the model template Updated to:

[0063]

[0064] P (t) The linear operation is defined as element-wise matrix multiplication (P (t) i (t) )(m)=P (t) u (t) (m), that is, the eigenvector Projection to P (t) In the detection phase, the model template The response graph is calculated as follows: Apply a filter based on HOG+CN features, the specific form is similar to formula (8):

[0065]

[0066] A Hanning window W is used to avoid boundary effects, and the color information adaptive mapping is applied to the adaptive dimension feature in a simple way to generate the final accurate response: ψ' = ψ⊙{χ(H⊙W)+(1-χ)W}, where χ is the adaptive mapping fusion parameter, ψ' represents the adaptive mapping feature after dimensionality reduction, H is the adaptive mapping obtained by calculating the pixel-by-pixel score of the target template based on the CM histogram model h, and ψ is the adaptive dimension feature.

[0067] Adaptive time-scale interval estimation

[0068] This method uses adaptive time scale interval (in represents the frequency of switching scale filter, The video frames for which the target size estimation is required are selected using the original temporal scale interval (representing the original temporal scale interval). The present invention always uses the original fixed scale step size in the spatial domain to ensure tracking stability. A scale filter is then applied to these selected video frames with the adaptive temporal scale interval to adapt to the target size. Even if the selected video frames with the adaptive scale interval in the temporal domain may not contain a suitable scale, given the generally small and smooth changes in target scale across all video frames, it is likely that a suitable scale can be determined using the numerous adaptive scale intervals of subsequently selected video frames.

[0069] The ARIST (Adaptive Response Interference Suppression Tracker) tracker is proposed by combining response interference suppression correlation filtering, adaptive mapping feature response based on dimensionality reduction and histogram score map, and adaptive time scale interval estimation. Its flow chart is as follows Figure 1 As shown in Figure 1. The entire tracking process is divided into two parts: filter training and target prediction. In the training and update phases, the translation filter based on adaptive dimensional features is trained by suppressing response interference, similar to the RISTrack tracker. In addition, a scale filter based on adaptive time scale intervals and a histogram model based on a color map are prepared in parallel. In the target prediction phase, the translation filter with adaptive dimensional features and the histogram model with adaptive mapping are first applied to predict the target position using formula (12). The scale filter with adaptive time scale intervals is then used to calculate the final target size.

[0070] The present invention proposes an adaptive mapping feature and scale scheme based on response interference suppression correlation filter to solve the problem of fixed feature dimension and sparse spatiotemporal scale sampling. This paper proposes adaptive mapping feature response and adaptive time scale estimation based on histogram score map dimensionality reduction, which adopts features with fewer dimensions and sparse time scale intervals to further improve the effectiveness and efficiency of target state estimation. Extensive tests on UAV123@10fps, DTB70, UAV112 and UAVDT datasets show that compared with baseline trackers with fixed features and sparse scale intervals, the proposed tracker provides higher performance in terms of computational efficiency and outperforms state-of-the-art trackers under real-time operation, which makes it suitable for resource-constrained terminal devices in various IoT applications.

[0071] The above is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field, within the technical scope disclosed by the present invention, can make equivalent replacements or changes based on the technical solutions and inventive concepts of the present invention, which should be covered by the scope of protection of the present invention.

Claims

1. A target tracking method for resource-constrained terminal devices in the Internet of Things, characterized in that: The method comprises: Based on the adaptive feature dimension and the adaptive time scale interval, the translation filter and the scale filter are trained to obtain the scale filter based on the adaptive time scale interval and the translation filter with the adaptive dimension feature; The target position is predicted by a translation filter with adaptive dimensionality features and a histogram model; then a scaling filter with adaptive time scale interval is used to calculate the final target size.

2. The target tracking method for resource-constrained terminal devices of the Internet of Things according to claim 1, characterized in that: Training translation filters based on adaptive feature dimensions includes: Principal component analysis is used to perform feature dimensionality reduction. A projection matrix is ​​constructed by minimizing the reconstruction error, the features are projected into a low-dimensional subspace, the features are reconstructed by Fourier transform, and the features are compressed to train the translation filter.

3. The target tracking method for resource-constrained terminal devices of the Internet of Things according to claim 1, characterized in that: The expression of the adaptive time scale interval is: in, represents the frequency of switching scale filter, is the original time domain scale interval.

4. The target tracking method for resource-constrained terminal devices of the Internet of Things according to claim 1, characterized in that: The target position is predicted by the translation filter and histogram model with adaptive dimensionality features as follows: A histogram model is constructed based on the color map, and the pixel-by-pixel score map of the target template is calculated. The score map is fused with the Hanning window through the adaptive mapping fusion parameter to generate the adaptive mapping feature.

5. The target tracking method for resource-constrained terminal devices of the Internet of Things according to claim 4, characterized in that: The adaptive mapping feature formula is: ψ'=ψ⊙{χ(H⊙W)+(1-χ)W}, Among them, χ is the adaptive mapping fusion parameter, ψ' represents the adaptive mapping feature after dimensionality reduction, H is the adaptive mapping obtained by calculating the pixel-by-pixel score of the target template based on the histogram model, W is the Hanning window, and ψ is the adaptive dimensionality feature.

6. A target tracking system for resource-constrained terminal devices in the Internet of Things, characterized in that: The system comprises: A filter training module is configured to train a translation filter and a scale filter based on an adaptive feature dimension and an adaptive time scale interval to obtain a scale filter based on an adaptive time scale interval and a translation filter with an adaptive dimension feature; The training and target prediction module is configured to predict the target location through a translation filter and a histogram model with adaptive dimensionality features; and then a scaling filter with adaptive time scale interval is used to calculate the final target size.

7. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the target tracking method according to claims 1-5.

8. A computer-readable storage medium, characterized in that The computer program is stored therein, and when the computer program is executed by a processor, the target tracking method as claimed in claims 1 to 5 is implemented.

Images