Filter channel selection method and device based on DCF and medium
By introducing a DCF-based filter channel selection method in the target tracking algorithm, using FR scores and weight calculations to optimize channel selection, the redundancy and interference problems in high-dimensional features and large-scale channel information processing are solved, and the tracking performance and calculation are improved efficiency.
Patent Information
- Application Number
- CN202411561935.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-04
- Publication Date
- 2025-02-18
AI Technical Summary
When existing target tracking algorithms process high-dimensional features and large amounts of channel information, it is difficult to effectively filter out effective information and suppress redundant or interference information, resulting in low computing efficiency and unstable tracking performance.
A filter channel selection method based on DCF is proposed. By calculating the FR score of each channel corresponding to each frame of image, selecting the channel whose FR score is greater than the minimum score threshold, and calculating the weight of each channel based on the FR score to optimize channel selection and characteristics extract.
Improve the accuracy of feature channel selection during target tracking, enhance the robustness and accuracy of tracking, and reduce computing costs and memory requirements.
Smart Images
Figure CN119477982A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of software development, and in particular to a DCF-based filter channel selection method, device and medium. Background Art
[0002] Object tracking technology aims to estimate the position and size of the object in subsequent frames based on its position, size and appearance features in the initial frame. Currently, mainstream object tracking technologies are divided into two categories: generative tracking models and discriminative tracking models.
[0003] The generative model tracking algorithm generates a large number of random candidate targets based on the geometric state of the target box in the previous frames, and selects the candidate target with the smallest reconstruction error with the target template as the tracking result. Common generative models include tracking methods based on mean shift, tracking methods based on particle filtering, optical flow algorithms, etc.
[0004] The discriminative model tracking algorithm distinguishes between the target and the background by training a classifier, and takes the candidate sample with the highest response as the tracking result. Among them, the Discriminative Correlation Filters (DCF) method collects samples through a circulant matrix and uses Fast Fourier Transform (FFT) to convert the time domain correlation operation into frequency domain point-by-point multiplication, thereby reducing the amount of calculation and improving the algorithm execution speed.
[0005] At present, the feature representation in the Discriminative Correlation Filter (DCF) framework based on deep neural networks has shown excellent performance and robustness in a variety of visual applications compared to traditional hand-designed features. In addition to the development direction of enhancing discriminative correlation filters based on deep neural networks, end-to-end learning of target classification and regression tasks through neural networks has also significantly improved the accuracy of feature extraction and effectively improved the robustness of target tracking algorithms.
[0006] Nevertheless, there is still redundant information in these high-dimensional features and their large number of channels, which has prompted researchers to propose a series of methods to use this information more effectively. For example, some trackers use principal component analysis (PCA) to reduce the dimension of deep features to reduce computational and memory costs. In addition, there are algorithms that introduce attention mechanisms and dynamic weight allocation strategies to optimize tracking efficiency. However, the number of feature channels used by these algorithms is still quite large, and not only does the computational efficiency need to be improved, but more importantly, it is impossible to filter out valid information from redundant information and suppress useless or even disruptive information.
[0007] Therefore, in recent years, some researchers have introduced saliency detection technology into the correlation filter tracking framework, developed a number of advanced algorithms, and made significant progress. For example, discriminative tracking distinguishes visual scenes into salient targets and non-salient backgrounds through top-down saliency mechanism regulation. Based on this concept, many studies have incorporated saliency awareness into DCF trackers. However, most of these methods are limited to using image saliency information to construct spatial or temporal regularization terms to alleviate boundary effect problems, or to handle target appearance changes through reinforcement learning, but ignore the potential correlation between multi-channel features and target saliency information. Studies have shown that although different channels in multi-channel features have their own contributions, many of them may contain information that is irrelevant to the target and redundant. Direct integration of these dimensions may lead to severe overfitting, thereby affecting the stability of tracking performance. Summary of the invention
[0008] In order to solve the above problems, the present invention proposes a filter channel selection method, device and medium based on DCF.
[0009] The specific plan is as follows:
[0010] A filter channel selection method based on DCF, when tracking a target through DCF, selects the channel selected for each frame and calculates the weight of the corresponding channel through the following steps;
[0011] S1: Calculate the FR score of each channel corresponding to the current frame image:
[0012]
[0013] in, represents the FA value of the cth channel of the tth frame; Indicates that the c-th channel feature of the t-th frame corresponds to the average energy of the target perception area of the image; Indicates that the c-th channel feature of the t-th frame corresponds to the average energy of the background area of the image;
[0014] S2: Based on the set minimum score threshold, all channels with FR scores greater than the minimum score threshold are selected as the channels selected for the current frame;
[0015] S3: Calculate the weight of each channel in the selected channels based on the FR score of each channel.
[0016] Furthermore, the c-th channel feature of the t-th frame corresponds to the average energy of the target perception area of the image The calculation formula is:
[0017]
[0018] in, A represents the feature value of the pixel (i, j) in the cth channel of the tth frame; O represents the area of the target perception area of the image; O represents the target perception area of the image; (i, j) represents the coordinates of the pixel point in the image.
[0019] Furthermore, the cth channel feature of the tth frame corresponds to the average energy of the background area of the image The calculation formula is:
[0020]
[0021] Among them, A S represents the area of the target search area of the image; S represents the target search area of the image.
[0022] Furthermore, the calculation formula for the weight of each channel is:
[0023]
[0024] in, represents the weight of the cth channel corresponding to the tth frame; max(FR t ) represents the maximum value of the FR scores of all channels selected in the tth frame; min(FR t ) represents the minimum FR score of all channels selected in the tth frame.
[0025] A filter channel selection terminal device based on DCF comprises a processor, a memory and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method described above in the embodiment of the present invention are implemented.
[0026] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described above in an embodiment of the present invention are implemented.
[0027] The present invention adopts the above technical solution, which effectively improves the accuracy of feature channel selection during target tracking and enhances the robustness and accuracy of tracking. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 Shown is a flow chart of a method according to a first embodiment of the present invention. DETAILED DESCRIPTION
[0029] To further illustrate various embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention, which are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in this field should be able to understand other possible implementations and advantages of the present invention.
[0030] The present invention will now be further described with reference to the accompanying drawings and specific implementation methods.
[0031] Embodiment 1:
[0032] Localizing the target in consecutive video frames is the core task of visual target tracking, and the method based on correlation filtering aims to determine the initial predicted position of the target in the video. Assuming that the predicted position of the tracked object has been determined in the tth frame, in order to predict the position of the target in the next frame, that is, to locate the tracked target in the (t+1)th frame, DCF is trained by filtering It is implemented by defining a C-dimensional channel feature with a width of W and a height of H. Then, the training sample X of the tth frame is used t , Y is the expected response mapping of the corresponding Gaussian shape. In order to obtain a multi-channel correlation filter, DCF expresses the target as a regularized least squares problem:
[0033]
[0034] Where * is the circular convolution operator, is the regularization term, is the corresponding discriminant correlation filter, represents the cth channel feature. Then the least squares problem can be transformed into the Fourier domain, and the closed solution of equation (1) can be derived. Through Fourier transform and complex conjugate operation, the filter can be obtained The solution is as follows:
[0035]
[0036] Wherein, the superscript ^ denotes discrete Fourier transform, ⊙ denotes dot product, the right superscript * denotes the complex conjugate operator, and λ is the regularization parameter.
[0037] In the next frame, the feature vector extracted Calculate the response graph This process is achieved through inverse Fourier transform, which multiplies the features of all channels by the filter in the Fourier domain, then sums them, and finally performs inverse Fourier transform. The formula for obtaining the response graph R is as follows:
[0038]
[0039] in represents the inverse Fourier transform, the target position in the (t+1) frame is determined by the peak position in the response graph R, and real represents the real part operator.
[0040] The success of deep learning in the field of target tracking is mainly attributed to the ability of neural networks to extract more superior and sophisticated deep features. However, even though there are many methods that use image saliency information to construct spatial or temporal regularization terms to alleviate boundary effect problems, or process target appearance changes through reinforcement learning, the limited number of visual tracking training samples has led to the need to ignore the potential association between certain multi-channel features and target saliency information. If a deep network trained by a target is used to extract multi-channel features of a specific target, a large number of interference channels will be included. Therefore, when the DCF tracker extracts search area features based on the target position and generates a response map, it should pay more attention to the energy of the target area feature channel. This embodiment will allocate target channel attention information and select channel features based on saliency in the feature space.
[0041] In order to quantify the confidence of the feature channel, an asymmetric background-aware sampling scheme is used in this embodiment to obtain the target spatial saliency. Then, the target-aware region feature map and the background-aware region feature map are obtained by weighting the feature map extracted from the target. Finally, the average energy of these two parts is calculated, and the FR (Feature Reliability) score is used as an evaluation indicator to allocate the target channel attention information and select channel features, such as Figure 1 As shown, the specific steps include:
[0042] S1: Calculate the FR score of each channel corresponding to the current frame image
[0043]
[0044] in, represents the FA value of the cth channel of the tth frame; Indicates that the c-th channel feature of the t-th frame corresponds to the average energy of the target perception area of the image; It indicates that the c-th channel feature of the t-th frame corresponds to the average energy of the background area of the image, and C represents the total number of channels.
[0045] The c-th channel feature of the t-th frame corresponds to the average energy of the target perception area of the image The calculation formula is:
[0046]
[0047] in, A represents the feature value of the pixel (i, j) in the cth channel of the tth frame; Orepresents the area of the target perception area of the image; O represents the target perception area of the image; (i, j) represents the coordinates of the pixel point in the image.
[0048] The c-th channel feature of the t-th frame corresponds to the average energy of the background area of the image The calculation formula is:
[0049]
[0050] Among them, A S represents the area of the target search area of the image; S represents the target search area of the image.
[0051] S2: Based on the set minimum score threshold, all channels whose FR scores are greater than the minimum score threshold are selected as channels selected for the current frame.
[0052] Based on the above formula, it can be found that a high FR score indicates that the feature channel is rich in target-related information, while a low FA score indicates that the channel contains more background interference. Therefore, in this embodiment, by selecting a feature channel with a higher FR score for filter training, the negative impact of the low confidence channel on filter learning is reduced.
[0053] S3: Calculate the weight of each channel in the selected channels based on the FR score of each channel.
[0054] In this embodiment, a corresponding weight is assigned to each selected channel based on the FR score of each channel. The specific calculation formula is as follows:
[0055]
[0056] in, represents the weight of the cth channel corresponding to the tth frame; max(FR t ) represents the maximum value of the FR scores of all channels selected in the tth frame; min(FR t ) represents the minimum FR score of all channels selected in the tth frame.
[0057] In one implementation, the method described in this embodiment can be directly applied to the invention patent "A target tracking method, device and medium based on asymmetric background perception" with application number 2023110307458. In a specific application, when calculating the relevant filter response of the t-th frame sample, the channel weight used is calculated by the method of this embodiment. The weight of each channel selected for each frame Replace it with the relevant filter response calculation formula in the invention patent Replace with
[0058] Through the method of this embodiment, the accuracy of feature channel selection in the target tracking process is effectively improved, and the robustness and accuracy of tracking are enhanced.
[0059] Embodiment 2:
[0060] The present invention also provides a DCF-based filter channel selection terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the above-mentioned method embodiment of the first embodiment of the present invention when executing the computer program.
[0061] Further, as an executable solution, the filter channel selection terminal device based on DCF can be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The filter channel selection terminal device based on DCF may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the composition structure of the filter channel selection terminal device based on DCF is only an example of the filter channel selection terminal device based on DCF, and does not constitute a limitation on the filter channel selection terminal device based on DCF. It may include more or less components than the above, or combine certain components, or different components. For example, the filter channel selection terminal device based on DCF may also include input and output devices, network access devices, buses, etc., and the embodiments of the present invention do not limit this.
[0062] Further, as an executable solution, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the DCF-based filter channel selection terminal device, and uses various interfaces and lines to connect various parts of the entire DCF-based filter channel selection terminal device.
[0063] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the DCF-based filter channel selection terminal device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and an application required for at least one function; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0064] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method in the embodiment of the present invention are implemented.
[0065] If the module / unit integrated in the DCF-based filter channel selection terminal device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory) and software distribution medium, etc.
[0066] Although the present invention has been specifically shown and described in conjunction with the preferred embodiments, it should be understood by those skilled in the art that various changes may be made to the present invention in form and details without departing from the spirit and scope of the present invention as defined by the appended claims, all of which are within the scope of protection of the present invention.
Claims
1. A filter channel selection method based on DCF, characterized in that: When tracking a target through DCF, the following steps are used to select the channel used for each frame and calculate the weight of the corresponding channel; S1: Calculate the FR score of each channel corresponding to the current frame image: in, represents the FA value of the cth channel of the tth frame; Indicates that the c-th channel feature of the t-th frame corresponds to the average energy of the target perception area of the image; Indicates that the c-th channel feature of the t-th frame corresponds to the average energy of the background area of the image; S2: Based on the set minimum score threshold, all channels with FR scores greater than the minimum score threshold are selected as the channels selected for the current frame; S3: Calculate the weight of each channel in the selected channels based on the FR score of each channel.
2. The filter channel selection method based on DCF according to claim 1, characterized in that: The c-th channel feature of the t-th frame corresponds to the average energy of the target perception area of the image The calculation formula is: in, A represents the feature value of the pixel (i, j) in the cth channel of the tth frame; O represents the area of the target perception area of the image; O represents the target perception area of the image; (i, j) represents the coordinates of the pixel point in the image.
3. The filter channel selection method based on DCF according to claim 2, characterized in that: The c-th channel feature of the t-th frame corresponds to the average energy of the background area of the image The calculation formula is: Among them, A S represents the area of the target search area of the image; S represents the target search area of the image.
4. The filter channel selection method based on DCF according to claim 1, characterized in that: The calculation formula for the weight of each channel is: in, represents the weight of the cth channel corresponding to the tth frame; max(FR t ) represents the maximum value of the FR scores of all channels selected in the tth frame; min(FR t ) represents the minimum FR score of all channels selected in the tth frame.
5. A filter channel selection terminal device based on DCF, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 4 when executing the computer program.
6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.