Visual tracking method and device and computer program product
The visual tracking method iteratively updates a kernel function-based feature histogram to enhance tracking accuracy in infrared light scenarios, addressing the limitations of deep learning models by eliminating the need for extensive training data.
Patent Information
- Application Number
- CN202411222545.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-02
- Publication Date
- 2024-12-31
AI Technical Summary
Existing visual tracking solutions, particularly those using deep learning models, require large amounts of training data and struggle to distinguish between tracking objects and background areas in infrared light-based tracking scenarios due to small pixel feature differences, leading to inaccuracies.
A visual tracking method utilizing a kernel function to characterize a feature histogram, with iterative updates based on variance and mean shift clustering to predict and correct the kernel function bandwidth, enabling accurate tracking without deep learning models.
The method achieves continuous and accurate tracking of objects by iteratively updating the feature histogram, improving tracking accuracy and reducing the need for extensive training data, suitable for infrared light-based scenarios.
Smart Images
Figure CN119228842A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of visual tracking technology, and in particular to a visual tracking method, device and computer program product. Background Art
[0002] Visual tracking is a research hotspot in the field of computer vision, and has important research and application value in scenarios such as virtual reality, human-computer interaction, intelligent monitoring, augmented reality, and machine perception. Visual tracking mainly analyzes video image sequences, matches the detected candidate target areas, and locates the position of the tracked object in the video sequence.
[0003] At present, the mainstream visual tracking solution is to use deep learning models to identify the target area to be tracked in the image sequence. This method requires a large amount of training data to support the training of the deep learning model, and different tracking objects correspond to different training data, which has a high implementation threshold. In addition, for infrared light-based tracking scenes (at night), the pixel feature difference between the target area to be tracked and the background area is small, which results in the trained deep learning model not necessarily being able to distinguish between the target area and the background area. Summary of the invention
[0004] The purpose of this application is to provide a visual tracking method, device and electronic device that can continuously and accurately lock the tracking object area without relying on a deep learning model.
[0005] In order to achieve the above purpose, the embodiment of the present application is implemented as follows:
[0006] In a first aspect, a visual tracking method is provided, comprising:
[0007] Determine a first feature histogram of a tracked object region from the image sequence, wherein the first feature histogram is characterized by a kernel function;
[0008] The multiple iterations are performed to: predict the number of pixels in the tracking object area in the next iteration based on the variance and / or the mean of the variance of the first feature histogram in the current iteration under the mean shift clustering algorithm; correct the kernel function bandwidth of the first feature histogram based on the predicted number of pixels in the tracking object area in the next iteration; update the first feature histogram in the current iteration according to the kernel function bandwidth corrected in the current iteration to determine the first feature histogram in the next iteration;
[0009] The tracking object area is determined based on the first feature histogram after the multiple rounds of iterations.
[0010] In a second aspect, a visual tracking device is provided, comprising:
[0011] A histogram determination module, used to determine a first feature histogram of a tracked object area from an image sequence, wherein the first feature histogram is represented by a kernel function;
[0012] The histogram updating module is used for performing multiple rounds of iterations: predicting the number of pixels in the tracking object area in the next round of iteration based on the variance and / or variance mean of the first feature histogram in the current round of iteration under the mean shift clustering algorithm; correcting the kernel function bandwidth of the first feature histogram based on the predicted number of pixels in the tracking object area in the next round of iteration; updating the first feature histogram in the current round of iteration according to the kernel function bandwidth corrected in the current round of iteration to determine the first feature histogram in the next round of iteration;
[0013] The tracking object locking module is used to determine the tracking object area based on the first feature histogram after the multiple rounds of iterations.
[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor; and a memory configured to store computer-executable instructions, wherein the computer-executable instructions, when executed, cause the processor to execute the method described in the first aspect.
[0015] According to a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store computer-executable instructions, and the computer-executable instructions implement the method described in the first aspect when executed by a processor.
[0016] The present application uses a kernel function to characterize the first feature histogram of the tracking object area in the image sequence. When it is necessary to continuously lock the tracking object area, based on the variance and / or variance mean of the first feature histogram under the mean shift clustering algorithm, the number of pixels in the tracking object area at a later time is predicted, and based on the predicted number of pixels, the kernel function bandwidth of the first feature histogram is corrected, and the first feature histogram is updated according to the corrected kernel function bandwidth. Since the kernel function bandwidth affects the size and shape of the tracking object area, updating the first feature histogram based on continuously correcting the kernel function bandwidth can avoid the first feature histogram from being too large. Figure 1 The same kernel function is used to avoid position inaccuracy, which can more accurately lock the tracking object area. The entire solution does not rely on the deep learning model to predict the tracking object area, so there is no need to prepare a large amount of training data, and it is suitable for night tracking scenarios based on infrared light. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0018] Figure 1 Schematic diagram of the flow of the visual tracking method according to an embodiment of the present application.
[0019] Figure 2 Schematic diagram of dividing the tracking object area and the background area in an image.
[0020] Figure 3 Schematic diagram comparing the tracking effects of the visual tracking method according to an embodiment of the present application and the traditional visual tracking method.
[0021] Figure 4 Schematic diagram of the structure of the visual tracking device according to an embodiment of the present application.
[0022] Figure 5 A schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION
[0023] As mentioned above, the current mainstream visual tracking solution is to use a deep learning model to identify the target area to be tracked in the image sequence. This method requires a large amount of training data to support the training of the deep learning model, and different tracking objects correspond to different training data, which has a high implementation threshold. In addition, for infrared light-based tracking scenes (at night), the pixel feature gap between the target area to be tracked and the background area is small, which results in the trained deep learning model not necessarily being able to distinguish between the target area and the background area.
[0024] To address the above issues, this application proposes a visual tracking solution that can accurately track object areas without relying on deep learning models, and is suitable for infrared light tracking scenarios at night.
[0025] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.
[0026] An embodiment of the present application provides a visual tracking method. Figure 1It is a flowchart of the visual tracking method, including:
[0027] S102, determining a first feature histogram of a tracking object region from an image sequence, wherein the first feature histogram is represented by a kernel function.
[0028] In this embodiment, the image sequence is composed of multiple frames of images showing the tracked object. As an exemplary introduction, the image sequence can be video data collected by a night infrared camera for the tracked object, and the source of the image sequence is not specifically limited in this article.
[0029] Specifically, this embodiment divides each frame of image into a tracking object area and a background area. Figure 2 As shown, the blue frame area is the tracking object area, the area outside the red frame is the background area, and the area between the blue frame and the red frame is the isolation area used to distinguish the background area from the tracking object.
[0030] exist Figure 2 On this basis, this embodiment uses a feature histogram (such as a grayscale feature histogram) to express the tracking object area and the background area. For the convenience of distinction, the feature histogram of the tracking object area is defined as the first feature histogram, and the feature histogram of the background area is defined as the second feature histogram. Correspondingly, the position of the first feature histogram is the position of the tracking object area, and the position of the second feature histogram is the position of the background area.
[0031] In a feasible implementation, this embodiment uses a probability density function to characterize the feature histogram, that is, the target model of the feature histogram adopts the mathematical formula of the probability density function.
[0032] The probability density function is introduced in detail below.
[0033] This embodiment defines the target model of the tracking object area extracted from the first frame image as:
[0034]
[0035] Among them, u represents the uth level, and m is the grayscale quantization level, that is, It is an m-level histogram, usually m<256. When determining the tracking object area in subsequent frames, the target model can be described using the following formula:
[0036]
[0037] Based on formulas (1) and (2), in order to facilitate modeling, we first need to normalize all pixel positions in the tracking object area in the row and column directions so that the pixel position coordinates are controlled within the range of 0 to 1. Assume If it is the normalized pixel coordinate in formula (1) and the central pixel coordinate of the target area is (0, 0), then the probability of the target model can be calculated by the following formula:
[0038]
[0039] where δ is the delta function, n represents the number of pixels, represents the pixel position in the image, is the position The quantization value of the pixel at the corresponding feature space, where the feature here can be either the spatial position or the gray feature, u represents the mean. C is the normalization coefficient, and k(·) is the kernel function, which should satisfy:
[0040] K(x) = k(‖x‖ 2 ) (4)
[0041] At the same time, it should also satisfy a: k(·) is a piecewise continuous function, and is a non-increasing function, and if a < b, then k(a) ≥ k(b); k(·) is a non-negative function.
[0042] Among them, two commonly used kernel functions of k(·) are as follows:
[0043] (1) Gaussian kernel function
[0044]
[0045] The corresponding function shape is:
[0046]
[0047] (2) Epanechnikov kernel function
[0048]
[0049] where c d is the volume of the d-dimensional sphere, and the corresponding function shape is:
[0050]
[0051] Assume that in the current frame image, {X i} i＝1,…,n , is the normalized candidate target pixel position, and Y is the pixel coordinate of the center position of the candidate target area. Then the probability of the u-th level of the updated candidate target model at Y can be calculated by the following formula:
[0052]
[0053] where C h is the normalization parameter, nh is the number of pixels, k is the kernel function, Y is the current position, h is the bandwidth, which represents the number of pixels and is usually set to 1 / 2 of the target number of pixels. The formula for the normalization coefficient is:
[0054]
[0055] The above formula (3) is a normalized expression of formula (1), and the above formula (8) is a normalized expression of formula (2). If the present embodiment updates the tracking object area of the subsequent frame image according to the previous frame image, then the initial first feature histogram of the tracking object area and the initial second feature histogram of the background area in the previous frame image can refer to formula (3), and the initial first feature histogram of the tracking object area and the initial second feature histogram of the background area in the subsequent frame image can refer to formula (8).
[0056] S104, multiple rounds of iterations are performed to: predict the number of pixels in the tracking object area of the next iteration based on the variance and / or variance mean of the first feature histogram of the current iteration under the mean shift clustering algorithm; correct the kernel function bandwidth of the first feature histogram of the current iteration based on the predicted number of pixels in the tracking object area of the next iteration; update the first feature histogram of the current iteration according to the corrected kernel function bandwidth of the current iteration to determine the first feature histogram of the next iteration.
[0057] It should be understood that the bandwidth of the kernel function, as a parameter-free fast pattern matching algorithm based on kernel density estimation, reflects the size and shape of the tracking object area. Therefore, whether the bandwidth of the kernel function is appropriate is a key factor affecting the tracking accuracy.
[0058] In the field of visual tracking, the tracking object in the image sequence is dynamic, which causes the first feature histogram corresponding to it to change at any time. Therefore, in the multiple rounds of iterations of this embodiment, the first feature histogram is updated based on the modified kernel function bandwidth, thereby improving the matching degree between the position area of the first feature histogram and the actual tracking object area.
[0059] Among them, the conditions for the termination of the above-mentioned multiple rounds of iterations may include: the similarity between the first feature histogram of the current iteration and the first feature histogram of the first iteration reaches a preset standard. In practical applications, this embodiment can perform multiple rounds of iterative updates on the first feature histogram in the current frame image, and the first feature histogram of the first iteration may be the first feature histogram in the previous frame image (such as the previous frame image). Assuming that the frame rate of the video is 24fps (frames per second), then one frame is 1 / 24 second; usually, the tracked object (person) cannot make a large movement in such a short time, so setting the preset standard for the termination of multiple rounds of iterations is to prevent the updated first feature histogram from being too different from the first feature histogram in the previous frame image, resulting in distortion.
[0060] The following is a detailed introduction to the process of multiple rounds of iterations.
[0061] This embodiment adopts the mean shift target tracking algorithm of the neutrosophic weighted histogram to iteratively update the first feature histogram, and specifically involves the neutrosophic set theory and the mean shift target tracking algorithm theory.
[0062] Regarding Neutrosophic Set Theory:
[0063] The neutrosophic set contains three parameters, namely, truthfulness, uncertainty, and distortion.
[0064] Suppose there is a non-empty set C, called A={ <c,T A (c),I A (c),F A (c)>|c∈C} is a neutrosophic set, called T A (c) I A (c) F A (c) are the true degree, uncertain degree, and untrue degree of c belonging to A in set C. A (c) I A (c) F A (c) are single values on the standard interval [0,1], and the sum of the three parameters is not restricted. Neutrosophic sets explicitly quantify uncertainty, and the unreal component, uncertainty component, and real component are independent of each other.
[0065] A multi-criteria neutrosophic set can be expressed as follows:
[0066] Assume A={A1,A2,…A n} is a series of choices, C = {C1, C2, ... C m} is a series of criteria, then choose A i The features of (i=1,2,…,n) can be represented by the following information:
[0067]
[0068] in Representative choice A i Meets Criteria C j degree. Representative choice A i Meets or does not meet Criteria C j degree of uncertainty. Representative choice A i Does not meet Criterion C j degree.
[0069] Two options A i and A j The similarity between them can be defined as:
[0070]
[0071] In this embodiment, A is the neutron-weighted histogram of the tracking object area, and A * is the ideal neutrosophic weight histogram of the tracked object area.
[0072] Assumptions Then A i and A * The similarity between them can be calculated by the following formula:
[0073]
[0074] Assume ω k ∈[0,1] is the standard C k The weight of Then A i and A * The weighted correlation coefficient between can be defined as:
[0075]
[0076] Theory of mean shift target tracking algorithm:
[0077] The mean shift target tracking algorithm is essentially a pattern matching algorithm, that is, it searches for the pattern area that is most similar to the reference target model in the current frame. The similarity requires a measurement index. If this index reaches the maximum value in a certain area, then the area is the area where the target is located in the current frame. The description formula (6) of the reference target model and the description formula (11) of the candidate target model are used to measure the similarity through the following similarity function:
[0078]
[0079] Where θ is the similarity coefficient, which is the Bhattacharyya coefficient between p and q:
[0080]
[0081] The position Y where formula (12) takes the minimum value is the best matching area of the reference target, that is, the position where the target is most likely to appear in the current frame.
[0082] Target tracking based on mean shift locates the target by minimizing formula (12). Since the maximization formula (13) is equivalent to the minimization formula (12), the target can be located by finding the maximum value of formula (13). The Bhattacharyya coefficient between q and p(Y) is Perform Taylor expansion at the position and discard the high-order terms. The approximate formula of formula (13) can be expressed as:
[0083]
[0084] Substituting formula (11) into formula (14) yields:
[0085]
[0086] in:
[0087]
[0088] Maximizing equation (13) is to maximize the second term of equation (15), because there is no position Y variable in the first term. The maximum value of the second term of equation (15) can be searched by using the mean shift principle. The mean shift algorithm can recursively move the center position (core) of the target from the target position of the previous frame to the new position, using the following equation:
[0089]
[0090] in, is the position of the first feature histogram in the next iteration; h is the kernel function bandwidth modified in this iteration; n h is the predicted number of pixels in the tracking object area in the next iteration; i is n h The pixel number in X i is the position of the i-th pixel; is the position of the first feature histogram of this iteration; w i is the neutrosophic weight histogram of this iteration, and its calculation formula refers to formula (34) below; g() is a piecewise continuous function.
[0091] By continuously iterating formula (17), the position coordinates that make the Bhattacharyya coefficient in the current frame reach the maximum value can be obtained. This position coordinate is the position where the target is most likely to appear in the current frame. That is, the purpose of locating the target is achieved.
[0092] In the traditional mean shift algorithm, it is assumed that the sample space is x n , n=1,...N, the mean u of the sample space x Set as the mean estimator for the main cluster The initial value of Consider a kernel function p(x; μw, σw) with mean μw and variance σw, assign the mean of the main cluster to the mean of the kernel function The standard deviation here is not discussed for now. The weight of each sample can be obtained by OK, α n A in the above equation is the weight α n The sum of the normalized coefficients is equal to 1, such as With weight α n Calculate sample x n ,n=1,...N mean u x ,like The value of the mean estimator for the main class can be updated as If the value of the average estimator If the change of is equal to or less than a predetermined fixed value, the updating process is terminated. Otherwise, the iteration is repeated.
[0093] This embodiment adopts the mean shift algorithm based on the update kernel, and defines the sample standard deviation σ x Derive the main cluster standard deviation σ n :
[0094] Assuming that a Gaussian distribution function with mean u and variance σ can be expressed as p(x;u,σ), which can be abbreviated as p(x;σ), the following formula can be used to express the Gaussian distribution:
[0095]
[0096]
[0097] Assume that the sample mode does not deviate from the mean u N ,If a sufficiently small fixed kernel bandwidth determined by the standard deviation of the kernel function is iterated from an appropriate initial value, the influence of outliers on the estimate gradually decreases as the estimate converges.
[0098] Therefore, when the estimate converges to the true value, only the clusters from the main cluster x are considered. n, n=1,...N samples are sufficient. In addition, the average μN of the main cluster and the average μW of the Gaussian kernel are consistent near the convergence point. Even if the coordinate transformation is performed so that both are 0, there is no loss of generality. Therefore, μW=μN=0 can be set.
[0099]
[0100] Sample variance The expected value is as follows:
[0101]
[0102] The variance of is sufficiently smaller than that of other parts. Therefore, based on the assumption that the main cluster follows a Gaussian distribution, it can be approximated by the following equation, as shown below:
[0103]
[0104] The definition of A and the mean can be expanded to:
[0105]
[0106] The remainder is:
[0107]
[0108]
[0109] in, Indicates the standard deviation σ W The standard deviation of the sample σ x The corresponding kernel function variance is estimated as:
[0110]
[0111] Based on formula (21), the number of samples belonging to the main cluster can be estimated as:
[0112]
[0113] Wherein, A in formula (25) is the normalization coefficient.
[0114] In order to enhance the tracking accuracy of the traditional mean shift tracker in complex backgrounds, a new weight histogram ω is defined in the neutral region. NS . Weight histogram ω NS Each level of is expressed by three membership functions T (true value), I (uncertainty) and T (false value).
[0115] Since the proposed target feature is an uncertain feature, T CO, I CO and F CO Represent the probability of the hypothesis being true, uncertain, and false, respectively. Finding the position of the tracked object in a new frame is the main task of the tracker, and the target model (the target feature histogram in the initial frame) is often used as the main information to distinguish the target from the background. The area with more similarity to the target feature is more likely to be the target area. Using the target feature similarity criterion, the definition can be further given:
[0116]
[0117]
[0118] F CO( u)=1-T CO (u) (28)
[0119] in, is the target region G in the first frame of the tracking process O The probability of the corresponding histogram level u is calculated using equation (6). Equation (19) defines the uncertainty level I CO . is the probability of the target feature histogram at level u updated in the previous frame. Assume is the probability of the u-th level of the feature histogram of the target area predicted by the tracker in the t-1th frame, then Calculated by the following formula:
[0120]
[0121] where λ is the update rate and λ∈(0,1).
[0122] The camera tracker may be offset from the target due to similar background, so using target features with high similarity to the background will bring risks to the accuracy of the tracker. If the background region feature similarity criterion is considered, the corresponding three membership functions are defined as follows:
[0123]
[0124]
[0125]
[0126] in is the probability of the uth level of the target background feature histogram, which is in the background area G of the first frame. B Initialize in, such as Figure 1 Equation (6) is also used to calculate When the surrounding environment of the tracked object changes, Will be updated.
[0127] By substituting the target feature similarity and background feature similarity into equation (3) with the corresponding T (true value), I (uncertainty) and F (false value), the probability of the uth level of the neutrosophic weighted histogram can be calculated as follows:
[0128]
[0129] Among them, ω CO ,ω CB is the weight in the interval [0,1], ω CO +ω CB =1; T(u) is the true degree parameter of the u-th gray level; I(u) is the uncertainty degree parameter of the u-th gray level; F(u) is the untrue degree C of the u-th gray level k , CO, CB are different standards of the intelligence set theory; A * is an ideal neutrosophic weight histogram of the tracking object area.
[0130] Based on the above principles, the algorithm steps of the neutrosophic mean shift based on the update kernel include:
[0131] Initialization phase:
[0132] Step 1: Read the t-th frame image and select the object to be tracked.
[0133] Step 2: Calculate the first feature histogram of the tracking object area in the first frame image according to formula (6): And the second feature histogram of the background area
[0134] Tracking phase:
[0135] Step 3: Use the position y0 of the first feature histogram in the t-th frame image as the starting position of the first feature histogram in the t+1-th frame image. For the t+1-th frame image, calculate the variance and the mean of the variance of the first feature histogram according to formula (18) and formula (19), calculate the variance of the kernel function according to formula (24), and then calculate the number of pixels contained in the tracking object area according to formula (25), thereby obtaining a new kernel function bandwidth.
[0136] Step 4: For the t+1 frame image, use the new kernel function bandwidth, based on the mean shift algorithm and the neutrino weighted histogram, and update the position of the first feature histogram again according to formula (34) and formula (17):
[0137]
[0138] Step 5: If Then stop; otherwise, When When Assign to Then go to step 4 and recalculate Until stop.
[0139] in, is the mathematical distance between two coordinate points, and ε0 is the preset judgment threshold. In is the first feature histogram of the first iteration.
[0140] Step 6: For the t+1th frame image, calculate according to equation (29) Then, when the Bhattacharyya coefficient season in is the current background area G B The corresponding second feature histogram.
[0141] S106: Determine a tracking object area based on the first feature histogram after multiple rounds of iterations.
[0142] It should be understood that the first feature histogram position after multiple rounds of iterations in this embodiment is the tracking object area.
[0143] In practical applications, assuming that the image sequence includes a first frame image and a second frame image, wherein the second frame image is located after the first frame image. Then, in S102, a first feature histogram of the tracking object area can be determined from the first frame image of the image sequence; correspondingly, in S106, the tracking object area in the second frame image of the image sequence can be determined based on the first feature histogram after multiple rounds of iterations.
[0144] Figure 3The following is a schematic diagram comparing the tracking effects of the traditional visual tracking method (deep learning model) and the visual tracking method of the present embodiment. The red frame area is the tracking object area. In the first frame, both the traditional visual tracking method and the visual tracking method of the present embodiment can normally determine the tracking object area, but after the 40th frame, the tracking object coincides with the tree. It is difficult for the deep learning model of the traditional visual tracking method to distinguish the difference between the tracking object and the tree, resulting in a deviation in the locking of the tracking object area. However, if the visual tracking method of the present embodiment performs multiple rounds of iterations on the first feature histogram for each frame, the first feature histogram after the current frame image is finally completed maintains a certain degree of similarity with the first feature histogram of the first round of iteration, thereby preventing the first feature histogram from being updated too much, and the visual tracking method of the present embodiment updates the first feature histogram according to the corrected kernel function bandwidth, so that the first feature histogram will not consistently use the original kernel function to cause position misalignment, and finally the tracking object area can be accurately locked after the 40th frame.
[0145] Corresponds to Figure 1 According to the method shown, another embodiment of the present application provides a visual tracking device. Figure 4 Schematic diagram of the structure of the visual tracking device, including:
[0146] The histogram determination module 410 is used to determine a first feature histogram of the tracked object area from the image sequence, where the first feature histogram is represented by a kernel function.
[0147] The histogram update module 420 is used for multiple rounds of iterations to perform: based on the variance and / or variance mean of the first feature histogram of the current iteration under the mean shift clustering algorithm, predicting the number of pixels in the tracking object area of the next iteration; based on the predicted number of pixels in the tracking object area of the next iteration, correcting the kernel function bandwidth of the first feature histogram; updating the first feature histogram of the current iteration according to the corrected kernel function bandwidth of the current iteration to determine the first feature histogram of the next iteration.
[0148] The tracking object locking module 430 is used to determine the tracking object area based on the first feature histogram after the multiple rounds of iterations.
[0149] The device of this embodiment uses a kernel function to characterize the first feature histogram of the tracking object area in the image sequence. When it is necessary to continuously lock the tracking object area, based on the variance and / or variance mean of the first feature histogram under the mean shift clustering algorithm, the number of pixels in the tracking object area at a later time is predicted, and based on the predicted number of pixels, the kernel function bandwidth of the first feature histogram is corrected, and the first feature histogram is updated according to the corrected kernel function bandwidth. Since the kernel function bandwidth affects the size and shape of the tracking object area, updating the first feature histogram based on continuously correcting the kernel function bandwidth can avoid the first feature histogram from being too large. Figure 1 The same kernel function is used to avoid position inaccuracy, which can more accurately lock the tracking object area. The entire solution does not rely on the deep learning model to predict the tracking object area, so there is no need to prepare a large amount of training data, and it is suitable for night tracking scenarios based on infrared light.
[0150] Optionally, the condition for terminating the multiple rounds of iterations includes: the similarity between the first feature histogram of the current round of iteration and the first feature histogram of the first round of iteration reaches a preset standard.
[0151] Optionally, before multiple rounds of iterations, the histogram determination module 410 is also used to: determine a second feature histogram of the background area from the image sequence, wherein the second feature histogram is represented by a kernel function; the histogram updating module 420 updates the first feature histogram of the current iteration according to the kernel function bandwidth corrected in the current iteration to determine the first feature histogram of the next iteration, including: determining the authenticity parameter, uncertainty parameter and distortion parameter of the neutrosophic set theory of the current iteration based on the first feature histogram of the current iteration and the second feature histogram of the current iteration; constructing a neutrosophic weighted histogram of the tracking object area of the current iteration based on the authenticity parameter, the uncertainty parameter and the distortion parameter of the current iteration; updating the first feature histogram of the current iteration according to the kernel function bandwidth corrected in the current iteration and in combination with the neutrosophic weighted histogram of the current iteration to determine the first feature histogram of the next iteration.
[0152] Optionally, the histogram updating module 420 updates the first feature histogram of the current iteration according to the kernel function bandwidth corrected in the current iteration and in combination with the neutrosophic weight histogram of the current iteration to determine the first feature histogram of the next iteration, including:
[0153] According to the formula Updating the first feature histogram of the current iteration to determine the first feature histogram of the next iteration;
[0154] in, is the position of the first feature histogram in the next iteration; h is the kernel function bandwidth modified in this iteration; n h is the predicted number of pixels in the tracking object area in the next iteration; i is n h The pixel number in X i is the position of the i-th pixel; is the position of the first feature histogram of this iteration; w i is the neutrosophic weight histogram of this iteration; g() is a piecewise continuous function.
[0155] The neutrosophic weight histogram w of this iteration i The calculation formula is:
[0156]
[0157] Where m is the number of gray levels; u is the gray level serial number; b(X i ) is X i The characteristic value of the pixel at is the kernel function of the first feature histogram of this iteration; is the kernel function of the first feature histogram in the next round; is the weighted correlation coefficient of neutrosophic set theory.
[0158] Weighted Correlation Coefficient of Neutrosophic Set Theory The calculation formula is:
[0159]
[0160] Among them, ω CO ,ω CB is the weight in the interval [0,1], ω CO +ω CB =1; T(u) is the true degree parameter of the u-th gray level; I(u) is the uncertainty degree parameter of the u-th gray level; F(u) is the untrue degree C of the u-th gray level k , CO, CB are different standards of the intelligence set theory; A * is an ideal neutrosophic weight histogram of the tracking object area.
[0161] Optionally, the histogram determination module 410 determines a first feature histogram of the tracked object area from a first frame image of the image sequence; correspondingly, the tracked object locking module 430 determines the tracked object area in a second frame image of the image sequence based on the first feature histogram after the multiple rounds of iterations; wherein the second frame image is located after the first frame image.
[0162] It should be understood that the visual tracking device of this embodiment can be used as Figure 1 The execution subject of the method shown can thus realize Figure 1 The steps and functions of the method shown.
[0163] Figure 5 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 5 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.
[0164] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0165] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.
[0166] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming the above-mentioned visual tracking device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:
[0167] A first feature histogram of a tracked object region is determined from an image sequence, wherein the first feature histogram is characterized by a kernel function.
[0168] Multiple rounds of iterations are performed to: predict the number of pixels in the tracking object area in the next round of iteration based on the variance and / or variance mean of the first feature histogram in the current round of iteration under the mean shift clustering algorithm; correct the kernel function bandwidth of the first feature histogram based on the predicted number of pixels in the tracking object area in the next round of iteration; update the first feature histogram in the current round of iteration according to the corrected kernel function bandwidth in the current round of iteration to determine the first feature histogram in the next round of iteration.
[0169] The tracking object area is determined based on the first feature histogram after the multiple rounds of iterations.
[0170] The above application Figure 1 The method disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in one or more embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with one or more embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.
[0171] Of course, in addition to software implementation methods, the electronic device of the present application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.
[0172] The present application also provides a computer program product, which includes a computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the following operations:
[0173] A first feature histogram of a tracked object region is determined from an image sequence, wherein the first feature histogram is characterized by a kernel function.
[0174] Multiple rounds of iterations are performed to: predict the number of pixels in the tracking object area in the next round of iteration based on the variance and / or variance mean of the first feature histogram in the current round of iteration under the mean shift clustering algorithm; correct the kernel function bandwidth of the first feature histogram based on the predicted number of pixels in the tracking object area in the next round of iteration; update the first feature histogram in the current round of iteration according to the corrected kernel function bandwidth in the current round of iteration to determine the first feature histogram in the next round of iteration.
[0175] The tracking object area is determined based on the first feature histogram after the multiple rounds of iterations.
[0176] In short, the above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of the present application shall be included in the protection scope of one or more embodiments of the present application.
[0177] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
[0178] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0179] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0180] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
Claims
1. A visual tracking method, characterized in that: include: Determine a first feature histogram of a tracked object region from the image sequence, wherein the first feature histogram is characterized by a kernel function; Multiple rounds of iterations are performed to: predict the number of pixels in the tracking object area in the next round of iteration based on the variance and / or variance mean of the first feature histogram in the current round of iteration under the mean shift clustering algorithm; Based on the predicted number of pixels in the tracking object area of the next iteration, the kernel function bandwidth of the first feature histogram is corrected; according to the kernel function bandwidth corrected in the current iteration, the first feature histogram of the current iteration is updated to determine the first feature histogram of the next iteration; The tracking object area is determined based on the first feature histogram after the multiple rounds of iterations.
2. The method according to claim 1, characterized in that The condition for terminating the multiple rounds of iterations includes: the similarity between the first feature histogram of the current round of iteration and the first feature histogram of the first round of iteration reaches a preset standard.
3. The method according to claim 1, characterized in that Before the multiple rounds of iterations, the method further includes: Determine a second feature histogram of a background area from the image sequence, wherein the second feature histogram is characterized by a kernel function; The first feature histogram of the current iteration is updated according to the kernel function bandwidth corrected in the current iteration to determine the first feature histogram of the next iteration, including: Determine the truth parameter, uncertainty parameter and distortion parameter of the neutrosophic set theory of this round of iteration based on the first feature histogram of this round of iteration and the second feature histogram of this round of iteration; Based on the truth parameter, the uncertainty parameter and the distortion parameter of this iteration, constructing a neutrosophic weighted histogram of the tracking object area of this iteration; According to the kernel function bandwidth corrected in this round of iteration and in combination with the neutrosophic weight histogram of this round of iteration, the first feature histogram of this round of iteration is updated to determine the first feature histogram of the next round of iteration.
4. The method according to claim 3, characterized in that According to the kernel function bandwidth corrected in this round of iteration, combined with the neutrosophic weight histogram of this round of iteration, the first feature histogram of this round of iteration is updated to determine the first feature histogram of the next round of iteration, including: According to the formula Updating the first feature histogram of the current iteration to determine the first feature histogram of the next iteration; in, is the position of the first feature histogram in the next iteration; h is the kernel function bandwidth modified in this iteration; n h is the predicted number of pixels in the tracking object area in the next iteration; i is n h The pixel number in X i is the position of the i-th pixel; is the position of the first feature histogram of this iteration; w i is the neutrosophic weight histogram of this iteration; g() is a piecewise continuous function.
5. The method according to claim 4, characterized in that Where m is the number of gray levels; u is the gray level serial number; b(X i ) is X i The characteristic value of the pixel at is the kernel function of the first feature histogram of this iteration; is the kernel function of the first feature histogram in the next round; is the weighted correlation coefficient of neutrosophic set theory.
6. The method according to claim 5, characterized in that Among them, ω CO ,ω CB is the weight in the interval [0,1], ω CO +ω CB =1; T(u) is the true degree parameter of the u-th gray level; I(u) is the uncertainty degree parameter of the u-th gray level; F(u) is the untrue degree C of the u-th gray level k , CO, CB are different standards of the intelligence set theory; A * is an ideal neutrosophic weight histogram of the tracking object area.
7. The method according to claim 4, characterized in that Determining a first feature histogram of a tracking object area from an image sequence includes: Determine a first feature histogram of the tracking object area from a first frame image of the image sequence; Determining the tracking object area based on the first feature histogram after the multiple rounds of iterations includes: Based on the first feature histogram after the multiple rounds of iterations, the tracking object region in the second frame image of the image sequence is determined; wherein the second frame image is located after the first frame image.
8. A visual tracking device, characterized in that: include: A histogram determination module, used to determine a first feature histogram of a tracked object area from an image sequence, wherein the first feature histogram is represented by a kernel function; A histogram updating module, used for multiple rounds of iterations to perform: based on the variance and / or variance mean of the first feature histogram of the current iteration under the mean shift clustering algorithm, predicting the number of pixels in the tracking object area of the next iteration; Based on the predicted number of pixels in the tracking object area of the next iteration, the kernel function bandwidth of the first feature histogram is corrected; according to the kernel function bandwidth corrected in the current iteration, the first feature histogram of the current iteration is updated to determine the first feature histogram of the next iteration; The tracking object locking module is used to determine the tracking object area based on the first feature histogram after the multiple rounds of iterations.
9. An electronic device, comprising: processor; and a memory arranged to store computer executable instructions which, when executed, cause the processor to perform a method as claimed in any one of claims 1 to 7.
10. A computer program product, comprising a computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to execute the method according to any one of claims 1 to 7.