Method, device and apparatus for target tracking

Abstract

The present application relates to the field of image processing technology, and discloses a method for target tracking. It includes: obtaining first position information of a target; extracting a first image area from a first image frame according to a first search range and the first position information; obtaining second position information according to the first image area; extracting a second image area from a second image frame according to a second search range and the second position information; obtaining third position information according to the second image area; the first search range is larger than the second search range. By obtaining the second position information according to the first search range and the first position information, and then obtaining the third position information according to the second search range and the second position information, the estimation accuracy of the target position can be significantly improved through the strategy of first searching a large range and then positioning in a small range, thereby ensuring the calculation accuracy of target tracking. The present application also discloses a device and equipment for target tracking.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, for example, to a method, device and equipment for target tracking. Background Art

[0002] Object tracking aims to estimate the state of a specific target in an image sequence in real time and stably. It is the basis of high-level visual computing tasks such as visual navigation, video content understanding, and three-dimensional reconstruction.

[0003] In the process of implementing the embodiments of the present disclosure, it is found that there are at least the following problems in the related art: the estimation accuracy of the target position is not high. Summary of the invention

[0004] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical components or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0005] The embodiments of the present disclosure provide a method, an apparatus and a device for target tracking, so as to improve the estimation accuracy of the target position.

[0006] In some embodiments, the method for target tracking includes:

[0007] Acquiring first position information of the target object;

[0008] extracting a first image area from a first image frame according to a first search range and the first position information;

[0009] Acquire second position information according to the first image area;

[0010] extracting a second image area from a second image frame according to a second search range and the second position information;

[0011] Acquire third position information according to the second image area;

[0012] The first search range is larger than the second search range.

[0013] In some embodiments, obtaining second position information according to the first image area includes:

[0014] Acquire the color features and the first directional gradient histogram HOG texture features of the first image area,

[0015] Obtain a color confidence map corresponding to the color feature; obtain a texture confidence map corresponding to the first HOG texture feature;

[0016] Fusion the color confidence map with the texture confidence map to obtain a first position confidence map;

[0017] Acquire second position information according to the first position confidence map.

[0018] In some embodiments, obtaining a color confidence map corresponding to the color feature includes:

[0019] Classifying the color features by a preset classifier to obtain a foreground probability map;

[0020] Window aggregation is performed according to the foreground probability map to obtain a color confidence map.

[0021] In some embodiments, obtaining a texture confidence map corresponding to the first HOG texture feature includes:

[0022] Expand the preset filters;

[0023] Obtain a texture confidence map corresponding to the first HOG texture feature according to the expanded filter.

[0024] In some embodiments, obtaining a texture confidence map corresponding to the first HOG texture feature according to the expanded filter includes:

[0025] Filtering the first HOG texture feature according to the expanded filter;

[0026] The filtering result of the first HOG texture feature is interpolated to obtain a texture confidence map.

[0027] In some embodiments, obtaining third position information according to the second image area includes:

[0028] Acquire a second HOG texture feature of the second image area;

[0029] Obtaining a second position confidence map corresponding to the second HOG texture feature according to a preset filter;

[0030] Acquire third position information according to the second position confidence map.

[0031] In some embodiments, obtaining a second position confidence map corresponding to the second HOG texture feature according to a preset filter includes:

[0032] Filtering the second HOG texture feature according to a preset filter;

[0033] Interpolate the filtering result of the second HOG texture feature to obtain a second position confidence map.

[0034] In some embodiments, after obtaining the third position information according to the second position confidence map, the method further includes:

[0035] The filter is trained and updated according to the third position information.

[0036] In some embodiments, the device for target tracking includes a processor and a memory storing program instructions, wherein the processor is configured to execute the above-mentioned method for target tracking when executing the program instructions.

[0037] In some embodiments, the device comprises the above-mentioned apparatus for target tracking.

[0038] The method, apparatus, and binding device for target tracking provided by the embodiments of the present disclosure can achieve the following technical effects: by obtaining the second position information based on the first search range and the first position information, and then obtaining the third position information based on the second search range and the second position information, the estimation accuracy of the target position can be significantly improved through the strategy of first searching a large range and then positioning in a small range, thereby ensuring the calculation accuracy of target tracking.

[0039] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein:

[0041] Figure 1 is a schematic diagram of a method for target tracking provided by an embodiment of the present disclosure;

[0042] Figure 2 It is a schematic diagram of a device for target tracking provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0043] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0044] The terms "first", "second", etc. in the specification and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0045] Unless otherwise stated, the term "plurality" means two or more.

[0046] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.

[0047] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0048] Combination Figure 1 As shown, the present disclosure provides a method for target tracking, including:

[0049] Step S101, obtaining first position information of a target object;

[0050] Step S102, extracting a first image area from a first image frame according to a first search range and first position information;

[0051] Step S103, acquiring second position information according to the first image area;

[0052] Step S104, extracting a second image area from the second image frame according to the second search range and the second position information;

[0053] Step S105, obtaining third position information according to the second image area;

[0054] The first search range is larger than the second search range.

[0055] By adopting the method for target tracking provided by the embodiment of the present disclosure, the second position information is obtained based on the first search range and the first position information, and then the third position information is obtained based on the second search range and the second position information. In this way, the estimation accuracy of the target position can be significantly improved by adopting the strategy of first searching a large range and then positioning in a small range, thereby ensuring the calculation accuracy of target tracking.

[0056] Optionally, the image acquisition module acquires images of the target object in real time at a preset acquisition rate to obtain image frames of the target object.

[0057] Optionally, the first image frame is a previous image frame or a preset image frame; optionally, the first position information of the target is the target tracking result of the previous image frame, or the target tracking result of the preset image frame. In the first image frame, the image area of ​​the preset first search range is extracted with the first position information of the target as the center as the first image area; for example, the preset target size is M×N, and in the first image frame, the image area of ​​the first search range of size 5M×5N is extracted with the first position information of the target as the center as the first image area, which is the first image area, wherein M is the length of the target and N is the width of the target.

[0058] Optionally, the preset first search range is adjusted according to the application scenario and the requirement of computing efficiency. Selecting the first search range of 5M×5N can achieve a compromise between large-range search and real-time performance to a certain extent.

[0059] Optionally, acquiring the second position information according to the first image area includes:

[0060] Obtain the color features of the first image area and the first directional gradient histogram HOG texture features,

[0061] Obtain a color confidence map corresponding to the color feature; obtain a texture confidence map corresponding to the first HOG texture feature;

[0062] The color confidence map and the texture confidence map are fused to obtain a first position confidence map;

[0063] The second position information is obtained according to the first position confidence map.

[0064] Optionally, obtaining the color features of the first image area includes: for each pixel in the first image area, calculating its eigenvalue according to the color space; optionally, the eigenvalue is a three-dimensional vector; optionally, the color space is RGB (RGB colormode, RGB color mode), HSV (Hue, Saturation, Value, HSV color model) or Lab (CIELab colormodel, Lab color model), etc.; for example, the HSV color space includes hue, saturation and brightness, so that different colors have a higher degree of distinction, which is more suitable for foreground / background classification of pixel points during target tracking.

[0065] Optionally, the color feature of the first image region is expressed as a first two-dimensional multi-channel feature map consistent with the first image region.

[0066] Optionally, obtaining the first HOG (Histogram of Oriented Gradient) texture feature of the first image region includes: selecting a cell of a first set size, and calculating the histogram of the gradient direction of each cell; weighting the histogram value using the gradient amplitude; and performing a local normalization operation on the gradient direction histogram of each cell; this can suppress image noise and increase robustness. Optionally, a cell of a set size of 4×4 is selected, so that the accuracy of search positioning can be guaranteed while maintaining a full description of the image texture.

[0067] Optionally, the first HOG texture feature of the first image region is expressed as a second two-dimensional multi-channel feature map reduced by cells.

[0068] Optionally, a color confidence map corresponding to the color feature is obtained, including:

[0069] The color features are classified by a preset classifier to obtain a foreground probability map;

[0070] Window aggregation is performed based on the foreground probability map to obtain the color confidence map.

[0071] Optionally, a pixel point in the first image area is randomly selected, and the color feature c of the pixel point is input into a preset classifier, such as a Bayesian classifier, to obtain the posterior probability P(A|c) that the pixel point belongs to the foreground A; then the posterior probabilities of all pixels in the first image area constitute the foreground probability map of the first image area; wherein c is the color feature of the pixel point, A is the foreground of the pixel point, and P(A|c) is the posterior probability that the pixel point belongs to the foreground A; optionally, the size of the first image area is 5M×5N, then a foreground probability map of size 5M×5N is obtained. In this way, by introducing a Bayesian classifier that can realize pixel-level foreground / background classification, it is possible to quickly adapt to changes in target posture and effectively improve the resistance to noise interference.

[0072] Optionally, the foreground probability map is subjected to window aggregation with a window of a second set size, for example, the second set size is consistent with the size M×N of the target object, to obtain a color confidence map.

[0073] Optionally, taking each pixel point in the first image area as the center, a local window of a second set size is selected, for example: a local window of size M×N is selected, and the average value of the posterior probability that all pixels in the local window belong to the foreground is calculated to obtain the confidence that each pixel point in the first image area is the target center point; wherein, the larger the average value, the higher the expected value of the number of foreground pixels contained in the local window, and the higher the possibility that the pixel point is the target center. Optionally, the window aggregation operation is implemented by the integral graph method, thereby ensuring the real-time performance of the target tracking method.

[0074] Optionally, obtaining a texture confidence map corresponding to the first HOG texture feature includes:

[0075] Expand the preset filters;

[0076] The texture confidence map corresponding to the first HOG texture feature is obtained according to the expanded filter.

[0077] In some embodiments, the preset correlation filter with the first HOG texture feature as input is expressed as a two-dimensional multi-channel matrix of the same size as the second two-dimensional multi-channel feature map, and is represented in the frequency domain. Optionally, the preset correlation filter is trained on a smaller search range; optionally, the smaller search range is consistent with the second search range. In some embodiments, the size of the second search range is set to 2.5M×2.5N, and the size of the corresponding HOG texture feature and the preset correlation filter is

[0078] Optionally, the search range of the preset correlation filter is expanded, and the expansion of the preset correlation filter is performed in the time domain, and its size is expanded to be consistent with the search range by keeping the center position of the filter unchanged and filling zero values ​​around it, and multiple channels of the correlation filter are operated independently; for example: expanded to 5M×5N; Optionally, the correlation filter is transformed into a time domain representation through a two-dimensional discrete Fourier inverse transform, and its size is expanded to by keeping the center position of the correlation filter unchanged and filling zero values ​​around it. Then it is transformed into a frequency domain representation through a two-dimensional discrete Fourier transform. Optionally, for multiple channels of the correlation filter, the expansion process is the same and independent. Optionally, the expanded correlation filter can perform a correlation filtering operation on the first HOG texture feature extracted from the first search area of ​​size 5M×5N to search for possible target locations. In this way, the correlation filter is expanded to zero value in the time domain, maintaining the ability of the correlation filter to discriminate target characteristics in the original search area, without applying additional operations to the expanded area, and dense sampling and target confidence calculation of the entire area are achieved through cyclic shifting.

[0079] By expanding the preset correlation filter, the search area of ​​the correlation filter is expanded without the need for an additional context modeling process. This can handle the problem of large displacement between current image frames while maintaining high computational efficiency.

[0080] Optionally, obtaining a texture confidence map corresponding to the first HOG texture feature according to the expanded filter includes:

[0081] Filtering the first HOG texture feature according to the expanded filter;

[0082] The filtering result of the first HOG texture feature is interpolated to obtain a texture confidence map.

[0083] Optionally, for each channel of the second two-dimensional multi-channel feature map, it is transformed into a frequency domain representation through a two-dimensional discrete Fourier transform, and a spectral product is performed with the correlation filter of the corresponding channel. The spectral product results of all channels are added together to obtain the frequency domain representation result of the correlation filter output, and the target confidence of all positions in the first image area is obtained by a two-dimensional discrete Fourier inverse transform of the frequency domain representation result of the correlation filter output.

[0084] Optionally, the filtering result of the first HOG texture feature output by the correlation filtering, that is, the target confidence of all positions in the first image area, is interpolated to obtain a texture confidence map, that is, to obtain a correlation filtering output result consistent with the size of the original image area. In this way, the search positioning accuracy can be improved.

[0085] In this way, the texture confidence map is calculated by the correlation filter trained on the HOG texture feature, which contains strong target position information, but is easily disturbed by the background and is more sensitive to changes in target posture, especially when performing related operations on a large search area, there is a lot of noise; the color confidence map is calculated by the Bayesian classifier trained on the color feature, which contains strong foreground / background discrimination information. The classification and aggregation operations in pixels can effectively handle changes in target posture, but lack precise target position information; the two are complementary to a certain extent. By weighted fusion of the texture confidence map and the color confidence map, the respective advantages of the texture confidence map and the color confidence map can be retained, avoiding noise interference during a large-scale search and maintaining a certain search accuracy.

[0086] Optionally, the color confidence map f color and texture confidence map f texture Perform weighted fusion by calculating

[0087] Get the first position confidence map f; where f is the first position confidence map, f color is the color confidence map, f texture is the texture confidence map; color is the first fusion weight, λ texture is the second fusion weight; wherein the first fusion weight and the second fusion weight are determined by the maximum confidence values ​​of the color confidence map and the texture confidence map respectively;

[0088] Optionally, the position with the highest confidence in the first position confidence map f is the position where the target object in the current frame is most likely to appear, and the position with the highest confidence in the first position confidence map f is used as the second position information, which is the target search result of the current image frame.

[0089] By expanding the preset correlation filter and combining it with a Bayesian classifier for pixel-level foreground / background classification, the most likely location of the target object can be estimated within a larger search range, centering on the target tracking result of the previous image frame.

[0090] Optionally, the second image frame is the current image frame. In the second image frame, an image area within a preset second search range is extracted as the second image area with the second position information of the target object, i.e., the target search result of the current image frame, as the center; for example: an image area within the second search range with a size of 2.5M×2.5N is extracted as the second image area.

[0091] Optionally, acquiring third position information according to the second image area includes:

[0092] Acquire a second HOG texture feature of the second image area;

[0093] Obtaining a second position confidence map corresponding to the second HOG texture feature according to a preset filter;

[0094] The third position information is obtained according to the second position confidence map.

[0095] Optionally, obtaining the second HOG texture feature of the second image region includes: selecting a cell of a second set size, calculating a histogram of the gradient direction of each cell; weighting the histogram value using the gradient amplitude; and performing a local normalization operation on the gradient direction histogram of each cell; this can suppress image noise and increase robustness. Optionally, a cell of a set size of 4×4 is selected, so that the positioning accuracy can be guaranteed while maintaining a full description of the image texture.

[0096] Optionally, the second HOG texture feature of the second image region is expressed as a third two-dimensional multi-channel feature map reduced by cells.

[0097] Optionally, obtaining a second position confidence map corresponding to the second HOG texture feature according to a preset filter includes:

[0098] Filtering the second HOG texture feature according to a preset filter;

[0099] The filtering result of the second HOG texture feature is interpolated to obtain a second position confidence map.

[0100] Optionally, for each channel of the third two-dimensional multi-channel feature map, it is transformed into a frequency domain representation through a two-dimensional discrete Fourier transform, and a spectral product is performed with the correlation filter preset for the corresponding channel. The spectral product results of all channels are added together to obtain the frequency domain representation result of the correlation filtering output, and the target confidence of all positions in the second image area is obtained through a two-dimensional discrete inverse Fourier transform.

[0101] Optionally, the filtering result of the second HOG texture feature output by the correlation filtering is interpolated to obtain a second position confidence map.

[0102] Optionally, the position with the highest confidence in the second position confidence map is used as the third position information, that is, the position of the target in the current frame.

[0103] In this way, taking the target search result of the current image frame as the center, the target is located in a relatively small range through a preset correlation filter, and the precise position of the target in the current frame can be obtained.

[0104] Optionally, after obtaining the third position information, it also includes: estimating the size of the target object through a scale filter to obtain the target tracking result of the current image frame; in this way, a more accurate and complete target tracking result can be obtained, and at the same time, the adaptability of the target tracking method to different application scenarios can be improved.

[0105] In this way, since the second search range is smaller than the first search range, the strategy of first searching a large range and then accurately positioning a small range can significantly improve the estimation accuracy of the target position, thereby ensuring the calculation accuracy of target tracking; the relevant filter trained on a smaller search range in the current image frame contains less background information. On the premise of having a preliminary estimate of the target position of the current image frame, that is, obtaining the second position information, the estimated relatively rough current frame target search result can be further refined to obtain accurate positioning of the target.

[0106] Optionally, after obtaining the third position information according to the second position confidence map, the method further includes:

[0107] The filter is trained and updated according to the third position information.

[0108] Optionally, with the third position information as the center, optionally, with the current image frame target tracking result as the center, the image area of ​​the preset third search range is extracted as the third image area, for example: extracting the third image area of ​​the third search range of size 2.5M×2.5N, and calculating the color features and third HOG texture features of the third image area. Optionally, the third search range can be narrowed according to different application scenarios and the requirements of computing efficiency; optionally, setting the size of the third search range to 2.5M×2.5N can reduce the proportion of background information in the model while ensuring computing efficiency.

[0109] Optionally, taking the third HOG feature as input and the two-dimensional Gaussian distribution as output, a single-frame optimal two-dimensional multi-channel correlation filter is learned, and the correlation filter is updated by an exponential time weighted average strategy, and its update weight is determined by the current frame tracking state.

[0110] Optionally, color features are used as input, feature distributions belonging to the foreground / background are counted, color histograms of the foreground / background are calculated, and the Bayesian classifier is updated using an exponential time weighted average strategy. Optionally, the update weight is determined according to the current image frame tracking state.

[0111] Optionally, learning a single-frame optimal two-dimensional multi-channel correlation filter includes: a third HOG texture feature g of the third image region is consistent with a size of an expected output o constructed by a two-dimensional Gaussian distribution, two dimensions are independent of each other, a mean is a center position, and a variance is related to a size of the target object;

[0112] Optionally, the optimal two-dimensional multi-channel correlation filter h consists of multiple channels, Wherein, d is the number of channels of the third HOG texture feature, l=1,...,d, and l and d are both positive integers; optionally, the size of the optimal two-dimensional multi-channel correlation filter h is consistent with the third HOG texture feature g and the expected output o, and the optimal two-dimensional multi-channel correlation filter h is obtained by minimizing the error between the correlation filter output and the expected output;

[0113] By calculating: Obtain the optimal two-dimensional multi-channel correlation filter h;

[0114] Among them, h is the optimal two-dimensional multi-channel correlation filter, h l is the correlation filter of the lth channel, the symbol * represents the correlation operation, o is the expected output, g is the third HOG texture feature, d is the number of channels of the third HOG texture feature, is a regular term with λ as weight, which is used to limit the complexity of the learning model and prevent overfitting.

[0115] Optionally, by calculating, Get the representation of each channel of the correlation filter in the frequency domain;

[0116] Among them, capitalized variables represent the frequency domain representation of the corresponding variables obtained by two-dimensional discrete Fourier transform, and the variables with horizontal lines represent the complex conjugate of the corresponding variables; lis the frequency domain representation of the correlation filter of the lth channel obtained by two-dimensional discrete Fourier transform, O is the frequency domain representation of the expected output obtained by two-dimensional discrete Fourier transform, G is the frequency domain representation of the third HOG texture feature obtained by two-dimensional discrete Fourier transform, l=1,...,d, k=1,......,d, k, l and d are all positive integers; is the complex conjugate of the desired output in the frequency domain, and λ is the weight.

[0117] Optionally, the correlation filter is updated by the exponential time weighted average strategy, and the correlation filter of the single-frame optimal two-dimensional multi-channel frequency domain representation learned at the current moment is The correlation filter represented by the frequency domain of the lth channel at time t-1, that is, the historical model Perform weighted summation to obtain the correlation filter represented in the frequency domain of the lth channel at the current time t Enables the model to adaptively handle changes in target appearance;

[0118] By calculation Get the correlation filter represented in the frequency domain of the lth channel at the current time t

[0119] in, is the correlation filter represented in the frequency domain of the lth channel at the current time t, η t Update the weights for the current time t, is the correlation filter of the lth channel frequency domain at time t-1, H l is the correlation filter of the lth channel in frequency domain.

[0120] In this way, the training process of the correlation filter is completed in a relatively small image area, which can alleviate the overfitting and tracking drift problems.

[0121] Optionally, the current moment update weight is obtained according to the current frame tracking state; by calculating Get the updated weight η at the current time t t ;

[0122] Among them, η0 is the basic learning rate, f t is the first position confidence map at the current moment, f0 is the first position confidence threshold, is a constant.

[0123] In some embodiments, the calculation of the updated weight is a dynamic process. If the maximum value of the confidence of the first position at the current moment is greater than the threshold value f0, the tracking result at the current moment is more reliable, and the weight of the model update increases accordingly. On the contrary, if the maximum value of the confidence of the first position at the current moment is less than or equal to the threshold value f0, the reliability of the tracking result at the current moment is reduced, and the weight of the model update is reduced accordingly. In this way, the tracking drift problem can be alleviated.

[0124] Optionally, a color feature c′ of a pixel point in the third image region is preset. According to the Bayesian estimation theory, the posterior probability P(A|c′) of the pixel point belonging to the foreground A is calculated as follows:

[0125] Get the posterior probability P(A|c′) belonging to foreground A;

[0126] Among them, P(A|c′) is the posterior probability of the foreground, P(c′|A) is the color feature distribution of the foreground, P(c′|B) is the color feature distribution of the background, P(A) is the prior probability of the foreground, and P(B) is the prior probability of the background.

[0127] Optionally, the prior probabilities P(A) and P(B) represent the probability that a pixel belongs to the foreground / background without observing its features, and are constants independent of the model, calculated by the proportion of the size of the target object in the third image region. For example, if the Bayesian classifier is trained on a third image region of size 2.5M×2.5N, then P(A)=1 / (2.5×2.5)=0.16, and P(B)=0.84.

[0128] Optionally, the color feature distributions P(c′|A) and P(c′|B) are described by histograms. To simplify the amount of calculation, the three-dimensional color feature space is discretized into a size of 32×32×32. Optionally, an image region is preset, and the color features of all the pixels belonging to the target are counted by histogram to obtain the foreground color feature distribution P(c′|A); the color features of all the pixels belonging to the background are counted by histogram to obtain the background color feature distribution P(c′|B).

[0129] Optionally, the Bayesian classifier is updated through an exponential time weighted average strategy, similar to the update weight of the correlation filter, and the dynamic weight is calculated according to the current frame tracking state; the foreground / background color feature distribution of the image area extracted in the current frame is counted; the foreground / background color feature distribution is updated online, and the posterior probability is recalculated. In this way, the online update of the foreground / background color feature distribution can effectively retain the historical information within a certain time window, avoiding the non-smoothing problem caused by directly adjusting the posterior probability model.

[0130] In this way, the third HOG texture feature is used as input, the two-dimensional Gaussian distribution is used as output, and the target tracking result of the current image frame is used as the center to extract a relatively small image area, train the optimal two-dimensional multi-channel correlation filter for a single frame, and dynamically update the correlation filter according to the current image frame tracking state. Taking the color feature as input, the feature distribution belonging to the foreground / background is counted, the color histogram of the foreground / background is calculated, and the Bayesian classifier is dynamically updated according to the current image frame tracking state.

[0131] The method for target tracking provided by the embodiment of the present disclosure expands the preset correlation filter, combines the Bayesian classifier for pixel-level foreground / background classification, takes the target tracking result of the previous frame as the center, estimates the most likely position of the target in a larger search range, that is, the target search result of the current image frame; takes the target search result of the current image frame as the center, uses the preset correlation filter to locate the target in a relatively small range, and obtains the target tracking result of the current image frame, that is, the precise position of the target in the current image frame; in this way, the estimation accuracy of the target position can be significantly improved by the strategy of first searching a large range and then locating a small range, thereby ensuring the calculation accuracy of target tracking. Taking the target tracking result of the current image frame as the center, extracting the image area of ​​a relatively small range, training the optimal correlation filter of a single frame, statistically analyzing the feature distribution of the foreground / background, and adaptively updating the correlation filter and the Bayesian classifier according to the current image frame tracking state. It can effectively solve the problem that the search area of ​​the correlation filtering method is limited and seriously interfered by the background, and can handle the situation that affects the target tracking performance in practical applications such as rapid movement of the target and reappearance after short-term occlusion, and can still accurately track and locate the target in the case of rapid movement of the target and reappearance after short-term occlusion.

[0132] Combination Figure 2 As shown, an embodiment of the present disclosure provides a device for target tracking, including a processor 100 and a memory 101 storing program instructions. Optionally, the device may also include a communication interface 102 and a bus 103. The processor 100, the communication interface 102, and the memory 101 may communicate with each other through the bus 103. The communication interface 102 may be used for information transmission. The processor 100 may call the program instructions in the memory 101 to execute the method for target tracking of the above embodiment.

[0133] In addition, the program instructions in the above-mentioned memory 101 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0134] The memory 101 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor 100 executes the functional application and data processing by running the program instructions / modules stored in the memory 101, that is, the method for target tracking in the above embodiment is implemented.

[0135] The memory 101 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 101 may include a high-speed random access memory and may also include a non-volatile memory.

[0136] The device for target tracking provided by the embodiment of the present disclosure is adopted. By expanding the preset correlation filter and combining the Bayesian classifier for pixel-level foreground / background classification, the target tracking result of the previous frame is taken as the center, and the most likely position of the target is estimated within a larger search range, that is, the target search result of the current image frame; the target search result of the current image frame is taken as the center, and the preset correlation filter is used to locate the target within a relatively small range to obtain the target tracking result of the current image frame, that is, the precise position of the target in the current image frame; in this way, the estimation accuracy of the target position can be significantly improved by the strategy of first searching a large range and then locating a small range, thereby ensuring the calculation accuracy of target tracking. Taking the target tracking result of the current image frame as the center, a relatively small range of image area is extracted, the optimal correlation filter of a single frame is trained, the feature distribution of the foreground / background is statistically analyzed, and the correlation filter and the Bayesian classifier are adaptively updated according to the current image frame tracking state. It can effectively solve the problem that the search area of ​​the correlation filtering method is limited and seriously interfered by the background, and can handle the situation that affects the target tracking performance in practical applications such as rapid movement of the target and reappearance after short-term occlusion, and can still accurately track and locate the target in the case of rapid movement of the target and reappearance after short-term occlusion.

[0137] An embodiment of the present disclosure provides a device, comprising the above-mentioned apparatus for target tracking.

[0138] Optionally, the device is provided with an image acquisition module, and the device acquires images of the target object in real time at a preset acquisition rate through the image acquisition module to obtain image frames of the target object.

[0139] The device expands the preset correlation filter and combines the Bayesian classifier for pixel-level foreground / background classification. The device takes the target tracking result of the previous frame as the center, estimates the most likely position of the target in a larger search range, that is, the target search result of the current image frame; takes the target search result of the current image frame as the center, uses the preset correlation filter to locate the target in a relatively small range, and obtains the target tracking result of the current image frame, that is, the precise position of the target in the current image frame; in this way, the estimation accuracy of the target position can be significantly improved through the strategy of first searching a large range and then locating a small range, thereby ensuring the calculation accuracy of target tracking. Taking the target tracking result of the current image frame as the center, extract the image area of ​​a relatively small range, train the optimal correlation filter of a single frame, count the feature distribution of the foreground / background, and adaptively update the correlation filter and the Bayesian classifier according to the current image frame tracking state. It can effectively solve the problems of limited search area and severe background interference of the correlation filtering method, and can handle the situations that affect the target tracking performance in practical applications such as rapid movement of the target and reappearance after short-term occlusion. In the case of rapid movement of the target and reappearance after short-term occlusion, it can still accurately track and locate the target.

[0140] An embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute the above-mentioned method for target tracking.

[0141] An embodiment of the present disclosure provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the above-mentioned method for target tracking.

[0142] The computer-readable storage medium mentioned above may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0143] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of the embodiment of the present disclosure. The aforementioned storage medium may be a non-transient storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and other media that can store program codes, or a transient storage medium.

[0144] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible changes. Unless explicitly required, separate components and functions are optional, and the order of operation may vary. The parts and features of some embodiments may be included in or replace the parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and the claims, unless the context clearly indicates, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of listings containing one or more associated ones. In addition, when used in the present application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the presence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the various embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0145] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods for each specific application to implement the described functions, but such implementations should not be considered to exceed the scope of the embodiments of the present disclosure. The technicians may clearly understand that, for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above may refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0146] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.

[0147] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A method for target tracking, characterized in that: include: Acquiring first position information of the target object; extracting a first image area from a first image frame according to a first search range and the first position information; Acquire second position information according to the first image area; extracting a second image area from a second image frame according to a second search range and the second position information; Acquire third position information according to the second image area; The first search range is larger than the second search range; Acquiring second position information according to the first image area includes: acquiring color features and first directional gradient histogram HOG texture features of the first image area; Obtain a color confidence map corresponding to the color feature; expand a preset filter; obtain a texture confidence map corresponding to the first HOG texture feature according to the expanded filter; fuse the color confidence map with the texture confidence map to obtain a first position confidence map; and obtain second position information according to the first position confidence map.

2. The method according to claim 1, characterized in that Obtaining a color confidence map corresponding to the color feature, including: Classifying the color features by a preset classifier to obtain a foreground probability map; Window aggregation is performed according to the foreground probability map to obtain a color confidence map.

3. The method according to claim 1, characterized in that Obtaining a texture confidence map corresponding to the first HOG texture feature according to the expanded filter, including: Filtering the first HOG texture feature according to the expanded filter; The filtering result of the first HOG texture feature is interpolated to obtain a texture confidence map.

4. The method according to claim 1, characterized in that Acquiring third position information according to the second image area includes: Acquire a second HOG texture feature of the second image area; Obtaining a second position confidence map corresponding to the second HOG texture feature according to a preset filter; Acquire third position information according to the second position confidence map.

5. The method according to claim 4, characterized in that Obtaining a second position confidence map corresponding to the second HOG texture feature according to a preset filter, including: Filtering the second HOG texture feature according to a preset filter; Interpolate the filtering result of the second HOG texture feature to obtain a second position confidence map.

6. The method according to any one of claims 1 to 5, characterized in that: After obtaining the third position information according to the second position confidence map, the method further includes: The filter is trained and updated according to the third position information.

7. A device for target tracking, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to perform the method for target tracking according to any one of claims 1 to 6 when executing the program instructions.

8. A device for target tracking, characterized in that: Comprising the device for target tracking as described in claim 7.

Images