A target tracking method and device based on number theory transform

Abstract

The present disclosure relates to a kind of relevant filtering target tracking method and device based on number theory transformation, the method includes: obtaining the target position frame of tracking target in upper frame image;Target position frame is scaled to multiple scales, and the sampling position frame of scale is obtained;According to each sampling position frame, it is sampled on current frame image respectively, and the sampling image of scale is obtained;Sampling image is scaled to the size of the target position frame of upper frame image, and the candidate image of scale is obtained;For each candidate image, the image feature of candidate image is extracted and is quickly number theory transformation, and the first transformed feature is obtained;First transformed feature is correlated with first template, and multiplied with second template, and the first operation result is obtained;First operation result is quickly number theory inverse transformation, and the response value of candidate image is obtained;According to the response value of candidate image, based on the scale and position of the candidate image corresponding to maximum response value, the target scale and target position frame of target in current frame image are determined.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of target tracking, in particular, to a target tracking method and device. BACKGROUND

[0002] The main process of the current target tracking algorithm based on correlation filtering is as follows: step one, the position and size of the tracking target in the first frame are given, and the HOG feature of the tracking target is calculated, wherein the size of the first frame is defaulted to 1, and the HOG feature is processed by using the autocorrelation function to obtain the target template of the tracking target; step two, after the subsequent frame image is transmitted, the image near the position of the tracking target in the previous frame image is intercepted on the current frame image, the HOG feature is calculated and autocorrelation calculation is performed to obtain the Gaussian response value of the current frame image; step three, the target template of the previous frame image and the Gaussian response value of the current frame image are cross-correlated, and the position with the maximum response value is the position of the tracking target in the current frame image; step four, according to the position of the tracking target in the current frame image, the Gaussian response value is calculated again and used as the target template corresponding to the current frame image, and the next frame image is processed from step two.

[0003] The above-mentioned target tracking algorithm has the following disadvantages: it cannot adapt to the size change of the target, when the size of the tracking target changes greatly, the initially given position frame is difficult to continuously and stably track the target, and even the target is lost, a large amount of correlation operation is mixed with Fourier transform calculation, the resource usage on the hardware is large, and there is a certain error. SUMMARY

[0004] The purpose of the present disclosure is to provide a target tracking method and device to solve the problems existing in the prior art target tracking algorithm.

[0005] In order to achieve the above-mentioned purpose, the first aspect of the present disclosure provides a target tracking method, comprising:

[0006] obtaining a target position frame of a tracking target in a previous frame image of a current frame image;

[0007] scaling the target position frame according to a plurality of scales in a scale pool to obtain a sampling position frame corresponding to each scale;

[0008] sampling according to the sampling position frame corresponding to each scale on the current frame image to obtain a sampling image corresponding to each scale;

[0009] scaling the sampling image to the size of the target position frame of the previous frame image to obtain a candidate image corresponding to each scale;

[0010] For each of the candidate images, image features of the candidate image are extracted, and a fast number theory transformation is performed on the image features to obtain first transformed features;

[0011] A first template and a second template determined based on the previous frame image are obtained;

[0012] The first transformed features are correlated with the first template, and a result of the correlation operation is multiplied by the second template to obtain a first operation result;

[0013] A fast number theory inverse transformation is performed on the first operation result, and an inverse transformation result is taken as a response value corresponding to the candidate image;

[0014] A maximum response value is determined according to the response value corresponding to each of the candidate images, and a target scale and a target position frame corresponding to the tracking target in the current frame image are determined based on a scale and a position of the candidate image corresponding to the maximum response value.

[0015] Optionally, the method further comprises:

[0016] It is judged whether the maximum response value is greater than a preset threshold value;

[0017] If the maximum response value is not greater than the preset threshold value, a target image is cropped from the current frame image based on the target scale and the target position frame corresponding to the tracking target in the current frame image;

[0018] Image features of the target image are extracted, a fast number theory transformation is performed on the image features to obtain second transformed features, and the second transformed features are taken as a first reference template;

[0019] A self-correlation operation is performed on the first reference template, and a result of the self-correlation operation is taken as a second reference template;

[0020] The first reference template, a first template determined based on a previous frame image, and a first template determined based on a first frame image are weighted and superimposed to obtain a first template determined based on the current frame image;

[0021] The second reference template, a second template determined based on the previous frame image, and a second template determined based on the first frame image are weighted and superimposed to obtain a second template determined based on the current frame image.

[0022] Optionally, the method further comprises:

[0023] If the maximum response value is greater than the preset threshold value, a target image is cropped from the current frame image based on the target scale and the target position frame corresponding to the tracking target in the current frame image;

[0024] extract image features of the target image, and perform fast number theory transformation on the image features to obtain second transformed features, and take the second transformed features as a first reference template;

[0025] perform autocorrelation operation on the first reference template, and take a result of the autocorrelation operation as a second reference template;

[0026] perform weighted superposition on the first reference template and a first template determined based on the previous frame image to obtain a first template determined based on the current frame image;

[0027] perform weighted superposition on the second reference template and a second template determined based on the previous frame image to obtain a second template determined based on the current frame image.

[0028] Optionally, the scaling of the target position frame according to multiple scales in the scale pool includes:

[0029] obtaining target scales of the tracking target in the previous previous frame image and the previous frame image;

[0030] predicting a scale of the tracking target in the current frame image through Kalman filtering according to the target scales of the tracking target in the previous previous frame image and the previous frame image, to obtain a predicted scale;

[0031] selecting a scale in a neighboring numerical range of the predicted scale from a preset scale pool to form a scale pool corresponding to the current frame image;

[0032] scaling the target position frame according to multiple scales in the scale pool corresponding to the current frame image.

[0033] The second aspect of the present disclosure provides a target tracking device, comprising:

[0034] a target position frame obtaining module configured to obtain a target position frame of a tracking target in a previous frame image of a current frame image;

[0035] a sampling position frame obtaining module configured to scale the target position frame according to multiple scales in a scale pool to obtain a sampling position frame corresponding to each scale;

[0036] an image sampling module configured to sample the current frame image according to the sampling position frame corresponding to each scale to obtain a sampling image corresponding to each scale;

[0037] an image scaling module configured to scale the sampling image to a size of the target position frame of the previous frame image to obtain a candidate image corresponding to each scale;

[0038] a number theory transformation module, configured to extract image features of each of the candidate images and perform fast number theory transformation on the image features to obtain first transformed features;

[0039] a template obtaining module, configured to obtain a first template and a second template determined based on the previous frame image;

[0040] a correlation operation module, configured to perform correlation operation on the first transformed features and the first template, and multiply a result of the correlation operation with the second template to obtain a first operation result;

[0041] a response determining module, configured to perform fast number theory inverse transformation on the first operation result, and take a result of the inverse transformation as a response value corresponding to each of the candidate images;

[0042] a target determining module, configured to determine a maximum response value according to the response value corresponding to each of the candidate images, and determine a target scale and a target position frame corresponding to the tracking target in the current frame image based on a scale and a position of a candidate image corresponding to the maximum response value.

[0043] Optionally, the device further comprises a template updating module, configured to:

[0044] determine whether the maximum response value is greater than a preset threshold;

[0045] if the maximum response value is not greater than the preset threshold, cut a target image from the current frame image based on the target scale and the target position frame corresponding to the tracking target in the current frame image;

[0046] extract image features of the target image, perform fast number theory transformation on the image features to obtain second transformed features, and take the second transformed features as a first reference template;

[0047] perform autocorrelation operation on the first reference template, and take a result of the autocorrelation operation as a second reference template;

[0048] perform weighted superposition on the first reference template, the first template determined based on the previous frame image, and the first template determined based on the first frame image to obtain the first template determined based on the current frame image;

[0049] perform weighted superposition on the second reference template, the second template determined based on the previous frame image, and the second template determined based on the first frame image to obtain the second template determined based on the current frame image.

[0050] Optionally, the template updating module is further configured to:

[0051] if the maximum response value is greater than the preset threshold value, then based on the target scale and target position frame corresponding to the tracking target in the current frame image, a target image is cropped from the current frame image;

[0052] extracting image features of the target image, and performing fast number theory transformation on the image features to obtain second transformation features, and taking the second transformation features as a first reference template;

[0053] performing autocorrelation operation on the first reference template, and taking the autocorrelation operation result as a second reference template;

[0054] performing weighted superposition on the first reference template and a first template determined based on the previous frame image to obtain a first template determined based on the current frame image;

[0055] performing weighted superposition on the second reference template and a second template determined based on the previous frame image to obtain a second template determined based on the current frame image.

[0056] Optionally, the sampling position frame acquisition module comprises:

[0057] a historical scale acquisition module configured to acquire target scales corresponding to the tracking target in the previous previous frame image and the previous frame image;

[0058] a scale prediction module configured to predict a scale of the tracking target in the current frame image by Kalman filtering according to the target scales corresponding to the tracking target in the previous previous frame image and the previous frame image, to obtain a predicted scale;

[0059] a scale pool determination module configured to select scales in a neighboring numerical range of the predicted scale from a preset scale pool to form a scale pool corresponding to the current frame image;

[0060] a scale scaling module configured to scale the target position frame according to multiple scales in the scale pool corresponding to the current frame image.

[0061] The technical solution provided by the present disclosure has the following technical effects:

[0062] In the present technical solution, scale adaptive technology is introduced, and multiple scales in the scale pool are used to search for the optimal target scale, so that the target position frame changes with the target scale, thereby more stably tracking the target. In addition, there are many fast Fourier transforms in the related calculations, in which there are a large number of floating point calculations. The fast number theory transformation is replaced by integer operation, which not only reduces the resource occupation of hardware, but also avoids rounding errors.

[0063] Other features and advantages of the present disclosure will be described in detail in the following specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0064] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings:

[0065] Figure 1 This is a flowchart of a target tracking method in an exemplary embodiment;

[0066] Figure 2 This is yet another flowchart of a target tracking method in an exemplary embodiment;

[0067] Figure 3 This is yet another flowchart of a target tracking method in an exemplary embodiment;

[0068] Figure 4 This is a block diagram of a target tracking device in an exemplary embodiment. Detailed Implementation

[0069] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0070] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with authorization from the owner of the relevant device.

[0071] To address the technical problems mentioned in the background art, this disclosure provides a target tracking method that achieves scale adaptation through scale pooling, so that the bounding box of the tracked target changes with scale, thereby tracking the target more stably and reducing hardware resource consumption.

[0072] Figure 1 A flowchart of a target tracking method in an exemplary embodiment is shown. (Refer to...) Figure 1 The target tracking method includes:

[0073] S101, Obtain the target location bounding box of the tracked target in the previous frame image of the current frame image.

[0074] In the first frame, the user provides the scale and position of the target to be tracked, resulting in the target's bounding box in the first frame image. It's understood that the target doesn't need to be tracked in the first frame; tracking begins in subsequent frames based on the target specified in the first frame. The scale of the first frame is set to 1 by default.

[0075] The target location bounding box of the tracking target in the previous frame of the current frame image can be determined based on steps S101 to S109 in the embodiments of this disclosure.

[0076] S102, scale the target location box according to the scales in the scale pool to obtain the sampling location box corresponding to each scale.

[0077] This scale pool includes multiple scales.

[0078] In some embodiments, the scale pool may be a preset fixed scale pool.

[0079] In some embodiments, the scale pool can be an adaptively varying scale pool.

[0080] In an exemplary embodiment, the above steps may specifically include: obtaining the target scale corresponding to the tracked target in the previous frame image and the previous frame image; predicting the scale of the tracked target in the current frame image by Kalman filtering based on the target scale corresponding to the tracked target in the previous frame image and the previous frame image, and obtaining the predicted scale; selecting scales within the neighboring numerical range of the predicted scale from a preset scale pool to form a scale pool corresponding to the current frame image; and scaling the target location box according to the corresponding scales in the scale pool corresponding to the current frame image.

[0081] S103, sample the image in the current frame according to the sampling position box corresponding to each scale, and obtain the sampled image corresponding to each scale.

[0082] Specifically, sampling is performed on the current frame image according to the sampling location box corresponding to each scale. The sampling process is as follows: the image corresponding to the sampling location box is cropped from the current frame image and used as the sampling image of the corresponding scale.

[0083] S104, scale the sampled image to the size of the target location box of the previous frame image to obtain the candidate image corresponding to each scale.

[0084] S105, for each candidate image, extract the image features of the candidate image, and perform a fast number theory transformation on the image features to obtain the first transformed features.

[0085] It should be noted that the fast number theory transformation in this disclosure is based on one-dimensional fast number theory transformation. For two-dimensional image features, one-dimensional number theory transformation can be performed on each row of the image feature separately, then on each column separately, and finally the results of the row and column number theory transformations are combined. HOG features are three-dimensional image features. For a three-dimensional image feature, such as a 128×128×64 HOG feature, it can be regarded as 64 two-dimensional matrices of size 128×128. Therefore, it is only necessary to perform one-dimensional number theory transformation multiple times for each two-dimensional matrix in the aforementioned manner. Therefore, whether it is a two-dimensional or three-dimensional image feature, it is based on one-dimensional number theory transformation, so it is only necessary to understand the specific process of one-dimensional number theory transformation.

[0086] Understandably, one-dimensional number theory transformations are applied to one-dimensional sequences. In this disclosure, a one-dimensional sequence may be the value of each row or column in an image feature.

[0087] In this disclosure, the process of one-dimensional number theory transformation includes: performing a butterfly transformation on a one-dimensional sequence to obtain the butterfly transformation result, and then performing a positional inversion permutation on the butterfly transformation result to obtain the one-dimensional number theory transformation result.

[0088] For example, a one-dimensional sequence includes x0 to x7. A butterfly transformation is performed on this one-dimensional sequence to obtain the butterfly transformation result:

[0089] X0, X4, X2, X6, X1, X5, X3, X7

[0090] Optionally, the fast number theory transformation is based on the modulo operation. The values ​​in the one-dimensional sequence and the values ​​in the transformation process (including the results of each operation in the butterfly transformation) need to be moduloed. For example, if the modulus is M, the values ​​in the one-dimensional sequence and the results of each operation need to be modulo M, so that the values ​​in the transformation process are in the range of 0 to M-1, which can effectively avoid overflow.

[0091] Then, the butterfly transform result is subjected to bit reversal permutation. For example, the bit reversal permutation formula is as follows:

[0092]

[0093] Where n represents the position of each value in the butterfly transformation result, R(n) represents the position after permutation, R(0)=0, and k represents the power of 2 corresponding to the length of the one-dimensional sequence. In the example above, the length of the one-dimensional sequence is 8, so k=3. This indicates the quotient operation.

[0094] Using the example above, we can obtain the following by performing a bit reversal on the butterfly transformation result:

[0095] R(0)=0; R(1)=4; R(2)=2; R(3)=6; R(4)=1; R(5)=5; R(6)=3; R(7)=7

[0096] Thus, we obtain the result of the one-dimensional number theory transformation:

[0097] X0, X1, X2, X3, X4, X5, X6, X7

[0098] S106, Obtain the first template and the second template determined based on the previous frame image.

[0099] Specifically, a first template, denoted as XF, is obtained based on the previous frame image, and a second template, denoted as α, is obtained based on the previous frame image.

[0100] Understandably, after determining the target scale and target location bounding box of the tracking target in each frame, it is necessary to update the first template XF and the second template α based on the target location bounding box. The update strategy for the first template XF and the second template α will be explained later.

[0101] S107, perform correlation operations between the first transformation feature and the first template, and multiply the correlation operation result with the second template to obtain the first operation result.

[0102] The first transformation feature of the candidate image is denoted as YF. For each candidate image, the first transformation feature YF is correlated with the first template XF, and the correlation result is multiplied by the second template α to obtain the first operation result.

[0103] S108, perform a fast number-theoretic inverse transform on the first operation result, and use the inverse transform result as the response value corresponding to the candidate image.

[0104] The process of the fast number theory inverse transform can be reversed by referring to the forward transform process described above, and will not be explained in detail here.

[0105] S109, determine the maximum response value based on the response value corresponding to each candidate image, and determine the target scale and target location box corresponding to the tracking target in the current frame image based on the scale and position of the candidate image corresponding to the maximum response value.

[0106] The scale of the candidate image corresponding to the maximum response value is the target scale.

[0107] In the above process, scale adaptation technology is introduced. By constructing a scale pool, multiple scales in the pool are used to search for the optimal target scale, so that the target bounding box changes with the target scale, thereby tracking the target more stably. In addition, many fast Fourier transforms are involved in the related calculations, which involve a large number of floating-point calculations. Fast number theory transforms are used to replace them with integer operations, which not only reduces the hardware resource consumption, but also avoids rounding errors.

[0108] Furthermore, during tracking, factors such as non-rigid deformation of the target, scale changes, illumination variations, and occlusion can easily lead to tracking drift. Therefore, the tracking template must be updated during tracking. Understandably, the response value of a candidate image reflects the degree of matching between the candidate image and the sample template. Therefore, the template update strategy can be adjusted by reasonably utilizing the response value. Strengthening the first frame information in the template when the response value is low can improve the robustness of this technical solution against target occlusion.

[0109] Figure 2 Another flowchart of a target tracking method in an exemplary embodiment is shown, such as... Figure 2 As shown, in this method, the first template and the second template can be updated through the following process:

[0110] S201: After determining the target scale and target location box corresponding to the tracking target in the current frame image, determine whether the maximum response value is greater than a preset threshold; if the maximum response value is not greater than the preset threshold, proceed to step S202.

[0111] S202, based on the target scale and target location box corresponding to the tracked target in the current frame image, extract the target image from the current frame image.

[0112] S203, extract the image features of the target image, perform a fast number theory transformation on the image features to obtain the second transformation features, and use the second transformation features as the first reference template.

[0113] The first reference template is denoted as XF'.

[0114] S204, perform autocorrelation operation on the first reference template, and use the autocorrelation operation result as the second reference template.

[0115] Specifically, an autocorrelation operation is performed between the first reference template XF' and the second reference template XF', and the result of the autocorrelation operation is used as the second reference template, denoted as α'.

[0116] S205, the first reference template, the first template determined based on the previous frame image, and the first template determined based on the first frame image are weighted and superimposed to obtain the first template determined based on the current frame image.

[0117] S206, the second reference template, the second template determined based on the previous frame image, and the second template determined based on the first frame image are weighted and superimposed to obtain the second template determined based on the current frame image.

[0118] Figure 3 Another flowchart of a target tracking method in an exemplary embodiment is shown. In an optional embodiment, in step S201, if the maximum response value is greater than a preset threshold, then execution is performed. Figure 3 The steps are shown.

[0119] S301, based on the target scale and target location box corresponding to the tracked target in the current frame image, extract the target image from the current frame image.

[0120] S302, extract the image features of the target image, perform a fast number theory transformation on the image features to obtain the second transformation features, and use the second transformation features as the first reference template.

[0121] S303, perform autocorrelation operation on the first reference template, and use the autocorrelation operation result as the second reference template.

[0122] S304, the first reference template and the first template determined based on the previous frame image are weighted and superimposed to obtain the first template determined based on the current frame image.

[0123] S305, the second reference template and the second template determined based on the previous frame image are weighted and superimposed to obtain the second template determined based on the current frame image.

[0124] based on Figure 2 and Figure 3 The process shown can determine new first and second templates based on the current frame image after determining the target scale and target location box corresponding to the tracked target in the current frame image, so that they can be used to participate in the calculation in the next frame image.

[0125] Understandably, in the above process, when the maximum response value of the current frame image is greater than the preset threshold, it is only necessary to obtain the first template XF of the current frame image based on the first reference template XF' of the current frame image and the first template XF of the previous frame image, and to obtain the second template α of the current frame image based on the second reference template α' of the current frame image and the second template α of the previous frame image. When the maximum response value of the current frame image is not greater than the preset threshold, it is necessary to obtain the first template XF of the current frame image based on the first reference template XF' of the current frame image, the first template XF of the previous frame image, and the first template XF of the first frame image, and to obtain the second template α of the current frame image based on the second reference template α' of the current frame image, the second template α of the previous frame image, and the second template α of the first frame image. Since the first frame information in the template is strengthened under low response value conditions, the robustness against target occlusion can be improved.

[0126] Based on the same inventive concept Figure 4 A block diagram of a target tracking device in an exemplary embodiment is shown, such as Figure 4 As shown, the target tracking device 400 includes:

[0127] The target location box acquisition module 401 is used to acquire the target location box of the tracked target in the previous frame image of the current frame image;

[0128] The sampling location box acquisition module 402 is used to scale the target location box according to the multiple scales in the scale pool to obtain the sampling location box corresponding to each scale.

[0129] The image sampling module 403 is used to sample the current frame image according to the sampling position box corresponding to each scale, so as to obtain the sampled image corresponding to each scale.

[0130] Image scaling module 404 is used to scale the sampled image to the size of the target location box of the previous frame image to obtain candidate images corresponding to each scale;

[0131] The number theory transformation module 405 is used to extract image features of each candidate image and perform a fast number theory transformation on the image features to obtain a first transformation feature;

[0132] The template acquisition module 406 is used to acquire the first template and the second template determined based on the previous frame image;

[0133] The correlation operation module 407 is used to perform correlation operation between the first transformation feature and the first template, and multiply the correlation operation result with the second template to obtain the first operation result;

[0134] The response determination module 408 is used to perform a fast number theory inverse transform on the first calculation result and use the inverse transform result as the response value corresponding to the candidate image.

[0135] The target determination module 409 is used to determine the maximum response value based on the response value corresponding to each candidate image, and to determine the target scale and target location box corresponding to the tracking target in the current frame image based on the scale and position of the candidate image corresponding to the maximum response value.

[0136] Optionally, the target tracking device 400 further includes a template update module for:

[0137] Determine whether the maximum response value is greater than a preset threshold;

[0138] If the maximum response value is not greater than the preset threshold, then the target image is extracted from the current frame image based on the target scale and target location box corresponding to the tracked target in the current frame image;

[0139] The image features of the target image are extracted, and the image features are subjected to fast number theory transformation to obtain second transformation features, and the second transformation features are used as the first reference template;

[0140] Perform an autocorrelation operation on the first reference template, and use the autocorrelation result as the second reference template;

[0141] The first reference template, the first template determined based on the previous frame image, and the first template determined based on the first frame image are weighted and superimposed to obtain the first template determined based on the current frame image.

[0142] The second reference template, the second template determined based on the previous frame image, and the second template determined based on the first frame image are weighted and superimposed to obtain the second template determined based on the current frame image.

[0143] Optionally, the template update module is also used for:

[0144] If the maximum response value is greater than the preset threshold, then the target image is extracted from the current frame image based on the target scale and target location box corresponding to the tracked target in the current frame image;

[0145] The image features of the target image are extracted, and the image features are subjected to fast number theory transformation to obtain second transformation features, and the second transformation features are used as the first reference template;

[0146] Perform an autocorrelation operation on the first reference template, and use the autocorrelation result as the second reference template;

[0147] The first reference template and the first template determined based on the previous frame image are weighted and superimposed to obtain the first template determined based on the current frame image;

[0148] The second reference template and the second template determined based on the previous frame image are weighted and superimposed to obtain the second template determined based on the current frame image.

[0149] Optionally, the sampling location box acquisition module 402 includes:

[0150] The historical scale acquisition module is used to acquire the target scale of the tracked target in the previous frame image and the previous frame image;

[0151] The scale prediction module is used to predict the scale of the tracked target in the current frame image by using Kalman filtering based on the target scale corresponding to the target scale in the previous frame image and the previous frame image, so as to obtain the predicted scale.

[0152] The scale pooling determination module is used to select scales within the neighboring numerical range of the predicted scale from the preset scale pool to form the scale pool corresponding to the current frame image.

[0153] The scaling module is used to scale the target location box according to the corresponding scales in the scale pool corresponding to the current frame image.

[0154] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0155] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and all such simple modifications fall within the protection scope of this disclosure. For example...

[0156] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0157] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A target tracking method, characterized in that, include: Obtain the bounding box of the tracked target in the previous frame image within the current frame image; The target location box is scaled according to the multiple scales in the scale pool to obtain the sampling location box corresponding to each scale. Based on the sampling location box corresponding to each scale, sample the image in the current frame to obtain the sampled image corresponding to each scale; The sampled image is scaled to the size of the target location box of the previous frame image to obtain candidate images for each scale; For each candidate image, image features of the candidate image are extracted, and fast number theory transformation is performed on the image features to obtain the first transformation features; Obtain the first and second templates determined based on the previous frame image; Perform correlation operations between the first transformation feature and the first template, and multiply the correlation operation result by the second template to obtain the first operation result; Perform a fast number-theoretic inverse transform on the first calculation result, and use the inverse transform result as the response value corresponding to the candidate image; The maximum response value is determined based on the response value corresponding to each candidate image, and the target scale and target location box corresponding to the tracking target in the current frame image are determined based on the scale and position of the candidate image corresponding to the maximum response value.

2. The method according to claim 1, characterized in that, The method further includes: Determine whether the maximum response value is greater than a preset threshold; If the maximum response value is not greater than the preset threshold, then the target image is extracted from the current frame image based on the target scale and target location box corresponding to the tracked target in the current frame image; The image features of the target image are extracted, and the image features are subjected to fast number theory transformation to obtain second transformation features, and the second transformation features are used as the first reference template; Perform an autocorrelation operation on the first reference template, and use the autocorrelation result as the second reference template; The first reference template, the first template determined based on the previous frame image, and the first template determined based on the first frame image are weighted and superimposed to obtain the first template determined based on the current frame image. The second reference template, the second template determined based on the previous frame image, and the second template determined based on the first frame image are weighted and superimposed to obtain the second template determined based on the current frame image.

3. The method according to claim 2, characterized in that, The method further includes: If the maximum response value is greater than the preset threshold, then the target image is extracted from the current frame image based on the target scale and target location box corresponding to the tracked target in the current frame image; The image features of the target image are extracted, and the image features are subjected to fast number theory transformation to obtain second transformation features, and the second transformation features are used as the first reference template; Perform an autocorrelation operation on the first reference template, and use the autocorrelation result as the second reference template; The first reference template and the first template determined based on the previous frame image are weighted and superimposed to obtain the first template determined based on the current frame image; The second reference template and the second template determined based on the previous frame image are weighted and superimposed to obtain the second template determined based on the current frame image.

4. The method according to any one of claims 1-3, characterized in that, The step of scaling the target location box according to multiple scales in the scale pool includes: Obtain the target scale of the tracked target in the previous frame and the previous frame; Based on the target scale of the tracked target in the previous frame and the previous frame, the scale of the tracked target in the current frame is predicted by Kalman filtering to obtain the predicted scale. Scales within the neighboring numerical range of the predicted scale are selected from the preset scale pool to form the scale pool corresponding to the current frame image; The target location box is scaled according to the scale pool corresponding to the current frame image.

5. A target tracking device, characterized in that, include: The target location bounding box acquisition module is used to acquire the target location bounding box of the tracked target in the previous frame image of the current frame image; The sampling location box acquisition module is used to scale the target location box according to multiple scales in the scale pool to obtain the sampling location box corresponding to each scale. The image sampling module is used to sample the current frame image according to the sampling position box corresponding to each scale, so as to obtain the sampled image corresponding to each scale. An image scaling module is used to scale the sampled image to the size of the target location box of the previous frame image to obtain candidate images corresponding to each scale; The number theory transformation module is used to extract image features of each candidate image and perform a fast number theory transformation on the image features to obtain a first transformation feature; The template acquisition module is used to acquire the first template and the second template determined based on the previous frame image; The correlation operation module is used to perform correlation operations on the first transformation feature and the first template, and multiply the correlation operation result by the second template to obtain the first operation result; The response determination module is used to perform a fast number-theoretic inverse transform on the first calculation result and use the inverse transform result as the response value corresponding to the candidate image. The target determination module is used to determine the maximum response value based on the response value corresponding to each candidate image, and to determine the target scale and target location box corresponding to the tracked target in the current frame image based on the scale and position of the candidate image corresponding to the maximum response value.

6. The apparatus according to claim 5, characterized in that, The device further includes a template update module for: Determine whether the maximum response value is greater than a preset threshold; If the maximum response value is not greater than the preset threshold, then the target image is extracted from the current frame image based on the target scale and target location box corresponding to the tracked target in the current frame image; The image features of the target image are extracted, and the image features are subjected to fast number theory transformation to obtain second transformation features, and the second transformation features are used as the first reference template; Perform an autocorrelation operation on the first reference template, and use the autocorrelation result as the second reference template; The first reference template, the first template determined based on the previous frame image, and the first template determined based on the first frame image are weighted and superimposed to obtain the first template determined based on the current frame image. The second reference template, the second template determined based on the previous frame image, and the second template determined based on the first frame image are weighted and superimposed to obtain the second template determined based on the current frame image.

7. The apparatus according to claim 6, characterized in that, The template update module is also used for: If the maximum response value is greater than the preset threshold, then the target image is extracted from the current frame image based on the target scale and target location box corresponding to the tracked target in the current frame image; The image features of the target image are extracted, and the image features are subjected to fast number theory transformation to obtain second transformation features, and the second transformation features are used as the first reference template; Perform an autocorrelation operation on the first reference template, and use the autocorrelation result as the second reference template; The first reference template and the first template determined based on the previous frame image are weighted and superimposed to obtain the first template determined based on the current frame image; The second reference template and the second template determined based on the previous frame image are weighted and superimposed to obtain the second template determined based on the current frame image.

8. The apparatus according to any one of claims 5-7, characterized in that, The sampling location box acquisition module includes: The historical scale acquisition module is used to acquire the target scale of the tracked target in the previous frame image and the previous frame image; The scale prediction module is used to predict the scale of the tracked target in the current frame image by using Kalman filtering based on the target scale corresponding to the target scale in the previous frame image and the previous frame image, so as to obtain the predicted scale. The scale pooling determination module is used to select scales within the neighboring numerical range of the predicted scale from the preset scale pool to form the scale pool corresponding to the current frame image. The scaling module is used to scale the target location box according to the corresponding scales in the scale pool corresponding to the current frame image.

Images