Embedded target tracking search area self-adaptive cutting resampling method

Abstract

The invention relates to a self-adaptive cutting and resampling method for an embedded target tracking search area, and belongs to the technical field of computer vision. The invention provides a self-adaptive cutting and resampling method, which comprises the following steps of: when the resolution of a candidate region is relatively small, adopting a method of cutting out the candidate region and then resampling the candidate region, and when the resolution of the candidate region is relatively large, adopting a method of resampling the whole image and then cutting out the candidate region. And the total time consumption of target tracking preprocessing is greatly reduced.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision, and in particular relates to an embedded target tracking search area adaptive clipping and resampling method. Background Art

[0002] In the target tracking task, the algorithm does not search for the target in the entire image, but searches in an area 2-3 times the size of the target near the location where the target appeared in the previous frame. In the preprocessing stage of the target tracking algorithm, the search area is first cropped from the current image, and then the search area is resampled to a fixed size. After completing the above preprocessing, the template area is matched with the search area to determine the location of the target at the current moment. For example, in the SiamRPN target tracking algorithm, the template area size is set to 128×128 and the search area size is set to 256×256. In the preprocessing process of the target tracking algorithm, cropping and resampling operations are required. Embedded devices usually have poor performance, and these two steps take a long time. The application scenario of target tracking usually requires real-time tracking of the target. The long cropping and resampling time is difficult to meet the real-time requirements. Summary of the invention

[0003] 1. Technical issues to be resolved

[0004] The technical problem to be solved by the present invention is to design a target tracking search area adaptive clipping and resampling method to reduce the total time consumption of target tracking preprocessing.

[0005] (II) Technical solution

[0006] In order to solve the above technical problems, the present invention provides an embedded target tracking search area adaptive cropping and resampling method, which is implemented based on an embedded processing chip; when the resolution of the candidate area in the target tracking search area is small, a method of first cropping the candidate area and then resampling the candidate area is adopted for processing; when the resolution of the candidate area is large, a method of first resampling the entire image and then cropping the candidate area is adopted for processing, thereby realizing adaptive cropping and resampling.

[0007] Preferably, the method specifically comprises the following steps:

[0008] (1) Calculate the side length of the target tracking search area, that is, the square candidate area in the target area:

[0009]

[0010] Among them, S is the side length of the candidate area, N is the multiple of the candidate area area relative to the target tracking search area area, w and h are the width and height of the target area at the previous moment respectively;

[0011] (2) If S ≤ K, execute step 3, otherwise execute step 4; K is a preset constant threshold;

[0012] (3) According to the side length calculated in step (1), the candidate region is cropped from the original image, and then the candidate region is resampled to a resolution of L×L; the resolution of the original image is S*S, and L is the resolution size input to the target tracking neural network, and the target tracking neural network is used to find the target in the candidate region and regress the target size; at this point, the processing process is completed and the flow ends;

[0013] (4) Resample the original image, and the target resolution of the resampled image is calculated as follows:

[0014]

[0015] w1×h1 is the target resolution of resampling, w1, h1 are the width and height of the resampled image, w0×h0 is the original resolution of the original image, w0, h0 are the width and height of the original image; then the candidate area is cropped from the original image; at this point, the processing is completed and the process ends.

[0016] The present invention further provides a target tracking method implemented based on the method.

[0017] The present invention also provides a system for implementing the method.

[0018] (III) Beneficial effects

[0019] The present invention provides an embedded target tracking search area adaptive cropping and resampling method. When the resolution of the candidate area is small, a method of first cropping the candidate area and then resampling the candidate area is adopted. When the resolution of the candidate area is large, a method of first resampling the entire image and then cropping the candidate area is adopted, which greatly reduces the total time consumption of target tracking preprocessing. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 The figure is a time-consuming test comparison diagram of implementing target tracking preprocessing on the HiSilicon 3516AV300 chip based on the method of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, content, and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the accompanying drawings and examples.

[0022] The time taken for a crop operation is proportional to the number of rows in the cropped area, and the time taken for a resampling operation is proportional to the target resolution.

[0023] If the candidate area is cropped first and then resampled, the cropping operation time can be expressed as aS, where a is a constant determined by the chip and memory performance, and S is the side length of the cropped area; the resampling operation has a fixed time consumption, which is bL 2 , b is a constant determined by the chip performance, and L is the input resolution of the target tracking neural network. In this way, the preprocessing time can be expressed as:

[0024] t=aS+bL 2

[0025] Among them, a, b, and L are all constants. Therefore, in this method, the preprocessing time t is linearly related to the side length S of the candidate region. When the resolution of the candidate region is small, the preprocessing time is short, and when the resolution of the candidate region is large, the preprocessing takes more time.

[0026] If the entire image is resampled first and then the candidate region is cropped, the resampling time is The cropping operation takes a fixed amount of time, which is aL. In this way, the preprocessing time can be expressed as:

[0027]

[0028] Among them, a, b, L, w0, and h0 are all constants. In this way, the preprocessing time t is the inverse of the square of the side length of the candidate region. When the resolution of the candidate region is large, the preprocessing time is short, and when the resolution of the candidate region is small, the cropping operation takes more time.

[0029] Therefore, the present invention comprehensively considers the above two methods. When the resolution of the candidate area is small, a method of first cropping the candidate area and then resampling the candidate area is adopted (corresponding to the following step (3)). When the resolution of the candidate area is large, a method of first resampling the entire image and then cropping the candidate area is adopted (corresponding to the following step (4)).

[0030] Based on the above design ideas, the present invention proposes an embedded target tracking search area adaptive cropping and resampling method, and its specific steps are:

[0031] (1) Calculate the side length of the square candidate area in the target tracking search area:

[0032]

[0033] Here, S is the side length of the candidate region, N is the multiple of the area of ​​the candidate region relative to the area of ​​the target region (i.e., the target tracking search region), S and N are constant hyperparameters, which are the same for all moments, preferably, N is 2; w and h are the width and height of the target region at the previous moment, respectively;

[0034] (2) If S≤K, execute step 3, otherwise execute step 4; K is a preset constant threshold, preferably, K is 384;

[0035] (3) According to the side length calculated in step 1, the candidate region is cropped from the original image, and then the candidate region is resampled to a resolution of L×L; the resolution of the original image is S*S, and L is the resolution size input to the target tracking neural network, which is used to find the target in the candidate region and regress the target size. Preferably, L is 256. At this point, the preprocessing process of target tracking is completed and the process ends.

[0036] (4) Resample the original image, and the target resolution of the resampled image is calculated as follows:

[0037]

[0038] w1×h1 is the target resolution of the resampled image, w1, h1 are the width and height of the resampled image, w0×h0 is the original resolution of the original image, w0, h0 are the width and height of the original image; then the candidate area is cropped from the original image. At this point, the preprocessing process of target tracking is completed and the process ends.

[0039] The present invention compares and verifies the present invention and the conventional target tracking preprocessing method on the HiSilicon 3516AV300 chip. The resampling operation is performed in the HiSilicon IVE image resampling dedicated module. Figure 1 As shown, when the candidate area is large, the preprocessing time of cropping first and then resampling is long. Similarly, when the candidate area is small, the preprocessing time of resampling first and then cropping is also long. However, when using the method of the present invention, as the size of the candidate area increases, the preprocessing time first increases and then decreases, and the total preprocessing time can be controlled within 12ms.

[0040] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An embedded target tracking search area adaptive cropping and resampling method, characterized in that: The method is implemented based on an embedded processing chip; when the resolution of the candidate area in the target tracking search area is small, the candidate area is first cropped out and then resampled for processing; when the resolution of the candidate area is large, the entire image is first resampled and then the candidate area is cropped out to achieve adaptive cropping and resampling.

2. The method according to claim 1, characterized in that The method specifically comprises the following steps: (1) Calculate the side length of the target tracking search area, that is, the square candidate area in the target area: Among them, S is the side length of the candidate area, N is the multiple of the candidate area area relative to the target tracking search area area, w and h are the width and height of the target area at the previous moment respectively; (2) If S ≤ K, execute step 3, otherwise execute step 4; K is a preset constant threshold; (3) According to the side length calculated in step (1), the candidate region is cropped from the original image, and then the candidate region is resampled to a resolution of L×L; the resolution of the original image is S*S, and L is the resolution size input to the target tracking neural network, and the target tracking neural network is used to find the target in the candidate region and regress the target size; at this point, the preprocessing process of target tracking is completed and the process ends; (4) Resample the original image, and the target resolution of the resampled image is calculated as follows: w1×h1 is the target resolution of the resampled image, w1, h1 are the width and height of the resampled image, w0×h0 is the original resolution of the original image, w0, h0 are the width and height of the original image; then the candidate area is cropped from the original image; at this point, the preprocessing process of target tracking is completed and the process ends.

3. The method according to claim 2, characterized in that S and N are constant hyperparameters, and their values ​​are the same at all times.

4. The method according to claim 2, characterized in that The value of N is 2.

5. The method according to claim 2, characterized in that The value of K is 384.

6. The method according to claim 2, characterized in that The value of L is 256.

7. The method according to any one of claims 1 to 6, characterized in that: This method is applied in the field of computer vision technology.

8. A target tracking method implemented based on the method according to any one of claims 1 to 6.

9. A system for implementing the method according to any one of claims 1 to 6.

10. The system according to claim 9, characterized in that The system is applied in the field of computer vision technology.

Images