An Adaptive Cropping and Resampling Method for Embedded Target Tracking Search Region

Abstract

The present invention relates to an embedded target tracking search area adaptive cropping and resampling method, belonging to the field of computer vision technology. The present invention provides an adaptive cropping and resampling method. When the resolution of the candidate region is small, the method first crops out the candidate region and then resamples the candidate region. When the resolution of the candidate region is large, the method first resamples the entire image and then crops the candidate region. This method significantly reduces the total time consumed in target tracking preprocessing.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to an embedded target tracking search area adaptive clipping and resampling method. Background Art

[0002] In the target tracking task, the algorithm does not search for the target in the entire image, but searches in an area 2-3 times the size of the target, near the location where the target appeared in the previous frame. In the preprocessing stage of the target tracking algorithm, the search area is first cropped from the current image, and then the search area is resampled to a fixed size. After completing the above preprocessing, the template area is matched with the candidate target area to determine the location of the target at the current moment. For example, in the SiamRPN target tracking algorithm, the template area size is set to , the search area size is set to During the preprocessing of target tracking algorithms, cropping and resampling operations are required. Embedded devices typically have poor performance, and these two steps take a long time. However, target tracking applications typically require real-time tracking, and the long cropping and resampling time makes it difficult to meet real-time requirements. Summary of the Invention

[0003] (1) Technical issues to be resolved

[0004] The technical problem to be solved by the present invention is to design a target tracking search area adaptive clipping and resampling method to reduce the total time consumption of target tracking preprocessing.

[0005] (2) Technical solution

[0006] In order to solve the above technical problems, the present invention provides an embedded target tracking search area adaptive cropping and resampling method, which is implemented based on an embedded processing chip; when the resolution of the candidate area in the target tracking search area is small, a method of first cropping the candidate area and then resampling the candidate area is adopted for processing; when the resolution of the candidate area is large, a method of first resampling the entire image and then cropping the candidate area is adopted for processing, thereby realizing adaptive cropping and resampling.

[0007] Preferably, the method specifically comprises the following steps:

[0008] (1) Calculate the side length of the target tracking search area, that is, the square candidate area in the target area:

[0009]

[0010] Among them, S is the side length of the candidate region, It is the multiple of the candidate area relative to the target tracking search area. 、 are the width and height of the target area at the previous moment respectively;

[0011] (2) If , then execute step 3, otherwise execute step 4; is the preset constant threshold;

[0012] (3) According to the side length calculated in step (1), the candidate region is cropped from the original image, and then the candidate region is resampled to The resolution of the original image is S*S, is the resolution size input to the target tracking neural network, which is used to find the target in the candidate area and regress the target size; at this point, the processing process is completed and the process ends;

[0013] (4) Resample the original image, and the target resolution of the resample is calculated as follows:

[0014]

[0015] is the target resolution of the resampling, w1, h1 are the width and height of the resampled image, is the original resolution of the original image, w0 and h0 are the width and height of the original image; then the candidate area is cropped from the original image; at this point, the processing is completed and the process ends.

[0016] The present invention further provides a target tracking method implemented based on the method.

[0017] The present invention also provides a system for implementing the method.

[0018] (3) Beneficial effects

[0019] The present invention provides an embedded target tracking search area adaptive cropping and resampling method. When the resolution of the candidate area is small, the method of first cropping the candidate area and then resampling the candidate area is adopted. When the resolution of the candidate area is large, the method of first resampling the entire image and then cropping the candidate area is adopted, which greatly reduces the total time consumption of target tracking preprocessing. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 The figure is a comparison chart of the time consumption of the target tracking preprocessing implemented on the HiSilicon 3516AV300 chip based on the method of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, content, and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below with reference to the accompanying drawings and examples.

[0022] The time taken for a crop operation is proportional to the number of rows in the cropped area, and the time taken for a resampling operation is proportional to the target resolution.

[0023] If the candidate area is cropped first and then resampled, the cropping operation time can be expressed as , is a constant determined by the chip and memory performance, is the side length of the cropping area; the resampling operation is a fixed time consuming operation. , is a constant determined by the chip performance, is the input resolution of the target tracking neural network. In this way, the preprocessing time can be expressed as:

[0024]

[0025] in, 、 、 are all constants, so the preprocessing time in this way is The side length of the candidate region When the resolution of the candidate region is small, the preprocessing time is short. When the resolution of the candidate region is large, the preprocessing time is long.

[0026] If the entire image is resampled first and then the candidate region is cropped, the resampling time is , the cropping operation is a fixed time-consuming operation, which takes In this way, the preprocessing time can be expressed as:

[0027]

[0028] in, 、 、 、 、 are all constants. In this way, the preprocessing time The inverse of the square of the side length of the candidate region When the resolution of the candidate region is large, the preprocessing time is short. When the resolution of the candidate region is small, the cropping operation takes more time.

[0029] Therefore, the present invention comprehensively considers the above two methods. When the resolution of the candidate area is small, the method of first cropping the candidate area and then resampling the candidate area is adopted (corresponding to step (3) below). When the resolution of the candidate area is large, the method of first resampling the entire image and then cropping the candidate area is adopted (corresponding to step (4) below).

[0030] Based on the above design ideas, the present invention proposes an embedded target tracking search area adaptive cropping and resampling method, the specific steps of which are:

[0031] (1) Calculate the side length of the square candidate area in the target tracking search area:

[0032]

[0033] here, is the side length of the candidate region, It is the multiple of the candidate area relative to the target area (i.e., target tracking search area). S and N are constant hyperparameters that are the same at all times. Preferably, Take 2; 、 are the width and height of the target area at the previous moment respectively;

[0034] (2) If , then execute step 3, otherwise execute step 4; is a preset constant threshold, preferably, Take 384;

[0035] (3) According to the side length calculated in step 1, the candidate region is cropped from the original image, and then the candidate region is resampled to The resolution of the original image is S*S, is the resolution size input to the target tracking neural network, which is used to find the target in the candidate area and regress the target size. Preferably, Take 256. At this point, the preprocessing process of target tracking is completed and the process ends.

[0036] (4) Resample the original image, and the target resolution of the resampling is calculated as follows:

[0037]

[0038] is the target resolution of the resampling, w1, h1 are the width and height of the resampled image, The original resolution of the original image is w0, h0 is the width and height of the original image; the candidate region is then cropped from the original image. At this point, the target tracking preprocessing process is completed and the process ends.

[0039] The present invention and conventional target tracking preprocessing methods are compared and verified on the HiSilicon 3516AV300 chip. The resampling operation is performed in the HiSilicon IVE image resampling dedicated module. Figure 1 As shown in the figure, when the candidate area is large, the preprocessing time of cropping first and then resampling is long. Similarly, when the candidate area is small, the preprocessing time of resampling first and then cropping is also long. However, when using the method of the present invention, as the size of the candidate area increases, the preprocessing time first increases and then decreases, and the total preprocessing time can be controlled within 12ms.

[0040] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An embedded target tracking search area adaptive cropping and resampling method, characterized in that: This method is based on an embedded processing chip. When the resolution of the target tracking search area is small, the search area is first cropped and then resampled. When the resolution of the search area is large, the whole image is resampled and then cropped. The method of processing the area is used to achieve adaptive cropping and resampling; The method specifically comprises the following steps: (1) Calculate the side length of the search area: Where S is the side length of the search area, is the multiple of the side length of the search area relative to the side length of the target area, 、 are the width and height of the target area at the previous moment respectively; (2) If , then execute step 3, otherwise execute step 4; is the preset constant threshold; (3) Cut out the search area from the original image according to the side length calculated in step (1), and then resample the search area to The resolution of the search area is S*S, is the resolution size input to the target tracking neural network, which is used to find the target in the search area and regress the target size; at this point, the preprocessing process of target tracking is completed and the process ends; (4) Resample the original image, and the target resolution of the resampling is calculated as follows: is the target resolution of the resampling, w1, h1 are the width and height of the image area after resampling, is the original resolution of the original image, w0, h0 are the width and height of the original image; then crop out from the original image area; at this point, the preprocessing process of target tracking is completed and the process ends.

2. The method according to claim 1, wherein N is a constant hyperparameter that has the same value at all times.

3. The method according to claim 1, wherein The value of N is 2.

4. The method according to claim 1, wherein The value is 384.

5. The method according to claim 1, wherein The value is 256.

6. A target tracking method implemented based on the method according to any one of claims 1 to 5.

Images