An infrared dim target tracking method and system based on OSTrack model

Abstract

This invention discloses a method and system for tracking small infrared targets based on the OSTrack model. The method belongs to the field of target tracking technology and includes the following steps: S1: model training; S2: target tracking. This method obtains the position information of small infrared targets by inputting video of the target into the OSTrack-based model for tracking. This method also adds post-processing, improving the tracking accuracy of small infrared targets. Furthermore, the post-processing takes minimal time, achieving real-time tracking. The method is expected to be applicable to remote monitoring and early warning for anti-UAV (unmanned aerial vehicle) systems, with high tracking accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of target tracking, and in particular to an infrared dim small target tracking method and system based on an OSTrack model. Background Art

[0002] The definition of a small infrared target identifies two characteristics of a target: "weak" and "small." "Weak" refers to a low signal-to-noise ratio, poor contrast with the background, and weak infrared radiation intensity; "small" refers to the small number of pixels and size of the target. Therefore, tracking small infrared targets has always been a difficult and hot topic in the field of target tracking, and is widely used in military and civilian applications such as missile early warning systems, precision-guided weapons, maritime surveillance systems, and security monitoring systems. However, due to various factors, tracking small infrared targets remains a significant challenge.

[0003] Currently, target tracking methods are mainly divided into two categories: those based on correlation filtering and convolutional neural networks. Correlation filtering methods have faster execution speeds, but their tracking performance is poor when the target is deformed or occluded. Convolutional neural network-based methods have higher tracking accuracy. However, most methods lack sufficient tracking accuracy for small infrared targets. Therefore, research on high-precision infrared small target tracking algorithms is crucial, and these issues need to be addressed urgently. To this end, a method and system for tracking small infrared targets based on the OSTrack model are proposed. Summary of the Invention

[0004] The technical problem to be solved by the present invention is how to solve the problem of insufficient tracking accuracy in infrared dim small target tracking technology, and provides an infrared dim small target tracking method based on the OSTrack model.

[0005] The present invention solves the above technical problems through the following technical solutions, which include the following steps:

[0006] S1: Model training

[0007] Create an infrared dim target dataset, and feed the dataset into the OSTrack model for training to obtain the trained OSTrack model.

[0008] S2: Target Tracking

[0009] The video to be tracked is input into the trained OSTrack model, the target position is manually selected on the first frame of the video, and then the next frame of the video is input. The target position is obtained using the trained OStrack model, and the target position is post-processed to obtain the final target position. It is then determined whether the tracking is complete. If it is not complete, the next frame of the video is input again to continue obtaining the target position and post-processing it to obtain the final target position. Otherwise, the tracking is complete.

[0010] Furthermore, in step S2, the specific process of post-processing the target position is as follows:

[0011] S41: Calculate the relevant information of the target, then determine whether the camera is moving. If the camera is moving, perform evaluation processing. If the camera is not moving, perform assignment processing, and then go to step S42;

[0012] S42: judging whether an abnormality occurs according to the correlation value in step S41; if no abnormality occurs, proceeding to step S43; if an abnormality occurs, proceeding to step S44;

[0013] S43: When there is no abnormality, that is, the target is not lost and integrated into the background, the target frame is updated in two cases based on the scale factor K, and then the process goes to step S46;

[0014] S44: When an abnormal situation occurs, different abnormal situations are processed respectively to update the target frame, and then go to step S45;

[0015] S45: Determine whether the target is lost; when the target is lost and the camera does not move, update the target frame; when the target is lost and the camera moves, update the target frame; when the target is not lost and the target does not blend into the background, update the target frame; when the target is not lost and the target blends into the background, update the target frame; then go to step S46;

[0016] S46: Determine the target frame, update the target frame, and obtain the final target frame.

[0017] Furthermore, in step S41, the specific process is as follows:

[0018] S411: Calculate the average pixel value of the target area image of the previous frame, the horizontal speed, vertical speed, and total speed of the target in the previous frame and the current frame, wherein the target area image is the image within the target frame;

[0019] S412: If the target speed of the previous frame and the current frame is greater than the threshold A, the camera is considered to have moved, and the target area image pixel ratio value of the previous frame, the target area image pixel change ratio value of the current frame, and the target area image pixel ratio value of the current frame are calculated;

[0020] S413: If at least one of the speeds of the previous frame and the current frame is less than the threshold A, the camera is deemed not to have moved, and the target area image pixel ratio value of the previous frame, the target area image pixel change ratio value of the current frame, and the target area image pixel ratio value of the current frame are set to the default value B.

[0021] Furthermore, in step S411, the horizontal speed is the subtraction of the x-axis pixel coordinate value of the center point of the target frame and the center point of the target frame of the previous frame, and the initial value of the average horizontal speed is 0; the vertical speed is the subtraction of the y-axis pixel coordinate value of the center point of the target frame and the center point of the target frame of the previous frame, and the initial value of the average vertical speed is 0; the speed is the vector sum of the horizontal speed and the vertical speed.

[0022] Furthermore, in step S42, the abnormal conditions are: the camera does not move, the length and width of the target frame are greater than the thresholds Cw and Cy respectively; the camera moves, the image pixels of the target area of ​​the current frame change ratio value, the image pixels of the target area of ​​the current frame change ratio value is greater than the threshold D, and the horizontal and vertical speeds of the target are greater than the thresholds Ex and Ey respectively.

[0023] Furthermore, in step S43, the target frame is updated in two cases: if both the length and width of the target frame are not greater than the corresponding threshold F multiplied by K, the coordinates of the target frame are saved, the target frame is updated, and the horizontal velocity and the average vertical velocity are updated; if one of the length and width of the target frame is greater than the corresponding threshold F multiplied by K, the target frame is updated.

[0024] Furthermore, in the step S44, there are two abnormal situations. When processing the first abnormal situation, the target area image pixel ratio value of the previous frame is recalculated, and the target area image pixel change ratio value of the current frame is recalculated to obtain the maximum target area image pixel change ratio value; then it is necessary to determine whether it is integrated into the background, and it is considered to be integrated into the background in the following two situations: the maximum target area image pixel change ratio value is less than the threshold G; the maximum target area image pixel change ratio value is less than the threshold H and the horizontal speed and the vertical speed are respectively greater than the thresholds Ix and Iy; in other situations, the target is not integrated into the background, and the target is searched in the near, middle, and far areas of the target frame; when processing the second abnormal situation, the target is processed in the same way as the first abnormal situation when it is not integrated into the background, and the target is searched in the near, middle, and far areas of the target frame.

[0025] Furthermore, in step S45, the target is considered lost in the following two cases: the target is integrated into the background, and the pixel change ratio value of the target area image in the current frame is less than the threshold L; the target is not integrated into the background, and the length and width of the target frame are respectively greater than the thresholds Mw and My; in other cases, the target is not lost.

[0026] Furthermore, in step S46, the target frame is judged, and when the length is greater than the threshold N multiplied by the width or the width is greater than the threshold N multiplied by the length, the target frame is updated.

[0027] The present invention also provides an infrared small target tracking system based on the OSTrack model, which is used to perform target tracking using the above tracking method, including:

[0028] The model training module is used to generate a dataset of infrared small targets and send the dataset to the OSTrack model for training to obtain the trained OSTrack model.

[0029] The target tracking module is used to input the video to be tracked into the trained OSTrack model, manually select the target position on the first frame of the video, then input the next frame of the video, use the trained OStrack model to obtain the target position, post-process the target position to obtain the final target position, and determine whether the tracking is completed. If it is not completed, the next frame of the video is input again to continue to obtain the target position and post-process it to obtain the final target position. Otherwise, the tracking is completed.

[0030] Compared with the existing technology, the present invention has the following advantages: the infrared dim small target tracking method based on the OSTrack model obtains the position information of the infrared dim small target by inputting the infrared dim small target video to be tracked into the OSTrack-based model for tracking, thereby increasing post-processing and improving the tracking accuracy of the infrared dim small target; and the post-processing takes only a very short time, achieving real-time tracking, and is expected to be applied to anti-UAV remote monitoring and early warning, with high tracking accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 1 is a flow chart of an infrared dim small target tracking method based on the OStrack model in an embodiment of the present invention;

[0032] Figure 2 This is the overall framework diagram of the OSTrack model in an embodiment of the present invention;

[0033] Figure 3 (a) is the first frame tracking result of three consecutive frames of a public infrared dim target video by the OSTrack model in an embodiment of the present invention;

[0034] Figure 3 (b) is a graph showing the second frame tracking result of three consecutive frames of a public infrared dim target video using the OSTrack model according to an embodiment of the present invention;

[0035] Figure 3(c) is a graph showing the third frame tracking result of three consecutive frames of a public infrared dim target video using the OSTrack model according to an embodiment of the present invention;

[0036] Figure 4 (a) is the first frame tracking result of three consecutive frames of the public infrared dim target video according to the present invention;

[0037] Figure 4 (b) is the second frame tracking result of three consecutive frames of the public infrared dim target video according to the present invention;

[0038] Figure 4 (c) is a graph showing the third frame tracking result of three consecutive frames of a public infrared small target video;

[0039] Figure 5 (a) is the first frame tracking result of three consecutive frames of infrared dim target video shot by the present invention;

[0040] Figure 5 (b) is the second frame tracking result of three consecutive frames of infrared dim target video shot by the present invention;

[0041] Figure 5 (c) is the third frame tracking result of three consecutive frames of infrared dim target video shot by the present invention;

[0042] Figure 6 Schematic diagram of the search area for infrared dim targets in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process. However, the protection scope of the present invention is not limited to the following embodiment. This embodiment is an infrared dim small target tracking method based on the OSTrack model, such as Figure 1 The system used in the tracking method specifically includes two modules:

[0044] The processing of the training module is as follows:

[0045] S11: Create infrared small target dataset;

[0046] Process the infrared dim target videos taken by ourselves and the public infrared dim target videos into the dataset format required by the Ostrack model;

[0047] S12: Input the data set obtained in step S11 into the OSTrack model for training.

[0048] In step S12, during the training process, classification loss and regression loss are used simultaneously; in terms of loss function: for the classification branch, the same weighted focal loss as in CornerNet is used, that is, L cls Indicates that the farther the position is from the GT center, the lower the weight is; for the regression branch, the commonly used IoU loss and L1 loss combination is used, that is, L iou , L1 represents; the overall loss function L track :

[0049] L track =L cls +λ iou L iou +λ L1 L1

[0050] Where: iou =2,λ L1 =5;

[0051] The tracking module processes as follows:

[0052] S21: Input the video to be tracked into the trained OStrack model, and manually select the target position init_rect on the first frame image im of the video;

[0053] In step S21, when manually selecting the target position, the target object should be placed in the center of the selected area as much as possible.

[0054] S22: input the next frame image;

[0055] S23: Use OStrack model to get the target position; Figure 2 , is the overall framework of the OSTrack model, the process from input to output.

[0056] The template image is obtained by the artificially selected target position [x0, y0, w0, h0] on the first frame image im; wherein, the upper left corner of the image is the coordinate origin, the horizontal direction of the image is the x-axis, the vertical direction is the y-axis, (x0, y0) is the pixel coordinate of the upper left corner of the artificially selected target box, w0 is the length of the artificially selected target box, and h0 is the width of the artificially selected target box.

[0057] First, calculate the template image cropping size crop_sz_template = sqrt(w0*h0)*template_factor, where the template image magnification factor template_factor = 2, the x-axis pixel coordinate value of the upper left corner of the enlarged template image x1 = round(x0+0.5*w0-crop_sz*0.5), the x-axis pixel coordinate value of the lower right corner of the enlarged template image x2 = x1+crop_sz, the y-axis pixel coordinate value of the upper left corner of the enlarged template image y1 = round(y0+0.5*h0-crop_sz*0.5), and the y-axis pixel coordinate value of the lower right corner of the enlarged template image y2 = y1+crop_sz. The correction value x1_pad = max(0, -x1) of the x-axis pixel coordinate value of the upper left corner of the enlarged template image, the correction value x2_pad = max(x2-im.shape[1]+1,0) of the x-axis pixel coordinate value of the lower right corner of the enlarged template image, the correction value y1_pad = max(0, -y1) of the y-axis pixel coordinate value of the upper left corner of the enlarged template image, and the correction value y2_pad = max(y2-im.shape[0]+1,0) of the y-axis pixel coordinate value of the lower right corner of the enlarged template image.

[0058] The template image im_crop = im[y1+y1_pad:y2-y2_pad,x1+x1_pad:x2-x2_pad] is cropped from the first frame image im. The template padding image im_crop_padded is obtained using the CV2.copyMakeBorder function. The resolution of im_crop_padded is then resized to 128x128 using the cv2.resize function to obtain the template region image. The template region image is then tiled into multiple patches of size P*P, where P*P is the patch resolution. This entire process is performed only once.

[0059] The search image of the current frame image im_cur is obtained by the target position state_pre of the previous frame = [x_pre, y_pre, w_pre, h_pre] and the initial value [x0, y0, w0, h0].

[0060] First, calculate the search image crop size crop_sz_search = sqrt(w_pre*h_pre)*search_factor, where the search image magnification factor search_factor = 4; then calculate:

[0061] Zoom in on the x-axis pixel coordinate of the upper left corner of the search image x1_cur = round(x_pre + 0.5 * w_pre - crop_sz * 0.5);

[0062] The x-axis pixel coordinate value of the lower right corner of the enlarged search image is x2_cur=x1_cur+crop_sz, and the y-axis pixel coordinate value of the upper left corner of the enlarged search image is y1_cur=round(y_pre+0.5*h_pre-crop_sz*0.5);

[0063] The y-axis pixel coordinate value y2_cur of the lower right corner of the enlarged search image is equal to y1_cur + crop_sz, and the correction value x1_pad_cur of the x-axis pixel coordinate value of the upper left corner of the enlarged search image is equal to max(0, -x1_cur);

[0064] The correction value of the x-axis pixel coordinate value of the lower right corner of the enlarged search image is x2_pad_cur = max(x2_cur-im.shape[1]+1,0);

[0065] A correction value y1_pad_cur = max(0, -y1_cur) for the y-axis pixel coordinate value of the upper left corner of the enlarged search image;

[0066] The correction value y2_pad_cur = max(y2_cur-im.shape[0]+1,0) of the y-axis pixel coordinate value of the upper left corner of the enlarged search image.

[0067] Then the search image is cropped from the current frame image im_cur, im_crop = im_cur[y1+y1_pad:y2-y2_pad,x1+x1_pad:x2-x2_pad]. The search filling image im_crop_padded is obtained using the CV2.copyMakeBorder function. Finally, the resolution of im_crop_padded is adjusted to 256x256 using the cv2.resize function to obtain the search area image.

[0068] Split the search area image into multiple patches of size P*P and expand these patches into vectors, where P*P is the resolution of the patch.

[0069] Feature extraction and relationship modeling are performed on the patches obtained from the template area image and the patches obtained from the search area image. These patches are mapped to D dimensions through the linear projection layer E of the neural network (OStrack model). The learnable position encoding is added to the patch embedding to obtain the token embedding. Finally, the two are concatenated together as the input of the Transformr encoder layer of the OStrack model.

[0070] After being processed by the Transformer encoder layer, the part of its output features corresponding to the search area image is directly extracted and input into the head of the OStrack model. The head consists of three branches: the predicted classification score, the predicted offset value O to compensate for the downsampling quantization error, and the normalized bounding box size S. Each branch is composed of L convolutional layers stacked together. The point with the highest predicted score (x d ,y d ) as the target position. Take the value of the corresponding position in the offset value O and size S to calculate the final target bounding box (i.e., target box), such as Figure 3 As shown, the following expression is the coordinate expression of the target bounding box:

[0071] (x,y,z,w)=(x d +O(0,x d ,y d ),y d +O(1,x d ,y d ),S(0,x d ,y d ),S(1,x d ,y d ))

[0072] Among them, (x, y) represents the pixel coordinates of the upper left corner of the target box, z represents the length of the target box, and w represents the width of the target box.

[0073] S24: Post-process the target position to obtain the final target position;

[0074] The current frame target box output_box_cur = [curx, cury, curw, curh] is obtained in step S23 of the tracking module, (curx, cury) is the pixel coordinate of the upper left corner of the current frame target box, curw and curh are the length and width of the current frame target box respectively; state is the target box after post-processing, state_pre is the target box after post-processing of the previous frame; meanw and meanh are the average length and width of the target box respectively, and the initial values ​​are the length and width of the manually selected target box respectively; meanvx and meanvy are the average horizontal speed and average vertical speed of the target respectively, and the initial value is 0; the target slow movement times slowlymovetimes is defined as the number of frames in which the target blends into the background; i represents the i-th frame of the video, and stepx is defined as the number of steps in the x-axis direction. Step size, stepy, is defined as the step size in the y-axis direction, len_end is defined as the length, r0 is defined as the pixel value of the upper left x-axis corner in state_pre, r1 is defined as the pixel value of the upper left y-axis corner in state_pre, rm is defined as a list of length len_end, and rone is defined as a list of length len_end. The CV2.mean function is used to calculate the mean pixel value mean_object of the target area image in the previous frame. The horizontal velocity v_x of the target in the current frame is the difference between the x-axis pixel coordinates of the target box center and the previous frame center; the vertical velocity v_y is the difference between the y-axis pixel coordinates of the target box center and the previous frame center; the current frame velocity v_cur is the vector sum of v_x and v_y. Similarly, the velocity v_pre of the previous frame is defined. The relationship between v_cur, v_pre, and meanw is used to determine whether the camera has moved. If both v_cur and v_pre are greater than 0.25meanw, the camera is considered to have moved. The camera move times cameramovetimes is defined as the number of frames in which the camera moved. Next, the target area image is processed, and the pixel ratio value pix_change_ratio_pre of the target area image of the previous frame is calculated. That is, the number of pixels whose difference between the pixel values ​​of the target area image of the previous frame and the pixel values ​​of the target area image of the current frame is greater than 15 is accumulated, and the value is divided by the area of ​​the target rectangular frame of the previous frame. The pixel change ratio value pix_change_ratio_cur of the target area image of the current frame is calculated. That is, the number of pixels whose difference between the pixel values ​​of the target area image of the current frame and the pixel values ​​of the target area image of the previous frame is greater than 15 is accumulated, and the value is divided by the area of ​​the target rectangular frame of the current frame. The object pixel ratio value pix_object_ratio_cur of the target area image of the current frame is calculated. That is, the number of pixels whose pixel values ​​of the target area image of the current frame are greater than the mean pixel value mean_object is accumulated, and the value is divided by the area of ​​the target rectangular frame of the current frame.If at least one of v_cur and v_pre is less than 0.25meanw, it is considered that the camera has not moved, and pix_change_ratio_pre, pix_change_ratio_cur, and pix_object_ratio_cur are set to the default value of 0.05;

[0075] Based on the values ​​of meanw, meanh, pix_change_ratio_pre, pix_change_ratio_cur, pix_object_ratio_cur, v_x, v_y, and cameramovetimes, determine whether: (1) the camera does not move, and the curw and curh of the current frame target frame are greater than 1.25meanw and 1.25meanh, respectively. (2) the camera moves, pix_object_ratio_cur is greater than 0.98 and pix_change_ratio_cur is greater than 0.98, and the absolute values ​​of v_x and v_y are greater than 0.5meanw and 0.5meanh, respectively.

[0076] If no abnormality occurs, the target does not blend into the background. Calculate the scaling factor K, k = 1.25 + min(1, cameramovetimes) * 0.25. Update in two cases: (1) curw and curh are not greater than meanw or meanh multiplied by K. Tracking is successful. Add output_box_cur to the list output_boxes and update the relevant values: meanw = meanw + (curw - meanw) / min(i, 20), meanh = meanh + (curh - meanh) / min(i, 20), meanvx = meanvx + (v_x - meanvx) / min(i, 5), meanvy = meanvy + (v_y - meanvy) / min(i, 5). (2) At least one of curw and curh is greater than meanw, and meanh is multiplied by K, then curx = output_boxes[-1][0]+np.sign(meanvx)*min(0.5*meanw,abs(meanvx)), cury = output_boxes[-1][1]+np.sign(meanvy)*min(0.5*meanh,(absmeanvy)), curw = meanw, curh = meanh.

[0077] When the above-mentioned abnormal situation (1) occurs, it is necessary to recalculate pix_change_ratio_pre and pix_change_ratio_cur. Then calculate the maximum ratio value of the pixel change in the target area image, pix_change_max_ratio = max(pix_change_ratio_cur, pix_change_ratio_pre). Then it is necessary to determine whether the target is integrated into the background. In two cases, the target is considered to be integrated into the background: (1) When pix_change_max_ratio is less than 0.01, the target is integrated into the background; (2) pix_change_max_ratio is less than 0.05 and the absolute values ​​of v_x and v_y are greater than 0.2meanw and 0.2meanh respectively, the target is integrated into the background. When the target is integrated into the background, the state of the previous frame is used as output_box_cur. In other cases, if the target is not integrated into the background, the target is searched in the near, middle, and far areas of output_box_cur, such as Figure 6 shown.

[0078] Target box rect = state_pre, (1) When searching for the target in the nearby area, stepx = meanw / 6, stepy = meanh / 6, len_end = 3, r0 = state_pre[0], r1 = state_pre[1], rm = np.random.random(len_end) / 4, rone = np.random.choice([-1,1], size = len_end);

[0079] (2) When searching for the target in the middle area, stepx = meanw / 4, stepy = meanh / 4, len_end = 4, r0 = state_pre[0], r1 = state_pre[1], rm = np.random.random(len_end) / 4, rone = np.random.choice([-1,1], size = len_end);

[0080] (3) When searching for a target in a distant area, stepx=1.25*rect[2] / 9,stepy=1.25*rect[3] / 9,len_end=12,r0=state_pre[0],r1=state_pre[1],rm=np.random.random(len_end) / 3,

[0081] rone = np.random.choice([-1,1],size = len_end). Loop through the values ​​j from 0 to len_end-1, r0 = r0 + stepx, r1 = r1 + stepy, rect[0] = r0 + rm[j] + rone[j] + meanw, rect[1] = r1 + rm[j] + rone[j] + meanh. Calculate the pixel value pix_object at the position of the rect (target frame). That is, if the pixel value of the rect (target frame) image in the current frame is greater than mean_object + 10, accumulate it. pix_lights_max, rectxys is a list. Add pix_object - j * 4 to pix_lights_max, and add the pixel coordinates of the upper left corner of rect (rect[0], rect[1]) to rectxys. After the loop is completed, the maximum pixel brightness pix_light_max = max(pix_lights_max) is calculated, and the index of the maximum pixel brightness idx = pix_lights_max.index(pix_light_max).

[0082] That is, curx=rectxys[idx][0], cury=rectxys[idx][1]], curw=rect[2], curh=rect[3].

[0083] When the above-mentioned abnormal situation (2) occurs, the processing is the same as (1) when the target does not blend into the background. Secondly, determine whether the target is lost. In two cases, the target is considered lost: (1) When the target blends into the background and pix_change_ratio_cur is less than 0.1, the target is lost. (2) The target does not blend into the background, but the length and width of the target frame are greater than 1.5meanw and 1.5meanh respectively, the target is lost. In other cases, the target is not lost. When the target is lost and cameramovetimes=0, the update value is:

[0084] curx=output_boxes[-1][0]+np.sign(meanvx)*min(0.5*meanw,abs(meanvx))*(2+min(10,slowlymovetimes),

[0085] cury=output_boxes[-1][1]+np.sign(meanvy)*min(0.5*meanh,abs(meanvy))*(2+min(10,slowlymovetimes),

[0086] curw=meanw,curh=meanh.

[0087] When the target is lost and cameramovetimes>0, update the value: use the last saved coordinate parameter output_boxes[-1] as output_box_cur. When the target is not lost and does not blend into the background, update the value as if no abnormality occurs. When the target is not lost and blends into the background, update the value:

[0088] curx=output_boxes[-1][0]+np.sign(meanvx)*min(0.2*meanw,abs(meanvx))*(1+min(10,slowlymovetimes),

[0089] cury=output_boxes[-1][1]+np.sign(meanvy)*min(0.2*meanh,abs(meanvy))*(1+min(10,slowlymovetimes),

[0090] curw=meanw,curh=meanh.

[0091] Judge output_box_cur, when curw is greater than 2.5curh or curh is greater than 2.5curw, width = (curw + curh) / 2,

[0092] curw=curh=width, curx=(curx+curw / 2)-width / 2, cury=(cury+curh / 2)-width / 2.

[0093] Update the target position, state = output_box_cur. The tracking result is as follows Figure 4 express.

[0094] It is determined whether the tracking is finished, if not, the process returns to step S22 to continue the process. Otherwise, the tracking is finished.

[0095] The final results of the present invention for tracking the downloaded infrared dim target video and the infrared dim target video shot by the user are as follows: Figure 4 、 5 It can be seen that the present invention has achieved a good tracking effect. Figure 3 , Figure 4 By comparison, it is obvious that the method described in the present invention has a better tracking effect, the tracking frame and the target are highly consistent, and has higher stability.

[0096] It should be noted that, in this embodiment, the correction value is the corrected coordinate value, the search image is cropped from the current frame image, and the search area image is obtained by resizing the search filling image.

[0097] In summary, the infrared small target tracking method based on the OSTrack model of the above embodiment obtains the position information of the infrared small target by inputting the infrared small target video to be tracked into the OSTrack-based model for tracking, thereby adding post-processing and improving the tracking accuracy of the infrared small target; and the post-processing takes only a very short time, achieving real-time tracking, and is expected to be applied to anti-UAV remote monitoring and early warning, with high tracking accuracy.

[0098] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for tracking small infrared targets based on the OSTrack model, characterized in that: The following steps are involved: S1: Model training Create an infrared dim target dataset, and feed the dataset into the OSTrack model for training to obtain the trained OSTrack model. S2: Target Tracking The video to be tracked is fed into the trained OSTrack model. The target position is manually selected on the first frame of the video. The next frame of the video is then fed into the trained OStrack model to obtain the target position. The target position is post-processed to obtain the final target position. The tracking is then judged to be complete. If it is not complete, the next frame of the video is fed into the model again to continue obtaining the target position and post-processing it to obtain the final target position. Otherwise, the tracking is complete. In step S2, the specific process of post-processing the target position is as follows: S41: Calculate the relevant information of the target, then determine whether the camera is moving. If the camera is moving, perform evaluation processing. If the camera is not moving, perform assignment processing, and then go to step S42; S42: judging whether an abnormality occurs according to the correlation value in step S41; if no abnormality occurs, proceeding to step S43; if an abnormality occurs, proceeding to step S44; S43: When there is no abnormality, the target is not lost and is not integrated into the background, the target frame is updated in two cases based on the scale factor K, and then the process goes to step S46; S44: When an abnormal situation occurs, different abnormal situations are processed respectively to update the target frame, and then go to step S45; S45: Determine whether the target is lost; When the target is lost and the camera does not move, update the target frame; When the target is lost and the camera moves, update the target box; When the target is not lost and does not blend into the background, update the target frame; When the target is not lost and is integrated into the background, update the target frame; Then proceed to step S46; S46: Determine the target frame, update the target frame, and obtain the final target frame.

2. The infrared small target tracking method based on the OSTrack model according to claim 1, characterized in that: In step S41, the specific process is as follows: S411: Calculate the average pixel value of the target area image of the previous frame, the horizontal speed, vertical speed, and total speed of the target in the previous frame and the current frame, wherein the target area image is the image within the target frame; S412: If the combined speed of the previous frame and the current frame is greater than the threshold A, the camera is considered to have moved, and the target area image pixel ratio value of the previous frame, the target area image pixel change ratio value of the current frame, and the target area image pixel ratio value of the current frame are calculated; S413: If at least one of the combined velocities of the current frame and the current frame is less than a threshold value A, the camera is deemed not to have moved, and the target area image pixel ratio value of the previous frame, the target area image pixel change ratio value of the current frame, and the target area image pixel ratio value of the current frame are set to a default value B.

3. The infrared small target tracking method based on the OSTrack model according to claim 2, characterized in that: In step S411, the horizontal speed is the subtraction of the x-axis pixel coordinate values ​​of the center point of the target frame and the center point of the target frame of the previous frame, and the initial value of the horizontal speed is 0; the vertical speed is the subtraction of the y-axis pixel coordinate values ​​of the center point of the target frame and the center point of the target frame of the previous frame, and the initial value of the average vertical speed is 0; the total speed is the vector sum of the horizontal speed and the vertical speed.

4. The infrared small target tracking method based on the OSTrack model according to claim 1, characterized in that: In step S42, the abnormal conditions are: the camera does not move, the length and width of the target frame are greater than the thresholds Cw and Cy respectively; the camera moves, the image pixels of the target area of ​​the current frame change ratio, the image pixels of the target area of ​​the current frame change ratio is greater than the threshold D, and the horizontal and vertical speeds of the target are greater than the thresholds Ex and Ey respectively.

5. The infrared small target tracking method based on the OSTrack model according to claim 1, characterized in that: In step S43, the target frame is updated in two cases: if both the length and width of the target frame are not greater than the corresponding threshold F multiplied by K, the coordinates of the target frame are saved, the target frame is updated, and the horizontal velocity and the average vertical velocity are updated; if one of the length and width of the target frame is greater than the corresponding threshold F multiplied by K, the target frame is updated.

6. The infrared small target tracking method based on the OSTrack model according to claim 1, characterized in that: In step S45, the target is considered lost in the following two cases: the target is integrated into the background, and the pixel change ratio of the target area image in the current frame is less than the threshold L; the target is not integrated into the background, and the length and width of the target frame are respectively greater than the thresholds Mw and My; in other cases, the target is not lost.

7. The infrared small target tracking method based on the OSTrack model according to claim 1, characterized in that: In step S46 , the target frame is judged, and when the length is greater than the threshold N multiplied by the width or the width is greater than the threshold N multiplied by the length, the target frame is updated.

8. An infrared small target tracking system based on the OSTrack model, characterized in that: Used to perform target tracking using the tracking method according to any one of claims 1 to 7, comprising: The model training module is used to generate a dataset of infrared small targets and send the dataset to the OSTrack model for training to obtain the trained OSTrack model. The target tracking module is used to input the video to be tracked into the trained OSTrack model, manually select the target position on the first frame of the video, then input the next frame of the video, use the trained OStrack model to obtain the target position, post-process the target position to obtain the final target position, and determine whether the tracking is completed. If it is not completed, the next frame of the video is input again to continue to obtain the target position and post-process it to obtain the final target position. Otherwise, the tracking is completed.

Images