Anti-occlusion target tracking method based on multi-region feature matching

Abstract

The invention relates to an anti-occlusion target tracking method based on multi-region feature matching. The method comprises the following steps: designing trackers including a main tracker, an upper sub-tracker, a lower sub-tracker, a left sub-tracker and a right sub-tracker; and selecting a tracking area, and selecting a target area to be tracked in the initial frame of the video stream or the image sequence. And initializing the main tracker and the sub-tracker, and reading the next frame. Calculating a main tracking state, and if the state of the main tracker is normal, outputting a tracking prediction result by the main tracker; if the main tracker judges that the target is shielded, comprehensive scores of the four sub-trackers are calculated, and the prediction result of the sub-tracker with the highest comprehensive score is selected as the tracking result of the current frame. And finally, updating the Kalman filter.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to an anti-occlusion target tracking method based on multi-region feature matching. Background Technology

[0002] Object tracking is a crucial research area in computer vision. Its main task is to select a target of interest from the first frame of an image sequence or video stream and continuously predict its position and size in subsequent frames. It's important to emphasize that the target category selected in the initial frame can be arbitrary. This process presents numerous challenges, including changes in target size, pose adjustment, occlusion, and interference from complex backgrounds. For applications requiring rapid response to dynamic scenes, real-time performance and accuracy are the core evaluation criteria for object tracking algorithms.

[0003] With the rapid development of deep learning technology, Siamese network-based deep learning object tracking algorithms have gradually become mainstream. These algorithms significantly enhance their adaptability to complex scenes by learning the feature representation of the target. Compared with traditional methods, the Siamese network structure not only utilizes sample data more efficiently but also significantly improves the quality of feature extraction, while balancing algorithm accuracy and operational efficiency. However, the field of object tracking still faces many challenges, such as occlusion, interference from similar targets, and image jitter, thus requiring continuous exploration of new methods to further optimize algorithm performance.

[0004] Occlusion is a common challenge in target tracking tasks. Occlusion occurs when a target is partially or completely blocked by other objects or obstacles, making it partially or completely invisible in an image or video. This can lead to changes in the target's appearance, such as partial disappearance, shape distortion, or color shift. If occlusion is not detected and addressed promptly, these changes can cause tracking algorithms to fail. Furthermore, occlusion can cause biases in target position determination because the target may move during occlusion. If the tracking algorithm does not account for occlusion, it may incorrectly locate the target at the location of the occluder, resulting in tracking errors. In cases of complete occlusion, the algorithm may even fail to locate the target correctly. Therefore, to address occlusion issues, tracking algorithms need strong adaptability to function properly even when the target is partially or completely occluded, thereby improving the system's reliability in complex real-world environments. Summary of the Invention

[0005] Therefore, it is necessary to provide an anti-occlusion target tracking method based on multi-region feature matching to address the above-mentioned technical problems.

[0006] Technical Solution: To address the aforementioned technical problems, this invention proposes an anti-occlusion target tracking method based on multi-region feature matching, the method comprising:

[0007] Step S1: Design the trackers. Design five target trackers, including one master tracker and four sub-trackers: upper sub-tracker, lower sub-tracker, left sub-tracker, and right sub-tracker;

[0008] Step S2: Select the tracking region, in the k=1 frame I of the video stream or image sequence. (1) In the middle, select the tracking region (x (1) ,y (1) ,w (1) ,h (1) ) as a template, where (x (1) ,y (1) ) represents the pixel coordinates of the top-left corner of the tracking region in the image, w (1) and h (1) These are the width and height of the tracking area, respectively;

[0009] Step S3: Initialize the main tracker, and set the tracking area (x (1) ,y (1) ,w (1) ,h (1) The input is fed into the template branch of the main tracker to initialize the Kalman filter and the target's state space vector;

[0010] Step S4: Initialize the sub-tracker, and set the upper half of the tracking region (x (1) ,y (1) ,w (1) ,h (1) / 2) Input to the template branch of the upper sub-tracker, and set the lower half tracking region (x (1) ,y (1) +h (1) / 2,w (1) ,h (1) / 2) Input to the template branch of the next tracker, and set the left half of the tracking region (x (1) ,y (1) ,w (1) / 2,h (1) The input is sent to the template branch of the left child tracker, and the right half of the tracking region (x) is transferred. (1) +w (1) / 2,y (1) ,w (1) / 2,h (1) Input to the template branch of the right child tracker;

[0011] Step S5: Read the next frame image. If the image sequence or video stream has not finished reading all the image frames, let k = k + 1, and read the k-th frame image I.(k) Proceed to step S6; if all image frames of the image sequence or video have been read, proceed to step S11.

[0012] Step S6: Calculate the tracking state of the master tracker, input the search region into the search branch of the master tracker, and calculate the tracking state of the master tracker in this frame. if At this point, it is determined that the tracking status of the main tracker is normal, and the process proceeds to step S7; if... At this point, it is determined that the target is obscured, and the process proceeds to step S8;

[0013] Step S7: The main tracker tracks the image in the k-th frame I. (k) At that time, the prediction results of the master tracker will be used. As the tracking result for this frame, The pixel coordinates of the top-left corner of the main tracker's prediction result in the image. and These represent the width and height of the prediction result from the main tracker, respectively; the tracking result of the upper child tracker is set to... in, The coordinates of the top-left corner of the tracking result from the sub-tracker are the pixel coordinates in the image. and These represent the width and height of the tracking result from the upper child tracker, respectively; the tracking result from the lower child tracker is set to... in, The coordinates of the top-left corner of the sub-tracker's tracking result in the image are the pixel coordinates. and These represent the width and height of the tracking result of the lower child tracker, respectively; the tracking result of the left child tracker is set to... in, The left child tracker tracks the top-left corner of the image, which is the pixel coordinate of the top-left corner. and These represent the width and height of the tracking result for the left child tracker, respectively; the tracking result for the right child tracker is set to... in, The coordinates of the top-left corner of the right child tracker's tracking result in the image are the pixel coordinates. and The width and height of the tracking result of the right child tracker are respectively set, and the process jumps to step S10;

[0014] Step S8: Calculate the tracking state of the sub-tracker, input the search region corresponding to the upper sub-tracker into its search branch, and calculate the tracking state of the upper sub-tracker in this frame. Similarly, calculate the tracking states of the lower sub-tracker, left sub-tracker, and right sub-tracker in that frame. if At this point, it is determined that there is a sub-tracker with a normal tracking status;

[0015] Step S9: Sub-trackers perform tracking, calculating the comprehensive score for each of the four sub-trackers. and And calculate the maximum value among them. Sure The corresponding sub-trackers and their prediction results The prediction result is used as the tracking result for this frame, and the tracking results of each tracker are set according to the prediction result;

[0016] Step S10, Kalman filtering: between two adjacent frames, assuming the target moves in a straight line at a constant speed, in the k-th frame I... (k) In this process, the state space vector of the target is predicted by the state transition matrix, and the Kalman filter parameters are updated according to the tracking state of each tracker.

[0017] Step S11: End tracking. Repeat steps S5 to S10 until all image frames in the image sequence or video stream have been read, and end target tracking.

[0018] Furthermore, during the initialization of the master tracker in step S3, the method further includes:

[0019] Step S301, define the target's 8-dimensional state space vector x (k) for:

[0020]

[0021] Among them, (x (k) ,y (k) Let be the pixel coordinates of the top-left corner of the tracking region in the k-th frame, and let a be the pixel coordinates of that region. (k) The aspect ratio of the target area is calculated as a. (k) =w (k) / h (k) w (k) and h (k) These are the width and height of the target area, respectively. and x (k) y (k) a (k) and h (k) The velocity in the k-th frame;

[0022] Step S302, initialize the Kalman filter when k=1, in image I (1) Select the target region to track (x (1) ,y (1) ,w (1) ,h(1) Using the target region as the tracking template, the target's state filter update value when k=1 is... In and Each is determined by the initial target state x (1) x in (1) y (1) w (1) and h (1) Give, and The value of is 0.

[0023] Furthermore, in step S6, when calculating the tracking state of the master tracker, the method further includes:

[0024] Step S601: Introduce the Average Peak Correlation Energy (APCE) to assist in evaluating the current tracking status. Let the main tracker output response score be in the k-th frame. To determine the size of the output response score, the formula for calculating the APCE of the master tracker in the k-th frame is:

[0025]

[0026] Where 1≤i,j≤os for The response score at position (i,j), where mean(·) represents the mean operation;

[0027] Step S602: Let The APCE value of the master tracker in frame k. The response score of the master tracker in the kth frame. The historical average APCE value of the master tracker over the first k-1 frames. The tracking state of the master tracker in the k-1th frame is defined by the historical average score of the master tracker over the previous k-1 frames. The calculation method is as follows:

[0028]

[0029] Where α1 and α2 are parameters that take values ​​between 0 and 1, if This indicates that the main tracker's tracking status is normal; conversely, if... This indicates that the target being tracked by the main tracker may be occluded.

[0030] Furthermore, when calculating the tracking state of the sub-tracker in step S8, the method further includes:

[0031] Step S801: Let the sub-tracker output response score be in the k-th frame. To determine the size of the output response score, the formula for calculating the APCE of the sub-tracker in the k-th frame is:

[0032]

[0033] Where 1≤i,j≤os for The response score at position (i,j), where mean(·) represents the mean operation;

[0034] Step S802: Let Let $\frac{ ... The score for the response of the sub-tracker in the k-th frame is given. The average APCE value of the upper sub-tracker over the previous k-1 frames. The tracking state of the upper sub-tracker in the k-1th frame is defined as the historical average score of the upper sub-tracker over the previous k-1 frames. The calculation method is as follows:

[0035]

[0036] Where α1 and α2 are parameters that take values ​​between 0 and 1, if This indicates that the tracking status of the upper tracker is normal; conversely, if... This indicates that the target being tracked by the sub-tracker may be in a state of occlusion;

[0037] Step S803: Calculate in step S801 The method is the same; calculate the APCE value of the sub-tracker in the k-th frame. Calculated in step S802 The method is the same; calculate the tracking state of the sub-tracker in the k-th frame.

[0038] Step S804: Calculate in step S801 The method is the same; calculate the APCE value of the left child tracker in the k-th frame. Calculated in step S802 The method is the same; calculate the tracking state of the left sub-tracker in the k-th frame.

[0039] Step S805: Calculate in step S801 The method is the same; calculate the APCE value of the right sub-tracker in the k-th frame. Calculated in step S802 The method is the same; calculate the tracking state of the right sub-tracker in the k-th frame.

[0040] Furthermore, during the sub-tracker tracking in step S9, the method further includes:

[0041] Step S901: Calculated based on steps S801 to S805 and Further calculate the maximum APCE of the four sub-trackers in the k-th frame. and minimum value The calculation method is as follows

[0042]

[0043] Step S902: Normalize the APCE values ​​of the four sub-trackers in the k-th frame, ensuring that their values ​​are distributed between 0 and 1. The normalization calculation method is as follows:

[0044]

[0045] Step S903: Calculate the combined score of the four sub-trackers in the k-th frame. and The calculation method is as follows:

[0046]

[0047] Where β is a parameter that takes a value between 0 and 1;

[0048] Step S904: Calculate the maximum combined score of the four sub-trackers in the k-th frame. Sure The corresponding sub-trackers and their prediction results:

[0049]

[0050] in, The coordinates of the top-left corner of the predicted result from the sub-tracker are the pixel coordinates in the image. and These represent the width and height of the prediction result from the upper tracker, respectively; This refers to the pixel coordinates of the top-left corner of the sub-tracker's prediction result in the image. and These represent the width and height of the sub-tracker's prediction result, respectively; The left child tracker predicts the top-left corner of the pixel in the image, which is the pixel coordinate of that corner. and These represent the width and height of the prediction result from the left child tracker, respectively. The pixel coordinates of the top-left corner of the right child tracker's prediction result in the image. and These represent the width and height of the prediction result from the right child tracker, respectively.

[0051] Step S905: Calculate the prediction results of the sub-tracker. This serves as the tracking result for that frame;

[0052] Step S906: Set the tracking result of the master tracker to In addition, the tracking results of the upper sub-tracker are set as Set the tracking result of the lower-level tracker to Set the tracking result of the left child tracker to Set the tracking result of the right child tracker to

[0053] Furthermore, during Kalman filtering in step S10, the method further includes:

[0054] Step S1001: Let the current state prediction value be... Specifically

[0055]

[0056] in, Let be the pixel coordinates of the top-left corner of the prediction region in the k-th frame within the image. The aspect ratio of the predicted area is calculated as follows: and These represent the width and height of the prediction region, respectively. and They are respectively and The velocity in the k-th frame;

[0057] Let the current filter update value Specifically:

[0058]

[0059] in, Update the pixel coordinates of the top-left corner of the region in the image for the k-th frame. To update the aspect ratio of the region, the calculation method is as follows: and These are the width and height of the update area, respectively. and They are respectively and The velocity in the k-th frame;

[0060] Step S1002: In the kth frame I (k)The state space vector of the target is predicted by the state transition matrix, thus obtaining the predicted state value. The calculation method is as follows:

[0061]

[0062] Where F is the state transition matrix, This is the state filter update value from the previous time step. P is the predicted state value at the current moment. (k-1|k-1) Let P be the covariance matrix of the filtering error at the previous time step. (k|k-1) Let Q be the covariance matrix of the prediction error at the current moment, and let Q be the covariance matrix of the noise.

[0063] Step S1003: If Perform only step S1003; otherwise, proceed to step S1004.

[0064]

[0065] As observations, the tracking results of the master tracker As given in step S7, For the current frame tracking result The velocities in the x, y, a, and h directions are obtained by subtracting the results from the main tracker in the previous frame.

[0066] Calculate the Kalman gain matrix

[0067] K (k) =P (k|k-1) H T HP (k|k-1) H T +R) -1

[0068] Where H is the observation matrix and R is the covariance matrix of the observation noise;

[0069] The predicted values ​​are combined with the observed values ​​using the Kalman gain matrix to obtain the filtered updated values. The calculation method is as follows:

[0070]

[0071] P (k|k ) = P (k|k-1 )-K (k) HP (k | k-1 )

[0072] Step S1004: If and Perform only step S1004; otherwise, proceed to step S1005.

[0073] Will

[0074]

[0075] As an observation, the tracking result of the sub-tracker As given in step S904, For the current frame tracking result The velocities in the x, y, a, and h directions are obtained by subtracting the results from the main tracker in the previous frame.

[0076] Calculate the Kalman gain matrix

[0077] K (k) =P (k|k-1) H T HP (k|k-1) H T +R) -1

[0078] Where H is the observation matrix and R is the covariance matrix of the observation noise;

[0079] The predicted values ​​are combined with the observed values ​​using the Kalman gain matrix to obtain the filtered updated values. The calculation method is as follows

[0080]

[0081] P (k|k) =P (k|k-1) -K (k) HP (k|k-1)

[0082] Step S1005: If For the predicted quantity given by step S1002 make

[0083]

[0084] That is, keeping the predicted height and width of the target area unchanged, let

[0085]

[0086] That is, keeping the predicted values ​​of the acceleration in the x and y directions of the target region constant, The filtered update value, i.e. And will As the prediction result for this frame, the tracking results of each tracker are set according to the prediction result;

[0087] Step S1006: Set the tracking result of the main tracker to In addition, the tracking results of the upper sub-tracker are set as Set the tracking result of the lower-level tracker to Set the tracking result of the left child tracker to Set the tracking result of the right child tracker to Attached Figure Description

[0088] Figure 1 This is a flowchart illustrating an anti-occlusion target tracking method based on multi-region feature matching. Detailed Implementation

[0089] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0090] In one embodiment, such as Figure 1 As shown, this invention proposes an anti-occlusion target tracking method based on multi-region feature matching, the method comprising:

[0091] Step S1: Design the trackers. Design five target trackers, including one master tracker and four sub-trackers: upper sub-tracker, lower sub-tracker, left sub-tracker, and right sub-tracker;

[0092] Step S2: Select the tracking region, in the k=1 frame I of the video stream or image sequence. (1) In the middle, select the tracking region (x (1) ,y (1) ,w (1) ,h (1) ) as a template, where (x (1) ,y (1) ) represents the pixel coordinates of the top-left corner of the tracking region in the image, w (1) and h (1) These are the width and height of the tracking area, respectively;

[0093] Step S3: Initialize the main tracker, and set the tracking area (x (1) ,y (1) ,w (1) ,h (1) The input is fed into the template branch of the main tracker to initialize the Kalman filter and the target's state space vector;

[0094] Step S4: Initialize the sub-tracker, and set the upper half of the tracking region (x (1) ,y (1) ,w (1) ,h (1) / 2) Input to the template branch of the upper sub-tracker, and set the lower half tracking region (x (1) ,y (1) +h(1) / 2,w (1) ,h (1) / 2) Input to the template branch of the next tracker, and set the left half of the tracking region (x (1) ,y (1) ,w (1) / 2,h (1) The input is sent to the template branch of the left child tracker, and the right half of the tracking region (x) is transferred. (1) +w (1) / 2,y (1) ,w (1) / 2,h (1) Input to the template branch of the right child tracker;

[0095] Step S5: Read the next frame image. If the image sequence or video stream has not finished reading all the image frames, let k = k + 1, and read the k-th frame image I. (k) Proceed to step S6; if all image frames of the image sequence or video have been read, proceed to step S11.

[0096] Step S6: Calculate the tracking state of the master tracker, input the search region into the search branch of the master tracker, and calculate the tracking state of the master tracker in this frame. if At this point, it is determined that the tracking status of the main tracker is normal, and the process proceeds to step S7; if... At this point, it is determined that the target is obscured, and the process proceeds to step S8;

[0097] Step S7: The main tracker tracks the image in the k-th frame I. (k) At that time, the prediction results of the master tracker will be used. As the tracking result for this frame, The pixel coordinates of the top-left corner of the main tracker's prediction result in the image. and These represent the width and height of the prediction result from the main tracker, respectively; the tracking result of the upper child tracker is set to... in, The coordinates of the top-left corner of the tracking result from the sub-tracker are the pixel coordinates in the image. and These represent the width and height of the tracking result from the upper child tracker, respectively; the tracking result from the lower child tracker is set to... in, The coordinates of the top-left corner of the sub-tracker's tracking result in the image are the pixel coordinates. and These represent the width and height of the tracking result of the lower child tracker, respectively; the tracking result of the left child tracker is set to... in, The left child tracker tracks the top-left corner of the image, which is the pixel coordinate of the top-left corner. and These represent the width and height of the tracking result for the left child tracker, respectively; the tracking result for the right child tracker is set to... in, The coordinates of the top-left corner of the right child tracker's tracking result in the image are the pixel coordinates. and The width and height of the tracking result of the right child tracker are respectively set, and the process jumps to step S10;

[0098] Step S8: Calculate the tracking state of the sub-tracker, input the search region corresponding to the upper sub-tracker into its search branch, and calculate the tracking state of the upper sub-tracker in this frame. Similarly, calculate the tracking states of the lower sub-tracker, left sub-tracker, and right sub-tracker in that frame. if At this point, it is determined that there is a sub-tracker with a normal tracking status;

[0099] Step S9: Sub-trackers perform tracking, calculating the comprehensive score for each of the four sub-trackers. and And calculate the maximum value among them. Sure The corresponding sub-trackers and their prediction results The prediction result is used as the tracking result for this frame, and the tracking results of each tracker are set according to the prediction result;

[0100] Step S10, Kalman filtering: between two adjacent frames, assuming the target moves in a straight line at a constant speed, in the k-th frame I... (k) In this process, the state space vector of the target is predicted by the state transition matrix, and the Kalman filter parameters are updated according to the tracking state of each tracker.

[0101] Step S11: End tracking. Repeat steps S5 to S10 until all image frames in the image sequence or video stream have been read, and end target tracking.

[0102] Furthermore, during the initialization of the master tracker in step S3, the method further includes:

[0103] Step S301, define the target's 8-dimensional state space vector x (k) for:

[0104]

[0105] Among them, (x (k) ,y (k) Let be the pixel coordinates of the top-left corner of the tracking region in the k-th frame, and let a be the pixel coordinates of that region. (k)The aspect ratio of the target area is calculated as a. (k) =w (k) / h (k) w (k) and h (k) These are the width and height of the target area, respectively. and x (k) y (k) a (k) and h (k) The velocity in the k-th frame;

[0106] Step S302, initialize the Kalman filter when k=1, in image I (1) Select the target region to track (x (1) ,y (1) ,w (1) ,h (1) Using the target region as the tracking template, the target's state filter update value when k=1 is... In and Each is determined by the initial target state x (1) x in (1) y (1) w (1) and h (1) Give, and The value of is 0.

[0107] Furthermore, in step S6, when calculating the tracking state of the master tracker, the method further includes:

[0108] Step S601: Introduce the Average Peak Correlation Energy (APCE) to assist in evaluating the current tracking status. Let the main tracker output response score be in the k-th frame. To determine the size of the output response score, the formula for calculating the APCE of the master tracker in the k-th frame is:

[0109]

[0110] Where 1≤i,j≤os for The response score at position (i,j), where mean(·) represents the mean operation;

[0111] Step S602: Let The APCE value of the master tracker in frame k. The response score of the master tracker in the kth frame. The historical average APCE value of the master tracker over the first k-1 frames. The tracking state of the master tracker in the k-1th frame is defined by the historical average score of the master tracker over the previous k-1 frames. The calculation method is as follows:

[0112]

[0113] Where α1 and α2 are parameters that take values ​​between 0 and 1, if This indicates that the main tracker's tracking status is normal; conversely, if... This indicates that the target being tracked by the main tracker may be occluded.

[0114] Furthermore, when calculating the tracking state of the sub-tracker in step S8, the method further includes:

[0115] Step S801: Let the sub-tracker output response score be in the k-th frame. To determine the size of the output response score, the formula for calculating the APCE of the sub-tracker in the k-th frame is:

[0116]

[0117] Where 1≤i,j≤os for The response score at position (i,j), where mean(·) represents the mean operation;

[0118] Step S802: Let Let $\frac{ ... The score for the response of the sub-tracker in the k-th frame is given. The average APCE value of the upper sub-tracker over the previous k-1 frames. The tracking state of the upper sub-tracker in the k-1th frame is defined as the historical average score of the upper sub-tracker over the previous k-1 frames. The calculation method is as follows:

[0119]

[0120] Where α1 and α2 are parameters that take values ​​between 0 and 1, if This indicates that the tracking status of the upper tracker is normal; conversely, if... This indicates that the target being tracked by the sub-tracker may be in a state of occlusion;

[0121] Step S803: Calculate in step S801 The method is the same; calculate the APCE value of the sub-tracker in the k-th frame. Calculated in step S802 The method is the same; calculate the tracking state of the sub-tracker in the k-th frame.

[0122] Step S804: Calculate in step S801 The method is the same; calculate the APCE value of the left child tracker in the k-th frame. Calculated in step S802 The method is the same; calculate the tracking state of the left sub-tracker in the k-th frame.

[0123] Step S805: Calculate in step S801 The method is the same; calculate the APCE value of the right sub-tracker in the k-th frame. Calculated in step S802 The method is the same; calculate the tracking state of the right sub-tracker in the k-th frame.

[0124] Furthermore, during the sub-tracker tracking in step S9, the method further includes:

[0125] Step S901: Calculated based on steps S801 to S805 and Further calculate the maximum APCE of the four sub-trackers in the k-th frame. and minimum value The calculation method is as follows

[0126]

[0127] Step S902: Normalize the APCE values ​​of the four sub-trackers in the k-th frame, ensuring that their values ​​are distributed between 0 and 1. The normalization calculation method is as follows:

[0128]

[0129] Step S903: Calculate the combined score of the four sub-trackers in the k-th frame. and The calculation method is as follows:

[0130]

[0131] Where β is a parameter that takes a value between 0 and 1;

[0132] Step S904: Calculate the maximum combined score of the four sub-trackers in the k-th frame. Sure The corresponding sub-trackers and their prediction results:

[0133]

[0134] in, The coordinates of the top-left corner of the predicted result from the sub-tracker are the pixel coordinates in the image. and These represent the width and height of the prediction result from the upper tracker, respectively; This refers to the pixel coordinates of the top-left corner of the sub-tracker's prediction result in the image. and These represent the width and height of the sub-tracker's prediction result, respectively; The left child tracker predicts the top-left corner of the pixel in the image, which is the pixel coordinate of that corner. and These represent the width and height of the prediction result from the left child tracker, respectively. The pixel coordinates of the top-left corner of the right child tracker's prediction result in the image. and These represent the width and height of the prediction result from the right child tracker, respectively.

[0135] Step S905: Calculate the prediction results of the sub-tracker. This serves as the tracking result for that frame;

[0136] Step S906: Set the tracking result of the master tracker to In addition, the tracking results of the upper sub-tracker are set as Set the tracking result of the lower-level tracker to Set the tracking result of the left child tracker to Set the tracking result of the right child tracker to

[0137]

[0138] Furthermore, during Kalman filtering in step S10, the method further includes:

[0139] Step S1001: Let the current state prediction value be... Specifically

[0140]

[0141] in, Let be the pixel coordinates of the top-left corner of the prediction region in the k-th frame within the image. The aspect ratio of the predicted area is calculated as follows: and These represent the width and height of the prediction region, respectively. and They are respectively and The velocity in the k-th frame;

[0142] Let the current filter update value Specifically:

[0143]

[0144] in, Update the pixel coordinates of the top-left corner of the region in the image for the k-th frame. To update the aspect ratio of the region, the calculation method is as follows: and These are the width and height of the update area, respectively. and They are respectively and The velocity in the k-th frame;

[0145] Step S1002: In the kth frame I (k) The state space vector of the target is predicted by the state transition matrix, thus obtaining the predicted state value. The calculation method is as follows:

[0146]

[0147] P (k|k-1) =FP (k-1|k-1) F T +Q

[0148] Where F is the state transition matrix, This is the state filter update value from the previous time step. P is the predicted state value at the current moment. (k-1|k-1) Let P be the covariance matrix of the filtering error at the previous time step. (k|k-1) Let Q be the covariance matrix of the prediction error at the current moment, and let Q be the covariance matrix of the noise.

[0149] Step S1003: If Perform only step S1003; otherwise, proceed to step S1004.

[0150]

[0151] As observations, the tracking results of the master tracker As given in step S7, For the current frame tracking result The velocities in the x, y, a, and h directions are obtained by subtracting the results from the main tracker in the previous frame.

[0152] Calculate the Kalman gain matrix

[0153] K (k) =P (k|k-1) H THP (k|k-1) H T +R) -1

[0154] Where H is the observation matrix and R is the covariance matrix of the observation noise;

[0155] The predicted values ​​are combined with the observed values ​​using the Kalman gain matrix to obtain the filtered updated values. The calculation method is as follows:

[0156]

[0157] P (k | k) =P (k | k-1) -K (k) HP (k | k-1)

[0158] Step S1004: If and Perform only step S1004; otherwise, proceed to step S1005.

[0159] Will

[0160]

[0161] As an observation, the tracking result of the sub-tracker As given in step S904, For the current frame tracking result The velocity in the x, y, a, and h directions is obtained by subtracting the result from the master tracker in the previous frame.

[0162] Calculate the Kalman gain matrix

[0163] K (k) =P (k|k-1) H T HP (k|k-1) H T +R) -1

[0164] Where H is the observation matrix and R is the covariance matrix of the observation noise;

[0165] The predicted values ​​are combined with the observed values ​​using the Kalman gain matrix to obtain the filtered updated values. The calculation method is as follows

[0166]

[0167] P (k|k) =P (k|k-1) -K (k) HP(k | k-1)

[0168] Step S1005: If For the predicted quantity given by step S1002 make

[0169]

[0170] That is, keeping the predicted height and width of the target area unchanged, let

[0171]

[0172] That is, keeping the predicted values ​​of the acceleration in the x and y directions of the target region constant, The filtered update value, i.e. And will As the prediction result for this frame, the tracking results of each tracker are set according to the prediction result;

[0173] Step S1006: Set the tracking result of the main tracker to In addition, the tracking results of the upper sub-tracker are set as Set the tracking result of the lower-level tracker to Set the tracking result of the left child tracker to Set the tracking result of the right child tracker to

[0174]

[0175] The aforementioned anti-occlusion target tracking method based on multi-region feature matching, after extensive verification and testing, demonstrates that, when combined with existing target tracking algorithms, it can significantly improve the robustness and accuracy of target tracking algorithms in occluded scenarios, effectively preventing target loss due to occlusion. This method relies solely on the algorithm to implement the anti-occlusion function, reducing system hardware costs and making the target tracking algorithm simpler and lighter in engineering applications. Therefore, the anti-occlusion target tracking method proposed in this invention possesses the characteristics of low cost, low complexity, strong real-time performance, and high stability, enabling continuous target tracking in complex scenarios, and has significant application value and broad application prospects. Compared with traditional single models, this method fully utilizes the matching ability of multi-region features and the advantages of response fusion, exhibiting higher adaptability and accuracy in complex environments, and possessing significant application value and broad application prospects.

[0176] It should be understood that, although Figure 1The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but may be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0177] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered to be within the scope of this specification.

Claims

1. A method for anti-occlusion target tracking based on multi-region feature matching, characterized in that, The method includes: Step S1: Design the trackers. Design five target trackers, including one master tracker and four sub-trackers: upper sub-tracker, lower sub-tracker, left sub-tracker, and right sub-tracker; Step S2: Select the tracking region, in the k=1 frame I of the video stream or image sequence. (1) In the middle, select the tracking region (x (1) ,y (1) ,w (1) ,h (1) ) as a template, where (x (1) ,y (1) ) represents the pixel coordinates of the top-left corner of the tracking region in the image, w (1) and h (1) These are the width and height of the tracking area, respectively; Step S3: Initialize the main tracker, and set the tracking area (x (1) ,y (1) ,w (1) ,h (1) The input is fed into the template branch of the main tracker to initialize the Kalman filter and the target's state space vector; Step S4: Initialize the sub-tracker, and set the upper half of the tracking region (x (1) ,y (1) ,w (1) ,h (1) / 2) Input to the template branch of the upper sub-tracker, and set the lower half tracking region (x (1) ,y (1) +h (1) / 2,w (1) ,h (1) / 2) Input to the template branch of the next tracker, and set the left half of the tracking region (x (1) ,y (1) ,w (1) / 2,h (1) The input is sent to the template branch of the left child tracker, and the right half of the tracking region (x) is transferred. (1) +w (1) / 2,y (1) ,w (1) / 2,h (1) Input to the template branch of the right child tracker; Step S5: Read the next frame image. If the image sequence or video stream has not finished reading all the image frames, let k = k + 1, and read the k-th frame image I. (k) Proceed to step S6; if all image frames of the image sequence or video have been read, proceed to step S11. Step S6: Calculate the tracking state of the master tracker, input the search region into the search branch of the master tracker, and calculate the tracking state of the master tracker in this frame. if At this point, it is determined that the tracking status of the main tracker is normal, and the process proceeds to step S7; if... At this point, it is determined that the target is obscured, and the process proceeds to step S8; Step S7: The main tracker tracks the image in the k-th frame I. (k) At that time, the prediction results of the master tracker will be used. As the tracking result for this frame, The pixel coordinates of the top-left corner of the main tracker's prediction result in the image. and These represent the width and height of the prediction result from the main tracker, respectively; the tracking result of the upper child tracker is set to... in, The coordinates of the top-left corner of the tracking result from the sub-tracker are the pixel coordinates in the image. and These represent the width and height of the tracking result from the upper child tracker, respectively; the tracking result from the lower child tracker is set to... in, The coordinates of the top-left corner of the sub-tracker's tracking result in the image are the pixel coordinates. and These represent the width and height of the tracking result of the lower child tracker, respectively; the tracking result of the left child tracker is set to... in, The left child tracker tracks the top-left corner of the image, which is the pixel coordinate of the top-left corner. and These represent the width and height of the tracking result for the left child tracker, respectively; the tracking result for the right child tracker is set to... in, The coordinates of the top-left corner of the right child tracker's tracking result in the image are the pixel coordinates. and The width and height of the tracking result of the right child tracker are respectively set, and the process jumps to step S10; Step S8: Calculate the tracking state of the sub-tracker, input the search region corresponding to the upper sub-tracker into its search branch, and calculate the tracking state of the upper sub-tracker in this frame. Similarly, calculate the tracking states of the lower sub-tracker, left sub-tracker, and right sub-tracker in that frame. if At this point, it is determined that there is a sub-tracker with a normal tracking status; Step S9: Sub-trackers perform tracking, calculating the comprehensive score for each of the four sub-trackers. and And calculate the maximum value among them. Sure The corresponding sub-trackers and their prediction results The prediction result is used as the tracking result for this frame, and the tracking results of each tracker are set according to the prediction result; Step S10, Kalman filtering: between two adjacent frames, assuming the target moves in a straight line at a constant speed, in the k-th frame I... (k) In this process, the state space vector of the target is predicted by the state transition matrix, and the Kalman filter parameters are updated according to the tracking state of each tracker. Step S11: End tracking. Repeat steps S5 to S10 until all image frames in the image sequence or video stream have been read, and end target tracking.

2. The method according to claim 1, characterized in that, When initializing the master tracker in step S3, the method further includes: Step S301, define the target's 8-dimensional state space vector x (k) for: Among them, (x (k) ,y (k) Let be the pixel coordinates of the top-left corner of the tracking region in the k-th frame, and let a be the pixel coordinates of that region. (k) The aspect ratio of the target area is calculated as a. (k) =w (k) / h (k) w (k) and h (k) These are the width and height of the target area, respectively. and x (k) y (k) a (k) and h (k) The velocity in the k-th frame; Step S302, initialize the Kalman filter when k=1, in image I (1) Select the target region to track (x (1) ,y (1) ,w (1) ,h (1) Using the target region as the tracking template, the target's state filter update value when k=1 is... In and Each is determined by the initial target state x (1) x in (1) y (1) w (1) and h (1) Give, and The value of is 0.

3. The method according to claim 1, characterized in that, When calculating the tracking state of the master tracker in step S6, the method further includes: Step S601: Introduce the Average Peak Correlation Energy (APCE) to assist in evaluating the current tracking status. Let the main tracker output response score be in the k-th frame. To determine the size of the output response score, the formula for calculating the APCE of the master tracker in the k-th frame is: Where 1≤i,j≤os for The response score at position (i,j), where mean(·) represents the mean operation; Step S602: Let The APCE value of the master tracker in frame k. The response score of the master tracker in the kth frame. The historical average APCE value of the master tracker over the first k-1 frames. The tracking state of the master tracker in the k-1th frame is defined by the historical average score of the master tracker over the previous k-1 frames. The calculation method is as follows: Where α1 and α2 are parameters that take values ​​between 0 and 1, if This indicates that the main tracker's tracking status is normal; conversely, if... This indicates that the target being tracked by the main tracker may be occluded.

4. The method according to claim 1, characterized in that, When calculating the tracking state of the sub-tracker in step S8, the method further includes: Step S801: Let the sub-tracker output response score be in the k-th frame. To determine the size of the output response score, the formula for calculating the APCE of the sub-tracker in the k-th frame is: Where 1≤i,j≤os for The response score at position (i,j), where mean(·) represents the mean operation; Step S802: Let Let $\frac{ ... The score for the response of the sub-tracker in the k-th frame is given. The average APCE value of the upper sub-tracker over the previous k-1 frames. The tracking state of the upper sub-tracker in the k-1th frame is defined as the historical average score of the upper sub-tracker over the previous k-1 frames. The calculation method is as follows: Where α1 and α2 are parameters that take values ​​between 0 and 1, if This indicates that the tracking status of the upper tracker is normal; conversely, if... This indicates that the target being tracked by the sub-tracker may be in a state of occlusion; Step S803: Calculate the same steps as in step S801. The method is the same; calculate the APCE value of the sub-tracker in the k-th frame. Calculated in step S802 The method is the same; calculate the tracking state of the sub-tracker in the k-th frame. Step S804: Calculate in step S801 The method is the same; calculate the APCE value of the left child tracker in the k-th frame. Calculated in step S802 The method is the same; calculate the tracking state of the left sub-tracker in the k-th frame. Step S805: Calculate in step S801 The method is the same; calculate the APCE value of the right sub-tracker in the k-th frame. Calculated in step S802 The method is the same; calculate the tracking state of the right sub-tracker in the k-th frame.

5. The method according to claim 1, characterized in that, During step S9, when the sub-tracker is tracking, the method further includes: Step S901: Calculated based on steps S801 to S805 and Further calculate the maximum APCE of the four sub-trackers in the k-th frame. and minimum value The calculation method is as follows Step S902: Normalize the APCE values ​​of the four sub-trackers in the k-th frame, ensuring that their values ​​are distributed between 0 and 1. The normalization calculation method is as follows: Step S903: Calculate the combined score of the four sub-trackers in the k-th frame. and The calculation method is as follows: Where β is a parameter that takes a value between 0 and 1; Step S904: Calculate the maximum combined score of the four sub-trackers in the k-th frame. Sure The corresponding sub-trackers and their prediction results: in, The coordinates of the top-left corner of the predicted result from the sub-tracker are the pixel coordinates in the image. and These represent the width and height of the prediction result from the upper tracker, respectively; This refers to the pixel coordinates of the top-left corner of the sub-tracker's prediction result in the image. and These represent the width and height of the sub-tracker's prediction result, respectively; The left child tracker predicts the top-left corner of the pixel in the image, which is the pixel coordinate of that corner. and These represent the width and height of the prediction result from the left child tracker, respectively. The pixel coordinates of the top-left corner of the right child tracker's prediction result in the image. and These represent the width and height of the prediction result from the right child tracker, respectively. Step S905: Calculate the prediction results of the sub-tracker. This serves as the tracking result for that frame; Step S906: Set the tracking result of the master tracker to In addition, the tracking results of the upper sub-tracker are set as Set the tracking result of the lower-level tracker to Set the tracking result of the left child tracker to Set the tracking result of the right child tracker to 6. The method according to claim 1, characterized in that, In step S10, during Kalman filtering, the method further includes: Step S1001: Let the current state prediction value be... Specifically in, Let be the pixel coordinates of the top-left corner of the prediction region in the k-th frame within the image. The aspect ratio of the predicted area is calculated as follows: and These represent the width and height of the prediction region, respectively. and They are respectively and The velocity in the k-th frame; Let the current filter update value Specifically: in, Update the pixel coordinates of the top-left corner of the region in the image for the k-th frame. To update the aspect ratio of the region, the calculation method is as follows: and These are the width and height of the update area, respectively. and They are respectively and The velocity in the k-th frame; Step S1002: In the kth frame I (k) The state space vector of the target is predicted by the state transition matrix, thus obtaining the predicted state value. The calculation method is as follows: P (k|k-1) =FP (k-1|k-1) F T +Q Where F is the state transition matrix, This is the state filter update value from the previous time step. P is the predicted state value at the current moment. (k-1|k-1) Let P be the covariance matrix of the filtering error at the previous time step. (k|k-1) Let Q be the covariance matrix of the prediction error at the current moment, and let Q be the covariance matrix of the noise. Step S1003: If Perform only step S1003; otherwise, proceed to step S1004. As observations, the tracking results of the master tracker As given in step S7, For the current frame tracking result The velocities in the x, y, a, and h directions are obtained by subtracting the results from the main tracker in the previous frame. Calculate the Kalman gain matrix K (k) =P (k|k-1) H T (HP (k|k-1) H T +R) -1 Where H is the observation matrix and R is the covariance matrix of the observation noise; The predicted values ​​are combined with the observed values ​​using the Kalman gain matrix to obtain the filtered updated values. The calculation method is as follows: P (k|k) =P (k|k-1) -K (k) HP (k|k-1) Step S1004: If and Perform only step S1004; otherwise, proceed to step S1005. Will As an observation, the tracking result of the sub-tracker As given in step S904, For the current frame tracking result The velocities in the x, y, a, and h directions are obtained by subtracting the results from the main tracker in the previous frame. Calculate the Kalman gain matrix K (k) =P (k|k-1) H T (HP (k|k-1) H T +R) -1 Where H is the observation matrix and R is the covariance matrix of the observation noise; The predicted values ​​are combined with the observed values ​​using the Kalman gain matrix to obtain the filtered updated values. The calculation method is as follows P (k|k) =P (k|k-1) -K (k) HP (k|k-1) Step S1005: If For the predicted quantity given by step S1002 make That is, keeping the predicted height and width of the target area unchanged, let That is, keeping the predicted values ​​of the acceleration in the x and y directions of the target region constant, The filtered update value, i.e. And will As the prediction result for this frame, the tracking results of each tracker are set according to the prediction result; Step S1006: Set the tracking result of the master tracker to In addition, the tracking results of the upper sub-tracker are set as Set the tracking result of the lower-level tracker to Set the tracking result of the left child tracker to Set the tracking result of the right child tracker to

Images