Infrared weak and small target tracking system and method based on spatio-temporal information fusion

Abstract

The invention discloses an infrared weak and small target tracking system and method based on spatio-temporal information fusion, and relates to the technical field of infrared detection and target tracking, and the method comprises the steps: building a pixel coordinate system, obtaining the initial position and state of a starting frame target, and completing the tracking initialization; predicting a current target state based on the historical state and determining a prediction area; candidate measurement is generated through a double-layer sliding window in the prediction area, effective measurement is screened, the association probability is calculated by fusing time coherence and space statistical distance, and dynamic adjustment is performed in combination with the measurement relative position; based on the normalized weight of the adjusted probability, the target state is updated through Kalman filtering weighted fusion; the corresponding system comprises an initialization module, a state prediction and region determination module, a measurement and probability processing module and a state updating module, and each module comprises a subdivision unit to realize corresponding functions. The method improves the accuracy and robustness of infrared weak and small target tracking through deep fusion of time-space information, and is suitable for a target tracking task of a complex infrared scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of infrared detection and target tracking, in particular to an infrared dim small target tracking system and method based on spatiotemporal information fusion. BACKGROUND

[0002] Infrared detection technology has been widely used in military reconnaissance, security monitoring, aerospace, precision guidance and other fields due to its advantages of passive detection, strong anti-interference ability and all-weather working. As a key technology of infrared detection system, the main task of infrared dim small target tracking is to lock the target and output its motion state in the infrared image sequence with low signal-to-clutter ratio and strong background interference, which is the premise of subsequent target recognition, decision and attack.

[0003] However, the infrared dim small target itself has the characteristics of weak signal strength, small size, lack of obvious texture and shape features, etc. In addition, the strong interference of complex background makes the traditional tracking method face many challenges. At present, the existing infrared dim small target tracking technology mainly has the following problems: First, the measurement generation precision is insufficient. Traditional measurement methods mostly use a single window for target detection, which is difficult to effectively distinguish the target from the background clutter, and is easy to misjudge the background noise as a candidate target, resulting in an increase in false measurements for subsequent tracking, increasing the computational burden of the tracking system and reducing the tracking reliability.

[0004] Second, the spatiotemporal information fusion is insufficient. Most tracking algorithms only focus on using spatial dimension information (such as the gray difference between target and background) for decision-making, ignoring the temporal coherence characteristics of target motion. When the target appears to be temporarily blocked, changes in posture or the background mutates, tracking drift or even target loss may occur.

[0005] Third, the correlation probability calculation and state update logic are not perfect. In the process of matching the effective measurement and the target, the existing method is not reasonable in the weight distribution of the spatiotemporal distance, which is difficult to accurately reflect the real correlation degree of the measurement and the target. At the same time, in the state update stage, the fusion method of multiple measurement information is simple, and the effective information of each measurement is not fully utilized, resulting in insufficient accuracy and robustness of target state estimation.

[0006] Therefore, how to optimize the measurement generation method, deeply fuse the spatiotemporal information, and perfect the correlation probability calculation and state update mechanism to improve the stability and precision of infrared dim small target tracking in complex background has become a key technical problem to be solved in the field. SUMMARY

[0007] The purpose of the present application is to provide an infrared dim small target tracking system and method based on spatiotemporal information fusion to solve the problems in the prior art.

[0008] To achieve the above object, the application provides the following technical scheme: an infrared dim small target tracking system and method based on space-time information fusion, The method comprises the following steps: Step S1: establishing a coordinate system of an infrared image, obtaining initial position information of a target in a starting frame, determining an initial state of the target, completing tracking initialization, and taking the initial state as a starting reference for subsequent target state prediction; Step S2: predicting a target state at a current time based on a target state at a historical time, and determining a corresponding prediction region in a current frame of infrared image according to the predicted state; Step S3: generating candidate measurements in the prediction region and screening to obtain effective measurements, combining time dimension continuity information and spatial dimension statistical information, calculating an association probability of the effective measurements and the target, and dynamically adjusting the association probability according to the position of the effective measurements relative to the prediction region; Step S4: updating the target state at the current time based on the dynamically adjusted association probability, and taking the update result as a historical state for prediction at the next time.

[0009] Further, step S1 comprises the following steps: Step S1-1: establishing a pixel coordinate system of the infrared image with the upper left corner of the infrared image as the origin, the horizontal right as the x-axis, and the vertical down as the y-axis, and taking the row and column positions of a pixel as the pixel coordinates; Step S1-2: processing the starting frame of infrared image through an external detection algorithm, and obtaining initial pixel coordinates of the infrared dim small target in the starting frame under the pixel coordinate system ; Step S1-3: determining an initial state vector and an initial covariance matrix of the target based on the initial pixel coordinates , wherein the initial state vector contains initial position and velocity information of the target, completing the initialization configuration of the tracking system, and taking the initial state vector and the initial covariance matrix as the starting reference for subsequent target state prediction.

[0010] Further, the step S2 comprises the following steps: Step S2-1: predicting a target prediction state vector and a prediction covariance matrix at the k time through a filtering algorithm based on the target state vector and the covariance matrix at the k-1 time, and the prediction formula is as follows: , Where F is the state transition matrix and Q is the process noise covariance matrix. Let F be the transpose of F, and we obtain the target prediction state at the current moment; Step S2-2: Predict the state vector from the target Extract the predicted pixel coordinates of the target at time k in the pixel coordinate system described in step S1-1. Centered on the predicted pixel coordinates, a rectangular region of a preset size of M×N pixels is set on the infrared image at time k as the prediction region.

[0011] Furthermore, step S3 includes: Step S3-1: Construct a double-layer sliding window within the prediction area determined in step S2-2. The double-layer sliding window includes a central window and a background window. The central window is a target region window with a size of s×s pixels, and the background window is a ring-shaped region window with a size of 3s×3s pixels centered on the central window. Here, s is a positive integer determined based on the preset size of the infrared weak target in the image. Calculate the local contrast response value pixel by pixel using the following formula: ,in, The largest element in the central window. The mean of the elements in the background window that are greater than or equal to the mean. The average value of all elements in the background window; responses with values ​​greater than a preset threshold. The pixels are identified as candidate measurement points, forming a candidate measurement set; Step S3-2: Calculate the Mahalanobis distance between each measurement in the candidate measurement set and the predicted target position at the current time, and set the Mahalanobis distance to a preset gate threshold. The measurement is determined to be a valid measurement if the candidate measurement set is empty or the Mahalanobis distance of all candidate measurements is not less than the preset gate threshold. If so, the valid measurements for the current frame are empty; Step S3-3: For valid measurements, calculate the temporal continuity distance and spatial statistical distance. The formula for calculating the temporal continuity distance is: ,in, Measure the coordinates at time k. Let H be the predicted state of the target at time k-1, and H be the observation matrix; the formula for calculating the spatial statistical distance is: ,in, For the information at time k, S(k) is the transpose of the new information, and S(k) is the covariance of the new information. Let be the inverse matrix of the information covariance; the association probability is calculated by combining the coherence distance in the time dimension and the statistical distance in the spatial dimension. The formula for calculating the spatiotemporal association probability is: ,in, To detect the probability, is the time coherence distance, S(k) represents the innovation covariance of the target t, is the Mahalanobis distance of the measurement at time k and the target, V represents the volume of the target tracking gate; Step S3-4: dynamically adjust the association probability according to the position of the effective measurement relative to the predicted region, if the effective measurement is in the gate, use the space-time association probability described in step S3-3 as the probability value; if there is no effective measurement in the predicted region, calculate the clutter probability based on the missed detection probability and the gate volume, and the clutter probability calculation formula is: , if the measurement is outside the gate, the probability is assigned as zero.

[0012] Further, the step S4 includes: Step S4-1: normalize the dynamically adjusted association probability to generate the association weight corresponding to each effective measurement, and the normalization formula is: , wherein, is the number of effective measurements; Step S4-2: based on the association weight , combine the effective measurement and the target predicted state at the current time, and weight and fuse the state estimates of different measurements through Kalman filtering to update the target state vector and covariance matrix at the current time: Update the target state vector at time k: ; Update the covariance matrix at time k: ; wherein, is the normalized probability corresponding to the missed detection, is the Kalman gain, is the innovation covariance, is the transpose of the Kalman gain, is the state estimate corresponding to the jth measurement, is its transpose, is the fused state estimate of the target, is its transpose, and the update of the target state at the current time is completed.

[0013] The system includes: an initialization module, a state prediction and region determination module, a measurement and probability processing module, and a state update module; The initialization module is used to establish the coordinate system of the infrared image, obtain the initial position information of the target in the starting frame, determine the initial state of the target, and complete the tracking initialization, and the initial state is used as the starting reference for subsequent target state prediction; The state prediction and region determination module is used to predict the target state at the current time based on the target state at the historical time, and determine the corresponding predicted region in the current frame of infrared image according to the predicted state. The measurement and probability processing module is used to generate candidate measurements and filter them to obtain valid measurements within the prediction area. It combines the continuity information in the time dimension and the statistical information in the spatial dimension to calculate the association probability between the valid measurements and the target, and dynamically adjusts the association probability according to the position of the valid measurements relative to the prediction area. The state update module is used to update the target state at the current moment based on the dynamically adjusted association probability, and the update result is used as the historical state predicted for the next moment.

[0014] The initialization module includes: The coordinate system establishment unit is used to establish the pixel coordinate system of the infrared image with the upper left corner of the infrared image as the origin, the horizontal x-axis to the right, and the vertical y-axis downward. The pixel coordinates are represented by the row and column positions of the pixels. The initial coordinate acquisition unit is used to process the infrared image of the starting frame using an external detection algorithm, and to acquire the initial pixel coordinates of the infrared weak target in the starting frame in the pixel coordinate system. ; An initial state determination unit is used to determine the initial state based on the initial pixel coordinates. Determine the initial state vector of the target. and the initial covariance matrix , where the initial state vector Including the target's initial position and velocity information, the initial state vector completes the initial configuration of the tracking system. and the initial covariance matrix This serves as the starting benchmark for subsequent target state prediction.

[0015] The state prediction and region determination module includes: State prediction unit, used for the target state vector at time k-1 Covariance Matrix The target prediction state vector at time k is predicted using a filtering algorithm. and predicting covariance matrix The prediction formula is: , Where F is the state transition matrix and Q is the process noise covariance matrix. Let F be the transpose of F, and we obtain the target prediction state at the current moment; Prediction region determination unit, used to predict state vector from the target. Extract the predicted pixel coordinates of the target at time k in the pixel coordinate system of the coordinate system establishment unit. Centered on the predicted pixel coordinates, a rectangular region of a preset size of M×N pixels is set on the infrared image at time k as the prediction region.

[0016] The measurement and probability processing module includes: A candidate measurement generation unit is used to construct a double-layer sliding window within the prediction region determined by the prediction region determination unit. The double-layer sliding window includes a central window and a background window. The central window is a target region window with a size of s×s pixels, and the background window is a ring-shaped region window with a size of 3s×3s pixels centered on the central window. Here, s is a positive integer determined based on a preset size of the infrared weak target in the image. The local contrast response value is calculated pixel-by-pixel using the following formula: ,in, The largest element in the central window. The mean of the elements in the background window that are greater than or equal to the mean. The average value of all elements in the background window; responses with values ​​greater than a preset threshold. The pixels are identified as candidate measurement points, forming a candidate measurement set; The effective measurement screening unit is used to calculate the Mahalanobis distance between each measurement in the candidate measurement set and the target's predicted position at the current time, and selects measurements whose Mahalanobis distance is less than a preset gate threshold. The measurement is determined to be a valid measurement if the candidate measurement set is empty or the Mahalanobis distance of all candidate measurements is not less than the preset gate threshold. If so, the valid measurements for the current frame are empty; The correlation probability calculation unit is used to calculate the temporal continuity distance and spatial statistical distance for valid measurements. The formula for calculating the temporal continuity distance is as follows: ,in, Measure the coordinates at time k. Let H be the predicted state of the target at time k-1, and H be the observation matrix; the formula for calculating the spatial statistical distance is: ,in, For the information at time k, S(k) is the transpose of the new information, and S(k) is the covariance of the new information. Let be the inverse matrix of the information covariance; the association probability is calculated by combining the coherence distance in the time dimension and the statistical distance in the spatial dimension. The formula for calculating the spatiotemporal association probability is: ,in, For detection probability, It is the temporal coherence distance, and S(k) represents the information covariance of the target t. The Mahalanobis distance to the target is measured at time k, and V represents the volume of the target tracking gate. The correlation probability adjustment unit is used to dynamically adjust the correlation probability based on the position of the effective measurement relative to the prediction area. If the effective measurement is within the gate, the spatiotemporal correlation probability described in the correlation probability calculation unit is used. As a probability value; if there are no effective measurements in the prediction area, the clutter probability is calculated based on the missed detection probability and the gate volume. The clutter probability calculation formula is: If the measurement is outside the gate, the probability is assigned to zero.

[0017] The status update module includes: The weight generation unit is used to normalize the dynamically adjusted association probabilities and generate association weights corresponding to each valid measurement. The normalization formula is as follows: ,in, The number of valid measurements; A state update unit is used to update the state based on the association weight. By combining valid measurements and the target's predicted state at the current moment, a weighted fusion of state estimates from different measurements is performed using Kalman filtering to update the target's state vector and covariance matrix at the current moment. Update the target state vector at time k: ; Update the covariance matrix at time k: ;in, This represents the normalized probability corresponding to a missed detection. For Kalman gain, For the new information covariance, This is the transpose of the Kalman gain. For the state estimate corresponding to the j-th measurement, Transpose it. For the fusion state estimation of the target, Transpose it to update the target state at the current moment.

[0018] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention constructs a two-layer sliding window consisting of a central window and a ring-shaped background window, and combines local contrast response value calculation to screen candidate measurements. This enables accurate differentiation between weak infrared targets and background clutter, effectively reducing the false measurement problem that easily occurs in traditional single-window detection. This not only reduces the redundant burden of subsequent correlation probability calculations but also provides a high-quality measurement foundation for the tracking process, improving the reliability of the initial decision-making of the tracking system. 2. Compared to the shortcomings of traditional algorithms that focus on a single spatial dimension, this invention innovatively integrates temporal coherence distance and spatial statistical distance into the correlation probability calculation and dynamically adjusts the probability allocation. This design enables the system to lock onto the target based on spatiotemporal cooperative features even when faced with sudden background changes, brief target occlusion, or slight changes in attitude, effectively avoiding tracking drift or loss and significantly enhancing tracking robustness in complex infrared scenes; 3. This invention first normalizes the correlation probability, then uses Kalman filtering to weightedly fuse the state estimates of multiple effective measurements. This fully utilizes the effective information from each measurement and avoids the information waste caused by traditional simple fusion methods. Simultaneously, the state update process considers both the probability of missed detection and the reliability of measurements, making the estimation of the target state vector and covariance matrix more closely match the actual motion situation. This significantly improves the estimation accuracy of state parameters such as position and velocity of weak infrared targets, providing accurate data support for subsequent target decision-making. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the infrared weak target tracking system based on spatiotemporal information fusion according to the present invention; Figure 2 This is a schematic diagram of the measurement and probability processing module of the infrared weak target tracking system based on spatiotemporal information fusion of the present invention. Figure 3 This is a flowchart illustrating the infrared weak target tracking method based on spatiotemporal information fusion according to the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example: Figure 1 , Figure 2 and Figure 3 As shown, this invention provides a technical solution: an infrared weak target tracking system and method based on spatiotemporal information fusion. The methods include: Step S1: Establish the coordinate system of the infrared image, obtain the initial position information of the target in the starting frame, determine the initial state of the target, and complete the tracking initialization. The initial state serves as the starting reference for subsequent target state prediction. Step S2: Based on the target state at a historical moment, predict the target state at the current moment, and determine the corresponding prediction area in the current frame infrared image according to the predicted state; Step S3: Generate candidate measurements within the prediction area and filter to obtain valid measurements. Combine the continuity information in the time dimension with the statistical information in the spatial dimension to calculate the association probability between the valid measurements and the target, and dynamically adjust the association probability according to the position of the valid measurements relative to the prediction area. Step S4: Based on the dynamically adjusted association probability, update the target state at the current moment, and use the update result as the predicted historical state for the next moment.

[0022] Furthermore, step S1 includes: Step S1-1: Take the upper left corner of the infrared image as the origin O(0,0), the horizontal direction to the right as the x-axis, and the vertical direction downward as the y-axis. The pixel coordinates are represented by (column number, row number), that is, the coordinates (x,y) of any pixel point correspond to the x-th column and y-th row of the image. Step S1-2: Using an infrared image sequence with a resolution of 640×480 pixels, the target is a small, distant aerial target (approximately 3×3 pixels in size), with a background containing cloud clutter and slight atmospheric scattering interference. An external detection algorithm combining adaptive thresholding segmentation and morphological filtering is used to process the initial frame (k=0) of the infrared image, detecting the initial pixel coordinates of the target. ; Step S1-3: Based on the initial pixel coordinates, set the initial target velocity to 2 pixels / frame horizontally to the right and 1 pixel / frame vertically downwards. Therefore, the initial state vector... The first two digits represent position, and the last two digits represent velocity; initial covariance matrix. The tracking initialization is completed, and this initial state serves as the starting baseline for subsequent predictions. Furthermore, step S2 includes: Step S2-1: For time k=1, based on the state vector at time k-1=0 Covariance Matrix Set the process noise covariance matrix Observation matrix To observe only positional information, the state transition matrix of the uniform motion model is used. The state at time k=1 is predicted using a filtering algorithm: ; ; Step S2-2: From the predicted state vector Extract the predicted pixel coordinates at time k=1. Set the prediction region size to M×N=31×31 pixels. Centered on this coordinate, delineate a 31×31 pixel rectangular region on the infrared image at time k=1 as the prediction region, that is, the region range is x∈[106,137], y∈[66,96].

[0023] Furthermore, step S3 includes: Step S3-1: Within the 31×31 prediction region, set the center window size s=3 (3×3 pixels), and the background window to a 9×9 pixel annular region. Construct a double-layer sliding window and calculate the local contrast response value (SLCM) pixel by pixel; set the local contrast response threshold. Pixels with response values ​​greater than the threshold are identified as candidate measurement points; at coordinates (123, 82), the largest element in the center window... Corresponding threshold Background window average The mean of elements in the background window that are greater than or equal to the mean. Calculated If the value is greater than the threshold of 100, the point is identified as a candidate measurement point; finally, 5 candidate measurements are selected to form a candidate measurement set. ; Step S3-2: Set the elliptic gate threshold The Mahalanobis distance between each candidate measurement and the predicted coordinates (122, 81) was calculated, and measurements with a Mahalanobis distance less than 3 were identified as valid measurements. The calculated Mahalanobis distances for candidate measurements (121, 80), (122, 81), and (123, 82) were 2.1, 1.0, and 2.5, respectively, all less than 3, and were therefore identified as valid measurements. The remaining two candidate measurements had Mahalanobis distances of 3.2 and 3.5, respectively, and were therefore discarded. The final set of valid measurements was... ; Step S3-3: Set the detection probability The specific calculations are as follows: Temporal coherence distance: measured effectively The predicted state of the target at time k-1=0 After transformation by the observation matrix H, it becomes [120, 80], and the calculation yields... Similarly, calculate the other two valid measurements. They are 1.414 and 2.236 respectively; Spatial Statistical Distance: News New information covariance Where R is the observation noise, denoted as Calculated , The determinant is , With the first valid measurement, , =2, The other two valid measurements They are 0 and 1.118 respectively; Elliptic gate volume calculation: based on the obtained new information covariance matrix Using formula Calculate the volume by substituting the data. , where px represents pixels; Spatiotemporal correlation probability: Substitute into the formula to calculate the first valid measurement. Similarly, calculate the other two valid measurements. They are respectively and ; Step S3-4: Since all three effective measurements are within the gate, the calculated values ​​are used directly. As the adjusted probability.

[0024] Furthermore, step S4 includes: Step S4-1: Normalize the association probabilities of the three valid measurements. The sum of the total probabilities is: The normalized weights are respectively , , ; Step S4-2: Kalman Gain ; State vector update: The calculated state estimates for the three measurements are as follows: , , After fusion ; Covariance matrix update: Substituting into the formula yields... ; The updated state vector Covariance Matrix This will be used as the historical state predicted at time k=2 and entered into the tracking loop of the next frame.

[0025] The system includes: an initialization module, a state prediction and region determination module, a measurement and probability processing module, and a state update module; The initialization module is used to establish the coordinate system of the infrared image, obtain the initial position information of the target in the starting frame, determine the initial state of the target, and complete the tracking initialization. The initial state serves as the starting reference for subsequent target state prediction. The state prediction and region determination module is used to predict the target state at the current moment based on the target state at a historical moment, and determine the corresponding prediction region in the current frame infrared image according to the predicted state. The measurement and probability processing module is used to generate candidate measurements and filter them to obtain valid measurements within the prediction area. It combines the continuity information in the time dimension and the statistical information in the spatial dimension to calculate the association probability between the valid measurements and the target, and dynamically adjusts the association probability according to the position of the valid measurements relative to the prediction area. The state update module is used to update the target state at the current moment based on the dynamically adjusted association probability, and the update result is used as the historical state predicted for the next moment.

[0026] The initialization module includes: The coordinate system establishment unit is used to establish the pixel coordinate system of the infrared image with the upper left corner of the infrared image as the origin, the horizontal x-axis to the right, and the vertical y-axis downward. The pixel coordinates are represented by the row and column positions of the pixels. The initial coordinate acquisition unit is used to process the infrared image of the starting frame using an external detection algorithm, and to acquire the initial pixel coordinates of the infrared weak target in the starting frame in the pixel coordinate system. ; An initial state determination unit is used to determine the initial state based on the initial pixel coordinates. Determine the initial state vector of the target. and the initial covariance matrix , where the initial state vector Including the target's initial position and velocity information, the initial state vector completes the initial configuration of the tracking system. and the initial covariance matrix This serves as the starting benchmark for subsequent target state prediction.

[0027] The state prediction and region determination module includes: State prediction unit, used for the target state vector at time k-1 Covariance Matrix The target prediction state vector at time k is predicted using a filtering algorithm. and predicting covariance matrix The prediction formula is: , Where F is the state transition matrix and Q is the process noise covariance matrix. Let F be the transpose of F, and we obtain the target prediction state at the current moment; Prediction region determination unit, used to predict state vector from the target. Extract the predicted pixel coordinates of the target at time k in the pixel coordinate system of the coordinate system establishment unit. Centered on the predicted pixel coordinates, a rectangular region of a preset size of M×N pixels is set on the infrared image at time k as the prediction region.

[0028] The measurement and probability processing module includes: A candidate measurement generation unit is used to construct a double-layer sliding window within the prediction region determined by the prediction region determination unit. The double-layer sliding window includes a central window and a background window. The central window is a target region window with a size of s×s pixels, and the background window is a ring-shaped region window with a size of 3s×3s pixels centered on the central window. Here, s is a positive integer determined based on a preset size of the infrared weak target in the image. The local contrast response value is calculated pixel-by-pixel using the following formula: ,in, The largest element in the central window. The mean of the elements in the background window that are greater than or equal to the mean. The average value of all elements in the background window; responses with values ​​greater than a preset threshold. The pixels are identified as candidate measurement points, forming a candidate measurement set; The effective measurement screening unit is used to calculate the Mahalanobis distance between each measurement in the candidate measurement set and the target's predicted position at the current time, and selects measurements whose Mahalanobis distance is less than a preset gate threshold. The measurement is determined to be a valid measurement if the candidate measurement set is empty or the Mahalanobis distance of all candidate measurements is not less than the preset gate threshold. If so, the valid measurements for the current frame are empty; The correlation probability calculation unit is used to calculate the temporal continuity distance and spatial statistical distance for valid measurements. The formula for calculating the temporal continuity distance is as follows: ,in, Measure the coordinates at time k. Let H be the predicted state of the target at time k-1, and H be the observation matrix; the formula for calculating the spatial statistical distance is: ,in, For the information at time k, S(k) is the transpose of the new information, and S(k) is the covariance of the new information. Let be the inverse matrix of the information covariance; the association probability is calculated by combining the coherence distance in the time dimension and the statistical distance in the spatial dimension. The formula for calculating the spatiotemporal association probability is: ,in, For detection probability, It is the temporal coherence distance, and S(k) represents the information covariance of the target t. The Mahalanobis distance to the target is measured at time k, and V represents the volume of the target tracking gate. The correlation probability adjustment unit is used to dynamically adjust the correlation probability based on the position of the effective measurement relative to the prediction area. If the effective measurement is within the gate, the spatiotemporal correlation probability described in the correlation probability calculation unit is used. As a probability value; if there are no effective measurements in the prediction area, the clutter probability is calculated based on the missed detection probability and the gate volume. The clutter probability calculation formula is: If the measurement is outside the gate, the probability is assigned to zero.

[0029] The status update module includes: The weight generation unit is used to normalize the dynamically adjusted association probabilities and generate association weights corresponding to each valid measurement. The normalization formula is as follows: ,in, The number of valid measurements; A state update unit is used to update the state based on the association weight. By combining valid measurements and the target's predicted state at the current moment, a weighted fusion of state estimates from different measurements is performed using Kalman filtering to update the target's state vector and covariance matrix at the current moment. Update the target state vector at time k: ; Update the covariance matrix at time k: ;in, This represents the normalized probability corresponding to a missed detection. For Kalman gain, For the new information covariance, This is the transpose of the Kalman gain. For the state estimate corresponding to the j-th measurement, Transpose it. For the fusion state estimation of the target, Transpose it to update the target state at the current moment.

[0030] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for tracking weak infrared targets based on spatiotemporal information fusion, characterized in that: Includes the following steps: Step S1: Establish the coordinate system of the infrared image, obtain the initial position information of the target in the starting frame, determine the initial state of the target, and complete the tracking initialization. The initial state serves as the starting reference for subsequent target state prediction. Step S2: Based on the target state at a historical moment, predict the target state at the current moment, and determine the corresponding prediction area in the current frame infrared image according to the predicted state; Step S3: Generate candidate measurements within the prediction area and filter to obtain valid measurements. Combine the continuity information in the time dimension with the statistical information in the spatial dimension to calculate the association probability between the valid measurements and the target, and dynamically adjust the association probability according to the position of the valid measurements relative to the prediction area. Step S4: Based on the dynamically adjusted association probability, update the target state at the current moment, and use the update result as the predicted historical state for the next moment.

2. The infrared weak target tracking method based on spatiotemporal information fusion according to claim 1, characterized in that: Step S1 includes: Step S1-1: Establish a pixel coordinate system for the infrared image with the top left corner as the origin, the x-axis pointing horizontally to the right, and the y-axis pointing vertically downwards. The pixel coordinates are represented by the row and column positions of the pixels. Step S1-2: Process the infrared image of the starting frame using an external detection algorithm to obtain the initial pixel coordinates of the weak infrared target in the starting frame in the pixel coordinate system. ; Step S1-3: Based on the initial pixel coordinates Determine the initial state vector of the target. and the initial covariance matrix , where the initial state vector Including the target's initial position and velocity information, the initial state vector completes the initial configuration of the tracking system. and the initial covariance matrix This serves as the starting benchmark for subsequent target state prediction.

3. The infrared weak target tracking method based on spatiotemporal information fusion according to claim 2, characterized in that, Step S2 includes: Step S2-1: Based on the target state vector at time k-1 Covariance Matrix The target prediction state vector at time k is predicted using a filtering algorithm. and predicting covariance matrix The prediction formula is: , Where F is the state transition matrix and Q is the process noise covariance matrix. Let F be the transpose of F, and we obtain the target prediction state at the current moment; Step S2-2: Predict the state vector from the target Extract the predicted pixel coordinates of the target at time k in the pixel coordinate system described in step S1-1. Centered on the predicted pixel coordinates, a rectangular region of a preset size of M×N pixels is set on the infrared image at time k as the prediction region.

4. The infrared weak target tracking method based on spatiotemporal information fusion according to claim 3, characterized in that, Step S3 includes: Step S3-1: Construct a double-layer sliding window within the prediction area determined in step S2-2. The double-layer sliding window includes a central window and a background window. The central window is a target region window with a size of s×s pixels, and the background window is a ring-shaped region window with a size of 3s×3s pixels centered on the central window. Here, s is a positive integer determined based on the preset size of the infrared weak target in the image. Calculate the local contrast response value pixel by pixel using the following formula: ,in, The largest element in the central window. The mean of the elements in the background window that are greater than or equal to the mean. The average value of all elements in the background window; responses with values ​​greater than a preset threshold. The pixels are identified as candidate measurement points, forming a candidate measurement set; Step S3-2: Calculate the Mahalanobis distance between each measurement in the candidate measurement set and the predicted target position at the current time, and set the Mahalanobis distance to a preset gate threshold. The measurement is determined to be a valid measurement if the candidate measurement set is empty or the Mahalanobis distance of all candidate measurements is not less than the preset gate threshold. If so, the valid measurements for the current frame are empty; Step S3-3: For valid measurements, calculate the temporal continuity distance and spatial statistical distance. The formula for calculating the temporal continuity distance is: ,in, Measure the coordinates at time k. Let H be the predicted state of the target at time k-1, and H be the observation matrix; the formula for calculating the spatial statistical distance is: ,in, For the information at time k, S(k) is the transpose of the new information, and S(k) is the covariance of the new information. Let be the inverse matrix of the information covariance; the association probability is calculated by combining the coherence distance in the time dimension and the statistical distance in the spatial dimension. The formula for calculating the spatiotemporal association probability is: ,in, For detection probability, It is the temporal coherence distance, and S(k) represents the information covariance of the target t. The Mahalanobis distance to the target is measured at time k, and V represents the volume of the target tracking gate. Step S3-4: Dynamically adjust the correlation probability based on the position of the effective measurement relative to the prediction area. If the effective measurement is within the gate, the spatiotemporal correlation probability described in step S3-3 is used. As a probability value; if there are no effective measurements in the prediction area, the clutter probability is calculated based on the missed detection probability and the gate volume. The clutter probability calculation formula is: If the measurement is outside the gate, the probability is assigned to zero.

5. The infrared weak target tracking method based on spatiotemporal information fusion according to claim 4, characterized in that, Step S4 includes: Step S4-1: Normalize the dynamically adjusted association probabilities to generate association weights corresponding to each valid measurement. The normalization formula is as follows: ,in, The number of valid measurements; Step S4-2: Based on the association weight By combining valid measurements and the target's predicted state at the current moment, a weighted fusion of state estimates from different measurements is performed using Kalman filtering to update the target's state vector and covariance matrix at the current moment. Update the target state vector at time k: ; Update the covariance matrix at time k: ;in, This represents the normalized probability corresponding to a missed detection. For Kalman gain, For the new information covariance, This is the transpose of the Kalman gain. For the state estimate corresponding to the j-th measurement, Transpose it. For the fusion state estimation of the target, Transpose it to update the target state at the current moment.

6. An infrared weak target tracking system based on spatiotemporal information fusion, used to execute the infrared weak target tracking method based on spatiotemporal information fusion as described in any one of claims 1-5, characterized in that: The system includes: The module includes an initialization module, a state prediction and region determination module, a measurement and probability processing module, and a state update module. The initialization module is used to establish the coordinate system of the infrared image, obtain the initial position information of the target in the starting frame, determine the initial state of the target, and complete the tracking initialization. The initial state serves as the starting reference for subsequent target state prediction. The state prediction and region determination module is used to predict the target state at the current moment based on the target state at a historical moment, and determine the corresponding prediction region in the current frame infrared image according to the predicted state. The measurement and probability processing module is used to generate candidate measurements and filter them to obtain valid measurements within the prediction area. It combines the continuity information in the time dimension and the statistical information in the spatial dimension to calculate the association probability between the valid measurements and the target, and dynamically adjusts the association probability according to the position of the valid measurements relative to the prediction area. The state update module is used to update the target state at the current moment based on the dynamically adjusted association probability, and the update result is used as the historical state predicted for the next moment.

7. The infrared weak target tracking system based on spatiotemporal information fusion according to claim 6, characterized in that: The initialization module includes: The coordinate system establishment unit is used to establish the pixel coordinate system of the infrared image with the upper left corner of the infrared image as the origin, the horizontal x-axis to the right, and the vertical y-axis downward. The pixel coordinates are represented by the row and column positions of the pixels. The initial coordinate acquisition unit is used to process the infrared image of the starting frame using an external detection algorithm, and to acquire the initial pixel coordinates of the infrared weak target in the starting frame in the pixel coordinate system. ; An initial state determination unit is used to determine the initial state based on the initial pixel coordinates. Determine the initial state vector of the target. and the initial covariance matrix , where the initial state vector Including the target's initial position and velocity information, the initial state vector completes the initial configuration of the tracking system. and the initial covariance matrix This serves as the starting benchmark for subsequent target state prediction.

8. The infrared weak target tracking system based on spatiotemporal information fusion according to claim 7, characterized in that: The state prediction and region determination module includes: State prediction unit, used for the target state vector at time k-1 Covariance Matrix The target prediction state vector at time k is predicted using a filtering algorithm. and predicting covariance matrix The prediction formula is: , Where F is the state transition matrix and Q is the process noise covariance matrix. Let F be the transpose of F, and we obtain the target prediction state at the current moment; Prediction region determination unit, used to predict state vector from the target. Extract the predicted pixel coordinates of the target at time k in the pixel coordinate system of the coordinate system establishment unit. Centered on the predicted pixel coordinates, a rectangular region of a preset size of M×N pixels is set on the infrared image at time k as the prediction region.

9. The infrared weak target tracking system based on spatiotemporal information fusion according to claim 8, characterized in that: The measurement and probability processing module includes: A candidate measurement generation unit is used to construct a double-layer sliding window within the prediction region determined by the prediction region determination unit. The double-layer sliding window includes a central window and a background window. The central window is a target region window with a size of s×s pixels, and the background window is a ring-shaped region window with a size of 3s×3s pixels centered on the central window. Here, s is a positive integer determined based on a preset size of the infrared weak target in the image. The local contrast response value is calculated pixel-by-pixel using the following formula: ,in, The largest element in the central window. The mean of the elements in the background window that are greater than or equal to the mean. The average value of all elements in the background window; responses with values ​​greater than a preset threshold. The pixels are identified as candidate measurement points, forming a candidate measurement set; The effective measurement screening unit is used to calculate the Mahalanobis distance between each measurement in the candidate measurement set and the target's predicted position at the current time, and selects measurements whose Mahalanobis distance is less than a preset gate threshold. The measurement is determined to be a valid measurement if the candidate measurement set is empty or the Mahalanobis distance of all candidate measurements is not less than the preset gate threshold. If so, the valid measurements for the current frame are empty; The correlation probability calculation unit is used to calculate the temporal continuity distance and spatial statistical distance for valid measurements. The formula for calculating the temporal continuity distance is as follows: ,in, Measure the coordinates at time k. Let H be the predicted state of the target at time k-1, and H be the observation matrix; the formula for calculating the spatial statistical distance is: ,in, For the information at time k, S(k) is the transpose of the new information, and S(k) is the covariance of the new information. Let be the inverse matrix of the information covariance; the association probability is calculated by combining the coherence distance in the time dimension and the statistical distance in the spatial dimension. The formula for calculating the spatiotemporal association probability is: ,in, For detection probability, It is the temporal coherence distance, and S(k) represents the information covariance of the target t. The Mahalanobis distance to the target is measured at time k, and V represents the volume of the target tracking gate. The correlation probability adjustment unit is used to dynamically adjust the correlation probability based on the position of the effective measurement relative to the prediction area. If the effective measurement is within the gate, the spatiotemporal correlation probability described in the correlation probability calculation unit is used. As a probability value; if there are no effective measurements in the prediction area, the clutter probability is calculated based on the missed detection probability and the gate volume. The clutter probability calculation formula is: If the measurement is outside the gate, the probability is assigned to zero.

10. The infrared weak target tracking system based on spatiotemporal information fusion according to claim 9, characterized in that: The status update module includes: The weight generation unit is used to normalize the dynamically adjusted association probabilities and generate association weights corresponding to each valid measurement. The normalization formula is as follows: ,in, The number of valid measurements; A state update unit is used to update the state based on the association weight. By combining valid measurements and the target's predicted state at the current moment, a weighted fusion of state estimates from different measurements is performed using Kalman filtering to update the target's state vector and covariance matrix at the current moment. Update the target state vector at time k: ; Update the covariance matrix at time k: ;in, This represents the normalized probability corresponding to a missed detection. For Kalman gain, For the new information covariance, This is the transpose of the Kalman gain. For the state estimate corresponding to the j-th measurement, Transpose it. For the fusion state estimation of the target, Transpose it to update the target state at the current moment.

Images