An infrared sea surface target multi-frame detection method and system based on deep time series modeling

Abstract

This invention provides a method and system for multi-frame infrared sea surface target detection based on deep temporal modeling. Belonging to the field of image processing and target detection, it is suitable for continuous detection and tracking of weak infrared targets under complex sea conditions. The method relies on the collaborative processing logic of single-frame detection preprocessing, temporal state prediction, cross-frame association matching, and target trajectory decision and management modules. It sequentially executes single-frame detection and redundancy filtering, temporal state prediction, prediction-guided cross-frame association matching, and target trajectory decision and management steps. Through deep temporal modeling, it mines the temporal features of target movement and existence, guides cross-frame association with predicted position, and constructs a unified judgment mechanism for joint confidence verification. This achieves bidirectional optimization of missing target completion and false detection suppression, while simultaneously implementing full lifecycle management of target trajectories. This invention effectively solves the problem of both missed detections and false detections in existing algorithms, improving the continuity and accuracy of weak infrared sea surface target detection under complex sea conditions.

Description

Technical Field

[0001] This invention relates to the field of image processing and target detection technology, and more particularly to a method and system for multi-frame detection of infrared sea surface targets based on depth temporal modeling. Background Technology

[0002] Infrared imaging technology, due to its all-weather operation and strong adaptability to complex environments, has become an important technical means in the field of sea surface target detection. This technology forms images by receiving the difference in thermal radiation between the target and the background, effectively overcoming the limitations of visible light imaging under conditions such as nighttime and fog. In infrared imaging systems, sea surface targets typically appear as small spots with gray values ​​slightly higher than the background, and their detection performance is limited by inherent characteristics such as small target size and low signal-to-noise ratio. Simultaneously, the sea surface environment contains various high-intensity interference sources such as dynamic waves, solar flares, and cloud edges. These interferences are highly similar to the appearance of the real target in a single frame image, posing a serious challenge to traditional detection methods based on single-frame analysis.

[0003] Existing technologies primarily employ detection algorithms based on spatial filtering or temporal accumulation. Spatial methods, such as Top-hat transform and local contrast measurement, can suppress some background noise, but lack modeling of target motion patterns across multiple frames, making it difficult to distinguish between real targets with continuous motion characteristics and transient noise. Temporal methods, such as three-frame difference and trajectory correlation, introduce temporal information, but are limited by fixed thresholds or simple linear motion assumptions, easily leading to trajectory breaks or false correlations in complex scenarios such as wave transients and target occlusion. More critically, existing algorithms generally lack a temporal propagation model of target existence probability. They cannot use high-confidence detection from historical frames to fill in missed targets in the current frame, nor do they have a cross-frame consistency verification mechanism for falsely detected targets, resulting in a significant decrease in system detection stability under low signal-to-noise ratio conditions. Summary of the Invention

[0004] To address the aforementioned problems in existing technologies, this invention aims to design a multi-frame infrared sea surface target detection method based on depth temporal modeling that can fully utilize temporal information and improve detection continuity and accuracy.

[0005] The technical means employed in this invention are as follows:

[0006] A multi-frame infrared sea surface target detection method based on depth temporal modeling includes: S1. Perform target detection frame by frame on the input infrared image sequence to obtain several candidate detection boxes, and then perform redundancy filtering on each candidate detection box to obtain a stable detection result for a single frame. S2. Extract the most recent consecutive multi-frame historical state information of each tracked target in the system, input it into the deep temporal modeling network, output the predicted displacement vector of each target in the current frame and the prediction existence confidence, and obtain the predicted center position of each target in the current frame based on the predicted displacement vector. S3. Based on the single-frame stable detection results obtained in step S1 and the target trajectory prediction state obtained in step S2, construct a cost matrix. Use the optimal allocation algorithm to solve the cost matrix to realize the association and matching between the current frame detection results and the historical target trajectories, and obtain three types of matching results: successfully matched detection-trajectory pairs, unmatched detection results, and unmatched trajectory predictions. S4. Based on the three types of matching results obtained in step S3, perform multi-level decision-making to complete the target trajectory status update, missing detection completion, false detection suppression, new trajectory initialization, and failed trajectory termination management, and output reliable infrared sea surface target detection results with continuous identity identifiers.

[0007] Further, step S1 includes: S11. Let the current processing frame be the [number]th frame. For each frame, perform single-frame object detection to obtain several candidate detection boxes, each containing a spatial location. Detection confidence And category information; S12. Perform redundancy filtering on each candidate detection box, including: Calculate the coordinates of the center point of each detection box The formula is as follows:

[0008] Calculate the Euclidean distance between the center points of any two detection boxes. The formula is as follows:

[0009] in,( ), ( ) are respectively the first The, the The coordinates of the center point of each detection box; Compare the Euclidean distance with a preset spatial threshold, and discard those with a distance less than the preset spatial threshold. The candidate detection boxes with lower confidence are selected, and the retained detection boxes are the single-frame stable detection results.

[0010] Furthermore, the preset spatial threshold It is 10 to 20 pixels.

[0011] Further, step S2 includes: S21. For target trajectories that have been established in the system and are continuously updated, maintain their most recent continuous... Historical displacement sequence and historical detection confidence sequence of frame window; S22. When the trajectory history length reaches the window, the historical information is input into a deep learning-based temporal prediction network for joint modeling, and the predicted displacement of the current frame is output. The confidence level of the prediction exists. ,in , used to represent the temporal probability that the trajectory continues to exist in the current frame; S23. The predicted center position of the current frame is obtained based on the predicted displacement, as follows:

[0012] in, This represents the actual center position of the target in the previous frame.

[0013] Further, step S3 includes: S31. Let the number of detection boxes in the current frame be... The number of target trajectories is When constructing the cost matrix, the first The center of the first detection frame and the first The cost of a trajectory prediction center is defined as:

[0014] when Greater than the preset association threshold When this happens, set it to infinity to eliminate unreasonable matches; S32. Solve the cost matrix using the optimal allocation algorithm to obtain the matching relationship between the current frame detection and the historical trajectory, and form three types of results: successfully matched detection-trajectory pairs, unmatched detection boxes, and unmatched trajectory predictions.

[0015] Furthermore, the preset association threshold It is 10 to 20 pixels.

[0016] Further, step S4 includes: S41. For successfully matched detection – the trajectory pair – update the trajectory status and introduce a trajectory confirmation mechanism to divide the trajectory into confirmed and unconfirmed trajectories, wherein: When the trajectory successfully matches the number of frames consecutively satisfy If the trajectory is confirmed, it is marked as a confirmed trajectory. A confirmed trajectory indicates that the target has been stably observed over a continuous period of time, and its identity and movement trend are more reliable. A dynamic confidence threshold is applied to confirmed trajectories to improve the retention of weakly salient targets under complex sea conditions.

[0017] in, Based on the detection threshold, Take a value of 0.3 to 0.5; S42. Construct a unified output rule based on joint confidence verification, eliminating the need to distinguish complex intermediate states. The system directly detects the confidence level based on the single-frame detection of the current frame. There is a confidence level with the prediction output of the time series model. Perform joint determination; S43. Let the single-frame detection threshold be... The prediction threshold is Establish a joint confidence verification logic, and the system will finally output the target bounding box. Determined by the following piecewise function:

[0018] in, This indicates a virtual completion box; S44. For detection boxes that do not match any historical trajectories, to avoid frequent initialization of new trajectories due to background noise causing false detection accumulation, a spatiotemporal consistency constraint is applied based on the occurrence time and spatial location, as follows: Initialization to a new trajectory is only allowed within the first few frames of the video sequence, or when the center point of the detection box is located in the left or right edge region of the image; otherwise, the detection results lack temporal continuity support, are judged as false targets and filtered out, thereby reducing the false detection rate caused by complex sea state background interference. S45. The system introduces a lifecycle management mechanism for the target trajectory, and performs unified management of trajectory generation, confirmation, loss and termination.

[0019] Furthermore, the joint confidence verification logic is as follows: For scenarios where the target appears stably, the system implements a dual confirmation mechanism. Only when both the single-frame detection confidence and the temporal prediction confidence reach the threshold is the target deemed real and valid, and the actual detection bounding box of the current frame is directly output. This ensures the existence of the target while maximizing the accuracy of the observation location; For scenarios where the target encounters occlusion or weak signals, the system implements a missed detection completion mechanism. When the confidence level of a single-frame detection falls below a threshold due to a decrease in the signal-to-noise ratio, but the temporal prediction still determines that the target exists with a high probability, the system activates the completion function and forces the output of a virtual completed bounding box constructed from the predicted location and historical scale. This is to maintain the continuity of the trajectory and avoid tracking interruptions; In response to ocean clutter interference scenarios, the system implements a false alarm suppression mechanism. For candidate targets that exist in a single frame but have low prediction confidence, the system treats them as transient noise and determines them as invalid because they lack support from historical motion patterns, thus effectively filtering out false alarms.

[0020] Furthermore, the virtual completion box The construction method is as follows: With prediction center Based on the previous frame, the target scale is used. Generate a rectangle, that is: .

[0021] This invention also provides a multi-frame infrared sea surface target detection system based on depth temporal modeling, implemented using the aforementioned method for multi-frame detection of infrared sea surface targets. The system includes a single-frame detection preprocessing module, a temporal state prediction module, a cross-frame association matching module, and an adaptive decision-making module, wherein: The single-frame detection preprocessing module is used to perform target detection frame by frame on the input infrared image sequence to obtain several candidate detection boxes, and then perform redundancy filtering on each candidate detection box to obtain a stable single-frame detection result. The temporal state prediction module is used to extract the most recent consecutive multi-frame historical state information of each tracked target in the system, input it into the deep temporal modeling network, output the predicted displacement vector of each target in the current frame and the prediction existence confidence, and obtain the predicted center position of each target in the current frame based on the predicted displacement vector. The cross-frame association matching module is used to construct a cost matrix based on the obtained single-frame stable detection results and target trajectory prediction state, and to solve the cost matrix using an optimal allocation algorithm to realize the association matching between the current frame detection results and historical target trajectories, thereby obtaining three types of matching results: successfully matched detection-trajectory pairs, unmatched detection results, and unmatched trajectory predictions. The adaptive decision-making module is used to perform multi-level decisions based on the three types of matching results obtained, complete the target trajectory status update, missing detection completion, false detection suppression, new trajectory initialization and failed trajectory termination management, and output reliable infrared sea surface target detection results with continuous identity identifiers.

[0022] Compared with the prior art, the present invention has the following advantages: 1. To address the problem of "easy to miss detection and easy to false alarm" for weak targets under complex sea conditions, this invention uses multi-frame temporal information to model the continuity of target motion and the stability of its existence, thereby achieving the completion output for missed targets and reducing false alarm output caused by background interference.

[0023] 2. To address the problems of traditional cross-frame association relying on the observation of the previous frame, which is prone to mismatch and trajectory breakage, this invention adopts a cross-frame association strategy guided by predicted position to improve the association accuracy and trajectory continuity under platform jitter and short-term occlusion conditions.

[0024] 3. To address the contradiction in existing methods where "completing omissions can easily introduce false detections, while suppressing false detections can easily aggravate missed detections," this invention constructs a unified judgment mechanism: performing completion on high-confidence predicted targets and suppression on low-confidence detected targets, effectively achieving bidirectional optimization.

[0025] 4. This invention is a pure algorithm process and can be deployed independently as a post-processing module for a single-frame detector without modifying the original detection network structure, making it easy to integrate into existing infrared sea surface monitoring systems.

[0026] Based on the above reasons, this invention can be widely applied in fields such as image processing and target detection. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of the process of the infrared sea surface target multi-frame detection method based on depth temporal modeling of the present invention.

[0029] Figure 2 This is a schematic diagram of the target trajectory time series analysis module of the present invention.

[0030] Figure 3 This is a schematic diagram of the missed detection repair and false detection suppression strategy of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products or devices.

[0033] like Figure 1 As shown, this invention provides a multi-frame detection method for infrared sea surface targets based on depth temporal modeling, comprising: S1. Perform target detection frame by frame on the input infrared image sequence to obtain several candidate detection boxes, and then perform redundancy filtering on each candidate detection box to obtain a stable detection result for a single frame. S2. Extract the most recent consecutive multi-frame historical state information of each tracked target in the system, input it into the deep temporal modeling network, output the predicted displacement vector of each target in the current frame and the prediction existence confidence, and obtain the predicted center position of each target in the current frame based on the predicted displacement vector; for example Figure 2 As shown, after obtaining the stable detection input of the current frame, the system establishes and maintains the "target trajectory". The target trajectory described in this invention refers to a time series obtained by merging the states of the same target across multiple consecutive frames through cross-frame correlation. It describes the target's central position and confidence level over time and serves as the basis for subsequent temporal inference. The trajectory is necessary because single-frame observations are severely disturbed by wave textures and random high-brightness areas, while real targets typically exhibit more stable motion continuity and existence continuity over short periods. Therefore, this invention not only predicts the target's displacement vector to characterize its motion trend but also predicts the target's existence confidence level to quantify its continued existence in the current frame. These two types of prediction information together constitute the decision-making basis for missed detection correction and false detection suppression.

[0034] S3. Based on the single-frame stable detection results obtained in step S1 and the target trajectory prediction state obtained in step S2, construct a cost matrix. Use the optimal allocation algorithm to solve the cost matrix to realize the association and matching between the current frame detection results and the historical target trajectories, and obtain three types of matching results: successfully matched detection-trajectory pairs, unmatched detection results, and unmatched trajectory predictions. S4. Based on the three types of matching results obtained in step S3, perform multi-level decision-making to complete the target trajectory status update, missing detection completion, false detection suppression, new trajectory initialization, and failed trajectory termination management, and output reliable infrared sea surface target detection results with continuous identity identifiers.

[0035] In a specific implementation, as a preferred embodiment of the present invention, step S1 includes: S11. Let the current processing frame be the [number]th frame. For each frame, perform single-frame object detection to obtain several candidate detection boxes, each containing a spatial location. Detection confidence And category information; S12. Due to the tendency for the same real target to be captured multiple times in complex sea conditions, directly proceeding to subsequent cross-frame processing could easily lead to duplicate output or incorrect initialization. Therefore, redundant filtering is performed on each candidate detection box, including: Calculate the coordinates of the center point of each detection box The formula is as follows:

[0036] Calculate the Euclidean distance between the center points of any two detection boxes. The formula is as follows:

[0037] in,( ), ( ) are respectively the first The, the The coordinates of the center point of each detection box; Compare the Euclidean distance with a preset spatial threshold, and discard those with a distance less than the preset spatial threshold. The candidate detection boxes with lower confidence are selected, and the retained detection boxes are the single-frame stable detection results.

[0038] In a specific implementation, as a preferred embodiment of the present invention, the preset spatial threshold... The resolution is 10 to 20 pixels, and in this embodiment, 15 pixels is preferred.

[0039] In a specific implementation, as a preferred embodiment of the present invention, step S2 includes: S21. For target trajectories that have been established in the system and are in a continuously updated state (i.e., trajectories that have been associated with detection results at least once in recent frames and have not been terminated), maintain their most recent continuous trajectories. Historical displacement sequence and historical detection confidence sequence of frame window; In this embodiment, The length can be 8 to 12, preferably 10; The historical window is designed to allow for the use of the motion consistency and confidence statistics of multiple frames to determine whether a target persists in time and moves along a reasonable path, even when background disturbances such as ocean clutter cause instability in single-frame observations and significant fluctuations in confidence. This reduces the probability of transient noise being misidentified as a target and avoids premature removal of weakly significant targets due to a short-term drop in confidence.

[0040] S22. When the trajectory history length reaches the window, the historical information is input into a deep learning-based temporal prediction network for joint modeling, and the predicted displacement of the current frame is output. The confidence level of the prediction exists. ,in , used to represent the temporal probability that the trajectory continues to exist in the current frame; S23. The predicted center position of the current frame is obtained based on the predicted displacement, as follows:

[0041] in, This represents the actual center position of the target in the previous frame.

[0042] In this embodiment, displacement prediction enables the system to infer the possible location of the target even when there is short-term occlusion or weak salience, which is used for missed detection repair; existence confidence prediction enables the system to determine whether the detection in the current frame may be "noise impersonation", which is used for false detection suppression.

[0043] In a specific implementation, as a preferred embodiment of the present invention, step S3 includes: S31. Let the number of detection boxes in the current frame be... The number of target trajectories is When constructing the cost matrix, the first The center of the first detection frame and the first The cost of a trajectory prediction center is defined as:

[0044] when Greater than the preset association threshold When this happens, set it to infinity to eliminate unreasonable matches; S32. Solve the cost matrix using the optimal allocation algorithm to obtain the matching relationship between the current frame detection and historical trajectories, forming three categories of results: successfully matched detection-trajectory pairs, unmatched detection boxes, and unmatched trajectory predictions. Then proceed to... Figure 3 The strategy determination step is shown.

[0045] In a specific implementation, as a preferred embodiment of the present invention, the preset association threshold... The resolution is 10 to 20 pixels, and in this embodiment, 15 pixels is preferred.

[0046] In a specific implementation, as a preferred embodiment of the present invention, step S4 includes: S41. For successfully matched detection – the trajectory pair – update the trajectory status and introduce a trajectory confirmation mechanism to divide the trajectory into confirmed and unconfirmed trajectories, wherein: When the trajectory successfully matches the number of frames consecutively satisfy If the trajectory is confirmed, it is marked as a confirmed trajectory. A confirmed trajectory indicates that the target has been stably observed over a continuous period of time, and its identity and movement trend are more reliable. A dynamic confidence threshold is applied to confirmed trajectories to improve the retention of weakly salient targets under complex sea conditions.

[0047] in, The basic detection threshold can be set to 0.12 to 0.20, preferably 0.15. Take 0.3 to 0.5 (preferably 0.4); S42. To achieve synergistic optimization of "missed detection repair" and "false detection suppression", this invention constructs a unified output rule based on joint confidence verification, eliminating the need to distinguish complex intermediate states. The system directly detects the confidence level of the current frame based on the single-frame detection of the current frame. (If no detection box is matched in the current frame, then) The confidence level of the prediction output by the time series model is consistent with that of the prediction. Perform joint determination; S43. Let the single-frame detection threshold be... The prediction threshold is (Preferred value 0.20), establish a joint confidence verification logic, and the system finally outputs the target bounding box. Determined by the following piecewise function:

[0048] in, This represents a virtual completion box; the virtual completion box The construction method is as follows: With prediction center Based on the previous frame, the target scale is used. Generate a rectangle, that is: .

[0049] In this embodiment, as Figure 3 As shown, this invention unifies the repair of missed detections and the suppression of false detections into a single set of judgment logic, avoiding conflicts caused by "repairing missed detections first and then removing false detections" or "removing false detections first and then repairing missed detections".

[0050] S44. For detection boxes that do not match any historical trajectories, to avoid frequent initialization of new trajectories due to background noise causing false detection accumulation, a spatiotemporal consistency constraint is applied based on the occurrence time and spatial location, as follows: Initialization to a new trajectory is only allowed within the first few frames of the video sequence, or when the center point of the detection box is located in the left or right edge region of the image; otherwise, the detection results lack temporal continuity support, are judged as false targets and filtered out, thereby reducing the false detection rate caused by complex sea state background interference. S45. The system introduces a lifecycle management mechanism for the target trajectory, and manages the trajectory generation, confirmation, loss, and termination in a unified manner. In this embodiment, when a trajectory fails to match the detection result for multiple consecutive frames and the prediction confidence is consistently lower than the threshold, or its predicted position exceeds the image range, the system determines that the trajectory is invalid and performs termination management to avoid invalid trajectories occupying system resources for a long time and introducing erroneous output.

[0051] In a specific implementation, as a preferred embodiment of the present invention, the confidence joint verification logic is as follows: For scenarios where the target appears stably, the system implements a dual confirmation mechanism. Only when both the single-frame detection confidence and the temporal prediction confidence reach the threshold is the target deemed real and valid, and the actual detection bounding box of the current frame is directly output. This ensures the existence of the target while maximizing the accuracy of the observation location; For scenarios where the target encounters occlusion or weak signals, the system implements a missed detection completion mechanism. When the confidence level of a single-frame detection falls below a threshold due to a decrease in the signal-to-noise ratio, but the temporal prediction still determines that the target exists with a high probability, the system activates the completion function and forces the output of a virtual completed bounding box constructed from the predicted location and historical scale. This is to maintain the continuity of the trajectory and avoid tracking interruptions; In response to ocean clutter interference scenarios, the system implements a false alarm suppression mechanism. For candidate targets that exist in a single frame but have low prediction confidence, the system treats them as transient noise and determines them as invalid because they lack support from historical motion patterns, thus effectively filtering out false alarms.

[0052] This invention also provides a multi-frame infrared sea surface target detection system based on depth temporal modeling, implemented using the aforementioned method for multi-frame detection of infrared sea surface targets based on depth temporal modeling. The system includes a single-frame detection preprocessing module, a temporal state prediction module, a cross-frame association matching module, and an adaptive decision-making module, wherein: The single-frame detection preprocessing module is used to perform target detection frame by frame on the input infrared image sequence to obtain several candidate detection boxes, and then perform redundancy filtering on each candidate detection box to obtain a stable single-frame detection result. In this embodiment, the single-frame detection preprocessing module, as the front-end processing unit of the method, is configured to: perform frame-by-frame target detection on the input infrared image sequence to obtain initial detection boxes, detection confidence, and category information; and perform redundant filtering based on the spatial distance relationship between the detection boxes to eliminate duplicate detection results caused by local bright noise or detector jitter, thereby suppressing significantly redundant false detections at the single-frame level and providing stable and reliable single-frame detection input for subsequent cross-frame processing.

[0053] The temporal state prediction module is used to extract the most recent consecutive multi-frame historical state information of each tracked target in the system, input it into the deep temporal modeling network, output the predicted displacement vector of each target in the current frame and the prediction existence confidence, and obtain the predicted center position of each target in the current frame based on the predicted displacement vector. In this embodiment, the cross-frame association matching module is connected to the single-frame detection preprocessing module and the temporal state prediction module, and is configured to: construct a cost matrix describing the difference between the predicted position and the actual observed position based on the actual detection box position obtained by the single-frame detection preprocessing module in the current frame and the predicted position of each target trajectory output by the temporal state prediction module; and use an optimal allocation algorithm to complete the association matching between the detection result of the current frame and the historical target trajectory; according to the matching result, the detection and trajectory are divided into three cases: detection-trajectory pairs that have successfully completed the association, detection boxes that have failed to be associated with any trajectory, and trajectory predictions that have failed to match the detection result.

[0054] The cross-frame association matching module is used to construct a cost matrix based on the obtained single-frame stable detection results and target trajectory prediction state, and to solve the cost matrix using an optimal allocation algorithm to realize the association matching between the current frame detection results and historical target trajectories, thereby obtaining three types of matching results: successfully matched detection-trajectory pairs, unmatched detection results, and unmatched trajectory predictions. In this embodiment, the temporal state prediction module serves as the core computational unit of the method, used to model the continuous motion state and stability of the target. This module is configured to: receive the historical state sequence of the target trajectory maintained by the cross-frame association matching module; learn the target's motion change trend and confidence evolution law using a deep learning-based temporal prediction model; and simultaneously output the predicted displacement vector of the target in the current frame. ) and the confidence level of the prediction ( The predicted displacement vector is used to infer the possible position of the target in the current frame to support cross-frame association and missing detection completion. The predicted existence confidence is used to characterize the stability of the trajectory in the time dimension, so as to suppress transient false targets and provide a criterion for trajectory termination management when background interference such as wave crests and specular reflection occurs.

[0055] The adaptive decision-making module is used to perform multi-level decisions based on the three types of matching results obtained, complete the target trajectory status update, missing detection completion, false detection suppression, new trajectory initialization and failed trajectory termination management, and output reliable infrared sea surface target detection results with continuous identity identifiers.

[0056] In this embodiment, the target trajectory decision and management module, the cross-frame association matching module, and the time-series state prediction module form a closed-loop processing structure. This structure is configured to comprehensively utilize cross-frame association results, historical trajectory states, and the predicted existence confidence level to execute multi-level judgment logic. This logic is used to complete target trajectory state updates, missed detection completion, false detection suppression, new trajectory initialization, and failed trajectory termination management. Specifically, by applying spatiotemporal consistency constraints to unmatched detection boxes to suppress randomly occurring false targets, and by performing termination judgments on trajectories with persistently low predicted existence confidence or long-term unmatched trajectories, the system actively eliminates falsely detected trajectories, ultimately outputting reliable detection results with continuous identity identifiers.

[0057] Example This invention is integrated into the information processing unit of a shipborne infrared search and rescue system. The system's electro-optical pod incorporates a mid-wave infrared imaging sensor, capable of stably outputting an infrared video stream with a resolution of 640×512 and a frame rate of 25 frames per second, regardless of day or night conditions or complex sea states. The information processing unit employs an embedded industrial computer equipped with a high-performance GPU, responsible for running the multi-frame detection algorithm of this invention.

[0058] The key algorithm parameters are set as follows: single-frame detection uses a YOLOv5s model trained on sea surface target data; the spatial distance threshold for redundancy removal of detection boxes is 15 pixels; the historical frame window length of the temporal prediction module is 10 frames; the distance threshold for cross-frame association matching is 20 pixels; trajectory confirmation requires 10 consecutive successful matches and a prediction confidence greater than 0.15; the basic detection threshold is 0.15, and a dynamic threshold (0.4 times the basic threshold) is used for confirmed trajectories; the initialization constraint for new trajectories is: within the first 20 frames of the sequence, or the center of the detection box is located within the left and right 10-pixel edge area of ​​the image.

[0059] In a real-world infrared sequence of strong winds and waves, the system continuously tracks a small vessel. The target exhibits low signal-to-noise ratio in single frames and is partially obscured by waves, leading to missed detections. The algorithm models its trajectory using a temporal prediction module, and when the target... When a frame is not output by the single-frame detector due to occlusion, its corresponding trajectory prediction confidence level The calculated value is 0.28, which is higher than the dynamic threshold (0.15) set for confirmed trajectories. (0.4 = 0.06). Based on this, the target trajectory decision and management module determined it to be a missed detection. A virtual detection box was generated according to the predicted position (312.5, 256.8) and the target size recorded on the trajectory, effectively maintaining the continuity of the trajectory. Simultaneously, a bright interference formed by wave crest reflection in the center of the image was detected. Because it did not meet the spatiotemporal constraints, it was determined to be a false alarm and filtered out, preventing the generation of false trajectories. Verification in a real-world scenario showed that after applying the method of this invention, the system's continuous detection performance for small targets on the sea surface was significantly improved: the overall false detection rate decreased by 63%, and the overall missed detection rate decreased by 74%. This effectively verifies the ability of this invention to collaboratively optimize missed detection completion and false detection suppression under complex sea conditions, demonstrating good engineering practical value.

[0060] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-frame detection method for infrared sea surface targets based on depth temporal modeling, characterized in that, include: S1. Perform target detection frame by frame on the input infrared image sequence to obtain several candidate detection boxes, and then perform redundancy filtering on each candidate detection box to obtain a stable detection result for a single frame. S2. Extract the most recent consecutive multi-frame historical state information of each tracked target in the system, input it into the deep temporal modeling network, output the predicted displacement vector of each target in the current frame and the prediction existence confidence, and obtain the predicted center position of each target in the current frame based on the predicted displacement vector. S3. Based on the single-frame stable detection results obtained in step S1 and the target trajectory prediction state obtained in step S2, construct a cost matrix. Use the optimal allocation algorithm to solve the cost matrix to realize the association and matching between the current frame detection results and the historical target trajectories, and obtain three types of matching results: successfully matched detection-trajectory pairs, unmatched detection results, and unmatched trajectory predictions. S4. Based on the three types of matching results obtained in step S3, perform multi-level decision-making to complete the target trajectory status update, missing detection completion, false detection suppression, new trajectory initialization, and failed trajectory termination management, and output reliable infrared sea surface target detection results with continuous identity identifiers.

2. The multi-frame detection method for infrared sea surface targets based on depth temporal modeling according to claim 1, characterized in that, Step S1 includes: S11. Let the current processing frame be the [number]th frame. For each frame, perform single-frame object detection to obtain several candidate detection boxes, each containing a spatial location. Detection confidence And category information; S12. Perform redundancy filtering on each candidate detection box, including: Calculate the coordinates of the center point of each detection box The formula is as follows: Calculate the Euclidean distance between the center points of any two detection boxes. The formula is as follows: in,( ), ( ) are respectively the first The, the The coordinates of the center point of each detection box; Compare the Euclidean distance with a preset spatial threshold, and discard those with a distance less than the preset spatial threshold. The candidate detection boxes with lower confidence are selected, and the retained detection boxes are the single-frame stable detection results.

3. The multi-frame detection method for infrared sea surface targets based on depth temporal modeling according to claim 2, characterized in that, The preset space threshold It is 10 to 20 pixels.

4. The multi-frame detection method for infrared sea surface targets based on depth temporal modeling according to claim 1, characterized in that, Step S2 includes: S21. For target trajectories that have been established in the system and are continuously updated, maintain their most recent continuous... Historical displacement sequence and historical detection confidence sequence of frame window; S22. When the trajectory history length reaches the window, the historical information is input into a deep learning-based temporal prediction network for joint modeling, and the predicted displacement of the current frame is output. The confidence level of the prediction exists. ,in , used to represent the temporal probability that the trajectory continues to exist in the current frame; S23. The predicted center position of the current frame is obtained based on the predicted displacement, as follows: in, This represents the actual center position of the target in the previous frame.

5. The multi-frame detection method for infrared sea surface targets based on depth temporal modeling according to claim 1, characterized in that, Step S3 includes: S31. Let the number of detection boxes in the current frame be... The number of target trajectories is When constructing the cost matrix, the first The center of the first detection frame and the first The cost of a trajectory prediction center is defined as: when Greater than the preset association threshold When this happens, set it to infinity to eliminate unreasonable matches; S32. Solve the cost matrix using the optimal allocation algorithm to obtain the matching relationship between the current frame detection and the historical trajectory, and form three types of results: successfully matched detection-trajectory pairs, unmatched detection boxes, and unmatched trajectory predictions.

6. The multi-frame detection method for infrared sea surface targets based on depth temporal modeling according to claim 5, characterized in that, The preset association threshold It is 10 to 20 pixels.

7. The method for multi-frame detection of infrared sea surface targets based on depth temporal modeling according to claim 1, characterized in that, Step S4 includes: S41. For successfully matched detection – the trajectory pair – update the trajectory status and introduce a trajectory confirmation mechanism to divide the trajectory into confirmed and unconfirmed trajectories, wherein: When the trajectory successfully matches the number of frames consecutively satisfy If the trajectory is confirmed, it is marked as a confirmed trajectory. A confirmed trajectory indicates that the target has been stably observed over a continuous period of time, and its identity and movement trend are more reliable. A dynamic confidence threshold is applied to confirmed trajectories to improve the retention of weakly salient targets under complex sea conditions. in, Based on the detection threshold, Take a value of 0.3 to 0.5; S42. Construct a unified output rule based on joint confidence verification, eliminating the need to distinguish complex intermediate states. The system directly detects the confidence level based on the single-frame detection of the current frame. There is a confidence level with the prediction output of the time series model. Perform joint determination; S43. Let the single-frame detection threshold be... The prediction threshold is Establish a joint confidence verification logic, and the system will finally output the target bounding box. Determined by the following piecewise function: in, This indicates a virtual completion box; S44. For detection boxes that do not match any historical trajectories, to avoid frequent initialization of new trajectories due to background noise causing false detection accumulation, a spatiotemporal consistency constraint is applied based on the occurrence time and spatial location, as follows: Initialization to a new trajectory is only allowed within the first few frames of the video sequence, or when the center point of the detection box is located in the left or right edge region of the image; otherwise, the detection results lack temporal continuity support, are judged as false targets and filtered out, thereby reducing the false detection rate caused by complex sea state background interference. S45. The system introduces a lifecycle management mechanism for the target trajectory, and performs unified management of trajectory generation, confirmation, loss and termination.

8. The method for multi-frame detection of infrared sea surface targets based on depth temporal modeling according to claim 7, characterized in that, The joint confidence level verification logic is as follows: For scenarios where the target appears stably, the system implements a dual confirmation mechanism. Only when both the single-frame detection confidence and the temporal prediction confidence reach the threshold is the target deemed real and valid, and the actual detection bounding box of the current frame is directly output. This ensures the existence of the target while maximizing the accuracy of the observation location; For scenarios where the target encounters occlusion or weak signals, the system implements a missed detection completion mechanism. When the confidence level of a single-frame detection falls below a threshold due to a decrease in the signal-to-noise ratio, but the temporal prediction still determines that the target exists with a high probability, the system activates the completion function and forces the output of a virtual completed bounding box constructed from the predicted location and historical scale. This is to maintain the continuity of the trajectory and avoid tracking interruptions; In response to ocean clutter interference scenarios, the system implements a false alarm suppression mechanism. For candidate targets that exist in a single frame but have low prediction confidence, the system treats them as transient noise and determines them as invalid because they lack support from historical motion patterns, thus effectively filtering out false alarms.

9. The multi-frame detection method for infrared sea surface targets based on depth temporal modeling according to claim 7, characterized in that, The virtual completion box The construction method is as follows: With prediction center Based on the previous frame, the target scale is used. Generate a rectangle, that is: 。 10. A multi-frame infrared sea surface target detection system based on depth temporal modeling, implemented according to the depth temporal modeling-based multi-frame detection method for infrared sea surface targets as described in any one of claims 1-9, characterized in that, It includes a single-frame detection preprocessing module, a temporal state prediction module, a cross-frame association matching module, and an adaptive decision-making module, wherein: The single-frame detection preprocessing module is used to perform target detection frame by frame on the input infrared image sequence to obtain several candidate detection boxes, and then perform redundancy filtering on each candidate detection box to obtain a stable single-frame detection result. The temporal state prediction module is used to extract the most recent consecutive multi-frame historical state information of each tracked target in the system, input it into the deep temporal modeling network, output the predicted displacement vector of each target in the current frame and the prediction existence confidence, and obtain the predicted center position of each target in the current frame based on the predicted displacement vector. The cross-frame association matching module is used to construct a cost matrix based on the obtained single-frame stable detection results and target trajectory prediction state, and to solve the cost matrix using an optimal allocation algorithm to realize the association matching between the current frame detection results and historical target trajectories, thereby obtaining three types of matching results: successfully matched detection-trajectory pairs, unmatched detection results, and unmatched trajectory predictions. The adaptive decision-making module is used to perform multi-level decisions based on the three types of matching results obtained, complete the target trajectory status update, missing detection completion, false detection suppression, new trajectory initialization and failed trajectory termination management, and output reliable infrared sea surface target detection results with continuous identity identifiers.

Images