A multi-target tracking method and device for a marine scene

Abstract

The application provides a multi-target tracking method and device for a marine scene. The method provided by the application comprises: acquiring a current frame target detection result, an active track set and a lost track set; performing observation anchoring prediction processing on the active track based on the latest real observation information, historical motion information and historical scale statistical information in the active track set, to generate a current frame prediction box and corresponding direction prior information; performing two-stage association matching based on the current frame prediction box, the direction prior information and the current frame target detection result, to obtain a first unmatched track set and a first unmatched detection set, and further performing supplementary association matching to obtain a second unmatched detection set; performing occlusion post-track reconnection processing based on the second unmatched detection set and the lost track set, to recover the corresponding historical track identity; creating a new track for a target detection whose identity is not recovered, and outputting a current frame target tracking result.

Description

Technical Field

[0001] This application relates to the field of multi-target tracking technology, and in particular to a multi-target tracking method and apparatus for maritime scenarios. Background Technology

[0002] With the increasing demand for applications such as marine traffic monitoring, intelligent maritime patrol, autonomous navigation of unmanned vessels, and situational awareness of maritime targets, multi-target tracking technology based on optical video has gradually become an important research direction in the field of marine intelligent sensing. Therefore, how to achieve stable, continuous, and reliable target tracking in complex marine environments has become one of the key technical issues in marine intelligent monitoring systems.

[0003] Currently, most multi-target tracking methods employ an online processing framework of "target detection—trajectory prediction—data association—trajectory management." Target detection is used to obtain the target's position in the current frame; trajectory prediction typically extrapolates the target's next state based on Kalman filtering or its variants; data association uses metrics such as intersection-overlap ratio (IoU), center distance, or appearance features to match the detection results with historical trajectories; and trajectory management retains, updates, or deletes trajectories based on continuous updates. This type of method achieves good tracking results in general land traffic scenarios and is therefore widely used in vehicle tracking, pedestrian tracking, and video surveillance. However, maritime scenarios differ significantly from traditional land scenarios. Affected by factors such as the ship's six-degree-of-freedom motion, external sea state disturbances like wind, waves, and currents, and the resulting jitter of the shipborne imaging platform, the target motion in the image often overlaps with a large number of non-target motion components. This leads to problems such as prediction box drift, scale fluctuations, and velocity error accumulation in traditional prediction methods based on the constant velocity assumption, thus reducing the stability of the prediction results. Meanwhile, under conditions of platform disturbance, short-term occlusion, or detection offset, insufficient or even no overlap may occur between the target detection bounding box and the predicted bounding box. Existing association methods that rely on intersection-union ratio (IUU) struggle to maintain stable matching, easily leading to failure in associating with real targets and trajectory interruptions. Furthermore, when a target reappears due to wave occlusion, ship hull overlap, or after a short-term missed detection by the detector, existing trajectory management methods typically reassign the target's identity, resulting in increased identity switching and trajectory fragmentation.

[0004] Therefore, there is an urgent need for a multi-target tracking method suitable for complex maritime scenarios, in order to improve the stability of target prediction, enhance the correlation capability in low-overlap scenarios, and achieve continuous recovery of target identity after occlusion, thereby improving the continuity and stability of multi-target tracking at sea. Summary of the Invention

[0005] In view of this, this application provides a multi-target tracking method and apparatus for maritime scenarios, which can improve the stability of target prediction, enhance the correlation capability in low-overlap scenarios, and realize the continuous recovery of target identity after occlusion, thereby improving the continuity and stability of multi-target tracking at sea.

[0006] Specifically, this application is implemented through the following technical solution:

[0007] The first aspect of this application provides a multi-target tracking method for maritime scenarios, the method comprising:

[0008] Obtain the target detection results, active trajectory set, and lost trajectory set for the current frame;

[0009] Based on the most recent real observation information, historical motion information and historical scale statistical information in the set of activity trajectories, observation anchoring prediction processing is performed on the activity trajectory to generate the current frame prediction box and the corresponding directional prior information.

[0010] Based on the current frame prediction box, orientation prior information and current frame target detection result, a two-stage association matching is performed to obtain the first unmatched trajectory set and the first unmatched detection set.

[0011] Based on the first set of unmatched trajectories and the first set of unmatched detections, a supplementary association matching is performed to obtain a second set of unmatched detections.

[0012] Based on the second unmatched detection set and the lost trajectory set, perform trajectory reconnection processing after occlusion to restore the identity of the corresponding historical trajectory;

[0013] Create new trajectories for targets whose identities have not been restored, update the active trajectory set, lost trajectory set, and trajectory history statistics, and output the target tracking results for the current frame.

[0014] A second aspect of this application provides a multi-target tracking device for maritime scenarios, the device comprising an acquisition module, a prediction module, a matching module, a processing module, and an output module;

[0015] The acquisition module is used to acquire the target detection results, active trajectory set, and lost trajectory set of the current frame;

[0016] The prediction module is used to perform observation anchoring prediction processing on the activity trajectory based on the most recent real observation information, historical motion information and historical scale statistical information in the activity trajectory set, and generate the current frame prediction box and the corresponding direction prior information.

[0017] The matching module is used to perform a two-stage association matching based on the current frame prediction box, the orientation prior information and the current frame target detection result to obtain a first set of unmatched trajectories and a first set of unmatched detections.

[0018] The matching module is further configured to perform supplementary association matching based on the first unmatched trajectory set and the first unmatched detection set to obtain a second unmatched detection set;

[0019] The processing module is used to perform occlusion reconnection processing based on the second unmatched detection set and the lost trajectory set to restore the identity of the corresponding historical trajectory;

[0020] The output module is used to create new trajectories for targets whose identities have not been restored, update the active trajectory set, the lost trajectory set, and historical trajectory statistics, and output the target tracking result of the current frame.

[0021] The multi-target tracking method and apparatus for maritime scenarios provided in this application acquire the target detection results, active trajectory set, and lost trajectory set of the current frame to provide a unified trajectory basis for subsequent prediction, association, and reconnection processing. Subsequently, based on the most recent real observation information, historical motion information, and historical scale statistical information, the current frame prediction box and orientation prior information are generated, enabling the prediction results to maintain high stability and orientation reliability under maritime platform disturbance conditions. Furthermore, through target association matching and supplementary association matching, the success rate of target association under low overlap, prediction offset, and short-term occlusion conditions is improved. Afterward, based on the second unmatched detection set and the lost trajectory set, trajectory reconnection processing after occlusion is performed to restore the historical trajectory identity after the target reappears, reducing identity switching and trajectory fragmentation. Finally, by creating new trajectories and updating the active trajectory set, lost trajectory set, and trajectory historical statistical information, the continuous output of the target tracking results of the current frame is achieved, thereby improving the continuity and stability of maritime multi-target tracking. Attached Figure Description

[0022] Figure 1 A flowchart of a multi-target tracking method for maritime scenarios provided in Embodiment 1 of this application;

[0023] Figure 2 This is a schematic diagram of the structure of the multi-target tracking device for maritime scenarios provided in Embodiment 2 of this application. Detailed Implementation

[0024] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0025] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used herein are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0026] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0027] The following specific embodiments are given to illustrate the technical solution of this application in detail.

[0028] Figure 1 This is a flowchart of a multi-target tracking method for maritime scenarios provided in Embodiment 1 of this application. Please refer to... Figure 1 The method provided in this embodiment may include:

[0029] S101. Obtain the target detection results, active trajectory set, and lost trajectory set of the current frame.

[0030] Specifically, the target detection result of the current frame refers to the set of target information obtained after target recognition of the current input video frame by the target detection network. Each target detection result includes at least the target location, target category, and target detection confidence. The target location represents the region of the target in the current frame image, usually represented by a bounding box; the target category represents the type of the target; and the target detection confidence represents the reliability of the target detection result. It should be noted that the target detection network can be implemented using existing target detection models. This application does not limit the specific network structure, training method, or detection implementation process of the target detection network, but only uses the target detection results output by the target detection network as input data for subsequent multi-target tracking processing.

[0031] The active trajectory set refers to the set of target trajectories that are currently under normal tracking and have maintained effective updates in the most recent consecutive frames. Each active trajectory in the active trajectory set corresponds to a target with an established identity and records the target's historical motion information, historical position change information, historical scale statistics, and trajectory status information for subsequent trajectory prediction, target association, and trajectory update processing.

[0032] The lost trajectory set refers to the set of target trajectories that have not yet completed matching updates in the current frame, but still retain historical identity information. The trajectories in the lost trajectory set usually correspond to target trajectories that have temporarily disappeared due to wave obstruction, ship hull overlap, platform disturbance, or short-term missed detection by detectors. The set retains the historical motion information, historical location information, and historical identity identifier of the corresponding targets, which are used to perform trajectory reconnection and identity restoration processing when the targets reappear.

[0033] In practice, the system acquires the maritime video stream to be processed and decodes it frame by frame in chronological order to obtain the current frame image. The current frame image is then input into a target detection network to perform target recognition processing on the targets in the current frame image, outputting the target detection result for the current frame. The system reads the trajectory data updated in the previous frame and classifies the trajectories according to their current state. Trajectories in normal tracking status and maintaining effective updates in the most recent consecutive frames are classified as active trajectories. Trajectories that have not yet completed matching updates in the current frame but have not exceeded a preset retention frame threshold are classified as lost trajectories. For each active trajectory in the active trajectory set, the system reads its historical motion information, historical position change information, historical scale statistics, and trajectory status information. For each lost trajectory in the lost trajectory set, the system reads its historical motion information, historical position information, and historical identification.

[0034] S102. Based on the most recent real observation information, historical motion information and historical scale statistics in the set of activity trajectories, perform observation anchoring prediction processing on the activity trajectory to generate the current frame prediction box and the corresponding direction prior information.

[0035] Specifically, the most recent true observation information refers to the true target observation information corresponding to the most recent successful match between the activity trajectory and the target detection result, representing the actual state of the target when it was most reliably detected. The most recent true observation information includes at least the most recent true observation box position, target center position, target width, target height, and corresponding observation time information, which are used as observation anchor points in the current frame prediction processing.

[0036] Historical motion information refers to the motion changes of the activity trajectory in consecutive historical frames, representing the target's motion trend over time. Historical motion information includes at least historical center displacement information, historical motion direction information, historical motion velocity information, and historical position change sequence; specifically, historical center displacement information describes the change in the target's center position between adjacent frames, historical motion direction information describes the change in the target's motion direction, historical motion velocity information describes the change in the target's motion amplitude, and historical position change sequence describes the change in the target's position.

[0037] Historical scale statistics refer to the statistical results of target bounding box scale changes in continuous historical frames of the activity trajectory, characterizing the stable scale features of the target in the time series. Historical scale statistics include at least historical height information, historical aspect ratio information, reference height information, and average scale information; among them, the reference height information is used to constrain the range of predicted bounding box scale changes, and the average scale information is used to assess the stability of the target scale.

[0038] The current frame prediction bounding box refers to the target bounding box generated after predicting the target position and scale in the current frame based on the most recent real observation information, historical motion information, and historical scale statistics. It represents the predicted position and scale range of the target in the current frame. Orientation prior information refers to directional constraint information generated based on the historical motion direction changes of the activity trajectory, representing the possible motion direction trend of the target in the current frame. Orientation prior information includes at least orientation consistency information and corresponding orientation prior weights, used to constrain the matching direction between the trajectory and the detection during subsequent target association processes, improving the stability of target association.

[0039] In specific implementation, based on the most recent real observation information, historical motion information, and historical scale statistics in the activity trajectory set, observation anchoring prediction processing is performed on the activity trajectory to generate the current frame prediction box and the corresponding directional prior information, including:

[0040] (1) Obtain the most recent real observation box of the activity trajectory, and use the most recent real observation box as the prediction anchor point of the current frame.

[0041] Specifically, the current frame prediction anchor point refers to the reference observation position used as the starting reference position for prediction during the current frame target state prediction process. The current frame prediction anchor point serves as the reference starting point for extrapolating historical motion displacements, ensuring that the current frame prediction position is updated based on the most recent actual observation position. This reduces prediction drift caused by platform disturbances, short-term abnormal target motion, or continuous mismatches, and improves the positional stability of the current frame prediction box.

[0042] In practice, the historical trajectory information corresponding to each activity trajectory is read from the activity trajectory set. The ground truth bounding box corresponding to the most recent successful match between each activity trajectory and the target detection result is obtained. The target position information and target bounding box scale information corresponding to the ground truth bounding box are read, the center position of the ground truth bounding box is extracted as the reference position for the current frame prediction, and the position state corresponding to the ground truth bounding box is written into the current frame prediction initialization state, using the ground truth bounding box as the anchor point for the current frame prediction.

[0043] (2) Extract window motion trend information based on historical center displacement information of activity trajectory, and generate the current frame predicted center position based on the window motion trend information.

[0044] Specifically, window motion trend information refers to the motion change trend information extracted based on the changes in the center position of the activity trajectory within a preset historical time window, representing the stable motion direction and amplitude variation law of the target in short-term continuous frames. Window motion trend information includes at least the historical center displacement sequence, short-window median displacement, motion direction change information, and motion amplitude change information; among which, the short-window median displacement represents the stable displacement trend of the activity trajectory within the preset time window, and the motion direction change information represents the direction change law of the target in continuous frames. The current frame predicted center position refers to the position result obtained after predicting the center position of the target in the current frame based on the current frame prediction anchor point and window motion trend information, representing the predicted center region of the target in the current frame.

[0045] In specific implementation, the historical center displacement sequence of the activity trajectory within a preset time window is obtained, and the median displacement is calculated based on the historical center displacement sequence; the median displacement represents the stable motion trend of the activity trajectory; the most recent actual observation position of the activity trajectory is obtained, and the most recent actual observation position is used as the observation anchor point for the predicted center position of the current frame; the reference height information of the activity trajectory is obtained, and the predicted displacement corresponding to the median displacement is scale-constrained based on the reference height information to limit the extrapolation range of the predicted center position of the current frame; the number of consecutive mismatch frames of the activity trajectory is obtained, and the predicted displacement is attenuated based on the number of consecutive mismatch frames to reduce the motion extrapolation intensity under the continuous mismatch state; the predicted center position of the current frame is generated based on the observation anchor point, the predicted displacement after scale constraint, and the motion extrapolation result after attenuation adjustment.

[0046] Specifically, the historical center position of the target in the most recent consecutive frames is read from the activity trajectory. For each frame, the position of the target bounding box center point in the image coordinate system is recorded. Then, by calculating the position change between the center points of adjacent frames, the corresponding historical center displacement sequence is obtained. Since the target motion in a maritime scene is easily affected by the ship's roll, pitch, and wave disturbances, the displacement of a single frame usually fluctuates significantly. Therefore, the displacement of the latest frame is not directly used as the basis for motion prediction. Instead, median statistical processing is performed on the historical displacement sequence within a preset time window to obtain a relatively stable median displacement within a short time window. The target observation position corresponding to the most recent successful match of the activity trajectory is read, and this observation position is used as the observation anchor point for the predicted center position of the current frame. After obtaining the median displacement, the reference height information of the activity trajectory in the historical consecutive frames is read, and the median displacement is constrained by the reference height. When the predicted displacement exceeds the allowable extrapolation range corresponding to the target reference height, the predicted displacement is compressed and limited. The number of consecutive mismatch frames corresponding to the current activity trajectory is read. The number of consecutive mismatch frames indicates how many consecutive frames the trajectory has failed to successfully match the detection result. The motion attenuation coefficient is determined based on the number of consecutive mismatch frames. The larger the number of consecutive mismatch frames, the smaller the motion attenuation coefficient. This attenuation coefficient is used to adjust the predicted displacement, so that the trajectory gradually reduces the motion extrapolation amplitude under continuous mismatch conditions. Using the most recent actual observation position as the base position, the predicted displacement after scale constraint and attenuation adjustment is superimposed on the observation anchor point to generate the predicted center position of the current frame.

[0047] For example, in one embodiment, the predicted center position of the current frame can be represented as:

[0048] ;

[0049] ;

[0050] ;

[0051] in, Predict the center position for the current frame; The observation anchor point is used to predict the center position of the current frame; This represents the number of consecutive mismatched frames. This represents the median of the short window displacement. This is the displacement scale constraint coefficient, used to constrain the predicted displacement corresponding to the median displacement of the short window based on the target reference height. For reference height; To prevent stable terms with a denominator of zero.

[0052] The method provided in this embodiment, when determining the predicted center position of the current frame, can extract a stable motion trend based on the median displacement of the historical center displacement sequence, reducing the impact of abnormal displacement caused by disturbances of the offshore platform and image jitter. At the same time, by using the most recent real observation position as the observation anchor point, the prediction result always converges to the real observation position, reducing the error accumulation caused by continuous recursion. Furthermore, by combining the target reference height to impose scale constraints on the predicted displacement, the extrapolation range of the predicted center position can be limited, avoiding unreasonable target drift. And by adjusting the motion extrapolation intensity by the number of consecutive mismatched frames, the problem of continuous offset of the prediction result under continuous mismatch is reduced, thereby improving the stability, reliability, and spatial rationality of the predicted center position of the current frame, providing a more reliable prediction basis for subsequent target association.

[0053] (3) Recover the scale of the current frame prediction box based on the historical height information and historical aspect ratio information of the activity trajectory, and generate the current frame prediction box.

[0054] Specifically, historical height information refers to the changes in the height of the target bounding box corresponding to the activity trajectory in consecutive historical frames, representing the scale changes of the target over time. Historical height information can include the target bounding box height value, average height value, median height value, and height change trend information for each historical frame. The median or average height value is typically used to characterize the stable scale characteristics of the target. Historical aspect ratio information refers to the changes in the ratio between the width and height of the target bounding box corresponding to the activity trajectory in consecutive historical frames, representing the stable morphological characteristics of the target's appearance scale structure. Historical aspect ratio information can include the aspect ratio value, average aspect ratio, and aspect ratio change trend information for each historical frame, used to recover the width-to-height relationship of the target bounding box during prediction.

[0055] In practice, the historical height sequence and aspect ratio sequence of the target bounding boxes in consecutive historical frames are read from the activity trajectory. Statistical processing is performed on the historical height sequence to obtain the reference height information corresponding to the activity trajectory, using the median of the historical height sequence as the reference height. Simultaneously, statistical processing is performed on the historical aspect ratio sequence to obtain the corresponding reference aspect ratio information, using the median of the historical aspect ratio sequence as the reference aspect ratio. The reference height is used as the predicted height of the current frame's prediction bounding box. The product of the reference height and the reference aspect ratio is calculated as the predicted width of the current frame's prediction bounding box. The recovered predicted width and predicted height are combined with the current frame's prediction center position to generate the current frame's prediction bounding box.

[0056] For example, in one embodiment, the scale of the current frame prediction box can be expressed as:

[0057] ;

[0058] ;

[0059] ;

[0060] in, The scale of the predicted bounding box for the current frame; Predict the width of the bounding box for the current frame; Predict the height of the bounding box for the current frame; For the reference height information corresponding to the activity trajectory, this embodiment uses the median of the historical height sequence as the reference height; To reference aspect ratio information, this embodiment uses the median of the historical aspect ratio sequence as the reference aspect ratio.

[0061] (4) Update the trajectory status information corresponding to the activity trajectory synchronously based on the prediction box of the current frame.

[0062] Specifically, trajectory state information refers to a set of trajectory management information used to characterize the current tracking state and historical motion state of an activity trajectory, describing the prediction state, update state, and lifecycle state of the target trajectory at the current moment. Trajectory state information includes at least the current trajectory position state, trajectory velocity state, trajectory scale state, trajectory continuous matching state, number of consecutive mismatch frames, trajectory existence duration, and trajectory identification information. Specifically, the current trajectory position state represents the current predicted or updated position of the target; the trajectory velocity state represents the target's current direction and amplitude of motion; the trajectory scale state represents the current width, height, and scale changes of the target bounding box; the trajectory continuous matching state represents whether the trajectory is currently in a normal update state; the number of consecutive mismatch frames records the number of consecutive frames for which the trajectory has not completed a match; the trajectory existence duration represents the length of time the trajectory has existed continuously; and the trajectory identification information distinguishes the trajectory identities corresponding to different targets. In this application, the trajectory state information is updated synchronously with the update of the prediction bounding box in the current frame to ensure that the prediction state, motion state, and scale state of the activity trajectory remain consistent with the prediction result of the current frame.

[0063] In practice, the system reads the trajectory state information corresponding to the previous frame, including the trajectory center position, prediction box scale, historical velocity status, and trajectory continuous matching status. It obtains the prediction center position and prediction box scale corresponding to the current frame prediction box, calculates the position difference between the current frame prediction center position and the trajectory state center position of the previous frame, and obtains the positional motion change amount corresponding to the current frame. Based on the scale change relationship between the current frame prediction box scale and the previous frame prediction box scale, it calculates the current scale change velocity status of the active trajectory. To avoid abrupt changes in scale status due to disturbances of the offshore platform, detection fluctuations, or abnormal predictions, a saturation constraint is applied to the scale change amount, limiting the scale change amplitude to a preset proportion range corresponding to the previous frame prediction box scale. After completing the position and scale status updates, the current frame prediction center position, current frame prediction box scale, current position velocity status, and current scale change velocity status are written into the trajectory state information corresponding to the active trajectory, and the current timestamp, continuous mismatch status, and trajectory lifecycle status are updated synchronously. The updated trajectory state information is saved to the active trajectory set.

[0064] For example, in one embodiment, the trajectory state information update process can be represented as:

[0065] ;

[0066] ;

[0067] ;

[0068] in, The position and velocity state of the activity trajectory in the current frame; Predict the center position for the current frame; The position of the center of the trajectory recorded in the trajectory status information of the previous frame; This represents the scale change rate state of the activity trajectory in the current frame. The scale of the predicted bounding box for the current frame; The scale is the predicted bounding box size of the previous frame.

[0069] (5) Extract motion direction information based on historical displacement information of the activity trajectory, and perform reliability assessment on the motion direction information according to direction consistency, displacement amplitude, trajectory maturity and continuous mismatch state.

[0070] Specifically, motion direction information refers to the target motion direction features extracted based on the displacement changes of the activity trajectory in consecutive historical frames, representing the target's main motion orientation in the time series. Motion direction information can include historical displacement direction, average motion direction, direction change trend, and direction stability. Direction consistency refers to the degree to which the motion direction of the activity trajectory remains consistent across consecutive historical frames, representing the stability of the target's motion direction.

[0071] Displacement amplitude refers to the magnitude of displacement of the activity trajectory within consecutive historical frames, characterizing the actual motion intensity of the target. Displacement amplitude can be obtained from the distance of position change between historical center positions and is used to assess whether the current motion direction has sufficiently obvious motion characteristics. When the displacement amplitude is small, the target may be in a state of slow movement, stationary, or affected by disturbances, and the reliability of the corresponding motion direction is low.

[0072] Trajectory maturity refers to the degree of stable tracking accumulated during the current tracking process, characterizing the reliability of the trajectory status. Trajectory maturity can be determined based on the duration of continuous trajectory existence, the number of consecutive matching frames, or historical stable update data; the higher the trajectory maturity, the longer the trajectory has been stably tracked, and the more reliable its corresponding motion direction information is usually.

[0073] A continuous mismatch state refers to the state information where the activity trajectory fails to successfully match the target detection result in a number of consecutive frames, representing the stability of the current trajectory state. A continuous mismatch state is usually represented by the number of consecutive mismatch frames; the larger the number of consecutive mismatch frames, the longer the current trajectory has not received a true detection update, and the reliability of its motion state will gradually decrease, as will the reliability of the corresponding motion direction information.

[0074] In specific implementation, the sequence of center position changes of the activity trajectory within a preset historical time window is read, and the corresponding historical displacement vector is calculated based on the positional change relationship between the center positions of adjacent frames. The motion direction information of the activity trajectory in consecutive historical frames is extracted based on the direction information of each historical displacement vector, generating a corresponding historical motion direction sequence. After obtaining the historical motion direction sequence, the directional difference between each historical displacement direction is calculated, and a directional consistency index is generated based on the statistical results of the directional difference. When the directional change between consecutive historical displacements is small, the corresponding directional consistency is high; when the directional change fluctuates greatly, the corresponding directional consistency is low. The displacement amplitude corresponding to the activity trajectory is calculated based on the length information of the historical displacement vectors, using the average, median, or robust statistical results of the historical displacement lengths as the displacement amplitude representation result. The trajectory maturity information corresponding to the current activity trajectory is read, and the corresponding trajectory maturity level is determined based on the number of consecutive frames the trajectory has existed, the number of consecutive successfully matched frames, or the trajectory's stable update duration. The continuous mismatch status information corresponding to the current activity trajectory is read, including the current number of consecutive mismatch frames and whether the trajectory is currently in a continuous mismatch state. After acquiring the above information, a reliability assessment is performed on the current motion direction information, taking into account directional consistency, displacement amplitude, trajectory maturity, and continuous mismatch status. If directional consistency is high, displacement amplitude meets motion conditions, trajectory maturity is high, and continuous mismatch status is weak, the current motion direction information is determined to have high reliability; otherwise, the reliability of the current motion direction information is reduced, and the motion direction information after reliability assessment is output.

[0075] (6) When the motion direction information meets the reliability conditions, generate the corresponding direction prior information.

[0076] In specific implementation, the first parameter is determined based on the trajectory maturity of the activity trajectory; the directional consistency score is calculated based on the historical displacement direction information of the activity trajectory within a preset time window; the median displacement and reference height information of the activity trajectory are obtained, and the corresponding weight interval is determined based on the relationship between the directional consistency score and the corresponding first threshold, and the relationship between the median displacement and the reference height; when the weight interval is a first preset interval, the directional prior weight is 0; when the weight interval is a second preset interval, the directional prior weight is calculated based on the number of consecutive mismatch frames of the activity trajectory and the first parameter; the current motion direction information is weighted based on the directional prior weight to generate the corresponding directional prior information.

[0077] Specifically, the system reads the current trajectory maturity information of the active trajectory and determines the corresponding first parameter based on the trajectory maturity level. For mature trajectories with long continuous stable tracking time and a large number of consecutive successful matching frames, a higher first parameter is assigned; for newly initialized or less stable trajectories, a lower first parameter is assigned. The system reads the historical displacement direction sequence of the active trajectory within a preset historical time window and generates a corresponding directional stability evaluation result by statistically analyzing the degree of angle change between consecutive historical displacement directions. The system reads the median displacement and reference height information corresponding to the active trajectory and determines the weight interval corresponding to the current motion state based on the relationship between the directional consistency score and the corresponding first threshold, as well as the ratio of the median displacement to the average reference height. Specifically, when the directional consistency score is low or the median displacement is small relative to the average reference height, the current motion direction is deemed unstable and enters the first preset interval; when the directional consistency score meets the preset conditions and the median displacement relative to the reference height reaches the preset motion conditions, the current motion direction is deemed to have high reliability and enters the second preset interval. When the weight interval is the first preset interval, the prior weight of the direction is directly set to 0, so that the current motion direction information does not participate in subsequent direction constraint processing. When the weight interval is the second preset interval, the number of consecutive mismatch frames corresponding to the current activity trajectory is read, and the motion attenuation coefficient corresponding to the number of consecutive mismatch frames is matched. The product of the motion attenuation coefficient and the first parameter is determined as the direction prior weight. The calculated direction prior weight is used to perform weighted processing on the current motion direction information to generate the corresponding direction prior information.

[0078] For example, in one embodiment, the directional prior weights can be expressed as:

[0079] ;

[0080] ;

[0081] in, Prior weights for direction; Scoring for directional consistency; This is the median displacement; Average height; This represents the number of consecutive mismatched frames.

[0082] S103. Based on the current frame prediction box, direction prior information and the current frame target detection result, perform two-stage association matching to obtain the first unmatched trajectory set and the first unmatched detection set.

[0083] Specifically, two-stage association matching refers to performing two rounds of association processing sequentially on the active trajectory and the target detection result in the current frame during target tracking, in order to improve the target association success rate and reduce trajectory interruptions. The first stage of association matching is performed based on the predicted bounding box of the active trajectory in the current frame and the target detection result in the current frame. This stage uses the predicted bounding box of the current frame generated by observation anchoring prediction processing as the trajectory's current position estimate. The association relationship between the predicted bounding box and the target detection box is calculated to complete the matching of the trajectory and the detection result, and the successfully matched active trajectory is updated. For trajectories that have not yet completed matching after the first stage of association matching, and for target detection results that have not completed matching, the second stage of association matching is entered. The second stage of association matching no longer uses the predicted bounding box of the current frame, but instead uses the target bounding box corresponding to the most recent real observation of the unmatched trajectory and the unmatched detection result to perform association processing again. The correspondence between the trajectory and the detection result is re-established through real observation information, thereby recovering targets that failed to be associated due to prediction offset, target jitter, or short-term state changes.

[0084] The first set of unmatched trajectories refers to the set of active trajectories that failed to successfully match the target detection result in the current frame during the target association matching process. Trajectories in the first set of unmatched trajectories include one of the following situations: insufficient overlap between the predicted bounding box and the detection bounding box, prediction offset caused by platform disturbance, short-term occlusion of the target, sudden change in the target's motion state, or short-term missed detection in the detection result.

[0085] The first unmatched detection set refers to the set of target detection results that failed to successfully match the activity trajectory during the target association matching process in the current frame. The target detection results in the first unmatched detection set include one of the following situations: the target is a newly appearing target, the trajectory prediction of the target has a large deviation, the target reappears after being occluded, or the association conditions are not met during the current association process.

[0086] In specific implementation, the system reads the current frame predicted bounding box, orientation prior information, and target detection result corresponding to the activity trajectory. Candidate association pairs are constructed based on the positional relationship between each activity trajectory and the target detection result. The overlap and spatial positional relationship between each current frame predicted bounding box and the target detection box are calculated, and an initial association cost is generated based on this correspondence. Simultaneously, the orientation prior information corresponding to the activity trajectory is read, and the initial association cost is adjusted based on the consistency between the actual motion direction of the target detection result relative to the predicted bounding box and its historical motion direction. When the motion direction of the target detection result is consistent with the historical motion direction of the trajectory, the corresponding association cost is reduced; when the direction deviation is large, the corresponding association cost is increased. After calculating the association cost between each candidate trajectory and candidate detection, a trajectory-detection association cost matrix is ​​constructed. Based on this association cost matrix, the Hungarian matching algorithm or other optimal matching algorithm is used to perform the first stage of association matching. The matching results are filtered for validity based on the association cost threshold, and the corresponding activity trajectories are updated based on the valid matching results. Subsequently, the trajectories that have not yet been matched after the first stage of association matching, as well as the target detection results that have not been matched, are obtained, along with the most recent real observation information corresponding to the unmatched trajectories. Furthermore, the predicted bounding boxes of the current frame are replaced with the bounding boxes corresponding to the most recently observed real information, and candidate association pairs are reconstructed with the unmatched detection results, and the corresponding association cost matrix is ​​calculated. A second-stage association matching is performed based on the association cost matrix to recover the target correspondences that failed to be associated due to prediction offset, short-term motion anomalies, or accumulated prediction errors, and the corresponding activity trajectories are updated based on the second-stage association results. After completing the two-stage association matching, the activity trajectories that still have not successfully completed matching are determined as the first unmatched trajectory set; the target detection results that still have not successfully completed matching are determined as the first unmatched detection set.

[0087] S104. Perform supplementary association matching based on the first unmatched trajectory set and the first unmatched detection set to obtain the second unmatched detection set.

[0088] Specifically, the second unmatched detection set refers to the set of target detection results that, after completing the first-stage target association matching and supplementary association matching, have still not successfully established an association with any historical trajectory. The target detection results in the second unmatched detection set typically include the following situations: newly added targets entering the monitoring area in the current frame; targets whose identity recovery has not yet been completed due to prolonged target occlusion, large trajectory prediction deviations, or historical trajectories being lost; and target detection results that still do not meet the association conditions during the supplementary association matching process.

[0089] In specific implementation, category consistency screening is performed on unmatched trajectories in the first unmatched trajectory set and unmatched detections in the first unmatched detection set; when the target category of an unmatched trajectory is consistent with that of an unmatched detection, the predicted center position of the corresponding unmatched trajectory and the detection center position of the corresponding unmatched detection are obtained; the trajectory-detection spatial distance is calculated based on the positional relationship between the predicted center position and the detection center position; the historical target scale information corresponding to the unmatched trajectory and the current target scale information corresponding to the unmatched detection are obtained; the trajectory-detection spatial distance is normalized based on the historical target scale information and the current target scale information to generate a normalized spatial distance; a height consistency modulation term is calculated based on the historical reference height of the unmatched trajectory and the current height of the unmatched detection; a comprehensive association cost is generated based on the normalized spatial distance and the height consistency modulation term; a one-to-one supplementary association matching is performed between the trajectory and the detection based on the comprehensive association cost, and the target detections that still do not complete the matching after the supplementary association matching are output as the second unmatched detection set.

[0090] Optionally, generating a comprehensive association cost based on the normalized spatial distance and the high consistency modulation term includes: determining whether the unmatched trajectory is consistent with the target category of the unmatched detection; when the unmatched trajectory is inconsistent with the target category of the unmatched detection, setting the comprehensive association cost to a preset maximum cost; wherein the preset maximum cost is greater than the association cost threshold corresponding to the supplementary association matching, so that the corresponding unmatched trajectory and the unmatched detection do not meet the association condition; when the unmatched trajectory is consistent with the target category of the unmatched detection, calculating the difference between the preset baseline value 1 and the high consistency modulation term; generating a candidate association cost based on the product of the difference and the normalized spatial distance; comparing the candidate association cost with the preset lower limit value 0, and determining the larger value as the comprehensive association cost.

[0091] Specifically, the process reads each unmatched trajectory from the first unmatched trajectory set and each unmatched detection from the first unmatched detection set, and performs category consistency screening based on the correspondence between the trajectory target category and the detection target category. For trajectory-detection pairs with consistent target categories, they are retained as candidate association pairs; for trajectory-detection pairs with inconsistent target categories, a preset maximum cost value is directly assigned, preventing them from meeting subsequent association conditions. For candidate trajectory-detection pairs that pass the category consistency screening, the predicted center position corresponding to the unmatched trajectory and the detection center position corresponding to the unmatched detection are read, and the corresponding spatial deviation result is obtained by calculating the Euclidean distance between the predicted center position and the detection center position. The process also reads the historical target scale information corresponding to the unmatched trajectory and the current target scale information corresponding to the unmatched detection, including parameters such as the target box width, height, and target size, and constructs a normalized scale factor based on the historical target scale and the current target scale. Subsequently, the normalized scale factor is used to perform scale normalization processing on the trajectory-detection spatial distance to generate a normalized spatial distance.

[0092] The system reads the historical reference height corresponding to the unmatched trajectory and the current detection height corresponding to the unmatched detection, calculates the normalized height difference between the historical reference height and the current detection height, and generates the corresponding height consistency modulation result according to the preset height consistency segmentation rule. When the height difference is small, the height consistency modulation value is increased; when the height difference is large, the corresponding modulation value is decreased. Further, a comprehensive association cost is generated based on the normalized spatial distance and the height consistency modulation term. First, it determines whether the target category of the current candidate trajectory and the candidate detection is consistent; when the target category is consistent, the difference between the preset baseline value 1 and the height consistency modulation term is further calculated, and the candidate association cost is generated by multiplying the difference by the normalized spatial distance. Subsequently, the candidate association cost is compared with the preset lower limit value 0, and the larger value is determined as the comprehensive association cost. After completing the comprehensive association cost calculation for all candidate trajectory-detection pairs, a supplementary association matching cost matrix is ​​constructed, and a one-to-one supplementary association matching between the trajectory and the detection is performed based on this cost matrix. Target detection results that still fail to complete the matching after supplementary association matching are divided into a second unmatched detection set.

[0093] For example, in one embodiment, category consistency filtering can be represented as:

[0094] ;

[0095] in, The cost of filtering for category consistency; To preset the maximum cost value; , These are the unmatched trajectories and the target categories of the unmatched detections, respectively. This is the threshold for association cost.

[0096] Normalized spatial distance can be expressed as:

[0097] ;

[0098] ;

[0099] in, Normalized spatial distance; The predicted center location corresponding to the unmatched trajectory; The location of the detection center corresponding to the unmatched detection; The width of the predicted bounding box corresponding to the unmatched trajectory; The width of the unmatched detection box; The height of the predicted bounding box corresponding to the unmatched trajectory; For detecting height.

[0100] The highly consistent modulation term can be expressed as:

[0101] ;

[0102] ;

[0103] in, It is a highly consistent modulation term; , These are the positive and negative height modulation coefficients, respectively, set according to the trajectory maturity level. For high consistency segmentation threshold, this embodiment uses 0.1; For historical reference; For detecting height; It is a numerically stable term.

[0104] The overall associated cost can be expressed as:

[0105] ;

[0106] in, To comprehensively consider the associated costs; To preset the maximum cost value; Normalized spatial distance; It is a highly consistent modulation term; , These are the unmatched trajectories and the target categories of the unmatched detections, respectively. This is the threshold for association cost.

[0107] S105. Based on the second unmatched detection set and the lost trajectory set, perform trajectory reconnection processing after occlusion to restore the identity of the corresponding historical trajectory.

[0108] Specifically, historical trajectory identity refers to the unique trajectory identifier information corresponding to a target during continuous tracking. It is used to distinguish the trajectory affiliation of different targets and maintain the continuity of the identity of the same target in different time frames. When a target is first detected in a video sequence and its trajectory initialization is completed, the target is assigned a unique trajectory identifier. During subsequent continuous frame tracking, regardless of changes in the target's position, scale, or motion state, as long as the target maintains its association with the original trajectory, its corresponding trajectory identity remains unchanged.

[0109] In specific implementation, stability screening is performed on each lost trajectory in the lost trajectory set, retaining stable lost trajectories that meet the preset trajectory length condition; the center position of the bottom edge of the target box corresponding to the last observation of the stable lost trajectory is obtained, and the center position of the bottom edge of the target box is used as the trajectory position anchor point; stable pixel velocity is calculated based on the historical motion information of the stable lost trajectory, and the current position extrapolation prediction is performed on the trajectory position anchor point according to the stable pixel velocity; based on the current position extrapolation prediction result, the stable lost trajectory, and the target detection results in the second unmatched detection box set, three types of consistency scores are calculated; the three types of consistency scores are weighted and summed to obtain a comprehensive reconnection score; the highest comprehensive reconnection score and the second highest comprehensive reconnection score corresponding to the target detection result are obtained; when the highest comprehensive reconnection score meets the preset reconnection threshold, and the highest comprehensive reconnection score and the second highest comprehensive reconnection score meet the uniqueness constraint condition, the historical trajectory identity of the corresponding stable lost trajectory is restored to the target detection result.

[0110] Optionally, three types of consistency scores are calculated, including: spatial prediction consistency score based on the positional relationship between the extrapolated prediction result of the current position and the target detection result in the second unmatched detection set; motion velocity consistency score based on the relationship between the historical motion information of the stable lost trajectory and the current motion information corresponding to the target detection result; and target area consistency score based on the relationship between the historical target area information of the stable lost trajectory and the current target area information corresponding to the target detection result.

[0111] Optionally, a motion speed consistency score is calculated based on the relationship between the historical motion information of the stable lost trajectory and the current motion information corresponding to the target detection result. This includes: obtaining the historical displacement information of the stable lost trajectory within a preset historical time window; calculating the historical motion direction and historical motion speed based on the historical displacement information; obtaining the current position information corresponding to the target detection result; calculating the current motion direction and current motion speed based on the position change relationship between the current position information and the last observed position of the stable lost trajectory; calculating a direction consistency index based on the directional difference between the historical motion direction and the current motion direction; calculating a speed consistency index based on the speed difference between the historical motion speed and the current motion speed; generating a motion speed consistency score based on the direction consistency index and the speed consistency index; and increasing the corresponding motion speed consistency score when the direction consistency index and the speed consistency index meet preset consistency conditions.

[0112] Specifically, each lost trajectory in the lost trajectory set is read, and stability filtering is performed on each lost trajectory based on its historical existence duration, continuous stable tracking length, and final trajectory state. Only stable lost trajectories that meet the preset trajectory length condition and have not been determined to have left the image area are retained to construct a stable lost trajectory set. The target bounding box corresponding to the last successful observation of each stable lost trajectory is obtained, and the center position of the bottom edge of the target bounding box is extracted as the trajectory position anchor point. The position of the midpoint of the bottom edge of the target bounding box in the image coordinate system is read as the initial reference position for subsequent current position extrapolation prediction. After obtaining the trajectory position anchor point, the historical motion information of the stable lost trajectory in historical consecutive frames is further read, including parameters such as historical displacement information, historical motion direction, and historical motion velocity, and the stable pixel velocity is calculated based on the historical motion information. Subsequently, based on the time interval between the current time and the last visible time of the trajectory, the current position extrapolation prediction is performed on the trajectory position anchor point using the stable pixel velocity to generate the predicted position result of the stable lost trajectory in the current frame.

[0113] The system reads the target detection results from the second unmatched detection set and calculates three types of consistency scores based on the extrapolated prediction results at the current position, the stable lost trajectory, and the correspondence between the target detection results. When calculating the spatial prediction consistency score, the spatial deviation between the predicted and detected positions is calculated based on the positional relationship between the extrapolated prediction results at the current position and the target detection results, and a corresponding spatial prediction consistency score is generated based on this spatial deviation. When calculating the motion velocity consistency score, the system reads the historical displacement information of the stable lost trajectory within a preset historical time window and calculates the corresponding historical motion direction and historical motion velocity based on this historical displacement information. Then, the system reads the current position information corresponding to the target detection results and calculates the current motion direction and current motion velocity based on the positional change relationship between the current position information and the last observed position of the stable lost trajectory. A direction consistency index is calculated based on the directional difference between the historical and current motion directions, and a velocity consistency index is calculated based on the velocity difference between the historical and current motion velocities. Finally, a motion velocity consistency score is generated based on the direction consistency index and the velocity consistency index; when both the direction consistency index and the velocity consistency index simultaneously meet preset consistency conditions, the corresponding motion velocity consistency score is increased. When calculating the target area consistency score, the historical target area information corresponding to the stable lost trajectory and the current target area information corresponding to the target detection result are read, and the target area consistency score is calculated based on the area difference relationship between the two.

[0114] After calculating the three types of consistency scores, a weighted summation process is performed on the spatial prediction consistency score, motion velocity consistency score, and target area consistency score to generate a comprehensive reconnection score. Subsequently, for each target detection result, the highest and second-highest comprehensive reconnection scores among the corresponding candidate stable lost trajectories are obtained. It is then determined whether the highest comprehensive reconnection score meets a preset reconnection threshold, and whether a uniqueness constraint is satisfied between the highest and second-highest comprehensive reconnection scores. When the highest comprehensive reconnection score meets the preset reconnection threshold, and a uniqueness constraint is satisfied between the highest and second-highest comprehensive reconnection scores, the historical trajectory identity of the corresponding stable lost trajectory is restored to the corresponding target detection result, and trajectory reconnection is updated. The target detection results with restored trajectory identities are then added back to the active trajectory set for subsequent continuous tracking processing.

[0115] For example, in one embodiment, the process of filtering stable lost trajectories can be represented as:

[0116] and ;

[0117] in, The trajectory number corresponding to the lost trajectory; For the set of lost trajectories; The length of the trajectory; To determine the minimum trajectory length threshold for stabilizing lost trajectories; This represents the left boundary position of the target bounding box at the last moment of the lost trajectory. For boundary buffering; The horizontal velocity at the last moment of the trajectory; The boundary velocity threshold corresponding to the target leaving the image region; This represents the right boundary position of the target bounding box at the last moment of the lost trajectory. This represents the image width.

[0118] The extrapolation prediction of the current position for the trajectory position anchor point can be expressed as:

[0119] ;

[0120] in, Predict the anchor point for the current frame; Anchor points for trajectory positions; The interval between the reconnection time and the last visible time; To stabilize pixel speed.

[0121] Spatial prediction consistency score can be expressed as:

[0122] ;

[0123] in, Scoring the consistency of spatial predictions; Centered at the bottom edge of the candidate detection box; To stabilize the lost trajectory at the predicted anchor point position in the current frame; This is a parameter for adjusting spatial consistency.

[0124] The consistency score for movement speed can be expressed as:

[0125] ;

[0126] in, Scoring for consistency in movement speed; This represents the observation speed corresponding to the current detection. To stabilize the motion velocity corresponding to the lost trajectory; For speed consistency adjustment parameters; Scoring is given for consistency in the direction of movement.

[0127] The target area consistency score can be expressed as:

[0128] ;

[0129] in, Scoring is given for the consistency of the target area; The current target area corresponding to the unmatched detection; To stabilize the historical reference area corresponding to the lost trajectory; This is the parameter for adjusting area consistency.

[0130] The overall reconnection score can be expressed as:

[0131] ;

[0132] in, For comprehensive reconnection score; Scoring the consistency of spatial predictions; Scoring for consistency in movement speed; Scoring is given for the consistency of the target area.

[0133] S106. Create a new trajectory for the target detection that has not recovered its identity, update the active trajectory set, the lost trajectory set and trajectory history statistics, and output the target tracking result of the current frame.

[0134] Specifically, targets whose identities have not been restored refer to target detection results that, after target association matching, supplementary association matching, and trajectory reconnection after occlusion in the current frame, have still not successfully established an identity correspondence with any historical trajectory. Targets whose identities have not been restored typically include the following situations: newly added targets entering the monitoring area in the current frame; targets whose original trajectory identities cannot be restored due to prolonged occlusion, deletion of historical trajectories, or significant changes in historical motion states; and target detection results that do not meet the comprehensive reconnection conditions or uniqueness constraints during trajectory reconnection. For such targets, the system will reassign a new trajectory identity and create a corresponding new trajectory.

[0135] The target tracking result for the current frame includes at least the target position, target bounding box scale, target category, trajectory identification, target motion state, and trajectory status information. Specifically, the target position represents the final tracking position of the target in the current frame; the target bounding box scale represents the width and height information of the target; the target category represents the category to which the target belongs; the trajectory identification is used to distinguish the trajectory identities of different targets; the target motion state represents the target's current motion direction and speed; and the trajectory status information indicates whether the target is currently in a normal tracking state, a short-term loss of tracking, or a reconnection recovery state.

[0136] In practice, the system reads the target detection results in the current frame that have not yet had their identities restored. A new trajectory identity is assigned to each target detection result without restored identity, and the center position, target scale, motion state, and trajectory lifecycle state of the corresponding trajectory are initialized based on the current target detection box. Simultaneously, the position, width, height, target category, and detection confidence of the current detection result are written into the initial trajectory state information of the corresponding new trajectory. After completing the initialization of the new trajectory, the active trajectory set is updated. Active trajectories that have successfully completed association matching in the current frame, trajectories that have been restored after occlusion reconnection, and newly created trajectories are all added to the active trajectory set, and the current state information of the corresponding trajectories is updated synchronously. The lost trajectory set is updated. Trajectories that have not yet completed matching in the current frame and do not meet the trajectory deletion conditions are retained in the lost trajectory set; trajectories with continuous mismatch time exceeding a preset retention threshold or that have been determined to have left the image area are removed from the lost trajectory set. The historical statistics of each trajectory are updated synchronously. The system updates statistical information such as historical center position, historical displacement, historical direction of movement, historical speed of movement, historical target height, historical aspect ratio, historical target area, and trajectory duration corresponding to the current active trajectory and the stable lost trajectory. The trajectory status corresponding to the current frame is then written into the historical cache sequence. Finally, the system summarizes the target position, target bounding box scale, target category, trajectory identifier, and trajectory status information corresponding to all valid trajectories in the current frame to generate the target tracking result for the current frame.

[0137] The method provided in this embodiment, in its first aspect, obtains the target detection results, active trajectory set, and lost trajectory set of the current frame to provide a unified trajectory basis for subsequent prediction, association, and reconnection processing. Subsequently, it generates the current frame prediction box and direction prior information based on the most recent real observation information, historical motion information, and historical scale statistical information, enabling the prediction results to maintain high stability and direction reliability under the disturbance conditions of the offshore platform. Furthermore, it improves the target association success rate under low overlap, prediction offset, and short-term occlusion conditions through target association matching and supplementary association matching. Afterward, it performs trajectory reconnection processing after occlusion based on the second unmatched detection set and the lost trajectory set, realizing the restoration of the historical trajectory identity after the target reappears, reducing identity switching and trajectory fragmentation. Finally, by creating new trajectories and updating the active trajectory set, lost trajectory set, and trajectory historical statistical information, it realizes the continuous output of the target tracking results of the current frame, thereby improving the continuity and stability of multi-target tracking at sea.

[0138] Secondly, by performing observation-anchored prediction processing on the activity trajectory, the current frame prediction box and direction prior information are generated using the most recent real observation information, historical motion information, and historical scale statistics. This allows the prediction results to not only extrapolate the position based on historical motion trends but also to anchor the prediction results using the most recent real observation position, thus avoiding the problem of continuous error accumulation in traditional continuous recursive prediction. Simultaneously, by extracting stable motion trends from the median of historical displacements and combining this with reference height to limit the extrapolation range of predicted displacements, the impact of abnormal displacements caused by the platform's roll, pitch, wave disturbances, and image jitter can be reduced, ensuring high stability between the predicted center position and the prediction box scale. Furthermore, by combining direction consistency, trajectory maturity, and continuous mismatch states to generate direction prior information, the ability to constrain the target's motion direction can be improved, thereby enhancing the reliability of matching the real target in subsequent association processes.

[0139] Thirdly, through a two-stage association mechanism of target association matching and supplementary association matching, the system can not only complete the association of conventional highly overlapping targets, but also perform supplementary recovery for targets with low overlap, prediction offset, and short-term occlusion. In the supplementary association matching process, a comprehensive association cost is jointly generated through category consistency screening, normalized spatial distance, and height consistency modulation terms. This ensures that the association results not only consider the proximity between targets but also further constrain the relationship between target scale and height changes, thereby reducing erroneous association problems caused by relying solely on intersection-union ratio or single spatial distance. Especially in maritime scenarios, where platform disturbances can easily lead to insufficient overlap between predicted and detected bounding boxes, the supplementary association mechanism can maintain a high target association success rate even under low overlap conditions, thus reducing trajectory breakage issues.

[0140] Fourthly, through trajectory reconnection processing after occlusion, the system can restore the original trajectory identity of targets that reappear after a short-term mismatch. Specifically, by performing stability screening on lost trajectories, only trajectories with stable historical motion characteristics are retained for reconnection processing, thereby reducing the problem of erroneous reconnection caused by unstable trajectories. At the same time, by generating extrapolation prediction results of the current position based on historical motion information, and combining spatial prediction consistency scores, motion velocity consistency scores, and target area consistency scores to perform comprehensive reconnection score calculation, the system can simultaneously determine the correspondence between the current detection results and historically lost trajectories from three dimensions: spatial location, motion trend, and target scale. In this way, even if the target is occluded by waves, overlapped by ship hulls, or temporarily missed by the detector, the system can still restore its original trajectory identity after the target reappears, avoiding the repeated creation of new trajectories, thereby reducing the problems of identity switching and trajectory fragmentation.

[0141] Corresponding to the aforementioned embodiment of a multi-target tracking method for maritime scenarios, this application also provides an embodiment of a multi-target tracking device for maritime scenarios.

[0142] Figure 2 This is a schematic diagram of the multi-target tracking device for maritime scenarios provided in Embodiment 2 of this application. Please refer to... Figure 2 The apparatus provided in this embodiment includes an acquisition module 210, a prediction module 220, a matching module 230, a processing module 240, and an output module 250.

[0143] The acquisition module 210 is used to acquire the target detection result, the set of active trajectories, and the set of lost trajectories in the current frame.

[0144] The prediction module 220 is used to perform observation anchoring prediction processing on the activity trajectory based on the most recent real observation information, historical motion information and historical scale statistical information in the activity trajectory set, and generate the current frame prediction box and the corresponding direction prior information.

[0145] The matching module 230 is used to perform a two-stage association matching based on the current frame prediction box, the direction prior information and the current frame target detection result to obtain a first unmatched trajectory set and a first unmatched detection set.

[0146] The matching module 230 is further configured to perform supplementary association matching based on the first unmatched trajectory set and the first unmatched detection set to obtain a second unmatched detection set;

[0147] The processing module 240 is used to perform occlusion reconnection processing based on the second unmatched detection set and the lost trajectory set to restore the identity of the corresponding historical trajectory.

[0148] The output module 250 is used to create new trajectories for targets whose identities have not been restored, update the active trajectory set, the lost trajectory set, and historical trajectory statistics, and output the target tracking result of the current frame.

[0149] The apparatus of this embodiment can be used to perform... Figure 1 The steps of the method embodiment shown are similar in principle and process, and will not be repeated here.

[0150] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0151] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0152] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A multi-target tracking method for maritime scenarios, characterized in that, The method includes: Obtain the target detection results, active trajectory set, and lost trajectory set for the current frame; Based on the most recent real observation information, historical motion information and historical scale statistical information in the set of activity trajectories, observation anchoring prediction processing is performed on the activity trajectory to generate the current frame prediction box and the corresponding directional prior information. Based on the current frame prediction box, orientation prior information and current frame target detection result, a two-stage association matching is performed to obtain the first unmatched trajectory set and the first unmatched detection set. Based on the first set of unmatched trajectories and the first set of unmatched detections, a supplementary association matching is performed to obtain a second set of unmatched detections. Based on the second unmatched detection set and the lost trajectory set, perform trajectory reconnection processing after occlusion to restore the identity of the corresponding historical trajectory; For targets whose identities have not been recovered, create new trajectories, update the active trajectory set, lost trajectory set, and historical trajectory statistics, and output the target tracking results for the current frame; Based on the most recent real observation information, historical motion information, and historical scale statistics in the activity trajectory set, observation anchoring prediction processing is performed on the activity trajectory to generate the current frame prediction box and the corresponding direction prior information, including: Obtain the most recent real observation box of the activity trajectory, and use the most recent real observation box as the prediction anchor point of the current frame; Based on the historical center displacement information of the activity trajectory, window motion trend information is extracted, and the predicted center position of the current frame is generated according to the window motion trend information. Based on the historical height and aspect ratio information of the activity trajectory, the scale of the current frame prediction box is recovered, and the current frame prediction box is generated. The trajectory status information corresponding to the active trajectory is synchronously updated based on the current frame prediction box; Motion direction information is extracted based on historical displacement information of the activity trajectory, and a reliability assessment is performed on the motion direction information based on direction consistency, displacement amplitude, trajectory maturity, and continuous mismatch status. When the motion direction information meets the reliability condition, corresponding direction prior information is generated; wherein, a corresponding first parameter is determined based on the trajectory maturity of the activity trajectory; a direction consistency score is calculated based on the historical displacement direction information of the activity trajectory within a preset time window; the median displacement and reference height information of the activity trajectory are obtained, and a corresponding weight interval is determined based on the relationship between the direction consistency score and the corresponding first threshold, and the relationship between the median displacement and the reference height; when the weight interval is a first preset interval, the direction prior weight is 0; when the weight interval is a second preset interval, the direction prior weight is calculated based on the number of consecutive mismatch frames of the activity trajectory and the first parameter; the current motion direction information is weighted based on the direction prior weight to generate the corresponding direction prior information.

2. The method according to claim 1, characterized in that, Extracting window motion trend information based on historical center displacement information of the activity trajectory, and generating the predicted center position of the current frame based on the window motion trend information, including: Obtain the historical center displacement sequence of the activity trajectory within a preset time window, and calculate the median displacement based on the historical center displacement sequence; the median displacement characterizes the stable motion trend of the activity trajectory. Obtain the most recent real observation position of the activity trajectory, and use the most recent real observation position as the observation anchor point of the current frame prediction center position; Obtain reference height information of the activity trajectory, and apply scale constraints to the predicted displacement corresponding to the median displacement based on the reference height information to limit the extrapolation range of the predicted center position of the current frame. The number of consecutive mismatch frames of the activity trajectory is obtained, and the predicted displacement is attenuated based on the number of consecutive mismatch frames to reduce the motion extrapolation intensity under the continuous mismatch state. Based on the observed anchor point, the predicted displacement after scale constraint, and the motion extrapolation result after attenuation adjustment, the predicted center position of the current frame is generated.

3. The method according to claim 1, characterized in that, Based on the second unmatched detection set and the lost trajectory set, occlusion reconnection processing is performed to restore the identity of the corresponding historical trajectory, including: Stability screening is performed on each lost trajectory in the set of lost trajectories, and stable lost trajectories that meet the preset trajectory length condition are retained. Obtain the center position of the bottom edge of the target box corresponding to the last observation of the stable lost trajectory, and use the center position of the bottom edge of the target box as the trajectory position anchor point; Based on the historical motion information of the stable lost trajectory, the stable pixel velocity is calculated, and the current position extrapolation prediction is performed on the trajectory position anchor point according to the stable pixel velocity. Based on the extrapolated prediction results of the current location, the stable lost trajectory, and the target detection results in the second set of unmatched detection boxes, three types of consistency scores are calculated. The three types of consistency scores are weighted and summed to obtain the comprehensive reconnection score. Obtain the highest and second-highest comprehensive reconnection scores corresponding to the target detection results; When the highest comprehensive reconnection score meets the preset reconnection threshold, and the highest comprehensive reconnection score and the second highest comprehensive reconnection score satisfy the uniqueness constraint, the historical trajectory identity of the corresponding stable lost trajectory is restored to the target detection result.

4. The method according to claim 3, characterized in that, Calculate the three types of consistency scores, including: Based on the positional relationship between the extrapolated prediction result at the current location and the target detection results in the second unmatched detection set, a spatial prediction consistency score is calculated. Based on the relationship between historical motion information of stable lost trajectories and current motion information corresponding to target detection results, a motion speed consistency score is calculated. Based on the relationship between historical target area information of stable lost trajectories and current target area information corresponding to target detection results, a target area consistency score is calculated.

5. The method according to claim 4, characterized in that, Based on the relationship between historical motion information of stable lost trajectories and current motion information corresponding to target detection results, a motion velocity consistency score is calculated, including: Obtain historical displacement information of the stable lost trajectory within a preset historical time window, and calculate the historical motion direction and historical motion speed based on the historical displacement information; Obtain the current position information corresponding to the target detection result, and calculate the current direction of motion and current speed based on the position change relationship between the current position information and the last observed position of the stable lost trajectory. Calculate the direction consistency index based on the directional difference between the historical direction of motion and the current direction of motion; A speed consistency index is calculated based on the speed difference between the historical speed and the current speed. A motion speed consistency score is generated based on the direction consistency index and the speed consistency index; when the direction consistency index and the speed consistency index meet the preset consistency conditions, the corresponding motion speed consistency score is increased.

6. The method according to claim 1, characterized in that, Based on the first set of unmatched trajectories and the first set of unmatched detections, supplementary correlation matching is performed to obtain a second set of unmatched detections, including: Perform category consistency screening on the unmatched trajectories in the first unmatched trajectory set and the unmatched detections in the first unmatched detection set; When an unmatched trajectory matches the target category of an unmatched detection, obtain the predicted center position of the corresponding unmatched trajectory and the detection center position of the corresponding unmatched detection. The trajectory-detection spatial distance is calculated based on the positional relationship between the predicted center location and the detection center location; Obtain historical target scale information corresponding to unmatched trajectories and current target scale information corresponding to unmatched detections; Based on the historical target scale information and the current target scale information, the trajectory-detection spatial distance is normalized to generate a normalized spatial distance; Calculate the height consistency modulation term based on the historical reference height of the unmatched trajectory and the current height of the unmatched detection; A comprehensive correlation cost is generated based on the normalized spatial distance and the highly consistent modulation term; Based on the one-to-one supplementary association matching between the comprehensive association cost execution trajectory and the detection, the target detections that still do not complete the matching after the supplementary association matching are output as the second unmatched detection set.

7. The method according to claim 6, characterized in that, A comprehensive correlation cost is generated based on the normalized spatial distance and the highly consistent modulation term, including: Determine whether the unmatched trajectory matches the target category of the unmatched detection; When the unmatched trajectory does not match the target category of the unmatched detection, the comprehensive association cost is set to a preset maximum cost; wherein, the preset maximum cost is greater than the association cost threshold corresponding to the supplementary association matching, so that the corresponding unmatched trajectory and the unmatched detection do not meet the association conditions. When the unmatched trajectory matches the target category of the unmatched detection, calculate the difference between the preset baseline value 1 and the high consistency modulation term; Candidate association values ​​are generated based on the product of the difference and the normalized spatial distance. The candidate association cost is compared with the preset lower limit of 0, and the larger value is determined as the comprehensive association cost.

8. A multi-target tracking device for maritime scenarios, characterized in that, The multi-target tracking device for maritime scenarios is applied to the multi-target tracking method for maritime scenarios according to any one of claims 1-7. The device includes an acquisition module, a prediction module, a matching module, a processing module, and an output module. The acquisition module is used to acquire the target detection results, active trajectory set, and lost trajectory set of the current frame; The prediction module is used to perform observation anchoring prediction processing on the activity trajectory based on the most recent real observation information, historical motion information and historical scale statistical information in the activity trajectory set, and generate the current frame prediction box and the corresponding direction prior information. The matching module is used to perform a two-stage association matching based on the current frame prediction box, the orientation prior information and the current frame target detection result to obtain a first set of unmatched trajectories and a first set of unmatched detections. The matching module is further configured to perform supplementary association matching based on the first unmatched trajectory set and the first unmatched detection set to obtain a second unmatched detection set; The processing module is used to perform occlusion reconnection processing based on the second unmatched detection set and the lost trajectory set to restore the identity of the corresponding historical trajectory; The output module is used to create new trajectories for targets whose identities have not been restored, update the active trajectory set, the lost trajectory set, and historical trajectory statistics, and output the target tracking result of the current frame.

Images