An unmanned ship efficient multi-target tracking method based on visual geometry prior

Abstract

The embodiment discloses a kind of unmanned ship high-efficiency multi-target tracking methods based on visual geometry prior, by introducing displacement attenuation and velocity smoothing mechanism based on visual geometry prior, effectively overcome the motion prediction drift problem caused by camera shaking on sea;At the same time, the multi-dimensional total matching cost matrix of fusing scale cost and confidence cost is constructed, and the secondary optimization matching strategy of intersection over union and motion distance is combined, which significantly improves the data association accuracy under the condition of target scale dramatic change and background dynamic interference.Candidate pool is set to manage the time sequence of unmatched detection frame, and the time sequence non-overlapping track is intelligently merged according to time interval, spatial interval and area scale ratio.The present application avoids high complexity appearance feature extraction and transformer operation under the premise of ensuring high tracking accuracy, realizes the efficient use of computing resources, and is especially suitable for real-time, robust multi-target tracking of unmanned ship edge platform.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and multi-target tracking technology, and in particular to an efficient multi-target tracking method for unmanned surface vessels based on visual geometric priors. Background Technology

[0002] Multi-target tracking is a fundamental task in computer vision, aiming to simultaneously locate multiple targets in a video sequence while maintaining the consistency of each target's identity across frames, and outputting bounding box trajectories with unique identifiers. Unlike traditional pedestrian or vehicle tracking scenarios, the maritime tracking environment is characterized by a wide field of view, large variations in target motion scale, and relatively simple but dynamically disturbed backgrounds. In recent years, with the widespread application of unmanned surface vessels (USVs) in autonomous maritime navigation, port monitoring, and traffic flow analysis, multi-target tracking technology for USV platforms has received increasing attention.

[0003] Existing detection-based tracking paradigms primarily rely on appearance features and Kalman filtering for data association. However, in long-range maritime scenarios, appearance features are unreliable, and Kalman filtering motion predictions are susceptible to drift due to camera shake. While Transformer-based trackers offer superior performance, their high computational complexity makes them unsuitable for USV edge deployment. Summary of the Invention

[0004] This invention discloses an efficient multi-target tracking method for unmanned surface vessels based on visual geometric priors to overcome the aforementioned technical problems.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows: An efficient multi-target tracking method for unmanned surface vessels based on visual geometric priors includes the following steps: S1: Obtain the temporal sequence of images including the current frame of the marine image, and use a target detector to obtain the output of the target detector and perform preprocessing; the output of the target detector includes multiple active trajectories, the ID of the target in the marine image, the target detection box, and the confidence of the detection box; S2: Based on the output of the target detector, obtain the original displacement of the target between adjacent frames; then obtain the attenuated displacement of the target between adjacent frames. S3: Based on the attenuated displacement, obtain the smoothed final velocity of the target to obtain the predicted detection box position of the active trajectory in the current frame of the sea image; S4: Based on the predicted detection box position of the active trajectory in the current frame of the marine image, obtain the distance cost between the detection box in the current frame of the marine image and the active trajectory. S5: Obtain the sequential matching cost between the detection box and the active trajectory in the current frame of the marine image, and obtain the total matching cost between the detection box and the active trajectory in the marine image, including the scale cost and the confidence cost, based on the distance cost between the detection box and the active trajectory in the current frame of the marine image. S6: Based on the total matching cost matrix, the Hungarian matching algorithm is used to obtain preliminary trajectory-detection box matching pairs; the total matching cost matrix is ​​updated based on the intersection-union ratio and movement distance of the trajectory-detection box matching pairs to obtain the updated total matching cost matrix; and the Hungarian algorithm is used based on the updated total matching cost matrix to obtain the updated trajectory-detection box matching pairs; and the positions of unmatched detection boxes in the current frame of the maritime image are obtained. S7: Establish a candidate pool for storing the information of unmatched detection boxes in all previous marine images of the current frame as candidate entries; update the candidate entries in the candidate pool according to the position of the unmatched detection boxes in the current marine image; wherein, the information of unmatched detection boxes in the marine image includes the center position of the detection box, the confidence of the detection box, and the size of the detection box. S8: Based on the candidate entries in the updated candidate pool, determine whether the candidate entries can be promoted to trajectories to obtain an updated list of trajectories; S9: If there are two non-overlapping trajectories in the updated trajectory list, obtain the time interval, spatial interval, and area scale ratio of the two trajectories to determine whether the two trajectories can be merged. If the two trajectories can be merged, they will be merged to obtain the final trajectory list. Based on the trajectories in the final trajectory list, multi-target tracking of unmanned surface vessels based on visual geometric priors can be achieved.

[0006] Furthermore, the formula used to obtain the original displacement of the target in adjacent frames is as follows:

[0007] In the formula: For the first The original displacement of the active trajectory between the k-th and (k-1)-th marine images; , For the first The active trajectory in the first Frame and the Coordinates of the observation center in the frame of the marine image; This is the index of the frame number of the marine image in the image time series; Index of active trajectories; The formula used to obtain the attenuated displacement is as follows:

[0008] In the formula: It is the first The attenuated displacement of an active trajectory between the k-th and (k-1)-th frames of the marine image; The attenuation coefficient; Let L be the L2 norm of the vector, i.e., the Euclidean distance; This is the threshold for determining pixel jitter.

[0009] Furthermore, the smoothed final velocity of the target and the predicted detection box position of the current frame of the sea image are obtained using the following formula:

[0010]

[0011] In the formula: For the first The final velocity of an active trajectory after exponential smoothing in the k-th frame; and These are the smooth fusion weights for historical velocity and current displacement, respectively; For the first The final velocity of an active trajectory after exponential smoothing in frame k-1; It is the first The attenuated displacement of an active trajectory between the k-th and (k-1)-th frames of the marine image; For the first The predicted center coordinates of the target detection box for each active trajectory in the (k+1)th frame of the marine image; For the first The active trajectory in the first The coordinates of the center position of the detection frame in the observation of the frame at sea.

[0012] Furthermore, the formula used to obtain the distance cost between the detection box and the active trajectory in the marine image is as follows: In the formula, The first one in the sea picture The detection box and the first The distance cost between active trajectories; For the first The predicted center coordinates of the active trajectory in the (k+1)th frame of the marine image; For the (k+1)th frame of the sea image, the first... The center coordinates of each detection box; Let L be the L2 norm of the vector; The maximum distance between the detection box and the active trajectory in a normalized maritime image; Index of active trajectories; This is the index of the detection boxes in the marine image.

[0013] Furthermore, the formula used to obtain the sequential matching cost between detection boxes and active trajectories in marine images is as follows:

[0014] In the formula: Cost of sequential matching; It is indexed by ID number, and also by active trajectory; This is the index of the detection boxes in the marine image; It is the absolute value; This represents the number of currently active trajectories. This represents the number of detection boxes in the current frame of the marine image.

[0015] Furthermore, the formula used to obtain the total matching cost between detection boxes and active trajectories in marine images is as follows:

[0016] In the formula: The first one in the sea picture The detection box and the first The total matching cost between active trajectories; The first one in the sea picture The detection box and the first The distance cost between active trajectories; The first one in the sea picture The detection box and the first Scale cost between active trajectories; Cost of sequential matching; For the cost of confidence; , , , All are weighting coefficients; in,

[0017] In the formula: , The first one in the sea picture The width and height of each detection box; The first Width and height of the predicted bounding box for each active trajectory in the image at sea;

[0018] In the formula: The first one in the sea picture The confidence score of each detection box.

[0019] Furthermore, the method used to update the candidate entries in the candidate pool is as follows: If no candidate entry exists in the candidate pool, then the unmatched detection box information in the current frame's marine image is added to the candidate pool as a candidate entry; otherwise, if a candidate entry exists in the candidate pool, then: If the absolute values ​​of the differences between the x-coordinate and y-coordinate of the center position of the detection box in the candidate entry and the position of the unmatched detection box in the current frame of the marine image are both less than a set pixel threshold, then the hit count of the candidate entry is incremented by one; the average confidence value of the candidate entry is updated; the position of the detection box in the candidate entry is updated to the position of the unmatched detection box in the current frame of the marine image; and the size of the detection box in the candidate entry is updated to the size of the unmatched detection box in the current frame of the marine image. If the absolute values ​​of the differences between the x-coordinate and y-coordinate of the center position of the detection box in the candidate entry and the position of the unmatched detection box in the current frame of the marine image are not both less than the set pixel threshold, but the confidence of the unmatched detection box in the current frame of the marine image is not less than the set confidence threshold, then the information of the unmatched detection box in the current frame of the marine image is added to the candidate pool as a new candidate entry.

[0020] Furthermore, the method used to determine whether the candidate entry can be promoted to a trajectory is as follows: S81: If the number of hits in a candidate entry reaches the minimum promotion threshold and the average confidence level is not lower than the average confidence level threshold, then execute S82; otherwise, the candidate entry cannot be promoted to a trajectory. S82: Traverse the trajectories in the trajectory list. If there is a trajectory whose number of sea image frames between the last frame and the current frame is less than a set image frame threshold, obtain the actual detection box position of the trajectory in the last frame. If the absolute value of the difference between the horizontal coordinate of the detection box position in the candidate entry and the actual detection box position of the trajectory in the last frame is less than the first promotion pixel threshold, and the absolute value of the difference between the vertical coordinate of the detection box position in the candidate entry and the actual detection box position of the trajectory in the last frame is less than the second promotion pixel threshold, then the candidate entry cannot be promoted to a trajectory; otherwise, execute S83. S83: When the number of frames between the last frame corresponding to the trajectory in the shadow library and the current frame of the sea image is greater than the set second image frame threshold, obtain the actual detection box position of the trajectory in the shadow library in the last frame, obtain the extrapolated predicted position coordinates of the trajectory in the shadow library, and then obtain the comprehensive matching score between the trajectory in the shadow library and the detection box corresponding to the candidate item. Obtain the trajectory from the shadow library with the highest comprehensive matching score between the detection boxes corresponding to the candidate item. If the comprehensive matching score is greater than the inheritance threshold, the candidate item is promoted to a trajectory and added to the trajectory list. The promoted trajectory inherits the ID number of the trajectory in the shadow library, and the candidate item is removed from the candidate pool. Otherwise, the candidate entry cannot be promoted to a trajectory.

[0021] Furthermore, the formula used to obtain the overall matching score is as follows: In the formula: The overall matching score; This is the weighted balance coefficient between the athletic score and the scale score; A score for matching the movements; The scale matching score; in,

[0022] In the formula: The predicted center coordinates are the extrapolated trajectories stored in the shadow library. All of these are the center coordinates of the detection box in the corresponding marine image at the moment the trajectory disappears, which are stored in the shadow library; All of these represent the speed at which the trajectories stored in the shadow library disappear; The frame difference between the current frame and the image frame where the trajectory stored in the shadow library disappears; For the tolerance of movement distance; All are the average position coordinates of the candidate in the kth frame; In the formula: The area of ​​the candidate box; The area of ​​the detection box in the corresponding sea image at the moment the trajectory disappears, stored in the shadow library. This is the scale sensitivity control parameter.

[0023] Furthermore, the formulas used to obtain the time interval and spatial interval between the two trajectories are as follows:

[0024]

[0025] In the formula: For time intervals; For the next trajectory segment Start frame number; For the previous trajectory segment End frame number; Spatial distance; For the previous trajectory segment The coordinates of the target observation center corresponding to the end frame; For the next trajectory segment The target observation center coordinates corresponding to the starting frame; The formula used to obtain the area scale ratio of the two trajectories is as follows:

[0026] In the formula: Area scale ratio; , Each is a segment of the previous trajectory. and the next trajectory segment The average area of ​​the detection frame; The conditions for determining whether two trajectories can be merged are as follows:

[0027]

[0028]

[0029]

[0030] In the formula: For time intervals; This is the maximum allowed frame splicing interval; Spatial distance; This is the maximum allowable splicing space distance; Area scale ratio; This represents the maximum permissible scale difference ratio. This represents the average detection confidence of the previous trajectory segment a; This represents the average detection confidence of the subsequent trajectory segment b. The minimum detection confidence threshold; This is the set of all frame numbers covered by the previous trajectory segment a; This is the set of all frame numbers covered by the next trajectory segment b.

[0031] Beneficial Effects: This invention provides an efficient multi-target tracking method for unmanned surface vessels (USVs) based on visual geometric priors. By introducing a displacement attenuation and velocity smoothing mechanism based on visual geometric priors, it effectively overcomes the motion prediction drift problem caused by camera shake at sea and eliminates the reliance on easily disturbed Kalman filters. Simultaneously, it constructs a multi-dimensional total matching cost matrix that integrates scale cost and confidence cost, and combines a secondary optimization matching strategy based on intersection-union ratio (IUU) and motion distance, significantly improving data association accuracy under conditions of drastic changes in target scale and dynamic background interference. Furthermore, by setting up a candidate pool for temporal management of unmatched detection boxes and intelligently merging temporally non-overlapping trajectories based on time intervals, spatial intervals, and area scale comparisons, it successfully solves the problem of trajectory interruption and fragmentation caused by target occlusion due to waves or detector misses. In summary, this invention, while ensuring high tracking accuracy, avoids highly complex appearance feature extraction and Transformer operations, achieving efficient utilization of computational resources. It is particularly suitable for real-time, robust multi-target tracking on the edge platforms of USVs. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a flowchart of the efficient multi-target tracking method for unmanned surface vessels based on visual geometric priors of the present invention; Figure 2 This is a schematic diagram of the overall architecture of the multi-target tracking system for unmanned vessels in an embodiment of the present invention; Figure 3 This is a schematic diagram of the overall workflow of the efficient multi-target tracking method for unmanned vessels in this embodiment of the invention; Figure 4 This is a schematic diagram of the horizontal order stable geometric prior principle in an embodiment of the present invention; Figure 5 This is a schematic diagram of the hierarchical verification mechanism structure of the observation access control mechanism in this embodiment of the invention; Figure 6 This is a schematic diagram of the marine target detection dataset constructed according to an embodiment of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] This embodiment introduces an efficient multi-target tracking method for unmanned surface vessels based on visual geometric priors, including the following steps: Figure 1 As shown: S1: Obtain the temporal sequence of images including the current frame of the marine image, and use the YOLO11L target detector to obtain the output of the YOLO11L target detector and perform preprocessing. The output of the YOLO11L target detector includes multiple active trajectories, the ID of the target in the marine image, the target detection box, and the confidence of the detection box. This embodiment makes targeted improvements to the general detection-then-track baseline framework to construct a multi-target tracking model at sea. During system initialization, the trajectory list is empty, with no active trajectories. Furthermore, initially, due to the lack of trajectory objects, they are automatically skipped, and all detection boxes are directly processed by the new trajectory confirmation mechanism, gradually establishing the initial trajectory set through multi-frame cumulative verification.

[0036] The preprocessing of the YOLOv11L target detector output includes: filtering out targets with confidence levels below a minimum threshold. The detection box, and the loss of tracking duration exceeding the maximum forgetting threshold. The trajectory is temporarily removed from the matching pool.

[0037] S2: Based on the output of the YOLO11L object detector, obtain the original displacement of the target between adjacent frames; then obtain the attenuated displacement of the target between adjacent frames to attenuate small displacements less than the jitter threshold and suppress prediction drift caused by camera jitter. Preferably, the formula used to obtain the original displacement of the target in adjacent frames is as follows: (1) In the formula: For the first The original displacement of the active trajectory between the k-th and (k-1)-th marine images; , For the first The active trajectory in the first Frame and the Coordinates of the observation center in the frame of the marine image; This is the index of the maritime image frame number in the image time sequence, representing the video frame number currently being processed; This is an index for active trajectories, where each ID number represents a tracking target and corresponds to a trajectory.

[0038] The formula used to obtain the attenuated displacement is as follows: (2) In the formula: It is the first The attenuated displacement of an active trajectory between the k-th and (k-1)-th frames of the marine image; The attenuation coefficient is... =0.5; The L2 norm of the vector, i.e., the Euclidean distance, is used to calculate the pixel length of the displacement vector; The threshold for determining pixel jitter. =4.

[0039] The motion prediction mechanism in this embodiment adopts an online velocity decay prediction scheme. By calculating the displacement after decay of each active trajectory, prediction drift caused by camera shake can be suppressed with extremely low computational overhead.

[0040] S3: Based on the attenuated displacement, obtain the smoothed final velocity of the target to obtain the predicted detection box position of the active trajectory in the current frame of the sea image; Preferably, the smoothed final velocity of the target and the predicted detection box position of the current frame of the marine image are obtained using the following formula: (3) (4) In the formula: For the first The final velocity of an active trajectory after exponential smoothing in the k-th frame; and These are the smooth fusion weights for historical velocity and current displacement, respectively; For the first The final velocity of an active trajectory after exponential smoothing in frame k-1; It is the first The attenuated displacement of an active trajectory between the k-th and (k-1)-th frames of the marine image; For the first The predicted center coordinates of the target detection box for each active trajectory in the (k+1)th frame of the marine image; For the first The active trajectory in the first The coordinates of the center position of the detection frame in the observation of the frame at sea.

[0041] After solving for the target frame prediction box and center coordinates, the prediction center and predicted width and height of all active trajectories are output to the sequential constraint data association unit as the geometric input basis for cross-frame matching.

[0042] In this embodiment, the velocity is updated smoothly by exponential moving average, and the predicted position for the next frame is extrapolated: the width and height of the trajectory are kept at the latest observation value, and the new trajectory is fixed at the initial observation position when less than two frames have been observed.

[0043] In this embodiment, based on the predicted position of each active trajectory output by the motion prediction mechanism in the current frame, the predicted center coordinates of the trajectories and the center coordinates of each detection box in the same frame of the marine image are obtained. All active trajectories and detection boxes in the current frame are sorted in ascending order of their horizontal center coordinates to obtain an ordered trajectory sequence and an ordered detection sequence. The total matching cost between the detection boxes and active trajectories is calculated. When the total matching cost is minimized, a matching pair of detection boxes and active trajectories in the marine image is obtained.

[0044] S4: Based on the predicted detection box position of the active trajectory in the current frame of the marine image, obtain the distance cost between the detection box in the current frame of the marine image and the active trajectory. Preferably, the formula used to obtain the distance cost between the detection box and the active trajectory in the marine image is as follows: (5) In the formula, The first one in the sea picture The detection box and the first The distance cost between active trajectories; For the first The predicted center coordinates of the active trajectory in the (k+1)th frame of the marine image; For the (k+1)th frame of the sea image, the first... The center coordinates of each detection box; Let L be the L2 norm of the vector; The maximum distance between the detection box and the active trajectory in a normalized maritime image; Index of active trajectories; This is the index of the detection boxes in the marine image; where each marine image may contain multiple targets, and each target corresponds to a detection box.

[0045] In this embodiment, the first [item] in the marine image is defined. The active trajectory and the first The distance cost between each detection box is the normalized result of the Euclidean distance between their center positions, which is used to quantify motion consistency.

[0046] S5: Obtain the sequential matching cost between the detection box and the active trajectory in the current frame of the marine image, and obtain the total matching cost between the detection box and the active trajectory in the marine image, including the scale cost and the confidence cost, based on the distance cost between the detection box and the active trajectory in the current frame of the marine image. Preferably, the formula used to obtain the sequential matching cost between detection boxes and active trajectories in the marine image is as follows: (6) In the formula: The cost of sequential matching, with a value range of [0,1], when The time cost is minimized, which means the position is perfectly matched, and it conforms to the order-preserving prior. It is indexed by ID number, and also by active trajectory; This is the index of the detection boxes in the marine image; It is the absolute value; This represents the number of currently active trajectories. This represents the number of detection boxes in the current frame of the marine image.

[0047] In this embodiment, the sorted number is defined as... The active trajectory and the first The order cost of each detection box is the normalized position difference, which is used to quantify the degree of matching between the two horizontal positions.

[0048] Preferably, the formula used to obtain the total matching cost between the detection box and the active trajectory in the marine image is as follows: (7) In the formula: The first one in the sea picture The detection box and the first The total matching cost between active trajectories; The first one in the sea picture The detection box and the first The distance cost between active trajectories is truncated to [0,1] after maximum distance normalization; The first one in the sea picture The detection box and the first The scale cost between active trajectories is the absolute value of the ratio of the logarithmic area of ​​the detection box to the area of ​​the trajectory prediction box, quantizing scale consistency and truncated to [0,1]. Cost of sequential matching; The confidence cost is the complement of the detection confidence. It prioritizes matching detections with higher confidence. The value range is [0,1]. The smaller the value, the higher the detection confidence. , , , All are weighting coefficients; in, (8) In the formula: , The first one in the sea picture The width and height of each detection box; The first The width and height of the predicted bounding box for each active trajectory in the marine image; this embodiment measures the width and height of the predicted bounding box (i.e., the first active trajectory) by detecting the area of ​​the bounding box and comparing it with the predicted bounding box (i.e., the first active trajectory). The cost is constructed by the log-absolute value of the ratio of the area of ​​the detection boxes at the predicted locations of the active trajectories, with a truncation cap of 1.

[0049] (9) In the formula: The first one in the sea picture The confidence score of each detection box is output by the YOLO11l object detector.

[0050] In this embodiment, the first is defined Trajectory and the first The confidence cost between detection boxes is the complement of the detection confidence score, which is 1 minus the detection confidence score. In other words, the higher the detection confidence, the lower the confidence cost and the higher the matching priority; conversely, the lower the detection confidence, the higher the confidence cost and the lower the matching priority. This method guides the matching algorithm to prioritize detection results with higher confidence, suppressing the interference of low-confidence false positives on the association results, and prioritizing the matching of detections with higher confidence.

[0051] In this embodiment, a sequence constraint data association mechanism is constructed to introduce target horizontal sequence prior constraints for cross-frame matching. Specifically, the sequence constraint data association mechanism is based on the geometric prior that the horizontal relative order of ships at a distance remains stable over a short period of time. In addition to the traditional motion, scale, and confidence costs, a sequence matching cost is added to constrain the matching relationship at the global position level, thereby reducing ID mismatches caused by camera shake.

[0052] S6: Based on the total matching cost matrix, the Hungarian matching algorithm is used to obtain preliminary trajectory-detection box matching pairs; the total matching cost matrix is ​​updated based on the intersection-union ratio and movement distance of the trajectory-detection box matching pairs to obtain the updated total matching cost matrix; and the Hungarian algorithm is used based on the updated total matching cost matrix to obtain the updated trajectory-detection box matching pairs; and the positions of unmatched detection boxes in the current frame of the maritime image are obtained. In this embodiment, before determining the final trajectory-detection box matching pair, a hard constraint check is first added: if the intersection-union ratio (IoU) of a trajectory-detection box matching pair is lower than the IoU rejection threshold... Furthermore, the motion distance of the trajectory-detection box matching pair exceeds the motion rejection threshold. The total matching cost corresponding to the trajectory-detection box matching pair is then... Setting the value to infinity forcibly prohibits this pairing, eliminating erroneous matches with low overlap over long distances, and thus obtaining an updated total matching cost matrix. Finally, based on the updated total cost matrix, the Hungarian algorithm is executed to perform binary matching, retaining only valid pairs with a total matching cost below a threshold, and updating the state of the corresponding trajectory with the matched detection boxes. Here, the motion distance of the trajectory-detection box matching pair is the distance between the predicted detection box position of the active trajectory in the current frame of the maritime image and the detection box in the matching pair.

[0053] S7: Establish a candidate pool to store the information of unmatched detection boxes in all previous marine images of the current frame as candidate entries; update the candidate entries in the candidate pool according to the position of the unmatched detection boxes in the current marine image. Information on unmatched bounding boxes in marine images includes the center position of the bounding box, the confidence level of the bounding box, and the size of the bounding box; Preferably, the method used to update the candidate entries in the candidate pool is as follows: If no candidate entry exists in the candidate pool, then the unmatched detection box information in the current frame's marine image is added to the candidate pool as a candidate entry; otherwise, if a candidate entry exists in the candidate pool, then: If the absolute values ​​of the differences between the x-coordinate and y-coordinate of the center position of the detection box in the candidate entry and the position of the unmatched detection box in the current frame's marine image are both less than a set pixel threshold, then the hit count of the candidate entry is incremented by one; and the average confidence score of the candidate entry (i.e., the average confidence score of the detection boxes in all marine images corresponding to the candidate entry) is updated; the position of the detection box in the candidate entry is updated to the position of the unmatched detection box in the current frame's marine image; and the size of the detection box in the candidate entry is updated to the size of the unmatched detection box in the current frame's marine image. If the absolute values ​​of the differences between the x-coordinate and y-coordinate of the center position of the detection box in the candidate entry and the position of the unmatched detection box in the current frame of the marine image are not both less than the set pixel threshold, but the confidence of the unmatched detection box in the current frame of the marine image is not less than the set confidence threshold, then the information of the unmatched detection box in the current frame of the marine image is added to the candidate pool as a new candidate entry.

[0054] If the center position of the detection box in the candidate item does not meet the above two conditions, the matched detection box is meaningless for path tracing and will not be discussed.

[0055] In this embodiment, the candidate pool is an empty list during system initialization. After each frame of Hungarian matching is completed, all detection boxes that do not match the active trajectory enter the candidate accumulation process. First, the existing candidate entries in the candidate pool are traversed for first-level spatial matching: if the difference between the coordinates of the unmatched detection box in the current frame of the marine image and the coordinates of the detection box in the candidate entry in the candidate pool is less than the set pixel threshold (in this embodiment, the absolute value of the x-coordinate difference is less than 40 pixels and the absolute value of the y-coordinate difference is less than 40 pixels), it is determined to be a subsequent observation of the same target, and the hit count of the candidate entry is incremented by one. The center coordinates of the position of the detection box in the candidate entry are the coordinates of the unmatched detection box in the current frame of the marine image. The confidence of the detection box in the candidate entry is updated to the average confidence of all marine image frames hit by the candidate entry, i.e., (average confidence of candidate entries in the candidate pool corresponding to the previous frame of the marine image × (hits-1) + current confidence) / hits, where hits is the total number of all marine image frames hit by the candidate entry. The width and height are updated to the latest observation values.

[0056] If there are no x-coordinate differences with an absolute value less than 40 pixels and y-coordinate differences with an absolute value less than 40 pixels, and the confidence level of any unmatched detection box in the current frame of the maritime image is not less than 0.6, then a new candidate entry is created and added to the candidate pool using the information of the unmatched detection boxes in the current frame of the maritime image. In this implementation, the candidate entry only records the latest center coordinates, cumulative hit count, moving average of confidence levels for each frame, and current size of the potential tracking target; it does not have a trajectory ID, historical position sequence, or movement speed, and is not a formal trajectory.

[0057] In this embodiment, an observation access control mechanism is constructed to perform tiered filtering of false detections and duplicate trajectories. The observation access control mechanism uses a four-layer collaborative sub-mechanism. When there are still remaining detection boxes after Hungarian matching, this mechanism is activated to control the matching access of detection boxes and the generation of new trajectories, suppressing false trajectories caused by false detections at sea, avoiding the generation of multiple duplicate trajectories for the same target, and restoring the target identity that has been lost for a short period of time through the shadow trajectory database.

[0058] The four-layer collaborative sub-mechanism includes: Scale Threshold: Utilizing the prior knowledge that the projected area of ​​distant ships changes gradually within a short time window, the area ratio of the detection box to the predicted trajectory box in the trajectory-detection box matching pair is calculated before matching. Only when the ratio is within the preset minimum and maximum area ratio range can the trajectory-detection box matching pair participate in the calculation of the total matching cost, filtering unreasonable pairings due to abrupt scale changes. Motion Outlier Filtering: Only extreme spatial jump pairings with extremely large center distance deviations (more than 180 pixels) are filtered, maintaining a relaxed motion constraint boundary to adapt to scenes with slight camera shake caused by ocean waves, avoiding over-filtering of effective matches. New Trajectory Confirmation Mechanism: Abandoning the strategy of directly generating new trajectories from single-frame detection, the candidate pool performs multi-frame cumulative verification of unmatched detections. IoU Deduplication: After each frame trajectory update, the recently updated active trajectories are traversed, and the intersection-union ratio (IoU) of the latest observation boxes is calculated pairwise; if the IoU exceeds the duplication threshold... If the value is 0.3, it is determined to be repeated tracking of the same target. The trajectory with the longer continuous observation history is retained, and the other redundant trajectory is deleted.

[0059] S8: Based on the candidate entries in the updated candidate pool, determine whether the candidate entries can be promoted to tracks. When a candidate entry can be promoted to a track, remove the candidate entry from the candidate pool and add the promoted track to the track list to obtain an updated track list. Preferably, the method used to determine whether the candidate entry can be promoted to a trajectory is as follows: S81: If the number of hits in a candidate entry reaches the minimum promotion threshold and the average confidence level is not lower than the average confidence level threshold, then execute S82; otherwise, the candidate entry cannot be promoted to a trajectory. S82: Traverse the trajectories in the trajectory list. If there is a trajectory where the number of sea image frames between the last frame and the current frame is less than a set image frame threshold, i.e., there is a trajectory where the target loss time is less than the set image frame threshold, then obtain the actual detection box position of the trajectory in the last frame. If the absolute value of the difference between the horizontal coordinate of the detection box position in the candidate entry and the actual detection box position of the trajectory in the last frame is less than the first promotion pixel threshold, and the absolute value of the difference between the vertical coordinate of the detection box position in the candidate entry and the actual detection box position of the trajectory in the last frame is less than the second promotion pixel threshold, then the candidate entry cannot be promoted to a trajectory; otherwise, execute S83. S83: When the number of frames between the last frame corresponding to the trajectory in the shadow library and the current frame of the sea image is greater than the set second image frame threshold, obtain the actual detection box position of the trajectory in the shadow library in the last frame, obtain the extrapolated predicted position coordinates of the trajectory in the shadow library, and then obtain the comprehensive matching score between the trajectory in the shadow library and the detection box corresponding to the candidate item. Obtain the trajectory from the shadow library with the highest comprehensive matching score between the detection boxes corresponding to the candidate item. If the comprehensive matching score is greater than the inheritance threshold, the candidate item is promoted to a trajectory and added to the trajectory list. The promoted trajectory inherits the ID number of the trajectory in the shadow library, and the candidate item is removed from the candidate pool. Otherwise, the candidate entry cannot be promoted to a trajectory.

[0060] In this embodiment, the second image frame threshold is greater than the image frame threshold; Preferably, the formula used to obtain the comprehensive matching score is as follows: (10) In the formula: The overall matching score; This is the weighted balance coefficient between the athletic score and the scale score; A score for matching the movements; The scale matching score; in, (11) (12) In the formula: The predicted center coordinates are the extrapolated trajectories stored in the shadow library. All of these are the center coordinates of the detection box in the corresponding marine image at the moment the trajectory disappears, which are stored in the shadow library; All of these represent the speed at which the trajectories stored in the shadow library disappear; The frame difference is the difference between the current frame and the image frame where the trajectory disappears, stored in the shadow library. The predicted position of the shadow trajectory in the current frame is obtained by multiplying the position and velocity at the moment of disappearance by the frame difference and adding them together. This prediction is then used to calculate the motion matching score with the candidate positions. For the tolerance of movement distance; These are the position coordinates of the detection box in the candidate item; (13) In the formula: The area of ​​the detection box in the candidate item; The area of ​​the detection box in the corresponding sea image at the moment the trajectory disappears, stored in the shadow library. This is the scale sensitivity control parameter.

[0061] Specifically, the initial trajectory list is empty. During the promotion evaluation phase, each frame performs a three-level verification on every candidate entry in the candidate pool. The first level is the basic condition: the cumulative number of hits for each candidate entry. Reaching the minimum promotion threshold And average confidence level Not lower than the confidence level mean threshold Entries that do not meet the requirements will remain in the pool for further observation.

[0062] The second level is spatial conflict verification: Trajectories in the trajectory list with a target loss time of less than 10 frames are traversed. The actual detection box position at the time of the last successful match is compared with the position of the unmatched detection box in the current frame's maritime image. If the absolute value of the x-coordinate difference is less than 15 pixels and the absolute value of the y-coordinate difference is less than 30 pixels, it is determined that the area has been covered by an existing trajectory, and the promotion is canceled. Otherwise, the third level of shadow library inheritance matching is performed. The third level is shadow library inheritance matching: The shadow library stores the final position, velocity, scale, and disappearance frame number of recently lost trajectories (more than 45 frames of lost target). Each shadow record in the shadow library is traversed through candidate shadows after spatial conflict verification. The actual detection box position and motion velocity at the disappearance time of the shadow are combined with the time interval and extrapolated to the current frame. The motion matching score between the extrapolated position and the detection box position in the candidate entry is calculated. Simultaneously, the scale matching score is calculated between the area of ​​the detection box in the candidate entry and the area of ​​the detection box in the last frame of the shadow trajectory. The motion score and scale score are weighted to obtain a comprehensive matching score. The highest comprehensive score among all shadow trajectories is taken. If it exceeds the inheritance threshold... If a candidate entry is successfully promoted to a track, it inherits the original track ID from the shadow record and is removed from the candidate pool. Upon successful promotion, a formal track object is created with the candidate entry's current position and size, inherits the candidate's hit count, and is added to the track list.

[0063] S9: If there are two non-overlapping trajectories in the updated trajectory list, obtain the time interval, spatial interval, and area scale ratio of the two trajectories to determine whether the two trajectories can be merged. If the two trajectories can be merged, they will be merged to obtain the final trajectory list. Based on the trajectories in the final trajectory list, multi-target tracking of unmanned surface vessels based on visual geometric priors can be achieved.

[0064] Preferably, the formulas used to obtain the time interval and spatial interval of the two trajectories are as follows: (14) (15) In the formula: For time intervals; For the next trajectory segment Start frame number; For the previous trajectory segment End frame number; Spatial distance, i.e. and The Euclidean distance between them; For the previous trajectory segment The coordinates of the target observation center corresponding to the end frame; For the next trajectory segment The coordinates of the target observation center corresponding to the starting frame.

[0065] The formula used to obtain the area scale ratio of the two trajectories is as follows: (16) In the formula: Area scale ratio; , Each is a segment of the previous trajectory. and the next trajectory segment The average detection frame area is calculated by dividing the larger area by the smaller area, and the ratio is always greater than or equal to 1.

[0066] The conditions for determining whether two trajectories can be merged are as follows: Specifically, after full video tracking is completed, global post-processing is performed to merge broken trajectory segments caused by short-term missed detections or occlusions, achieving offline trajectory segment stitching. Two segments can be determined to be mergeable segments of the same target only if they simultaneously meet all of the following constraints:

[0067] (17) In the formula: For time intervals; This is the maximum allowed frame splicing interval; Spatial distance; This is the maximum allowable splicing space distance; Area scale ratio; This represents the maximum permissible scale difference ratio. This represents the average detection confidence of the previous trajectory segment a; This represents the average detection confidence of the subsequent trajectory segment b. The minimum detection confidence threshold; This is the set of all frame numbers covered by the previous trajectory segment a; For the set of all frame numbers covered by the next trajectory segment b; in, Ensure that the frame numbers covered by the two segments do not overlap. When multiple candidate pairs exist, prioritize merging segments with smaller time intervals to generate a continuous and complete trajectory. This process is performed offline only and does not increase the computational burden of online inference.

[0068] The efficient multi-target tracking method for unmanned surface vessels (USVs) based on visual geometric priors (OP-MOT) provided in this embodiment is an improvement on the detection-before-track (TBD) baseline framework. First, it mines geometric priors with relatively stable horizontal coordinates of ships at long distances at sea and constructs a sequence-constrained data association mechanism to integrate global sequence constraints into cross-frame detection and matching. Second, it designs a hierarchical observation access control mechanism through scale gating, new trajectory confirmation mechanism, motion outlier filtering, and cross-validation deduplication. In addition, it constructs a lightweight linear extrapolation motion prediction and trajectory bridging mechanism to suppress prediction errors caused by camera shake and repair broken trajectories through bridging.

[0069] A specific embodiment of the present invention is as follows: A maritime multi-target tracking evaluation dataset was constructed for model validation and performance evaluation. This embodiment uses the Singapore Maritime Dataset (SMD) and the USVTrack public maritime tracking dataset as benchmarks. SMD covers open ocean scenes, including both shipborne and shore-based perspectives, with long target distances and significant camera shake; USVTrack covers complex port and inland waterway scenes, including challenging scenarios such as dense ship traffic and turning / intersection. The datasets were divided into training, validation, and test sets according to standards. YOLO series object detectors were uniformly fine-tuned on the corresponding datasets, generating a fixed detection result cache. All tracking algorithms participating in the comparison reused the same batch of detection results to ensure fairness in the tracking performance comparison.

[0070] This embodiment comprehensively verifies the tracking accuracy, identity stability, and engineering feasibility of the method through targeted fine-tuning of the detector, multi-dimensional performance evaluation, and lightweight edge deployment.

[0071] The target detector uses YOLOv11l as a baseline and is fine-tuned on the SMD and USVTrack maritime datasets based on publicly available pre-trained weights. Training is conducted on a single RTX 4090 GPU using the Ultralytics framework and PyTorch 2.1.2. The training employs the AdamW optimizer and a cosine learning rate scheduling strategy, along with techniques such as horizontal flipping and mosaic enhancement to improve generalization ability. An early stopping mechanism is implemented to select the weights with the best overall performance on the validation set. To ensure fairness in tracking comparisons, this embodiment constructs a unified detection benchmark and generates a cache of detection results using a fixed confidence level and a non-maximum suppression threshold. All tracking algorithms participating in the comparison reuse the same batch of detection results, eliminating the interference of detector performance differences on the evaluation.

[0072] Table 1 shows the quantitative performance comparison results of OP-MOT in this embodiment with more than ten mainstream multi-target tracking algorithms on the SMD marine dataset.

[0073] Table 1

[0074] The metrics used for comparison in Table 1 include: HOTA (Higher Order Tracking Accuracy), MOTA (Multiple Object Tracking Accuracy), IDF1 (IDF1-score), DetA (Detection Accuracy), Rec. (Recall), Prec. (Precision), AssA (Association Accuracy), IDSW (ID Switch), and Hz (Frames per Second).

[0075] The experiment employed the TrackEval tool and the HOTA official evaluation protocol to construct a multi-dimensional evaluation system: HOTA, MOTA, and IDF1 were used as core accuracy indicators, ID switching count and trajectory fragmentation were used to measure identity stability, and inference frame rate was used to verify the algorithm's real-time performance. To ensure fairness in the comparison, all algorithms based on the "detect-then-track" approach reused the same fixed detection result cache generated by the YOLO11L detector to eliminate the interference of detector performance differences on tracking evaluation.

[0076] As shown in Table 1, on the SMD open ocean dataset, the online tracker using this method achieved 60.98% HOTA, 56.52% MOTA, and 73.02% IDF1, with only 15 ID switching times. After overlaying offline segments, the HOTA increased to 61.05%, and the ID switching times decreased to 12. Both the MOTA and HOTA metrics ranked first among the pure geometric trackers evaluated.

[0077] Overall, OP-MOT, with its pure geometric lightweight design that does not rely on appearance feature embedding and has no camera motion compensation, has leading performance among similar pure geometric lightweight trackers, balancing tracking accuracy, identity stability, and real-time deployment of edge devices.

[0078] Performance evaluation employs the TrackEval tool and the official HOTA evaluation protocol to establish a multi-dimensional evaluation system: HOTA, MOTA, and IDF1 are used as core accuracy indicators, ID switching count and trajectory fragment quantity are used to measure identity stability, and inference frame rate is statistically verified for real-time performance. Comparison with several mainstream algorithms on two standard datasets shows that the online tracker of this embodiment achieves 60.98% HOTA and 56.52% MOTA on the SMD offshore dataset, with only 15 ID switching times; after overlaying offline fragments, HOTA increases to 61.05%, and ID switching times decrease to 12. On the USVTrack complex surface dataset, it achieves 80.62% HOTA and 83.58% MOTA, with 103 ID switching times; after stitching, HOTA reaches 80.67%, and ID switching times decrease to 92, demonstrating leading performance among similar purely geometric lightweight trackers.

[0079] Ablation experiments verified that the observation access control mechanism is the core contributing unit to identity stability, which can significantly reduce the number of ID switching. Sequential constraint data association provides global geometric bias, and the motion prediction mechanism further suppresses prediction drift caused by camera shake. The three work together to achieve a balance between accuracy and stability. Detection noise stress testing shows that the method in this embodiment has significantly better performance retention than the baseline algorithm in high-miss-detection scenarios, and is suitable for the unstable actual working conditions of maritime detection.

[0080] Figure 2 This embodiment demonstrates the complete architecture and core processing chain of OP-MOT (Visual Geometric Prior-Based Multi-Target Tracking System for Unmanned Surface Vessels). The system takes maritime video frames and target detection results as input and comprises four core collaborative functional units: 1) a lightweight velocity decay motion prediction unit to suppress predicted position drift caused by camera shake; 2) an order-preserving master association module to mine horizontally stable geometric priors to construct global matching constraints; 3) an observation access control module to suppress false detections, avoid duplicate trajectories, and restore short-term lost target identities through layered verification; and 4) an offline segment stitching post-processing module to repair broken trajectories caused by missed detections or occlusion. The system ultimately outputs a continuous ship tracking trajectory, which can be lightweightly deployed on unmanned surface vessel edge computing devices to support real-time environmental perception and navigation / collision avoidance for distant vessels.

[0081] Figure 3The entire workflow of the OP-MOT method, "online frame-by-frame tracking + offline global post-processing," is fully presented: In the online processing stage, position prediction is first achieved using a small displacement attenuation strategy via the motion recovery module. Then, the order-preserving master association stage is entered, where a total matching matrix is ​​constructed by fusing four types of costs: motion distance, scale, horizontal order, and confidence. The Hungarian algorithm performs initial matching between the trajectory and the detection box, and invalid pairings are eliminated through dual constraints of intersection-union ratio (IoU) and motion distance. Unmatched detection boxes enter a controlled admission process. After multi-frame accumulation in the candidate pool and three-level promotion verification, they either inherit historical trajectory IDs from the shadow library or are assigned new IDs to generate official trajectories. Simultaneously, redundant and duplicate trajectories for the same target are deleted using the IoU deduplication mechanism, ultimately outputting the online tracking result. In the offline post-processing stage, after all video tracking is completed, segment bridging is performed. Based on five constraints—time interval, spatial distance, area scale, detection confidence, and temporal non-overlap—fragmented trajectory segments belonging to the same target are identified and merged to generate continuous and complete trajectories. This process does not alter the original online tracking result.

[0082] Figure 4 This paper elucidates the geometric prior of distant maritime targets, a core component of OP-MOT, from two dimensions: Subfigure (a) shows that the relative horizontal order of multiple targets in the camera image plane remains consistent under different hull attitudes when the unmanned surface vessel (USV) encounters lateral undulations; Subfigure (b) illustrates, using the complete navigation process of the USV traveling in a straight line from point A to point B and then turning along an arc to point C as an example, that even if the USV undergoes displacement, under the perspective projection model, the depth of distant vessels is much greater than the camera translation, and the relative horizontal coordinate order of the targets in the image plane remains stable. The geometric prior in this embodiment forms the theoretical basis for the sequential constraint data association mechanism. It can constrain cross-frame matching from a global positional level without relying on appearance features, effectively reducing ID mismatch problems in camera shake scenarios.

[0083] Figure 5This section showcases the four-layer collaborative architecture of the observation access control mechanism in OP-MOT. Its core functions are filtering false detections, controlling new trajectory generation, and preventing repeated tracking of the same target: Layer 1: Size Gating: By constraining the area ratio of the detection box and the trajectory prediction box, unreasonable pairings with abrupt scale changes are filtered, adapting to the characteristics of gradual changes in the projected area of ​​distant ships. Layer 2: Abnormal Motion Filtering: Only extreme spatial jump pairings with large center distance deviations are removed, maintaining a loose motion constraint boundary, adapting to scenarios with minor camera shake caused by ocean waves. Layer 3: New Trajectory Confirmation: A candidate trajectory pool is maintained, and unmatched detections are cumulatively verified across multiple frames. Only when a candidate target meets the requirements for cumulative hit count, average confidence, and passes spatial conflict verification can a new ID be assigned to generate a formal trajectory or inherit historical trajectory IDs from the shadow library. Layer 4: IoU Deduplication: The intersection-over-union ratio (IoU) of some active trajectories is calculated pairwise. When a trajectory is determined to be a duplicate of the same target, the trajectory with a longer continuous observation history is retained, and redundant trajectories are deleted.

[0084] Figure 6 The presentation showcases sample frames from two types of publicly available marine benchmark datasets used in the OP-MOT method evaluation, covering various marine conditions to comprehensively verify the method's performance: Region (A) is the Singapore Maritime Dataset (SMD): covering open ocean scenes, including both shipborne and shore-based perspectives, characterized by long target distances and significant camera shake. The illustration shows the ocean target images in frames 231, 251, 271, and 291 of the sequence; Regions (B) and (C) are the USVTrack dataset: covering complex water surface scenes in ports and inland waterways, including challenging scenarios such as dense ship traffic, turning and converging, and near-shore occlusion. The illustrations show multiple consecutive sample frames from the corresponding sequences.

[0085] This embodiment addresses the problems of low tracking accuracy, frequent identity switching, severe trajectory fragmentation, and high computational cost making deployment difficult in existing maritime multi-target tracking methods under conditions of long-range weak texture, severe camera shake, and limited edge computing power. It fully utilizes the geometric prior of horizontal order stability of long-range maritime targets on the image plane, constructing a collaborative mechanism of three working mechanisms: order-preserving master association, observation access, and motion recovery. This achieves efficient and robust maritime multi-target tracking without relying on appearance features or Kalman filtering. It is a maritime-specific multi-target tracking method that is independent of appearance features and highly robust to motion noise. The tracking method in this embodiment has no deep neural network inference or Kalman filtering covariance iteration calculations throughout the process; it can run only through basic numerical calculations and logical judgments, resulting in extremely low computational overhead. Deploying the algorithm on the edge computing device of an unmanned surface vessel (USV), with the detector accelerated by TensorRT inference and the tracking algorithm lightweighted and compiled, the end-to-end frame rate meets the requirements of real-time environmental perception. It can be integrated with USV navigation and collision avoidance systems, demonstrating significant engineering application value.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for efficient multi-target tracking of unmanned surface vessels based on visual geometric priors, characterized in that, Includes the following steps: S1: Obtain the temporal sequence of images including the current frame of the marine image, and use a target detector to obtain the output of the target detector and perform preprocessing; the output of the target detector includes multiple active trajectories, the ID of the target in the marine image, the target detection box, and the confidence of the detection box; S2: Based on the output of the target detector, obtain the original displacement of the target between adjacent frames; then obtain the attenuated displacement of the target between adjacent frames. S3: Based on the attenuated displacement, obtain the smoothed final velocity of the target to obtain the predicted detection box position of the active trajectory in the current frame of the sea image; S4: Based on the predicted detection box position of the active trajectory in the current frame of the marine image, obtain the distance cost between the detection box in the current frame of the marine image and the active trajectory. S5: Obtain the sequential matching cost between the detection box and the active trajectory in the current frame of the marine image, and obtain the total matching cost between the detection box and the active trajectory in the marine image, including the scale cost and the confidence cost, based on the distance cost between the detection box and the active trajectory in the current frame of the marine image. S6: Based on the total matching cost matrix, the Hungarian matching algorithm is used to obtain preliminary trajectory-detection box matching pairs; The total matching cost matrix is ​​updated based on the intersection-union ratio and the motion distance of the trajectory-detection box matching pairs, thus obtaining the updated total matching cost matrix. Based on the updated total matching cost matrix, the Hungarian algorithm is used to obtain updated trajectory-detection box matching pairs; To obtain the locations of unmatched detection boxes in the current frame of the marine image; S7: Establish a candidate pool to store the detection box information of all maritime images before the current frame as candidate entries; The candidate entries in the candidate pool are updated based on the positions of unmatched detection boxes in the current frame of the marine image; the information of the unmatched detection boxes in the marine image includes the center position of the detection box, the confidence level of the detection box, and the size of the detection box. S8: Based on the candidate entries in the updated candidate pool, determine whether the candidate entries can be promoted to trajectories to obtain an updated list of trajectories; S9: If there are two non-overlapping trajectories in the updated trajectory list, obtain the time interval, spatial interval, and area scale ratio of the two trajectories to determine whether the two trajectories can be merged. If the two trajectories can be merged, they will be merged to obtain the final trajectory list. Based on the trajectories in the final trajectory list, multi-target tracking of unmanned surface vessels based on visual geometric priors can be achieved.

2. The efficient multi-target tracking method for unmanned surface vessels based on visual geometric priors according to claim 1, characterized in that, The formula used to obtain the original displacement of the target in adjacent frames is as follows: In the formula: For the first The original displacement of the active trajectory between the k-th and (k-1)-th marine images; , For the first The active trajectory in the first Frame and the Coordinates of the observation center in the frame of the marine image; This is the index of the frame number of the marine image in the image time series; Index of active trajectories; The formula used to obtain the attenuated displacement is as follows: In the formula: It is the first The attenuated displacement of an active trajectory between the k-th and (k-1)-th frames of the marine image; The attenuation coefficient; Let L be the L2 norm of the vector, i.e., the Euclidean distance; This is the threshold for determining pixel jitter.

3. The efficient multi-target tracking method for unmanned surface vessels based on visual geometric priors according to claim 1, characterized in that, The formula for obtaining the smoothed final velocity of the target and the predicted detection box position of the current frame of the marine image is as follows: In the formula: For the first The final velocity of an active trajectory after exponential smoothing in the k-th frame; and These are the smooth fusion weights for historical velocity and current displacement, respectively; For the first The final velocity of an active trajectory after exponential smoothing in frame k-1; It is the first The attenuated displacement of an active trajectory between the k-th and (k-1)-th frames of the marine image; For the first The predicted center coordinates of the target detection box for each active trajectory in the (k+1)th frame of the marine image; For the first The active trajectory in the first The coordinates of the center position of the detection frame in the observation of the frame at sea.

4. The efficient multi-target tracking method for unmanned surface vessels based on visual geometric priors according to claim 1, characterized in that, The formula used to obtain the distance cost between the detection box and the active trajectory in the marine image is as follows: In the formula, The first one in the sea picture The detection box and the first The distance cost between active trajectories; For the first The predicted center coordinates of the active trajectory in the (k+1)th frame of the marine image; For the (k+1)th frame of the sea image, the first... The center coordinates of each detection box; Let L be the L2 norm of the vector; The maximum distance between the detection box and the active trajectory in a normalized maritime image; Index of active trajectories; This is the index of the detection boxes in the marine image.

5. The efficient multi-target tracking method for unmanned surface vessels based on visual geometric priors according to claim 1, characterized in that, The formula used to obtain the sequential matching cost between bounding boxes and active trajectories in marine images is as follows: In the formula: Cost of sequential matching; It is indexed by ID number, and also by active trajectory; This is the index of the detection boxes in the marine image; It is the absolute value; This represents the number of currently active trajectories. This represents the number of detection boxes in the current frame of the marine image.

6. The efficient multi-target tracking method for unmanned surface vessels based on visual geometric priors according to claim 1, characterized in that, The formula used to obtain the total matching cost between bounding boxes and active trajectories in a marine image is as follows: In the formula: The first one in the sea picture The detection box and the first The total matching cost between active trajectories; The first one in the sea picture The detection box and the first The distance cost between active trajectories; The first one in the sea picture The detection box and the first Scale cost between active trajectories; Cost of sequential matching; For the cost of confidence; , , , All are weighting coefficients; in, In the formula: , The first one in the sea picture The width and height of each detection box; The first Width and height of the predicted bounding box for each active trajectory in the image at sea; In the formula: The first one in the sea picture The confidence score of each detection box.

7. The efficient multi-target tracking method for unmanned surface vessels based on visual geometric priors according to claim 1, characterized in that, The method used to update candidate entries in the candidate pool is as follows: If no candidate entry exists in the candidate pool, then the unmatched detection box information in the current frame's marine image is added to the candidate pool as a candidate entry; otherwise, if a candidate entry exists in the candidate pool, then: If the absolute values ​​of the differences between the x-coordinate and y-coordinate of the center position of the detection box in the candidate item and the position of the unmatched detection box in the current frame of the marine image are both less than the set pixel threshold, then the hit count of the candidate item is incremented by one; and the average confidence value of the candidate item is updated. The position of the detection box in the candidate entry is updated to the position of the unmatched detection box in the current frame of the marine image; the size of the detection box in the candidate entry is updated to the size of the unmatched detection box in the current frame of the marine image. If the absolute values ​​of the differences between the x-coordinate and y-coordinate of the center position of the detection box in the candidate entry and the position of the unmatched detection box in the current frame of the marine image are not both less than the set pixel threshold, but the confidence of the unmatched detection box in the current frame of the marine image is not less than the set confidence threshold, then the information of the unmatched detection box in the current frame of the marine image is added to the candidate pool as a new candidate entry.

8. The efficient multi-target tracking method for unmanned surface vessels based on visual geometric priors according to claim 1, characterized in that, The method used to determine whether the candidate entry can be promoted to a trajectory is as follows: S81: If the number of hits in a candidate entry reaches the minimum promotion threshold and the average confidence level is not lower than the average confidence level threshold, then execute S82; otherwise, the candidate entry cannot be promoted to a trajectory. S82: Traverse the trajectories in the trajectory list. If there is a trajectory whose number of sea image frames between the last frame and the current frame is less than a set image frame threshold, obtain the actual detection box position of the trajectory in the last frame. If the absolute value of the difference between the horizontal coordinate of the detection box position in the candidate entry and the actual detection box position of the trajectory in the last frame is less than the first promotion pixel threshold, and the absolute value of the difference between the vertical coordinate of the detection box position in the candidate entry and the actual detection box position of the trajectory in the last frame is less than the second promotion pixel threshold, then the candidate entry cannot be promoted to a trajectory; otherwise, execute S83. S83: When the number of frames between the last frame corresponding to the trajectory in the shadow library and the current frame of the sea image is greater than the set second image frame threshold, obtain the actual detection box position of the trajectory in the shadow library in the last frame, obtain the extrapolated predicted position coordinates of the trajectory in the shadow library, and then obtain the comprehensive matching score between the trajectory in the shadow library and the detection box corresponding to the candidate item. Obtain the trajectory from the shadow library with the highest comprehensive matching score between the detection boxes corresponding to the candidate item. If the comprehensive matching score is greater than the inheritance threshold, the candidate item is promoted to a trajectory and added to the trajectory list. The promoted trajectory inherits the ID number of the trajectory in the shadow library, and the candidate item is removed from the candidate pool. Otherwise, the candidate entry cannot be promoted to a trajectory.

9. The efficient multi-target tracking method for unmanned surface vessels based on visual geometric priors according to claim 1, characterized in that, The formula used to obtain the overall matching score is as follows: In the formula: The overall matching score; This is the weighted balance coefficient between the athletic score and the scale score; A score for matching the movements; The scale matching score; in, In the formula: The predicted center coordinates are the extrapolated trajectories stored in the shadow library. All of these are the center coordinates of the detection box in the corresponding marine image at the moment the trajectory disappears, which are stored in the shadow library; All of these represent the speed at which the trajectories stored in the shadow library disappear; The frame difference between the current frame and the image frame where the trajectory stored in the shadow library disappears; For the tolerance of movement distance; All are the average position coordinates of the candidate in the kth frame; In the formula: The area of ​​the candidate box; The area of ​​the detection box in the corresponding sea image at the moment the trajectory disappears, stored in the shadow library. This is the scale sensitivity control parameter.

10. The efficient multi-target tracking method for unmanned surface vessels based on visual geometric priors according to claim 1, characterized in that, The formulas used to obtain the time interval and spatial interval between the two trajectories are as follows: In the formula: For time intervals; For the next trajectory segment Start frame number; For the previous trajectory segment End frame number; Spatial distance; For the previous trajectory segment The coordinates of the target observation center corresponding to the end frame; For the next trajectory segment The coordinates of the target observation center corresponding to the starting frame; The formula used to obtain the area scale ratio of the two trajectories is as follows: In the formula: Area scale ratio; , Each is a segment of the previous trajectory. and the next trajectory segment The average area of ​​the detection frame; The conditions for determining whether two trajectories can be merged are as follows: In the formula: For time intervals; This is the maximum allowed frame splicing interval; Spatial distance; This is the maximum allowable splicing space distance; Area scale ratio; This represents the maximum permissible scale difference ratio. This represents the average detection confidence of the previous trajectory segment a; This represents the average detection confidence of the subsequent trajectory segment b. This is the lowest detection confidence threshold; This is the set of all frame numbers covered by the previous trajectory segment a; This is the set of all frame numbers covered by the next trajectory segment b.

Images