A multi-target tracking and counting method for a smart ocean scene
By combining Kalman filtering prediction and dynamic confidence calibration with a two-stage matching strategy of historical context backtracking, the problem of target loss and trajectory fragmentation caused by occlusion and illumination changes in multi-target tracking is solved, and high-precision counting is achieved in smart ocean scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- QINGDAO UNIV OF SCI & TECH
- Filing Date
- 2026-04-22
- Publication Date
- 2026-06-19
Smart Images
Figure CN122066737B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and multi-target tracking technology, and in particular to a multi-target tracking and counting method for smart ocean scenarios. Background Technology
[0002] Multi-target tracking (MOT) aims to continuously locate targets in video sequences and maintain their identity consistency. It is a key supporting technology for applications such as autonomous driving, intelligent security, and human-computer interaction. In recent years, with the advancement of smart ocean construction, this technology has been widely used in smart ports, offshore platforms, and coastal scenic areas. For example, it is used for collaborative tracking of personnel and vehicles during container loading and unloading operations, and for real-time trajectory management and counting of tourists boarding and disembarking.
[0003] The current mainstream tracking paradigm is detection-based tracking (TBD), whose performance is highly dependent on the output of the target detector. However, in complex scenes, issues such as mutual occlusion between targets, motion blur, and changes in illumination can cause drastic fluctuations in detector confidence. Existing methods typically set a static confidence threshold and discard detection boxes below that threshold. This approach has significant drawbacks: many "low-confidence" detection boxes suppressed due to occlusion or poor image quality actually contain real targets. Directly discarding these detection boxes leads to missed detections of real targets, trajectory fragmentation, and identity switching (IDSwitch) problems, severely limiting the robustness and continuity of the tracking system. Furthermore, traditional data association methods often treat detection confidence and trajectory confidence as independent indicators during the matching process, failing to fully explore the intrinsic relationship between them and the impact of environmental complexity on them. This results in inaccurate association decisions in dense scenes (such as peak tourist boarding and disembarking times), thus affecting counting accuracy. Summary of the Invention
[0004] To overcome the aforementioned problems in the existing technology, this invention proposes a multi-target tracking and counting method for smart ocean scenarios.
[0005] The technical solution adopted by this invention to solve its technical problem is: a multi-target tracking and counting method for smart ocean scenarios, comprising the following steps:
[0006] Step 1: Acquire surveillance video of the target scene, process each frame of the surveillance video using the target detector, and output the bounding boxes of all targets and their corresponding detection confidence scores.
[0007] Step 2: For the current frame, use Kalman filtering to predict the bounding box position of the existing trajectory in the current frame, construct the similarity matrix between the predicted box and all detection boxes, and calibrate the detection boxes using the predicted boxes.
[0008] Step 3: Calculate the confidence score for each trajectory and normalize the confidence scores of all trajectories;
[0009] Step 4: Using the calibrated set of detection boxes from Step 2 and the confidence level obtained in Step 3, construct the first association cost matrix, and use the Hungarian algorithm to perform the first association to obtain preliminary matching results, unmatched detection boxes, and unmatched trajectories.
[0010] Step 5: For the unmatched trajectories in Step 4, extract their historical context information, associate the unmatched detection boxes with the historical context information for the second time, and merge the successfully matched pairs from the second association to obtain the complete tracking result;
[0011] Step 6: Based on the tracking results, update the status, position, and confidence of all trajectories, and perform trajectory elimination and new trajectory initialization operations for unmatched trajectories and detection boxes respectively, and output continuous target tracking trajectories.
[0012] In the aforementioned multi-target tracking and counting method for smart ocean scenarios, step 2 specifically comprises:
[0013] Step 2.1, Trajectory Prediction: For all current active trajectories T={t1,t2,...,t...} n}, each trajectory t i Maintain a Kalman filter to describe the motion state of the targets on the image plane; using the prediction equation of the Kalman filter, predict their bounding box positions in the current frame based on the posterior state estimate of the previous frame, obtaining the prediction box set P={p1,p2,...,p...} n}, each prediction box ;
[0014] Step 2.2, Construct the composite similarity matrix: Calculate the similarity matrix for each predicted box p. i With each detection box b j The composite similarity S(p) between i ,b j );
[0015] Step 2.3, Confidence Boosting: Traverse all detection boxes b j Its detection confidence level is c j If c j If <τ_high, then find the prediction box p with the highest composite similarity. i The similarity is S_max; if S_max is higher than a preset similarity threshold τ_sim, then the low confidence detection box is considered to be a true positive target that needs to be found, and the confidence of the detection box is dynamically increased.
[0016] In the multi-target tracking and counting method for smart ocean scenarios described above, the similarity in step 2.2 includes overlap and shape similarity. The overlap is specifically calculated by comparing the cross-union ratio of the predicted box and the detection box; the shape similarity is calculated by comparing the aspect ratio of the predicted box and the detection box.
[0017] To integrate these two indicators, an interval is set for each indicator. When the indicator value falls within this interval, it is mapped to a score between 0 and 1 using linear interpolation. The two scores are then weighted and summed to obtain the composite similarity: S(p i ,b j )=w iou *f iou (IoU(p i ,b j ))+w shape *f shape (shape_sim(p i ,b j )), where w iou and w shape Preset weights.
[0018] The above-mentioned multi-target tracking and counting method for smart ocean scenarios, in step 3, for each trajectory t i Confidence level C_trk(t) i The calculation formula is:
[0019] C_trk(t i )=α*C_prior(t i )*C_quality(t i )
[0020] Where α is the environmental complexity factor, α=max(0,1-β*N), where β is the attenuation coefficient, and N is the number of points on trajectory t. i The number N of detection boxes around the predicted bounding box whose IoU exceeds the threshold τ_iou; C_prior(t i C_prior is the prior confidence score of the trajectory, where C_prior = score^(n-age), and score is the initial confidence base, age is the number of frames the trajectory has survived, and n is the set mature frame number; C_quality(t iC_quality is the confidence score of the trajectory. If c_last ≥ γ, then C_quality = score^(time_since_update), where time_since_update is the number of frames since the last match, and c_last is the confidence score of the most recent successfully matched detection box based on the trajectory. If c_last < γ, then C_quality = max(0, 1 - (time_since_update) / T_max), where T_max is the maximum number of unmatched frames allowed.
[0021] In the aforementioned multi-target tracking and counting method for smart ocean scenarios, step 4 specifically involves: for trajectory t... i and detection box b j ', calculate its geometric distance Dist_geo: Dist_geo = 1 - IoU(p i ,b j A Gaussian function based on confidence similarity is introduced as a weighting factor W_conf:
[0022] W_conf=exp(-(C_det(b j ')-C_trk(t i )) 2 / (2σ 2 ))
[0023] Where C_det is the detection confidence after calibration in step 2, C_trk is the trajectory confidence calculated in step 3, and σ is the preset difference penalty factor; W_conf and Dist_geo are combined to form the comprehensive matching cost: Cost = Dist_geo + λ * (1 - W_conf), where λ is the balance factor; the Hungarian algorithm is run based on this cost matrix to obtain the first matching result M1, the unmatched trajectory set T_unmatched, and the unmatched detection set D_unmatched.
[0024] In the aforementioned multi-target tracking and counting method for smart ocean scenarios, step 5 specifically comprises:
[0025] Step 5.1, Extract historical context: For each trajectory t in T_unmatched i Extract the detection box b_hist that was most recently successfully matched in the current frame from its trajectory buffer as its historical state;
[0026] Step 5.2, Construct the matching cost: Match the detection boxes in D_unmatched with b_hist. The matching cost is composed of geometric similarity, including IoU and shape similarity, and a lower IoU matching threshold than the first association is used; the matching cost matrix is calculated by weighting IoU and shape similarity.
[0027] Step 5.3, perform matching: Based on the above matching cost matrix, run the Hungarian algorithm again to obtain the second matching result M2;
[0028] Step 5.4, Result merging: Merge the first matching result M1 with the second matching result M2 to form the final matching set M_final.
[0029] In the aforementioned multi-target tracking and counting method for smart ocean scenarios, step 6 specifically comprises:
[0030] Step 6.1, Successfully Matched Trajectories: For each matching pair in M_final, use the matched detection box b j As a new observation, update the state of its Kalman filter to obtain the prediction for the next frame. At the same time, store the detection result of the current frame into the trajectory buffer as a new historical state.
[0031] Step 6.2, Unmatched Tracks: For tracks in T_unmatched that are still not matched, increment their miss count (miss_count). If the miss_count exceeds the preset maximum number of lost frames threshold, it is determined that the target has permanently left the screen or disappeared, and it is removed from the list of active tracks.
[0032] Step 6.3, Unmatched detection boxes: For detection boxes that are still unmatched in D_unmatched, initialize them as new trajectories, initialize the Kalman filter for the new trajectory, the state of which is determined by the current detection box, the velocity is initialized to 0, and at the same time, assign a new and unique identifier to the trajectory;
[0033] Step 6.4, Track Confidence Update: Based on the matching results, update the confidence-related parameters of each track for the confidence calculation of the next frame. After updating all tracks, output the tracking results of the current frame.
[0034] The aforementioned multi-target tracking and counting method for smart ocean scenarios, specifically for the scenario of tourists boarding and disembarking, also includes setting up virtual counting lines at dock entrances or ship side areas, combining trajectory direction judgment with identity identification, to achieve bidirectional counting and statistics of tourists boarding and disembarking, and output the counting results.
[0035] The above-mentioned multi-target tracking and counting method for smart ocean scenarios specifically includes bidirectional counting of tourists boarding and disembarking: pre-setting boarding and disembarking counting lines in the video footage at the connection between the dock and the ship's side or at the entrance and exit of the tourist passage, which are used to count the number of tourists boarding and disembarking respectively.
[0036] For each trajectory, its direction of movement is determined based on the change in the position of the center point of the bounding box in consecutive frames. When the trajectory intersects the counting line and the direction of movement is consistent with the preset direction, a counting event is triggered. The system records the identity identifier of each counted trajectory to ensure that the same tourist is counted only once.
[0037] The system counts the number of people boarding and disembarking in real time and overlays the results onto the video feed in a visual format.
[0038] The beneficial effects of the present invention are: (1) Improve the recall rate of low confidence targets: Through the dynamic confidence calibration mechanism, the trajectory prediction information is used as the anchor point to actively mine and improve the confidence of low confidence detection boxes, so that they can participate in data association, effectively reducing the missed detection of real targets due to occlusion or blurring, and improving the continuity of tracking, especially suitable for dense crowds during peak tourist boarding and disembarking periods.
[0039] (2) Enhance the accuracy of trajectory confidence modeling: For the first time, environmental complexity is introduced as a dynamic weight factor into trajectory confidence calculation, and trajectory historical quality and detection quality are integrated to achieve fine modeling of trajectory reliability. This avoids mismatches caused by over-trust in dense scenes in traditional methods, and provides higher identity stability for counting tourists boarding and disembarking.
[0040] (3) Effectively alleviate trajectory breakage caused by occlusion: A two-stage matching strategy based on historical context backtracking is proposed. After the first matching fails, the trajectory is reconnected by using the stable and reliable historical state of the trajectory instead of the drifting prediction box. This significantly improves the trajectory reconnection ability in long-term occlusion or sudden motion scenarios, reduces the number of identity switching, and ensures that the identity of tourists is continuous and the counting is accurate during the boarding and disembarking process.
[0041] (4) Excellent overall tracking performance: Through confidence-driven dynamic association and environmental perception, this method achieves excellent HOTA and AssA scores on public datasets such as MOT17 and MOT20, verifying its strong adaptability and robustness in both sparse and dense scenarios. In the task of counting passengers boarding and disembarking in smart oceans, this method can achieve high-precision, low-error real-time statistics, and has good engineering application value.
[0042] This invention is applicable to handling problems such as target occlusion and detection confidence fluctuation in complex scenarios such as smart ports, maritime traffic monitoring, ship and personnel management, and counting of tourists boarding and disembarking at coastal scenic spots. Attached Figure Description
[0043] Figure 1 This is a schematic diagram of the overall process of the method described in this invention;
[0044] Figure 2 This is the pseudocode of the overall algorithm flow in the embodiments of the present invention;
[0045] Figure 3 This is the pseudocode of the algorithm for finding true positives in this embodiment of the invention;
[0046] Figure 4 This is the pseudocode for data association based on multi-feature fusion in this embodiment of the invention;
[0047] Figure 5 This is a schematic diagram of the tracking effect in an embodiment of the present invention. Detailed Implementation
[0048] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
[0049] This embodiment provides a multi-target tracking method based on trajectory context awareness, the overall process of which is as follows: Figure 1 As shown, the pseudocode for the overall algorithm flow is as follows: Figure 2 As shown in the figure. This method aims to effectively solve the problems of target loss and trajectory fragmentation caused by occlusion and detection confidence fluctuations in complex scenarios through dynamic confidence calibration, trajectory confidence modeling based on environmental awareness, and historical context backtracking matching. The following section provides a detailed explanation of this method using a scenario of counting passengers boarding and disembarking.
[0050] S1: Data Acquisition and Preprocessing
[0051] This embodiment uses high-definition camera equipment mounted on a fixed camera at the dock or a drone to acquire RGB video streams. The video covers the area where tourists board and disembark, ensuring clear capture of tourists entering and exiting cabins or dock entrances / exits. Before inputting the video frames into the object detector, necessary preprocessing (such as size normalization, image enhancement, etc.) is performed. A pre-trained YOLOX object detector is used to process each frame and outputs the bounding boxes B={b1,b2,...,b...} of all detected targets in the current frame. m Each bounding box is defined by its center coordinates, width, and height, i.e., b j =(x j ,y j ,w j ,h j Simultaneously, the detector outputs the detection confidence score C_det={c1,c2,...,c...} for each bounding box. mThe detection confidence level ranges from 0 to 10, representing the detector's certainty that the bounding box contains the target.
[0052] S2: Dynamic Confidence Calibration (DCC)
[0053] Before filtering detection boxes based on the detection confidence threshold τ_high, this embodiment introduces a dynamic confidence calibration mechanism to recover false positive targets from low-confidence detections. The core idea of this mechanism is that even if a target is occluded or has poor imaging quality, its spatial location and shape should be highly consistent with the predicted trajectory of a successfully tracked target in the previous frame, despite the low detection confidence. Therefore, by using the predicted trajectory bounding box as an "anchor point," the confidence of these "potential true positive" detection boxes can be actively searched for and improved. The specific steps are as follows:
[0054] Trajectory prediction: For all current active trajectories T={t1,t2,...,t... n}, each trajectory t i Maintain a Kalman filter to describe the motion state of objects on the image plane (typically including position, velocity, size, etc.). Using the prediction equation of the Kalman filter, predict their bounding box positions in the current frame based on the posterior state estimate of the previous frame, obtaining the predicted box set P={p1,p2,...,p...} n Each prediction box .
[0055] Construct a composite similarity matrix: calculate p for each predicted box i With each detection box b j The composite similarity S(p) between i ,b j This similarity measure aims to assess the spatial matching degree of two bounding boxes and comprises two parts: Intersection over Union (IoU): Calculated by the intersection-union ratio of the predicted and detected bounding boxes. IoU is a classic metric for measuring the degree of overlap between two boxes, with a value range of 0, 10, and 1. Shape similarity: Calculated by comparing the aspect ratios of the predicted and detected bounding boxes. For example, the absolute value of the aspect ratio difference can be calculated and mapped to the 0, 10, and 1 range using an exponential function or linear interpolation. Shape similarity helps distinguish between different targets that overlap in the same region but have significant differences in size and shape. To fuse these two metrics, an interval is set for each metric. When the metric value falls within this interval, it is mapped to a score between 0 and 1 using linear interpolation. Finally, the two scores are weighted and summed to obtain the composite similarity: S(p i ,b j =w_iou*f_iou(IoU(p i ,b j ))+w_shape*f_shape(shape_sim(p i,b j In this context, w_iou and w_shape are preset weights, typically set to a larger w_iou (e.g., 0.7) and a smaller w_shape (e.g., 0.3). Confidence enhancement: Iterate through all bounding boxes b. j Its detection confidence level is c j If c j If <τ_high, then find the prediction box p with the highest composite similarity. i The similarity is S_max. If S_max is higher than a preset similarity threshold τ_sim (e.g., 0.5), the low-confidence detection box is considered to be highly likely to be a true positive target that needs to be retrieved. In this case, the confidence of the detection box is dynamically increased. A simple and effective way is to directly increase its confidence to the same level as the high-confidence threshold: c j =τ_high. Another smoother approach is to use linear interpolation: c j '=c j +(τ_high-c j )*S_max, meaning the higher the similarity, the greater the increase in confidence. After this step, the confidence of all calibrated detection boxes is adjusted to above τ_high, forming the set of detection boxes B' to be matched. The pseudocode for the true positive search algorithm is as follows: Figure 3 As shown.
[0056] S3: Trajectory Confidence and Environmental Complexity Joint Perception Modeling (TCAEM)
[0057] This embodiment describes each trajectory t. i Confidence level C_trk(t) i This approach employs refined modeling, moving away from treating trajectory confidence as a static variable or solely determined by matching history. Instead, it comprehensively considers the trajectory's environment, its lifecycle, and the quality of historical matches. This confidence level is then used for subsequent data association to dynamically adjust the matching strategy. The calculation formula is as follows:
[0058] C_trk(t i )=α*C_prior(t i )*C_quality(t i )
[0059] The meanings and calculation methods of each part are as follows:
[0060] Environmental complexity factor α: defined as the environmental complexity factor α in trajectory t iThe number N of detection boxes whose IoU with the prediction box exceeds the threshold τ_iou around it. This number N directly reflects the degree of crowding around the trajectory in the current frame. The larger N is, the more crowded the environment is, and the higher the probability that the object is occluded. At this time, the confidence in this trajectory should be lowered to avoid incorrect associations in dense crowds. Therefore, α is designed as a decreasing function of N, for example, linear attenuation: α = max(0, 1 - β * N), where β is the attenuation coefficient. In a sparse scenario (N = 0 or very small), α remains 1 or close to 1, and the trajectory confidence is not penalized by the environment.
[0061] The prior confidence C_prior(t i ): Used to measure the reliability of the trajectory in the initial stage. Since the Kalman filter of the newly generated trajectory has not converged, the uncertainty of its predicted position is relatively large, so a lower confidence should be given. An exponential decay model is adopted: C_prior = score^(n - age). Where, score is the initial confidence base (such as 0.6), age is the number of frames the trajectory has survived, and n is the set number of mature frames (for example, 7 frames). When age < n, C_prior increases exponentially with age; when age >= n, the trajectory is considered to have entered the stable tracking state, and C_prior = 1.
[0062] The trajectory quality confidence C_quality(t i ): Dynamically adjusts its decay rate according to the confidence c_last of the detection box that the trajectory successfully matched last time. This design aims to distinguish the true disappearance of the object and temporary occlusion. If c_last is higher than the threshold γ (for example, 0.7), it indicates that the object the trajectory matched last time is a clear object with high confidence. At this time, if the object disappears, a quick response should be made. Therefore, exponential decay is adopted: C_quality = score^(time_since_update), where time_since_update is the number of frames since the last match. If c_last is lower than γ, it indicates that the object the trajectory matched last time may be a blurred object affected by occlusion. At this time, if the object disappears briefly, it should not be prematurely determined to have perished. Therefore, it switches to linear decay: C_quality = max(0, 1 - (time_since_update) / T_max), where T_max is the maximum number of unmatched frames allowed.
[0063] Finally, the confidence of all trajectories is normalized so that it is distributed within the interval of 0, 10, 1 for unified comparison with the detection confidence.
[0064] S4: The first data association based on confidence similarity weighting
[0065] This embodiment performs the first data association to match trajectories with detection boxes. The association process is based on the Hungarian algorithm, whose cost matrix construction no longer relies solely on geometric distance but integrates the similarity between detection confidence and trajectory confidence. Specifically, for trajectory t... i and detection box b j '(from B'), calculate its geometric distance Dist_geo, usually using IoU distance: Dist_geo = 1 - IoU(p i ,b j Then, a Gaussian function based on confidence similarity is introduced as a weighting factor W_conf:
[0066] W_conf=exp(-(C_det(b j ')-C_trk(t i )) 2 / (2σ 2 ))
[0067] Where C_det is the detection confidence after calibration in step S2, C_trk is the trajectory confidence calculated in step S3, and σ is a preset difference penalty factor (e.g., 0.2). The characteristic of this function is: when the detection confidence and trajectory confidence are very close, W_conf approaches 1, and does not penalize the matching cost; when the difference between the two increases, W_conf decreases rapidly, thereby increasing the matching cost and acting as a penalty. Finally, W_conf is combined with Dist_geo to form the comprehensive matching cost: Cost = Dist_geo * (2 - W_conf) or Cost = Dist_geo + λ * (1 - W_conf), where λ is a balancing factor. Based on this cost matrix, the Hungarian algorithm is run to obtain the first matching result M1, the unmatched trajectory set T_unmatched, and the unmatched detection set D_unmatched.
[0068] S5: Secondary Data Association (HCC) Based on Historical Context Backtracking
[0069] For trajectories T_unmatched that failed the first match, this embodiment initiates a second association. The core of this mechanism lies in recognizing that the Kalman filter predicted bounding boxes of trajectories that failed in the first match may have experienced significant positional drift due to not being updated for several consecutive frames. Continuing to rely on these predicted bounding boxes for matching is ineffective. Therefore, the "historical stable state" of the trajectory—i.e., the true detection bounding box from the previous successful match—is used for secondary association. This mechanism effectively handles long-term occlusion, successfully connecting the target to the original trajectory when it reappears. The algorithm flow is as follows: Figure 4 As shown. The specific steps are as follows:
[0070] Extracting historical context: for each trajectory t in T_unmatched i The system extracts the most recently successfully matched detection box, b_hist, from its trajectory buffer as its historical state. This detection box is a true observation, recording the accurate position and size of the target before it was occluded, and is more reliable than the potentially drifted Kalman filter prediction.
[0071] Calculating the matching cost: Match the bounding boxes in D_unmatched with b_hist. Considering that the target may reappear after disappearing for a period of time, its appearance may change due to changes in viewpoint and lighting, and appearance information during occlusion is unreliable, appearance features are not used in this matching. The matching cost is mainly composed of geometric similarity, including IoU and shape similarity. At the same time, a lower IoU matching threshold than the first association is used to allow for greater positional deviations.
[0072] The matching cost matrix is calculated by weighting IoU and shape similarity, for example: Cost_hist=1-(w_iou'*IoU(b_hist,b j )+w_shape'*shape_sim(b_hist,b j )).
[0073] Matching is performed: Based on the cost matrix described above, the Hungarian algorithm is run again to obtain the second matching result M2. These matching results represent the trajectories successfully retrieved using historical information after the occlusion ends.
[0074] Result merging: The first matching result M1 and the second matching result M2 are merged to form the final matching set M_final. It should be noted that the second matching is performed in the "unmatched set" of the first matching to avoid duplicate matching.
[0075] S6: Track Management and Update
[0076] Based on the final matching result M_final, perform state updates and management on all trajectories:
[0077] Successfully matched trajectory: For each matching pair in M_final, use the matched detection box b j As new observations, the state of the Kalman filter (including position, velocity, etc.) is updated to obtain a more accurate prediction for the next frame. At the same time, the detection results of the current frame are stored in the trajectory buffer as new historical states.
[0078] Unmatched tracks: For tracks in T_unmatched that still have no match, increment their miss count (miss_count). If miss_count exceeds the preset maximum number of lost frames threshold, the target is determined to have permanently left the screen or disappeared, and it is removed from the list of active tracks.
[0079] Unmatched detection boxes: For detection boxes that are still unmatched in D_unmatched, initialize them as new trajectories. Initialize the Kalman filter for the new trajectory, the state of which is determined by the current detection box, and initialize the velocity to 0. At the same time, assign a new, unique identifier (ID) to the trajectory.
[0080] Track confidence update: Based on the matching results, update the confidence-related parameters of each track, such as time_since_update and c_last, for use in the confidence calculation of the next frame.
[0081] After updating all trajectories, output the tracking result for the current frame, such as... Figure 5 As shown, this includes the ID and bounding box of each target.
[0082] S7: Passenger boarding and disembarking counting application
[0083] This embodiment, based on target tracking, further implements an automatic counting function for passengers boarding and disembarking. The specific implementation method is as follows:
[0084] Virtual counting lines are preset in the video footage, usually placed at the junction of the dock and the ship's side, or at the entrance and exit of the tourist passage. Boarding and disembarking counting lines are set up to count the number of tourists boarding and disembarking, respectively.
[0085] For each trajectory, its direction of movement is determined by the change in the center point position of its bounding box in consecutive frames. A counting event is triggered when the trajectory intersects the counting line and its direction of movement is consistent with the preset direction. To avoid duplicate counting, the system records the identifier (ID) of each counted trajectory to ensure that the same visitor is counted only once.
[0086] The system tracks and displays real-time statistics on the number of passengers boarding and disembarking, overlaying the results onto the video feed in a visual format. The data is then uploaded to the smart ocean management platform for passenger flow monitoring, scheduling decisions, and safety management.
[0087] Through the above steps, this embodiment not only achieves highly robust multi-target tracking, but also provides accurate counting services in the smart ocean tourist boarding and disembarking scenario, fully verifying the practicality and promotional value of this method.
[0088] The above embodiments are merely exemplary embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art can make various modifications or equivalent substitutions to the present invention within its scope and spirit, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of the present invention.
Claims
1. A multi-target tracking and counting method for smart ocean scenarios, characterized in that, Includes the following steps: Step 1: Acquire surveillance video of the target scene, process each frame of the surveillance video using the target detector, and output the bounding boxes of all targets and their corresponding detection confidence scores. Step 2: For the current frame, use Kalman filtering to predict the bounding box position of the existing trajectory in the current frame, construct the similarity matrix between the predicted box and all detection boxes, and calibrate the detection boxes using the predicted boxes. Step 3: Calculate the confidence score for each trajectory and normalize the confidence scores of all trajectories; Step 4: Using the calibrated set of detection boxes from Step 2 and the confidence level obtained in Step 3, construct the first association cost matrix, and use the Hungarian algorithm to perform the first association to obtain preliminary matching results, unmatched detection boxes, and unmatched trajectories. Step 5: For the unmatched trajectories in Step 4, extract their historical context information, associate the unmatched detection boxes with the historical context information for the second time, and merge the successfully matched pairs from the second association to obtain the complete tracking result; Step 6: Based on the tracking results, update the state, position, and confidence of all trajectories, and perform trajectory elimination and new trajectory initialization operations for unmatched trajectories and detection boxes respectively, outputting continuous target tracking trajectories; Step 2 specifically involves: Step 2.1, Trajectory Prediction: For all current active trajectories T={t1,t2,...,t...} n }, each trajectory t i Maintain a Kalman filter to describe the motion state of the targets on the image plane; using the prediction equation of the Kalman filter, predict their bounding box positions in the current frame based on the posterior state estimate of the previous frame, obtaining the prediction box set P={p1,p2,...,p...} n }, each prediction box ; Step 2.2, Construct the composite similarity matrix: Calculate the similarity matrix for each predicted box p. i With each detection box b j The composite similarity S(p) between i ,b j ); Step 2.3, Confidence Boosting: Traverse all detection boxes b j Its detection confidence level is c j If c j If <τ_high, then find the prediction box p with the highest composite similarity. i The similarity is S_max; if S_max is higher than a preset similarity threshold τ_sim, then the low-confidence detection box is considered a true positive target that needs to be found, and the confidence of the detection box is dynamically increased; In step 2.2, the similarity includes overlap and shape similarity. The overlap is specifically calculated by measuring the intersection-union ratio of the predicted bounding box and the detection bounding box. The shape similarity is calculated by comparing the aspect ratio of the predicted bounding box and the detection bounding box. To integrate these two indicators, an interval is set for each indicator. When the indicator value falls within this interval, it is mapped to a score between 0 and 1 using linear interpolation. The two scores are then weighted and summed to obtain the composite similarity: S(p i ,b j )=w iou *f iou (IoU(p i ,b j ))+w shape *f shape (shape_sim(p i ,b j )), where w iou and w shape Preset weights; Each trajectory t in step 3 i Confidence level C_trk(t) i The calculation formula is: C_trk(t i )=α*C_prior(t i )*C_quality(t i ) Where α is the environmental complexity factor, α=max(0,1-β*N), where β is the attenuation coefficient, and N is the number of points on trajectory t. i The number N of detection boxes around the predicted bounding box whose IoU exceeds the threshold τ_iou; C_prior(t i C_prior is the prior confidence score of the trajectory, where C_prior = score^(n-age), and score is the initial confidence base, age is the number of frames the trajectory has survived, and n is the set mature frame number; C_quality(t i Let C_quality be the trajectory quality confidence score. If c_last ≥ γ, then C_quality = score^(time_since_update), where time_since_update is the number of frames since the last match, c_last is the confidence score of the most recently successfully matched detection box based on the trajectory, and γ is a set threshold. If c_last < γ, then C_quality = max(0, 1 - (time_since_update) / T_max), where T_max is the maximum allowed number of unmatched frames. Step 4 specifically involves: for trajectory t i and detection box b j ', calculate its geometric distance Dist_geo: Dist_geo = 1 - IoU(p i ,b j A Gaussian function based on confidence similarity is introduced as a weighting factor W_conf: W_conf=exp(-(C_det(b j ')-C_trk(t i )) 2 / (2σ 2 )) Where C_det is the detection confidence after calibration in step 2, C_trk is the trajectory confidence calculated in step 3, and σ is the preset difference penalty factor; W_conf and Dist_geo are combined to form the comprehensive matching cost: Cost = Dist_geo + λ*(1-W_conf), where λ is the balance factor; the Hungarian algorithm is run based on this cost matrix to obtain the first matching result M1, the unmatched trajectory set T_unmatched, and the unmatched detection set D_unmatched; Step 5 specifically involves: Step 5.1, Extract historical context: For each trajectory t in T_unmatched i Extract the detection box b_hist that was most recently successfully matched in the current frame from its trajectory buffer as its historical state; Step 5.2, Construct the matching cost: Match the detection boxes in D_unmatched with b_hist. The matching cost is composed of geometric similarity, including IoU and shape similarity, and a lower IoU matching threshold than the first association is used; the matching cost matrix is calculated by weighting IoU and shape similarity. Step 5.3, perform matching: Based on the above matching cost matrix, run the Hungarian algorithm again to obtain the second matching result M2; Step 5.4, Result merging: Merge the first matching result M1 with the second matching result M2 to form the final matching set M_final.
2. The multi-target tracking and counting method for smart ocean scenarios according to claim 1, characterized in that, Step 6 specifically involves: Step 6.1, Successfully Matched Trajectories: For each matching pair in M_final, use the matched detection box b j As a new observation, update the state of its Kalman filter to obtain the prediction for the next frame. At the same time, store the detection result of the current frame into the trajectory buffer as a new historical state. Step 6.2, Unmatched Tracks: For tracks in T_unmatched that are still not matched, increment their miss count (miss_count). If the miss_count exceeds the preset maximum number of lost frames threshold, it is determined that the target has permanently left the screen or disappeared, and it is removed from the list of active tracks. Step 6.3, Unmatched detection boxes: For detection boxes that are still unmatched in D_unmatched, initialize them as new trajectories, initialize the Kalman filter for the new trajectory, the state of which is determined by the current detection box, the velocity is initialized to 0, and at the same time, assign a new and unique identifier to the trajectory; Step 6.4, Track Confidence Update: Based on the matching results, update the confidence-related parameters of each track for the confidence calculation of the next frame. After updating all tracks, output the tracking results of the current frame.
3. The multi-target tracking and counting method for smart ocean scenarios according to claim 1, characterized in that, For scenarios involving tourists boarding and disembarking, the system also includes setting up virtual counting lines at dock entrances or ship side areas. By combining trajectory direction judgment with identity identification, it enables two-way counting and statistics of tourists boarding and disembarking, and outputs the counting results.
4. The multi-target tracking and counting method for smart ocean scenarios according to claim 3, characterized in that, The two-way counting of passengers boarding and disembarking is specifically achieved by setting up boarding and disembarking counting lines in the video footage at the connection between the dock and the ship's side or at the entrance and exit of the passenger passage, which are used to count the number of passengers boarding and disembarking respectively. For each trajectory, its direction of movement is determined based on the change in the position of the center point of the bounding box in consecutive frames. When the trajectory intersects the counting line and the direction of movement is consistent with the preset direction, a counting event is triggered. The system records the identity identifier of each counted trajectory to ensure that the same tourist is counted only once. The system counts the number of people boarding and disembarking in real time and overlays the results onto the video feed in a visual format.