Nonlinear target tracking method and device suitable for dynamic marine environment

Abstract

The invention provides a nonlinear target tracking method and device suitable for a dynamic marine environment. The method provided by the invention comprises the following steps: acquiring all target detection frames and confidence coefficients in a current frame; for all the continued tracks in the previous frame, predicting the state of each continued track to obtain a prediction bounding box of each continued track in the current frame; based on the historical bounding box feature points corresponding to each continued track in the previous frame, horizontal direction displacement information is extracted for motion momentum estimation, and the horizontal motion momentum of each continued track is obtained; dividing a target detection frame in the current frame according to the confidence coefficient, and performing association matching on the continued track and the detection frame in stages in combination with the prediction bounding box, the horizontal motion momentum and the target appearance characteristics; updating the successfully matched track by using a corresponding detection frame; for the unmatched track, determining whether to terminate the track based on the number of continuous unmatched frames, otherwise, keeping the track as a continued track; a new trajectory is initialized based on the first unmatched detection frame.

Description

Technical Field

[0001] This application relates to the field of target tracking technology, and in particular to a nonlinear target tracking method and apparatus suitable for dynamic marine environments. Background Technology

[0002] The safe navigation of autonomous surface vessels (MASS) relies heavily on accurate situational awareness of surrounding vessels and floating targets. Visual multi-target tracking (MOT) technology is crucial for achieving this awareness, requiring target detection, identity maintenance, and trajectory association through continuous video sequences to provide data support for collision avoidance decisions and route planning. However, MASS is susceptible to severe swaying caused by wave impacts in dynamic maritime environments, resulting in highly nonlinear apparent target motion. Therefore, a suitable nonlinear target tracking solution is urgently needed.

[0003] Currently, mainstream multi-target tracking is mainly based on the "detect-after-tracking" (TBD) paradigm, such as SORT, DeepSORT, and ByteTrack. It relies on two major assumptions: first, the target detector outputs reliable observations; second, the target's motion in adjacent frames is approximately linear, and the state can be predicted by a standard Kalman filter (KF), combined with the intersection-over-union ratio (IoU) to complete trajectory-detection matching.

[0004] However, this type of method has significant shortcomings in dynamic maritime environments: First, the linear motion assumption of KF fails, and the apparent trajectory of the target deviates greatly from the prediction, leading to trajectory interruption and identity switching; second, traditional IoU is sensitive to vertical disturbances, and changes in the target's vertical displacement or scale can easily lead to mismatch; third, motion estimation is affected by vertical noise, and when calculating horizontal and vertical displacements in combination, high-frequency noise in the vertical direction cannot reflect the target's true navigation intention, reducing the reliability of association.

[0005] Therefore, there is an urgent need for a robust dynamic multi-target tracking method in waters to solve the problems of state prediction failure, inaccurate correlation measurement, and interference with motion estimation, so as to achieve stable tracking of targets in complex marine environments. Summary of the Invention

[0006] In view of this, this application provides a nonlinear target tracking method and apparatus suitable for dynamic marine environments, which provides a robust multi-target tracking method in dynamic waters, solves the problems of state prediction failure, inaccurate correlation measurement, and interference with motion estimation, and achieves stable tracking of targets in complex marine environments.

[0007] Specifically, this application is implemented through the following technical solution:

[0008] The first aspect of this application provides a nonlinear target tracking method suitable for dynamic marine environments, the method comprising:

[0009] Obtain all target detection boxes output by the target detector in the current frame and the confidence score of each target detection box;

[0010] For all existing trajectories in the previous frame, the state of each existing trajectory is predicted based on the observation center Kalman filter strategy to obtain the predicted bounding box of each existing trajectory in the current frame; the existing trajectory is the target trajectory in the previous frame that has not reached the preset termination condition and needs to be tracked.

[0011] Based on the historical bounding box feature points corresponding to each existing trajectory in the previous frame, horizontal displacement information is extracted to estimate the motion momentum, and the horizontal motion momentum of each existing trajectory is obtained.

[0012] The target detection boxes in the current frame are divided according to the confidence level. Combining the predicted bounding box, the horizontal motion momentum and the target appearance features, the existing trajectory and the detection box are matched in stages, and the successfully matched trajectory-detection pair, the unmatched trajectory and the unmatched detection box are output.

[0013] For a successfully matched trajectory, the observation center Kalman filter state and horizontal momentum of the trajectory are updated using the corresponding detection box, and the trajectory is retained as a continuing trajectory.

[0014] For unmatched trajectories, determine whether the number of consecutive unmatched frames has reached a preset termination threshold. If it has, terminate the trajectory; otherwise, retain it as a continuing trajectory.

[0015] The first unmatched detection box with a confidence level higher than a preset value is selected from the unmatched detection boxes, and a new trajectory is initialized based on the first unmatched detection box; the new trajectory is included in the existing trajectory of the current frame for target tracking in the next frame.

[0016] A second aspect of this application provides a nonlinear target tracking device suitable for dynamic marine environments, the device comprising an acquisition module, a prediction module, a matching module, and a processing module;

[0017] The acquisition module is used to acquire all target detection boxes output by the target detector in the current frame and the confidence level corresponding to each target detection box.

[0018] The prediction module is used to predict the state of each existing trajectory based on the observation center Kalman filter strategy for all existing trajectories in the previous frame, and obtain the predicted bounding box of each existing trajectory in the current frame; the existing trajectory is the target trajectory in the previous frame that has not reached the preset termination condition and needs to be tracked.

[0019] The prediction module is also used to extract horizontal displacement information based on the historical bounding box feature points corresponding to each existing trajectory in the previous frame to estimate the motion momentum and obtain the horizontal motion momentum of each existing trajectory.

[0020] The matching module is used to divide the target detection boxes in the current frame according to the confidence level, and combine the predicted bounding box, the horizontal motion momentum and the target appearance features to perform association matching between the existing trajectory and the detection box in stages, and output the successfully matched trajectory-detection pair, the unmatched trajectory and the unmatched detection box.

[0021] The processing module is used to update the observation center Kalman filter state and horizontal motion momentum of the successfully matched trajectory using the corresponding detection box, and retain it as a continuing trajectory.

[0022] The processing module is also used to determine whether the number of consecutive unmatched frames for an unmatched trajectory has reached a preset termination threshold. If it has, the trajectory is terminated; otherwise, it is retained as a continuing trajectory.

[0023] The processing module is further configured to filter out a first unmatched detection box with a confidence level higher than a preset value from the unmatched detection boxes, and initialize a new trajectory based on the first unmatched detection box; the new trajectory is incorporated into the existing trajectory of the current frame for target tracking in the next frame.

[0024] The nonlinear target tracking method and apparatus provided in this application, applicable to dynamic marine environments, firstly calculates horizontal motion momentum by extracting horizontal displacement information based solely on historical bounding box feature points, discarding vertical displacement data susceptible to wave turbulence and platform heave disturbances. This filters out invalid vertical noise interference, focusing on the target's true horizontal navigation motion characteristics, avoiding momentum estimation bias caused by vertical disturbances, and allowing motion momentum to more accurately represent the target's actual navigation trend. This provides a reliable motion basis for subsequent association matching and improves the stability of momentum estimation in complex dynamic marine environments. Secondly, after dividing the detection boxes according to confidence levels, association matching is carried out in stages. This staged matching strategy can adapt to the uneven quality of detection boxes in marine environments, effectively reducing matching errors caused by target occlusion and lighting changes, reducing trajectory interruption and identity switching problems, improving the robustness of nonlinear target tracking, and achieving accurate and stable multi-target tracking results when combined with the dynamic nonlinear disturbance characteristics of the marine environment. Attached Figure Description

[0025] Figure 1 A flowchart of a nonlinear target tracking method suitable for dynamic marine environments provided in Embodiment 1 of this application;

[0026] Figure 2 A schematic diagram of the four types of feature points provided in this application;

[0027] Figure 3 This is a schematic diagram of the structure of a nonlinear target tracking device suitable for dynamic marine environments, provided in Embodiment 2 of this application. Detailed Implementation

[0028] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0029] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used herein are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0030] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0031] The following specific embodiments are given to illustrate the technical solution of this application in detail.

[0032] Figure 1 This is a flowchart illustrating a nonlinear target tracking method suitable for dynamic marine environments, provided in Embodiment 1 of this application. Please refer to... Figure 1 The method provided in this embodiment may include:

[0033] S101. Obtain all target detection boxes output by the target detector in the current frame and the confidence level of each target detection box.

[0034] Specifically, the target detector is a mature and widely used target recognition tool, which can be a ship-specific target detection network or a general target detection network (such as the YOLO series, Faster R-CNN, etc.). Its core function is to identify objects of interest such as ships and floating targets from video frame images. This is not the focus of improvement in this application. This application only uses its output results as tracking input. The target detection box is a rectangular area delineated by the target detector in the image coordinate system for the identified target. It is used to locate the target position and is usually described by the coordinates of the upper left and lower right corners or the center coordinates and width and height dimensions. The confidence score is a value between 0 and 1 output by the target detector, which is used to characterize the degree of confidence that the target actually exists within the detection box. The higher the value, the more reliable the detection result.

[0035] S102. For all existing trajectories in the previous frame, predict the state of each existing trajectory based on the observation center Kalman filter strategy to obtain the predicted bounding box of each existing trajectory in the current frame.

[0036] The continuing trajectory is the target trajectory in the previous frame that did not meet the preset termination condition and needs to be continued to be tracked.

[0037] Specifically, the continuing trajectory refers to the target trajectory that did not meet the preset termination condition during the tracking process of the previous frame and needs to continue tracking. It includes trajectories that successfully matched the detection box and completed the state update in the previous frame, as well as trajectories that did not match in the previous frame but whose consecutive unmatched frame count did not reach the threshold and were not terminated. It is the core carrier for maintaining the continuity of target identity across frames.

[0038] The predicted bounding box refers to the estimated position and size range of the target in the current frame, which is inferred from the historical state (position, velocity, etc.) of the existing trajectory in the previous frame through the observation center Kalman filtering strategy and presented in the form of a rectangular box.

[0039] In specific implementation, a multi-dimensional state vector is constructed for each existing trajectory. For the target existing trajectory, a state transition matrix is ​​constructed based on the assumption of approximately uniform motion. Combining the preset process noise, the multi-dimensional state vector of the target existing trajectory in the previous frame is used to calculate the predicted state vector of the target existing trajectory in the current frame through linear deduction. The center coordinates and size information of the corresponding target bounding box are extracted from the predicted state vector to generate the predicted bounding box of the target existing trajectory in the current frame. When a detection box matching the target existing trajectory is obtained in the current frame, the measurement noise covariance corresponding to the detection box is set to a first threshold, or the measurement noise covariance is adaptively adjusted according to the confidence of the detection box. The higher the confidence of the detection box, the smaller the measurement noise covariance. Based on the adjusted measurement noise covariance and combined with the Kalman filter update rule, the predicted state vector of the target existing trajectory is corrected using the matching detection box to obtain the final state vector of the target existing trajectory in the current frame.

[0040] Specifically, firstly, for each existing trajectory, a multi-dimensional state vector is constructed, containing the center x-coordinate, center y-coordinate, width, and height of the target bounding box, as well as the first-order velocity components corresponding to each of these coordinates. This comprehensively represents the position, size, and motion state of the existing trajectory. Then, based on the assumption that the target moves at approximately uniform speed between adjacent frames, a state transition matrix is ​​constructed for state transfer. This matrix must linearly map the multi-dimensional state vector from the previous frame to the predicted state in the current frame. Simultaneously, pre-defined process noise (to characterize uncertainties such as environmental interference and motion model bias) is introduced. The determined multi-dimensional state vector of the existing trajectory from the previous frame is substituted into the state transition matrix, and combined with the process noise, a linear deduction calculation is performed to obtain the predicted state vector of the trajectory in the current frame. Finally, the corresponding center x-coordinate is extracted from this predicted state vector. The system uses the target, center ordinate, width, and height information to generate the predicted bounding box of the existing trajectory in the current frame. When a detection box matching the existing trajectory is found in the current frame, the measurement noise covariance corresponding to the detection box is directly set to a preset first threshold (the first threshold is a small value). Alternatively, it can be adaptively adjusted according to the confidence level of the detection box itself. The higher the confidence level of the detection box, the smaller the measurement noise covariance is set, so that the high-confidence detection results have a higher weight in the state update. Finally, based on the adjusted measurement noise covariance, the system strictly follows the state update rules of Kalman filtering, substitutes the actual observation information (position, size) of the target carried by the matched detection box into the calculation, corrects the previously obtained predicted state vector, eliminates prediction bias, and finally obtains the accurate final state vector of the existing trajectory in the current frame, completing the trajectory state update iteration.

[0041] For example, in one embodiment, an 8-dimensional state vector is constructed for each existing trajectory:

[0042] ;

[0043] in, It is a multidimensional state vector; The coordinates of the bounding box center; Define the width and height of the bounding box; This corresponds to the first-order velocity component.

[0044] In the After the frame, a linear prediction of the state is performed:

[0045] ;

[0046] in, This is the predicted state vector of the trajectory existing in the current frame; This is the multidimensional state vector of the trajectory remaining in the previous frame; This is the state transition matrix based on the "approximately uniform velocity" assumption. This is process noise.

[0047] The predicted bounding box of the target's trajectory in the current frame can be represented as:

[0048] ;

[0049] in, The predicted bounding box of the target's existing trajectory in the current frame; The x-coordinate is the center. The central ordinate; Width; For height.

[0050] The final state vector of the target's trajectory in the current frame can be represented as:

[0051] ;

[0052] in, The final state vector of the target's existing trajectory in the current frame; This is the predicted state vector of the trajectory existing in the current frame; Kalman gain; The predicted bounding box of the target's existing trajectory in the current frame; This is the observation matrix.

[0053] The method provided in this embodiment generates a predicted bounding box by "constructing a multidimensional state vector containing position, size, and corresponding velocity components + a state transition matrix based on the assumption of approximate uniform velocity + linear derivation combined with process noise". This method can comprehensively characterize the target's motion characteristics through the multidimensional state vector, and adapt to the target's basic motion law and environmental interference by using the state transition matrix and process noise, thus initially locking the target's estimated range in the current frame. The subsequent "setting the detection box measurement noise covariance to a small value (or adaptively adjusting according to confidence level) + combining the state update formula of Kalman gain and observation matrix" essentially strengthens the driving role of detection observation on the state. When the detection quality is good, the Kalman gain will tilt towards the observation side, so that the final trajectory state is highly aligned with the observation box height. This solves the problem of linear prediction deviation caused by nonlinear disturbances of the platform in dynamic marine environments, and avoids the prediction error from continuously amplifying under strong disturbances. The predicted bounding box retains the rationality based on historical states and corrects deviations by conforming to the actual detection results, significantly improving the stability of the trajectory in complex aquatic environments, while ensuring the continuous maintenance of the target identity (ID) and reducing trajectory interruptions and identity switching.

[0054] S103. Based on the historical bounding box feature points corresponding to each existing trajectory in the previous frame, extract the horizontal displacement information to estimate the motion momentum and obtain the horizontal motion momentum of each existing trajectory.

[0055] Specifically, historical bounding box feature points are predefined feature locations on the target bounding box corresponding to the existing trajectory in the previous frame, including the left midpoint, right midpoint, top midpoint, and bottom midpoint, used to accurately extract the local position information of the bounding box. Horizontal displacement information is the change in position of the corresponding historical bounding box feature points in the horizontal direction (e.g., x-axis) of the image coordinate system in adjacent historical frames (including the previous frame) for the same existing trajectory; that is, the difference in the horizontal coordinates of the feature points from the adjacent previous frame to the previous frame. Horizontal motion momentum is the trajectory motion state calculated based on the horizontal displacement information of the historical bounding box feature points, including the horizontal motion velocity (the ratio of the horizontal displacement to the time interval between adjacent frames) and the horizontal motion direction (determined by the positive or negative direction of the horizontal displacement), and only the horizontal component is retained for subsequent trajectory and detection box association matching.

[0056] Optional, Figure 2 This is a schematic diagram illustrating the four types of feature points provided in this application. Please refer to... Figure 2 Wherein, the left midpoint is LM, the right midpoint is RM, the top midpoint is TM, and the bottom midpoint is BM. The left midpoint is the intersection of the left edge of the historical bounding box and the center height of the bounding box, where the center height is the vertical position corresponding to 1 / 2 of the bounding box height; the right midpoint is the intersection of the right edge of the historical bounding box and the center height of the bounding box; the top midpoint is the intersection of the top edge of the historical bounding box and the center width of the bounding box, where the center width is the horizontal position corresponding to 1 / 2 of the bounding box width; and the bottom midpoint is the intersection of the bottom edge of the historical bounding box and the center width of the bounding box.

[0057] In specific implementation, four types of feature points are defined for the historical bounding boxes corresponding to each existing trajectory in the previous frame; the four types of feature points include the left midpoint, right midpoint, top midpoint, and bottom midpoint; the bounding boxes of each existing trajectory in adjacent historical frames are obtained, and the coordinates of the four types of feature points of the bounding boxes of the same trajectory in adjacent frames are determined respectively; outlier removal is performed on the coordinates of the four types of feature points to obtain valid feature points; the valid feature points at least include the left midpoint and the right midpoint; the horizontal displacement values ​​of the valid feature points between adjacent frames of the same existing trajectory are calculated, and the arithmetic mean of the horizontal displacements of all left midpoints and right midpoints is taken to obtain the average horizontal displacement of the existing trajectory between adjacent frames; the horizontal motion velocity of the existing trajectory is calculated based on the time interval between the average horizontal displacement and the adjacent frames, and the horizontal motion direction is determined based on the positive and negative directions of the average horizontal displacement; the horizontal motion velocity and the horizontal motion direction together constitute the horizontal motion momentum of the existing trajectory.

[0058] Specifically, firstly, for each historical bounding box corresponding to a persistent trajectory in the previous frame, four types of feature points are defined: the left midpoint is the intersection of the left edge of the historical bounding box and the center height of the bounding box; the right midpoint is the intersection of the right edge of the historical bounding box and the center height; the top midpoint is the intersection of the top edge of the historical bounding box and the center width of the bounding box; and the bottom midpoint is the intersection of the bottom edge of the historical bounding box and the center width. Then, the bounding boxes of the persistent trajectory in adjacent historical frames (including the previous frame) are obtained. According to the definition rules of the four types of feature points, the coordinates of the left midpoint, right midpoint, top midpoint, and bottom midpoint of the bounding box of the same trajectory in adjacent frames are calculated and determined. Next, outlier values ​​are removed from the coordinates of these four types of feature points, retaining those that at least include the left midpoint. The effective feature points are the left and right midpoints. Then, the horizontal displacement values ​​of the effective feature points between adjacent frames of the same persistent trajectory (i.e., the difference in the horizontal coordinates of the same effective feature point in adjacent frames) are calculated. The horizontal displacements of all effective left and right midpoints are selected, and the arithmetic mean of these displacement values ​​is taken to obtain the average horizontal displacement of the persistent trajectory between adjacent frames. Finally, the horizontal motion velocity of the trajectory is obtained by removing the horizontal displacement with respect to the time difference between the two frames. At the same time, the horizontal motion direction is determined according to the positive and negative directions of the average horizontal displacement (e.g., positive for right and negative for left). The horizontal motion velocity and the horizontal motion direction together constitute the horizontal motion momentum of the persistent trajectory.

[0059] Optionally, outlier removal is performed on the coordinates of the four types of feature points to obtain valid feature points. This includes: obtaining the bounding box of the existing trajectory in adjacent historical frames, calculating the horizontal displacement of each of the four types of feature points between adjacent frames of the same trajectory; performing consistency judgment on the horizontal displacement of the four types of feature points of the same trajectory; if the difference between the horizontal displacement of the target feature point and the horizontal displacement of the other three types of feature points exceeds a preset displacement deviation threshold, or if the horizontal displacement direction of the target feature point is opposite to the estimated horizontal momentum direction of the existing trajectory in the previous frame and the included angle exceeds a preset angle threshold, then the target feature point is determined to be an outlier feature point and removed to obtain valid feature points.

[0060] In practice, the target bounding box of the trajectory to be processed is first obtained in adjacent historical frames (including the previous frame). Based on the defined position rules of the left, right, top, and bottom midpoints, the specific coordinates of each type of feature point of the trajectory in the two adjacent frames are determined. Then, by calculating the difference in the horizontal coordinates of the same feature point between adjacent frames, the horizontal displacement values ​​corresponding to the four types of feature points of the trajectory are obtained. Subsequently, a consistency judgment operation is performed on the horizontal displacements of the four types of feature points of the trajectory. Each type of feature point is taken as the target feature point in turn. The horizontal displacement of the target feature point and the horizontal displacement of the other three types of feature points are calculated first. The displacement difference is compared one by one to see if it exceeds a preset displacement deviation threshold. At the same time, the horizontal momentum direction estimated in the previous frame of the existing trajectory is retrieved to determine whether the horizontal displacement direction of the current target feature point is opposite to the historical momentum direction and whether the angle between them exceeds a preset angle threshold. If the target feature point meets any of the conditions of displacement difference exceeding the threshold or displacement direction being opposite and angle exceeding the threshold, the target feature point is determined to be an abnormal feature point and a removal operation is performed. All feature points that are not removed after the above judgment are retained as valid feature points for the existing trajectory to be used for subsequent horizontal momentum estimation.

[0061] The method provided in this embodiment firstly utilizes four types of feature points to achieve multi-dimensional extraction of target bounding box position information, providing a comprehensive data foundation for horizontal displacement calculation. Secondly, it eliminates abnormal feature points by using a dual judgment of displacement difference exceeding a preset threshold and displacement direction opposite to historical momentum direction with an angle exceeding a threshold. This effectively filters out erroneous displacement data caused by platform turbulence and image noise, avoiding interference from outliers in momentum calculation. Finally, it retains only the horizontal displacement of the valid left and right midpoints and takes the arithmetic mean, discarding other dimensions of information that are easily affected by vertical noise, focusing on the horizontal motion features that reflect the target's true navigation intention. This setting ensures the accuracy and stability of momentum calculation while avoiding the negative impact of vertical disturbances in the dynamic marine environment. The resulting horizontal motion momentum can accurately represent the target's true horizontal motion state, providing a reliable motion basis for subsequent trajectory and detection box association matching, and improving the robustness of multi-target tracking in complex water environments.

[0062] S104. Divide the target detection boxes in the current frame according to the confidence level, and combine the predicted bounding box, the horizontal motion momentum and the target appearance features to perform correlation matching between the existing trajectory and the detection box in stages, and output the successfully matched trajectory-detection pair, the unmatched trajectory and the unmatched detection box.

[0063] Specifically, target appearance features are feature information extracted from the target region image corresponding to the target detection box in the current frame or the historical trajectory, used to characterize the visual uniqueness of the target. They usually include quantitative data of the target's contour, texture, color distribution, etc. Feature vectors can be generated through a preset feature extraction network (such as a pedestrian re-identification network or a convolutional neural network) to distinguish different targets and reduce matching errors caused by overlapping target positions or similar appearances.

[0064] In specific implementation, the existing trajectory and detection box are associated and matched in stages, combining the predicted bounding box, the horizontal motion momentum, and the target appearance features. This includes: calculating a comprehensive cost for each pair of existing trajectory-detection box combinations; the comprehensive cost is obtained by weighting the height modulation intersection-union ratio, horizontal motion momentum, and target appearance features; selecting a set of first detection boxes with a confidence level higher than a first preset confidence threshold from the detection boxes in the current frame, constructing a first association cost matrix with the existing trajectory as the row and the first detection box as the column, where the elements of the first association cost matrix are the comprehensive costs of the corresponding existing trajectory-first detection box; solving the minimum cost matching of the first association cost matrix based on the Hungarian algorithm, retaining the matching pairs with a comprehensive cost lower than the first preset cost threshold as the trajectory-detection pairs successfully matched in the first stage, and marking unmatched existing trajectories and unmatched first detection boxes.

[0065] Among them, the height modulation cross-union ratio cost is determined based on the height modulation cross-union ratio of the predicted bounding box and the detection box, the horizontal motion momentum consistency cost is determined based on the difference between the trajectory horizontal motion momentum and the target horizontal motion estimate corresponding to the detection box, and the appearance feature cost is determined based on the distance between the trajectory historical appearance features output by the preset pedestrian re-identification network and the appearance features of the detection box.

[0066] Specifically, firstly, the target detection boxes in the current frame are divided into high-confidence and low-confidence sets based on their confidence levels. Then, all combinations of existing trajectories and target detection boxes in the current frame are traversed. For each pair of existing trajectory-(target) detection box combinations, three types of cost components are calculated sequentially: The first type is the height modulation cross-union ratio (HMOCR) cost, where the sum of the HMOCR cost and the HMOCR is 1. The second type is the horizontal motion momentum consistency cost. First, the horizontal motion momentum of the target corresponding to the target detection box is estimated, then the difference between this estimate and the existing horizontal motion momentum of the existing trajectory is calculated. The horizontal motion momentum consistency cost is mapped based on the magnitude of the difference; the larger the difference, the higher the cost. The third type is the appearance feature cost. A preset pedestrian re-identification network is called to extract the historical appearance feature vector of the existing trajectory and the appearance feature vector of the current detection box. The cosine distance or Euclidean distance between the two feature vectors is calculated, and this distance is directly used as the appearance feature cost; the larger the distance, the higher the cost. Then, according to the preset... The weighted coefficients are used to sum the weighted costs of height modulation intersection-over-union, horizontal motion momentum consistency, and appearance features to obtain the comprehensive cost corresponding to each pair of surviving trajectory-detection box combinations. Next, the confidence data of all target detection boxes in the current frame are retrieved, and detection boxes with confidence scores higher than a first preset threshold are selected to form a first detection box set. Then, a first association cost matrix is ​​constructed with the number of surviving trajectories as the number of rows and the number of detection boxes in the first detection box set as the number of columns. The comprehensive cost corresponding to each pair of surviving trajectory-first detection box combinations is filled into the corresponding positions in the matrix, and positions of combinations not involved in the matrix can be filled with infinity. Finally, the Hungarian algorithm is used to solve the first association cost matrix for minimum cost matching. The obtained matching pairs are traversed, and matching pairs with comprehensive costs lower than the first preset cost threshold are selected and marked as successfully matched trajectory-detection pairs in the first stage. Simultaneously, surviving trajectories that did not match any first detection boxes in this stage, as well as first detection boxes that did not match any surviving trajectories, are recorded one by one. The specific implementation process of the Hungarian algorithm can be found in the descriptions in related technologies, and will not be elaborated here.

[0067] For example, in one embodiment, the overall cost is expressed as:

[0068] ;

[0069] in, As a comprehensive cost; , , These are the weighting coefficients; For high modulation crossover ratio; The cost of consistent momentum for horizontal motion; Cost of appearance features.

[0070] Optionally, the calculation process of the height modulation intersection-union ratio includes: determining the predicted bounding box of the existing trajectory and the detection box of the current frame, and extracting the position information and planar coordinate information of the two candidate boxes in the vertical direction respectively; determining the overlapping area of ​​the two candidate boxes in the planar coordinate system and calculating the area of ​​the overlapping area; calculating the planar area of ​​each of the two candidate boxes, summing the planar areas and subtracting the area of ​​the overlapping area to obtain the planar union area of ​​the two candidate boxes; taking the ratio of the area of ​​the overlapping area to the planar union area as the planar intersection-union ratio of the two candidate boxes; determining the overlapping interval in the vertical direction according to the starting and ending coordinates of the two candidate boxes in the vertical direction, calculating the length of the overlapping interval as the vertical overlap height; calculating the length of the union interval in the vertical direction of the two candidate boxes as the vertical union height; taking the ratio of the vertical overlap height to the vertical union height as the height overlap ratio of the two candidate boxes; and multiplying the planar intersection-union ratio and the height overlap ratio to obtain the height modulation intersection-union ratio.

[0071] In practice, the first step is to determine two candidate boxes to be calculated: the predicted bounding box corresponding to the existing trajectory and the target detection box in the current frame. The vertical position information (including the starting and ending coordinates) and planar coordinate information (including the starting and ending coordinates, width, and height) of these two candidate boxes are extracted. Next, the overlapping region of the two candidate boxes in the planar coordinate system is determined. The area of ​​the overlapping region is calculated based on its horizontal and vertical boundary coordinates. Then, the planar areas of the predicted bounding box and the detection box are calculated separately. Finally, the area of ​​the overlapping region is subtracted from the sum of the two planar areas. The planar union area of ​​the two candidate boxes is obtained, and the ratio of the overlapping area to the planar union area is used as the planar intersection-union ratio of the two candidate boxes. Then, based on the starting and ending coordinates of the two candidate boxes in the vertical direction, the vertical overlap interval is determined, and the length of the overlap interval is calculated to obtain the vertical overlap height. At the same time, the length of the vertical union interval of the two candidate boxes is calculated to obtain the vertical union height. The ratio of the vertical overlap height to the vertical union height is used as the height overlap ratio of the two candidate boxes. Finally, the calculated planar intersection-union ratio and the height overlap ratio are multiplied to obtain the final height-modulated intersection-union ratio.

[0072] For example, in one embodiment, the height overlap ratio is expressed as:

[0073] ;

[0074] in, The overlap ratio is the height. The vertical overlap height between the two candidate boxes; The vertical height union of the two candidate boxes.

[0075] The intersection-union ratio of the planes is Then the high-modulation crossover ratio is defined as:

[0076] ;

[0077] in, For high modulation crossover ratio; For planar interaction ratio; This represents the high overlap ratio.

[0078] Optionally, after simultaneously marking unmatched trajectories and unmatched first detection boxes, the method further includes: selecting a set of second detection boxes from the current frame detection boxes whose confidence level is not higher than a first preset confidence threshold and is higher than a second preset confidence threshold; constructing a second association cost matrix with the unmatched trajectories in the first stage as rows and the second detection boxes as columns; solving for the minimum cost matching of the second association cost matrix, retaining matching pairs with a comprehensive cost lower than a second preset cost threshold as successfully matched trajectory-detection pairs in the second stage, while marking unmatched trajectories and unmatched second detection boxes; constructing a third association cost matrix based on the height modulation intersection-union ratio of the predicted bounding box and the detection box for the unmatched trajectories and unmatched second detection boxes in the second stage; matching the third association cost matrix, retaining matching pairs with a comprehensive cost lower than a third preset cost threshold as successfully matched trajectory-detection pairs in the third stage; the third preset cost threshold is greater than the first and second preset cost thresholds; integrating the three-stage matching results, and outputting all successfully matched trajectory-detection pairs, the final unmatched trajectories, and the final unmatched detection boxes.

[0079] Specifically, after the first stage of matching is completed and unmatched trajectories and unmatched first detection boxes are marked, target detection boxes with confidence levels not higher than a first preset confidence threshold and higher than a second preset confidence threshold are first selected from the target detection boxes in the current frame to form a second detection box set. Then, a second association cost matrix is ​​constructed with the unmatched trajectories from the first stage as rows and the detection boxes in the second detection box set as columns. Each element in the matrix is ​​the comprehensive cost of the corresponding "unmatched trajectories-second detection boxes" combination. The minimum cost matching of the second association cost matrix is ​​then solved based on the Hungarian algorithm. Matching pairs with comprehensive costs lower than the second preset cost threshold are retained as successfully matched trajectory-detection pairs in the second stage, and this stage is marked. The remaining unmatched trajectories and unmatched second detection boxes are then identified. Subsequently, for these unmatched trajectories and second detection boxes from the second stage, the multi-dimensional comprehensive cost is no longer calculated. Instead, a third association cost matrix is ​​constructed based solely on the height modulation intersection-union ratio (HUC) of each trajectory prediction bounding box and each detection box. A matching algorithm is then used to match this third association cost matrix, retaining pairs with a comprehensive cost lower than a third preset cost threshold as successfully matched trajectory-detection pairs in the third stage. This third preset cost threshold is greater than the first and second preset cost thresholds. Finally, the matching results from the first, second, and third stages are integrated, summarizing all successfully matched trajectory-detection pairs. The final unmatched trajectories and unmatched detection boxes are then identified and output. For the specific implementation process of the matching algorithm, please refer to the descriptions in related technologies; they will not be elaborated here.

[0080] The method provided in this embodiment first divides the target detection boxes into a high-confidence first set of detection boxes and a low-confidence second set of detection boxes based on their confidence levels. It prioritizes using a multi-dimensional comprehensive cost, including height modulation cross-union ratio, horizontal motion momentum, and appearance features, to perform strict threshold matching between high-confidence detection boxes and existing trajectories. This approach relies on high-confidence observations and comprehensive cost dimensions to ensure the accuracy of core matching results and reduce interference from low-quality detection boxes. Secondly, it performs a second-stage matching process for trajectories that did not match in the first stage and low-confidence detection boxes, continuing the matching logic of multi-dimensional comprehensive cost and medium thresholds to further explore effective matching relationships and avoid trajectory loss due to a single strict screening. In the final stage, for trajectories and detection boxes that still do not match in the first two stages, only the cost of the high-modulation intersection comparison at this core position is considered, and a higher matching threshold is set for a third-stage fallback matching. This simplifies the computational complexity and can recover some potential matching pairs caused by disturbances while ensuring matching reliability. At the same time, by setting progressive thresholds of "high threshold → medium threshold → low strictness high threshold", a balance between matching accuracy and recall rate is achieved. This phased matching strategy is well adapted to the characteristics of inconsistent detection box quality and target appearance being easily disturbed in dynamic maritime environments. Ultimately, it effectively reduces the problems of trajectory interruption and identity switching, and improves the robustness and stability of multi-target tracking.

[0081] S105. For the successfully matched trajectory, update the observation center Kalman filter state and horizontal motion momentum of the trajectory using the corresponding detection box, and retain it as a continuing trajectory.

[0082] Specifically, for all successfully matched trajectory-detection pairs, the predicted state vector of the observation center Kalman filter corresponding to the trajectory is first retrieved. The position and size information of the matched detection box are converted into observation vectors. The measurement noise covariance corresponding to the detection box is set to the first threshold according to preset rules or adaptively adjusted according to the confidence of the detection box. Then, based on the adjusted measurement noise covariance and combined with the Kalman filter update rules, the predicted state vector is corrected with the observation vector to obtain the final state vector of the trajectory in the current frame, thus completing the update of the observation center Kalman filter state. Subsequently, the coordinates of four types of feature points—left midpoint, right midpoint, top midpoint, and bottom midpoint—of the matched detection box of the trajectory are extracted. Combined with the coordinates of the historical bounding box feature points of the trajectory in the previous frame, the horizontal displacement of the effective feature points between adjacent frames is calculated. The arithmetic mean of the horizontal displacements of the effective left midpoint and right midpoint is taken. Then, the updated horizontal motion velocity is calculated based on the time interval between adjacent frames. The horizontal motion direction is determined according to the positive and negative directions of the horizontal displacement. The updated horizontal motion velocity and direction together constitute the new horizontal motion momentum of the trajectory. Finally, the trajectory that has completed the filter state update and horizontal motion momentum update is retained as the continuation trajectory of the current frame.

[0083] S106. For unmatched trajectories, determine whether the number of consecutive unmatched frames has reached a preset termination threshold. If it has, terminate the trajectory; otherwise, retain it as a continuing trajectory.

[0084] Specifically, firstly, all unmatched existing trajectories are traversed, and the statistical data of the number of consecutive unmatched frames corresponding to each existing trajectory is retrieved. This statistical data is compared with a preset trajectory termination threshold. If the number of consecutive unmatched frames of a certain trajectory reaches or exceeds the preset termination threshold, it is determined that the target corresponding to the trajectory has left the tracking range or disappeared, and the trajectory termination operation is performed. If the number of consecutive unmatched frames of a certain trajectory does not reach the preset termination threshold, the existing status of the trajectory is maintained, its consecutive unmatched frame statistical data is incremented by 1 and updated and stored, and reserved as the existing trajectory required for the next frame of tracking.

[0085] S107. Select a first unmatched detection box with a confidence level higher than a preset value from the unmatched detection boxes, and initialize a new trajectory based on the first unmatched detection box.

[0086] The new trajectory is incorporated into the existing trajectory of the current frame for target tracking in the next frame.

[0087] Specifically, firstly, all unmatched detection boxes are traversed, and detection boxes with a confidence level higher than a preset value are selected as the first unmatched detection boxes. Next, the multi-dimensional state vector of the corresponding new trajectory is initialized based on the position and size information of each first unmatched detection box. At the same time, the number of consecutive unmatched frames of the new trajectory is initialized to 0, and the appearance features of the detection box are extracted as the initial historical appearance features of the new trajectory. The initial horizontal motion momentum of the target corresponding to the detection box is calculated. Finally, all the newly initialized trajectories are included in the existing trajectory set of the current frame for target tracking in the next frame.

[0088] Optionally, after initializing a new trajectory based on the first unmatched detection box, the method further includes: setting a pending confirmation flag for the initialized new trajectory and starting a continuous frame matching count, with the initial count value set to 0; during the tracking process of each frame for a subsequent preset number of consecutive frames, for the new trajectory to be confirmed, if the new trajectory in the current frame successfully matches the corresponding detection box through phased association matching, then the continuous frame matching count is incremented by 1; if the new trajectory in the current frame does not match any detection box, then the continuous frame matching count is directly reset to 0, and the pending confirmation flag is maintained; when the continuous frame matching count reaches a preset confirmation threshold, the pending confirmation flag of the new trajectory is removed, updated to a confirmed trajectory, and included in the formal list of existing trajectories; if the continuous frame matching count never reaches the confirmation threshold in subsequent preset frames, or if two or more consecutive frames fail to match, causing the count to be reset multiple times, then the new trajectory is determined to be a pseudo trajectory, and a trajectory deletion operation is performed.

[0089] In specific implementation, after initializing a new trajectory based on the first unmatched detection box, a pending confirmation mark is immediately set for each initialized new trajectory, and a continuous frame matching count is started, with the initial value of the count set to 0. During the tracking process of each frame for the subsequent preset number of consecutive frames, for a new trajectory with a pending confirmation mark, if the new trajectory successfully matches the corresponding detection box through phased association matching in the current frame, its continuous frame matching count is incremented by 1. If the new trajectory does not match any detection box in the current frame, its continuous frame matching count is directly reset to 0, and the pending confirmation mark is maintained. When the continuous frame matching count of a pending new trajectory reaches the preset confirmation threshold, the pending confirmation mark of the new trajectory is immediately removed, it is updated as a confirmed trajectory, and included in the formal list of existing trajectories. If, during the subsequent preset frame tracking period, the continuous frame matching count of a pending new trajectory never reaches the preset confirmation threshold, or if the count is reset multiple times due to two or more consecutive unmatched frames, the new trajectory is determined to be a pseudo trajectory, and a trajectory deletion operation is performed.

[0090] The method provided in this embodiment, in its first aspect, employs a Kalman filtering strategy from an observation center to predict the state of the existing trajectory. By constructing a multidimensional state vector containing position, size, and corresponding velocity components, and combining the assumption of approximate uniform motion with the state transition matrix, the trajectory state is deduced. Simultaneously, based on the confidence level of the detection box, the measurement noise covariance is adaptively adjusted, causing the Kalman gain to tilt towards high-confidence observation results. This effectively offsets the nonlinear prediction bias caused by the turbulence of the offshore platform and water flow disturbance, avoids the continuous amplification of prediction errors, improves the fit between the predicted bounding box and the true position of the target, and provides accurate position references for subsequent association matching.

[0091] Secondly, after dividing the detection boxes according to their confidence levels, the matching process is carried out in stages. High-confidence detection boxes are prioritized for strict matching in combination with predicted bounding boxes, horizontal motion momentum, and appearance features to ensure the accuracy of core matching results and reduce interference from low-confidence detection boxes. Then, a second matching is performed on unmatched trajectories and medium-confidence detection boxes to further explore effective matching relationships and avoid trajectory loss caused by a single strict screening. Finally, a fallback matching is used to recover potential effective matching pairs, achieving a balance between matching accuracy and recall. At the same time, the staged matching strategy can adapt to the uneven quality of detection boxes in the marine environment, effectively reducing matching errors caused by target occlusion and changes in lighting, reducing trajectory interruption and identity switching problems, and improving the robustness of nonlinear target tracking.

[0092] Thirdly, a height-modulated cross-union ratio is introduced in the association matching. By combining the calculation methods of planar cross-union ratio and vertical overlap ratio, it not only retains the traditional cross-union ratio's ability to represent the degree of overlap of the target's planar position, but also strengthens the weight of vertical position consistency. This can adapt to the vertical position fluctuations of the target at sea due to the rise and fall of the waves, reduce matching misjudgments caused by the vertical displacement of the target, and improve the accuracy of cost calculation under complex working conditions.

[0093] Fourthly, by initializing new trajectories using high-confidence detection boxes selected from unmatched detection boxes, interference from low-confidence noise detection boxes can be filtered out, reducing the generation of pseudo-trajectories. At the same time, a confirmation mark and a continuous frame matching count mechanism are set for the new trajectory. The effectiveness of the new trajectory is verified through subsequent multi-frame tracking. Only trajectories with a continuous matching count reaching the confirmation threshold are included in the official existing trajectory list. Pseudo-trajectories that do not reach the threshold or have been reset multiple times are deleted. This further improves the reliability of trajectory initialization, avoids pseudo-trajectories occupying computing resources, and ensures the overall stability and efficiency of multi-target tracking in dynamic maritime environments.

[0094] Corresponding to the aforementioned embodiment of a nonlinear target tracking method suitable for dynamic marine environments, this application also provides an embodiment of a nonlinear target tracking device suitable for dynamic marine environments.

[0095] Figure 3 This is a schematic diagram of the nonlinear target tracking device suitable for dynamic marine environments provided in Embodiment 2 of this application. Please refer to... Figure 3 The apparatus provided in this embodiment includes an acquisition module 310, a prediction module 320, a matching module 330, and a processing module 340.

[0096] The acquisition module 310 is used to acquire all target detection boxes output by the target detector in the current frame and the confidence level corresponding to each target detection box;

[0097] The prediction module 320 is used to predict the state of each existing trajectory based on the observation center Kalman filter strategy for all existing trajectories in the previous frame, and obtain the predicted bounding box of each existing trajectory in the current frame; the existing trajectory is the target trajectory in the previous frame that has not reached the preset termination condition and needs to be tracked.

[0098] The prediction module 320 is also used to extract horizontal displacement information based on the historical bounding box feature points corresponding to each existing trajectory in the previous frame to estimate the motion momentum and obtain the horizontal motion momentum of each existing trajectory.

[0099] The matching module 330 is used to divide the target detection box in the current frame according to the confidence level, and combine the predicted bounding box, the horizontal motion momentum and the target appearance features to perform association matching between the existing trajectory and the detection box in stages, and output the successfully matched trajectory-detection pair, the unmatched trajectory and the unmatched detection box.

[0100] The processing module 340 is used to update the observation center Kalman filter state and horizontal motion momentum of the successfully matched trajectory using the corresponding detection box, and retain it as a continuing trajectory.

[0101] The processing module 340 is also used to determine whether the number of consecutive unmatched frames for the unmatched trajectory has reached a preset termination threshold. If it has, the trajectory is terminated; otherwise, it is retained as a continuing trajectory.

[0102] The processing module 340 is further configured to filter out a first unmatched detection box with a confidence level higher than a preset value from the unmatched detection boxes, and initialize a new trajectory based on the first unmatched detection box; the new trajectory is incorporated into the existing trajectory of the current frame for target tracking in the next frame.

[0103] The apparatus of this embodiment can be used to perform... Figure 1 The steps of the method embodiment shown are similar in principle and process, and will not be repeated here.

[0104] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0105] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0106] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A nonlinear target tracking method suitable for dynamic marine environments, characterized in that, The method includes: Obtain all target detection boxes output by the target detector in the current frame and the confidence score of each target detection box; For all existing trajectories in the previous frame, the state of each existing trajectory is predicted based on the observation center Kalman filter strategy to obtain the predicted bounding box of each existing trajectory in the current frame; the existing trajectory is the target trajectory in the previous frame that has not reached the preset termination condition and needs to be tracked. Based on the historical bounding box feature points corresponding to each existing trajectory in the previous frame, horizontal displacement information is extracted to estimate the motion momentum, and the horizontal motion momentum of each existing trajectory is obtained. The target detection boxes in the current frame are divided according to the confidence level. Combining the predicted bounding box, the horizontal motion momentum and the target appearance features, the existing trajectory and the detection box are matched in stages, and the successfully matched trajectory-detection pair, the unmatched trajectory and the unmatched detection box are output. For a successfully matched trajectory, the observation center Kalman filter state and horizontal momentum of the trajectory are updated using the corresponding detection box, and the trajectory is retained as a continuing trajectory. For unmatched trajectories, determine whether the number of consecutive unmatched frames has reached a preset termination threshold. If it has, terminate the trajectory; otherwise, retain it as a continuing trajectory. The first unmatched detection box with a confidence level higher than a preset value is selected from the unmatched detection boxes, and a new trajectory is initialized based on the first unmatched detection box; the new trajectory is included in the existing trajectory of the current frame for target tracking in the next frame.

2. The method according to claim 1, characterized in that, Based on the observation center's Kalman filter strategy, the state of each surviving trajectory is predicted, resulting in the predicted bounding box of each trajectory in the current frame, including: Construct a multi-dimensional state vector for each existing trajectory; For the target trajectory, a state transition matrix is ​​constructed based on the assumption of approximate uniform motion. Combined with the preset process noise, the multidimensional state vector of the target trajectory in the previous frame is used to obtain the predicted state vector of the target trajectory in the current frame through linear deduction calculation. The center coordinates and size information of the corresponding target bounding box are extracted from the predicted state vector to generate the predicted bounding box of the target trajectory in the current frame. When a detection box matching the target's trajectory is obtained in the current frame, the measurement noise covariance corresponding to the detection box is set to a first threshold, or the measurement noise covariance is adaptively adjusted according to the confidence level of the detection box; the higher the confidence level of the detection box, the smaller the measurement noise covariance. Based on the adjusted measurement noise covariance and combined with the Kalman filter update rule, the predicted state vector of the target trajectory is corrected using the matched detection box to obtain the final state vector of the target trajectory in the current frame.

3. The method according to claim 1, characterized in that, Based on the historical bounding box feature points corresponding to each existing trajectory in the previous frame, horizontal displacement information is extracted to estimate motion momentum, resulting in the horizontal motion momentum of each existing trajectory, including: Define four types of feature points for the historical bounding boxes corresponding to each existing trajectory in the previous frame; the four types of feature points include the left midpoint, right midpoint, top midpoint, and bottom midpoint; Obtain the bounding boxes of each existing trajectory in adjacent historical frames, and determine the coordinates of four types of feature points of the bounding boxes of the same trajectory in adjacent frames respectively; Outlier removal is performed on the coordinates of the four types of feature points to obtain valid feature points; the valid feature points include at least the left midpoint and the right midpoint. Calculate the horizontal displacement values ​​of effective feature points between adjacent frames of the same persistent trajectory, and take the arithmetic mean of the horizontal displacements of all left and right midpoints to obtain the average horizontal displacement of the persistent trajectory between adjacent frames. The horizontal motion velocity of the existing trajectory is calculated based on the average horizontal displacement and the time interval between adjacent frames, and the horizontal motion direction is determined based on the positive and negative directions of the average horizontal displacement. The horizontal motion velocity and the horizontal motion direction together constitute the horizontal motion momentum of the existing trajectory.

4. The method according to claim 3, characterized in that, Outlier removal is performed on the coordinates of four types of feature points to obtain valid feature points, including: Obtain the bounding boxes of the existing trajectory in adjacent historical frames, and calculate the horizontal displacements of the four types of feature points between adjacent frames of the same trajectory. Consistency judgment is made on the horizontal displacement of four types of feature points on the same trajectory. If the difference between the horizontal displacement of the target feature point and the horizontal displacement of the other three types of feature points exceeds the preset displacement deviation threshold, or if the horizontal displacement direction of the target feature point is opposite to the estimated horizontal momentum direction of the existing trajectory in the previous frame and the included angle exceeds the preset angle threshold, then the target feature point is determined to be an abnormal feature point and is removed to obtain a valid feature point.

5. The method according to claim 1, characterized in that, Combining the predicted bounding box, the horizontal motion momentum, and the target appearance features, the existing trajectory and the detection box are associated and matched in stages, including: For each pair of surviving trajectory-detection box combinations, a comprehensive cost is calculated; the comprehensive cost is obtained by weighting the height modulation crossover ratio, horizontal motion momentum, and target appearance features. The first set of detection boxes with a confidence level higher than the first preset confidence threshold is selected from the detection boxes in the current frame. The first association cost matrix is ​​constructed with the existing trajectory as the row and the first detection box as the column. The element of the first association cost matrix is ​​the comprehensive cost of the corresponding existing trajectory – the first detection box. The minimum cost matching of the first association cost matrix is ​​solved based on the Hungarian algorithm. Matching pairs with a comprehensive cost lower than the first preset cost threshold are retained as the first stage of successful trajectory-detection pairs. At the same time, unmatched existing trajectories and unmatched first detection boxes are marked.

6. The method according to claim 5, characterized in that, After simultaneously marking unmatched existing trajectories with unmatched first detection boxes, the method further includes: Filter out a second set of detection boxes from the current frame detection boxes, whose confidence level is not higher than the first preset confidence threshold and is higher than the second preset confidence threshold; A second association cost matrix is ​​constructed using the unmatched trajectories from the first stage as rows and the second detection boxes as columns. Solve the minimum cost matching of the second association cost matrix, retain the matching pairs whose comprehensive cost is lower than the second preset cost threshold as the trajectory-detection pairs that are successfully matched in the second stage, and mark the still unmatched existing trajectories and the unmatched second detection boxes. For the unmatched trajectories and unmatched second detection boxes in the second stage, a third association cost matrix is ​​constructed based on the height modulation intersection-union ratio of the predicted bounding boxes and the detection boxes. The third correlation cost matrix is ​​matched, and the matching pairs with a comprehensive cost lower than the third preset cost threshold are retained as the trajectory-detection pairs that are successfully matched in the third stage; the third preset cost threshold is greater than the first and second preset cost thresholds. Integrate the three-stage matching results and output all successfully matched trajectory-detection pairs, the final unmatched surviving trajectories, and the final unmatched detection boxes.

7. The method according to claim 5, characterized in that, The calculation process for the high-modulation cross-connection ratio includes: Determine the predicted bounding box of the existing trajectory and the detection box of the current frame, and extract the position information and planar coordinate information of the two candidate boxes in the vertical direction respectively; Determine the overlapping region of the two candidate boxes in the planar coordinate system and calculate the area of ​​the overlapping region; Calculate the planar area of ​​each of the two candidate boxes, sum the planar areas and subtract the area of ​​the overlapping region to obtain the area of ​​the planar union of the two candidate boxes; The ratio of the area of ​​the overlapping region to the area of ​​the plane union is used as the plane intersection-union ratio of the two candidate boxes; Based on the start and end coordinates of the two candidate boxes in the vertical direction, determine the overlapping interval in the vertical direction, calculate the length of the overlapping interval, and use it as the vertical overlap height. Calculate the length of the union interval between the two candidate boxes in the vertical direction, and use it as the vertical union height; The ratio of the vertical overlap height to the vertical union height is used as the height overlap ratio of the two candidate boxes. Multiplying the planar crossover ratio by the height overlap ratio yields the height modulation crossover ratio.

8. The method according to claim 1, characterized in that, After initializing a new trajectory based on the first unmatched detection box, the method further includes: Set a confirmation marker for the newly initialized trajectory and start continuous frame matching counting, with the initial count value set to 0; During the tracking process of each frame for the subsequent preset number of consecutive frames, if a new trajectory to be confirmed is successfully matched with the corresponding detection box in the current frame through phased association matching, the consecutive frame matching count is incremented by 1. If no detection box is matched with the new trajectory in the current frame, the consecutive frame match count is reset to 0 and the pending confirmation mark is maintained. When the consecutive frame matching count reaches the preset confirmation threshold, the pending confirmation mark of the new trajectory is removed, updated to a confirmed trajectory, and included in the formal list of existing trajectories. If the consecutive frame matching count fails to reach the confirmation threshold within subsequent preset frames, or if two or more consecutive frames fail to match, causing the count to reset multiple times, the new trajectory is determined to be a pseudo trajectory, and the trajectory deletion operation is performed.

9. The method according to claim 3, characterized in that, The left midpoint is the intersection of the left edge of the historical bounding box and the center height of the bounding box, where the center height is the vertical position corresponding to 1 / 2 of the bounding box height; the right midpoint is the intersection of the right edge of the historical bounding box and the center height of the bounding box; the top midpoint is the intersection of the top edge of the historical bounding box and the center width of the bounding box, where the center width is the horizontal position corresponding to 1 / 2 of the bounding box width; and the bottom midpoint is the intersection of the bottom edge of the historical bounding box and the center width of the bounding box.

10. A nonlinear target tracking device suitable for dynamic marine environments, characterized in that, The device includes an acquisition module, a prediction module, a matching module, and a processing module; The acquisition module is used to acquire all target detection boxes output by the target detector in the current frame and the confidence level corresponding to each target detection box. The prediction module is used to predict the state of each existing trajectory based on the observation center Kalman filter strategy for all existing trajectories in the previous frame, and obtain the predicted bounding box of each existing trajectory in the current frame; the existing trajectory is the target trajectory in the previous frame that has not reached the preset termination condition and needs to be tracked. The prediction module is also used to extract horizontal displacement information based on the historical bounding box feature points corresponding to each existing trajectory in the previous frame to estimate the motion momentum and obtain the horizontal motion momentum of each existing trajectory. The matching module is used to divide the target detection boxes in the current frame according to the confidence level, and combine the predicted bounding box, the horizontal motion momentum and the target appearance features to perform association matching between the existing trajectory and the detection box in stages, and output the successfully matched trajectory-detection pair, the unmatched trajectory and the unmatched detection box. The processing module is used to update the observation center Kalman filter state and horizontal motion momentum of the successfully matched trajectory using the corresponding detection box, and retain it as a continuing trajectory. The processing module is also used to determine whether the number of consecutive unmatched frames for an unmatched trajectory has reached a preset termination threshold. If it has, the trajectory is terminated; otherwise, it is retained as a continuing trajectory. The processing module is further configured to filter out a first unmatched detection box with a confidence level higher than a preset value from the unmatched detection boxes, and initialize a new trajectory based on the first unmatched detection box; the new trajectory is incorporated into the existing trajectory of the current frame for target tracking in the next frame.

Images