A ship target recognition and tracking method and system based on multi-source data fusion

Abstract

A ship target identification and tracking method and system based on multi-source data fusion relates to the field of electronic digital data processing. The method includes: collecting ship monitoring data from multiple sensors, including position coordinates, area, and color features; generating initial ship trajectories based on the monitoring data; calculating the speed, area, and color similarity of the detection points and correlating them to obtain trajectory pairs; identifying outliers and optimizing trajectories based on a similarity energy function that includes options such as dynamics and persistence; calculating the state differences between the optimized trajectories for similarity testing, and determining trajectories exceeding a threshold as the same target to form a multi-source trajectory group; and calculating dynamic fusion weights based on the standard deviation of the trajectory errors relative to the average trajectory. The target trajectory is then obtained through weighted fusion. Implementing this method can improve the fusion accuracy of ship target trajectories.

Description

Technical Field

[0001] The present application relates to the field of electronic digital data processing, and in particular to a method and system for ship target identification and tracking based on multi-source data fusion. Background Art

[0002] Accurate identification and real-time tracking of ship targets are crucial safeguards for maritime safety and regulation. With the increasing frequency of maritime vessel activity and the expansion of navigation ranges, higher requirements are being placed on the accuracy and reliability of ship monitoring.

[0003] Among related technologies, maritime ship identification and tracking primarily utilizes radar detection, optical remote sensing, and satellite remote sensing. These technologies employ sensors to collect ship information, acquiring characteristic data such as the ship's position and speed, and then process this data to determine the ship's trajectory. Radar detection operates 24 / 7 but is susceptible to interference from waves, optical remote sensing offers high resolution but is significantly affected by weather, and satellite remote sensing offers wide coverage but has a long update cycle.

[0004] In practical applications, due to the limited detection capabilities of a single sensor, the acquired ship trajectory data often suffers from issues such as unstable accuracy, missing data, and outliers. When sea conditions are complex or weather conditions change, the quality of the data collected by the sensor can significantly degrade, leading to deviations in the generated ship trajectory and compromising subsequent monitoring and early warning. Summary of the Invention

[0005] The present application provides a ship target recognition and tracking method and system based on multi-source data fusion, which is used to improve the fusion accuracy of ship target trajectories.

[0006] In a first aspect, the present application provides a ship target identification and tracking method based on multi-source data fusion, which uses a ship target identification and tracking system. The method includes: collecting ship monitoring data of the same sea area from multiple source sensors, the ship monitoring data including ship position coordinates, area characteristics and color characteristics; generating an initial ship trajectory of each sensor based on the ship monitoring data, calculating the detection point similarity of the initial ship trajectory and associating them to obtain an associated trajectory pair, the detection point similarity including speed similarity, area similarity and color similarity; performing outlier identification and trajectory optimization on the associated trajectory pair based on a similarity energy function to obtain an optimized ship trajectory, the similarity energy function including a dynamic option, a persistence option, a fusion interval option, a color similarity option and an adjustment option; performing a similarity test by calculating the state difference between the optimized ship trajectories, and determining the optimized ship trajectories whose similarity exceeds a preset threshold as the same target and associating them to form a multi-source trajectory group; calculating a dynamic fusion weight based on the standard deviation of the measurement error of each optimized ship trajectory in the multi-source trajectory group relative to its average trajectory, and performing weighted fusion to obtain a ship target trajectory.

[0007] By employing this technical solution, ship monitoring data collected by multi-source sensors includes location, area, and color features, providing a comprehensive data foundation for subsequent similarity calculations. By correlating detection point similarities and optimizing the similarity energy function, outliers are effectively identified and eliminated. Combined with a multi-source trajectory group fusion mechanism using dynamic fusion weights, this mechanism leverages the measurement strengths of different sensors, ultimately generating ship target trajectories with greater accuracy and reliability.

[0008] In combination with some embodiments of the first aspect, in some embodiments, the step of calculating the similarity of the detection points of the initial ship trajectory and associating them to obtain an associated trajectory pair specifically includes: calculating the speed similarity of adjacent detection points based on the position information of the detection points in the initial ship trajectory; calculating the area similarity of adjacent detection points based on the rectangular box of the detection points in the initial ship trajectory; calculating the color similarity of adjacent detection points based on the color features of the detection points in the initial ship trajectory; and associating adjacent detection points based on a weighted combination of the speed similarity, the area similarity and the color similarity to obtain an associated trajectory pair.

[0009] By employing this technical solution, the similarities of the three dimensions of speed, area, and color of the detection points are weighted and combined to correlate. Speed similarity reflects motion consistency, area similarity reflects the scale variation of the target, and color similarity ensures accurate target recognition. The synergistic effect of these multi-dimensional features significantly improves the accuracy of trajectory correlation and reduces the occurrence of false and missed associations.

[0010] In combination with some embodiments of the first aspect, in some embodiments, the steps of identifying outliers and optimizing the trajectory of the initial ship trajectory based on the similarity energy function to obtain the optimized ship trajectory specifically include: using the similarity energy function to calculate the similarity of adjacent trajectory points in the initial ship trajectory, and the similarity energy function includes a dynamic option, a persistence option, a fusion interval option, a color similarity option and an adjustment option; constructing a linear model based on the speed drift coefficient and the disturbance variance factor to calculate the mutation probability of the trajectory point, eliminating the trajectory points whose mutation probability exceeds a preset threshold, and using the moving mean method to supplement the missing segments after eliminating the outliers to obtain the optimized ship trajectory.

[0011] By employing these technical solutions, the similarity energy function integrates multiple options, including dynamics and persistence, to comprehensively characterize the correlations between trajectory points. A linear model based on the velocity drift coefficient and the disturbance variance factor accurately captures trajectory mutations, while the moving mean method complements missing segments to maintain trajectory continuity. The optimized ship trajectory is smoother and more coherent, effectively eliminating abnormal fluctuations.

[0012] In combination with some embodiments of the first aspect, in some embodiments, the step of performing a similarity test by calculating the state difference between the optimized ship trajectories, determining the optimized ship trajectories with a similarity exceeding a preset threshold as the same target and associating them to form a multi-source trajectory group, specifically includes: calculating the state estimation difference of the optimized ship trajectories of different sensors, where the state estimation difference includes position, heading and speed information; performing a likelihood ratio test based on the state estimation difference through a joint probability density function based on an uncertain distribution to determine whether the different sensor trajectories belong to the same target, and obtaining the associated multi-source trajectory group.

[0013] By employing this technical solution, the state estimate difference encompasses position, heading, and speed information, comprehensively reflecting the ship's motion. A likelihood ratio test based on the joint probability density function of the uncertainty distribution accurately assesses the degree of correlation between different sensor trajectories. The resulting multi-source trajectory group fully utilizes the detection advantages of each sensor, providing a high-quality data foundation for subsequent fusion.

[0014] In combination with some embodiments of the first aspect, in some embodiments, the step of calculating the dynamic fusion weight based on the standard deviation of the measurement error of each optimized ship trajectory in the multi-source trajectory group relative to its average trajectory, and performing weighted fusion to obtain the ship target trajectory, specifically includes: calculating the standard deviation of the measurement error of each optimized ship trajectory in the multi-source trajectory group relative to the average trajectory of the multi-source trajectory group; calculating the dynamic fusion weight based on the measurement error standard deviation; and performing weighted fusion on the multi-source trajectory group according to the dynamic fusion weight to obtain the ship target trajectory.

[0015] By employing this technical solution, the standard deviation of the measurement error of each optimized ship trajectory relative to the average trajectory in a multi-source trajectory group reflects the trajectory's reliability. Dynamic fusion weights are adjusted in real time as measurement errors change, giving higher-accuracy trajectories a greater role in the fusion process. The weighted fusion of the target ship trajectory inherits the advantages of each sensor.

[0016] In combination with some embodiments of the first aspect, in some embodiments, the step of generating the initial ship trajectory of each sensor based on the ship monitoring data specifically includes: when no detection point associated with the target detection point is found in the previous frame image, a new trajectory segment is generated with the detection point as the starting point of the trajectory; when no detection point associated with the target detection point is found in the next frame image, the target detection point is used as the end point of the current trajectory segment; judging whether the number of frames in which the trajectory appears continuously is less than a preset threshold, and if so, determining it as a trajectory fragment; calculating the energy function value after the trajectory fragment is fused with other trajectory segments, and retaining the trajectory fragment when the energy function value decreases, otherwise removing it from the trajectory space to obtain the initial ship trajectory.

[0017] By employing this technical solution, trajectory fragments are dynamically generated and terminated based on the correlation of detection points, achieving automatic trajectory initialization. The energy function value judgment mechanism effectively screens trajectory fragments and eliminates false trajectories caused by noise. The generated initial ship trajectory exhibits strong spatiotemporal continuity, laying a solid foundation for subsequent trajectory optimization.

[0018] In combination with some embodiments of the first aspect, in some embodiments, the step of eliminating trajectory points whose mutation probability exceeds a preset threshold and using the moving mean method to supplement the missing fragments after eliminating the abnormal points to obtain the optimized ship trajectory specifically includes: setting a sliding time window, calculating the speed drift coefficient and the disturbance variance factor of the trajectory point within the sliding time window; constructing a linear model of the trajectory point position based on the speed drift coefficient and the disturbance variance factor, and calculating the mutation probability of the trajectory point according to the deviation between the trajectory point and the predicted position of the linear model; eliminating the abnormal points whose mutation probability exceeds the preset probability threshold to obtain the trajectory fragment after eliminating the abnormal points; using the moving mean method to supplement the missing part in the trajectory fragment after eliminating the abnormal points, and when the time interval between the trajectory fragments exceeds the preset interval threshold, using a two-way search method to supplement them to obtain the optimized ship trajectory.

[0019] By employing this technical solution, the velocity drift coefficient and disturbance variance factor calculated within a sliding time window reflect the changing characteristics of the local trajectory. A linear model-based mutation probability calculation accurately identifies outliers, while a moving mean method and bidirectional search method adaptively select supplementary strategies based on time intervals. The optimized ship trajectory maintains the original motion pattern while eliminating trajectory fluctuations caused by sensor errors, improving trajectory continuity and smoothness.

[0020] In a second aspect, an embodiment of the present application provides a ship target identification and tracking system, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the ship target identification and tracking system to perform the method described in the first aspect and any possible implementation of the first aspect.

[0021] In a third aspect, an embodiment of the present application provides a computer program product comprising instructions. When the above-mentioned computer program product is run on a ship target identification and tracking system, the above-mentioned ship target identification and tracking system executes the method described in the first aspect and any possible implementation method of the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium comprising instructions. When the instructions are executed on a ship target identification and tracking system, the ship target identification and tracking system executes the method described in the first aspect and any possible implementation of the first aspect.

[0023] It is understood that the ship target identification and tracking system provided in the second aspect, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the methods provided in the embodiments of this application. Therefore, the beneficial effects achievable by these methods can be referenced to the beneficial effects of the corresponding methods and will not be further elaborated here.

[0024] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0025] 1. This application uses multi-source sensor data collected from ships, including location, area, and color features, to provide a comprehensive data foundation for subsequent similarity calculations. This method effectively identifies and removes outliers based on the correlation of detection point similarities and the optimization of similarity energy functions. A multi-source trajectory group fusion mechanism, combined with dynamic fusion weights, leverages the measurement strengths of different sensors, ultimately generating ship target trajectories with greater accuracy and reliability.

[0026] 2. This application uses a weighted combination of similarities in three dimensions: speed, area, and color. Speed similarity reflects motion consistency, area similarity reflects the scale variation of the target, and color similarity ensures target recognition accuracy. The synergistic effect of these multi-dimensional features significantly improves the accuracy of trajectory association and reduces the occurrence of false and missed associations.

[0027] 3. This application integrates multiple options, such as dynamics and persistence, through a similarity energy function to comprehensively characterize the correlation between trajectory points. A linear model constructed based on the velocity drift coefficient and the disturbance variance factor accurately captures the characteristics of trajectory mutations, and the moving mean method supplements missing segments to maintain trajectory continuity. The optimized ship trajectory is smoother and more coherent, effectively eliminating abnormal fluctuations. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a flow chart of a method for identifying and tracking ship targets based on multi-source data fusion in an embodiment of the present application;

[0029] Figure 2 This is another flowchart of the ship target identification and tracking method based on multi-source data fusion in an embodiment of the present application;

[0030] Figure 3 This is a schematic diagram of the physical device structure of the ship target identification and tracking system in the embodiment of the present application. DETAILED DESCRIPTION

[0031] The terms used in the following examples of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of the present application, the singular expressions "a", "an", "above", "the", and "this" are intended to include plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations of one or more of the listed items.

[0032] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of the technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0033] For ease of understanding, the application scenarios of the embodiments of the present application are introduced below.

[0034] Identification and tracking of ships at sea is one of the important means to maintain marine safety. With the increasing maneuverability of ships at sea and the diversification of the environment, a single sensor is unable to meet the needs of accurate ship tracking. Multi-source information fusion is mainly used in the field of target tracking, by coordinating the operation of multiple sensors to perform information fusion processing, so as to achieve a more accurate estimation and judgment of the actual situation of the target. At present, the methods used for ship tracking are mainly based on point trace correlation fusion based on AIS and satellite-borne SAR data, correlation fusion based on electronic and optical remote sensing, point trace fusion based on ground wave radar and AIS, and point trace fusion based on satellite-borne SAR, ground wave radar and AIS. With the increase in ship detection sensors, how to fuse multi-source sensors to carry out ship identification and tracking in order to further improve the accuracy of ship identification and tracking has become a hot research topic. The present invention is based on target and trajectory data obtained by electronic information, satellite-borne SAR, and high-resolution optical recognition, and through processing technologies such as trajectory optimization, trajectory correlation, and trajectory information fusion, it can achieve high-precision ship target identification and tracking.

[0035] For ease of understanding, the following describes the process of the method provided by this implementation in combination with the above scenario. Figure 1 , which is a flow chart of a ship target identification and tracking method based on multi-source data fusion in an embodiment of the present application.

[0036] S101. Collect ship monitoring data from multiple source sensors on the same sea area, where the ship monitoring data includes ship position coordinates, area characteristics, and color characteristics.

[0037] Multi-source sensors refer to multiple different types of monitoring methods capable of acquiring vessel information in the same ocean, including but not limited to radar satellites, optical satellites, AIS, and optoelectronics. Ship monitoring data refers to the collection of characteristic information about ship targets collected by sensors. Vessel position coordinates represent the ship's position on a two-dimensional plane, including longitude and latitude. Area features refer to the area occupied by a vessel target in the image. Color features represent the appearance of a vessel target and are described using a color histogram.

[0038] This step is performed when multiple sensors are monitoring the sea area. Specifically, the system first activates multiple sensors to monitor the designated sea area in real time. For each sensor's data, the system extracts the location coordinates (x, y) of the vessel target, calculates the area a of the rectangular box surrounding the vessel target, and constructs a color histogram h to describe the vessel's appearance. The system organizes and stores this feature data to form a complete vessel monitoring dataset.

[0039] In some embodiments, multi-source sensor data acquisition can be achieved through various methods: Optionally, fixed radar scanning can be used to acquire ship echo signals, extract ship position information through signal processing, analyze echo intensity to determine target area, and record echo characteristics; ship position messages can be acquired using AIS receivers, parsed to extract position coordinates, and area calculated based on ship dimensions; optoelectronic equipment can be used to capture ship images and perform image segmentation to determine target areas. Optionally, airborne radar can be used to perform round-trip scanning of the sea area to acquire dynamic information about ship targets; optoelectronic pods can be simultaneously activated to track and capture ship images, and ship AIS information can be received as auxiliary data. It is understood that other types of sensors or different combinations of sensors can also be used to achieve data acquisition, and this is not limited here.

[0040] S102: Generate an initial ship trajectory of each sensor based on the ship monitoring data, calculate the detection point similarity of the initial ship trajectory, and associate them to obtain an associated trajectory pair, where the detection point similarity includes speed similarity, area similarity, and color similarity.

[0041] Ship monitoring data refers to the raw position, speed, and other information obtained by sensors detecting a ship target. An initial ship trajectory represents a sequence of track points generated from ship monitoring data acquired by a single sensor. Detection point similarity indicates the degree of similarity between ship trajectory points acquired by different sensors. Associated trajectory pairs are pairs of track points from different sensors that are determined to be the same ship target through similarity calculations.

[0042] Specifically, this step, performed after acquiring ship monitoring data, is used to generate initial trajectories and perform similarity assessment. First, based on the ship monitoring data collected by each sensor, an initial trajectory sequence for each sensor is formed through time-series correlation. Then, velocity similarity, area similarity, and color similarity are calculated between trajectory points from different sensors. Speed similarity is calculated based on the Euclidean distance of velocity vectors, area similarity is calculated based on the ratio of the target's bounding rectangle areas, and color similarity is calculated based on the Bhattacharyya coefficient of the color histogram. Finally, based on a similarity threshold, trajectory points with high similarity are paired to form associated trajectory pairs.

[0043] In some embodiments, initial trajectory generation and similarity calculation can be achieved in a variety of ways: optionally, through a sliding time window method, the correlation between track points is calculated in each window based on the temporal continuity of the ship monitoring data, the points with high correlation are connected to form an initial trajectory segment, and then the trajectory segments are spliced to obtain a complete initial trajectory; similarity feature values are calculated for different sensor trajectory points in pairs, and the similarity is comprehensively evaluated in combination with a multi-feature fusion method, and the point pair with the highest similarity is selected as the associated trajectory pair. Optionally, a target tracking filter is used to filter the ship monitoring data to obtain a smooth trajectory; based on the trajectory segment feature extraction technology, the speed, area and color feature vectors are calculated respectively; the cosine similarity of the feature vectors is used to calculate the similarity of the trajectory points, and the trajectory is associated by setting a similarity threshold. It is understandable that other initial trajectory generation and similarity calculation methods can also be used, which are not limited here.

[0044] S103 , performing outlier identification and trajectory optimization on the associated trajectory pair based on a similarity energy function to obtain an optimized ship trajectory, wherein the similarity energy function includes a dynamics option, a persistence option, a fusion interval option, a color similarity option, and an adjustment option.

[0045] The similarity energy function represents the objective function used to evaluate trajectory optimization effectiveness. Outlier identification refers to the detection and removal of anomalous track points in a trajectory. Trajectory optimization involves smoothing and completing the trajectory. The optimized ship trajectory represents the high-quality trajectory obtained after outlier removal and optimization.

[0046] Specifically, this step is performed after obtaining associated trajectory pairs and is used to optimize trajectory quality. First, a similarity energy function is constructed, which includes dynamics, persistence, fusion interval, color similarity, and adjustment options. The dynamics option describes the changes in the track's motion state, the persistence option describes the continuity of the trajectory, the fusion interval option describes the interval between trajectory segments, the color similarity option describes the changes in the color characteristics of the trajectory points, and the adjustment option is used to control the number of trajectories. This energy function is then used to identify and optimize the trajectories, including removing outliers that do not conform to the motion rules, completing missing trajectory segments, and smoothing the trajectory curves.

[0047] In some embodiments, trajectory optimization can be achieved in a variety of ways: optionally, using a sliding window method to calculate the motion characteristics of local trajectory segments, identifying abnormal track points based on Mahalanobis distance; using cubic spline interpolation to complete missing trajectory segments; using Kalman filtering to smooth the trajectory; and finally obtaining the optimized trajectory by minimizing the energy function. Optionally, a trajectory anomaly detector is established based on a Bayesian probability model, combined with a multivariate statistical analysis method to identify abnormal points; using a dynamic programming algorithm to optimize the trajectory in segments, and using the least squares method to fit the trajectory curve within each segment; and adjusting the trajectory shape by iteratively optimizing the energy function. It is understandable that other trajectory optimization methods can also be used, which are not limited here.

[0048] S104: Perform similarity testing by calculating the state differences between the optimized ship trajectories, and determine the optimized ship trajectories with similarities exceeding a preset threshold as the same target and associate them to form a multi-source trajectory group.

[0049] State difference represents the degree of difference in motion state parameters between optimized ship trajectories. Similarity testing evaluates the similarity between trajectories by calculating trajectory state differences. A preset threshold represents the similarity criterion for determining trajectories as belonging to the same target. A multi-source trajectory group represents a collection of optimized trajectories from different sensors that are determined to be the same ship target.

[0050] Specifically, this step, performed after obtaining optimized ship trajectories, determines whether trajectories from different sensors correspond to the same vessel target. First, the state differences between the optimized trajectories are calculated, including differences in motion parameters such as position, velocity, and heading. A likelihood ratio statistic is then constructed based on these state differences to perform a similarity test, and the degree of similarity between the trajectories is evaluated using the Bayesian criterion. When the similarity exceeds a preset threshold, the trajectories are determined to be from the same vessel target and are linked together to form a multi-source trajectory group, laying the foundation for subsequent trajectory fusion.

[0051] In some embodiments, trajectory similarity testing and association can be achieved through a variety of methods: Optionally, first extract the trajectory's feature vectors, including average speed, steering angle, and distance traveled; then calculate the Mahalanobis distance of the feature vectors as a measure of state difference; finally, use hypothesis testing to determine the significance of the differences, and identify trajectories with a significance level below a threshold as the same target and associate them. Optionally, construct a trajectory similarity model based on joint probabilistic data association; calculate the state transition probability matrix between trajectory points; determine trajectory association based on a sequential probability ratio test, and dynamically update the association results using a sliding time window. It is understood that other trajectory similarity testing and association methods can also be used, and are not limited here.

[0052] S105: Calculate a dynamic fusion weight based on the standard deviation of the measurement error of each optimized ship trajectory in the multi-source trajectory group relative to its average trajectory, and perform weighted fusion to obtain the ship target trajectory.

[0053] The average trajectory represents the average state sequence of all trajectories in the multi-source trajectory group. The measurement error standard deviation indicates the degree of deviation of a single trajectory from the average trajectory. The dynamic fusion weight refers to the trajectory fusion coefficient that is dynamically adjusted based on the measurement error. The ship target trajectory represents the final trajectory obtained after weighted fusion of the multi-source trajectories.

[0054] Specifically, this step is performed after a multi-source trajectory group is formed and is used to fuse multiple trajectories into a single, high-precision trajectory. First, the average value of each trajectory point in the multi-source trajectory group is calculated to obtain the average trajectory, which serves as a reference. The standard deviation of the measurement error for each trajectory's position, heading, speed, and other state variables relative to the average trajectory is then calculated to assess the trajectory's measurement accuracy. Based on the measurement error standard deviation, the minimum variance criterion is used to calculate the dynamic fusion weights for each trajectory, giving trajectories with high measurement accuracy a greater weight. Finally, each trajectory is weighted averaged according to its corresponding weight to obtain the fused ship target trajectory.

[0055] In some embodiments, trajectory fusion can be achieved in a variety of ways: optionally, first time-aligning and spatially normalizing the multi-source trajectories; then calculating the root mean square error of the trajectories within a sliding time window as a measurement accuracy indicator; calculating the normalized fusion weights based on the inverse of the measurement accuracy; and finally, using a weighted Kalman filter for trajectory state estimation and fusion. Optionally, a trajectory accuracy assessment model based on covariance cross-validation is established; the fusion weights are dynamically adjusted using an adaptive weighting algorithm; and the fused trajectory is optimized by combining trajectory prediction and smoothing techniques to improve the smoothness and continuity of the trajectory. It is understood that other trajectory fusion methods can also be used and are not limited here.

[0056] The following is a more detailed description of the process of the method provided by this implementation. Figure 2 , is another flow chart of the ship target identification and tracking method based on multi-source data fusion in an embodiment of the present application.

[0057] S201. Collect ship monitoring data from multiple source sensors on the same sea area, where the ship monitoring data includes ship position coordinates, area characteristics, and color characteristics.

[0058] Multi-source sensors represent multiple different types of monitoring methods capable of acquiring vessel information in the same ocean, such as radar, Automatic Identification System (AIS), and optoelectronic equipment. Each of these sensors possesses unique monitoring principles and capabilities, enabling them to acquire vessel-related information from different perspectives. Ship monitoring data, a collection of characteristic information about vessel targets collected by sensors, is crucial for ship monitoring. Specifically, ship position coordinates accurately represent a vessel's position in a two-dimensional plane, typically defined by longitude and latitude. Area features refer to the area occupied by a vessel target in an image. Color features describe the apparent color characteristics of a vessel target through a color histogram.

[0059] When the ship target identification and tracking system is operational, it activates multiple sensors to simultaneously monitor ships in the same sea area. For example, radar transmits electromagnetic waves and receives reflected waves, then processes the signals to accurately determine a ship's position. The AIS system, on the other hand, receives messages from ships containing information such as their location and parses them to determine their position. To determine area features, if optoelectronic equipment is used to capture ship images, image segmentation techniques can be used to separate the ship target from the background and calculate its area within the image. Similarly, color features are analyzed based on the images captured by optoelectronic equipment. The distribution of different colors within the area is statistically analyzed, and a color histogram is constructed to describe the color features. The system integrates and stores the ship's position coordinates, area features, and color features acquired from these sensors to form a complete ship monitoring dataset, providing a critical data foundation for subsequent ship trajectory generation and analysis.

[0060] S202: When no detection point associated with the target detection point is found in the previous frame image, a new trajectory segment is generated with the detection point as the trajectory starting point.

[0061] Target detection points are feature points detected in an image that are associated with a ship and represent the ship's state at a specific moment. The previous image is the image immediately preceding the current image in the temporal sequence and is used for comparative analysis with the current image to determine the ship's trajectory. Associated detection points are detection points in different image frames that represent the same ship target and have similar characteristics and spatiotemporal continuity. The starting point is the detection point that serves as the starting position for a new trajectory segment. The new segment is the initial portion of a ship trajectory constructed based on this specific detection point.

[0062] When processing an image sequence, the system analyzes each detection point frame by frame. For each target detection point, the system compares it with all detection points in the previous frame according to preset rules. These rules are based on similarity calculations based on detection point features such as position, velocity, area, and color. For example, thresholds for position distance, velocity change, area difference, and color histogram similarity can be set. Position similarity is calculated using the Euclidean distance formula to measure the positional difference between two detection points on a two-dimensional plane. Velocity similarity is calculated by referring to the position change and time interval between detection points in the previous and next frames to derive the velocity vector and calculate the velocity difference. Area similarity is measured by calculating the ratio of the areas of the ship represented by the two detection points. Color similarity is calculated using a color histogram similarity algorithm. If these similarity calculations fail to find a matching detection point in the previous frame, meaning that all similarity metrics between all detection points and the current target detection point do not meet the preset thresholds, the system determines that target detection point as the starting point of a new trajectory segment. This is likely because the detection point represents a newly appeared ship target, or it was not correctly detected and associated in the previous frame due to a previous detection error. The system then uses this as a starting point to record subsequent related detection points, thereby generating a new ship track segment, laying the foundation for subsequent accurate tracking of the ship target.

[0063] S203 : When no detection point associated with the target detection point is found in the next frame image, the target detection point is used as the end point of the current trajectory segment.

[0064] The subsequent image is the image immediately following the currently processed image in chronological order and is used in conjunction with the current image to analyze the subsequent state of the vessel target. The current trajectory segment is a segment of the vessel trajectory that is being constructed or partially constructed, recording a portion of the vessel's motion trajectory over a period of time. The endpoint is the point where the current trajectory segment ends. This means that there are no subsequent detection points associated with this detection point, marking the end of this trajectory segment.

[0065] When the system analyzes a target detection point during ship trajectory construction, it searches for related detection points in the subsequent image frame. The association determination is based on similarity calculations based on multiple features, including position, velocity, area, and color. For example, when calculating velocity similarity between the current detection point and each detection point in the subsequent frame, the system calculates the absolute value of the velocity vector difference based on the position change and time interval between the detection points in the previous and subsequent frames. If this value exceeds a preset velocity difference threshold, the velocity similarity requirement is not met and the two are considered unrelated. Area similarity is calculated by calculating the relative difference between the areas of the ship represented by the two detection points. If this value exceeds a preset area difference threshold, the two are considered unrelated. For color similarity, algorithms such as the Bhattacharyya coefficient of the color histogram are used to calculate similarity. If the value falls below a preset color similarity threshold, the system determines that the target detection point is unrelated. If, after comprehensive similarity calculations and judgments, no matching detection point is found in the subsequent image frame, the system sets the target detection point as the endpoint of the currently constructed trajectory segment. This may be due to reasons such as the ship leaving the monitoring area or being blocked by other objects, resulting in the inability to detect the ship target in subsequent images, or an error in the detection process, making it impossible to find the associated detection point in subsequent images, thus ending the recording of the current trajectory segment.

[0066] S204: Determine whether the number of frames in which the track appears continuously is less than a preset threshold. If so, determine that the track is a track fragment.

[0067] A trajectory is formed by a series of detection points connected in chronological order and is used to characterize the motion path of a ship over a period of time. The number of consecutive frames refers to the number of frames in the image sequence in which the detection points contained in the trajectory appear in sequence without interruption, which reflects the continuity of the trajectory in the time dimension. The preset threshold is a fixed frame value set before the system is operated, which serves as a standard for judging whether the trajectory is complete or reliable. Track fragments refer to those parts of the trajectory whose consecutive frame number does not reach the preset threshold. Such trajectories may be caused by detection errors, temporary occlusion of the target, etc., and their integrity and reliability are relatively low.

[0068] As the system processes ship monitoring data to generate initial ship tracks, it analyzes each track. The system examines the presence of detection points on each track, sequentially following the image's chronological order. For each processed frame, if a detection point belonging to a track is detected, the track's consecutive frame count counter is incremented by 1; if no detection point is detected, the counter is reset to 0. For example, if the preset threshold is 5 frames, and the system processes a track, the detection point is detected from frames 1 to 3, but not in frame 4. In this case, the track's consecutive frame count is 3, which is less than the preset threshold of 5. The system compares the calculated consecutive frame count with the preset threshold. If it is less than the threshold, the track is classified as a track fragment. This determination is crucial for ensuring the quality of the initial ship track, as track fragments can interfere with subsequent track analysis and processing. By identifying and marking these track fragments, the system can handle them specifically, improving the accuracy and reliability of the entire ship target recognition and tracking system.

[0069] S205 , calculating the energy function value of the trajectory fragment after being fused with other trajectory segments. When the energy function value decreases, the trajectory fragment is retained; otherwise, it is removed from the trajectory space to obtain the initial ship trajectory.

[0070] Track fragments refer to the portion of a trajectory that was determined in step S204 to have too few consecutive frames. Other track segments are relatively complete or reliable track segments in the entire trajectory space, excluding the track fragment. The energy function value is a quantitative indicator used to evaluate the rationality of trajectory configuration by comprehensively considering multiple factors. It consists of multiple parts, such as dynamic options, persistence options, fusion interval options, color similarity options, and adjustment options. Its numerical value can intuitively reflect the overall quality and rationality of the trajectory. The trajectory space is a virtual space used by the system to store and manage all detected ship trajectory information, which includes various types of trajectories and their related attributes.

[0071] After identifying the track fragments, the system will attempt to fuse each track fragment with other track segments in the track space. Specifically, the system merges the track fragment with other track segments one by one, and then calculates the energy function value of the fusion according to the energy function formula. The energy function formula is:

[0072] After identifying the track fragments, the system will try to fuse each track fragment with other track segments in the track space. Specifically, the system merges the track fragment with other track segments one by one, and then calculates the energy function value of the fusion according to the energy function formula. The energy function formula is

[0073] Among them, the dynamic option E_{dyn}^{i}=α∑_{t=1}^{F}ξ‖v_{i}^{t}-v_{i}^{t+1}‖^2 is mainly used to constrain the range of change of the target's motion speed. When the speed changes too much, a larger penalty will be given to ensure the smoothness of the trajectory curve and prevent the target from undergoing identity transformation when crossing.

[0074] The persistence option E_{per}^{i}=β(d(Γ_{i}(e_{i}))+d(Γ_{i}(s_{i})))+χ(e_{i}-s_{i})^{-1} takes into account the rationality of the target appearing and disappearing in the image, prompting the trajectories to merge at the break points to form longer trajectories.

[0075] The fusion interval option E_{int}^{i}=λ∑_{k}M_{k} avoids the fused trajectory interval being too large by penalizing the situation where the fused trajectory does not pass through the detection point for a long time.

[0076] The color similarity option E_{col}^{i}=ψ(1-P(Γ_{i}(e_{i}), Γ_{j}(s_{j}))) ensures that the similarity of color features between each detection point in the trajectory segment varies little, helping to solve the identity conversion problem when multiple moving targets are close to each other.

[0077] The adjustment option E_{reg}^{i}=ϑN is used to optimize the existing trajectory space when generating the final trajectory so that the number of generated trajectories is as small as possible.

[0078] If the energy function value after fusion decreases compared to before fusion, it indicates that the fused trajectory is more reasonable and more consistent with the actual situation. The system will retain the trajectory fragment. Conversely, if the energy function value does not decrease, the system will remove the trajectory fragment from the trajectory space. After this processing of all trajectory fragments, the system can obtain a relatively accurate and reliable initial ship trajectory, providing a higher-quality data foundation for subsequent trajectory analysis, correlation, and fusion operations.

[0079] S206: Calculate the speed similarity of adjacent detection points based on the position information of the detection points in the initial ship trajectory.

[0080] The initial ship trajectory is generated by the system based on ship monitoring data. It consists of a series of detection points that record the ship's position and other related information at different times. A detection point identifies the ship's position in the image, and each detection point contains characteristic information such as the ship's position coordinates at that moment. Adjacent detection points are defined as two detection points that are closely connected in time within the initial ship trajectory. Velocity similarity measures the degree of similarity between the ship's motion speeds represented by these two adjacent detection points and is a key factor in determining whether adjacent detection points belong to the same trajectory segment.

[0081] After obtaining the initial ship trajectory, the system calculates the speed similarity for each pair of adjacent detection points on the trajectory in order to further analyze and process the trajectory data. For the adjacent detection points Di(t) and Dj(t+1) on the initial ship trajectory, the speed similarity calculation formula is:

[0082] In this formula, (vxi, vyi) and (vxj, vyj) represent the speeds of the two detection points in the X-axis and Y-axis directions, respectively. The speed is calculated by dividing the position change of the detection point in two adjacent frames by the time interval. For example, if the position vector of the detection point in frame t is pi(t), and the position vector in frame t+1 is pi(t+1), then the speed vector vi(t) of the detection point at time t = pi(t+1)-pi(t). σvx² and σvy² represent the variance of the speed in the two directions in the current trajectory segment. Taking the calculation of the speed variance in the X-axis direction as an example, it is first necessary to find the difference between the speed of each detection point in the trajectory segment in the X-axis direction and the average speed, square these differences, sum them, and then divide them by the number of detection points to obtain σvx². The calculation method of the speed variance in the Y-axis direction is the same. For example, in an initial ship trajectory, there are adjacent detection points A and B. A's velocity in the X-axis direction is vxA and its velocity in the Y-axis direction is vyA. B's velocity in the X-axis direction is vxB and its velocity in the Y-axis direction is vyB. The velocity variance of this trajectory segment in the X-axis direction is σvx², and the velocity variance in the Y-axis direction is σvy². Substituting these values into the velocity similarity calculation formula, we can determine the velocity similarity of adjacent detection points A and B.

[0083] S207 , calculating the area similarity of adjacent detection points based on the rectangular frame of the detection points in the initial ship trajectory.

[0084] When processing the initial ship trajectory, the system uses the rectangular box information of each pair of adjacent detection points on the trajectory to calculate area similarity. First, the system obtains the rectangular boxes corresponding to the adjacent detection points. For each adjacent detection point Di(t), the system calculates its area by multiplying the length and width of the rectangular box, denoted as si and sj, respectively.

[0085] The area similarity calculation formula is:

[0086] Among them, σs² represents the variance of the rectangular area of all detection points in the current trajectory segment. When calculating the variance, the system first calculates the average value s̄ of the rectangular area of all detection points in the trajectory segment. For example, there are 5 detection points in an initial ship trajectory segment, and their rectangular area are s1, s2, s3, s4, and s5 respectively, then s̄=(s1+s2+s3+s4+s5) / 5. Next, the square of the difference between the rectangular area of each detection point and the average value is calculated, that is, (s1-s̄)², (s2-s̄)², (s3-s̄)², (s4-s̄)², (s5-s̄)², these square values are summed and then divided by the number of detection points 5 to obtain σs². Substituting the area values corresponding to Di(t) and Dj(t+1) and the calculated σs² into the formula, the area similarity of the two adjacent detection points can be obtained. The system performs this calculation for each pair of adjacent detection points on the initial ship trajectory, generating a series of area similarity data. This data can be used to subsequently determine the correlation between adjacent detection points and support accurate ship trajectory construction.

[0087] S208: Calculate the color similarity of adjacent detection points based on the color features of the detection points in the initial ship trajectory.

[0088] When calculating the color similarity of adjacent detection points, the system uses color histogram to quantify the color characteristics of the detection points. For the adjacent detection points Di(t) and Dj(t+1) in the initial ship trajectory, the color similarity calculation formula is:

[0089] Here, N represents the number of levels in the color histogram, which determines the level of detail in describing color features. For example, if N = 32, the color space is divided into 32 levels to describe color features. q(d1)n and p(d2)n represent the proportion of the nth level in the color histograms of the two detection points, respectively. The system first constructs a color histogram for each detection point, statistically analyzing the distribution of different colors within the ship's target area, and then determining the proportion of each color level in the histogram. For example, for detection point Di(t), the proportion of the first level color in its color histogram is statistically determined to be q(d1)1, the proportion of the second level color is q(d1)2, and so on. For detection point Dj(t+1), there are also corresponding color proportions of each level, p(d2)n. The corresponding proportions of each level are then substituted into the above formula for calculation. For example, if the similarity contribution for the first color level is calculated as sqrt(q(d1)1p(d2)1), then the similarity contributions for all 32 color levels are summed, i.e., summing sqrt(q(d1)np(d2)n) for n=1 to 32 pairs. This gives the color similarity of the two adjacent detection points. The system calculates color similarity using this method for each pair of adjacent detection points on the initial ship trajectory. The resulting color similarity data helps to more accurately determine the relationship between adjacent detection points, further improving the accuracy of ship trajectory construction.

[0090] S209 , associating adjacent detection points based on a weighted combination of the speed similarity, the area similarity, and the color similarity to obtain an associated trajectory pair, where the detection point similarity includes speed similarity, area similarity, and color similarity.

[0091] After obtaining the speed, area, and color similarity data for adjacent detection points in the initial ship trajectory, the system performs a weighted combination to determine whether adjacent detection points belong to the same trajectory, thereby generating associated trajectory pairs. Assuming speed similarity is Avel, area similarity is Aare, and color similarity is Acol, they are assigned weights w1, w2, and w3, respectively (w1 + w2 + w3 = 1. The weight setting can be determined based on actual conditions and experience. For example, in complex marine environments where ship speeds vary significantly, speed similarity has a greater impact on trajectory accuracy; w1 can be set to 0.5. If the ship area features in the image are less disturbed, area similarity is more reliable, so w2 can be set to 0.3. Color similarity has a relatively small impact, so w3 can be set to 0.2). The comprehensive similarity is calculated as P(Di(t), Dj(t+1)) = w1Avel + w2Aare + w3*Acol). The system calculates this comprehensive similarity metric for each pair of adjacent detection points on the initial ship trajectory. For example, for a pair of adjacent detection points, the calculated velocity similarity Avel = 0.7, area similarity Aare = 0.6, and color similarity Acol = 0.5. Based on the aforementioned weighting, the overall similarity P(Di(t), Dj(t+1)) = 0.5 × 0.7 + 0.3 × 0.6 + 0.2 × 0.5 = 0.63. The system compares this overall similarity with a preset threshold (e.g., 0.6). If the overall similarity exceeds the threshold, the pair of adjacent detection points is considered to belong to the same trajectory and recorded as a correlated trajectory pair. If the overall similarity is less than the threshold, the pair is considered to belong to different trajectories. By performing this calculation and judgment on all adjacent detection points on the initial ship trajectory, the system generates a series of correlated trajectory pairs, which lay the foundation for subsequent generation of more accurate ship trajectories.

[0092] S210: Calculate the similarity of adjacent track points in the initial ship track using a similarity energy function, where the similarity energy function includes a dynamics option, a persistence option, a fusion interval option, a color similarity option, and an adjustment option.

[0093] When processing the initial ship trajectory, the system uses the similarity energy function to calculate the similarity of each pair of adjacent trajectory points in the trajectory. The formula of the similarity energy function is:

[0094] The dynamics option Edyn(i)=α∑(t=1 to F)ξ||vi(t)-vi(t+1)||², where vi(t) represents the velocity vector of target i in frame t, and ξ is adjusted based on the magnitude of the velocity change (ξ=10 when ||vi(t)-vi(t+1)||²>5; otherwise, ξ=1). α is a constant. This option penalizes trajectories with sudden velocity changes, resulting in smoother trajectories. For example, if the velocity of adjacent trajectory points varies significantly between two frames, resulting in ||vi(t)-vi(t+1)||²=8, then the value of this dynamics option is relatively large, significantly affecting the overall similarity energy function. The persistence option Eper(i)=β(d(Γi(ei))+d(Γi(si)))+χ(ei-si)^(-1), where d(Γi(ei)) and d(Γi(si)) represent the distance from the start or end node of a track to the edge of the image, respectively, and ei and si represent the frames of the end and start of the track, respectively. This option encourages tracks to merge at breakpoints and generate longer tracks. The fusion interval option Eint(i)=λ∑Mk, where Mk represents the total number of frames in the fused track that do not pass through any detection points for a long period of time, and λ is a constant. This penalizes tracks that have no detection points for a long period of time to ensure track validity. The color similarity option Ecol(i)=ψ(1-P(Γi(ei),Γj(sj))), where P(Γi(ei),Γj(sj)) represents the color histogram similarity between the end point of track Γi and the start point of track Γj, and ψ is a constant. This option, combined with the dynamics option, addresses the identity conversion problem when multiple targets are close together. The adjustment option Ereg(i) = ϑN, where N is the total number of trajectory segments in the current trajectory space and ϑ is a constant used to adjust the final number of generated trajectories. The system sums the results of these options to obtain the value Etal, which represents the similarity between adjacent trajectory points. By performing this calculation for all adjacent trajectory points in the initial ship trajectory, the system can assess trajectory quality, providing a basis for subsequent outlier identification and trajectory optimization.

[0095] S211. Based on the speed drift coefficient and the disturbance variance factor, a linear model is constructed to calculate the mutation probability of the trajectory point, a sliding time window is set, and the speed drift coefficient and the disturbance variance factor of the trajectory point within the sliding time window are calculated.

[0096] The system first sets a sliding time window. The window length can be set based on actual conditions, for example, l. When processing the initial ship trajectory, the system moves this sliding time window along the trajectory in chronological order. For the trajectory data within the window, the system calculates the velocity drift coefficient and the perturbation variance factor. Assuming that the ship travels along a great circle over a period of time, the ship's position at adjacent moments can be used to estimate the ship's speed during that time interval (denoted as the estimated position speed). The speed during that time interval can be estimated from the ship's actual speed (denoted as the actual speed). A linear model is established within the window. Assuming the estimated position speed is v̂ and the actual speed is v, the model is v̂ = β0 + β1v + ε, where ε is the model error and follows a normal distribution. Il×l is the l×l identity matrix, indicating that the model errors at each point are independent. The coefficient β0 is the velocity drift coefficient, which contains information about velocity drift affected by factors such as local ocean currents and sea surface wind speed. The variance factor τ^(-1) describes the perturbation of the data due to positioning errors, which is also known as the perturbation variance factor. For example, in a certain trajectory, the sliding time window contains five trajectory points. By processing the position and velocity data of these five points and using fitting methods such as least squares, the velocity drift coefficient β0 and the disturbance variance factor τ^(-1) can be obtained. By continuously moving the sliding time window, the system calculates the velocity drift coefficient and disturbance variance factor within each window, providing data support for the subsequent construction of a linear model to calculate the mutation probability of the trajectory points, thereby identifying anomalies in the trajectory and improving the accuracy and reliability of the trajectory.

[0097] S212: constructing a linear model of the trajectory point position based on the velocity drift coefficient and the disturbance variance factor, and calculating the mutation probability of the trajectory point according to the deviation between the trajectory point and the position predicted by the linear model.

[0098] The system first constructs a linear model for the trajectory point's position based on the previously calculated velocity drift coefficient and disturbance variance factor. Assume that in a two-dimensional plane, the trajectory point's position at time t is (xt, yt), the velocity drift coefficients are β0x and β0y in the x and y directions, respectively, the actual velocity components of the ship in the x and y directions are vx and vy, and the disturbance variance factors are τx^(-1) and τy^(-1) in the x and y directions, respectively. The linear model can be expressed as: xt+1=xt+β0x+vx∆t+εx, yt+1=yt+β0y+vy∆t+εy; where ∆t is the time interval, and εx and εy are error terms, which follow a normal distribution with mean 0 and variances τx^(-1) and τy^(-1), respectively.

[0099] Based on this linear model, the system predicts the position of each trajectory point at the next moment, obtaining the predicted position (xpred, ypred). The system then calculates the deviation between the actual position (xactual, yactual) of the trajectory point and the predicted position. This deviation can be calculated using the Euclidean distance, i.e., d = √((xactual - xpred)² + (yactual - ypred)²).

[0100] Next, the system calculates the probability of a mutation at a trajectory point based on the deviation. Assume that the deviation d follows a normal distribution N(0, σ²), where σ² is related to the perturbation variance factor. The mutation probability can be calculated by calculating the probability that the deviation d exceeds a certain threshold. For example, if a threshold T is set, the mutation probability P can be expressed as P = 1 - Φ((Td) / σ), where Φ is the cumulative distribution function of the standard normal distribution.

[0101] The system performs this calculation for each point in the initial ship trajectory, comparing the mutation probability with a preset threshold to determine whether the point is an outlier. If the mutation probability is greater than the threshold, the point is considered to have undergone a mutation, possibly due to measurement error, a sudden turn of the ship, or other factors. If the probability is less than the threshold, the point is considered to be in normal motion. In this way, the system can identify outliers in the trajectory, providing a basis for subsequent trajectory correction and optimization.

[0102] S213 , removing the abnormal points whose mutation probability exceeds a preset probability threshold, and obtaining the trajectory segments after removing the abnormal points.

[0103] After calculating the mutation probability for each point in the initial ship trajectory, the system compares it with a preset probability threshold. The preset probability threshold is set based on actual conditions and experience, for example, 0.7. For each point in the trajectory, the system accurately obtains its mutation probability value. For example, suppose a point A in a ship trajectory has a mutation probability of 0.8. Since 0.8 exceeds the preset probability threshold of 0.7, the system identifies point A as an outlier. The system performs this comparison on all points in the entire trajectory. Once a point is identified as an outlier, it is removed from the trajectory. For example, if a comparison reveals that the mutation probability of five points in a trajectory containing 100 points exceeds the preset probability threshold, the system removes these five outliers from the trajectory. After this removal process, the remaining points constitute the trajectory segment after the outliers have been removed. This process helps improve the quality of the trajectory, making subsequent analysis and processing based on the trajectory more accurate and reliable.

[0104] S214. A moving average method is used to supplement the missing parts in the trajectory segments after the outliers are removed. When the time interval between the trajectory segments exceeds a preset interval threshold, a bidirectional search method is used to supplement the missing parts to obtain the optimized ship trajectory. The similarity energy function includes a dynamics option, a persistence option, a fusion interval option, a color similarity option, and an adjustment option.

[0105] The system first examines the trajectory segments after removing outliers to identify any missing segments. For these missing segments, the system uses a moving average method to fill in the gaps. Specifically, the system sets a time window for each missing point, for example, a window size of five trajectory points. For a missing point, the system obtains the location information of two known trajectory points before and after it (a total of four). Assuming the coordinates of these four trajectory points are (x1, y1), (x2, y2), (x3, y3), and (x4, y4), the missing point's supplementary coordinates are ((x1+x2+x3+x4) / 4, (y1+y2+y3+y4) / 4). The system fills in all missing points in this manner. During the filling process, the system checks the time interval between trajectory segments. The system pre-sets a threshold, such as 3 minutes. When the system detects that the time interval between two trajectory segments exceeds this threshold, it uses a bidirectional search method to fill in the gaps. The system starts at the endpoints of the two trajectory segments and searches toward the center. During the search process, the system combines a similarity energy function (including dynamics, persistence, fusion interval, color similarity, and adjustment options) to evaluate potential supplementary track points. For example, the dynamics option constrains the range of target velocity, and the system uses this option to find supplementary track points with reasonable velocity variations. Through a bidirectional search, the system finds suitable track points to connect the two track segments. Using the moving average method and bidirectional search, the system generates an optimized ship trajectory that is more complete and accurate, better reflecting the ship's actual navigation conditions.

[0106] S215: Calculate the state estimation difference of the optimized ship trajectory from different sensors, where the state estimation difference includes position, heading, and speed information.

[0107] To optimize the ship's trajectory, the system processes the state information obtained by different sensors and calculates the difference in state estimates between them. For position information, assume there are two sensors A and B. At a certain time t, sensor A measures the ship's position coordinates as (xA, yA), and sensor B measures the position coordinates as (xB, yB). The system calculates the position state difference by calculating the distance between the two points using the formula dpos = √((xA - xB)² + (yA - yB)²). The system performs this calculation at each moment in the optimized ship's trajectory, resulting in a sequence of position state difference estimates for the entire trajectory. For heading information, at the same time t, sensor A measures the ship's heading as θA, and sensor B measures the heading as θB. The system calculates the angular difference between these two values. To avoid circular angle calculations, the system uses the formula dheading = min(|θA - θB|, 360 - |θA - θB|) to obtain the heading state difference. The system calculates the heading information at each moment, resulting in a sequence of heading state difference estimates. For speed information, at time t, sensor A measures the ship's speed as vA, and sensor B measures the ship's speed as vB. The system directly calculates their difference, dspeed = |vA - vB|, and performs this calculation at every moment along the entire optimized ship trajectory to obtain a sequence of speed state estimate differences. By performing this calculation on the position, heading, and speed information of different sensors at each moment along the optimized ship trajectory, the system obtains a complete state estimate difference. These state estimate differences can be used to evaluate the measurement accuracy and reliability of different sensors, providing an important basis for subsequent sensor fusion and data processing.

[0108] S216 , performing a likelihood ratio test based on the state estimation difference through a joint probability density function based on an uncertain distribution to determine whether different sensor trajectories belong to the same target, thereby obtaining an associated multi-source trajectory group.

[0109] After the system obtains the state estimation difference of different sensors for the optimized ship trajectory, it uses the joint probability density function based on the uncertain distribution to perform a likelihood ratio test. Suppose the state estimation difference of the sensor for targets i and j at time k is tijk(k)={tij(l)}; l=1, 2, ⋯, k, where i and j represent the trajectories of different sensors, and tij(l) is the state estimation difference at time l. Suppose H0 and H1 are two events, H0 indicates that Xi1(l|l) and Xj2(l|l) are the track data of the same target, and H1 indicates that they do not belong to the track data of the same target. Its joint probability density function is , where tij0(0)=Xi1(0|0)-Xj2(0|0) is the prior information, which is obtained by weighted track association. The joint probability density can also be written as ,in , Pi1(l|l) is the estimated error covariance of sensor 1 for target i at time l, and Pj2(l|l) is the estimated error covariance of sensor 2 for target j at time l. Perform a likelihood ratio test and calculate , the corresponding log-likelihood ratio is . Define the test statistic . The system compares λij(k) with the significance level α. If λij(k) is less than α, the null hypothesis H0 is accepted, that is, the trajectories of different sensors are considered to belong to the same target; if λij(k) is greater than or equal to α, the alternative hypothesis H1 is accepted, that is, they are considered not to belong to the same target. For example, if the significance level α is set to 0.05, for the trajectories of two sensors, the calculated value is λij(k)=0.03, which is less than α, then the system determines that the trajectories corresponding to the two sensors belong to the same target. The system performs such a likelihood ratio test on all pairs of trajectories from different sensors, and combines the trajectories belonging to the same target to obtain the associated multi-source trajectory group, providing an accurate data basis for subsequent trajectory fusion.

[0110] S217: Calculate the standard deviation of the measurement error of each optimized ship trajectory in the multi-source trajectory group relative to the average trajectory of the multi-source trajectory group.

[0111] After obtaining the associated multi-source trajectory group, the system begins calculating the standard deviation of the measurement error for each optimized ship trajectory relative to the average trajectory. First, the average trajectory of the multi-source trajectory group is calculated. For position information, assuming there are M trajectories in the multi-source trajectory group, and at a certain time t, the position coordinates of the i-th trajectory are (xi, yi), then the position coordinates of the average trajectory at that time are (x̄, ȳ) = (1 / M∑i=1Mxi, 1 / M∑i=1Myi). The calculation method for heading and speed information is similar, averaging the heading and speed of each trajectory at the same time. Next, for each optimized ship trajectory in the multi-source trajectory group, the measurement error relative to the average trajectory is calculated. Taking the position information as an example, at time t, the position coordinates of the j-th optimized ship trajectory are (xj, yj). The error relative to the average trajectory position is ∆xj=xj-x̄ and ∆yj=yj-ȳ. Then, the standard deviation of the measurement error is calculated. Assuming a multi-source trajectory group has N moments over a period of time, the standard deviation of position measurement error, σpos, is calculated as σpos = √(1 / N∑t = 1N(∆xj² + ∆yj²)). The standard deviations of heading and speed measurement errors, σang and σsp, are calculated similarly. For example, a multi-source trajectory group contains three optimized ship trajectories monitored for 10 moments. To calculate the standard deviation of the position measurement error for a particular trajectory, the error between the trajectory position and the average trajectory position is first calculated at each moment. The squares of these errors are then averaged, and the square root is taken to obtain σpos. This calculation yields the standard deviation of the measurement error for position, heading, and speed for each optimized ship trajectory in the multi-source trajectory group relative to the average trajectory. These standard deviations are then used to calculate the dynamic fusion weights.

[0112] S218. Calculate the dynamic fusion weight according to the measurement error standard deviation.

[0113] After obtaining the standard deviation of the measurement error in position, heading, and speed for each optimized ship trajectory in the multi-source trajectory group relative to the average trajectory, the system calculates the dynamic fusion weight based on these standard deviations. For position information, assuming there are M optimized ship trajectories in the multi-source trajectory group and the standard deviation of the position measurement error for the i-th trajectory is σpos_i, then its position fusion weight ωpos_i is calculated as ωpos_i=(1 / (σpos_i)²) / (∑j=1M1 / (σpos_j)²). For example, in a multi-source trajectory group, there are three optimized ship trajectories with standard deviations of position measurement error, σpos_1=0.5, σpos_2=0.8, and σpos_3=1.0, respectively. First, calculate the denominator ∑j=131 / (σpos_j)²=1 / (0.5)²+1 / (0.8)²+1 / (1.0)²=4+1.5625+1=6.5625. Then calculate ωpos_1=1 / (0.5)² / 6.5625=4 / 6.5625≈0.61. Similarly, calculate ωpos_2 and ωpos_3. For heading information, the heading fusion weight ωang_i of the i-th track is calculated as ωang_i=(1 / (σang_i)²) / (∑j=1M1 / (σang_j)²), where σang_i is the standard deviation of the heading measurement error of the i-th track. For speed information, the speed fusion weight ωsp_i for the i-th trajectory is calculated as ωsp_i=(1 / (σsp_i)²) / (∑j=1M1 / (σsp_j)²), where σsp_i is the standard deviation of the speed measurement error for the i-th trajectory. This calculation method determines dynamic fusion weights for each optimized ship trajectory in the multi-source trajectory group in terms of position, heading, and speed. During the subsequent weighted fusion process, these weights are used to rationally allocate the contribution of each trajectory to the final target ship trajectory based on its accuracy and reliability, ensuring that the fused trajectory more accurately reflects the ship's true motion state.

[0114] S219: Perform weighted fusion on the multi-source trajectory group according to the dynamic fusion weight to obtain the ship target trajectory.

[0115] After obtaining the dynamic fusion weights for each optimized ship trajectory in the multi-source trajectory group, the system begins weighted fusion to obtain the target ship trajectory. Assuming there are M optimized ship trajectories in the multi-source trajectory group, at a certain time t, the target track point information for each trajectory includes the position Posi(t), heading Angi(t), and speed Spi(t), and the corresponding dynamic fusion weights are ωpos_i, ωang_i, and ωsp_i, respectively. For the position information, the position Pos(t) of the target ship trajectory at that time is calculated using the following formula: Pos(t)=∑i=1Mωpos_iPosi(t). For example, if there are three optimized ship trajectories in the multi-source trajectory group, at time t, their positions are Pos1(t)=(x1, y1), Pos2(t)=(x2, y2), and Pos3(t)=(x3, y3), and the corresponding position dynamic fusion weights are ωpos_1=0.4, ωpos_2=0.3, and ωpos_3=0.3, respectively. Then the position of the ship target trajectory at this time is Pos(t)=0.4×(x1, y1)+0.3×(x2, y2)+0.3×(x3, y3)=(0.4x1+0.3x2+0.3x3, 0.4y1+0.3y2+0.3y3). For the heading information, the heading Ang(t) of the ship target trajectory at this time is calculated as Ang(t)=∑i=1Mωang_iAngi(t). Similarly, for speed information, the speed of the ship's target trajectory at that moment is Sp(t)=∑i=1Mωsp_iSpi(t). Following the aforementioned method, the system performs a weighted fusion calculation on the position, heading, and speed information at each moment in the multi-source trajectory group. This calculation is performed for each moment, from the start to the end of the multi-source trajectory group, to obtain a complete series of ship target trajectory points, which, when connected, form the ship's target trajectory. This weighted fusion approach comprehensively considers the reliability of the trajectories of different sensors, allowing the resulting ship target trajectory to more accurately reflect the ship's actual motion state, providing more reliable data support for ship monitoring and tracking.

[0116] The following describes the ship target identification and tracking system in the embodiment of the present invention from the perspective of hardware processing. Figure 3 , is a schematic diagram of the physical device structure of the ship target identification and tracking system in an embodiment of the present application.

[0117] It should be noted that Figure 3 The structure of the ship target identification and tracking system shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0118] like Figure 3As shown, the ship target identification and tracking system includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes, such as the methods described in the above embodiments, based on programs stored in a read-only memory (ROM) 302 or programs loaded from a storage unit 308 into a random access memory (RAM) 303. RAM 303 also stores various programs and data required for system operation. CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to bus 304.

[0119] The following components are connected to the I / O interface 305: an input section 306 including an audio input device, push button switches, and the like; an output section 307 including a liquid crystal display (LCD), an audio output device, indicator lights, and the like; a storage section 308 including a hard disk and the like; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. Removable media 311, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 310 as needed, so that computer programs read from the removable media can be installed in the storage section 308 as needed.

[0120] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for executing the methods illustrated in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 309 and / or installed from removable media 311. When executed by the central processing unit (CPU) 301, the computer program performs the various functions defined in the present invention.

[0121] It should be noted that specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0122] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. Each box in the flowchart or block diagram can represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings.

[0123] Specifically, the ship target identification and tracking system of this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, the ship target identification and tracking method based on multi-source data fusion provided in the above embodiment is implemented.

[0124] As another aspect, the present invention further provides a computer-readable storage medium, which may be included in the ship target identification and tracking system described in the above embodiments, or may exist independently and not be incorporated into the ship target identification and tracking system. The storage medium carries one or more computer programs, which, when executed by a processor of the ship target identification and tracking system, enable the ship target identification and tracking system to implement the ship target identification and tracking method based on multi-source data fusion provided in the above embodiments.

[0125] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0126] As used in the above embodiments, the term “when” may be interpreted to mean “if” or “after” or “in response to determining that” or “in response to detecting that”, depending on the context. Similarly, the phrases “upon determining that” or “if (stated condition or event) is detected” may be interpreted to mean “if determining that” or “in response to determining that” or “upon detecting (stated condition or event)” or “in response to detecting (stated condition or event)”, depending on the context.

[0127] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A ship target recognition and tracking method based on multi-source data fusion, characterized in that: Applying a ship target identification and tracking system, the method includes: Collecting ship monitoring data from multiple sensors in the same sea area, wherein the ship monitoring data includes ship position coordinates, area characteristics, and color characteristics; The speed similarity of adjacent detection points is calculated based on the position information of the detection points in the initial ship trajectory. The speed similarity calculation formula is: ； Specifically, Di(t) and Dj(t+1) represent two detection points, (vxi, vyi) and (vxj, vyj) represent the velocities of the two detection points in the X-axis direction and the Y-axis direction respectively, and σvx² and σvy² represent the variance of the velocities in the two directions in the current trajectory segment; The area similarity of adjacent detection points is calculated based on the rectangular frame of the detection points in the initial ship trajectory. The area similarity calculation formula is: ； Specifically, Di(t) and Dj(t+1) represent two detection points, and σs² represents the variance of the rectangular area of all detection points in the current trajectory segment. The difference between the area of the rectangular box of each detection point and the average value; The color similarity of adjacent detection points is calculated based on the color features of the detection points in the initial ship trajectory. The color similarity calculation formula is: ； Specifically, Di(t) and Dj(t+1) represent two detection points, N represents the level of the color histogram, q(d1)n and p(d2)n represent the proportion of the nth level in the color histograms of the two detection points respectively; Associating adjacent detection points based on a weighted combination of the speed similarity, the area similarity, and the color similarity to obtain an associated trajectory pair, wherein the detection point similarity includes speed similarity, area similarity, and color similarity; Calculating similarities between adjacent track points in the initial ship track using a similarity energy function, wherein the similarity energy function includes a dynamics option, a persistence option, a fusion interval option, a color similarity option, and an adjustment option; A linear model is constructed based on the velocity drift coefficient and the disturbance variance factor to calculate the mutation probability of the trajectory points, and the trajectory points whose mutation probability exceeds the preset threshold are eliminated. The moving mean method is used to supplement the missing segments after removing the abnormal points to obtain the optimized ship trajectory. The similarity energy function includes a dynamics option, a persistence option, a fusion interval option, a color similarity option and an adjustment option. A similarity test is performed by calculating the state difference between the optimized ship trajectories, and the optimized ship trajectories with similarity exceeding a preset threshold are determined to be the same target and associated to form a multi-source trajectory group; The dynamic fusion weight is calculated according to the standard deviation of the measurement error of each optimized ship trajectory in the multi-source trajectory group relative to its average trajectory, and the ship target trajectory is obtained by performing weighted fusion.

2. The method according to claim 1, characterized in that The step of performing similarity testing by calculating the state differences between the optimized ship trajectories, determining the optimized ship trajectories with similarities exceeding a preset threshold as the same target and associating them to form a multi-source trajectory group specifically includes: Calculating a state estimation difference of the optimized ship trajectory by different sensors, wherein the state estimation difference includes position, heading, and speed information; A likelihood ratio test is performed based on the state estimation difference through a joint probability density function based on uncertain distribution to determine whether different sensor trajectories belong to the same target, thereby obtaining an associated multi-source trajectory group.

3. The method according to claim 2, characterized in that The step of calculating the dynamic fusion weight according to the standard deviation of the measurement error of each optimized ship trajectory in the multi-source trajectory group relative to its average trajectory, and performing weighted fusion to obtain the ship target trajectory specifically includes: Calculating a standard deviation of measurement errors of each optimized ship trajectory in the multi-source trajectory group relative to an average trajectory of the multi-source trajectory group; Calculating a dynamic fusion weight according to the measurement error standard deviation; The multi-source trajectory group is weightedly fused according to the dynamic fusion weight to obtain the ship target trajectory.

4. The method according to claim 1, wherein The step of generating an initial ship trajectory for each sensor based on the ship monitoring data specifically includes: When no detection point associated with the target detection point is found in the previous frame image, a new trajectory segment is generated with the detection point as the trajectory starting point; When no detection point associated with the target detection point is found in the next frame image, the target detection point is used as the end point of the current trajectory segment; Determine whether the number of frames in which the track appears continuously is less than a preset threshold. If so, it is determined to be a track fragment. An energy function value after the trajectory fragment is fused with other trajectory segments is calculated. When the energy function value decreases, the trajectory fragment is retained; otherwise, it is removed from the trajectory space to obtain the initial ship trajectory.

5. The method according to claim 1, wherein The step of eliminating trajectory points whose mutation probability exceeds a preset threshold and using a moving average method to supplement the missing segments after eliminating abnormal points to obtain an optimized ship trajectory specifically includes: Setting a sliding time window, and calculating the velocity drift coefficient and disturbance variance factor of the trajectory points within the sliding time window; Constructing a linear model of the trajectory point position based on the velocity drift coefficient and the disturbance variance factor, and calculating the mutation probability of the trajectory point according to the deviation between the trajectory point and the position predicted by the linear model; Eliminate the abnormal points whose mutation probability exceeds a preset probability threshold to obtain the trajectory segment after eliminating the abnormal points; The moving mean method is used to supplement the missing parts in the trajectory segments after the abnormal points are removed. When the time interval between the trajectory segments exceeds a preset interval threshold, a two-way search method is used to supplement them to obtain the optimized ship trajectory.

6. A ship target identification and tracking system, characterized in that: The ship target identification and tracking system includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the ship target identification and tracking system to execute the method according to any one of claims 1 to 5.

7. A computer-readable storage medium comprising instructions, characterized in that: When the instruction is executed on a ship target identification and tracking system, the ship target identification and tracking system is caused to execute the method according to any one of claims 1 to 5.

8. A computer program product, characterized in that When the computer program product is run on a ship target identification and tracking system, the ship target identification and tracking system is enabled to execute the method according to any one of claims 1 to 5.

Images