A method for multi-source target track association of inland ship navigation

Abstract

The application relates to a kind of inland river ship navigation multi-source target track correlation method, the method comprises the following steps: step 1: obtaining the AIS data, radar data and image data of target ship;Step 2: the AIS data, radar data and image data of the ship and target ship are carried out coordinate transformation and time calibration;Step 3: by comprehensive factor fuzzy evaluation or support vector machine two-classifier, the AIS data and radar data after coordinate transformation and time calibration are carried out track correlation, and AIS radar fusion data is obtained, then AIS radar fusion data is used for track correlation with the image data after coordinate transformation and time calibration, the target track of the data obtained by three sensors can be directly correlated, and the track correlation effect is good.

Description

Technical Field

[0001] This invention belongs to the field of intelligent ship technology, specifically relating to a method for associating multi-source target trajectories during inland waterway vessel navigation. Background Technology

[0002] Intelligent navigation is a fundamental function of intelligent ships, primarily addressing the complex relationship between the ship and its external environment during navigation. It typically includes situational awareness, path planning, and control. The ship's situational awareness system uses sensors such as radar, Automatic Identification System (AIS), and cameras to detect nearby vessels and obstacles, analyzes and predicts the trajectories of the ship and other nearby vessels, assesses collision risks, and provides input for path planning and control decisions. Among these, the correlation of multi-source target tracks for inland waterway vessels is particularly important.

[0003] Ship AIS and radar / video data are two commonly used ship monitoring technologies, acquiring ship position and motion information through radio waves and satellite signals, and photoelectric information, respectively. However, due to the characteristics and incompleteness of different data sources, correlating the data from these two sources is a challenge. Support Vector Machine (SVM) is a commonly used supervised learning algorithm. Originally proposed by Vladimir Vapnik in the 1960s, SVM has been continuously developed and improved, becoming one of the important algorithms in machine learning and widely used in tasks such as pattern recognition, classification, and regression. In correlating ship AIS, radar, and video track data, SVM can effectively handle high-dimensional and complex data, improving the accuracy and reliability of data correlation. By selecting an appropriate kernel function, SVM can also handle nonlinear data, expanding its application scope. However, in practical applications, it is still necessary to select appropriate parameters and kernel functions based on the specific problem to achieve better classification results.

[0004] The difficulties in associating multi-source target tracks of inland waterway vessels are: (1) The spatiotemporal coordinates of AIS, radar and camera data are different, which makes it impossible to directly associate the target tracks obtained by the three sensors; (2) The traffic flow of inland waterway vessels is dense, and there are many ship tracks involved in the association with similar motion trajectory shapes, resulting in a high error rate in track association. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides a method for associating multi-source target trajectories in inland waterway navigation. First, it performs spatiotemporal registration on ship target identification and tracking data acquired from three sensors: AIS, radar, and cameras, resolving the issue of coordinate consistency among data from multiple heterogeneous sensors. Then, considering the diverse traffic scenarios in inland waterways, it compares and analyzes the effectiveness of a comprehensive factor fuzzy evaluation method and a support vector machine classification method for associating AIS, radar, and video target trajectories, thus obtaining a method for associating multi-source target trajectories.

[0006] A method for associating multi-source target tracks during inland waterway vessel navigation; the technical solution of this invention is as follows:

[0007] The method includes the following steps:

[0008] Step 1: Acquire AIS data, radar data, and image data of the target vessel;

[0009] Step 2: Perform coordinate transformation and time calibration on the AIS data, radar data, and image data of both the ship and the target ship;

[0010] Step 3: Use a comprehensive factor fuzzy evaluation or a support vector machine binary classifier to perform track association between the AIS data and radar data after coordinate transformation and time calibration to obtain AIS radar fusion data. Then, use the AIS radar fusion data and the image data after coordinate transformation and time calibration to perform track association.

[0011] Furthermore, the coordinate transformation in step 2 is specifically as follows:

[0012] Step 21: Convert AIS coordinates to radar polar coordinates to obtain the target ship's latitude and longitude. Planar coordinates are obtained using the Mercator projection based on the ship's current latitude and longitude. The conversion formula is as follows:

[0013] ;

[0014] ;

[0015] ;

[0016] ;

[0017] in, The target ship coordinates in the AIS system. For radar angle, For radar detection range,

[0018] The range variance detected by radar is Angular variance is , Here, K represents the current latitude and longitude of the ship, K represents the spherical curvature between the ship and the target ship, and R represents the Earth's radius.

[0019] Step 22: Convert radar polar coordinates to camera image pixel coordinates. The conversion formula is as follows:

[0020] ;

[0021] ;

[0022] In the formula, u is the x-coordinate of the pixel coordinate system, v is the y-coordinate of the pixel coordinate system, u0 is the origin of the x-coordinate of the pixel coordinate system, v0 is the origin of the y-coordinate of the pixel coordinate system, and d x d represents the physical size of a single pixel on the x-axis. y This represents the physical size of a single pixel on the y-axis. Indicates the current radar angle;

[0023] Step 23: Convert AIS coordinates to camera image pixel coordinates, as follows:

[0024] The latitude and longitude coordinates of the ship's center point are calculated based on the latitude and longitude of the AIS data and the installation location of the GPS antenna.

[0025] Calculate the relative orientation angle θ and relative distance r between the AIS ship and the camera using the ship's center point and the camera's latitude and longitude.

[0026] Calculate the pixel coordinates of theoretically distortion-free camera images.

[0027] Furthermore, step 3, which involves fuzzy correlation of comprehensive factors, specifically involves:

[0028] Step 31: Determine the coarsely relevant target vessel based on the target vessel's AIS and radar data;

[0029] Step 32: Target ships that meet the coarse correlation criteria are then correlated with fine correlations using a comprehensive factor fuzzy evaluation algorithm;

[0030] Step 33: Perform quality management on the correlation between AIS data and radar data tracks of the target vessel to obtain AIS radar fusion targets;

[0031] Step 34: Associate the target data with the image data from the AIS radar fusion system.

[0032] Furthermore, step 31 is further specified as follows:

[0033] Locate the latitude and longitude points of the ship identified by radar. Find the AIS target subset in the current AIS data set that satisfies the following formula 1: Formula 1 is as follows:

[0034] ;

[0035] Among them, the AIS target latitude and longitude points are , For trajectory-related thresholds.

[0036] Furthermore, step 32 is further specified as follows:

[0037] Step 321: Determine the fuzzy factor set, selecting longitude, rate of change of longitude, and latitude as the fuzzy factor set. The calculation formula is as follows:

[0038] ;

[0039] Among them, the latitude and longitude points of the ship radar target are: Radar target speed The ship's AIS target latitude and longitude coordinates are: AIS target heading ;U r1 It is the difference in image longitude between a ship radar target and an AIS target that meets the coarse correlation condition, U r2 It is the difference in speed, U r3 It is the difference in latitude;

[0040] Step 322: Construct a membership function model based on the difference U between the image longitudes of the collected ship radar targets and AIS targets that satisfy the coarse correlation conditions. r1 The difference in speed U r2 The difference between latitude and latitude U r3 Construct a training set of samples and estimate the probability density based on the training set samples to obtain the membership function S. r1 Membership function S r2 Membership function S r3 The corresponding probability density function and distribution plot are obtained, and then the probability density function is normalized.

[0041] Step 323: Comprehensively evaluate the computational tightness, using the judgment matrix g. ri As follows:

[0042] ;

[0043] in, The coefficients are determined by the membership degrees calculated from the training set samples. To be based on the membership function Calculate the membership degree;

[0044] The formula for the coefficient relationship is as follows:

[0045] ;

[0046] ;

[0047] The consecutive judgment matrices g of a ship radar target and its corresponding single AIS target ri Take the average value j * Then, the density coefficient is taken as the maximum value among multiple different AIS targets at the same time, and is calculated as follows:

[0048] ;

[0049] Where g rij Let be the judgment matrix for the j-th AIS target;

[0050] ;

[0051] in, This serves as the threshold for relevant judgments.

[0052] Furthermore, step 33 specifically involves the following: the track association quality is as follows:

[0053] ;

[0054] ;

[0055] in, Δq represents the association quality between radar target i and AIS target j at time n; Δq is the change in association quality. , ,when Furthermore, when there is only one, the two tracks are fixedly correlated and no further correlation is performed. The radar target's ID is fixedly associated with the AIS target's Maritime Mobile Service Identifier (MMSI) number.

[0056] Furthermore, step 34 specifically involves: during the ship target tracking and recognition process, a ship target bounding box will be formed in the image. The coarse correlation is based on the pixel coordinates of the midpoint of the bottom edge of the ship image target bounding box. The coordinates of the ship are considered as identified from the video. Then, a target subset of AIS and radar data that meets the following conditions is searched within the current set of AIS and radar data:

[0057] ;

[0058] The coordinates of the AIS data, radar data, and target ship image after fusion and transformation are: ,T u and Tv The pixel threshold;

[0059] The pixel x-coordinate, pixel x-coordinate change rate, and pixel y-coordinate are selected as the fuzzy factor set, and their calculation formula is as follows:

[0060] ;

[0061] Among them, the target pixel positions of the ship image are (u i v i The optical center pixel of the image is (u0, v0); f h denoted by , where is the camera focal length; h is the camera height; φ is the angle of motion of the target in the ship image; rAi is the distance from the target in the ship image to the camera; dx is the physical size of a single pixel on the x-axis; dy is the physical size of a single pixel on the y-axis; U1 is the difference in pixel abscissa between the target in the ship image and the AIS radar fusion target that satisfies the coarse correlation condition; U2 is the difference in the rate of change of pixel abscissa; and U3 is the difference in pixel ordinate.

[0062] A training set of samples is constructed based on the collected U1, U2, and U3. The probability density is estimated based on the training set samples to obtain the probability density functions and distribution maps corresponding to the membership functions S1, S2, and S3. The probability density functions are then normalized.

[0063] The comprehensive evaluation uses a judgment matrix as follows: ,in, The coefficients are determined by the membership degrees calculated from the training set samples. To be based on the membership function Calculate membership degree;

[0064] The formula for the coefficient relationship is as follows:

[0065] ;

[0066] ;

[0067] The consecutive judgment matrices g of a ship radar target and its corresponding single AIS target ri Take the average value j * Then, the density coefficient is taken as the maximum value among multiple different AIS targets at the same time, and is calculated as follows:

[0068] ;

[0069] Where g rij Let be the judgment matrix for the j-th AIS target;

[0070] ;

[0071] in, For relevant judgment thresholds;

[0072] Perform track association quality management as follows:

[0073] ;

[0074] ;

[0075] in, , ,when Furthermore, when two tracks are uniquely correlated, no further correlation is performed, and the ID of the image target is fixedly associated with the MMSI number of the AIS target's maritime mobile communication service identifier.

[0076] Furthermore, the track association using a support vector machine binary classifier specifically includes the following steps:

[0077] Step 3.1: Correlate AIS data and radar data to form an AIS radar fusion target;

[0078] Step 3.2: Associate the AIS radar fused target with the target track in the image data.

[0079] Furthermore, step 3.1 is further specified as follows:

[0080] Step 3.11: Preprocess and extract features from the ship's motion data. Feature extraction includes extracting the length difference feature between the radar trajectory and the AIS trajectory, the heading difference feature between the radar trajectory and the AIS trajectory, the average acceleration difference feature, the starting position distance difference feature between the radar trajectory and the AIS trajectory, the ending position distance difference feature between the radar trajectory and the AIS trajectory, and the DTW similarity feature between the radar trajectory and the AIS trajectory.

[0081] Step 3.12: Preprocessing includes data cleaning, noise reduction, and handling of missing data. Features are extracted from AIS trajectory and radar trajectory data, and a sample set is constructed to train an SVM classifier, thereby achieving automatic classification and association of unlabeled data. The samples include positive and negative samples.

[0082] The construction of positive samples is as follows: First, AIS trajectory and radar trajectory data are manually labeled; track features are extracted from AIS trajectory and radar trajectory data; for AIS and radar trajectories labeled as the same target, their features are combined into a positive sample; the sample length is set to the time length of three AIS data points; for each constructed positive sample, label 1 is assigned to indicate that the current sample belongs to the same target vessel.

[0083] The negative samples are constructed as follows: Negative samples consist of AIS and radar trajectory features labeled with different targets. First, AIS and radar trajectory data that are not labeled with the same target within the same time period are selected. These AIS and radar trajectory data contain ship motion and labeling information. Track features are extracted from the AIS and radar trajectory data. For each AIS and radar trajectory labeled with a different target, their features are combined into a negative sample. The AIS and radar trajectories are then time-aligned. Each constructed negative sample is assigned a label of 0, indicating that the current sample belongs to a different target ship. Synthetic samples are generated using the SMOTE algorithm to increase the number of samples in the minority categories, thus constructing the dataset.

[0084] Step 3.13: Establish evaluation metrics, including accuracy, recall, and F1 score;

[0085] Step 3.14: Perform AIS and radar fusion target track correlation analysis.

[0086] Furthermore, step 3.2 is further specified as follows:

[0087] A dataset is constructed by fusing target trajectory data and image data using labeled AIS radar data, extracting corresponding track features, and then establishing positive and negative samples.

[0088] By extracting features from labeled AIS radar target trajectory data and image data and constructing a sample set, a support vector machine (SVM) classifier is trained.

[0089] Precision, recall, and F1 score were selected as evaluation metrics.

[0090] The performance of the SVM model in the task of associating ship AIS, radar fusion target and video target tracks is evaluated by evaluation metrics; finally, track association analysis of AIS radar fusion target and image data is performed.

[0091] Compared with the prior art, the present invention has the following beneficial effects:

[0092] This invention addresses the spatiotemporal inconsistency problem of multi-sensor data from AIS, radar, and cameras by establishing a spatiotemporal registration method for multi-source sensing ship track data through the transformation between AIS coordinates, radar polar coordinates, and camera image coordinates, as well as time calibration.

[0093] A target trajectory association method based on comprehensive factor fuzzy evaluation and support vector machine (SVM) is proposed for AIS, radar and camera. The effects of comprehensive factor fuzzy evaluation and support vector machine (SVM) algorithm on target trajectory association of multi-sensor perception in inland waterway navigation are compared and analyzed. The performance of the two algorithms in different scenarios is verified, providing a practical solution for trajectory association of inland waterway vessels with high accuracy. Attached Figure Description

[0094] Figure 1 This is a schematic diagram of a pinhole imaging model;

[0095] Figure 2 A schematic diagram of the information time series model

[0096] Figure 3 Membership function S r1 Distribution map;

[0097] Figure 4 Membership function S r2 Distribution map;

[0098] Figure 5 Membership function S r3 Distribution map;

[0099] Figure 6 The distribution graph of membership function S1;

[0100] Figure 7 The distribution graph of membership function S2;

[0101] Figure 8 The distribution graph of membership function S3;

[0102] Figure 9 This is a positive sample trajectory diagram;

[0103] Figure 10 For negative sample trajectory plots;

[0104] Figure 11 The experimental evaluation results for correlating AIS data and radar data tracks;

[0105] Figure 12 Experimental evaluation results for the correlation of AIS data, radar data, and image data tracks. Detailed Implementation

[0106] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0107] A method for associating multi-source target tracks during inland waterway vessel navigation, the method comprising the following steps:

[0108] Step 1: Acquire AIS data, radar data, and image data of the target vessel;

[0109] Step 2: Perform coordinate transformation and time calibration on the AIS data, radar data, and image data of both the ship and the target ship;

[0110] Step 3: Use a comprehensive factor fuzzy evaluation or a support vector machine binary classifier to perform track association between the AIS data and radar data after coordinate transformation and time calibration to obtain AIS radar fusion data. Then, use the AIS radar fusion data and the image data after coordinate transformation and time calibration to perform track association.

[0111] Multi-source data preprocessing aims to avoid excessive computational complexity during fusion and improve system performance. It mainly includes two aspects: coordinate transformation and time calibration. Common sensors used in inland waterway vessel navigation situation intelligent perception systems include AIS, radar, and cameras, requiring preprocessing of their data such as coordinate transformation and time calibration. Image data consists of camera images and videos.

[0112] The fusion of AIS, radar, and video images requires a unified position coordinate system. This paper unifies all coordinates to a geodetic coordinate system during data fusion, and displays the results in both the radar polar coordinate system and the camera coordinate system. This invention enables the conversion between the three coordinate systems, thereby unifying ship targets into a single coordinate system for track association.

[0113] The coordinate transformation in step 2 is as follows:

[0114] Step 21: The conversion between AIS coordinates and radar polar coordinates is essentially a conversion between the geodetic coordinate system and the ship's own coordinate system. The radar coordinate system is a polar coordinate system with the radar as its origin, and the measured data includes distance... and rotation angle AIS target location data is the AIS target latitude and longitude in a geodetic coordinate system. The conversion between AIS coordinates and radar polar coordinates transforms the target ship's latitude and longitude. Planar coordinates are obtained using the Mercator projection based on the ship's current latitude and longitude. The conversion formula is as follows:

[0115] ;

[0116] ;

[0117] ;

[0118] ;

[0119] in, The target ship coordinates in the AIS system. For radar angle, For radar detection range,

[0120] The range variance detected by radar is Angular variance is , Here, K represents the current latitude and longitude of the ship, K represents the spherical curvature between the ship and the target ship, and R represents the Earth's radius.

[0121] Step 22: Use a pinhole imaging model to transform pixel coordinates to ship coordinates, such as... Figure 1 As shown, this model assumes that light rays from three-dimensional space are inversely mapped to a two-dimensional plane via a pinhole optics system, and that the camera is horizontally mounted with zero pitch, roll, and yaw angles. The camera's mounting height above the water surface is h. Therefore, a spatial transformation of the camera's imaging model's coordinate system is required. The transformation between the radar polar coordinates calculated from the ship's target data and the camera image pixel coordinates is essentially a transformation between the ship's coordinate system and the camera image pixel coordinate system. Ideal image coordinates.

[0122] The conversion formula between radar polar coordinates and camera image pixel coordinates is as follows:

[0123] ;

[0124] ;

[0125] In the formula, u is the x-coordinate of the pixel coordinate system, v is the y-coordinate of the pixel coordinate system, u0 is the origin of the x-coordinate of the pixel coordinate system, v0 is the origin of the y-coordinate of the pixel coordinate system, and d x d represents the physical size of a single pixel on the x-axis. y This represents the physical size of a single pixel on the y-axis. Indicates the current radar angle;

[0126] Step 23: The conversion between AIS coordinates and camera image pixel coordinates mainly involves the conversion between the geodetic coordinate system, the ship's own coordinate system, and the camera image pixel coordinate system. The main steps are as follows:

[0127] The conversion between AIS coordinates and pixel coordinates in image data is as follows:

[0128] The latitude and longitude coordinates of the ship's center point are calculated based on the latitude and longitude of the AIS data and the installation location of the GPS antenna.

[0129] Calculate the relative orientation angle θ and relative distance r between the AIS ship and the camera using the ship's center point and the camera's latitude and longitude.

[0130] For a given ship center point latitude and longitude (Lat_ship, Lon_ship)(Lat_ship, Lon_ship) and camera latitude and longitude (Lat_cam, Lon_cam)(Lat_cam, Lon_cam), we can calculate the relative orientation angle θ using the following formula.

[0131] ;

[0132] The relative distance r is calculated using the following formula:

[0133] ;

[0134] Where R is the radius of the Earth;

[0135] This allows us to obtain the pixel coordinates of theoretically distortion-free camera images.

[0136] In information fusion systems, differences in detection intervals and initial times between sensors can cause temporal discrepancies in sensor data. Video data is acquired at a frequency of 25 frames per second; simply select the corresponding time. The radar's operating cycle is approximately 36 r / min, roughly once every 2 seconds. AIS "Category B" vessels send position reports at intervals determined by their speed; if SOG > 2 knots, Tmin is 30 seconds; if SOG ≤ 2 knots, Tmin is 180 seconds. Figure 2 The information time series model shown exhibits significant time discrepancies between sensor data. A data interval of 2 seconds is proposed, therefore, AIS data extrapolation is necessary to align with the time of ship image and radar image data for AIS data prediction. The AISB ship message transmission cycle is as follows: 180 reports per second when the target ship distance is less than or equal to 2 km, and 30 reports per second when the target ship distance is greater than 2 km.

[0137] Ship tracks sensed by multiple sensors cannot directly determine whether they belong to the same target vessel. It is necessary to correlate tracks sensed by different sensors to solve the problem of repeated tracking of ships by different sensors. Solving the problem of track correlation between different systems can be mainly done through statistical and fuzzy mathematics methods. Among them, the fuzzy mathematics-based method is more practical for dense target environments. Considering the busy navigation and relatively dense ship targets in the studied waterway, this chapter adopts a comprehensive factor fuzzy evaluation method to correlate AIS targets, radar targets, and image targets, thereby determining the correlation between AIS targets, radar fusion targets, and ship image targets. The main process involves performing coarse correlation, fine correlation, and comprehensive judgment on track data. First, radar targets and AIS targets are correlated, and then the targets correlated with AIS and radar are correlated with video targets.

[0138] Step 3, which involves fuzzy correlation of comprehensive factors, specifically involves:

[0139] Step 31: Determine the coarsely relevant target vessel based on the target vessel's AIS and radar data;

[0140] Step 32: Target ships that meet the coarse correlation criteria are then correlated with fine correlations using a comprehensive factor fuzzy evaluation algorithm;

[0141] Step 33: Perform quality management on the correlation between AIS data and radar data tracks of the target vessel to obtain AIS radar fusion targets;

[0142] Step 34: Associate the target data with the image data from the AIS radar fusion system.

[0143] In this embodiment of the invention, step 31 is further specifically defined as follows:

[0144] Locate the latitude and longitude points of the ship identified by radar. Find the AIS target subset in the current AIS data set that satisfies the following formula 1: Formula 1 is as follows:

[0145] ;

[0146] Among them, the AIS target latitude and longitude points are , For track-related thresholds;

[0147] Ship AIS targets and radar targets that meet the coarse correlation conditions are considered to have a preliminary correlation, and then further fine correlation is performed.

[0148] In an embodiment of the present invention, step 32 is further specified as follows: fine correlation involves further associating the radar fused target with the AIS target that meets the coarse correlation conditions using a comprehensive factor fuzzy evaluation algorithm. The fine correlation process mainly includes determining the fuzzy factor set, constructing a membership function model, and comprehensively evaluating and calculating the tightness of the correlation.

[0149] Step 321: Determine the fuzzy factor set, selecting longitude, rate of change of longitude, and latitude as the fuzzy factor set. The calculation formula is as follows:

[0150] ;

[0151] Among them, the latitude and longitude points of the ship radar target are: Radar target speed The ship's AIS target latitude and longitude coordinates are: AIS target heading ;U r1 It is the difference in image longitude between a ship radar target and an AIS target that meets the coarse correlation condition, U r2It is the difference in speed, U r3 It is the difference in latitude.

[0152] Step 322: Construct a membership function model based on the difference U between the image longitudes of the collected ship radar targets and AIS targets that satisfy the coarse correlation conditions. r1 The difference in speed U r2 The difference between latitude and latitude U r3 Construct a training set of samples and estimate the probability density based on the training set samples to obtain the membership function S. r1 Membership function S r2 Membership function S r3 The corresponding probability density function and distribution plot are obtained, and then the probability density function is normalized.

[0153] First, the membership function S r1 The probability density function adopts a semi-normal distribution, according to U r1 The probability density function of the data sample is estimated as follows: ;

[0154] Here, x represents the difference in latitude.

[0155] Distribution map as follows Figure 3 As shown.

[0156] Membership function S r2 The probability density function adopts an exponential distribution, according to U r2 The probability density function of the data sample is estimated as follows: ;

[0157] Here, x represents the heading difference;

[0158] Distribution map as follows Figure 4 As shown.

[0159] Membership function S r3 The probability density function adopts a semi-normal distribution, according to U r3 The probability density function of the data sample is estimated as follows: ;

[0160] Here, x represents the difference in longitude.

[0161] Distribution map as follows Figure 5 As shown.

[0162] Secondly, the membership function S r1 Membership function S r2 and membership function S r3 The probability density function is normalized using the following formula:

[0163] ;

[0164] Wherein, the membership function S r1 Membership function S r2 and membership function S r3 It varies between [0,1], where 0 indicates no correlation and 1 indicates correlation.

[0165] Step 323: Comprehensively evaluate the computational tightness, using the judgment matrix g. ri As follows:

[0166] ;

[0167] in, The coefficients are determined by the membership degrees calculated from the training set samples. To be based on the membership function Calculate the membership degree;

[0168] The formula for the coefficient relationship is as follows:

[0169]

[0170]

[0171] The consecutive judgment matrices g of a ship radar target and its corresponding single AIS target ri Take the average value j * Then, the density coefficient is taken as the maximum value among multiple different AIS targets at the same time, and is calculated as follows:

[0172] ;

[0173] Where g rij Let be the judgment matrix for the j-th AIS target;

[0174] ;

[0175] in, The threshold for the relevant judgment is set at 0.8.

[0176] Step 33 further specifies that the track association quality is as follows:

[0177] ;

[0178] ;

[0179] in, Δq represents the association quality between radar target i and AIS target j at time n; Δq is the change in association quality. , ,when Furthermore, when there is only one, the two tracks are fixedly correlated and no further correlation is performed. The radar target's ID is fixedly associated with the AIS target's Maritime Mobile Service Identifier (MMSI) number.

[0180] In this embodiment of the invention, step 34 is further specified as follows: During the ship target tracking and recognition process, a ship target bounding box is formed in the image. The coarse correlation is based on the pixel coordinates of the midpoint of the bottom edge of the ship image target bounding box. The coordinates of the ship are considered as identified from the video. Then, a target subset of AIS and radar data that meets the following conditions is searched within the current set of AIS and radar data:

[0181] ;

[0182] The coordinates of the AIS data, radar data, and target ship image after fusion and transformation are: T u and T v The pixel threshold;

[0183] AIS radar fusion targets of ships that meet the coarse correlation conditions are considered to have preliminary correlation with image targets of ships, and then fine correlation is performed.

[0184] Fine correlation involves further associating ship image targets with AIS and radar fusion targets that meet coarse correlation conditions using a comprehensive factor fuzzy evaluation algorithm.

[0185] The pixel x-coordinate, pixel x-coordinate change rate, and pixel y-coordinate are selected as the fuzzy factor set, and their calculation formula is as follows:

[0186] ;

[0187] Among them, the target pixel positions of the ship image are (u i v i The optical center pixel of the image is (u0, v0); f h denoted by , where is the camera focal length; h is the camera height; φ is the angle of motion of the target in the ship image; rAi is the distance from the target in the ship image to the camera; dx is the physical size of a single pixel on the x-axis; dy is the physical size of a single pixel on the y-axis; U1 is the difference in pixel abscissa between the target in the ship image and the AIS radar fusion target that satisfies the coarse correlation condition; U2 is the difference in the rate of change of pixel abscissa; and U3 is the difference in pixel ordinate.

[0188] A training set of samples is constructed based on the collected U1, U2, and U3. The probability density is estimated based on the training set samples to obtain the probability density functions and distribution maps corresponding to the membership functions S1, S2, and S3. The probability density functions are then normalized.

[0189] First, the probability density function of the membership function S1 adopts a semi-normal distribution. Based on the U1 data sample, the probability density function is estimated as follows: ;

[0190] Here, x represents the pixel difference along the x-axis.

[0191] Distribution map as follows Figure 6 As shown.

[0192] The probability density function of the membership function S2 adopts an exponential distribution. Based on the U2 data sample, the probability density function is estimated as follows: ;

[0193] Here, x represents the difference in the rate of change of the x-axis coordinate.

[0194] Distribution map as follows Figure 7 As shown.

[0195] Membership function S r3 The probability density function adopts a semi-normal distribution. Based on the U3 data sample, the probability density function is estimated as follows: ;

[0196] Here, x represents the pixel difference along the y-axis.

[0197] Distribution map as follows Figure 8 As shown.

[0198] Next, the probability density functions of membership functions S1, S2, and S3 are normalized using the following formula:

[0199] ;

[0200] The comprehensive evaluation uses a judgment matrix as follows: ,in, The coefficients are determined by the membership degrees calculated from the training set samples. To be based on the membership function Calculate membership degree;

[0201] The formula for the coefficient relationship is as follows:

[0202] ;

[0203] ;

[0204] Among them, the statistical calculations obtained from the training samples of a set of experimental data are as follows: =0.378531341; =0.251551977; =0.369916682.

[0205] The consecutive judgment matrices g of a ship radar target and its corresponding single AIS target ri Take the average value j * The calculation is as follows:

[0206] ;

[0207] Where g rij Let be the judgment matrix for the j-th AIS target;

[0208] Then, the density coefficient is taken as the maximum value among multiple different AIS targets at the same time, and calculated as follows:

[0209] ;

[0210] in, The threshold for the relevant judgment is set at 0.8;

[0211] The following are the specific steps for implementing track association quality management:

[0212] ;

[0213] ;

[0214] in, , ,when Furthermore, when two tracks are uniquely correlated, no further correlation is performed, and the ID of the image target is fixedly associated with the MMSI number of the AIS target's maritime mobile communication service identifier.

[0215] Track features refer to meaningful information extracted from data sources such as ship AIS and radar. These features can help correlate trajectory data, understand ship motion patterns, target behavior, and potential safety risks. When applying SVM to calculate track features, the ship's motion data first needs to be preprocessed and features extracted. Preprocessing includes data cleaning, noise reduction, and handling of missing data to ensure the accuracy and completeness of the input data. Then, feature parameters such as speed, heading, turning rate, acceleration, and trajectory similarity are extracted from ship AIS and radar data to construct a feature vector representation of the ship's trajectory.

[0216] The specific steps for trajectory association using a support vector machine binary classifier are as follows:

[0217] Step 3.1: Correlate AIS data and radar data to form an AIS radar fusion target;

[0218] Step 3.2: Associate the AIS radar fused target with the target track in the image data.

[0219] In one embodiment of the present invention, step 3.1 is further specified as follows:

[0220] Step 3.11: Preprocess and extract features from the ship's motion data. Feature extraction includes extracting the length difference feature between the radar trajectory and the AIS trajectory, the heading difference feature between the radar trajectory and the AIS trajectory, the average acceleration difference feature, the starting position distance difference feature between the radar trajectory and the AIS trajectory, the ending position distance difference feature between the radar trajectory and the AIS trajectory, and the DTW similarity feature between the radar trajectory and the AIS trajectory.

[0221] The length difference feature between radar and AIS tracks is used to compare the differences in target trajectory lengths observed from the two data sources. By calculating the total length of the radar and AIS tracks, the consistency of target trajectory information across different data sources can be understood. (Length difference feature) It can be expressed by the following formula:

[0222] ;

[0223] in, Indicates the total length of the radar track; This represents the total length of the AIS track. A smaller length difference indicates a higher probability that the targets observed from the two data source tracks are the same target; conversely, a larger length difference suggests that the observed targets may be inconsistent. The length difference feature helps detect discrepancies in track lengths between AIS and radar data, thereby determining whether the targets are related. This contributes to improving the ability to analyze and understand the correlation between target tracks.

[0224] The heading difference feature between radar and AIS tracks is used to compare the differences in target heading observed from the two data sources. Heading refers to the ship's heading relative to the ground. By calculating the difference between the average heading of the radar track and the average heading of the AIS track, the consistency of target heading information across different data sources can be understood. Heading difference feature It can be expressed by the following formula:

[0225] ;

[0226] in, Indicates the heading of the radar trajectory. This indicates the heading of the AIS trajectory. A smaller heading difference suggests that the two data source trajectories are more likely to observe the same target, while a larger difference may indicate inconsistencies in heading information.

[0227] The average acceleration difference feature between radar and AIS tracks is used to compare the differences in target acceleration observed from the two data sources. Acceleration refers to the rate of change of a ship's velocity over time; different ships have different acceleration ranges. By calculating the difference between the average acceleration values ​​of radar and AIS track points, we can understand the consistency of target acceleration information across different data sources. Acceleration Difference Feature Expressed by the following formula:

[0228] ;

[0229] in, This represents the acceleration of radar trajectory point i. This represents the acceleration of point j in the AIS trajectory. n is the number of data points in the radar trajectory, and m is the number of data points in the AIS trajectory. If the difference in average acceleration is small, it indicates that the observations of the target acceleration from the two data sources are relatively consistent, suggesting that the two observed trajectories are likely to belong to the same target. Otherwise, there may be inconsistencies in acceleration information.

[0230] The starting position distance difference feature of radar and AIS tracks is used to compare the distance differences between the target's starting positions observed by the two data sources. By calculating the distance difference between the starting points of the radar track and the AIS track, the consistency of the target's starting position information under the two data sources can be understood. The starting position distance difference feature Diff_Start is expressed by the following formula:

[0231] ;

[0232] in, Indicates the endpoint of the radar trajectory. This indicates the endpoint of the AIS trajectory. If the difference in the distance between the starting positions is small, it indicates that the two data sources have relatively consistent observations of the target's starting position, suggesting that the two observed trajectory segments are likely to belong to the same target. Conversely, a large difference in the distance may indicate inconsistencies in the starting position information.

[0233] The DTW similarity feature of radar tracks and AIS tracks is obtained by treating radar tracks and AIS tracks as time series and using the DTW algorithm to calculate the similarity between them, which can quantify the dynamic similarity between the two tracks.

[0234] Given time series of two trajectories: AIS and radar. and Y= , where x i and y j Let i and j represent the elements of the two sequences at time points i and j, respectively.

[0235] First, construct an m×n cumulative distance matrix D, where D[i][j] represents the distance between the first i elements of sequence X and the first j elements of sequence Y. This distance can be calculated using a specific distance metric, such as Euclidean distance.

[0236] Secondly, the optimal path is calculated using dynamic programming to find the best alignment between sequence X and sequence Y. This can be achieved using the following recursive formula:

[0237] ;

[0238] Where, dist(x) i ,y j ) represents the sequence element x i and y j The distance between them.

[0239] Finally, the DTW similarity between sequence X and sequence Y can be obtained by using the bottom right element D[m][n] of the cumulative distance matrix D.

[0240] The main advantage of DTW similarity is its ability to handle situations where time series have inconsistent lengths and varying speeds. In ship trajectory analysis, ships may move at different speeds, and AIS and radar data may have different sampling frequencies, leading to mismatches in the time dimension between the two trajectories. The DTW algorithm effectively solves these problems by finding the optimal time alignment through dynamic programming, resulting in a more accurate similarity measure.

[0241] This feature plays a crucial role in ship trajectory matching and association. By comparing the DTW similarity of different target trajectories, it helps identify whether AIS data and radar data correspond to the same ship target, thereby enabling data association and consistency analysis. This contributes to more accurate ship monitoring and position tracking, providing more reliable decision support and safety assurance for ship navigation.

[0242] In an embodiment of the present invention, step 3.12: preprocessing includes data cleaning, noise reduction and processing of missing data, extracting features from AIS trajectory and radar trajectory data and constructing a sample set to train an SVM classifier, thereby realizing automatic classification and association of unlabeled data, and the samples include positive samples and negative samples;

[0243] The construction of positive sample pairs is as follows:

[0244] Data preparation: First, the AIS and radar trajectory data are manually labeled; this data includes ship motion information and labeling information, i.e., which radar trajectories correspond to the same ship as the AIS trajectory. Positive sample trajectories are as follows: Figure 6 As shown.

[0245] Feature extraction: Track features are extracted from AIS and radar track data, such as distance difference, heading difference, and average acceleration difference. These features reflect important characteristics of ship motion and help to build ship identification and association models.

[0246] Sample matching: For AIS and radar tracks marked as the same target, their features are combined into a positive sample; such a sample will help the model learn the characteristics of the target vessel, thereby achieving the identification and association of the target vessel.

[0247] Time alignment: Set the sample length to the time span of three AIS data points. Since the sampling frequencies of AIS and radar data may differ, mismatches in the time dimensions of the samples may occur. To ensure data continuity and consistency, time alignment of the AIS and radar trajectories is necessary when constructing positive samples. Typically, a sample length of three AIS data points can be set; such a time window can reduce the problem of inconsistent time intervals to some extent.

[0248] Sample Labels: For each positive sample constructed, assign a label of 1, indicating that the current sample belongs to the same target ship; these labels will be used as training data for supervised learning to help the model understand the characteristics of the target ship.

[0249] Negative samples are constructed as follows:

[0250] Data Preparation: In contrast to the positive sample construction process, negative samples are composed of AIS and radar trajectory features labeled with different targets. First, AIS and radar trajectory data that are not labeled with the same target within the same time period are selected. The AIS and radar trajectory data contain ship motion information and labeling information. Negative sample trajectories are as follows: Figure 7 ;

[0251] Feature extraction: Extract track features from AIS track data and radar track data, such as distance difference, heading difference, and average acceleration difference;

[0252] Sample matching: For each AIS track and radar track marked as a different target, its features are combined into a negative sample; such samples will help the model learn to distinguish the features of different target ships.

[0253] Time alignment: Similar to the positive sample construction process, AIS and radar trajectories need to be time aligned to ensure data continuity and consistency;

[0254] Sample label: For each negative sample constructed, assign a label of 0, indicating that the current sample belongs to a different target ship;

[0255] The constructed positive and negative sample sets are combined to form a complete dataset. In the task of associating ship AIS and radar data, the number of positive samples of ship trajectories belonging to the same target is relatively small, while the number of negative samples of ships not belonging to the same target is large. This imbalanced distribution may lead to a decrease in model performance during the training and testing phases, because the model tends to predict more of the more numerous classes and ignore the fewer numerous classes.

[0256] To ensure the accuracy and robustness of model training, positive and negative samples should be evenly distributed in the dataset. This requires balancing the dataset distribution. Imbalanced samples are addressed by generating synthetic samples using the SMOTE algorithm to increase the number of samples in the minority class, thereby making the sample distribution more balanced.

[0257] Step 3.13: Evaluation metrics are used to assess the performance of the SVM model in the task of associating ship AIS and radar trajectories, i.e., whether it can correctly distinguish whether AIS and radar trajectories are the same target. Evaluation metrics are established, including accuracy, recall, and F1 score.

[0258] Step 3.14: Perform AIS and radar fusion target track correlation analysis.

[0259] Step 3.2 is further specified as follows:

[0260] A dataset is constructed by fusing target trajectory data and image data using labeled AIS radar data, extracting corresponding track features, and then establishing positive and negative samples.

[0261] By extracting features from labeled AIS radar target trajectory data and image data and constructing a sample set, a support vector machine (SVM) classifier is trained.

[0262] Precision, recall, and F1 score were selected as evaluation metrics.

[0263] The performance of the SVM model in the task of associating ship AIS, radar fusion target and video target tracks is evaluated by evaluation metrics; finally, track association analysis of AIS radar fusion target and image data is performed.

[0264] In one embodiment of the present invention, the dataset includes AIS and radar fusion data, video data, and corresponding tagging information of different target vessels. Range difference features, heading difference features, average acceleration difference features, starting position range difference features, ending position range difference features, and DTW similarity features are extracted from the dataset as sample features. The constructed dataset is grouped in a 7:3 ratio to form a training set and a test set for track association.

[0265] Based on the association of ship AIS and radar trajectory data, experiments were conducted to associate image trajectory data, thereby achieving multi-sensor trajectory data association. As mentioned above, the association process for image trajectory data is the same as that for AIS and radar data, mainly including the calculation of trajectory features under image coordinates, and the construction of a dataset sample of image data and associated AIS-radar targets. Finally, the association of ship trajectory data from multiple sensors is achieved through the training and classification prediction of a support vector machine model. This mainly includes the following aspects.

[0266] Calculation of track features in image coordinates:

[0267] The target trajectory is extracted from the image and converted into relevant motion features. These features include the target's latitude, longitude, speed, heading, and acceleration. Based on the ship's motion trajectory in the image, image track features are extracted for each trajectory segment.

[0268] Constructing image data and AIS-radar target dataset samples:

[0269] Ship target features in the image are compared with features in the AIS-radar data to construct track pairs based on the Maritime Mobile Service Identifier (MMSI). Positive samples consist of image trajectory features and radar-AIS trajectory features labeled with the same MMSI, while negative samples consist of image trajectory features and radar-AIS trajectory features labeled with different MMSIs. Different samples are assigned a label indicating whether they are related (1 for positive samples, 0 for negative samples).

[0270] Training and classification prediction of support vector machine models:

[0271] The constructed sample dataset is divided into training and test sets. The training set data is used to train a Support Vector Machine (SVM) model, and techniques such as cross-validation are employed to evaluate model performance and select parameters. This allows the SVM model to learn the relationships between image features, AIS-radar features, and target associations. After model training, new image features and AIS-radar features can be input into the SVM model for classification prediction. Finally, the model's performance is evaluated using the test set data based on metrics such as accuracy, precision, recall, F1 score, ROC curve, and AUC.

[0272] The multi-sensor ship trajectory data association and integration process ultimately integrates the AIS-radar associated targets with the image recognition targets, resulting in a comprehensive ship trajectory dataset that includes information from multi-source data association. This experiment considers the case where the target ship, after association with AIS and radar data, is also associated with an image target. In the image target association, the dataset is divided into training and test sets in an 8:2 ratio, requiring simultaneous association and recognition of multiple test set images and radar-AIS trajectories. The experiment will explore the applicability and efficiency of the proposed association method in the case of image and radar-AIS association. The calculated F1 score of 0.90 and AUC value of 0.96 indicate that the proposed support vector machine-based ship image and radar-AIS trajectory data association method performs excellently on 346 ship encounter and navigation samples. The F1 score of 0.9 signifies a good balance between accuracy and recall, achieving excellent classification results even with imbalanced sample class distributions. The ROC curve is shown below. Figure 11 As shown, an AUC value of 0.96 indicates that the model effectively distinguishes between the image trajectory and radar-AIS trajectory of the target vessel, and its classification results are consistent with the true labels.

[0273] like Figure 11 As shown in the figure, this comparative experiment analyzed the correlation results using the established track correlation method based on fuzzy comprehensive evaluation and the track correlation method based on support vector machine binary classifier. The AUC value of the track correlation method based on support vector machine binary classifier was 0.99, while the AUC value of the track correlation method based on fuzzy comprehensive evaluation was 0.95. Therefore, when performing radar and AIS target track correlation, the track correlation method based on support vector machine binary classifier is more effective.

[0274] Based on the association of ship radar and AIS trajectory data, experiments were conducted to associate it with image trajectory data. The association results were analyzed using both the established fuzzy comprehensive evaluation-based trajectory association method and the support vector machine-based binary classifier method. Figure 12 As shown, experimental results indicate that the AUC value of the track association method based on the support vector machine binary classifier is 0.96, while the AUC value of the track association method based on fuzzy comprehensive evaluation is 0.94. Therefore, when performing track association between AIS radar fusion targets and image (video) targets, the track association method based on the support vector machine binary classifier performs better.

[0275] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for multi-source target track association of inland waterway vessel navigation, characterized in that, The method comprises the following steps: Step 1: obtaining AIS data, radar data and image data of a target ship; Step 2: performing coordinate transformation and time calibration on AIS data, radar data and image data of the target ship and the own ship; Step 3: performing track association on the AIS data and the radar data after coordinate transformation and time calibration by means of comprehensive factor fuzzy evaluation or a support vector machine binary classifier, obtaining AIS-radar fusion data, and then performing track association on the AIS-radar fusion data and the image data after coordinate transformation and time calibration; The track association by means of comprehensive factor fuzzy evaluation in step 3 is specifically as follows: Step 31: determining a coarsely associated target ship according to the AIS data and the radar data of the target ship; Step 32: performing fine association on the coarsely associated target ship by means of comprehensive factor fuzzy evaluation algorithm; Step 33: managing track association quality of the AIS data and the radar data of the target ship, and obtaining AIS-radar fusion targets; Step 34: performing track association on the AIS-radar fusion targets and the image data; The track association by means of a support vector machine binary classifier specifically comprises the following steps: Step 3.1: performing track association on the AIS data and the radar data, and forming AIS-radar fusion targets; Step 3.2: performing track association on the AIS-radar fusion targets and the image data targets.

2. The method for multi-source target track association of inland river vessel navigation according to claim 1, characterized in that, The coordinate transformation in step 2 is specifically as follows: Step 21: AIS coordinates and radar polar coordinate system conversion, the target ship's latitude and longitude , according to the current latitude and longitude of the ship by Mercator projection for plane coordinates conversion, conversion formula as follows: ； ； ； ； wherein, is the target ship coordinate for the AIS system, is the radar angle, is the radar detection distance, The distance variance of radar detection is , and the angle variance is , is the current latitude and longitude of the ship, K is the spherical radian between the ship and the target ship, and R is the earth radius. Step 22: converting radar polar coordinates into camera picture pixel coordinates, and the conversion formula is as follows: ； ； In the formula, u is a horizontal pixel coordinate, v is a vertical pixel coordinate, u0 is a horizontal pixel coordinate origin, v0 is a vertical pixel coordinate origin, d x is a physical size of a single pixel point on the x-axis, d y is a physical size of a single pixel point on the y-axis, represents a current radar angle; Step 23: converting AIS coordinates into camera picture pixel coordinates, and the conversion is specifically as follows: calculating the longitude and latitude coordinates of the center point of the ship according to the longitude and latitude of the AIS data and the installation position of the GPS antenna; calculating the relative direction angle θ and the relative distance r of the AIS ship and the camera according to the longitude and latitude of the center point of the ship and the camera.

3. The method for navigating multi-source target track association of inland river vessels according to claim 2, characterized in that, Step 31 is further specifically as follows: Finding the latitude and longitude point of a ship identified by a radar Finding a subset of AIS targets in the current set of AIS data that satisfy the following equation 1, equation 1 being as follows: ； Wherein, AIS target latitude and longitude point is , is the track correlation threshold.

4. The method for multi-source target track association of inland river vessel navigation according to claim 3, characterized in that, Step 32 is further specifically as follows: Step 321: determining a fuzzy factor set, and selecting longitude, longitude change rate and latitude as the fuzzy factor set, and the calculation formula is as follows: ； Wherein, the longitude and latitude point of the ship radar target is , the ship radar target speed is ; the longitude and latitude point of the ship AIS target is , the AIS target heading is ; U r1 is the longitude difference between the ship radar target and the AIS target meeting the coarse correlation condition, U r2 is the speed difference, and U r3 is the latitude difference. Step 322: constructing a membership function model, according to the difference U of longitude between the collected ship radar target and the AIS target satisfying the coarse correlation condition r1 , the difference U of speed r2 , and the difference U of latitude r3 Construct a training set sample, and perform probability density estimation according to the training set sample to obtain a membership function S r1 , a membership function S r2 , a membership function S r3 corresponding probability density function and distribution diagram, and then normalize the probability density function; Step 323: Calculate the closeness degree by comprehensive evaluation, and the judgment matrix g used is as follows: ri is as follows: ； wherein, The coefficients are determined by the membership of the training set samples, membership function calculated membership; The coefficient relationship formula is as follows: ； ； Continuous plurality of judgement matrices g of ship radar targets and corresponding single AIS targets ri Taking average value, then the compactness coefficient takes the maximum value among a plurality of different AIS targets at the same time, calculated as follows: ； where g rij is the judgment matrix for the jth AIS objective; ； wherein is a correlation decision threshold.

5. The method for navigating multi-source target track association of inland river vessels according to claim 4, characterized in that, Step 33 is further specifically as follows: the track association quality is as follows: ； ； wherein, represents the association quality of the association between the i-th radar target and the j-th AIS target at the n-th time; Δq is the change of the association quality; , When and unique, the two tracks are fixedly correlated, and the correlation is no longer performed, and the fixed correlation associates the ID of the radar target with the maritime mobile service identity (MMSI) number of the AIS target.

6. The method for navigating multi-source target track association of inland river vessels according to claim 5, characterized in that, Step 34 is further specified as follows: a ship target frame is formed in the image in the ship target tracking and identification process, the coarse correlation is performed on the bottom edge midpoint pixel coordinates of the ship image target frame The ship coordinate points recognized by the video are regarded as the ship coordinate points recognized by the video, and a target subset of the AIS data and radar data set satisfying the condition is searched in the current AIS data and radar data set, and the condition is shown in the following formula: ； Wherein, the AIS data, radar data and target ship image fusion conversion of the coordinates are , T u and T v are pixel thresholds; selecting pixel horizontal coordinate, pixel horizontal coordinate change rate and pixel vertical coordinate as the fuzzy factor set, and the calculation formula is as follows: ； Wherein, the ship image target pixel point is (u i , v i ); the image optical center pixel point is (u0, v0); f is the camera focal length; h is the camera height; θAi is the angle of the ship image target motion direction; rAi is the distance from the ship image target to the camera; dx is the physical size of a single pixel point on the x-axis; dy is the physical size of a single pixel point on the y-axis; U1 is the difference in pixel horizontal coordinates between the ship image target and the AIS radar fusion target meeting the coarse correlation condition; U2 is the difference in pixel horizontal coordinate change rate; and U3 is the difference in pixel vertical coordinates. constructing training set samples according to the pixel horizontal coordinate difference U1, the pixel horizontal coordinate change rate difference U2 and the pixel vertical coordinate difference U3 between the collected ship image targets and the AIS-radar fusion targets satisfying the coarse correlation condition, and performing probability density estimation according to the training set samples to obtain the probability density functions and distribution diagrams corresponding to the membership functions S1, S2 and S3, and then normalizing the probability density functions; The comprehensive evaluation adopts a judgment matrix as wherein, The coefficient is determined by the membership calculated from the training set sample, is the membership calculated according to the membership function ; The coefficient relationship formula is as follows: ； ； Continuous plurality of judgement matrices g of ship radar targets and corresponding single AIS targets ri Taking average value j * After that, the compactness coefficient takes the maximum value among a plurality of different AIS targets at the same time, and is calculated as follows: ； where g rij is the judgment matrix for the jth AIS objective; ； wherein is a correlation decision threshold; managing the track association quality, and the management is as follows: ； ； wherein, , When and unique, the two tracks are fixedly related, no longer correlation, fixed image target ID and AIS target MMSI number associated with the water mobile communication business mark.

7. The method for navigating multi-source target track association of inland river vessels according to claim 6, characterized in that, Step 3.1 is further specifically as follows: Step 3.11: pre-processing and feature extraction are performed on the motion data of the ship, and the feature extraction includes extracting length difference features of radar tracks and AIS tracks, heading difference features of radar tracks and AIS tracks, average acceleration difference features; starting position distance difference features of radar tracks and AIS tracks, terminal position distance difference features of radar tracks and AIS tracks and DTW similarity features of radar tracks and AIS tracks; Step 3.12: The preprocessing includes data cleaning, denoising and handling missing data, extracting features from AIS trajectory and radar trajectory data and constructing a sample set to train an SVM classifier, so as to realize automatic classification and association of unlabeled data, and the sample includes positive samples and negative samples; The positive sample pair is constructed as follows: first, manually label AIS trajectory and radar trajectory data; extract the track features from the AIS trajectory and radar trajectory data; for the AIS and radar trajectories labeled as the same target, combine their features into a positive sample; set the sample length to be the time length of three AIS data points; for each positive sample constructed, assign a label 1, indicating that the current sample belongs to the same target ship; The negative sample is constructed as follows: the negative sample is composed of AIS trajectory and radar trajectory features labeled as different targets, first, select AIS trajectory data and radar trajectory data that are not labeled as the same target in the same time period, the AIS trajectory data and radar trajectory data contain ship motion information and labeling information; extract the track features from the AIS trajectory data and radar trajectory data; For each AIS trajectory and radar trajectory labeled as a different target, combine their features into a negative sample; time align the AIS trajectory and radar trajectory; for each negative sample constructed, assign a label 0, indicating that the current sample belongs to different target ships; generate synthetic samples through the SMOTE algorithm to increase the number of samples in the minority class, and construct the data set; Step 3.13: Establish evaluation indicators, including accuracy, recall rate and F1 score; Step 3.14: Perform AIS and radar fusion target track association analysis.

8. The method for navigating multi-source target track association of inland river vessels according to claim 7, characterized in that, Step 3.2 is further specified as: Construct a data set by extracting the corresponding track features from the labeled AIS radar fusion target trajectory data and image data, and then establish positive samples and negative samples; Train a support vector machine (SVM) classifier by extracting features from the labeled AIS radar fusion target trajectory data and image data and constructing a sample set; Select accuracy, recall rate and F1 score as evaluation indicators; Evaluate the performance of the SVM model in the ship AIS, radar fusion target and video target track association task through the evaluation indicators; finally, perform AIS radar fusion target and image data track association analysis.

Images