Marine ship AIS and vision fusion automatic identification method

Abstract

The invention is suitable for the field of operation equipment, and provides a marine ship AIS and vision fusion automatic identification method, which comprises the following steps: S1, collecting AIS data and camera vision data to generate original data D1, unifying a coordinate system, synchronizing timestamps, and collecting multiple environments; s2, performing denoising, correction and the like on the D1, and converting the D1 into feature data D2; s3, a fusion model is trained based on D2, a fusion algorithm is selected, and a father-child label system is used for iterative training and then stored; s4, applying the model in a real-time environment, generating a preliminary fusion result, and visualizing the preliminary fusion result; s5, the preliminary result is post-processed, and a report is generated and fed back to a decision making system; according to the method, the identification precision and the trajectory prediction accuracy under the conditions of AIS deficiency and severe weather are improved, and human intervention is reduced.

Description

Technical Field

[0001] This invention belongs to the field of operational equipment, and particularly relates to an automatic identification method for marine vessels that integrates AIS and visual perception. Background Technology

[0002] With the development of intelligent ship technology, navigation safety and environmental perception have become research priorities. Although Automatic Identification Systems (AIS) can transmit ship dynamic data (position, heading, speed), they have significant limitations. The existing technical problems correspond one-to-one with their subsequent beneficial effects: Limited capabilities of AIS in single applications: It can only detect ships equipped with AIS, and cannot identify targets without AIS; AIS data update time is long (3-10 seconds / time) and transmission is prone to packet loss, resulting in delayed safety decisions in complex environments (such as port areas with obstructions); traditional methods lack visual redundancy supplementation, and the accuracy of ship identification within 500 meters is less than 70%. Limitations of single-application vision: Visual data quality deteriorates under adverse weather conditions such as low light, rain, and dense fog. In moderate rain (visibility of about 1km), the detection rate of single-application vision is only 85.3%, and the effective detection distance is short in dense fog (visibility ≤500m), which cannot meet the needs of long-distance early warning. Data spatiotemporal synchronization problem: AIS uses a geographic coordinate system (WGS84), while vision uses a pixel coordinate system. When the two are not unified, the target position is misaligned. In addition, AIS data has a delay. Without compensation through interpolation (linear or Lagrange), it is not spatiotemporally synchronized with visual features, resulting in low fusion accuracy. Hierarchical recognition is missing: a parent-child tag system has not been introduced. The loss function is prone to misclassification of similar types of vessels (such as motorboats and yachts), has a low ID retention rate in multi-target tracking in dense waterways, and has insufficient overall recognition accuracy. Low trajectory prediction accuracy: Single AIS or visual prediction without combining extended Kalman filter (EKF) and channel rules results in a large average position error in 30-second predictions, with AIS prediction alone being ±3.7m and visual prediction alone being ±4.8m. Under strong ocean current interference, the trajectory jitter is large, affecting route planning. Therefore, an automatic identification method that combines AIS and visual perception for marine vessels is needed to solve the above problems. Summary of the Invention

[0003] The purpose of this invention is to provide an automatic identification method for marine vessels by fusing AIS and visual perception, in order to solve the problems mentioned in the background art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: An automatic identification method for marine vessels by fusing AIS and visual perception includes the following steps: S1. Collect ship AIS data and camera visual data to generate raw data D1. The collection includes starting the AIS system to obtain dynamic parameters of surrounding ships, collecting environmental image information through high-definition low-light cameras, unifying the AIS geographic coordinate system and the visual pixel coordinate system (converted sequentially through geographic coordinate system → ship coordinate system → camera coordinate system → pixel coordinate system), recording timestamps of the same time window to achieve data synchronization, and covering multiple environments such as sunny days, rainy days, and foggy days to ensure data diversity. S2. Transform the original data D1 into feature data D2. The transformation includes denoising, correcting and standardizing D1, using linear interpolation or Lagrange interpolation to process the AIS data to compensate for the delay, extracting edge and color features from the visual data using YOLOv8, matching AIS and visual features based on timestamps and geographic coordinates to ensure spatiotemporal consistency, and using principal component analysis (PCA) to select and reduce the dimensionality of the feature data. S3. Train an AIS and visual data fusion model based on feature data D2. The training includes selecting a fusion algorithm framework (YOLOv8 deep learning model, weighted fusion method, tree-based decision model), dividing the training set and validation set into a ratio of 70%-80%:20%-30%, introducing a parent-child class label system (constructing parent and child class labels, adding corresponding branches to the model detection head, and using a weighted loss function), iteratively training, evaluating performance through the validation set, and saving the model after it meets the target. S4. Apply the trained fusion model in the real-time operating environment of the ship. The application includes loading the model, collecting AIS data and visual data in real time at intervals of 0.5-3 seconds and converting them into feature data D2 input to the model, generating preliminary fusion results of target detection, motion trajectory prediction (using Kalman filtering or extended Kalman filtering) and environmental analysis, and visually displaying them through a graphical interface and collecting feedback data. S5. Post-process the preliminary fusion results to obtain the final fusion perception results. The post-processing includes data cleaning (removing redundancy and detection results with confidence <0.8), multi-sensor weighted fusion (weighted average or Bayesian fusion), non-maximum suppression (IOU threshold 0.4-0.5), path smoothing (Kalman filtering or particle filtering), comprehensive environmental assessment (marking dangerous waters, obstacles and ship proximity), generating a visualization report and feeding it back to the ship decision-making system.

[0005] This method achieves AIS and vision fusion through five steps: data collection, data transformation, model training, real-time application, and post-processing. In the data collection stage, a unified coordinate system (through multiple transformations and corresponding formulas) and timestamps are used, and data collection in multiple environments ensures data diversity. In the data transformation stage, interpolation is used to compensate for AIS latency, and YOLOv8 is used to extract visual features. Hierarchical supervision is introduced in model training. Real-time application and post-processing optimize the results. This method solves the shortcomings of single sensors, realizes the complementary advantages of AIS dynamic parameters and visual environment information, avoids recognition blind spots caused by data isolation, and provides a comprehensive perception foundation for autonomous navigation. For example, even when AIS is missing in the port area, high-precision recognition can still be achieved through vision.

[0006] Further technical solutions, specifically the unified AIS geographic coordinate system in S1, include: S11. Camera Coordinate System and Ship Positioning Parameter Calibration: Set the camera coordinate system (origin at the camera position, X-axis pointing to the bow, Y-axis pointing to the starboard side, Z-axis perpendicular to the ship's surface), and obtain the camera's internal parameters (focal length). , Main point location , The system records the latitude and longitude of the ship containing the camera and external parameters (camera pose relative to the ship). , and the latitude and longitude of the target vessel , ; S12. Converting geographic coordinate system to ship coordinate system: Calculating latitude and longitude differences. , The relative distance between the target vessel and the camera vessel is calculated using the Havesing formula, and the relative direction is determined by angle calculation, thus transforming the geographic coordinate system coordinates into relative coordinates in the vessel coordinate system. ; S13. Transformation from ship coordinate system to camera coordinate system: via ship orientation rotation matrix The conversion is achieved using the following formula: , in, , , The coordinates of the target vessel in the camera coordinate system; S14. Projecting the camera coordinate system's 3D coordinates onto the 2D pixel coordinate system: Using a perspective projection model, the formula is as follows: , in, The pixel coordinates of the target ship in the image; Through camera parameter calibration and geographic-ship coordinate system transformation (calculation) , With Havesing formula), ship-camera coordinate system transformation ( The process involves four steps: rotation matrix, 3D-to-2D projection (perspective projection formula), to accurately map AIS latitude and longitude to visual pixel coordinates. This completely resolves the spatial misalignment issue between AIS and visual data, ensuring a one-to-one correspondence between their target positions, such as the latitude and longitude of a target ship. , It can be accurately converted into image pixels. This provides a spatial basis for feature matching and fusion, avoiding misassociation of targets due to coordinate deviation.

[0007] Further technical solutions, specifically the interpolation processing of AIS data in S2, include: If two time points , The ship's acceleration changes relatively little between intervals, so linear interpolation is used to calculate the acceleration at any given time. The acceleration is given by the formula: , in, , They are time points , The acceleration value; If the acceleration changes are complex, Lagrange interpolation is used to calculate the acceleration at any given time based on acceleration data from multiple moments. The acceleration is given by the formula: , in, Given the acceleration values ​​of the data points, For Lagrange basis functions; Acceleration obtained based on interpolation The speed prediction formula is derived by integrating the ship's speed and position: , The location prediction formula is: , in, The initial velocity, The initial position, For integration variables; Choose linear interpolation (for simple scenarios) or Lagrange interpolation (for complex scenarios) based on the acceleration change, and combine them. and The formula predicts ship motion; effectively compensates for AIS data delay (3-10 seconds), reduces trajectory prediction deviation caused by data lag. For example, when the AIS update interval is 3 seconds, the ship's midpoint position can be predicted in real time through interpolation, so that AIS features and real-time visual features are spatiotemporally synchronized, improving fusion accuracy.

[0008] Further technical solutions, the features of AIS in S2 include: Dynamic characteristics (updated in real time, reflecting the ship's status) include the ship's identification code, latitude and longitude location (… , ), heading, bow orientation, ground speed, and rate of turn; Static characteristics (relatively fixed, providing prior constraints) include ship type, ship size, call sign, and ship name; Derived / computational features (based on dynamic information calculation) include historical motion trajectories and predicted trajectories (based on...) , The formula yields the relative distance, relative speed, and collision risk index between the vessel and the target vessel. AIS features cover three categories: dynamic (latitude and longitude, speed, etc.), static (ship type, size, etc.), and derived (relative distance, collision risk, etc.). Dynamic features support real-time tracking, static features assist in identity verification, and derived features calculate and warn of collision risks based on formulas. Multi-dimensional features provide rich information for fusion. For example, the relative distance formula (Haversine formula) is used to calculate the distance between the ship and the target ship, and combined with collision risk indicators, a warning is given 5-10 minutes in advance to avoid collision accidents.

[0009] Further technical solutions include: in S3, the YOLOv8 deep learning model utilizes a cross-stage partial connectivity (CSP) structure and a feature pyramid network (FPN) to extract spatial target features from visual data, aligning and fusing them with AIS features (latitude, longitude, speed, and heading); a weighted fusion method dynamically adjusts weights based on environmental conditions, setting weights to 0.6-0.8 when AIS signals are strong and stable, and setting visual system weights to 0.7-0.9 when AIS signals are missing or lost; a tree-based decision model (random forest or gradient boosting tree) processes AIS numerical features and visual recognition results, learns the importance distribution of different features, and outputs classification and regression results; YOLOv8 extracts visual spatial features and aligns them with AIS features; weighted fusion dynamically adjusts weights; tree model handles multi-source feature decision-making; adapts to different navigation scenarios, YOLOv8 is selected for high-precision detection in port areas (recognition accuracy of over 93.5%), and weighted fusion is selected for low-latency tracking in open sea areas (response time < 0.5 seconds), balancing accuracy and real-time performance, and improving the adaptability of the fusion system.

[0010] Further technical solutions, specifically the parent-child tag system in S3, include: S31. Label System Construction: The parent category labels are set as "medium-sized ships", "large ships", and "floating objects". The sub-category labels are further subdivided under the parent category. "Medium-sized ships" is subdivided into "small motorboats", "motorboats", and "yachts". The label file retains both parent and sub-category information. The parent category labels provide macro-level category information (e.g., "medium-sized ships" vs. "floating objects"), while the sub-category labels provide fine-grained differences (e.g., "yachts" and "motorboats"). The network can simultaneously learn the common features of the major categories (hull shape, water surface outline) and the differences of the minor categories (ship structure, size ratio). In a practical system, it may only be necessary to first determine "whether it is a ship" or "the type of ship (medium / large)" before performing more detailed sub-category recognition. If the sub-category label of some samples is uncertain, the parent category label can still be used for training (e.g., only labeled as "medium-sized ships"). The "parent category + sub-category" label system allows the network to learn features from coarse to fine, which not only enhances generalization and stability but also facilitates task expansion and hierarchical reasoning. It is a commonly used label design strategy for highly robust intelligent recognition systems. S32. Network structure adjustment: Add hierarchical output branches to the detection head part of the fusion model. The sub-class branch outputs fine-grained categories (motorboats, yachts, etc.), and the parent class branch outputs coarse-grained categories (medium-sized ships, etc.). The two branches share the previous layer feature extraction part. S33. Loss Function Design: A multi-level loss function is adopted, with the following formula: , in, This is the subclass loss (cross-entropy loss or FocalLoss). The parent class loss is calculated using Softmax and cross-entropy. , To dynamically adjust the weights, The value ranges from 0.5 to 0.7. Values ​​range from 0.3 to 0.5; Through hierarchical labeling, branch design, The loss function is implemented to ensure fine recognition of subclasses and provide redundant supervision of parent classes; it alleviates the problem of small sample subclass recognition, and the recognition of parent classes (such as "medium-sized ships") remains stable under low light conditions, avoiding target omissions; it reduces misclassification of similar ships, improves the ID retention rate of dense waterways to 96.2%, and improves the overall recognition accuracy by 7.8%.

[0011] Further technical solutions include using an extended Kalman filter (EKF) combined with channel rule constraints for motion trajectory prediction in S4. This allows for 30-second predictions of the detected target, with the mean position error (RMSE) controlled within ±2.4m. During the prediction process, dynamic features of AIS (velocity, heading) and the visual target's motion trend are utilized to correct trajectory prediction bias. In scenarios with strong ocean current interference, EKF filtering reduces trajectory jitter by 43%. By combining EKF with channel rule constraints and correcting the trajectory based on AIS dynamic features and visual motion trends, the RMSE predicted in 30 seconds is ±2.4m, which is 35% and 50% lower than that of AIS alone (±3.7m) and visual (±4.8m), respectively. The amplitude of strong ocean current turbulence is reduced by 43%, which can avoid potential collision risks in advance and provide accurate dynamic basis for route planning.

[0012] A further technical solution is to use a weighted average method for multi-sensor fusion in S5, and to adjust the weights of AIS data. Weighting of visual data satisfy Based on AIS signal strength (signal-to-noise ratio > 20dB) Visual clarity (image resolution ≥ 1920×1080 and contrast > 50) Dynamic adjustment; when Kalman filtering is used for path smoothing, the state equation is based on the ship speed. With position Formula construction: The observation equation combines AIS and visual target position data to reduce noise in the predicted trajectory; Multi-sensor weighted fusion (dynamic adjustment) , The system employs NMS to eliminate overlap and filters to smooth trajectories, ensuring accurate and reliable final results. For example, the fusion detection rate reaches 98.7% in moderate rain (13.4% higher than visual detection alone). The visualization report is integrated with the decision-making system to achieve a closed loop of "perception-decision" and reduce human intervention.

[0013] Compared with the prior art, the beneficial effects of the present invention are: This invention improves robustness in the absence of AIS signals and under adverse weather conditions: By using a coordinate system (including perspective projection formula) and dynamic weight adjustment, it switches to an enhanced vision mode when AIS signals are missing, achieving a ship identification accuracy of 93.5% and a buoy accuracy of 95.1% within 500 meters (more than 23.5% higher than traditional methods); the fusion detection rate is 98.7% in moderate rain, and the effective detection distance in dense fog is 420m longer than that of a single AIS sensor, solving the problem of poor environmental adaptability of a single sensor. This invention optimizes hierarchical recognition and trajectory accuracy: the parent-child tag system improves overall recognition accuracy by 7.8%, and the ID retention rate for dense waterways is 96.2%; EKF combined with... , The formula predicts the trajectory, with an RMSE of ±2.4m in 30 seconds, reducing strong ocean current jitter by 43%, and solving the problems of multi-target mismatch and trajectory inaccuracy. This invention improves the efficiency of adaptive and autonomous decision-making: AIS interpolation compensates for delay, and multi-sensor weighted fusion enables seamless mode transition and reduces human intervention; the final result supports automated control, improves the efficiency of autonomous navigation decision-making, and increases navigation safety by more than 30% in complex environments.

[0014] To more clearly illustrate the structural features and effects of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the overall process of the present invention; Figure 2 This is a detailed schematic diagram of the coordinate system of the present invention; Figure 3 This is a flowchart illustrating the parent-child tag system of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0017] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.

[0018] Example 1: Scenarios where AIS signal is missing in the port area (severe obstruction); like Figures 1-3 As shown in the figure, this invention provides an automatic identification method for marine vessels by fusing AIS and visual perception, including the following steps: S1. Start the AIS system (record basic parameters even with obstruction), use the high-definition low-light camera (1920×1080 resolution, ISO1000) to acquire images within 500 meters; calibrate the camera's internal parameters ( Pixels Pixels Pixels (pixels), recording this ship , Target ship , ;calculate , The relative distance of 120 meters is obtained using the Havesing formula, which is then converted into the ship's coordinate system. After Transform the matrix (ship heading 30°, rotation matrix corresponding to angle parameters) into the camera coordinate system. Finally, the pixel coordinates are obtained using the perspective projection formula. The timestamp synchronization interval is 1 second, and data from the port area on sunny and cloudy days is collected to generate D1.

[0019] S2. Denoise D1 (Gaussian filtering for visual processing, marking missing parts of the AIS signal); skip interpolation if there is no valid AIS data, extract ship outline and color features using YOLOv8 for visual processing; match visual features with historical AIS features (if any), and generate D2 by dimensionality reduction using PCA.

[0020] S3. Select the YOLOv8+ weighted fusion algorithm (visual weights) D2 is divided into training and validation sets in a 7:3 ratio; parent and child labels are set to "medium-sized vessels - small powerboats / motorboats / yachts", and the loss function is... , After 50 rounds of iterative training, the model was saved once the accuracy on the validation set reached 94%.

[0021] S4. Load the model, collect visual data in real time (1 frame per second) and convert it into D2 input; generate ship / buoy detection results (confidence ≥ 0.85), trajectory prediction based on visual inter-frame motion, and environmental analysis (marking dock obstacles); the visualization interface displays the target pixel position and detection category.

[0022] S5. Data cleaning removes results with confidence levels < 0.8, NMS (IOU = 0.5) is used to eliminate overlap; particle filtering is used for path smoothing; environmental assessment marks hazardous areas within 100 meters of the dock; the generated results are fed back to the decision-making system.

[0023] In this embodiment: due to the lack of AIS, the system relies on visual and coordinate transformation formulas to achieve accurate positioning. The accuracy rate of ship identification within 500 meters is 93.5%, and that of buoys is 95.1%, which is more than 25% higher than the traditional non-redundant method (68%). This successfully avoids the risk of collision when ships meet at the dock and verifies the effectiveness of coordinate system one and visual redundancy.

[0024] Example 2: Moderate rain in severe weather (visibility 1km); The difference between this embodiment and Embodiment 1 is that: S1. Start the AIS system (data updates every 3 seconds, signal-to-noise ratio 18dB), and enable low-light mode (ISO800) to acquire images; calibrate camera parameters ( Pixels (pixels), this ship , Target ship , pixel coordinates are obtained after coordinate transformation. The timestamp synchronization interval is 1 second, and D1 is generated by collecting moderate rain data.

[0025] S2 and AIS use Kalman filtering for noise reduction, and linear interpolation is used because the acceleration change is small (the ship is sailing at a constant speed). seconds,² ; seconds,² ,calculate Seconds² ,pass ( )have to , ( )have to Visual noise reduction is achieved using bilateral filtering, and edge features are extracted using YOLOv8. D2 is generated by matching AIS with visual features.

[0026] S3, Select a weighted fusion + random forest model (AIS weights) Visual D2 is divided in an 8:2 ratio; the parent and child labels are set to "Large Vessels - Cargo Ships / Oil Tankers". , After 40 training rounds, the validation set detection rate was saved after reaching 97%.

[0027] S4. Load the model and collect AIS (once every 3 seconds) and visual (2 frames per second) data in real time; predict the trajectory for 30 seconds using EKF, with RMSE=±2.5m; analyze the environment and mark the visibility of the rain area at 1km; visualize the predicted trajectory.

[0028] S5. Data cleaning removes lost AIS packets; weighted average fusion is used for multiple sensors; NMS (IOU=0.45) eliminates false bounding boxes in rain and fog; EKF is used for path smoothing; high-risk areas within 200 meters are marked; feedback is sent to the decision system to adjust the speed to 6m / s.

[0029] In this embodiment: delay is compensated by linear interpolation of AIS, and rain and fog interference is reduced by dynamic weight adjustment. The fusion detection rate reaches 98.7%, which is 13.4% higher than that of vision alone (85.3%). The 30-second trajectory prediction is close to the ideal accuracy. The decision system adjusts the speed based on the results to avoid route deviation caused by rain and fog, which verifies the effectiveness of interpolation and weighted fusion.

[0030] Example 3: Multi-target scenario in dense waterways (15 ships converging); The difference between this embodiment and Embodiment 2 is that: S1. Start the AIS system (updates every 2 seconds, signal-to-noise ratio 25dB), dual cameras at the bow and stern acquire 360° images; calibrate the internal parameters of the dual cameras (make them consistent). (pixels), this ship , ; 15 target vessels, recorded separately , The coordinates of each target pixel are obtained through coordinate transformation; the timestamp synchronization interval is 0.5 seconds, and the channel data is collected to generate D1.

[0031] S2 and AIS use moving average noise reduction. Due to the complex acceleration changes in some ships due to turning, Lagrange interpolation is used. , , Seconds² ,² ,² ,pass calculate Seconds² , combined , The formula predicts the speed and position; visual denoising is achieved using guided filtering, and YOLOv8 extracts contour features; the AIS ship name is matched with the visual target, and PCA is used to reduce the dimensionality and generate D2.

[0032] S3, Select the YOLOv8+ gradient boosting tree model; D2 is divided into 7.5:2.5; Parent and child labels are set to "Ships - Cargo Ships / Fishing Boats / Yachts". , Training for 60 rounds, and saving the validation set IDs after achieving a 95% retention rate.

[0033] S4. Load the model and collect AIS data (once every 2 seconds) and dual camera data (3 frames per second) in real time; use EKF combined with waterway rules (prohibiting lateral crossing) to predict the 30-second trajectory; generate multi-target detection (distinguishing between cargo ships and fishing vessels) and collision risk indicators (<1km is considered high risk); visualize the ID tracking results.

[0034] S5. Data cleaning to remove duplicate AIS targets; Bayesian fusion of multiple sensors ( , NMS (IOU=0.4) is used to eliminate overlap; Kalman filtering is used for path smoothing; 5 high-risk vessels are marked; the results are fed back to the decision system to plan avoidance routes.

[0035] In this embodiment: Lagrange interpolation accurately compensates for the AIS delay of the turning vessel, the parent-child tag system enables the ID retention rate to reach 96.2%, and the collision risk warning accuracy rate to reach 94%; the decision system plans the avoidance route 5 minutes in advance based on the EKF predicted trajectory, with no collision accidents, verifying the effectiveness of the method in complex multi-objective scenarios.

[0036] Working principle and usage process of this invention: This method achieves automatic ship identification through a closed loop of "data input-processing-training-perception-optimization". Each step is linked by formulas and parameters. The specific process is as follows: Data collection: Constructing a unified spatiotemporal data source; The AIS system receives real-time dynamic parameters of the target vessel. , (Heading, speed), high-definition cameras capture environmental images; camera parameter calibration ( , , , Determine the imaging basis and calculate , And by combining the Havesing formula, the geographical coordinate system is transformed into the ship coordinate system, and then... The rotation matrix is ​​transformed into the camera coordinate system, and finally, the perspective projection formula is used. Mapping to the pixel coordinate system resolves spatial misalignment; synchronizing timestamps (recorded within the same window) with multi-environment acquisition generates raw data D1, providing a consistent foundation for subsequent processing; Data transformation: compensating for delays and extracting key features; D1 preprocessing: After denoising the AIS data, linear interpolation is selected based on the acceleration change. ) or Lagrange interpolation ( ), combined and The formula predicts motion and compensates for data latency; after denoising the visual data, YOLOv8 is used to extract edge and color features; based on timestamps and geographic coordinates, AIS features (latitude and longitude, speed) are matched with visual features, and PCA is used to reduce dimensionality and remove redundancy to generate feature data D2, ensuring data spatiotemporal synchronization and effectiveness; Model training: Building hierarchical fusion capabilities; The chosen fusion algorithm framework is: YOLOv8 extracts visual spatial features and aligns them with AIS features, then performs weighted fusion with dynamic adjustments to the weights (when the AIS signal is strong). When missing Tree-based models handle multi-feature decision-making; D2 partitions the training / validation set proportionally; a parent-child labeling system is introduced, and the model detection head has child / parent branches. Iterative training with loss function; periodic evaluation with validation set (accuracy, ID retention rate) to prevent overfitting; saving the model after training to give it the hierarchical capability of "fine recognition + coarse-grained redundancy"; Real-time application: generating dynamic perception results; The ship's real-time system loads a model, acquiring AIS and visual data at 0.5-3 second intervals and converting them into D2 input. The model outputs target detection results (identifying ships, buoys, etc., with a confidence level ≥ 0.8) and motion trajectory prediction (using EKF, combined with...). , Formulas and navigation rules, 30-second RMSE ± 2.4m; environmental analysis (visibility, water surface conditions); visually displaying target pixel positions, predicted trajectories and collision risks through a graphical interface; collecting feedback data for model iteration and optimization; providing real-time support for navigation decisions. Post-processing: Optimizing the connection between results and decision-making; Preliminary data cleaning (removal of redundancy and low-confidence detections); multi-sensor weighted fusion (adjustment based on AIS signal-to-noise ratio and visual sharpness). , such as during moderate rain , Non-maximum suppression (IOU 0.4-0.5) eliminates overlapping detection boxes; path smoothing uses Kalman filtering or particle filtering to reduce trajectory noise; comprehensive environmental assessment marks dangerous waters, obstacles, and ship proximity; generates visual reports and feeds them back to the ship decision-making system to support automated speed adjustment and path planning, achieving autonomous navigation safety assurance; Throughout the process, formulas and parameters are integrated into each stage: coordinate transformation formulas ensure spatial alignment, interpolation and integration formulas compensate for AIS latency, loss functions improve hierarchical recognition, filtering formulas optimize trajectories, and weight adjustments achieve adaptive fusion. Each stage works closely together to address the shortcomings of a single sensor, forming a complete automatic identification link for ships and improving navigation safety and intelligence.

[0037] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for automatic identification of marine vessels by fusing AIS and visual perception, characterized in that, Includes the following steps: S1. Collect ship AIS data and camera visual data to generate raw data D1. Start AIS to obtain dynamic parameters of surrounding ships. Collect environmental images through high-definition low-light cameras. Unify the AIS geographic coordinate system and the visual pixel coordinate system. Record the timestamp of the same time window to achieve data synchronization. Cover multiple environments such as sunny days, rainy days, and foggy days. S2. Transform D1 into feature data D2, denoise, correct and standardize D1, interpolate AIS data to compensate for delay, extract features from visual data, match AIS and visual features to ensure spatiotemporal consistency, and select dimensionality reduction for feature data. S3. Based on D2 training, an AIS and visual data fusion model is selected, and a fusion algorithm framework (YOLOv8 deep learning model, weighted fusion method, tree-based decision model) is selected. The training set and validation set are divided, a parent-child class label system is introduced to build a hierarchical supervision mechanism, and the model is saved after iterative training and validation. S4. Apply the trained model to the real-time environment of the ship, load the model, collect and input data into D2 in real time, generate preliminary fusion results of target detection, motion trajectory prediction and environmental analysis, visualize the results and collect feedback data. S5. Post-process the preliminary results to obtain the final fusion perception results, perform data cleaning, multi-sensor weighted fusion, non-maximum suppression, path smoothing and comprehensive environmental assessment, generate a report and feed it back to the ship decision-making system.

2. The automatic identification method for marine vessels by fusing AIS and vision according to claim 1, characterized in that, The unified AIS geographic coordinate system in S1 specifically includes: S11. Camera Coordinate System and Ship Positioning Parameter Calibration: Establish the camera coordinate system, acquire the camera's internal and external parameters, and record the latitude and longitude of the ship where the camera is located. , and the latitude and longitude of the target vessel , ; S12. Converting geographic coordinate system to ship coordinate system: Calculating latitude and longitude differences. , The relative distance between the target vessel and the camera vessel is calculated using the Havesing formula, and the relative direction is determined by angle calculation, thus transforming the geographic coordinate system coordinates into relative coordinates in the vessel coordinate system. ; S13. Transformation from ship coordinate system to camera coordinate system: via ship orientation rotation matrix The conversion is achieved using the following formula: ， in, , , The coordinates of the target vessel in the camera coordinate system; S14. Projecting the camera coordinate system's 3D coordinates onto the 2D pixel coordinate system: Using a perspective projection model, the formula is as follows: ， in, The pixel coordinates of the target ship in the image.

3. The automatic identification method for marine vessels by fusing AIS and vision according to claim 1, characterized in that, The interpolation processing of AIS data in S2 specifically includes: If two time points , The ship's acceleration changes relatively little between intervals, so linear interpolation is used to calculate the acceleration at any given time. The acceleration is given by the formula: ， in, , They are time points , The acceleration value; If the acceleration changes are complex, Lagrange interpolation is used to calculate the acceleration at any given time based on acceleration data from multiple moments. The acceleration is given by the formula: ， in, Given the acceleration values ​​of the data points, For Lagrange basis functions; Acceleration obtained based on interpolation The speed prediction formula is derived by integrating the ship's speed and position: ， The location prediction formula is: ， in, The initial velocity, The initial position, It is the integral variable.

4. The automatic identification method for marine vessels by fusing AIS and vision according to claim 1, characterized in that, The features of AIS in S2 include: Dynamic features include ship identification code, latitude and longitude location ( , ), heading, bow orientation, ground speed, and rate of turn; Static characteristics include vessel type, vessel size, call sign, and vessel name; Derived / computational features include historical motion trajectories and predicted trajectories (based on...) , The formula yields the relative distance, relative speed, and collision risk index between the ship and the target ship.

5. The automatic identification method for marine vessels by fusing AIS and vision according to claim 1, characterized in that, In S3, the YOLOv8 deep learning model utilizes a cross-stage partial connectivity (CSP) structure and a feature pyramid network (FPN) to extract spatial target features from visual data, and then aligns and fuses them with AIS features. The weighted fusion method dynamically adjusts the weights based on environmental conditions. When the AIS signal is strong and stable, the weights are set to 0.6-0.

8. When the AIS signal is missing or lost, the weights of the visual system are set to 0.7-0.

9. The tree-based decision model processes the numerical features of AIS and the visual recognition results, learns the importance distribution of different features, and outputs classification and regression results.

6. The automatic identification method for marine vessels by fusing AIS and vision according to claim 1, characterized in that, The parent-child tag system in S3 specifically includes: S31. Label System Construction: The parent category labels are set as "medium-sized ships", "large ships" and "floating objects". The sub-category labels are further subdivided under the parent category. "medium-sized ships" are subdivided into "small powerboats", "motorboats" and "yachts". The label file retains both parent and sub-category information. The "parent + sub-category" label system allows the network to learn features from coarse to fine, which not only enhances generalization and stability, but also facilitates task expansion and hierarchical reasoning. It is a commonly used label design strategy for highly robust intelligent recognition systems. S32. Network structure adjustment: Add hierarchical output branches to the detection head part of the fusion model. The sub-class branch outputs fine-grained categories, and the parent class branch outputs coarse-grained categories. The two branches share the previous layer feature extraction part. S33. Loss Function Design: A multi-level loss function is adopted, with the following formula: ， in, For subclass loss, Loss due to the parent class , To dynamically adjust the weights, The value ranges from 0.5 to 0.

7. The value ranges from 0.3 to 0.

5.

7. The automatic identification method for marine vessels by fusing AIS and vision according to claim 1, characterized in that, In S4, the motion trajectory prediction adopts extended Kalman filtering combined with channel rule constraints to predict the detected target in 30 seconds, with the average position error controlled within ±2.4m. During the prediction process, the dynamic features of AIS and the motion trend of the visual target are used to correct the trajectory prediction deviation. In the strong ocean current interference scenario, the trajectory jitter amplitude is reduced by 43% through EKF filtering.

8. The automatic identification method for marine vessels by fusing AIS and vision according to claim 1, characterized in that, When using a weighted average method for multi-sensor fusion in S5, the weight of AIS data... Weighting of visual data satisfy Dynamic adjustment based on AIS signal strength and visual clarity; when Kalman filtering is used for path smoothing, the state equation is based on ship speed. With position Formula construction combines AIS and visual target position data to reduce prediction trajectory noise.

Images