Ship visual perception and target identification method based on deep learning

Abstract

This invention belongs to the field of computer vision and artificial intelligence technology. Specifically, it discloses a deep learning-based method for ship visual perception and target recognition, which includes: collecting multi-source data such as visible light and infrared from multiple sensors, along with electronic nautical chart information, to generate a dynamic scene profile representing the dimensions of environment, task, and threat; inputting the profile into a metacognitive scheduling module to predict and select the optimal combination of models suitable for the current scene; inputting multi-source data into the model combination for collaborative detection to generate preliminary results containing target location and category; fusing ship AIS signals for correlation matching, and verifying and correcting the preliminary results using predictive physical rules; performing time-series analysis and trajectory prediction based on continuous multi-frame results to generate a dynamic risk situation map and attribute the causes; and adjusting sensor parameters in a closed loop and incrementally updating the model library based on risk attribution and performance evaluation to achieve continuous adaptive optimization of system performance.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and artificial intelligence technology, and relates to a deep learning-based method for ship visual perception and target recognition. Background Technology

[0002] Ship visual perception and target recognition technology is a key component for improving navigation safety and realizing ship intelligence. It mainly utilizes optical sensors such as visible light and infrared deployed on ships, combined with deep learning image processing algorithms, to detect, classify, and track other ships, navigation marks, obstacles, and other targets in the navigation environment in real time, providing decision support for the navigator or providing environmental perception information for autonomous navigation systems.

[0003] Existing technologies also include some solutions related to ship navigation safety. For example, Chinese patent publication number CN120998067A discloses a maritime vessel safety situational awareness system. This system integrates multiple sensors, including shipborne radar, Automatic Identification System (AIS), GPS, and optical cameras, to monitor the ship's surrounding environment. It fuses multi-source data to identify and track surface targets, calculates collision risks based on target motion parameters, and ultimately provides situational awareness and risk warnings to the crew on electronic nautical charts. This system provides ships with basic environmental awareness capabilities, improving navigation safety to a certain extent.

[0004] The aforementioned solutions also exhibit several limitations, specifically: First, these systems typically employ fixed data processing flows and algorithm models, making it difficult to accommodate diverse navigation scenarios such as fog, nighttime, strong light, and dense ship traffic. This leads to a sharp decline in recognition performance and insufficient robustness under specific adverse conditions. Second, their data fusion largely remains at the decision-making level, representing the combined results after independent processing by each sensor. This fails to fully leverage the complementarity and redundancy of multi-source data at the feature level, making it difficult to effectively suppress the inherent noise and interference of individual sensors, frequently resulting in false alarms or missed alarms. Finally, these systems lack the ability to continuously learn and improve from practical experience. Once deployed, their performance tends to be fixed, unable to adaptively optimize for new problems and scenarios encountered during operation, making it difficult to guarantee reliability and advancement over long-term operation. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art and to achieve the above objectives, the present invention proposes the following technical solution: a deep learning-based ship visual perception and target recognition method, including: Step 1, collecting multi-source perception data and electronic chart information at the current moment through multiple sensors deployed on the ship, and generating a dynamic scene profile representing the current comprehensive environment through multi-dimensional feature extraction.

[0006] Step 2: Input the dynamic scene profile into a metacognitive scheduling module for dynamic scheduling of models, filter out at least two candidate detection models, and use part of the multi-source perception data at the current moment as probe data to predict the expected performance of each candidate detection model in the current complete scene. Then, based on the expected performance, form an instant optimal model combination that is adapted to the current scene.

[0007] Step 3: Input the multi-source sensing data at the current moment into the instantaneous optimal model combination for collaborative feature extraction and target detection, and generate preliminary collaborative detection results containing target location, category, confidence level and low modality consistency label.

[0008] Step 4: Perform predictive physical rule verification on each target in the preliminary collaborative detection results, correct the preliminary collaborative detection results based on the verification results, and output the final target recognition results.

[0009] Step 5: Based on the final target recognition results of multiple consecutive frames and the ship's navigation parameter sensor data, perform time series analysis and trajectory prediction to generate a dynamic risk situation map for the future preset time period, and trace the source and cause of risk events in the dynamic risk situation map to generate risk attribution information.

[0010] Step 6: Based on risk attribution information and the system performance evaluation results of the current cycle, adjust the data acquisition parameters of multiple sensors in reverse, and update the prior knowledge of the metacognitive scheduling module for predicting performance and the model composition in the model warehouse to optimize the perception and recognition performance of the next processing cycle.

[0011] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) By constructing a dynamic scene profile and combining it with a metacognitive scheduling mechanism, the present invention enables the system to dynamically load the optimal target detection model combination according to the real-time changing navigation environment, thereby getting rid of the limitation of poor adaptability of traditional fixed models in complex scenarios and realizing high-precision and high-robust target recognition in all weather and all scenarios.

[0012] (2) This invention establishes a dual verification system that combines data-driven and rule-constrained approaches. At the data level, cross-modal attention verification utilizes multi-sensor information to mutually supervise each other at the feature layer, suppressing false detections caused by single-modal noise at the source. At the physical logic level, predictive physical rules such as AIS track comparison are introduced to review and correct the rationality of the perception results. The combination of these two approaches ensures that each output target is not only significant in the data but also credible in the physical world, thereby improving the reliability of the system.

[0013] (3) The present invention designs a complete closed-loop evolutionary system from risk attribution to parameter and model optimization, which can automatically diagnose and identify the root cause of failure, and adjust the sensor configuration and update the model library accordingly, so that the system has the ability to continuously learn and improve itself from practical experience, ensuring stable performance and continuous improvement under long-term operation. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of the implementation steps of the method of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Please see Figure 1 As shown, the deep learning-based ship visual perception and target recognition method proposed in this invention includes: Step 1, collecting multi-source perception data and electronic chart information at the current moment through multiple sensors deployed on the ship, and generating a dynamic scene profile representing the current comprehensive environment through multi-dimensional feature extraction.

[0018] In a preferred embodiment, multiple sensors deployed on the ship collect multi-source sensing data and electronic chart information at the current moment, and generate a dynamic scene profile representing the current comprehensive environment through multi-dimensional feature extraction. The multi-source sensing data includes at least visible light images, infrared images, and navigation parameter sensor data of the ship.

[0019] Electronic nautical charts contain geographic layers and navigation rules.

[0020] By analyzing visible light and infrared images, the visibility level and light intensity level of the current environment can be obtained.

[0021] By matching the position information in the navigation parameter sensor data with the electronic nautical chart information, the current area and mission attributes of the vessel are determined.

[0022] By statistically analyzing the historical final target identification results, we can obtain the current target density and velocity distribution characteristics in the surrounding area.

[0023] By integrating visibility level, light intensity level, regional task attributes, and target density and speed distribution characteristics, a dynamic scene profile is generated.

[0024] Specifically, the engineering implementation steps for generating a dynamic scene profile representing the current overall environment are as follows:

[0025] Step 1-1, Data Acquisition: At the beginning of each processing cycle, the system synchronously triggers each sensor to acquire various types of data at the same time to ensure data consistency over time, including:

[0026] Visible light image data, which is captured in real time by a visible light camera installed high on the ship, reflects scene information within the range of human vision. It is mainly used to identify the shape, color, texture and other detailed features of the target, and serves as a basis for assessing the current visibility level and light intensity level of the environment.

[0027] Infrared image data, which is collected in real time by a ship-borne infrared thermal imager, reflects the heat distribution of objects in a scene. It does not depend on ambient light, can penetrate thin fog, and is mainly used to detect and identify targets with thermal signals at night, in low visibility or camouflage conditions.

[0028] The dynamic data stream continuously acquired from the ship's onboard navigation parameter sensors represents the ship's own motion status and position information. This data specifically includes the ship's unique identification code obtained from the Automatic Identification System, real-time latitude and longitude coordinates, speed over land, and heading over land. This data is the key basis for geographic information matching, target trajectory verification, and dynamic risk situation analysis.

[0029] The structured geospatial database pre-stored within the system not only contains basic geographic layer information, but more importantly, these layers are pre-divided and labeled with regional attributes that have clear navigation rules, such as open waters, port access channels, anchorages, traffic control zones, or narrow waterways. This information provides the system with a legal and physical constraint context of the navigation environment.

[0030] Steps 1-2: The system performs a quantitative evaluation of the environmental dimensions. The engineering purpose of this is to accurately define the effective operating range and performance boundaries of the current photoelectric sensor. The specific implementation is as follows:

[0031] From the visible light image data stream, an algorithm based on dark channel priors is used to calculate the global mean of the dark channel in the non-sky region of the image. This mean reflects the concentration of suspended particulate matter in the atmosphere. Based on preset dark channel value segmentation thresholds, the calculated global mean is directly mapped to a visibility level of 1 to 5. In engineering implementation, a higher global mean of the dark channel value means denser fog and a lower visibility level. For example, the mean range [0, 0.2) can be defined as level 5 (excellent), [0.2, 0.4) as level 4, and so on, until [0.8, 1.0] is level 1 (very poor).

[0032] In parallel, the system analyzes the pixel intensity histogram of the visible light image, calculating its mean and standard deviation—two core statistics. Based on a pre-defined two-dimensional threshold decision table, this (mean, standard deviation) combination is mapped to a specific light intensity level. For example, when the histogram mean is below 50 and the standard deviation is below 30, the system classifies it as "nighttime" or "extremely low light"; when the mean is above 200 and the standard deviation is below 40, it is classified as "strong front lighting" or "overexposure".

[0033] Ultimately, the system outputs two discrete quantized values: visibility level and light intensity level, which together constitute the environmental dimension features that characterize the current scene.

[0034] Steps 1-3: Parallel Execution of Task Dimension Analysis: The system obtains the ship's real-time latitude and longitude coordinates from navigation parameter sensor data, then performs a spatial database query to overlay and match these coordinates with the geographic layers in the pre-loaded electronic nautical chart information. These electronic chart layers pre-define areas with clearly defined navigation rules, such as open waters, port access channels, anchorages, or narrow waterways. Upon successful matching, the system outputs a discrete code representing the current area's task attributes. For example, when entering a traffic-controlled channel, the output code is "TSS-01," which serves as a prerequisite for triggering subsequent, more stringent collision avoidance logic. The engineering purpose is to endow the system with contextual awareness of geospatial and navigational rules, enabling the system to understand its current legal and physical constraints.

[0035] Steps 1-4: Perform dynamic statistics on the threat dimension: The system maintains a time-series data buffer storing all final target identification results from the past 60 to 180 seconds. In each processing cycle, the system delineates a circular electronic fence area with a radius of 2 to 3 nautical miles centered on the ship.

[0036] The total number Y of valid targets currently located within the electronic fence area is filtered and counted within the time-series data buffer. The target density D is calculated using the following formula: ,in The radius of the electronic fence. Pi is the mathematical constant of a circle.

[0037] For each valid target within the fence, its motion vector is extracted, and the average and standard deviation of its velocity magnitude are calculated. These two values ​​together constitute the current velocity distribution characteristics.

[0038] The system compares real-time statistical data on target density and velocity standard deviation with preset benchmark thresholds. When both exceed their respective thresholds, the current navigation environment is deemed complex and highly unpredictable. Its engineering purpose is to assess the dynamic complexity and potential risk density of the surrounding environment.

[0039] Steps 1-5: Multi-dimensional feature fusion: A weighted normalization fusion algorithm is used, where visibility level and light intensity level are normalized to form environmental factors. The task factor is formed after the regional task attribute code is normalized. The target density and velocity distribution characteristics are normalized to form the threat factor. Each factor was normalized to the range of 0 to 1. Dynamic scene profiling. It can be expressed by the following formula:

[0040] ;

[0041] in, , , The weighting coefficients are for three dimensions: environment, mission, and threat, with a sum of 1. These weighting coefficients are preset or dynamically adjusted according to the focus of different voyage phases. For example, during port arrival and departure, the mission factor... With threat factors The weight will increase accordingly. The representative vector construction function combines three weighted factors into a standardized multidimensional feature vector. This vector, representing the dynamic scene profile, is passed to the subsequent metacognitive scheduling module as the sole input for its decision-making.

[0042] Step 2: Input the dynamic scene profile into a metacognitive scheduling module for dynamic scheduling of models, filter out at least two candidate detection models, and use part of the multi-source perception data at the current moment as probe data to predict the expected performance of each candidate detection model in the current complete scene. Then, based on the expected performance, form an instant optimal model combination that is adapted to the current scene.

[0043] In a preferred embodiment, step 2 includes: inputting a dynamic scene profile into a metacognitive scheduling module for dynamic scheduling models; the metacognitive scheduling module selects at least two candidate detection models from a model repository based on the dynamic scene profile.

[0044] A portion of the multi-source sensing data at the current moment is used as probe data and input into each candidate detection model for fast forward inference. The internal confidence distribution data and feature activation region data generated by each candidate detection model when processing the probe data are obtained.

[0045] Calculate the overlap consistency measure between the feature activation region data corresponding to the same target in different candidate detection models.

[0046] By integrating internal confidence distribution data with overlap consistency metrics, the expected performance of each candidate detection model is generated.

[0047] From the candidate detection models, select the model with the highest expected performance in processing visible light images as the first model branch, and select the model with the highest expected performance in processing infrared images as the second model branch. The first model branch and the second model branch together constitute the instantaneous optimal model combination.

[0048] Specifically, the engineering implementation steps for predicting the expected performance of each candidate detection model in the current complete scene are as follows:

[0049] Step 2-1: Selection of Candidate Detection Models: The dynamic scene profile is input into the metacognitive scheduling module, which maintains a knowledge base of "scene features - model applicability". By calculating the similarity between the current scene profile and the optimal applicable scene features of each expert model in the knowledge base, such as cosine similarity, the metacognitive scheduling module quickly selects the 2 to N models with the highest similarity scores as candidate detection models.

[0050] Step 2-2, Fast forward inference based on probe data: In order to evaluate the actual performance of candidate models without consuming a lot of resources, the system extracts one or more key region slices with a size of 256x256 to 512x512 pixels from the visible light and infrared images at the current time as probe data.

[0051] The key region selection strategy includes: convolving the entire image using the Laplacian or Sobel operator to obtain a gradient map; applying an M×M sliding window to the image, calculating the number of pixels with gradient values ​​higher than a preset threshold within each window, and selecting the window region with the most gradient values ​​as the information-rich region; predicting the motion trajectory of the identified target in the previous frame using a Kalman filter to determine its predicted position in the current frame, and marking the region centered on this predicted position as the dynamic target region; comprehensively considering the information-rich region and the dynamic target region, prioritizing overlapping parts, or selecting the dynamic target region as probe data when there is no overlap.

[0052] Subsequently, the system performs a fast forward inference. For each candidate detection model, it records the original confidence vector generated by the softmax layer before the final output layer for each potential target. This vector constitutes the internal confidence distribution data.

[0053] Simultaneously, the system utilizes Grad-weighted class activation mapping (Grad-CAM) to visualize the input image region that the model is most interested in when making decisions, generating a heatmap with the same size as the probe data. Subsequently, the system binarizes the heatmap, marking pixel regions with activation values ​​higher than a preset threshold as foreground (1) and the rest as background (0), thus forming a binary feature activation mask, which is the final feature activation region data.

[0054] Steps 2-3: Cross-compare the feature activation region data from different candidate detection models:

[0055] For each candidate model, the system analyzes its original score vector output from inference on the probe data and the corresponding bounding box prediction results. Predictions with scores higher than a preset threshold are selected to generate a preliminary set of target candidate boxes. Each candidate box contains location information and the predicted category.

[0056] Using the candidate bounding box from one model as a reference, candidate bounding boxes from the other models are matched against it. The matching is based on the spatial overlap between candidate bounding boxes. For example, the Intersection over Union (IoU) is calculated. If the IoU between two candidate bounding boxes from different models is greater than a preset matching threshold, such as 0.5, then they are considered to point to the same potential target in the real world.

[0057] Calculate the IoU (Inter-Activation Union) between models: For each potential target matched by at least two models, the system performs the following operations:

[0058] Obtain the candidate box position corresponding to the target in each model.

[0059] Using the candidate bounding box as a mask, the activation region overlapping with the target bounding box is extracted from the binarized feature activation mask of the corresponding model.

[0060] Calculate the pairwise intersection-union ratio (IoU) between these activation regions extracted from different models that point to the same target.

[0061] The average IoU values ​​calculated for the same objective are averaged to obtain the average overlap consistency score of the objective among the models.

[0062] For example, if model A and model B both identify the same target, and their corresponding activation regions are Area_A and Area_B respectively, then the overlap consistency measure... The calculation is the intersection area of ​​Area_A and Area_B divided by the union area.

[0063] Steps 2-4: Generate the final expected performance score for each candidate detection model: For each candidate detection model, the system no longer calculates its general expected performance, but instead calculates its specific performance score when processing visible light probe data and infrared probe data.

[0064] The formula for calculating the performance score of visible light is: ;

[0065] The formula for calculating the infrared performance score is as follows: ;

[0066] in, and These are the highest confidence levels in the internal confidence distribution data of the candidate detection models when processing visible light and infrared probe data. and These are the average values ​​of the overlap consistency measures generated by the feature activation regions of the candidate detection model and all other candidate models for the same target when processing visible light and infrared probe data. and These are preset weighting coefficients, summing to 1, used to balance the certainty of the model's own judgments with the importance of consensus among multiple models. For example, in scenarios with clear signals, The weight can be set to 0.6 to 0.7, but in scenarios with severe noise interference, The weight can be appropriately increased to rely more on consensus.

[0067] After calculating the specific performance scores of all candidate models in both modalities, the system performs the following selection operation:

[0068] Selecting the first model branch: Among all candidate detection models, find the model with the highest visible light specific performance score and determine it as the first model branch for processing visible light images.

[0069] Selecting the second model branch: Among all candidate detection models, find the model with the highest infrared specific performance score and determine it as the second model branch for processing infrared images.

[0070] The selected first and second model branches together form the final, instantaneous optimal model combination, which is then loaded into memory to perform subsequent full scene detection tasks.

[0071] Step 3: Input the multi-source sensing data at the current moment into the instantaneous optimal model combination for collaborative feature extraction and target detection, and generate preliminary collaborative detection results containing target location, category, confidence level and low modality consistency label.

[0072] In a preferred embodiment, the multi-source sensing data at the current moment is input into the instantaneous optimal model combination for collaborative feature extraction and target detection, generating preliminary collaborative detection results including target location, category, confidence level and low modality consistency markers, including: acquiring a first attention heatmap generated by a first model branch that processes visible light images, and a second attention heatmap generated by a second model branch that processes infrared images.

[0073] The first attention heatmap and the second attention heatmap are quantitatively analyzed to generate a high attention mask and a low response mask. The high attention mask is defined as the region in the heatmap where the activation value is higher than its own maximum activation value by a first preset proportion, and the low response mask is defined as the region in the heatmap where the activation value is lower than its own maximum activation value by a second preset proportion.

[0074] Disputed areas are identified by performing logical operations on high-attention masks and low-response masks.

[0075] The disputed region is a region that meets one of the following conditions: it is a region that simultaneously belongs to the high attention mask of the first attention heatmap and the low response mask of the second attention heatmap, or it is a region that simultaneously belongs to the high attention mask of the second attention heatmap and the low response mask of the first attention heatmap.

[0076] During the feature fusion stage, the features corresponding to the disputed region are multiplied by a preset suppression coefficient; and for targets whose bounding boxes overlap with the disputed region in the final detection result, a preset low-modal consistency label is added.

[0077] Specifically, the multi-source sensing data at the current moment is input into the instantaneous optimal model combination for collaborative feature extraction and target detection. The engineering implementation steps are as follows:

[0078] Step 3-1: Obtain the first and second attention heatmaps: For the first model branch processing visible light images and the second model branch processing infrared images, which run in parallel in the instantaneous optimal model ensemble, the system uses gradient-weighted class activation mapping (JEM) to apply to their respective final convolutional layers. This operation generates two original heatmaps for each detected target category: the first attention heatmap and the second attention heatmap. The pixel values ​​of these two heatmaps reflect the degree of attention the model pays to each region in the input image when making classification decisions. To facilitate subsequent calculations, the system normalizes the pixel values ​​of these two heatmaps to the range of 0 to 1.

[0079] Step 3-2, Quantitative identification of disputed regions: Threshold segmentation is performed on the normalized first attention heatmap and the second attention heatmap:

[0080] Pixel regions in the heatmap with activation values ​​higher than 70% of their maximum activation value are marked as foreground with a value of 1; the rest are marked as background with a value of 0, generating high-interest masks, specifically including visible light high-interest masks and infrared high-interest masks.

[0081] Pixel regions in the heatmap whose activation value is less than 20% of their maximum activation value are marked as foreground with a value of 1; the rest are marked as background with a value of 0, generating low-response masks, specifically including visible light low-response masks and infrared low-response masks.

[0082] Disputed regions are identified by performing a bitwise logical AND operation on the four types of binary masks mentioned above. A pixel is considered part of a disputed region if it meets one of the following conditions: The pixel is 1 in the high-interest mask of the first model branch and 1 in the low-response mask of the second model branch. Alternatively, the pixel is 1 in the high-interest mask of the second model branch and 1 in the low-response mask of the first model branch. All pixels that meet these conditions together constitute the final disputed region mask.

[0083] Step 3-3: Implement feature weight suppression based on disputed regions and generate low-modal consistency markers: Before feature fusion, the system uses the disputed region mask as a guide to perform suppression operations on the original feature map. Specifically, this is implemented as follows:

[0084] Generate a suppression weight mask with the same size as the feature map. The pixel value of this suppression weight mask is a preset suppression coefficient, such as 0.3, in the disputed region where the mask is 1, and the pixel value is 1.0 in the disputed region where the mask is 0.

[0085] Original feature map With suppression weight mask Element-wise multiplication is performed to obtain the modulated feature map. : ,in, This means multiplying the original feature map and the corresponding elements in the suppression weight mask one by one to generate a new feature map of the same size; This is a preset base weight, usually set to 0.4 to 0.6; This is a preset modulation weight, typically set to 0.4 to 0.6; and and The sum is 1. This operation weakens the feature contribution from the disputed region while keeping the features of the consensus region unchanged.

[0086] The system will assign the modulated first model branch to the corresponding feature map. Feature map corresponding to the second model branch The features are fused to generate a unified, multimodal fusion feature map. The fusion method can employ element-wise addition, splicing, or more complex attention fusion mechanisms.

[0087] Steps 3-4: Generate detection results: Combine the fused feature maps The input is fed into the model's detection head. The detection head typically consists of several convolutional and fully connected layers, and its function is to parse and fuse the feature maps, outputting a series of initial prediction boxes containing the following information:

[0088] Bounding box coordinates: predict the position and size of the target.

[0089] Category score: The raw score given for each possible target category.

[0090] Confidence score: The score used to determine whether an object exists within the predicted bounding box. The system then applies non-maximum suppression to these initial predicted bounding boxes to eliminate redundant detections of the same object, ultimately generating a set of candidate detection results. Each candidate detection result contains a clear object location, the most likely category, and the corresponding confidence score.

[0091] After generating preliminary collaborative detection results, the system calculates the overlap between the bounding box of each detected target and the mask of the disputed region. If the bounding box of a target overlaps with the disputed region by any number of pixels, the system adds a binary low-modal consistency flag with a value of 1 to the detection result of that target, for further review by the subsequent physical rule verification module.

[0092] The engineering objective of this process is to proactively identify and suppress potential false alarms caused by single-modal noise or artifacts through mutual questioning between different sensor perspectives in the early stages of feature fusion, thereby improving the signal-to-noise ratio and reliability of the fused features.

[0093] Step 4: Perform predictive physical rule verification on each target in the preliminary collaborative detection results, correct the preliminary collaborative detection results based on the verification results, and output the final target recognition results.

[0094] In a preferred embodiment, predictive physical rule verification is performed on each target in the preliminary collaborative detection results, including: acquiring the Automatic Identification System (AIS) signal of the ship and associating and matching it with the targets in the preliminary collaborative detection results.

[0095] Verify the consistency between the visual motion trajectory of the successfully associated target and its AIS reported trajectory to determine if it exceeds a trajectory difference threshold.

[0096] When the trajectory difference threshold is exceeded, the target's identification result is marked as a state of separation between AIS and visual signal, and a multi-source fusion tracking process to improve tracking robustness is triggered.

[0097] Specifically, the engineering implementation steps for performing predictive physical rule verification on each target in the preliminary collaborative detection results are as follows:

[0098] Step 4-1.1: Perform visual target and AIS target association matching. The engineering objective is to establish a unique correspondence between massive AIS broadcast data and visually detected targets. The system continuously receives and parses AIS signals broadcast by surrounding vessels, extracting AIS data frames containing the vessel's unique identification code (MMSI), latitude and longitude, speed of travel (SOG), and heading of travel (COG). For each visual target in the initial cooperative detection results, the system uses a Kalman filter to predict its latitude and longitude position at the current moment. Then, the system searches for AIS targets within a circular search area with a radius of 50 to 200 meters, centered on this predicted position. If a unique AIS target exists within this area, and the vessel type reported by the AIS target matches the category of the visual detection, the association matching is successful. The system assigns a corresponding MMSI to the visual target, completing the identity binding.

[0099] Step 4-1.2, Trajectory Consistency Verification: The system maintains two parallel trajectory data streams for each successfully associated target: one is the visual motion trajectory updated based on the preliminary collaborative detection results of multiple consecutive frames; the other is the AIS report trajectory updated based on the consecutive AIS data frames corresponding to the target's MMSI. The system calculates the average heading difference and average speed difference between the two trajectories within the most recent 10 to 30-second time window. When the average heading difference exceeds 5 to 15 degrees, or the average speed difference exceeds 2 to 5 knots, the system determines that the trajectory inconsistency exceeds a preset threshold.

[0100] Step 4-1.3: When inconsistencies in the detected trajectories are identified, a specific marking and response process is triggered. The engineering purpose is to mark the identified abnormal states and initiate a more robust algorithm to attempt to resolve data conflicts. The system marks the target's identification result as a state of separation between AIS and visual signals. This marking not only serves to highlight a warning to the driver on the user interface, but more importantly, it triggers the system to switch to a higher-confidence multi-source fusion tracking process. In this process, the system increases the weight of other non-visual sensors such as radar, or employs the interactive multi-model (IMM) algorithm. This algorithm can run multiple motion models in parallel, such as a uniform linear model and a coordinated turning model, and dynamically adjusts the probability weights of each model based on the real-time fit of the sensor data to more accurately track the target's true motion state and attempt to re-establish a stable correlation between vision and AIS.

[0101] In a further preferred embodiment, the preliminary collaborative detection results are corrected based on the verification results, and the final target recognition results are output, including: physical unreachable markers generated in the aggregated predictive physical rule verification, AIS and visual signal separation markers, and low modality consistency markers, and a comprehensive correction coefficient is generated.

[0102] The confidence level of the corresponding target in the preliminary collaborative detection results is weighted and adjusted using a comprehensive correction coefficient.

[0103] The measured noise parameters in a kinematic filter used to smooth the target trajectory are adjusted based on the comprehensive correction coefficients to correct the target's position.

[0104] Specifically, the preliminary collaborative detection results are corrected based on the verification results, and the final target recognition results are output. The engineering implementation steps are as follows:

[0105] Step 4-2.1: The system generates a comprehensive correction coefficient: The system collects all anomaly markers associated with a specific target. These markers include: a physically inaccessible marker generated when the target's location is outside its physical reach; an AIS and visual signal separation marker generated when the target's visual trajectory and AIS trajectory are significantly separated; and a low modal consistency marker generated by cross-modal attention verification in the previous step. The system presets a basic penalty weight for each marker. For example, the weight of the physically inaccessible marker is set to 0.8 to 1.0, the weight of the AIS and visual signal separation marker is set to 0.5 to 0.7, and the weight of the low modal consistency marker is set to 0.2 to 0.4. Correction Coefficient The following formula is used to calculate the effect to ensure that it is linear, rather than a simple linear superposition:

[0106]

[0107] in, , , The value indicates whether the above three flags exist; 1 indicates existence, and 0 indicates otherwise. , , These are the corresponding penalty weights. (Correction coefficient) The value ranges from 0 to 1. The smaller the value, the lower the credibility of the target and the greater the correction required.

[0108] Step 4-2.2, Weighted Adjustment: For an initial confidence level of... The target, its adjusted confidence level Obtained through the following methods:

[0109] ;

[0110] If the adjusted confidence level is lower than a preset final output threshold, such as 0.4, the target will be removed from the final target identification results. This step ensures that only targets that simultaneously meet the requirements of the data-driven model and physical rule verification can be identified as valid targets.

[0111] Step 4-2.3: The system performs kinematic model-based smoothing correction on the target positions of specific anomaly markers. This correction primarily targets those marked as having AIS separation from the visual signal or low modal consistency. The system activates an extended Kalman filter, which uses a state vector containing position, velocity, and acceleration to describe the target motion. At each time step, when a new visual measurement arrives—that is, a new position appears in the initial cooperative detection results—the system does not directly adopt it but compares it with the current position predicted based on the state at the previous time step. Correction coefficients. The noise covariance used to adjust the measurement value. A smaller one. The higher value indicates that the system considers the current visual measurement to be less reliable, thus increasing its noise covariance. This causes the Kalman filter to rely more on its own motion model predictions than on the measurements of the current frame. This produces a smoothing effect, helping to suppress abrupt changes in position that do not conform to kinematics, thereby generating a smoother and more physically reliable final target position.

[0112] Step 5: Based on the final target recognition results of multiple consecutive frames and the ship's navigation parameter sensor data, perform time series analysis and trajectory prediction to generate a dynamic risk situation map for the future preset time period, and trace the source and cause of risk events in the dynamic risk situation map to generate risk attribution information.

[0113] In a preferred embodiment, based on the final target identification results of multiple consecutive frames and the ship's navigation parameter sensor data, time series analysis and trajectory prediction are performed to generate a dynamic risk situation map for a future preset period, including: establishing a track file for the ship and each target containing historical position, speed, and heading data, and predicting the trajectories of the ship and each target for a future preset period.

[0114] Based on the predicted trajectory, the nearest encounter distance between the ship and each target and the remaining time to reach the nearest encounter point are calculated; and based on the comparison results with the preset safety threshold, a collision risk index is generated by quantification through a weighted penalty function.

[0115] The predicted trajectories of all targets and their risk levels as determined by the collision risk index are overlaid on the electronic nautical chart using different colored highlights to form a dynamic risk situation map.

[0116] Specifically, the engineering objective of generating a dynamic risk situation map in this invention is to transform the discrete final target identification results across multiple consecutive frames into continuous, forward-looking predictions of future traffic situations. By quantifying collision risks, it provides intuitive and actionable decision support for ship operators or autonomous navigation systems. The engineering implementation steps are as follows:

[0117] Step 5-1.1: This process is executed by a time series analysis and prediction module. This module first establishes a track file containing the historical status of the past 1 to 3 minutes for the ship and each stably tracked target within its field of view. This file records the final target identification results for multiple consecutive frames, namely the target's position, speed, heading, category, and other time series data, as well as the ship's own navigation parameter sensor data.

[0118] Step 5-1.2: The system performs time-series analysis and trajectory prediction based on these track archives: For each target, the system employs a kinematic model, such as a constant speed and rate of turn (CTRV) model, and estimates and predicts the target's motion state using an Extended Kalman Filter (EKF) or an Unscented Kalman Filter (UKF). This filter uses historical track data to smooth measurement noise and estimate the target's current motion state, including its potential rate of turn, and then propagates this state forward to generate a predicted trajectory for a preset time period, typically 3 to 10 minutes. This predicted trajectory is a series of future location points with timestamps. Similarly, the system also predicts the ship's trajectory within the same preset time period based on the ship's navigation parameter sensor data.

[0119] Step 5-1.3: After obtaining the predicted trajectories of the vessel and all surrounding targets, the system performs a dynamic risk assessment to generate a dynamic risk situation map. The core of this assessment is to calculate the nearest encounter distance (DCPA) and the remaining time to reach the nearest encounter point (TCPA) for each vessel-target vessel pair within the predicted time period. When the DCPA of a pair of vessels is less than a preset safe distance threshold (e.g., 0.5 to 1.0 nautical miles) and the TCPA is less than a preset safe time threshold (e.g., 12 minutes), the system determines that there is a collision risk. The system further quantifies this risk into a risk index R, the calculation of which can be expressed by the following formula:

[0120] ;

[0121] in, For this ship With the target ship The risk index between them. and These are the calculated DCPA and TCPA values, respectively. and It is a preset safe distance and time threshold. and It is a weighting coefficient used to adjust the importance of distance and time. and It is a penalty function, such as an exponential function, used to amplify the risk level when DCPA or TCPA is much smaller than the safety threshold.

[0122] Step 5-1.4: The system overlays the predicted trajectories of all targets and their risk levels calculated based on the risk index R onto the electronic nautical chart, forming a dynamic risk situation map. In this map, targets with different risk levels are highlighted with different colors; for example, high-risk targets are marked in red, and medium-risk targets in yellow. The system can also dynamically display their predicted paths and potential collision points, thus providing operators with clear warnings of future risks.

[0123] In a further preferred embodiment, risk events in the dynamic risk situation map are traced back to their causes to generate risk attribution information, including: analyzing the causes of high-risk events, such as the target being in the blind zone of multi-sensor fusion, the target's trajectory undergoing a sudden change, the target being a low-visibility small target, or the current environmental conditions causing a decline in the performance of specific sensors.

[0124] One or more of the causes are determined to be the primary cause.

[0125] By associating high-risk events with their corresponding primary causes, structured risk attribution information is generated.

[0126] Specifically, the engineering implementation steps for tracing and attributing the risk events in the dynamic risk situation map to generate risk attribution information are as follows:

[0127] Step 5-2.1: Perform parallel analysis of multi-dimensional causes: The engineering purpose is to collect evidence related to the current high-risk event from various stages of system operation. For targets identified as high-risk in the dynamic risk situation map, the system triggers a series of parallel diagnostic subroutines.

[0128] The first subroutine analyzes sensor fusion performance to determine if the target is in a blind zone of multi-sensor fusion. The system checks whether, in the past 3 to 5 seconds, the detection of the target has relied on a single sensor for more than 70% of the time; for example, only visible light is detected while infrared and radar do not respond. If so, it is marked as a "sensor fusion blind zone".

[0129] The second subroutine analyzes the target's dynamic behavior to determine if its trajectory has changed abruptly. The system calculates the target's heading angular velocity and acceleration within the most recent time window. If these values ​​exceed a preset threshold allowed by the physical performance of this type of vessel, such as a small speedboat having a turning angular velocity exceeding 10 degrees per second, it is marked as a "sudden change in trajectory".

[0130] The third subroutine analyzes the target's own attributes to determine whether it belongs to a low-visibility small target. Based on the category label in the final target recognition result and combined with the pixel area it occupies in the image, if the target category is a buoy, small fishing boat, etc., and the pixel area is less than 0.1% of the total image, it is marked as a "low-visibility small target".

[0131] The fourth subroutine analyzes the current environmental impact to determine if there are conditions that cause a specific sensor to degrade in performance. The system retrieves the dynamic scene profile at that moment. If environmental features, such as visibility level below 2 or light intensity level in a strong backlight condition, are present, it is marked as "environmental conditions causing sensor performance degradation".

[0132] Step 5-2.2, Correlation and Determination of Primary Causes: The engineering objective is to identify one or more primary causes that contribute the most to the current risk event from multiple possible triggers. The system assigns a base contribution score to each possible cause. Then, it weights the cause based on its duration, intensity, or frequency. For example, a persistent sensor fusion blind spot receives a higher weight than a momentary change in motion trajectory. The system aggregates all triggered cause tags and their weighted scores, selecting the one or two with the highest scores as primary causes.

[0133] Step 5-2.3: Generate structured risk attribution information. The engineering purpose is to encapsulate the analysis results into a standardized data package for subsequent modules to use. This risk attribution information is a data structure containing multiple fields. Its core fields include target ID, risk level, and one or more associated primary cause codes. For example, for the target "Fishing Vessel 001," its risk attribution information might be {TargetID:F001,RiskLevel:High,ReasonCodes:[SCB-01,MTP-02]}, where SCB-01 represents "sensor fusion blind spot" and MTP-02 represents "abrupt movement trajectory." This structured information package is sent to the system's closed-loop optimization and evolution module as the core basis for its decision-making.

[0134] Step 6: Based on risk attribution information and the system performance evaluation results of the current cycle, adjust the data acquisition parameters of multiple sensors in reverse, and update the prior knowledge of the metacognitive scheduling module for predicting performance and the model composition in the model warehouse to optimize the perception and recognition performance of the next processing cycle.

[0135] In a preferred embodiment, the data acquisition parameters of multiple sensors are adjusted in reverse based on risk attribution information and the system performance evaluation results of the current period, including: analyzing risk attribution information to identify repetitive sensor failure modes related to specific environmental conditions.

[0136] When a repetitive sensor failure mode is identified, a parameter adjustment instruction for that sensor is generated from a strategy library containing adjustment strategies.

[0137] In the next processing cycle, the corresponding sensors are controlled to collect data according to the parameter adjustment instructions.

[0138] Specifically, based on risk attribution information and the current period's system performance evaluation results, the data acquisition parameters of multiple sensors are adjusted in reverse. The engineering implementation steps are as follows:

[0139] Step 6-1.1: The system performs continuous attribution pattern analysis. Its engineering purpose is to identify recurring sensor failure modes strongly correlated with specific environmental conditions from a continuous stream of risk events. The closed-loop optimization and evolution module receives and caches all risk attribution information from the past 1 to 5 minutes in real time. The system performs statistical analysis on this information to find frequently occurring correlation pairs, i.e., combinations of "specific environmental conditions" and "specific sensor performance degradation." For example, if, within a sliding time window of 1 to 5 minutes, the cumulative number of occurrences of risk attribution information for a specific cause, such as "visible light camera performance degradation under strong backlight," exceeds a preset frequency threshold, such as 5 to 10 times, the system determines that the sensor has a persistent performance bottleneck under this specific environment and triggers a parameter adjustment process.

[0140] Step 6-1.2: Based on the preset adjustment strategy library, generate specific parameter adjustment instructions. The engineering purpose is to transform abstract performance bottleneck diagnoses into precise quantitative operation instructions for hardware parameters. This adjustment strategy library is stored in the form of "IF-THEN-ELSE" rules, mapping diagnosed problems to solutions. For example, when the system determines that "the visible light camera's performance degrades under strong backlight," a rule is triggered, generating an instruction to lower the corresponding camera's automatic exposure compensation value by -0.5EV to -1.0EV and enable the high dynamic range (HDR) mode for multi-frame synthesis. Similarly, if risk attribution information frequently points to "insufficient thermal contrast of the infrared thermal imager in sea fog," the system generates an instruction to increase the execution frequency of its non-uniformity correction NUC from once every 30 minutes to once every 5 minutes and increase its digital gain by 10% to 20%. The generated parameter adjustment instruction is a standardized data packet containing the target sensor device address, the address of the parameter register to be adjusted, and the new parameter value.

[0141] Step 6-1.3: The system executes the parameter adjustment command at the start of the next processing cycle. Its engineering purpose is to ensure that optimization measures are applied immediately at the source of data acquisition, thereby improving the original quality of the next frame image. This parameter adjustment command is sent to the ship's sensor management bus. Before the synchronization signal triggers acquisition in the next processing cycle, the corresponding sensor driver parses the command and updates the hardware registers. For example, the camera's ISP image signal processor applies the new exposure settings. Subsequently, the sensor acquires data according to the new parameter configuration and sends the optimized image data stream to the subsequent sensing and recognition processes, thus completing a full closed-loop optimization. A decrease in the incidence of risk events due to this attribution is expected to be observed in subsequent system performance evaluations.

[0142] The engineering objective of this process is to establish an adaptive feedback loop from the identification result to the data acquisition source, enabling the system to proactively address the performance degradation of specific sensors caused by environmental changes, thereby achieving continuous self-correction of sensing capabilities.

[0143] In a further preferred embodiment, updating the prior knowledge of the metacognitive scheduling module for predictive performance and the model composition in the model repository includes: storing the dynamic scene profile of the current period and the actual recognition performance evaluation of the optimal model combination in real time as a new experience data record in a scene-model performance experience library.

[0144] Based on the hard sample screening rules, a batch of hard sample data is automatically labeled from cases with poor identification results.

[0145] Incremental learning training can be performed on specific models in the model repository using difficult sample data, or new expert models can be trained and added to the model repository.

[0146] Specifically, the engineering implementation steps for updating the metacognitive scheduling module to use prior knowledge for prediction performance and the model composition in the model repository are as follows:

[0147] Step 6-2.1, System Execution Scenarios - Accumulation of Model Performance Experience: Its engineering purpose is to build a dynamically updated knowledge base, so that the decision-making of the metacognitive scheduling module can evolve from being based on static preset rules to being based on massive amounts of empirically verified experience data.

[0148] At the end of each processing cycle, the closed-loop optimization and evolution module records a set of data, forming a new empirical data record. This record is a structured data tuple, containing at least: the complete feature vector of the dynamic scene profile for the current cycle; the identifier of the optimal model combination ultimately selected in that cycle; and an evaluation of the actual recognition performance of that combination in this scene. The evaluation of actual recognition performance is a comprehensive score, calculated by weighting multiple indicators such as the false negative rate, false positive rate, and target position tracking stability of the final target recognition result.

[0149] This new experience data record is appended and stored in a database called the Scene-Model Performance Experience Library. When performing virtual performance evaluations, the metacognitive scheduling module will first query this experience library. If it finds a historical record with a similarity of more than 90% to the current dynamic scene profile, it can directly adopt the model combination with the highest performance score in the historical record, thereby significantly improving scheduling efficiency and accuracy.

[0150] Step 6-2.2, Automatic Hard Sample Screening and Labeling: The engineering purpose is to accurately locate the cognitive shortcomings of the current model combination and prepare high-quality training data for targeted model optimization. The system sets a set of hard sample screening rules and continuously monitors the recognition results. These rules include: First, targets marked as abnormal by multiple physical rule verification steps, but with an initial confidence level higher than 0.8; second, targets that appear and disappear in multiple consecutive frames, resulting in discontinuous trajectories; third, recognition results that are frequently manually corrected by human drivers through the interactive interface. Once a sample triggers any of the above rules, the system automatically collects the original multi-source perception data slice corresponding to the sample, along with its correct category and location label, into a dedicated hard sample database.

[0151] Step 6-2.3: Trigger incremental learning or iterative training of the model using accumulated difficult sample data: The engineering purpose is to fix the defects of existing models and expand the capabilities of the model repository. When the number of samples in a specific category or scenario in the difficult sample database reaches a preset threshold, such as 500 to 1000 samples, the system will trigger the background training process. For existing expert models, the system uses incremental learning techniques to fine-tune specific levels of the model using these difficult samples to improve its performance in these difficult scenarios without retraining the entire model. In addition, if the accumulated difficult samples present a completely new scenario that the model repository does not yet have a dedicated model for, such as "a small drone in the rain at night," the system will use these samples to train a new expert model specializing in this scenario from scratch. After training, this new model or the fine-tuned model will undergo a series of offline evaluations. If it passes, it will be automatically deployed to the model repository, thereby enabling the model composition to expand itself as the needs of real-world applications grow.

[0152] The engineering goal of this process is to enable the system to learn and evolve from continuous operational experience, so that its model scheduling strategy and the model's own performance can be continuously optimized over time and as the number of scenarios encountered increases.

[0153] The above content is merely an example and illustration of the concept of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the concept of the invention or exceed the scope defined by the present invention, and all such modifications and additions should fall within the protection scope of the present invention.

Claims

1. A method for ship visual perception and target identification based on deep learning, characterized in that, include: Step 1: Collect multi-source sensing data and electronic chart information at the current moment through multiple sensors deployed on the ship, and generate a dynamic scene profile representing the current comprehensive environment through multi-dimensional feature extraction; Step 2: Input the dynamic scene profile into a metacognitive scheduling module for dynamic scheduling models, filter out at least two candidate detection models, and use a portion of the multi-source perception data at the current moment as probe data to predict the expected performance of each candidate detection model in the current complete scene. Then, based on the expected performance, form an instant optimal model combination that is adapted to the current scene. Step 2 includes: The dynamic scene profile is input into a metacognitive scheduling module for dynamic scheduling models; the metacognitive scheduling module selects at least two candidate detection models from a model repository based on the dynamic scene profile. A portion of the multi-source sensing data at the current moment is used as probe data and input into each candidate detection model for fast forward inference. The internal confidence distribution data and feature activation region data generated by each candidate detection model when processing the probe data are obtained. Calculate the overlap consistency measure between feature activation region data corresponding to the same target in different candidate detection models; By integrating internal confidence distribution data with overlap consistency metrics, the expected performance of each candidate detection model is generated. From the candidate detection models, select the model with the highest expected performance in processing visible light images as the first model branch, and select the model with the highest expected performance in processing infrared images as the second model branch. The first model branch and the second model branch together constitute the instantaneous optimal model combination. Step 3: Input the multi-source sensing data at the current moment into the instantaneous optimal model combination for collaborative feature extraction and target detection, and generate preliminary collaborative detection results including target location, category, confidence level and low modality consistency label; The process involves inputting multi-source sensing data at the current moment into an instantaneous optimal model for collaborative feature extraction and target detection, generating preliminary collaborative detection results including target location, category, confidence level, and low-modal consistency markers. Obtain the first attention heatmap generated by the first model branch for processing visible light images, and the second attention heatmap generated by the second model branch for processing infrared images; Quantitative analysis is performed on the first attention heatmap and the second attention heatmap respectively to generate high attention mask and low response mask; The disputed area was identified by performing logical operations on the high-attention mask and the low-response mask; The disputed region is a region that meets one of the following conditions: it is a region that simultaneously belongs to the high attention mask of the first attention heatmap and the low response mask of the second attention heatmap, or it is a region that simultaneously belongs to the high attention mask of the second attention heatmap and the low response mask of the first attention heatmap. During the feature fusion stage, the features corresponding to the disputed region are multiplied by a preset suppression coefficient; and for targets whose bounding boxes overlap with the disputed region in the final detection result, a preset low-modal consistency marker is added. Step 4: Perform predictive physical rule verification on each target in the preliminary collaborative detection results, correct the preliminary collaborative detection results based on the verification results, and output the final target recognition results; Step 5: Based on the final target recognition results of multiple consecutive frames and the ship's navigation parameter sensor data, perform time series analysis and trajectory prediction to generate a dynamic risk situation map for a future preset time period, and trace and attribute the risk events in the dynamic risk situation map to generate risk attribution information. Step 6: Based on risk attribution information and the system performance evaluation results of the current cycle, adjust the data acquisition parameters of multiple sensors in reverse, and update the prior knowledge of the metacognitive scheduling module for predicting performance and the model composition in the model warehouse to optimize the perception and recognition performance of the next processing cycle.

2. The deep learning-based ship visual perception and target recognition method according to claim 1, characterized in that, The process involves collecting multi-source sensing data and electronic chart information from multiple sensors deployed on the ship, and generating a dynamic scene profile representing the current comprehensive environment through multi-dimensional feature extraction, wherein: Multi-source sensing data includes at least visible light images, infrared images, and data from ship navigation parameter sensors; Electronic nautical charts contain geographic layers and navigation rules; Analyze visible light and infrared images to obtain the visibility level and light intensity level of the current environment; By matching the position information in the navigation parameter sensor data with the electronic nautical chart information, the current regional mission attributes of the vessel can be determined. By analyzing historical final target identification results, the current target density and velocity distribution characteristics in the surrounding area can be obtained; By integrating visibility level, light intensity level, regional task attributes, and target density and speed distribution characteristics, a dynamic scene profile is generated.

3. The deep learning-based ship visual perception and target recognition method according to claim 1, characterized in that, The step of performing predictive physical rule verification on each target in the preliminary collaborative detection results includes: Acquire the Automatic Identification System (AIS) signal of the ship and associate and match it with the target in the preliminary collaborative detection results; Verify the consistency between the visual motion trajectory of the successfully associated target and its AIS reported trajectory to determine whether it exceeds a trajectory difference threshold; When the trajectory difference threshold is exceeded, the target's identification result is marked as a state of separation between AIS and visual signal, and a multi-source fusion tracking process to improve tracking robustness is triggered.

4. The deep learning-based ship visual perception and target recognition method according to claim 3, characterized in that, The step of correcting the preliminary collaborative detection results based on the verification results and outputting the final target recognition results includes: The physical unreachability markers, AIS and visual signal separation markers, and low modality consistency markers generated during the aggregation predictive physical rule verification are used to generate a comprehensive correction coefficient. The confidence levels of the corresponding targets in the preliminary collaborative detection results are weighted and adjusted using a comprehensive correction coefficient. The measured noise parameters in a kinematic filter used to smooth the target trajectory are adjusted based on the comprehensive correction coefficients to correct the target's position.

5. The deep learning-based ship visual perception and target recognition method according to claim 1, characterized in that, The final target recognition results based on multiple consecutive frames and the ship's navigation parameter sensor data are used to perform time-series analysis and trajectory prediction to generate a dynamic risk situation map for a future preset time period, including: Establish track archives for the ship and each target, including historical position, speed, and heading data, and predict the trajectories of the ship and each target within a predetermined time period in the future; Based on the predicted trajectory, the nearest encounter distance between the ship and each target and the remaining time to reach the nearest encounter point are calculated; and based on the comparison results with the preset safety threshold, a collision risk index is generated by quantification through a weighted penalty function. The predicted trajectories of all targets and their risk levels as determined by the collision risk index are overlaid on the electronic nautical chart using different colored highlights to form a dynamic risk situation map.

6. The deep learning-based ship visual perception and target recognition method according to claim 1, characterized in that, The process of tracing the sources and attributing risks in the dynamic risk situation map to generate risk attribution information includes: The causes of high-risk events are analyzed, including the target being in the blind zone of multi-sensor fusion, the target's trajectory changing abruptly, the target being a small, low-visibility target, or the sensor performance being degraded due to current environmental conditions. One or more of the causes are determined to be the primary cause; By associating high-risk events with their corresponding primary causes, structured risk attribution information is generated.

7. The deep learning-based ship visual perception and target recognition method according to claim 1, characterized in that, The data acquisition parameters of multiple sensors are adjusted in reverse based on risk attribution information and the system performance evaluation results of the current period, including: Analyze risk attribution information to identify repetitive sensor failure modes related to environmental conditions; When a repetitive sensor failure mode is identified, a parameter adjustment instruction for that sensor is generated from a strategy library containing adjustment strategies. In the next processing cycle, the corresponding sensors are controlled to collect data according to the parameter adjustment instructions.

8. The deep learning-based ship visual perception and target recognition method according to claim 1, characterized in that, The updated metacognitive scheduling module is used to construct prior knowledge of prediction performance and models in the model repository, including: The dynamic scene profile of the current period and the actual recognition performance evaluation of the combination of the best models in real time are stored as a new empirical data record in a scene-model performance experience library. Based on the hard sample screening rules, a batch of hard sample data is automatically labeled from cases with poor identification results; Incremental learning training can be performed on models in the model repository using difficult sample data, or new expert models can be trained and added to the model repository.

Images