Navigation mark visual identification and collision early warning method and system based on deep learning

Abstract

The invention discloses a navigation mark visual identification and collision early warning method and system based on deep learning, and relates to the technical field of intelligent shipping, and the method comprises the steps: collecting a navigation channel monitoring video stream in real time, and carrying out the preprocessing of a key image frame; constructing a navigation mark detection model based on an improved Officient Det network, and outputting the position and category of the navigation mark in the key image frame; the method comprises the following steps: constructing a ship track prediction model through ship historical track data, predicting a short-time track of a ship, calculating the relative position, speed and course angle of the ship and a navigation mark, and constructing a dynamic collision risk field; and fusing the visual detection result, the radar ranging data and the AIS information to obtain a ship collision risk, and triggering graded early warning based on a preset risk threshold. According to the method, the improved deep learning model and the real-time calculation framework are combined, the robustness of navigation mark identification is improved, an efficient collision early warning mechanism is established, and technical support is provided for intelligent shipping.

Description

Technical Field

[0001] This invention relates to the field of smart shipping technology, and more specifically, to a method and system for visual recognition and collision warning of navigation marks based on deep learning. Background Technology

[0002] With the rapid development of the shipping industry, waterway traffic is becoming increasingly busy, and the risk of ship collisions is also increasing. As important navigational aids, navigation marks provide ships with navigation, positioning, and warning functions, and their accurate identification and status monitoring are crucial for ensuring navigational safety. Traditional navigation mark identification mainly relies on manual observation or sensor technologies such as radar and AIS, but these methods suffer from low identification accuracy and insufficient real-time performance in complex weather conditions or dense waterway environments. Furthermore, traditional collision warning systems are mostly based on rule-based judgment or simple sensor fusion, making it difficult to cope with dynamically changing navigation environments.

[0003] In recent years, advancements in computer vision and deep learning technologies have provided new solutions for navigation mark recognition and collision warning. Convolutional neural network-based target detection algorithms can efficiently extract navigation mark features from images or video streams, achieving high-precision real-time detection. Furthermore, combining semantic segmentation techniques can further refine the contour and positional information of navigation marks, improving recognition robustness in complex backgrounds. In collision warning, deep learning models based on time-series data can analyze the spatial relationship between ship trajectories and navigation marks, predict potential collision risks, and integrate multi-sensor data (such as GPS and radar) to achieve more reliable warning decisions. However, existing methods face technical challenges such as insufficient environmental adaptability, difficulty in detecting small targets, and difficulty meeting real-time requirements. Therefore, focusing on lightweight networks, multimodal data fusion, and dynamic collision avoidance strategies based on reinforcement learning is an urgent problem to be solved. Summary of the Invention

[0004] To address the aforementioned technical issues, this invention proposes a deep learning-based method and system for visual recognition and collision warning of navigation marks. By combining an improved deep learning model with a real-time computing framework, the robustness of navigation mark recognition is enhanced, and an efficient collision warning mechanism is established, providing technical support for intelligent shipping.

[0005] The first aspect of this invention provides a method for visual recognition and collision warning of navigation marks based on deep learning, comprising the following steps: The ship's onboard camera collects real-time channel monitoring video streams, extracts key image frames from the channel monitoring video streams, and preprocesses the key image frames. A navigation mark detection model is constructed based on an improved Efficient Det network. Multi-level features of key image frames are extracted and processed. A bidirectional feature pyramid network is used for feature fusion to output the location and category of the navigation mark. By constructing a ship trajectory prediction model using historical ship trajectory data, the model predicts the ship's short-term trajectory. Based on the predicted trajectory information, the relative position, speed, and heading angle of the ship and navigation marks are calculated, and a dynamic collision risk field is constructed. By integrating visual inspection results, radar ranging data, and AIS information, ship collision risks are obtained, and graded early warnings are triggered based on preset risk thresholds.

[0006] In this solution, a real-time video stream of waterway monitoring is acquired using a camera mounted on the ship. Key image frames are extracted from the video stream, and these key image frames are preprocessed, specifically as follows: During ship navigation, the ship's high-definition camera continuously collects channel monitoring video streams based on preset video resolution and frame rate. Based on the characteristics of the video stream, the video stream frames are classified according to environmental conditions, and environmental tags are added according to the environmental conditions. The image entropy of the channel monitoring video stream is calculated frame by frame. A dynamic image entropy threshold is set according to the environmental label. Image frames with image entropy greater than the preset dynamic image entropy threshold are selected. Adjacent image frames are marked in the selected image frames. Image frames with low image entropy in the marked image frames are removed. Key image frames are output. Based on the environmental labels of key image frames, a differentiated preprocessing strategy is adopted. A preprocessing method library is constructed according to the preprocessing examples of waterway monitoring, and image quality labels are set in the preprocessing method library. The optimal combination is adaptively selected from the preprocessing method library using environmental labels. The parameters of each preprocessing method in the optimal combination are adjusted according to the image features of the key image frame. The preprocessing effect evaluation result of the key image frame after preprocessing is obtained based on the preset quality evaluation index. If the preprocessing effect evaluation result does not meet the standard, the optimal combination adjustment is triggered.

[0007] In this scheme, a beacon detection model is constructed based on an improved Efficient Det network, specifically as follows: The Efficient Det network is used as the basic framework of the beacon detection model. In the backbone network, depthwise separable convolutions are introduced to replace some standard convolutions, and a moving inverted bottleneck convolution module is embedded to enhance the feature expression capability. Based on the default BiFPN of EfficientDet, cross-layer skip connections are added, and weighted feature fusion is used to dynamically adjust the contribution of features at different scales. Contextual information is embedded into low-level features to make up for the lack of semantic information of small targets. An attention mechanism is introduced, embedding channel attention in the backbone network and BiFPN, adaptively calibrating channel weights to suppress background interference, and adding spatial attention in front of the detector head to enhance the spatial position of the beacon; Cluster analysis is performed on the ground truth bounding boxes of navigation marks in the training set to generate n sets of prior boxes that conform to the size distribution of navigation marks, and the matching between the prior boxes and the track targets is dynamically adjusted using the IoU threshold. Lightweight classification and regression branches are set up in parallel in the detection head. The beacon dataset is enhanced through environmental adaptation. The backbone network is pre-trained on a general dataset using a transfer learning strategy. The detection head is then trained and fine-tuned using the enhanced beacon dataset.

[0008] In this scheme, multi-level features of preprocessed key image frames are extracted, and a bidirectional feature pyramid network is used for feature fusion to output the location and category of navigation marks. Specifically: The processed key image frames are imported into the navigation mark detection model. The backbone network is used to extract high-resolution feature maps to obtain the visual features of the navigation mark as shallow features, medium-resolution feature maps to obtain the local structure of the navigation mark as medium-layer features, and low-resolution feature maps to output the high-level semantics as deep features. Feature enhancement is achieved by combining depthwise separable convolution with channel attention. Multi-level features obtained from the backbone network are then imported into a bidirectional feature pyramid, and feature fusion is performed through top-down and bottom-up paths. Learnable weights are assigned to the original features and fused features at the same scale. Based on the weights, normalization and fusion are performed to output an optimized feature map. The optimized feature map is then imported into the classification branch and the regression branch. In the classification branch, predict the beacon category probability for each prior box, and in the regression branch, predict the offset of the prior box from the ground truth box. Output the beacon category label and the beacon location coordinates.

[0009] In this scheme, a ship trajectory prediction model is constructed using historical ship trajectory data to predict short-term ship trajectories, specifically as follows: Historical trajectory data of ships is acquired and preprocessed. Based on the historical trajectory data, feature vectors of each ship are obtained. The feature vectors include position, speed, heading, and static information of the ship. The feature vectors are used as nodes, and spatial relationship edges and motion relationship edges are constructed according to the interaction relationship between ships. Graph structure modeling is performed based on nodes, spatial relationship edges, and motion relationship edges. The node features of mutual influence between ships are encoded by graph convolution modules, edge weights are dynamically adjusted by attention mechanisms, spatial features are extracted by hierarchical aggregation, and temporal dependency features are obtained by using GRU units to process the historical trajectory data of each ship and using a bidirectional GRU structure. A multi-head ProbSparse attention mechanism is introduced, which imports the GRU hidden state sequence into the ProbSparse attention head and obtains the query vector, key vector and value vector through linear transformation. The difference between each query vector and all keys is evaluated, and the n query vectors with the highest scores are retained to generate a sparse query matrix. Attention is calculated using a sparse query matrix, key vector, and value vector. After multi-head parallel computation, the vectors are concatenated to output a context vector. The spatial features, temporal dependency features, and context vector are then fused to obtain fused features. The fused features are imported into the prediction layer to output the position offset, speed, and heading changes at future time steps, and to output the ship's short-term track.

[0010] In this scheme, the relative position, speed, and heading angle of the ship and the navigation mark are calculated based on the predicted track information, and a dynamic collision risk field is constructed, specifically as follows: The ship's short-term track and the real-time status of the navigation mark are obtained. For each predicted time step, the relative distance, relative velocity vector, and relative heading angle between the ship and the navigation mark are calculated as kinematic indices. Based on the kinematic indices, the minimum encounter distance and the minimum encounter time are calculated. The dynamic risk value is calculated based on the minimum encounter distance, minimum encounter time and relative distance. Different risk value ranges are divided according to the dynamic risk value, and risk levels are generated. With the ship as the center, the distribution of dynamic risk values ​​on the future trajectory is plotted to construct a dynamic collision risk field.

[0011] This solution integrates visual inspection results, radar ranging data, and AIS information to assess ship collision risks, and triggers tiered early warnings based on preset risk thresholds. Specifically: Input visual inspection results, radar ranging data and AIS information, add timestamps, perform interpolation and alignment based on visual inspection data, and uniformly convert the pixel coordinates of visual inspection, radar polar coordinates and AIS latitude and longitude to the ship's local coordinate system to update the real-time status of navigation marks. Based on the updated real-time status of the navigation mark, the dynamic risk value distribution on the future trajectory is calculated. Different risk levels are generated by comparing the dynamic risk value distribution with the preset risk threshold, and the dynamic collision risk field is reconstructed. When the risk level is higher than the preset standard, the degree of invasion of the safety field is calculated, and the gradient descent method is used to find the optimal solution for speed and heading angle in the preset avoidance strategy library.

[0012] The second aspect of the present invention provides a navigation mark visual recognition and collision warning system based on deep learning, the system comprising: a visual perception module, a navigation mark detection module, a trajectory prediction module, a multi-source data fusion module, and a risk modeling module; The visual perception module acquires real-time channel monitoring video streams through the ship's onboard cameras, extracts key image frames from the channel monitoring video streams, and preprocesses the key image frames. The navigation mark detection module constructs a navigation mark detection model based on an improved Efficient Det network, extracts and processes multi-level features of key image frames, uses a bidirectional feature pyramid network for feature fusion, and outputs the location and category of the navigation mark. The trajectory prediction module constructs a ship trajectory prediction model based on historical ship trajectory data to predict the ship's short-term trajectory. The multi-source data fusion module integrates visual detection results, radar ranging data, and AIS information to update the real-time status of navigation marks. The risk modeling module calculates the relative position, speed, and heading angle of the ship and the navigation beacon based on the predicted track information and the real-time status of the navigation beacon, obtains the ship collision risk, constructs a dynamic collision risk field, and triggers graded early warnings based on preset risk thresholds.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention significantly improves navigation safety through the deep integration of multimodal perception and intelligent decision-making. Regarding adaptability to complex environments, an environment-adaptive image preprocessing strategy effectively overcomes traditional visual recognition challenges such as low visibility in foggy weather, insufficient nighttime lighting, and wave reflection interference, ensuring stable acquisition of navigation mark features under various weather conditions. The improved lightweight detection network achieves high-precision identification of small-sized navigation marks at long distances through multi-scale feature fusion and attention mechanisms, while significantly reducing computational resource consumption and meeting the real-time requirements of embedded shipboard devices.

[0014] The multi-source data fusion architecture overcomes the limitations of single sensors, constructing a more comprehensive understanding of the navigational situation through spatiotemporal alignment and collaborative perception. The complementary fusion of visual detection results and radar ranging data preserves pixel-level positioning accuracy while enhancing anti-obstruction capabilities, while the introduction of AIS information provides a standard data interface for moving targets that conforms to navigation rules. This fusion mechanism significantly improves the continuity and reliability of target tracking. The dynamic risk field modeling method innovatively combines traditional maritime encounter parameters with modern machine learning, assessing collision risk through a dual spatiotemporal dimension. The ship trajectory prediction module accurately captures the interaction effects between ships, predicting potential conflicts in advance, while the tiered early warning mechanism enables a progressive response from early risk alerts to emergency automatic collision avoidance. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments or examples of the present invention, the drawings used in the embodiments or examples will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained according to these drawings without creative effort.

[0016] Figure 1 A flowchart of a deep learning-based navigation mark visual recognition and collision warning method is shown. Figure 2 This diagram illustrates a flowchart of how a navigational aid detection model outputs the location and type of navigational aids. Figure 3 The flowchart illustrates the process of constructing a ship trajectory prediction model to predict short-term ship tracks. Figure 4 A block diagram of a navigation mark visual recognition and collision warning system based on deep learning is shown. Detailed Implementation

[0017] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0018] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0019] Figure 1 A flowchart of a deep learning-based navigation mark visual recognition and collision warning method is shown.

[0020] like Figure 1 As shown, this embodiment provides a deep learning-based method for visual recognition and collision warning of navigation marks, including: S102, real-time acquisition of waterway monitoring video stream through ship-mounted cameras, extraction of key image frames from the waterway monitoring video stream, and preprocessing of the key image frames; S104, a navigation mark detection model is constructed based on the improved Efficient Det network, multi-level features of key image frames are extracted and processed, and a bidirectional feature pyramid network is used for feature fusion to output the location and category of the navigation mark; S106, constructs a ship trajectory prediction model through historical ship trajectory data, predicts short-term ship trajectories, calculates the relative position, speed and heading angle of the ship and navigation marks based on the predicted trajectory information, and constructs a dynamic collision risk field; S108 integrates visual inspection results, radar ranging data, and AIS information to obtain ship collision risks and triggers graded early warnings based on preset risk thresholds.

[0021] It should be noted that during ship navigation, a channel monitoring video stream is continuously acquired using a shipborne high-definition camera based on a preset video resolution and frame rate. The video stream frames are classified according to environmental conditions based on video stream characteristics, and environmental tags are added based on these conditions, such as normal lighting, low lighting, severe weather, and strong interference scenarios. Image entropy is calculated frame-by-frame for the channel monitoring video stream to measure information complexity; a higher entropy value indicates richer visual information contained in the frame. A dynamic image entropy threshold is set based on the environmental tags. Image frames with image entropy greater than the preset dynamic image entropy threshold are selected. Adjacent image frames are marked within the selected frames, and image frames with low image entropy are discarded to avoid redundancy. Furthermore, optical flow is used to detect inter-frame motion changes, and repetitive high-entropy frames caused by ship turbulence are removed, outputting key image frames.

[0022] Based on environmental labels of key image frames, a differentiated preprocessing strategy is adopted. A preprocessing method library is constructed according to preprocessing examples of waterway monitoring. For example, under normal lighting conditions, histogram equalization is used to enhance the contrast between navigation marks and the background, and color space conversion is used to convert RGB images to HSV space to highlight the color features of navigation marks. Under low lighting conditions, the Retinex algorithm or a deep learning-based denoising model is used to improve image brightness and suppress noise. Under severe weather conditions, fog and haze interference is removed based on dark channel priors or DehazeNet networks. In strong interference scenarios, polarization filtering is used to suppress water surface reflections using polarized lenses, and HDR synthesis or local tone mapping is used to alleviate backlight overexposure. Image quality labels are set in the preprocessing method library, and the optimal combination is adaptively selected from the preprocessing method library using environmental labels. The parameters of each preprocessing method in the optimal combination are adjusted according to the image features of key image frames. For example, parameters such as transmittance estimation in the dehazing algorithm and the cropping threshold of histogram equalization are automatically adjusted according to the fog concentration, dynamic range, and other characteristics of the current image. The preprocessing effect evaluation results of key image frames after preprocessing are obtained based on preset quality evaluation indicators such as information entropy, contrast, and noise level. If the preprocessing effect evaluation results do not meet the standards, the optimal combination adjustment is triggered. The best preprocessing method is dynamically selected according to the real-time scene to avoid the limitations of fixed strategies and meet real-time requirements.

[0023] It should be noted that the EfficientDet network is used as the basic framework of the beacon detection model, with EfficientNet-Bx as the feature extraction backbone. The depth and width coefficients are balanced, and depthwise separable convolutions are introduced into the backbone network to replace some standard convolutions, reducing the number of parameters. A moving-inverted bottleneck convolution module is also embedded to enhance feature representation capabilities. Cross-layer skip connections are added to the default BiFPN in EfficientDet to improve the feature transfer efficiency of small targets at long distances. Weighted feature fusion is used to dynamically adjust the contribution of features at different scales, embedding contextual information into low-level features to compensate for the lack of semantic information for small targets. An attention mechanism is introduced, embedding channel attention in the backbone network and BiFPN to adaptively calibrate channel weights to suppress background interference. Spatial attention is added before the detection head to enhance the spatial position of the beacons. K-means++ is used to cluster the ground truth bounding boxes of beacons in the training set, generating n sets of prior boxes that conform to the beacon size distribution. An IoU threshold is used to dynamically adjust the matching between the prior boxes and the track targets, improving the matching rate of small targets. A lightweight classification and regression branch is set up in parallel in the detection head. The regression branch introduces GIoU Loss instead of traditional IoU Loss to optimize the bounding box localization accuracy. The beacon dataset is augmented through environmental adaptation. The backbone network is pre-trained on a general dataset using a transfer learning strategy, and the detection head is trained and fine-tuned using the augmented beacon dataset.

[0024] Figure 2 The flowchart shows the output of navigation mark location and category using the navigation mark detection model.

[0025] According to an embodiment of the present invention, multi-level features of preprocessed key image frames are extracted, and a bidirectional feature pyramid network is used for feature fusion to output the location and category of navigation marks, specifically: S202, the processed key image frames are imported into the navigation mark detection model. The backbone network is used to extract high-resolution feature maps to obtain the visual features of the navigation mark as shallow features, medium-resolution feature maps are extracted to obtain the local structure of the navigation mark as medium-layer features, and low-resolution feature maps are extracted to encode high-level semantics as deep features. S204 uses depthwise separable convolution combined with channel attention to enhance features, imports multi-level features obtained from the backbone network into a bidirectional feature pyramid, and performs feature fusion through top-down and bottom-up paths. S206: Learnable weights are assigned to the original features and fused features at the same scale. Based on the weights, normalized fusion is performed to output the optimized feature map. The optimized feature map is then imported into the classification branch and the regression branch. S208, predict the beacon category probability for each prior box in the classification branch, predict the offset of the prior box to the true box in the regression branch, and output the beacon category label and beacon location coordinates.

[0026] It should be noted that shallow features are sensitive to small navigation marks but have weak semantic information; mid-level features balance spatial details and semantic information; and deep features are robust to occluded navigation marks but have reduced positional accuracy. In the bidirectional feature pyramid fusion stage, the top-down path fuses high-level semantic features with lower-level features through upsampling, allowing high-level semantic information to guide low-level features. The bottom-up path fuses enhanced low-level features with upper-level features through downsampling, improving the spatial accuracy of high-level features. Cross-layer skip connections preserve original features and avoid information loss during fusion. The classification branch predicts the navigation mark category probability for each preset anchor, using Focal Loss to address class imbalance and embedding a channel attention mechanism to suppress misclassifications of background elements such as waves. The regression branch predicts the offset from the anchor to the ground truth bounding box, using GIoU Loss to optimize bounding box localization accuracy. Through cascaded feature extraction and fusion, a complete mapping from pixel-level features to semantic understanding is achieved in the waterway scene, providing high-precision input for subsequent collision warnings.

[0027] Figure 3 The flowchart for constructing a ship trajectory prediction model to predict short-term ship trajectories is shown.

[0028] According to an embodiment of the present invention, a ship trajectory prediction model is constructed using historical ship trajectory data to predict the short-term trajectory of a ship, specifically as follows: S302, acquire historical trajectory data of ships and preprocess it, obtain feature vectors for each ship based on the historical trajectory data of ships, the feature vectors include position, speed, heading and static information of ships, use the feature vectors as nodes, and construct spatial relationship edges and motion relationship edges according to the interaction relationship between ships; S304 is based on graph structure modeling of nodes and spatial and motion relationship edges. It encodes the node features of mutual influence between ships through graph convolution modules, dynamically adjusts edge weights through attention mechanisms, extracts spatial features through hierarchical aggregation, and obtains temporal dependency features by using GRU units to process the historical trajectory data of each ship and using a bidirectional GRU structure. S306 introduces a multi-head ProbSparse attention mechanism, which imports the GRU hidden state sequence into the ProbSparse attention head, obtains the query vector, key vector and value vector through linear transformation, evaluates the difference between each query vector and all keys, and retains the n query vectors with the highest scores to generate a sparse query matrix. S308, use a sparse query matrix and key vector and value vector to calculate attention, perform multi-head parallel computation and then concatenate to output context vector, and fuse the spatial features, temporal dependency features and context vector to obtain fused features; S310, the fused features are imported into the prediction layer to output the position offset, speed, and heading changes for future time steps, and the short-term trajectory of the ship is output.

[0029] It should be noted that each ship is treated as a node, with node features including dynamic characteristics such as latitude and longitude coordinates, speed (SOG), heading (COG), and rate of turn (ROT), as well as static characteristics such as ship type, length, and width. Physical relationships are established by constructing an adjacency matrix based on the distance between ships. Ships with heading angles <30° and similar speeds are linked by motion relationships, and the nodes and edges of the graph are dynamically updated using a sliding time window. In the graph convolutional network module, edge weights are dynamically adjusted through an attention mechanism, with closer ships receiving higher weights. In hierarchical aggregation, ships of the same type are aggregated first, followed by cross-category aggregation. For each query vector... Evaluation and Key Matrix The differences are analyzed, and the top n query vectors with the highest scores are retained to generate a sparse query matrix. The difference scores are represented as follows: , in Indicates the first A query vector, Indicates the first A key vector, Representing feature dimension, This represents the total number of key vectors.

[0030] Most time steps in the ship's trajectory involve straight, uniform, and stable navigation, with only a few moments (turning, accelerating) requiring close attention. A higher score indicates a more unique query vector and a greater impact on the output. Attention is calculated using a sparse query matrix, key vector, and value vector. After multi-head parallel computation, the vectors are concatenated to output a context vector. The spatial features, temporal dependency features, and context vector are fused to obtain fused features. The fully connected network outputs the position offset, speed, and heading changes for the next k steps.

[0031] It should be noted that the process involves acquiring short-term ship tracks, such as position sequences, speed sequences, and heading angle sequences, as well as real-time status of navigation marks, including their position and type. For each prediction time step, the relative distance between the ship and the navigation mark is calculated. The relative velocity vector and relative heading angle are used as kinematic indices. Based on these kinematic indices, the minimum encounter distance and minimum encounter time are calculated. (Minimum encounter distance...) Minimum meeting time , This represents the Euclidean distance between the two ships at the current moment. Indicates relative azimuth. Indicates the ship's heading angle. This indicates the phase adjustment term, which converts the relative azimuth into the geometric collision angle. This indicates the ship's speed.

[0032] The dynamic risk value is calculated based on the minimum encounter distance, minimum encounter time, and relative distance. , is represented as: , in Indicates the weight of the distance term. Represents the distance sensitivity coefficient. Indicates the weight of the time term. Indicates the time threshold. This represents the weight of the absolute distance decay term.

[0033] Based on the dynamic risk values, different risk value ranges are divided to generate risk levels. With the ship as the center, the distribution of dynamic risk values ​​on the future trajectory is plotted to construct a dynamic collision risk field.

[0034] The system inputs visual inspection results, radar ranging data, and AIS information, timestamps them, and performs interpolation and alignment based on the visual inspection data. It then unifies the pixel coordinates of the visual inspection, radar polar coordinates, and AIS latitude and longitude to the ship's local coordinate system, updating the real-time status of the navigation aids. Based on this updated real-time status, it calculates the dynamic risk value distribution on the future trajectory. Different risk levels are generated by comparing the dynamic risk value distribution with a preset risk threshold, reconstructing the dynamic collision risk field, including safe, low-risk, high-risk, and emergency levels. When the risk level exceeds a preset standard (e.g., high-risk, it suggests slowing down or adjusting the course), and in emergency situations, it automatically triggers collision avoidance decisions (e.g., stopping). It calculates the degree of intrusion into the safe zone and uses gradient descent to find the optimal speed and heading angle from a preset avoidance strategy library.

[0035] Figure 4 The diagram shows the architecture of a deep learning-based navigation mark visual recognition and collision warning system.

[0036] The second embodiment of the present invention provides a navigation mark visual recognition and collision warning system 4 based on deep learning. The system includes: a visual perception module 401, a navigation mark detection module 402, a trajectory prediction module 403, a multi-source data fusion module 404, and a risk modeling module 405. The visual perception module acquires real-time channel monitoring video streams through the ship's onboard cameras, extracts key image frames from the channel monitoring video streams, and preprocesses the key image frames. The navigation mark detection module constructs a navigation mark detection model based on an improved Efficient Det network, extracts and processes multi-level features of key image frames, uses a bidirectional feature pyramid network for feature fusion, and outputs the location and category of the navigation mark. The trajectory prediction module constructs a ship trajectory prediction model based on historical ship trajectory data to predict the ship's short-term trajectory. The multi-source data fusion module integrates visual detection results, radar ranging data, and AIS information to update the real-time status of navigation marks. The risk modeling module calculates the relative position, speed, and heading angle of the ship and the navigation beacon based on the predicted track information and the real-time status of the navigation beacon, obtains the ship collision risk, constructs a dynamic collision risk field, and triggers graded early warnings based on preset risk thresholds.

[0037] The third embodiment of the present invention provides a computer-readable storage medium, which includes a deep learning-based navigation mark visual recognition and collision warning method program. When the deep learning-based navigation mark visual recognition and collision warning method program is executed by a processor, it implements the steps of the deep learning-based navigation mark visual recognition and collision warning method.

[0038] In the several embodiments provided in this application, it should be understood that the disclosed methods and systems can be implemented in other ways. The system embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, and can be electrical, mechanical, or other forms. Furthermore, in the various embodiments of the present invention, all functional units can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0039] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0040] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for visual recognition and collision warning of navigational aids based on deep learning, characterized in that, Includes the following steps: The ship's onboard camera collects real-time channel monitoring video streams, extracts key image frames from the channel monitoring video streams, and preprocesses the key image frames. A navigation mark detection model is constructed based on an improved Efficient Det network. Multi-level features of key image frames are extracted and processed. A bidirectional feature pyramid network is used for feature fusion to output the location and category of the navigation mark. By constructing a ship trajectory prediction model using historical ship trajectory data, the model predicts the ship's short-term trajectory. Based on the predicted trajectory information, the relative position, speed, and heading angle of the ship and navigation marks are calculated, and a dynamic collision risk field is constructed. By integrating visual inspection results, radar ranging data, and AIS information, ship collision risks are obtained, and graded early warnings are triggered based on preset risk thresholds.

2. The navigation mark visual recognition and collision warning method based on deep learning according to claim 1, characterized in that, The system uses ship-mounted cameras to collect real-time video streams of waterway monitoring, extracts key image frames from these streams, and preprocesses these key image frames, specifically as follows: During ship navigation, the ship's high-definition camera continuously collects channel monitoring video streams based on preset video resolution and frame rate. Based on the characteristics of the video stream, the video stream frames are classified according to environmental conditions, and environmental tags are added according to the environmental conditions. The image entropy of the channel monitoring video stream is calculated frame by frame. A dynamic image entropy threshold is set according to the environmental label. Image frames with image entropy greater than the preset dynamic image entropy threshold are selected. Adjacent image frames are marked in the selected image frames. Image frames with low image entropy in the marked image frames are removed. Key image frames are output. Based on the environmental labels of key image frames, a differentiated preprocessing strategy is adopted. A preprocessing method library is constructed according to the preprocessing examples of waterway monitoring, and image quality labels are set in the preprocessing method library. The optimal combination is adaptively selected from the preprocessing method library using environmental labels. The parameters of each preprocessing method in the optimal combination are adjusted according to the image features of the key image frame. The preprocessing effect evaluation result of the key image frame after preprocessing is obtained based on the preset quality evaluation index. If the preprocessing effect evaluation result does not meet the standard, the optimal combination adjustment is triggered.

3. The navigation mark visual recognition and collision warning method based on deep learning according to claim 1, characterized in that, A beacon detection model is constructed based on an improved Efficient Det network, specifically as follows: The Efficient Det network is used as the basic framework of the beacon detection model. In the backbone network, depthwise separable convolutions are introduced to replace some standard convolutions, and a moving inverted bottleneck convolution module is embedded to enhance the feature expression capability. Based on the default BiFPN of EfficientDet, cross-layer skip connections are added, and weighted feature fusion is used to dynamically adjust the contribution of features at different scales. Contextual information is embedded into low-level features to make up for the lack of semantic information of small targets. An attention mechanism is introduced, embedding channel attention in the backbone network and BiFPN, adaptively calibrating channel weights to suppress background interference, and adding spatial attention in front of the detector head to enhance the spatial position of the beacon; Cluster analysis is performed on the ground truth bounding boxes of navigation marks in the training set to generate n sets of prior boxes that conform to the size distribution of navigation marks, and the matching between the prior boxes and the track targets is dynamically adjusted using the IoU threshold. Lightweight classification and regression branches are set up in parallel in the detection head. The beacon dataset is enhanced through environmental adaptation. The backbone network is pre-trained on a general dataset using a transfer learning strategy. The detection head is then trained and fine-tuned using the enhanced beacon dataset.

4. The navigation mark visual recognition and collision warning method based on deep learning according to claim 1, characterized in that, Multi-level features are extracted from key image frames after preprocessing, and a bidirectional feature pyramid network is used for feature fusion to output the location and category of navigation marks. Specifically: The processed key image frames are imported into the navigation mark detection model. The backbone network is used to extract high-resolution feature maps to obtain the visual features of the navigation mark as shallow features, medium-resolution feature maps to obtain the local structure of the navigation mark as medium-layer features, and low-resolution feature maps to output the high-level semantics as deep features. Feature enhancement is achieved by combining depthwise separable convolution with channel attention. Multi-level features obtained from the backbone network are then imported into a bidirectional feature pyramid, and feature fusion is performed through top-down and bottom-up paths. Learnable weights are assigned to the original features and fused features at the same scale. Based on the weights, normalization and fusion are performed to output an optimized feature map. The optimized feature map is then imported into the classification branch and the regression branch. In the classification branch, predict the beacon category probability for each prior box, and in the regression branch, predict the offset of the prior box from the ground truth box. Output the beacon category label and the beacon location coordinates.

5. The navigation mark visual recognition and collision warning method based on deep learning according to claim 1, characterized in that, A ship trajectory prediction model is constructed using historical ship trajectory data to predict short-term ship routes. Specifically: Historical trajectory data of ships is acquired and preprocessed. Based on the historical trajectory data, feature vectors of each ship are obtained. The feature vectors include position, speed, heading, and static information of the ship. The feature vectors are used as nodes, and spatial relationship edges and motion relationship edges are constructed according to the interaction relationship between ships. Graph structure modeling is performed based on nodes, spatial relationship edges, and motion relationship edges. The node features of mutual influence between ships are encoded by graph convolution modules, edge weights are dynamically adjusted by attention mechanisms, spatial features are extracted by hierarchical aggregation, and temporal dependency features are obtained by using GRU units to process the historical trajectory data of each ship and using a bidirectional GRU structure. A multi-head ProbSparse attention mechanism is introduced, which imports the GRU hidden state sequence into the ProbSparse attention head and obtains the query vector, key vector and value vector through linear transformation. The difference between each query vector and all keys is evaluated, and the n query vectors with the highest scores are retained to generate a sparse query matrix. Attention is calculated using a sparse query matrix, key vector, and value vector. After multi-head parallel computation, the vectors are concatenated to output a context vector. The spatial features, temporal dependency features, and context vector are then fused to obtain fused features. The fused features are imported into the prediction layer to output the position offset, speed, and heading changes at future time steps, and to output the ship's short-term track.

6. The navigation mark visual recognition and collision warning method based on deep learning according to claim 1, characterized in that, Based on the predicted track information, the relative position, speed, and heading angle of the ship and the navigation mark are calculated, and a dynamic collision risk field is constructed, specifically as follows: The ship's short-term track and the real-time status of the navigation mark are obtained. For each predicted time step, the relative distance, relative velocity vector, and relative heading angle between the ship and the navigation mark are calculated as kinematic indices. Based on the kinematic indices, the minimum encounter distance and the minimum encounter time are calculated. The dynamic risk value is calculated based on the minimum encounter distance, minimum encounter time and relative distance. Different risk value ranges are divided according to the dynamic risk value, and risk levels are generated. With the ship as the center, the distribution of dynamic risk values ​​on the future trajectory is plotted to construct a dynamic collision risk field.

7. The navigation mark visual recognition and collision warning method based on deep learning according to claim 1, characterized in that, By integrating visual inspection results, radar ranging data, and AIS information to assess ship collision risks, and triggering tiered early warnings based on preset risk thresholds, the following steps are taken: Input visual inspection results, radar ranging data and AIS information, add timestamps, perform interpolation and alignment based on visual inspection data, and uniformly convert the pixel coordinates of visual inspection, radar polar coordinates and AIS latitude and longitude to the ship's local coordinate system to update the real-time status of navigation marks. Based on the updated real-time status of the navigation mark, the dynamic risk value distribution on the future trajectory is calculated. Different risk levels are generated by comparing the dynamic risk value distribution with the preset risk threshold, and the dynamic collision risk field is reconstructed. When the risk level is higher than the preset standard, the degree of invasion of the safety field is calculated, and the gradient descent method is used to find the optimal solution for speed and heading angle in the preset avoidance strategy library.

8. A navigational aid visual recognition and collision warning system based on deep learning, characterized in that, To implement the deep learning-based navigation mark visual recognition and collision warning method as described in any one of claims 1-7, the system includes: a visual perception module, a navigation mark detection module, a trajectory prediction module, a multi-source data fusion module, and a risk modeling module; The visual perception module acquires real-time channel monitoring video streams through the ship's onboard cameras, extracts key image frames from the channel monitoring video streams, and preprocesses the key image frames. The navigation mark detection module constructs a navigation mark detection model based on an improved Efficient Det network, extracts and processes multi-level features of key image frames, uses a bidirectional feature pyramid network for feature fusion, and outputs the location and category of the navigation mark. The trajectory prediction module constructs a ship trajectory prediction model based on historical ship trajectory data to predict the ship's short-term trajectory. The multi-source data fusion module integrates visual detection results, radar ranging data, and AIS information to update the real-time status of navigation marks. The risk modeling module calculates the relative position, speed, and heading angle of the ship and the navigation beacon based on the predicted track information and the real-time status of the navigation beacon, obtains the ship collision risk, constructs a dynamic collision risk field, and triggers graded early warnings based on preset risk thresholds.

Images