Port long channel fog navigation system and method based on unmanned aerial vehicle navigation and small target detection positioning

Abstract

This invention provides a port long-channel fog navigation system and method based on UAV navigation and small target detection and localization, comprising: a guided vessel and a pilot terminal layer, an aerial UAV, and a maritime support platform; the guided vessel and pilot terminal layer includes: a guided merchant ship and a pilot tablet terminal; the aerial UAV includes: two UAVs operating in a forward formation navigation mode; the two UAVs respectively acquire images and point clouds at a fixed frequency, attach a unified timestamp and their own attitude information, and send them to a tugboat RTX edge computing base station, where the tugboat RTX base station performs data fusion; the maritime support platform includes: a tugboat RTX base station, which establishes a unified ENU coordinate system, transmits data through a dedicated wireless link and a data access gateway, fuses the observation data from the two UAVs, and performs 3D positioning and collision avoidance calculations. This invention significantly improves the detection accuracy of small targets and the ability to share 3D situational awareness, thereby reducing the risk of collision.

Description

Technical Field

[0001] This invention relates to the fields of unmanned aerial vehicle (UAV) navigation and target detection and positioning technology, and in particular to a port long channel fog navigation system and method based on UAV navigation and small target detection and positioning. Background Technology

[0002] With the increasing size of ships, port anchorages have gradually extended out to sea, resulting in a significant increase in the length of typical deep-water port access channels, reaching tens of kilometers. These channels are typically characterized by narrow boundaries and dense intersections. During seasons with limited visibility, such as sea fog or rain fog, the risk of ship collisions and groundings increases significantly. Under limited visibility conditions, ships need to complete lookout, judgment, and maneuvering within a shorter predictable distance, and their safety zone shrinks sharply as visibility decreases, significantly increasing the situational awareness burden on pilots and bridge teams. Case studies of maritime accidents show that improper radar operation, difficulty in identifying small targets, and insufficient awareness of the situation ahead are among the main contributing factors to fog navigation accidents.

[0003] Currently, port fog navigation mainly relies on traditional shipborne X / S band radar, ARPA system, AIS automatic identification system, electronic charts (ECDIS), and shore-based VTS radar / electro-optical monitoring equipment. Under conditions of limited visibility, these devices are designed to provide sufficient detection range and information redundancy to support compliance with collision avoidance rules such as COLREG Rule 19 and company navigation best practices.

[0004] However, traditional radar and AIS systems are primarily designed for medium to large-sized ships, with limited detection capabilities for small, non-metallic floating objects, rafts, fishing gear, small boats, and temporary buoys. Specifically, radar emits weak echo signals and has a high false alarm rate for small targets, and its resolution is affected by antenna length, sea clutter, and multipath effects; while AIS systems are completely unable to detect small targets without transponders. In foggy navigation scenarios on long waterways, these small targets may exist within hundreds to thousands of meters ahead of the ship, and traditional ship-to-shore equipment struggles to provide continuous, accurate, and three-dimensional forward-looking perception capabilities. This deficiency significantly increases the safety risks for ships in foggy conditions. Summary of the Invention

[0005] To address the challenges of timely and accurate detection and positioning of small surface obstacles in foggy conditions along long port channels, and the limitations of existing shipborne or shore-based monitoring systems in providing full-length 3D information on small targets from a high-angle forward view, this invention proposes a foggy navigation system and method for long port channels based on UAV navigation and small target detection and positioning. By employing technologies such as dual UAV collaborative detection, tugboat-mounted edge computing nodes, and real-time data transmission, the system significantly improves the detection accuracy of small targets and the ability to share 3D situational awareness, thereby reducing the risk of collisions and grounding.

[0006] Therefore, the present invention provides the following technical solution:

[0007] This invention provides a port long channel fog navigation system based on UAV navigation and small target detection and positioning, including: a guided vessel and a pilot terminal layer, an aerial UAV and a maritime support platform; The piloted vessel and pilotage terminal layer includes: the piloted merchant ship and the pilotage tablet terminal; the piloted merchant ship sends its own position and attitude to the pilotage tablet terminal, and the pilotage tablet terminal displays electronic nautical charts, UAV videos and overlays high-risk target alarms; The aerial drones include: two drones operating in a forward formation navigation mode; the two drones respectively acquire images and point clouds at a fixed frequency, attach a unified timestamp and their own attitude information and send them to the tugboat RTX base station, which performs data fusion. The maritime support platform includes: a tugboat RTX base station. The tugboat RTX base station establishes a unified ENU coordinate system and transmits data with a data access gateway through a dedicated wireless link. It fuses observation data from two UAVs and combines the fused data with the position and attitude information of the guided merchant ship to perform three-dimensional positioning and collision avoidance calculations.

[0008] Furthermore, the two drones were positioned relative to the guided merchant ship as follows: 1–2 km ahead of the centerline of the channel; 150–500 meters to the left and right horizontally; and 80–150 m in altitude.

[0009] Furthermore, both drones calculated their six degrees of freedom attitude and position using the RTK global navigation satellite system and inertial navigation unit, and periodically exchanged differential data with the tugboat's RTK base station.

[0010] Furthermore, each drone is equipped with at least one visible light camera and one lidar; the visible light camera provides high-resolution images; and the lidar provides high-precision geometric information at near to mid-range distances.

[0011] Furthermore, the dual UAVs preprocess the acquired images and point clouds, including time synchronization, candidate box determination, and feature extraction.

[0012] Furthermore, the pilotage tablet terminal displays the position, track, and current operation mode of the piloted vessel, tugboat, and dual UAVs on a high-resolution map or electronic nautical chart of the port area; it overlays a first-person view video window of the UAVs in the center area of ​​the map, displaying the channel ahead in real time, and overlays a category box and confidence percentage on the screen, while displaying the UAV status information below; it displays the calculated collision risk information in a list format in the sidebar, including the closest encounter time, closest distance, relative distance, and risk level automatically determined by the system for each target; and it provides measurement tools, allowing pilots to mark high-risk targets on the nautical chart.

[0013] In another aspect, the present invention also provides a method for fog navigation in long port channels based on UAV navigation and small target detection and localization. Based on the aforementioned fog navigation system for long port channels based on UAV navigation and small target detection and localization, the method includes: S1. Two drones patrol in a predetermined formation over the channel ahead of the guided vessel, equipped with visible light cameras and lidar, to continuously scan the water surface ahead. S2. The two drones will collect images and point clouds, and transmit them back to the edge server on the tugboat via a dedicated wireless link with a unified timestamp and their own attitude information. S3. The edge server on the tugboat performs data fusion on the observation data of the two UAVs in the RTK unified coordinate system, and performs three-dimensional positioning and collision avoidance calculation based on the fused data and the position and attitude information of the guided merchant ship. S4. The calculated obstacle positions and movement information are pushed in real time to the pilot tablet terminal on the piloted vessel via a private network between ships. The pilot software visualizes and marks the obstacles on the electronic nautical chart / port map, and at the same time, the key data is transmitted back to the port VTS for the reconstruction of the three-dimensional traffic situation of the long channel.

[0014] Furthermore, 3D localization and collision avoidance calculations include: Mapping from sensor coordinates to world coordinates: (using the first...) Taking a drone as an example, its camera intrinsic parameter matrix is ​​denoted as... The extrinsic parameters are denoted as rigid body transformation matrices. The attitude and position of the UAV body coordinate system to the world ENU coordinate system are denoted as... For a target detection box in an image, take its center pixel. Construct the normalized imaging plane vector: ; The vector is normalized and transformed to the UAV body coordinate system using extrinsic parameters: ; Then, by transforming the UAV's attitude to the world coordinate system, the direction of the observation ray originating from the camera's optical center is obtained: ; in Let this be the position of the camera's optical center in the world coordinate system. This refers to the translation of the camera relative to the body of the machine. Each target is represented as a parametric ray in the world coordinate system: ; (1) Single-machine sea surface plane constraint positioning mode: In maritime navigation scenarios, the sea surface is approximated as a flat plane for a short period of time. : ; in Typically, the vertically upward unit vector in the ENU coordinate system is taken. The constant is determined by the tide level and the datum; the target is a small boat or floating object on the water surface, and the solution is obtained by... With plane The intersection point yields its three-dimensional coordinates: ; (2) Dual-machine collaborative three-dimensional least squares positioning mode: Suppose that UAVs 1 and 2 observe the same target and obtain the ray. , Then the target's three-dimensional position Take the midpoint of the shortest line connecting the two rays: ; In the formula, The problem is a vector cross product; it can be transformed into a linear least squares solution, and its closed-form solution is expressed as: ; Conversion from world coordinates to piloted ship coordinates: The position of the piloted merchant ship in the world coordinate system is... The attitude matrix is The transformation from the world coordinate system to the ship's coordinate system is as follows: ; make Definition: Forward distance: Horizontal distance: Spatial distance: Relative azimuth: .

[0015] Furthermore, the three-dimensional localization and collision avoidance calculation also includes: establishing time-series tracking for each target and estimating its velocity vector and relative motion relationship; Set at two timestamps World coordinates for the same target The average velocity of the target in the world coordinate system is estimated as follows: ; The piloted vessel's own velocity vector Based on the trajectory calculation, the relative speed of the target relative to the ship is: ; Transform to ship coordinate system: ; At the target location With relative velocity Given the target and the ship, calculate the closest encounter time (TCPA) and closest distance (CPA) under the assumption of linear relative motion: ; Based on CPA, TCPA, and set thresholds, targets are classified into multiple levels, and the results are pushed to the pilot terminal and VTS system.

[0016] Furthermore, the reconstruction of the three-dimensional traffic situation of the long waterway includes: using the port area DEM / modeling results as a background, rendering the positions and trajectories of guided ships, tugboats, drones and small targets in three-dimensional coordinates in real time, and calling the first-person perspective videos of drones and guided merchant ships to achieve three-dimensional monitoring and remote verification.

[0017] Advantages and positive effects of the present invention: (1) The dual UAV forward-looking formation navigation system proposed in this invention establishes a stable high-level forward-looking observation zone several nautical miles in front of the guided vessel. It is the core infrastructure that breaks through the bottleneck of navigation restrictions / no-navigation under fog conditions by relying solely on shipborne radar.

[0018] In foggy navigation scenarios, the ability to safely clear a vessel essentially depends on the predictability of small targets and traffic conditions ahead. This invention utilizes two unmanned aerial vehicles (UAVs) maintaining a pre-set formation patrol within 1-2 nautical miles ahead of the guided vessel, continuously scanning the long channel from above to form a high-level perception layer independent of the vessel's own line-of-sight and radar blind spots. This formation not only covers the center of the channel but also provides additional detection capabilities for anchorages on both sides and temporary operating areas. This allows managers to move beyond being completely constrained by visibility and bridge visibility in foggy weather, instead dynamically assessing navigable windows based on the observations of the two UAVs. This is the first key point supporting the transition from prohibited / restricted navigation to controlled navigation.

[0019] (2) The present invention is based on a small target three-dimensional positioning and motion state estimation algorithm with dual UAVs and RTK unified coordinate system. It transforms the image detection results into accurate CPA / TCPA risk indicators that can be directly used for release decisions, thereby quantitatively supporting the "to release or not to release, and how to release" navigation decisions under fog conditions.

[0020] In traditional fog navigation, even if radar detects suspicious small targets, pilots and VTS often struggle to determine their true spatial location and future trajectory, relying instead on experience to increase safety distances and reduce speed, leading to forced navigation restrictions. This invention utilizes the spatial baseline of two unmanned aerial vehicles (UAVs) and a unified RTK coordinate system to obtain meter-level position and velocity vectors relative to the guided vessel from targets such as VESSEL, BUOY, and OBSTACLE detected by cameras through dual-ray least squares three-dimensional intersection and ship coordinate transformation. It further calculates risk parameters such as CPA / TCPA. The system outputs real-time quantitative information in fog such as "Small boat 1.8 nautical miles ahead, 0.2 nautical miles to port, expected closest encounter in 12 minutes, CPA 0.15 nautical miles," enabling both pilots and VTS to use unified numerical indicators to determine whether to maintain the current speed, whether to reduce speed, or adjust the order. This transforms experience-based navigation restrictions into data-driven, precise clearance. This entire algorithmic chain from image to risk indicators constitutes the second core protection point of this invention.

[0021] (3) This invention upgrades the traditional escort tugboat into a mobile fog navigation support platform that integrates UAV take-off and landing platform, RTK base station, edge computing node and communication relay. Without significantly modifying the port and large ships, it provides an engineering path that can be quickly deployed and replicated and promoted to lift the ban / restriction on navigation in foggy weather.

[0022] Existing solutions for fog navigation modifications mostly focus on building new shore-based radar towers or making large-scale upgrades to shipboard equipment. These approaches are costly, time-consuming, and disruptive to production, making them unsuitable as a universal solution to navigation restrictions in foggy weather. This invention utilizes existing tugboats as carriers, transforming them into mobile hubs between drones, piloted vessels, and VTS (Vehicle Transportation Services) by adding RTK base stations, industrial-grade edge servers, and dedicated network communication modules. This provides a stable take-off and landing platform and high-precision positioning reference for both drones, while also handling computation and data aggregation. Thus, ports only need to deploy a small amount of equipment within their pilotage and tugboat systems to provide on-demand drone navigation services for long waterways like Chaolian Island during critical fog seasons, without altering large merchant ships or shore-based infrastructure, achieving significant increased navigation capacity at a low modification cost.

[0023] (4) The UAV navigation-pilot tablet HMI-VTS three-dimensional situation reconstruction business closed loop constructed by the present invention upgrades the navigation decision-making in foggy weather from single-point experience judgment to a three-dimensional collaborative control mechanism shared by multiple parties, and provides institutional support for the dynamic optimization of port rules on foggy navigation prohibition / restriction.

[0024] In the traditional model, whether or not navigation is closed in foggy weather is determined by visibility thresholds and a limited amount of radar footage. Information is concentrated on individual bridges and VTS screens, making it difficult to form a traceable chain of global situational awareness. This invention uses drones to transmit forward-looking images and positioning results in real time to the tugboat's edge server. The fused target list and risk indicators are then pushed to the pilot's tablet and the VTS 3D scene, creating a multi-party shared framework with the same data presented from different perspectives. The VTS can formulate or adjust foggy navigation plans based on the 3D reconstruction results. Pilot stations can replay foggy navigation passages to assess risk control effectiveness. Port management departments can gradually replace simple visibility threshold rules with more granular operational procedures that allow for appropriate relaxation of navigation restrictions when drone-guided forward vision is available, based on extensive actual operational data. This closed loop from perception to procedures represents a systemic innovation in this invention, truly changing the rules of fog navigation through drone-guided navigation. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is an architecture diagram of a port long channel fog navigation system based on UAV navigation and small target detection and positioning in an embodiment of the present invention; Figure 2 This is a schematic diagram showing the distribution of positioning errors for single and dual machines in an embodiment of the present invention.

[0027] Figure 3 This is an example of how a drone target is integrated into the display screen of a navigation terminal in an embodiment of the present invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] This invention proposes a method and system for fog navigation in long port channels, which uses an escort tugboat as the core platform, dual UAVs for collaborative navigation, small target detection and positioning, and integrated sea-air-shore information sharing. This introduces UAV collaborative perception capabilities that can be implemented in engineering to traditional pilotage operations.

[0031] like Figure 1 As shown, a port long channel fog navigation system based on UAV navigation and small target detection and positioning includes: the guided vessel and the pilot terminal layer, the maritime support platform and the aerial UAV.

[0032] The piloted vessel and the pilot terminal layer include: the piloted merchant ship and the pilot tablet terminal; the piloted merchant ship sends Global Navigation Satellite System (GNSS) / compass / speed information to the pilot tablet terminal, which displays electronic nautical charts, UAV videos and overlays high-risk target warnings.

[0033] The maritime support platform includes a tugboat RTX base station. The tugboat RTX base station establishes a unified ENU coordinate system and transmits data via a dedicated wireless link to a data access gateway. It fuses observation data from both UAVs, aligning their time and coordinates. The fused data, combined with data from the piloted merchant ship and the pilot's tablet terminal, is used for 3D positioning and collision avoidance calculations, including target tracking and motion estimation, as well as 3D positioning of small targets.

[0034] The aerial unmanned aerial vehicle (UAV) system comprises two UAVs that collect video / point cloud data and perform lightweight preprocessing, including time synchronization, candidate bounding box determination, and feature extraction. The preprocessed data is then sent to the maritime support platform for further data fusion. The two UAVs operate in a forward formation navigation mode. They are positioned relative to the guided merchant ship as follows: 1–2 km ahead of the channel centerline; 150–500 meters horizontally to the left and right; and at an altitude of 80–150 meters to avoid interference with port obstacles and the superstructure of large vessels. Both UAVs calculate their six-DOF attitude and position using RTK GNSS and IMU, and periodically exchange differential data with the tugboat's RTK base station to ensure the entire system uses a unified port ENU (East-North-Up) coordinate system. The UAV flight control logic automatically adjusts its longitudinal position based on the guided merchant ship's GPS speed, maintaining the forward-looking distance within a set range, thus creating a continuous and stable forward-looking observation window throughout the long channel. In terms of sensor configuration, each drone is equipped with at least one visible light camera, superimposed with a LiDAR. The visible light camera provides high-resolution images for deep learning target detection; the LiDAR provides high-precision geometric information at near to mid-range distances, improving the depth constraint and robustness of small target localization. The drone acquires images and point clouds at a fixed frequency (e.g., 10–20 Hz), and adds a unified timestamp and its own attitude information.

[0035] The piloted vessel itself only needs to be equipped with lightweight communication equipment and a pilotage tablet terminal. The main system modifications are concentrated on the tugboat and drone side, which is conducive to the deployment of actual port projects.

[0036] Based on the aforementioned fog navigation system using UAV navigation and small target detection and localization, this embodiment presents a fog navigation method for long-channel ports based on UAV navigation and small target detection and localization. This method is constructed around four levels: dual UAV forward-looking perception, tugboat edge computing, pilotage terminal display, and VTS 3D reconstruction, including: S1. Two drones patrol in a predetermined formation over the channel ahead of the guided vessel, equipped with cameras and lidar, to continuously scan the water surface ahead. S2. The drone performs only light preprocessing on the airborne end, packaging the timestamped image frames, detection candidate boxes or sparse feature information, and transmitting them back to the edge server on the tugboat via a dedicated wireless link. S3. The tugboat edge server performs small target detection, matching, three-dimensional positioning and motion state estimation on the dual-machine observation under the RTK unified coordinate system, and transforms it to the coordinate system of the guided ship. Small target detection and coordinate transformation specifically include: This invention converts the two-dimensional target detected by the camera into a three-dimensional position and motion state in the coordinate system of the guided vessel. To adapt to different load combinations, this solution supports two modes: single-machine planar hypothetical intersection and dual-machine multi-view geometric positioning, and performs fusion uniformly in the RTK ENU coordinate system.

[0037] Mapping from sensor coordinates to world coordinates: (using the first...) Taking a drone as an example, its camera intrinsic parameter matrix is ​​denoted as... The extrinsic parameters (camera coordinates relative to the UAV body coordinate system) are denoted as the rigid body transformation matrix. The attitude and position of the UAV body coordinate system relative to the world ENU coordinate system are denoted as... For a target detection box in an image, its center pixel can be taken. Construct the normalized imaging plane vector: (1) The vector is normalized and transformed to the UAV body coordinate system using extrinsic parameters: (2) By transforming the UAV's attitude to the world coordinate system, the direction of the observation ray originating from the camera's optical center can be obtained: (3) in The position of the camera's optical center in the world coordinate system ( (This refers to the translation of the camera relative to the body).

[0038] Thus, each target can be represented as a parameterized ray in the world coordinate system: (4) (1) Single-machine + sea surface plane constraint positioning mode: In maritime navigation scenarios, the sea surface can be approximated as a flat plane for a short period of time. : (5) in Typically, the vertically upward unit vector in the ENU coordinate system is taken. This is a constant determined by the tide level and the datum. The targets are often small boats or floating objects on the water's surface, which can be solved by finding the ray... With plane The intersection point yields its three-dimensional coordinates: (6) This method requires minimal computation and can be solved in real time for each detection target on an edge server, making it suitable for single-machine payload scenarios.

[0039] (2) Dual-machine collaborative three-dimensional least squares positioning mode: To reduce uncertainties caused by sea surface fluctuations and tidal level estimation errors, this invention prioritizes a three-dimensional least squares positioning method using dual UAVs in collaboration. Assume that UAVs 1 and 2 observe the same target and obtain ray... , Then the target's three-dimensional position Take the midpoint of the shortest line connecting the two rays: (7) In the formula, Let be the cross product of vectors. This problem can be reduced to a linear least squares solution, whose closed-form solution can be expressed as: (8) By using two-machine baselines for spatial intersection in the RTK unified coordinate system, this invention can obtain more stable three-dimensional coordinate estimation of small targets without explicitly relying on sea surface parameters, and is especially suitable for situations with large waves or targets slightly higher than the sea surface.

[0040] Conversion from world coordinates to piloted vessel coordinates: The position and attitude of the piloted merchant ship are obtained through its own GNSS / RTK and compass / IMU, and its position in the world coordinate system is denoted as... The attitude matrix is The transformation from the world coordinate system to the ship's coordinate system (bow forward, port side positive) is as follows: (9) make Forward distance can be defined as: Horizontal distance: Spatial distance: Relative azimuth: .

[0041] The aforementioned physical quantities are directly used for the front-end display of the pilot terminal and for collision risk assessment.

[0042] To provide pilots with more decision-making information, this invention establishes time-series tracking for each target and estimates its velocity vector and relative motion relationship.

[0043] Set at two timestamps World coordinates for the same target The average velocity of the target in the world coordinate system is estimated as follows: (10) The piloted vessel's own velocity vector The relative speed of the target to the ship can be calculated from the flight path. (11) Transform to ship coordinate system: (12) At the target location With relative velocity Given the information, the closest encounter time (TCPA) and closest distance (CPA) between the target and the ship under the assumption of linear relative motion can be calculated: (13) This invention automatically classifies targets into normal, warning, and dangerous levels based on CPA / TCPA and set thresholds within an edge server, and pushes the results to the pilot terminal and VTS system. The interactive interface of the pilot tablet displays the MMSI / category, CPA, TCPA, and current distance of each target in a list on the right, using yellow / red to mark high-risk targets, consistent with the icon colors on the front-end map.

[0044] S4. The calculated obstacle positions and movement information are pushed in real time to the pilot tablet terminal on the piloted vessel via a private network between ships. The pilot software visualizes and marks the obstacles on the electronic nautical chart / port map, and at the same time, the key data is transmitted back to the port VTS for the reconstruction of the three-dimensional traffic situation of the long channel.

[0045] The pilot tablet terminal on the piloted vessel runs dedicated pilotage software. Its core functions include displaying the position, track, and current operating mode (normal, fog navigation, drone navigation, etc.) of the piloted vessel, tugboat, and two drones on a high-resolution port map or electronic nautical chart; Figure 3 As shown, a first-person view video window of the UAV is superimposed in the center area of ​​the map to display the forward flight path in real time. Category boxes such as VESSEL / BUOY / OBSTACLE and confidence percentages are superimposed on the screen. At the same time, the status information such as UAV altitude ALT, battery BAT, and link signal SIG are displayed below. The collision risk information calculated by formula (13) is displayed in a list in the sidebar, including the CPA, TCPA, relative distance and risk level automatically judged by the system for each target. Measurement tools such as ranging, bearing, VRM, EBL and plotting functions are provided, allowing the pilot to mark high-risk targets on the aeronautical chart.

[0046] The shore-based VTS system receives fused data streams from the tugboat edge server via a dedicated line, enabling the reconstruction of traffic conditions in a long waterway in a 3D scene: using the port area DEM / modeling results as a background, it renders the positions and trajectories of guided vessels, tugboats, drones, and small targets in 3D coordinates in real time, and can call up first-person view videos of drones and guided merchant ships to achieve three-dimensional monitoring and remote verification.

[0047] To facilitate understanding, the following simulation experiment will be used as an example to illustrate the specific application of this invention in the 30-nautical-mile long channel of Chaolian Island in Qingdao Port. This channel, approximately 30 nautical miles long from the outer anchorage to the port waters, is a typical long channel with high-density traffic and frequent fog navigation. In this embodiment, without large-scale modifications to the existing deep-water channel navigation system, the device and software described in this invention can be deployed only on the escort tugboat, the UAV platform, and the pilot terminal to form a dual-UAV fog navigation and small target detection and positioning system for the long channel of Chaolian Island.

[0048] During the preparation phase, the port pilot station configured a drone navigation template suitable for the Chaolian Island channel based on its geometric characteristics, bend locations, and typical fog distribution. This template, using the large merchant ship being piloted as a reference, pre-sets the formation positions of two drones in the channel: a longitudinal forward-looking distance of 1.5–2 nautical miles ahead of the channel centerline, and a lateral deviation of approximately 150 meters from the centerline, forming a V-shaped aerial observation baseline covering both sides of the channel and nearby anchorages. The altitude was set between 90 and 120 meters to ensure it could overcome obstructions from the superstructure and shoreline of large vessels while avoiding conflicts with existing port obstacles and airspace restrictions. Before departure, the accompanying tugboat completed RTK base station power-on, edge server self-testing, and dedicated network link testing to provide unified coordinate reference, calculation, and communication capabilities for subsequent operations.

[0049] When a large container ship receives pilotage instructions at its anchorage outside Chaolian Island and prepares to enter the port, the pilot boards the ship in a pilot boat and selects the UAV fog navigation scheme for the 30-nautical-mile long channel of Chaolian Island via the pilotage tablet terminal of this invention on the bridge. The tugboat then approaches the port or starboard aft position of the piloted vessel, and two UAVs automatically take off from the tugboat deck, flying to the preset formation position according to the template route and real-time RTK positioning, establishing a unified ENU coordinate system based on the tugboat's RTK. At this time, the piloted vessel connects its own position and attitude to the system through its own GNSS / compass equipment, achieving precise registration between the ship's coordinate system and the ENU coordinate system, laying the foundation for subsequent target coordinate transformation and motion state estimation.

[0050] During foggy navigation in the Chaolian Island channel, two UAVs maintained a constant following distance of 1.5–2 nautical miles ahead of the guided vessel, continuously scanning the channel for several nautical miles ahead from a top-down perspective. Each UAV acquired images and lidar point clouds at 10–20 Hz, along with pose information calculated by RTK / IMU, and transmitted them in real-time to the tugboat's edge server via a dedicated wireless link. The edge server used a deep learning target detection algorithm to identify small targets in the dual-UAV images, mapping the center of the detection box to the observation ray in the camera coordinate system, and then combined the UAV camera extrinsic parameters and attitude equations to complete the geometric description of the ray in the ENU coordinate system. For fishing boats, small boats, and floating objects that only float on the surface of the Chaolian Island channel, the system can use the sea surface plane assumption and ray intersection to obtain a rough three-dimensional position. When two drones observe the same target at the same time, the spatial intersection method based on dual-ray least squares proposed in this invention is used to accurately solve the target position using the spatial baseline between the drones, thereby obtaining the three-dimensional coordinates of small targets at the meter level or even sub-meter level without relying on the sea surface plane parameters.

[0051] After completing the positioning in the world coordinate system, the edge server transforms the target coordinates to the ship's coordinate system in real time using the attitude and position of the guided merchant ship. This yields the target's forward distance, lateral offset, and altitude information relative to the ship's bow. Simultaneously, it calculates the target's relative velocity, CPA, and TCPA using the target's position and the ship's trajectory at consecutive moments. All results are packaged into a structured target list and pushed to the pilotage tablet terminal via a dedicated network link. The pilotage software plots target points against the background of the electronic nautical chart of the Chaolian Island channel, using the ship as a reference, and distinguishes between ordinary, warning, and dangerous targets using different colors. Simultaneously, a floating window displaying the forward-looking view of the UAV pops up in the center of the interface, directly overlaying categories such as VESSEL, BUOY, and OBSTACLE identified through deep learning onto the video, and annotating the forward distance and risk level in text form. This allows the pilot to intuitively grasp the distribution and evolution trends of small targets within a few nautical miles ahead, even in fog.

[0052] At the Qingdao Port VTS center, fused data from dual drones, escorting tugboats, and piloted vessels is uploaded in real time to a shore-based server. Based on a 3D terrain and facility model of the 30-nautical-mile long channel around Chaolian Island, the server renders various vessels, tugboats, drones, and small targets as 3D symbols, creating a rotatable and scalable 3D traffic situation map of the long channel. VTS personnel can clearly see the relative positions of piloted vessels within the fog and the distribution of small targets in this view. They can also access the heading perspective of piloted vessels and the forward-looking view of any drone with a single click, allowing for manual verification and intervention command of high-risk targets automatically marked by the system. This air-sea-shore integrated 3D situation sharing mechanism upgrades traffic organization in the Chaolian Island long channel under fog conditions from traditional point-based radar symbols and voice command to visualized collaborative control based on 3D geometric relationships, significantly improving interpretability and safety margins in complex fog navigation scenarios.

[0053] This simulation assumes the presence of 200 randomly floating small targets in the 30-nautical-mile long channel to Chaolian Island, with a ship speed of approximately 15 knots (≈8 m / s) and target drift speeds ranging from ±2 m / s (longitudinal) to ±1 m / s (lateral). Two UAVs acquire images and point clouds at 10–20 Hz, outputting the target center through a deep learning model, and adding 0.5° of observation noise.

[0054] (1) Simulation Design and Methods Target generation: Near the baseline approximately 2 nautical miles (≈3704m) ahead of the tugboat, randomly generate the 3D real coordinates and velocities of 200 small targets. The target height is set to 0m, and they are distributed laterally within ±500m.

[0055] Single-unit positioning model: Simulates the two-dimensional positioning of a single UAV based on the assumption of the sea surface (intersection of the ray and the sea surface) to obtain the estimated coordinates of the target.

[0056] Dual-machine cooperative positioning model: Based on the two-ray least squares method, the midpoint of the shortest line connecting the two observation rays is obtained to achieve three-dimensional intersection positioning.

[0057] Error calculation: Compare the Euclidean distance between the estimated position and the true position of the single-machine and dual-machine systems.

[0058] Risk assessment: Risk is determined using the commonly used navigation indicators CPA and TCPA; when visibility is limited, a target is considered dangerous if the CPA is less than 2 nautical miles and the TCPA is less than 12 minutes.

[0059] (2) Main simulation results Table 1 presents the statistical results of positioning errors for single and dual-machine systems. It can be seen that the average error of dual-machine cooperative positioning is significantly lower than that of single-machine positioning.

[0060] Table 1

[0061] Figure 2 Further analysis revealed the error distribution: the errors of dual-machine positioning were concentrated within 100m, while the number of errors of single-machine method exceeding 300m was significantly greater.

[0062] The performance comparison results of risk assessment are shown in the table below. The precision and recall rates of the dual-machine solution are both higher than those of the single-machine solution, significantly reducing false negatives and false positives.

[0063] Table 2

[0064] (3) Simulation conclusions The spatial baseline formed by the dual UAVs enables 3D positioning of small targets ranging from meters to tens of meters in size, with an average error of only about 47 meters, improving accuracy by at least 50% compared to a single UAV. Combining high-precision positioning with relative motion estimation, the calculated CPA / TCPA is more accurate, increasing the detection rate of high-risk targets and reducing false alarms. Pushing the structured target list to the pilot board and shore-based VTS provides situational awareness several nautical miles in advance and intuitive video overlay display for fog navigation on long waterways, significantly reducing the cognitive burden on pilots. The simulation and experimental results fully demonstrate the innovative advantages of the dual UAV fog navigation and small target detection and positioning system in terms of positioning accuracy, risk assessment, and engineering feasibility.

[0065] The application of this invention in the 30-nautical-mile long channel of Chaolian Island fully utilizes the traditional pilotage element of tugboats, upgrading it into an intelligent support platform integrating UAV take-off and landing platforms, RTK reference stations, edge computing nodes, and communication relays. By utilizing dual UAV collaboration and a unified RTK coordinate system, high-precision 3D positioning and motion state estimation of small targets ahead are achieved in foggy navigation conditions along the long channel. Furthermore, through a dedicated tablet terminal for pilots and a 3D reconstruction system for VTS (Vehicle Traffic Support System), the forward-looking perception capabilities of UAVs are smoothly integrated into existing pilotage and traffic organization processes, forming a port long-channel fog navigation support technology system with significant innovation and engineering feasibility. The above embodiments are merely specific application examples of this invention. Without departing from the core ideas of this invention, the channel length of Chaolian Island, UAV formation parameters, sensor configuration, and communication methods can all be adjusted and expanded.

[0066] The above embodiments possess the following technical features and advantages: This invention, when applied to typical port navigation scenarios such as the 30-nautical-mile long channel of Chaolian Island, can significantly improve navigation safety under limited visibility conditions. Existing research shows that reduced visibility in foggy weather significantly increases the probability of collisions and groundings. Therefore, maritime management agencies in various countries generally maintain safety through speed limits, one-way traffic, or even closure of waterways. This, in turn, leads to a significant reduction in port and waterway utilization. In the traditional model, pilots mainly rely on shipborne radar, AIS, and experience for fog navigation maneuvering, often adopting a conservative "better to wait longer than to move" strategy when encountering small targets such as small boats, fishing gear, and floating objects. This invention, through dual UAVs high-position forward-looking perception and 3D positioning, maps small targets within a 1-2 nautical-mile range ahead onto the pilot terminal and VTS 3D situation map with meter-level accuracy. This greatly reduces the uncertainty for pilots due to poor visibility ahead, fundamentally improving the way fog navigation safety boundaries are determined, shifting the safety margin from experience-based conservatism to quantitative assessment based on objective perception data.

[0067] Based on this, the present invention directly impacts the port's navigation capacity and effective opening hours during foggy weather. Public reports indicate that many ports along China's coast frequently experience continuous dense fog in spring and early summer, causing the average waiting time for ships to increase from the normal 1-2 days to more than 3 days, and in some periods even requiring several days to clear the backlog of ships. Port congestion studies generally agree that increased waiting times and berth utilization rates not only directly reduce port throughput capacity but also trigger a chain reaction of declining terminal revenue, reduced service levels, and shipping companies diverting to other ports. This invention provides additional forward-looking aerial sensors and real-time small target positioning capabilities for long waterways during foggy navigation, allowing maritime authorities and ports to appropriately relax visibility closure thresholds or reduce mandatory speed limits, under safe conditions. This transforms some periods that would otherwise require closure or strict restrictions into controllable, limited navigation periods, thereby increasing effective voyages and daily throughput. This is equivalent to virtually expanding the port's foggy navigation capacity without expanding waterways or adding berths.

[0068] From an economic perspective, international research shows that severe weather and visibility restrictions can directly reduce container terminal revenue by up to 20% of monthly income, amounting to tens of millions of US dollars. Long-term effects of port congestion include reduced revenue, increased debt risk, and decreased port competitiveness. Globally, maritime transport handles over 80% of trade volume and is considered to contribute 2%–3% to world GDP. Therefore, any technology that can improve the available throughput capacity of key hub ports has a significant macroeconomic multiplier effect. This invention helps shipping companies reduce hidden costs such as fuel consumption, demurrage fees, and mismatched cargo space by reducing arrival delays, anchorage waiting times, and berth demurrage caused by fog, while simultaneously improving the utilization rate of terminal loading and unloading equipment and liner on-time performance. Referring to the economic assessment results of precision navigation in US ports, similar high-precision navigation and sensing technologies can significantly increase throughput and regional economic activity by improving loading rates, shortening shipping times, and reducing waiting times. The forward-looking perception capabilities provided by this invention in foggy navigation environments have a similar amplification effect.

[0069] Furthermore, the system architecture of this invention offers significant cost-effectiveness advantages. Compared to constructing new pilotage channels, expanding berths, or deploying large-scale shore-based radar and electro-optical towers, upgrading existing escort tugboats to UAV take-off and landing platforms + RTK base stations + edge computing nodes + communication relays, and equipping pilots with tablet terminals and software systems, represents a relatively controllable investment and minimal disruption to existing production. Considering that ports often face frequent navigation closures or restrictions lasting 1-3 days during foggy seasons, if this invention can convert a portion of these closure times into safe and controllable navigation time, the construction investment can be recovered within several foggy seasons, and net profits can continue to be generated for ports and shipping companies thereafter.

[0070] Finally, this invention also enhances the reliability of port brands and services on an intangible level. Port congestion and frequent weather-related closures often force liner companies to adjust their port of call or liner network structure, causing incalculable negative impacts on the long-term competitiveness of ports. By deploying the system of this invention, ports can provide shipping companies and cargo owners with differentiated services such as visualized pilotage in foggy weather and three-dimensional situational monitoring of long channels, improving on-time performance and predictability under adverse weather conditions, and enhancing customer confidence in port timeliness and safety. This dual improvement in safety and efficiency in foggy navigation scenarios elevates this invention from a simple technological improvement to a crucial infrastructure supporting the digitalization, intelligentization, and high reliability of port operations, with a long-term positive impact on regional and even national logistics competitiveness and economic development.

[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A port long-channel fog navigation system based on UAV navigation and small target detection and positioning, characterized in that, include: The guided vessel and pilotage terminal, aerial drones and maritime support platforms; The piloted vessel and pilotage terminal layer includes: the piloted merchant ship and the pilotage tablet terminal; the piloted merchant ship sends its own position and attitude to the pilotage tablet terminal, and the pilotage tablet terminal displays electronic nautical charts, UAV videos and overlays high-risk target alarms; The aerial drones include: two drones operating in a forward formation navigation mode; the two drones respectively acquire images and point clouds at a fixed frequency, attach a unified timestamp and their own attitude information and send them to the tugboat RTX base station, which performs data fusion. The maritime support platform includes: a tugboat RTX base station. The tugboat RTX base station establishes a unified ENU coordinate system and transmits data with a data access gateway through a dedicated wireless link. It fuses observation data from two UAVs and combines the fused data with the position and attitude information of the guided merchant ship to perform three-dimensional positioning and collision avoidance calculations.

2. A port long-channel fog navigation system based on UAV navigation and small target detection and positioning as described in claim 1, characterized in that, The two drones were positioned relative to the guided merchant ship as follows: 1–2 km ahead of the centerline of the channel; 150–500 meters to the left and right horizontally; and 80–150 m in altitude.

3. A port long-channel fog navigation system based on UAV navigation and small target detection and positioning as described in claim 1, characterized in that, Both drones calculated their six degrees of freedom attitude and position using the RTK global navigation satellite system and inertial navigation unit, and periodically exchanged differential data with the tugboat's RTK base station.

4. A port long-channel fog navigation system based on UAV navigation and small target detection and positioning as described in claim 1, characterized in that, Each drone is equipped with at least one visible light camera and one lidar; the visible light camera provides high-resolution images; and the lidar provides high-precision geometric information at near to mid-range distances.

5. A port long-channel fog navigation system based on UAV navigation and small target detection and positioning as described in claim 1, characterized in that, The dual UAVs preprocess the acquired images and point clouds. The preprocessing includes time synchronization, candidate box determination, and feature extraction.

6. A port long-channel fog navigation system based on UAV navigation and small target detection and positioning as described in claim 1, characterized in that, The pilotage tablet terminal displays the position, track, and current operation mode of the piloted vessel, tugboat, and two UAVs on a high-resolution map or electronic nautical chart of the port area; it overlays a first-person view video window of the UAVs in the center of the map, displaying the channel ahead in real time, and overlays a category box and confidence percentage on the screen, while displaying the UAV status information below; the sidebar displays the calculated collision risk information in a list format, including the closest encounter time, closest distance, relative distance, and risk level automatically determined by the system for each target; Provide measurement tools that allow pilots to mark high-risk targets on aeronautical charts.

7. A fog navigation method for long port channels based on UAV navigation and small target detection and localization, characterized in that, The port long-channel fog navigation system based on UAV navigation and small target detection and positioning as described in any one of claims 1 to 6, the method comprising: S1. Two drones patrol in a predetermined formation over the channel ahead of the guided vessel, equipped with visible light cameras and lidar, to continuously scan the water surface ahead. S2. The two drones will collect images and point clouds, and transmit them back to the edge server on the tugboat via a dedicated wireless link with a unified timestamp and their own attitude information. S3. The edge server on the tugboat performs data fusion on the observation data of the two UAVs in the RTK unified coordinate system, and performs three-dimensional positioning and collision avoidance calculation based on the fused data and the position and attitude information of the guided merchant ship. S4. The calculated obstacle positions and movement information are pushed in real time to the pilot tablet terminal on the piloted vessel via a private network between ships. The pilot software visualizes and marks the obstacles on the electronic nautical chart / port map, and at the same time, the key data is transmitted back to the port VTS for the reconstruction of the three-dimensional traffic situation of the long channel.

8. A method for fog navigation in long port channels based on UAV navigation and small target detection and positioning as described in claim 7, characterized in that, Three-dimensional positioning and collision avoidance calculations include: Mapping from sensor coordinates to world coordinates: (using the first...) Taking a drone as an example, its camera intrinsic parameter matrix is ​​denoted as... The extrinsic parameters are denoted as rigid body transformation matrices. The attitude and position of the UAV body coordinate system to the world ENU coordinate system are denoted as... For a target detection box in an image, take its center pixel. Construct the normalized imaging plane vector: ； The vector is normalized and transformed to the UAV body coordinate system using extrinsic parameters: ； Then, by transforming the UAV's attitude to the world coordinate system, the direction of the observation ray originating from the camera's optical center is obtained: ； in Let this be the position of the camera's optical center in the world coordinate system. This refers to the translation of the camera relative to the body of the machine. Each target is represented as a parametric ray in the world coordinate system: ； (1) Single-machine sea surface plane constraint positioning mode: In maritime navigation scenarios, the sea surface is approximated as a flat plane for a short period of time. : ； in Typically, the vertically upward unit vector in the ENU coordinate system is taken. The constant is determined by the tide level and the datum; the target is a small boat or floating object on the water surface, and the solution is obtained by... With plane The intersection point yields its three-dimensional coordinates: ； (2) Dual-machine collaborative three-dimensional least squares positioning mode: Suppose that UAVs 1 and 2 observe the same target and obtain the ray. , Then the target's three-dimensional position Take the midpoint of the shortest line connecting the two rays: ; In the formula, The problem is a vector cross product; it can be transformed into a linear least squares solution, and its closed-form solution is expressed as: ； Conversion from world coordinates to piloted ship coordinates: The position of the piloted merchant ship in the world coordinate system is... The attitude matrix is The transformation from the world coordinate system to the ship's coordinate system is as follows: ； make Definition: Forward distance: Horizontal distance: Spatial distance: Relative azimuth: .

9. A method for fog navigation in long port channels based on UAV navigation and small target detection and positioning as described in claim 8, characterized in that, Three-dimensional localization and collision avoidance calculation also includes: establishing time-series tracking for each target and estimating its velocity vector and relative motion relationship; Set at two timestamps World coordinates for the same target The average velocity of the target in the world coordinate system is estimated as follows: ; The piloted vessel's own velocity vector Based on the trajectory calculation, the relative speed of the target relative to the ship is: ; Transform to ship coordinate system: ; At the target location With relative velocity Given the target and the ship, calculate the closest encounter time (TCPA) and closest distance (CPA) under the assumption of linear relative motion: ； Based on CPA, TCPA, and set thresholds, targets are classified into multiple levels, and the results are pushed to the pilot terminal and VTS system.

10. A method for fog navigation in long port channels based on UAV navigation and small target detection and positioning according to claim 7, characterized in that, The reconstruction of the three-dimensional traffic situation of the long waterway includes: using the port area DEM / modeling results as a background, rendering the positions and trajectories of guided ships, tugboats, drones and small targets in three-dimensional coordinates in real time, and calling the first-person view videos of drones and guided merchant ships to achieve three-dimensional monitoring and remote verification.

Images