Navigation mark anti-collision early warning method based on multi-sensor fusion

Abstract

The invention belongs to the technical field of water traffic transportation safety and navigation mark navigation aiding, and mainly relates to a navigation mark anti-collision early warning method based on multi-sensor fusion, and the method comprises the steps: constructing a multi-sensor fusion architecture integrating an AIS terminal, a laser radar and an AI camera, and achieving the omnibearing monitoring and precise recognition of a ship in a navigation channel; a grading response mechanism is adopted, corresponding functions are dynamically started according to the real-time distance between the ship and the navigation mark, and targeted response from early warning to emergency warning is achieved; a legal evidence chain packaging mechanism is introduced, a unique ID is generated for each early warning event, formatted evidence is stored in an associated mode, and integrity and traceability of data are ensured; through a dynamic energy supply strategy, an equipment operation mode is intelligently adjusted according to a storage battery voltage threshold value, and continuous and stable work of the system under the conditions of cloudy and rainy days, night and the like is guaranteed; according to the invention, the accuracy, timeliness and reliability of navigation mark collision early warning information are effectively improved.

Description

Technical Field

[0001] This invention belongs to the field of waterway transportation safety and navigation aid technology, and mainly relates to a navigation aid collision avoidance early warning method based on multi-sensor fusion. Background Technology

[0002] Navigational aids are devices that provide positioning and warnings for ships, and are widely used in maritime, inland waterways, and waterways. Due to factors such as complex waterways, dense shipping traffic, and severe weather, navigational aids face significant collision risks. Ships may fail to effectively avoid navigational aids during navigation due to changes in direction or speed. In poor weather or low visibility, crew members may have difficulty clearly seeing navigational aids, leading to collisions. Existing ordinary navigational aids cannot detect the dynamic trajectory of ships and can only be dealt with after a collision has occurred. Furthermore, existing navigational aids have limited functionality, lacking environmental monitoring and ship interaction capabilities, and only providing visual navigation assistance. Some navigational aids are now beginning to incorporate collision warning functions.

[0003] For example, in Chinese patent application CN202010324360.2, "An Intelligent Navigation Aid Device and Its Early Warning Method," the invention relates to the field of navigation equipment technology, and particularly to an intelligent navigation aid device and its early warning method. This device includes an AIS receiving unit, a control unit, a compass sensor, a positioning unit, and an LED warning light. The control unit integrates a Raspberry Pi PC and an 8-channel switch controller. After receiving AIS messages, the AIS receiving unit forwards the messages to the Raspberry Pi PC in the control unit for data processing. The AIS receiving unit receives AIS messages from vessels in the vicinity of the navigation aid, parses them, and extracts the vessel's coordinates, speed, and heading information. The compass sensor is used to obtain the navigation aid's rotation angle, and the positioning unit is used to obtain the navigation aid's position. The control unit controls the LED warning light. This invention integrates multiple information technologies to collect real-time navigation environment data. When a vessel approaches, it can proactively warn the vessel to avoid it, reducing economic losses and navigation risks caused by collisions, greatly improving the efficiency of nighttime navigation aids, and making waterway navigation safer.

[0004] The aforementioned patents have provided a relatively complete method for warning collisions with navigation aids, but existing technologies still have certain technical bottlenecks:

[0005] Firstly, at the perception level, there is a lack of collaborative capabilities among multiple sensors, resulting in data silos. AIS, radar, and video surveillance typically operate independently without effective integration. For example, while AIS can identify the identity of ships equipped with the device within 200 meters, it cannot cover many small ships without AIS. Although radar has all-weather ranging capabilities, its classification error for targets beyond 25 meters often exceeds 30%. Video surveillance, on the other hand, heavily relies on lighting conditions, and the ship recognition rate drops significantly at night.

[0006] Secondly, when multiple targets appear simultaneously, it is difficult to accurately match the radar detection points with the ship identities reported by AIS, or the single detection method lacks an effective target matching method, resulting in inaccurate target matching.

[0007] Secondly, existing early warning mechanisms have certain problems. They mostly use a single fixed distance threshold to trigger alarms, resulting in invalid alarms for normally passing vessels outside the safe distance. Furthermore, once a vessel approaches rapidly, the system's response is delayed due to the lack of a tiered response buffer.

[0008] Moreover, the existing technology has weak evidence collection capabilities for early warning events, only retaining basic logs and failing to form a complete chain of evidence linking the ship's identity, real-time movement trajectory and on-site video recordings;

[0009] Meanwhile, the energy supply is unstable. Existing technology mainly relies on photovoltaic power. During continuous rain or at night, the battery voltage is prone to drop below 12V, causing all equipment to shut down and key early warning functions to be paralyzed.

[0010] Furthermore, the communication broadcasting process was ineffective. VHF warning broadcasts relied on public channels, which were highly susceptible to interference from other channels in areas with dense shipping traffic, resulting in an actual information delivery rate of less than 40%.

[0011] The interplay of these issues results in significant deficiencies in existing technologies regarding information accuracy, response timeliness, power supply stability, and communication reliability.

[0012] To address the aforementioned issues, this invention proposes a navigational aid collision avoidance and early warning method based on multi-sensor fusion. This method acquires real-time vessel position and identification information via an AIS terminal, combining high-precision distance measurement from lidar with the visual recognition capabilities of an AI camera to construct a multi-sensor fusion architecture, enabling comprehensive monitoring of vessels within the waterway. A tiered response mechanism dynamically triggers different levels of warning commands based on the distance between the vessel and the navigational aid, achieving early warning and targeted alerts, avoiding false alarms and missed alarms caused by indiscriminate broadcasting. A legal evidence chain encapsulation mechanism is employed to format and store vessel-related data, ensuring data integrity and traceability during collision warnings, further enhancing system reliability. A dynamic power supply strategy intelligently adjusts equipment operating modes based on voltage conditions, ensuring stable operation even in extreme weather and low-power conditions, meeting all-weather service requirements. Summary of the Invention

[0013] This invention provides a navigation mark collision avoidance and early warning method based on multi-sensor fusion, aiming to solve the problems in the existing technology such as lack of collaborative working ability among multiple sensors, difficulty in accurately matching multiple targets, crude early warning mechanism, insufficient energy supply stability, and poor effect of communication broadcasting.

[0014] To solve the above problems, the present invention employs the following technology:

[0015] A navigational aid collision avoidance and early warning method based on multi-sensor fusion:

[0016] Key technologies include: multi-sensor fusion architecture, hierarchical response mechanism, legal evidence chain encapsulation mechanism, and dynamic power supply strategy;

[0017] The implementation steps of the method are as follows:

[0018] Step S1: Build the hardware foundation on the navigation beacon and deploy the necessary hardware equipment for the multi-sensor fusion architecture, hierarchical response mechanism, legal evidence chain encapsulation mechanism, and dynamic power supply strategy.

[0019] Step S2: The hardware device of the multi-sensor fusion architecture searches for ships within range and uses a dynamic response mechanism to trigger different commands to ships at different distances.

[0020] Step S3: Activate the legal evidence chain encapsulation mechanism, encapsulate the formatted evidence data, and upload and store the key original data;

[0021] Step S4: During operation, the system power supply voltage is continuously monitored using a dynamic power supply strategy, and the power supply strategy is dynamically adjusted according to the real-time voltage status.

[0022] As a preferred embodiment, the main technology specifically includes:

[0023] The multi-sensor fusion architecture includes device-level collaborative design and data fusion algorithms. Device-level collaborative design refers to the fact that the device hardware includes at least an AIS terminal, VHF, LiDAR, and AI camera.

[0024] The graded response mechanism includes a two-level trigger chain design and dynamic response rules. The two-level trigger chain design refers to classifying the distance into 30 meters and 50 meters. The dynamic response rules refer to triggering different levels of response commands based on the preset distance from the ship to the trigger chain.

[0025] The legal evidence chain encapsulation mechanism includes evidence encapsulation technology and evidence solidification technology. Evidence encapsulation technology refers to formatting and encapsulating ship navigation data; evidence solidification technology refers to adding watermarks to video recordings and storing the original data directly in the cloud.

[0026] The dynamic power supply strategy includes voltage threshold control logic and corresponding hardware support. The voltage threshold control logic refers to automatically adjusting the operation of the equipment according to the different power supply capabilities. The corresponding hardware support includes at least solar panels, batteries, smart relays and MPPT controllers.

[0027] As a preferred implementation, the construction of the hardware infrastructure on the navigation beacon specifically includes:

[0028] The hardware devices for the multi-sensor fusion architecture include AIS terminals, VHF, LiDAR, and AI cameras;

[0029] The hardware equipment for the graded response mechanism includes: a VHF radio that executes voice and audible / visual warning commands, and an audible / visual alarm device;

[0030] The hardware equipment for the legal evidence chain encapsulation mechanism includes: an edge computing server for data fusion, logical judgment, evidence encapsulation and remote communication, and a 4G / 5G communication module;

[0031] The hardware for the dynamic power supply strategy includes: solar panels, batteries, MPPT controllers, smart relays; and other auxiliary equipment installed on the beacon.

[0032] As a preferred implementation, the hardware device of the multi-sensor fusion architecture searches for ships within its range, and employs a dynamic response mechanism to trigger different commands for ships at different distances. Specifically, this includes:

[0033] The AIS terminal continuously monitors the waterway, receives AIS messages broadcast by ships within its range, and calculates the distance between the monitored ships and the navigation mark.

[0034] When the AIS terminal device detects that a vessel within range is less than 50 meters from the navigation mark, it triggers the first-level response. It sends a short warning message and a warning voice to the vessel via AIS and VHF respectively, and starts the lidar. Using the AIS and radar target matching algorithm in the multi-sensor fusion architecture, it calculates the distance difference between each radar target and the vessel position reported by AIS. Based on the distance difference, it matches the radar target with the AIS vessel and identifies the vessel by combining the database. The lidar continuously outputs the target vessel's motion information.

[0035] When the lidar detects that the target is less than 30 meters away from the navigation mark, the final collision warning of the second-level response is triggered. The AI ​​camera rotates according to the target orientation data sent by the lidar, turns towards the target vessel, and continues to track the vessel. Using the vessel name extracted through the AIS terminal and dynamically filled in the vessel name with the preset template, the sound and light alarm is activated and the VHF warning is broadcast. At the same time, the AI ​​camera begins to record the vessel's video data and embeds watermark information in real time into the video stream.

[0036] Once the target vessel is more than 30 meters away from the buoy, the monitoring will stop recording, save the video evidence file, and stop the audible and visual alarms and VHF broadcasts. The AIS terminal will continue to monitor the vessel's distance until the target vessel leaves a range of more than 50 meters from the buoy.

[0037] As a preferred implementation, the AIS and radar target matching algorithm specifically includes:

[0038] After the LiDAR identifies the ship point cloud data within its range, point cloud clustering is performed using the Euclidean clustering algorithm. Clustering distance threshold, minimum cluster size, and maximum cluster size are set. All unlabeled points in the point cloud are traversed, and each point is used as a seed point to search for all nearest neighbors within the distance threshold. This process is recursively expanded until no new nearest neighbors can be found, thus forming a cluster.

[0039] After clustering, the location of the cluster centroid is calculated. For each cluster, the arithmetic mean of the X and Y coordinates of all its points is calculated. ;

[0040] After obtaining the centroid of the cluster, the distance D from the centroid is calculated using lidar. Then, the distance between the AIS and the beacon is calculated. The beacon's latitude and longitude are obtained using its own BeiDou positioning module. The ship's latitude and longitude are then extracted from its AIS signal. The spherical distance between two latitude and longitude points is calculated using the semi-versus formula:

[0041] ;

[0042] in, For the Earth's radius, The latitude of the navigation mark. For the ship's latitude, The longitude of the navigation mark. The longitude of the ship;

[0043] Calculate the ratio between the distance D from the centroid and the distance d from the AIS signal. If the ratio error is less than 5%, it is considered the same target. When the target within the range is not bound, the operation continues. Once all ship targets within the range are identified and bound, the operation stops.

[0044] As a preferred implementation, the activation of the legal evidence chain encapsulation mechanism, specifically encapsulating formatted evidence data, includes:

[0045] During operation, the evidence solidification technology is automatically activated to record the ship information output by the AIS terminal; during the lidar tracking phase, detailed motion information of the target ship is recorded; and during the AI ​​camera video recording phase, a video file containing a watermark is generated and recorded.

[0046] During operation, when the lidar detects a collision warning, it immediately generates a globally unique event case ID; this unique case ID is then used to tag the data collected for this event, enabling the association of all evidence elements.

[0047] When AIS detects that the target vessel has left the 50-meter range, the warning event ends and the final encapsulation of the evidence chain is automatically triggered, automatically generating a JSON format data file.

[0048] As a preferred implementation, uploading and storing the key raw data specifically includes:

[0049] Evidence preservation technology includes the preservation of original data, which involves locally storing the original AIS messages and the original LiDAR point cloud data.

[0050] The raw data is integrated with evidence elements in JSON format; the entire evidence data package is then encrypted and uploaded to the cloud evidence repository via wireless network.

[0051] As a preferred implementation, the step of continuously monitoring the system power supply voltage using a dynamic power supply strategy during operation and dynamically adjusting the power supply strategy according to the real-time voltage status specifically includes:

[0052] The output voltage of the solar battery is continuously collected by a smart relay, and its magnitude is monitored as an indicator for adjusting the state of energy. The MPPT controller is used to optimize the solar charging efficiency.

[0053] The power supply strategy is dynamically adjusted based on the output voltage. When the voltage is ≥13V, the full-function mode is activated, and all devices are started.

[0054] When the voltage is ≥12V and <13V, it enters power saving mode, continuously runs the AIS terminal and LiDAR, and maintains 4G / 5G communication to ensure core perception capabilities; visual evidence collection is activated as needed, and the AI ​​camera is only activated under daylight conditions; unnecessary equipment is turned off.

[0055] When the voltage is less than 12V, the system enters keep-alive mode, shutting down all sensing devices; only the 4G / 5G communication module remains online; and alarm information is immediately sent to the remote monitoring platform.

[0056] As a preferred implementation, in order to deal with ships without AIS terminals in real-world scenarios, a lightweight YOLO-based target recognition algorithm is integrated into the AI ​​camera. During the pre-training stage, the algorithm focuses on identifying ships from videos in real time and integrates a simple distance estimation algorithm based on monocular vision.

[0057] During use, the AI ​​camera maintains patrol monitoring. When a ship-type object is within 50 meters of the monitoring range, the AIS and LiDAR are activated to measure the distance to the target ship, triggering the first and second level strategies. Combined with preset alarm templates, the ship is warned to stay away.

[0058] As a preferred implementation, in cases where the ship is not equipped with an AIS terminal, considering the legality of ships without AIS terminals, after the lidar is directly activated by video surveillance, when the lidar calculates that the distance between the target ship and the navigation mark is clearly less than 50 meters, and the AIS terminal still has not received the AIS signal, an image is captured from the video stream, uploaded to the cloud, and an alarm message is sent simultaneously.

[0059] The beneficial effects of this invention are:

[0060] 1. Multi-level, high-precision fusion perception and early warning have been achieved; by constructing a response chain of "AIS identity recognition → radar trajectory tracking → laser precise ranging → video visual evidence collection", the entire process from long-distance identity recognition to close-range precise tracking and evidence collection has been covered, significantly improving the perception accuracy and early warning timeliness of the ship approach process.

[0061] 2. Enhanced cross-sensor collaboration and automated tracking capabilities; Through real-time conversion from radar coordinate system to gimbal coordinate system, the AI ​​camera achieved automatic and continuous tracking of target ships, solving the problems of inaccurate target matching and slow linkage response of multi-sensor systems, and enhancing the system's target locking and monitoring capabilities in complex scenarios;

[0062] 3. Enhanced the legal validity and traceability of early warning data; by synchronously storing multiple sources of raw data such as video recordings, lidar point clouds, and AIS messages, and realizing their correlation and encapsulation in the time and event dimensions, a complete data chain with legal evidentiary effect is formed, which can provide a reliable basis for the determination of liability in collision accidents;

[0063] 4. It ensures the system's continuous operation under harsh power supply conditions; through an adaptive power supply strategy based on voltage thresholds, it realizes intelligent hierarchical adjustment of equipment operation modes. Even in situations where photovoltaic power supply is insufficient, such as during continuous rain or at night, it can still maintain core sensing and communication functions, significantly improving the reliability of the system's 24 / 7 uninterrupted service. Attached Figure Description

[0064] Figure 1 This is a flowchart of the method of the present invention;

[0065] Figure 2 This is a flowchart of the multi-level early warning process of the present invention. Detailed Implementation

[0066] To make the technical means, creative features, and achieved objectives and effects of this invention easier to understand, the invention is further described below with reference to specific embodiments. However, the following embodiments are merely preferred embodiments of this invention and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments described herein without creative effort are all within the protection scope of this invention. Unless otherwise specified, the experimental methods in the following embodiments are conventional methods, and the materials and reagents used in the following embodiments are commercially available unless otherwise specified.

[0067] Example 1: This invention systematically solves the defects in the background technology through four core technologies: multi-sensor fusion architecture, hierarchical response mechanism, legal evidence chain encapsulation mechanism, and dynamic power supply strategy.

[0068] The multi-sensor fusion architecture includes device-level collaborative design and data fusion algorithms. Specifically, device-level collaborative design refers to the device hardware including at least an AIS terminal, LiDAR, and an AI camera.

[0069] The graded response mechanism includes a two-level trigger chain design and dynamic response rules. Specifically, the two-level trigger chain design refers to classifying the distance into 30 meters and 50 meters; the dynamic response rules refer to triggering different levels of response commands based on the preset distance from the ship to the trigger chain.

[0070] The legal evidence chain encapsulation mechanism includes evidence encapsulation technology and evidence solidification technology. Specifically, evidence encapsulation technology refers to formatting and encapsulating ship navigation data; evidence solidification technology refers to adding watermarks to video recordings and storing the original data directly in the cloud.

[0071] The dynamic power supply strategy includes voltage threshold control logic and corresponding hardware support. Specifically, the voltage threshold control logic refers to automatically adjusting the operation of the equipment according to the different power supply capabilities; the corresponding hardware support includes at least solar panels, batteries, smart relays and MPPT controllers.

[0072] like Figure 1 The flowchart of this invention provides a usage process for a navigation beacon collision avoidance warning method based on multi-sensor fusion, specifically including the following:

[0073] Step S1: Build the hardware foundation on the navigation beacon and deploy the necessary hardware equipment for the multi-sensor fusion architecture, hierarchical response mechanism, legal evidence chain encapsulation mechanism, and dynamic power supply strategy.

[0074] Specifically, the hardware foundation for building navigation marks includes hardware devices with a multi-sensor fusion architecture such as AIS terminals, LiDAR, and AI cameras.

[0075] The hardware equipment for the graded response mechanism includes: a VHF radio that executes voice and audible / visual warning commands, and an audible / visual alarm device;

[0076] The hardware equipment for the legal evidence chain encapsulation mechanism includes: an edge computing server for data fusion, logical judgment, evidence encapsulation and remote communication, and a 4G / 5G communication module;

[0077] The hardware for dynamic power supply strategies includes: solar panels, batteries, MPPT controllers, and smart relays;

[0078] In addition, to optimize the operation, other auxiliary equipment that can be added to the navigation beacon platform to improve the effect, such as Beidou positioning modules, or anti-shake brackets that are installed in conjunction with AI cameras and LiDAR. These brackets are equipped with gyroscopes and, based on the amplitude of the ship's sway, use motors to drive the brackets to compensate for the camera's sway, thereby reducing video image shake and optimizing the tracking effect.

[0079] Step S2: The hardware device of the multi-sensor fusion architecture searches for ships within range and uses a dynamic response mechanism to trigger different commands to ships at different distances.

[0080] Specifically, such as Figure 2 As shown in the multi-level early warning flowchart of this invention, after the device is started, different instructions are triggered by a dynamic response mechanism. The AIS terminal continuously monitors the waterway, receives AIS messages broadcast by ships within range, and calculates the distance between the monitored ships and the navigation mark.

[0081] When the AIS terminal device detects that a vessel within range is less than 50 meters from the navigation mark, it triggers the first-level response. It sends a short warning message and a warning voice message to the vessel via AIS and VHF respectively, and activates the lidar. The lidar begins scanning and returns a series of unidentified target point cloud data in terms of distance and azimuth. Using the AIS and radar target matching algorithm in the multi-sensor fusion architecture, it calculates the distance difference between each radar target and the vessel's reported position by AIS. If the distance difference is within a 5% tolerance threshold, the radar target and the AIS vessel are identified as the same target, and the target is bound to a pre-stored vessel identity in the database, thus assigning a clear identity to the anonymous radar point cloud target. After confirming that the target vessel matches the AIS signal identity, the lidar begins to accurately and frequently output its real-time distance, azimuth, and speed to the vessel, forming a precise trajectory.

[0082] When the lidar detects that the target is less than 30 meters away from the navigation mark, it triggers the final collision warning of the second-level response. The lidar further calculates the azimuth angle based on the acquired point cloud data and drives the AI ​​camera through the ONVIF interface via the PTZ control protocol. The AI ​​camera rotates according to the lidar, turns towards the target ship, and continuously tracks the ship based on the azimuth angle changes measured by the lidar. At the same time, the ship name extracted through the AIS terminal is dynamically filled in using a preset template, and then the audible and visual alarm is activated, starting to broadcast a VHF warning: "Warning: {ship name} stay away from this navigation mark!" At the same time, the AI ​​camera starts recording ship video data and embeds watermark information such as ship name, timestamp, and navigation mark location into the video stream in real time.

[0083] Continuous monitoring will continue until the lidar determines that the target vessel is moving further away and eventually leaves the 30-meter monitoring range. Once this is achieved, recording will stop, video evidence files will be saved, and audible and visual alarms and VHF broadcasts will be stopped. The AIS terminal will continue to monitor the vessel's distance until the target vessel leaves a range of more than 50 meters from the buoy.

[0084] Step S3: Activate the legal evidence chain encapsulation mechanism, encapsulate the formatted evidence data, and upload and store the key original data;

[0085] Specifically, during operation, the evidence solidification technology is automatically activated. Once the AIS terminal detects a vessel within 50 meters and captures its identity, the MMSI code and vessel name are immediately stored in the cache as core evidence elements. During the lidar tracking phase, trajectory points with high-precision timestamps are recorded at high frequency along with the lidar output. Each point includes distance, azimuth, and data source. During the AI ​​camera video recording phase, watermark information such as vessel name, MMSI code, timestamp, and navigation mark location is embedded in the video stream in real time, generating a video file containing watermarks.

[0086] Meanwhile, during operation, when the lidar detects that the final collision warning (30 meters) has been triggered, a globally unique event case ID is immediately generated. The generation rule is: Case ID = Beacon ID + "_" + Event start timestamp. After that, all data collected for this event (identity, trajectory, video, raw data) is tagged with this unique case ID, realizing the association of all evidence elements.

[0087] Evidence preservation technology also includes raw data preservation, which involves locally storing the original, uncompressed or unprocessed AIS messages and raw LiDAR point cloud data.

[0088] When AIS detects that the target vessel has left the 50-meter range, the warning event ends and the final encapsulation of the evidence chain is automatically triggered. The system automatically generates a JSON format data file, integrating evidence elements from different sources. The generation rule is: {navigation mark ID + timestamp, vessel name, MMSI code, radar trajectory, video stream address, and original laser point cloud data}. After generation, the entire evidence data package (including the JSON description file, video file, and original data file) is immediately encrypted and uploaded to the cloud evidence repository via a 4G / 5G network.

[0089] Step S4: During operation, the system power supply voltage is continuously monitored using a dynamic power supply strategy, and the power supply strategy is dynamically adjusted according to the real-time voltage status.

[0090] Since navigation marks need to be deployed on waterways for extended periods, and considering the impact of rainy weather on solar charging, a dynamic power supply strategy is designed to trigger equipment adjustment commands based on the equipment's power level, ensuring the long-term normal operation of the equipment.

[0091] Specifically, the output voltage of the solar battery is continuously collected by a smart relay, and the magnitude of the output voltage is monitored as an indicator for adjusting the state of energy. The MPPT controller is used to optimize the solar charging efficiency.

[0092] Based on the output voltage, the power supply strategy is dynamically adjusted. When the voltage is ≥13V, the full-function mode is activated, and all devices (AIS terminal, LiDAR, AI camera, edge server, 4G / 5G module, VHF radio) are started.

[0093] When the voltage is ≥12V and <13V, it enters power saving mode, continuously runs the AIS terminal and LiDAR, and maintains 4G / 5G communication to ensure core perception capabilities; visual evidence collection is started as needed, and the AI ​​camera is only started under daylight conditions; unnecessary devices are shut down, such as non-core computing tasks of edge servers, such as in-depth analysis of system operation logs, long-term storage and compression of historical data, etc.

[0094] When the voltage is less than 12V, it enters keep-alive mode, shutting down all sensing devices including the AIS terminal, LiDAR, and AI camera; only the 4G / 5G communication module remains online; and a "low battery alarm" message is immediately sent to the remote monitoring platform.

[0095] Example 2: The AIS and radar target matching algorithm mentioned in Example 1 includes the following: After the lidar identifies the ship point cloud data within its range, point cloud clustering is first performed using the Euclidean clustering algorithm. A clustering distance threshold is set, which should be greater than the point cloud spacing between different parts of the ship (such as the hull and deck) and less than the minimum safe distance between two ships, such as 2.0 meters. A minimum cluster number is set to filter noise and avoid misidentifying a few floating objects as targets, such as 10 points. A maximum cluster number is set to prevent misidentifying a continuous shoreline in the distance as a giant target, such as 5000 points. Subsequently, all unlabeled points in the point cloud are traversed, using them as seed points to search for all nearest neighbor points within the distance threshold, recursively expanding until no new nearest neighbor points can be found, thus forming a cluster.

[0096] After clustering, the position of the cluster centroid is calculated for distance comparison with the AIS position. The calculation method is as follows: for each cluster, calculate the arithmetic mean of all its points in the X and Y coordinates (horizontal plane). ;

[0097] After obtaining the centroid of the cluster, the distance D from the centroid is calculated using lidar. Then, the distance between the AIS and the beacon is calculated. The beacon's latitude and longitude are obtained using its own BeiDou positioning module. The ship's latitude and longitude are then extracted from its AIS signal. The spherical distance between two latitude and longitude points is calculated using the semi-versus formula:

[0098] ;

[0099] in, For the Earth's radius, The latitude of the navigation mark. For the ship's latitude, The longitude of the navigation mark. The longitude of the ship;

[0100] Calculate the ratio between the distance D from the centroid and the distance d from the AIS signal. If the ratio error is less than 5%, it is considered the same target. When the target within the range is not bound, the operation continues. Once all ship targets within the range are identified and bound, the operation stops.

[0101] Example 3: In waterways, there may be vessels without AIS terminals, such as small fishing boats, small cargo ships, and yachts. To address this, a lightweight YOLO-based target recognition algorithm is integrated into the AI ​​camera. The model undergoes pruning and quantization optimizations to ensure low computational overhead, sufficient for stable operation on edge devices alongside other core tasks. During the pre-training phase, it focuses on real-time identification of the "ship" category from video and integrates a simple distance estimation algorithm based on monocular vision. During use, the AI ​​camera maintains continuous monitoring. When a ship-type object is detected within 50 meters of the monitoring range, AIS and LiDAR are directly activated to measure the distance to the target ship, triggering first and second level strategies and pre-setting corresponding alarm templates to warn the ship to move away.

[0102] Furthermore, considering the legality of ships not equipped with AIS terminals during application, after the lidar is directly activated by video surveillance, when the lidar calculates that the distance between the target ship and the navigation mark is clearly less than 50 meters, and the AIS terminal still has not received the AIS signal, an image is captured from the video stream, uploaded to the cloud, and an alarm message is sent simultaneously.

[0103] Example 4: This example uses a left-hand navigation mark on the Wuhan section of the Yangtze River as an application scenario. After leaving Wuhan Port, a large cargo ship's course gradually shifts to the left side of the channel due to the influence of the water flow, approaching the red buoy of Tianxingzhou Bridge.

[0104] The AIS terminal on the buoy first captures the ship's dynamic signals and calculates its distance to the buoy in real time. When the system determines that the ship is only 48 meters away from the buoy, it immediately triggers the first-level warning. The lidar then activates, scans the target area, and quickly identifies the ship's outline using a point cloud clustering algorithm. The built-in fusion algorithm compares the ship's center of gravity position calculated by the lidar with the position reported by the AIS. The distance error between the two is within 3%, successfully confirming that the lidar target is the ship that sent the AIS signal. Subsequently, the lidar begins to continuously output the ship's precise distance, bearing, and speed information at a frequency of 10 Hz.

[0105] As the vessel continued to approach, the lidar detected that the distance had decreased to 28 meters, and the system immediately triggered a final collision warning. Guided by the lidar, the AI ​​camera quickly turned and locked onto the target vessel. Simultaneously, the system activated its audible and visual alarms and broadcast a clear voice warning via a pre-coordinated dedicated VHF radio on a dedicated channel: "Reminder: XX, stay away from this buoy!" The vessel name in the broadcast was dynamically generated from AIS data, achieving targeted warnings and avoiding interference from public channels and indiscriminate broadcasts.

[0106] At the same time, the AI ​​camera began recording video evidence and embedded watermark information such as the ship name, timestamp, and navigation mark location into the video in real time; the system generated a unique case ID (WH-TXZ-001_20251018_173205) for this event and associated all data such as AIS information, LiDAR trajectory, and video stream to this ID.

[0107] Upon hearing the targeted call, the ship's navigator promptly adjusted course, and the vessel began to gradually move away. Continuous lidar monitoring showed that once the distance returned to 31 meters, the system automatically stopped recording and issuing alarms, but AIS continued monitoring until the vessel was out of range by 50 meters. After the incident, the system automatically encrypted and packaged the formatted AIS message, raw lidar point cloud data, watermarked video files, and a JSON description file integrating all evidence chain information generated during the warning process, and uploaded them to a cloud storage platform, forming a complete legal evidence chain.

[0108] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A navigational aid collision avoidance and early warning method based on multi-sensor fusion, characterized in that: Key technologies include: multi-sensor fusion architecture, hierarchical response mechanism, legal evidence chain encapsulation mechanism, and dynamic power supply strategy; The implementation steps of the method are as follows: Step S1: Build the hardware foundation on the navigation beacon and deploy the necessary hardware equipment for the multi-sensor fusion architecture, hierarchical response mechanism, legal evidence chain encapsulation mechanism, and dynamic power supply strategy. Step S2: The hardware device of the multi-sensor fusion architecture searches for ships within range and uses a dynamic response mechanism to trigger different commands to ships at different distances. Step S3: Activate the legal evidence chain encapsulation mechanism, encapsulate the formatted evidence data, and upload and store the key original data; Step S4: During operation, the system power supply voltage is continuously monitored using a dynamic power supply strategy, and the power supply strategy is dynamically adjusted according to the real-time voltage status.

2. The navigation mark collision avoidance early warning method based on multi-sensor fusion according to claim 1, characterized in that: The main technologies specifically include: The multi-sensor fusion architecture includes device-level collaborative design and data fusion algorithms. Device-level collaborative design refers to the fact that the device hardware includes at least an AIS terminal, VHF, LiDAR, and AI camera. The graded response mechanism includes a two-level trigger chain design and dynamic response rules. The two-level trigger chain design refers to classifying the distance into 30 meters and 50 meters. The dynamic response rules refer to triggering different levels of response commands based on the preset distance from the ship to the trigger chain. The legal evidence chain encapsulation mechanism includes evidence encapsulation technology and evidence solidification technology. Evidence encapsulation technology refers to formatting and encapsulating ship navigation data; evidence solidification technology refers to adding watermarks to video recordings and storing the original data directly in the cloud. The dynamic power supply strategy includes voltage threshold control logic and corresponding hardware support. The voltage threshold control logic refers to automatically adjusting the operation of the equipment according to the different power supply capabilities. The corresponding hardware support includes at least solar panels, batteries, smart relays and MPPT controllers.

3. The navigational aid collision avoidance early warning method according to claim 1, characterized in that: The construction of the hardware infrastructure on the navigation mark specifically includes: The hardware devices for the multi-sensor fusion architecture include AIS terminals, VHF, LiDAR, and AI cameras; The hardware equipment for the graded response mechanism includes: a VHF radio that executes voice and audible / visual warning commands, and an audible / visual alarm device; The hardware equipment for the legal evidence chain encapsulation mechanism includes: an edge computing server for data fusion, logical judgment, evidence encapsulation and remote communication, and a 4G / 5G communication module; The hardware for the dynamic power supply strategy includes: solar panels, batteries, MPPT controllers, smart relays; and other auxiliary equipment installed on the beacon.

4. The navigational aid collision avoidance early warning method according to claim 1, characterized in that: The hardware device of the multi-sensor fusion architecture searches for ships within its range and uses a dynamic response mechanism to trigger different commands for ships at different distances. Specifically, the commands triggered for ships at different distances include: The AIS terminal continuously monitors the waterway, receives AIS messages broadcast by ships within its range, and calculates the distance between the monitored ships and the navigation mark. When the AIS terminal device detects that a vessel within range is less than 50 meters from the navigation mark, it triggers the first-level response. It sends a short warning message and a warning voice to the vessel via AIS and VHF respectively, and starts the lidar. Using the AIS and radar target matching algorithm in the multi-sensor fusion architecture, it calculates the distance difference between each radar target and the vessel position reported by AIS. Based on the distance difference, it matches the radar target with the AIS vessel and identifies the vessel by combining the database. The lidar continuously outputs the target vessel's motion information. When the lidar detects that the target is less than 30 meters away from the navigation mark, the final collision warning of the second-level response is triggered. The AI ​​camera rotates according to the target orientation data sent by the lidar, turns towards the target vessel, and continues to track the vessel. Using the vessel name extracted through the AIS terminal and dynamically filled in the vessel name with the preset template, the sound and light alarm is activated and the VHF warning is broadcast. At the same time, the AI ​​camera begins to record the vessel's video data and embeds watermark information in real time into the video stream. Once the target vessel is more than 30 meters away from the buoy, the monitoring will stop recording, save the video evidence file, and stop the audible and visual alarms and VHF broadcasts. The AIS terminal will continue to monitor the vessel's distance until the target vessel leaves a range of more than 50 meters from the buoy.

5. The navigational aid collision avoidance early warning method according to claim 4, characterized in that: The AIS and radar target matching algorithm specifically includes: After the LiDAR identifies the ship point cloud data within its range, point cloud clustering is performed using the Euclidean clustering algorithm. Clustering distance threshold, minimum cluster size, and maximum cluster size are set. All unlabeled points in the point cloud are traversed, and each point is used as a seed point to search for all nearest neighbors within the distance threshold. This process is recursively expanded until no new nearest neighbors can be found, thus forming a cluster. After clustering, the location of the cluster centroid is calculated. For each cluster, the arithmetic mean of the X and Y coordinates of all its points is calculated. ; After obtaining the centroid of the cluster, the distance D from the centroid is calculated using lidar. Then, the distance between the AIS and the beacon is calculated. The beacon's latitude and longitude are obtained using its own BeiDou positioning module. The ship's latitude and longitude are then extracted from its AIS signal. The spherical distance between two latitude and longitude points is calculated using the semi-versus formula: ； in, For the Earth's radius, The latitude of the navigation mark. For the ship's latitude, The longitude of the navigation mark. The longitude of the ship; Calculate the ratio between the distance D from the centroid and the distance d from the AIS signal. If the ratio error is less than 5%, it is considered the same target. When the target within the range is not bound, the operation continues. Once all ship targets within the range are identified and bound, the operation stops.

6. The navigation mark collision avoidance early warning method based on multi-sensor fusion according to claim 1, characterized in that: The aforementioned activation of the legal evidence chain encapsulation mechanism specifically includes the encapsulation of formatted evidence data, including: During operation, the evidence solidification technology is automatically activated to record the ship information output by the AIS terminal; during the lidar tracking phase, detailed motion information of the target ship is recorded; and during the AI ​​camera video recording phase, a video file containing a watermark is generated and recorded. During operation, when the lidar detects a collision warning, it immediately generates a globally unique event case ID; this unique case ID is then used to tag the data collected for this event, enabling the association of all evidence elements. When AIS detects that the target vessel has left the 50-meter range, the warning event ends and the final encapsulation of the evidence chain is automatically triggered, automatically generating a JSON format data file.

7. The navigational aid collision avoidance early warning method according to claim 1, characterized in that: The process of uploading and storing key raw data specifically includes: Evidence preservation technology includes the preservation of original data, which involves locally storing the original AIS messages and the original LiDAR point cloud data. The raw data is integrated with evidence elements in JSON format; the entire evidence data package is then encrypted and uploaded to the cloud evidence repository via wireless network.

8. The navigational aid collision avoidance early warning method according to claim 1, characterized in that: During operation, the use of a dynamic power supply strategy to continuously monitor the system power supply voltage and dynamically adjust the power supply strategy according to the real-time voltage status specifically includes: The output voltage of the solar battery is continuously collected by a smart relay, and its magnitude is monitored as an indicator for adjusting the state of energy. The MPPT controller is used to optimize the solar charging efficiency. The power supply strategy is dynamically adjusted based on the output voltage. When the voltage is ≥13V, the full-function mode is activated, and all devices are started. When the voltage is ≥12V and <13V, it enters power saving mode, continuously runs the AIS terminal and LiDAR, and maintains 4G / 5G communication to ensure core perception capabilities; visual evidence collection is activated as needed, and the AI ​​camera is only activated under daylight conditions; unnecessary equipment is turned off. When the voltage is less than 12V, the system enters keep-alive mode, shutting down all sensing devices; only the 4G / 5G communication module remains online; and alarm information is immediately sent to the remote monitoring platform.

9. The navigational aid collision avoidance early warning method according to claim 1, characterized in that: To address the issue of ships without AIS terminals in real-world scenarios, a lightweight YOLO-based target recognition algorithm is integrated into the AI ​​camera. During the pre-training phase, it focuses on identifying ships from videos in real time and integrates a simple distance estimation algorithm based on monocular vision. During use, the AI ​​camera maintains patrol monitoring. When a ship-type object is within 50 meters of the monitoring range, the AIS and LiDAR are activated to measure the distance to the target ship, triggering the first and second level strategies. Combined with preset alarm templates, the ship is warned to stay away.

10. The navigational aid collision avoidance early warning method according to claim 9, characterized in that: In cases where ships are not equipped with AIS terminals, considering the legality of ships without AIS terminals, after the lidar is directly activated by video surveillance, when the lidar calculates that the distance between the target ship and the navigation mark is clearly less than 50 meters, and the AIS terminal still has not received the AIS signal, an image is captured from the video stream, uploaded to the cloud, and an alarm message is sent simultaneously.

Images