An unmanned surface vehicle cruising method and system based on laser radar and vision fusion

Abstract

The application provides an unmanned ship cruising method and system based on laser radar and vision fusion, relates to the field of intelligent water equipment and maritime image recognition management, and specifically comprises the following steps: performing multi-modal original data synchronous acquisition, calculating the visual confidence of a current frame visual image and the point cloud confidence of a three-dimensional point cloud set, calculating the confidence difference, and adaptively cutting into the corresponding restriction and guidance decision state according to the numerical value falling area of the confidence difference; based on the decision state, performing state-driven feature guidance and three-dimensional information extraction, and outputting a continuous curve of a tide head height distribution and a distribution curve of an instantaneous propelling speed across the width of a channel; based on the continuous curve of the tide head height distribution and the distribution curve of the instantaneous propelling speed, performing tide head propelling trajectory and feature deduction, and calculating an instantaneous impact pressure array; finding an optimal risk avoidance coordinate point with the lowest cumulative risk value, and generating and sending maritime risk avoidance instructions to a target ship.

Description

Technical Field

[0001] This invention relates to the field of intelligent watercraft equipment and maritime image recognition management, and more specifically, to a method and system for unmanned surface vessel (USV) cruising based on lidar and vision fusion. Background Technology

[0002] With the continuous advancement of intelligent shipping and smart maritime construction, the demand for supervision of maritime traffic safety and navigation order is increasing. In the conventional maritime supervision field, the traditional patrol boat supervision model, which is mainly based on manpower, has long faced many practical challenges. Traditional patrol boats have high operating costs, limited operational efficiency, and personnel safety cannot be fully guaranteed in severe weather or high-risk environments. To address these limitations, unmanned surface vessel (USV) technology, which has advantages such as shallow draft, maneuverability, and no need for personnel to board, is gradually being introduced into maritime on-site supervision in order to achieve tasks such as maintaining navigation order, monitoring the aquatic environment, and emergency search and rescue.

[0003] However, in specific waters such as Hangzhou Bay and the Qiantang River, the hydrological conditions are extremely complex, placing stringent demands on the patrol and sensing capabilities of naval equipment. This area is a typical strong-tidal estuary, with an irregular semi-diurnal tidal system. The tidal range amplifies significantly along its course, averaging 5.45 meters, with a historical maximum measured range of 8.93 meters. Simultaneously, the hydrodynamics exhibit an asymmetrical distribution, with stronger flow during high tide and weaker flow during low tide. The average vertical velocity of the high tide in the core control area reaches 4 to 5 meters per second, with a maximum instantaneous velocity exceeding 6 meters per second. Under these conditions, the Qiantang River tidal bore typically reaches a height of 1 to 2 meters, with a historical maximum measured at 3.2 meters. The tidal wave propagation speed reaches 8 to 10 meters per second, and the peak impact pressure reaches as high as 7 tons per square meter. Furthermore, influenced by multiple factors such as runoff, tidal currents, and sediment supply, the sediment content in this area is extremely high, with a peak sediment content during high tide reaching 51 kilograms per cubic meter. The bottom sediment of the waterway is mainly silt, and the riverbed changes frequently and drastically due to erosion and deposition. This results in the waterway being wide, shallow, and scattered. At low tide, the water depth in many areas is less than 1 meter, creating extreme conditions where navigation is possible at high tide and the water can run aground at low tide.

[0004] The aforementioned extreme hydrological and topographical features result in significant blind spots and timeliness issues for traditional maritime regulatory methods. Due to the dynamic changes in channel depth and the dramatic fluctuations in tides, traditional patrol vessels, limited by their own draft, must strictly adhere to the principle of entering during high tide, operating during low tide, and exiting during low tide, with waiting times reaching up to 4 hours. During tidal bore, patrol vessels cannot approach core risk areas for monitoring; in shallow waters at low tide, their draft is limited, making it difficult to reach the edges of mudflats and shallow waters. This operational mode, constrained by hydrological conditions, often results in long delays from receiving an alarm to arriving at the scene, making it difficult to meet the needs for rapid response and 24 / 7 monitoring of maritime emergencies.

[0005] While existing unmanned surface vessels (USVs) have to some extent compensated for the drawbacks and maneuverability of traditional patrol vessels, their environmental perception and cruise decision-making systems still face technological bottlenecks in environments like the Qiantang River and Hangzhou Bay, which are characterized by extremely high sediment content, strong water mist interference, and severe hydrodynamic impacts. During tidal bores, high concentrations of sediment and the resulting water mist severely obstruct the view of optical equipment, causing a sharp decrease in image contrast or even failure of visual sensors. Furthermore, single-dimensional point cloud data generates significant physical noise within the complex splashing water, lacking accurate semantic boundary recognition capabilities. Current technologies lack deep feature fusion and reliability assessment mechanisms for such highly dynamic and turbid hydrological environments, making it impossible to accurately acquire the instantaneous three-dimensional characteristics of the tides. Consequently, it is difficult to scientifically predict the spatial impact pressure and navigation risks brought by tidal bores. This limits the cruise safety of existing USVs in extreme tidal environments and prevents them from effectively guiding and intervening in the avoidance of hydrological disasters by surrounding vessels. Summary of the Invention

[0006] This invention provides a method for unmanned surface vessel (USV) cruising based on lidar and vision fusion, comprising the following steps: Perform multimodal raw data synchronous acquisition to obtain the current frame visual image, 3D point cloud set and ship broadcast data in the surrounding waters; calculate the visual confidence of the current frame visual image and the point cloud confidence of the 3D point cloud set, calculate the confidence difference, and adaptively switch to the corresponding constraint and guidance decision state according to the numerical area of ​​the confidence difference. Based on the decision state, the feature guidance and three-dimensional information extraction driven by the execution state output a continuous curve of the tidal head height distribution and an instantaneous propulsion speed distribution curve spanning the width of the channel. Based on the continuous curve of tidal head height distribution and the instantaneous propulsion velocity distribution curve, the tidal head propulsion trajectory and characteristics are deduced, and the instantaneous impact pressure array is calculated by combining the introduced high turbidity sediment fluid density. Find the optimal risk avoidance coordinates with the lowest cumulative risk value, generate and send maritime risk avoidance commands to the target vessel.

[0007] The specific steps for calculating the visual confidence of the current frame's visual image and the point cloud confidence of the 3D point cloud set are as follows: Convert the current frame visual image into a single-channel grayscale image matrix. The second-order spatial differential operator is applied to extract the gradient magnitude matrix of the entire image, and the gradient variance of the image edges is calculated. And map the degree of optical distortion output. : in, This is a preset gradient baseline constant; A lightweight semantic segmentation network is invoked to process the current frame's visual image, outputting a binarized contour curve of the two-dimensional tidal wave peak. Extract the total number of unconnected contour segments in the binarized contour curve. Passing the exam Horizontal pixel span of each contour line segment Calculate the degree of contour continuity : The width of the image in pixels. The constant is the discontinuous decay penalty constant; and based on and Overall output visual confidence level ; In 3D space, a bounding box representing the region of interest is extracted. A subset of point clouds falling within this bounding box is then extracted, and the first echo energy, representing water mist interference, is statistically analyzed. With the second echo energy characterizing the solid water wall and Calculate the dual-echo energy ratio : It is a minimal positive real number. This is the critical energy transmission ratio threshold. The gain coefficient is used; principal component analysis of the covariance matrix is ​​performed on each point cloud data point within the bounding box to obtain the local surface normal vector. And calculate the consistency factor of local normal vectors in the spatial nearest neighborhood. : This represents the total number of points within the bounding box. The number of nearest neighbors; and based on and Overall output point cloud confidence .

[0008] Based on the numerical range of the confidence difference, the system adaptively identifies the corresponding constraint and guidance decision states, specifically including: calculating the confidence difference. ,according to Entering three decision states: when When the radar is in a dominant state, the system switches to the correction branch of the lidar's one-way guided vision. when When the system is determined to be in a vision-dominated state, it switches to the fog-breaking branch of the vision-guided unidirectional lidar. when and and When all values ​​exceed the system's minimum safe operating threshold, the system is determined to be in a dual-source equilibrium state, and the fusion branch with asymmetric timing bidirectional guidance is initiated; among them, This is a preset critical threshold for difference.

[0009] When entering the asymmetric temporally bidirectional guided fusion branch, cross-modal space dimensionality reduction and feature extraction are performed: the smallest two-dimensional rectangular bounding box that can completely enclose the binarized contour curve is calculated. Combined with the pre-calibrated camera intrinsic parameter matrix Homogeneous transformation matrix with external parameters Homogeneous coordinates of any point in a 3D point cloud set Perform spatial dimensionality reduction filtering constraint equations: in, As the scale-depth normalization factor, Perspective pixel coordinates; only the projection falling into the pixel coordinates are retained. The internal point cloud set constitutes a frustum dimensionality reduction subset. ; In the dimensionality reduction subset of the frustum In this process, the Euclidean distance difference between two adjacent point clouds is calculated as a continuous depth mutation operator. When the scanning line appears When the depth exceeds the rigidity threshold, lock the 3D water wall entity cluster and calculate its 3D spatial geometric center point. And force a dimensionality reduction projection back to the current frame's 2D image to obtain the target pixel coordinates. It is used as the sole high-priority absolute seed point for the growth of the two-dimensional tidal body region.

[0010] When switching to the correction branch of unidirectional vision guidance from the LiDAR, image enhancement and visual correction are performed: Extract the 3D water wall entity clusters locked by deep mutation and calculate their 3D geometric center points, then project them onto a 2D image to obtain their absolute physical coordinates. ;by As the core, a depth-space prior weight graph is generated. And superimposed on the original blurred image: These are the two-dimensional coordinates of the image pixels. and This represents the two-dimensional spatial broadening coefficient; based on the prior weight map, local edge gradient features are forcibly enhanced, and the enhanced gradient feature map is output. : In the formula, This represents the second-order differential gradient value of the original blurred image. The gain coefficient is defined by forcibly enhancing feature map nodes in the visual network using absolute physical coordinates, and the modified dynamic region growth threshold is defined accordingly. This forces visual algorithms to only... Limit region growth within the neighborhood: The growth threshold for the base region.

[0011] When switching to the fog-breaking branch of the vision-guided unidirectional LiDAR, point cloud filtering and inverse perspective mapping are performed: extracting the maximum vertical pixel coordinates of the lower edge of the 2D tide crest bounding box. Combined with the physical height of the installation Longitudinal focal length parameters and the coordinates of the longitudinal optical center principal point Calculate the absolute depth of a uniform virtual light curtain. : and all perspective coordinates within the bounding box The depth-distance function is uniformly assigned as ; For any point cloud, determine the true physical distance from the origin. The point data contains water mist noise, so the threshold for removing the first echo is increased. Simultaneously reduce the predetermined depth behind the light curtain. The second echo reflection intensity reception threshold within : Based on the basic water mist reflection filtration threshold, The entity reflection threshold is used; subsequently, a set of two-dimensional semantic masks for the main tidal subject is extracted from the output of the extreme region growing. The visual tidal bore height estimate is obtained by using inverse perspective transformation. : This is the scale-depth compensation coefficient. This represents the Z-axis elevation component of a spatial vector.

[0012] When entering the dual-source equalization state, in the horizontal coordinate system At each discrete grid point, perform point-by-point weighted averaging based on dynamic confidence to obtain the initial state fused height array. A one-dimensional smooth kernel was introduced along the cross section of the waterway. Perform discrete spatial convolution operations to output a high-precision fused height distribution curve. : For spatial convolution integral variables, Let the radius be the local sliding window radius. This is the fluid continuity smoothing coefficient.

[0013] The calculation of the instantaneous impact pressure array specifically includes: The density of high-turbidity sediment fluid is introduced based on pre-set hydrological empirical parameters. : In the formula, For standard freshwater density, The nominal density of suspended sediment particles. To determine real-time sediment concentration, a superposition model of hydrodynamic and hydrostatic pressure is used in a two-dimensional absolute geographic space system. The instantaneous impact pressure array is generated by calculation. : In the formula, It is the acceleration due to gravity. and respectively extrapolating to future moments The expected speed of advance and the expected height of the tidal bore.

[0014] The generation of a two-dimensional risk weight diagram specific to the target vessel and the optimization of the optimal avoidance coordinates specifically include: constructing a vulnerability coefficient based on deadweight tonnage. In the formula As a benchmark for disaster resistance load capacity, The tonnage attenuation constant is used to represent the instantaneous impact pressure array. With disaster vulnerability coefficient Linear coupling is performed to generate a two-dimensional risk weight graph. : The preset theoretical peak value of extreme impact pressure; based on the target ship's current absolute latitude and longitude coordinates, within its effective maneuver radius. Internally, a global minimum search is performed to find the optimal risk-avoidance coordinate point with the lowest cumulative risk value. : An unmanned surface vessel (USV) cruise system based on lidar and vision fusion, the system comprising: Confidence calculation module: Performs synchronous acquisition of multimodal raw data, obtains the current frame visual image, 3D point cloud set and ship broadcast data in the surrounding waters; calculates the visual confidence of the current frame visual image and the point cloud confidence of the 3D point cloud set, calculates the confidence difference, and adaptively switches to the corresponding constraint and guidance decision state according to the numerical area of ​​the confidence difference. Tidal crest feature calculation module: Based on decision state, execution state-driven feature guidance and 3D information extraction, outputting a continuous curve of tidal crest height distribution and instantaneous propulsion speed distribution curve spanning the width of the channel; Impact pressure calculation module: Based on the continuous curve of tidal head height distribution and the instantaneous propulsion velocity distribution curve, it performs tidal head propulsion trajectory and characteristic deduction, and calculates the instantaneous impact pressure array by combining the introduced high turbidity sediment fluid density; Guidance module: Finds the optimal risk avoidance coordinates with the lowest cumulative risk value, generates and sends maritime risk avoidance commands to the target vessel.

[0015] This invention discloses a method and system for unmanned surface vessel (USV) cruise based on lidar and visual fusion, applicable to extreme physical scenarios such as high dynamics, high turbidity, and strong water fog interference in estuaries and complex tidal bore waters. Traditional single-sensor or conventional static fusion algorithms are prone to failure under such extreme conditions due to splashing water or diffuse reflection from dense fog. This application constructs a cross-modal adaptive fusion architecture driven by data features. In particular, by deeply exploring the intrinsic relationship between lidar dual-echo characteristics and multi-dimensional features of visual images, it achieves high-precision, zero-latency three-dimensional tidal crest feature extraction in harsh hydrological environments, ensuring absolute physical safety and perception reliability for forward interception cruise of USVs from the underlying architecture.

[0016] This application is based on a dynamic evaluation and decision-making mechanism of multi-source original feature confidence. When facing strong backlighting on the river surface, water droplets adhering to the lens, or extremely dense suspended water mist barriers, it can independently quantify the optical imaging quality and the physical detection quality of the point cloud. Specifically, the extraction of visual confidence fully considers the continuity of image edge gradient distortion and semantic contours, while point cloud confidence combines the dual-echo energy ratio characterizing water mist interference and entity reflection, as well as the smoothness of local spatial surfaces. Based on this, by calculating the confidence differences between modes, it adaptively switches to different constraint and guidance decision states, such as radar-dominated, vision-dominated, or dual-source balanced. This mechanism avoids the problems of computational overload caused by blind global computation and the contamination of high-confidence perception results by low-confidence erroneous features in traditional fusion algorithms. After adaptively switching to a specific decision state, this application performs highly collaborative state-driven cross-modal feature guidance extraction, greatly improving the system's robustness under various transient interferences. When in a dual-source equilibrium state, the system executes asymmetric temporal bidirectional guidance. On one hand, it utilizes visual two-dimensional contours to construct physical view cones, reducing the dimensionality constraint of the LiDAR's three-dimensional spatial retrieval and significantly decreasing the ineffective computational power consumption on irrelevant backgrounds. On the other hand, it uses absolute depth mutation points extracted from point clouds as high-priority seed points to forcibly intervene in visual semantic segmentation, fundamentally eliminating the distance illusion defect where visual algorithms easily misjudge distant mudflats as dynamic tides. When encountering severe water splash obstruction and entering a radar-dominated state, the system can use the absolute physical prior coordinates established by the radar to forcibly generate a depth space prior weight map on the blurred image to locally enhance edge features, forcing the visual network to complete contour region growth under harsh conditions. In the visual-dominated state where dense fog causes significant radar failure, this application utilizes high-confidence visual boundaries to downmap and generate a unified virtual light curtain absolute depth, using this as a rigid physical boundary to implement spatial segmented dynamic filtering for the radar.

[0017] Based on the aforementioned sophisticated and error-proof fusion mechanism, the system can stably output a high-precision continuous distribution curve of tidal head height and instantaneous propulsion speed across the entire cross-section of a wide waterway. This fusion curve not only introduces a one-dimensional smoothing kernel to strictly satisfy the spatial continuity constraints of macroscopic fluid dynamics, but also utilizes radar anchor points to complete absolute deformation calibration of the visual height scale under extremely sparse and challenging observation conditions. Thanks to the highly reliable and complete three-dimensional hydrodynamic characteristic data provided by the preceding steps, this application can ultimately combine a high-turbidity sediment fluid density model to scientifically and accurately predict the instantaneous impact pressure array when the tidal bore sweeps across various areas of the waterway at future moments. This enables the system to overcome the limitations of traditional maritime early warning systems that lack specificity, accurately combining the static load-bearing and vulnerability attributes of surrounding vessels to optimize the absolutely safe waters with the lowest local pressure in a dynamic risk weighting map. This achieves automated, individually customized, precise maritime hazard avoidance scheduling and broadcasting, forming a complete closed loop for navigation safety. Attached Figure Description

[0018] Appendix Figure 1 This is a flowchart of the unmanned surface vessel (USV) cruise based on the fusion of lidar and vision according to the present invention; Appendix Figure 2 Comparison of full-section tidal head heights under different algorithms; Appendix Figure 3 This is a time series curve diagram of dynamic input and constraint of multimodal confidence in this invention. Detailed Implementation

[0019] This embodiment provides an unmanned surface vessel (USV) cruise system based on the fusion of lidar and vision. The system architecture of this embodiment provides the hardware foundation and data acquisition terminal for subsequent adaptive fusion algorithms and risk weight calculations.

[0020] The unmanned surface vessel (USV) cruise system in this embodiment is configured to operate in estuaries with strong tides and during specific periods of hydrological change, particularly during spring tides and tidal bores. Typical hydrophysical characteristics are as follows: tidal wave propagation speed is 8 to 10 meters per second, maximum instantaneous flow velocity is greater than 6 meters per second, tidal head height is 1 to 3.2 meters, and peak tidal impact pressure reaches 7 tons per square meter. The water exhibits highly dynamic turbidity, with peak sediment content reaching 51 kilograms per cubic meter during high tide, accompanied by high-density splashing water mist and suspended sediment. The core navigable channel width is limited to 200 to 300 meters, and there are dynamically changing erosion and sedimentation embankments underwater.

[0021] To adapt to the specific hydrological and waterway scenarios described above, the unmanned surface vessel (USV) cruise system in this embodiment includes an USV platform body, a propulsion and power module, a multimodal perception module, and a communication and edge computing module.

[0022] The unmanned surface vessel (USV) platform is preferably designed with a catamaran structure, configured to provide high lateral stability to resist lateral rolling caused by tidal surges. The USV platform is preferably 12.5 meters long, 4.5 meters wide, and has a draft of 0.53 meters, suitable for navigation in shallow waters at low tide. The propulsion and power module includes a main thruster, a side thruster, and a power supply unit. The main thruster is preferably a dual high-power four-stroke outboard motor, with a maximum power of 224 kW per motor, configured to provide a maximum speed of at least 20 knots. The USV's speed is synchronized with or slightly exceeds the tidal wave propagation speed of 8 to 10 m / s, enabling forward dynamic retreat navigation. The side thruster is preferably a shaftless flange-type thruster with a rated power of 6 kW, configured to provide lateral movement and low-speed attitude maintenance capabilities under strong lateral currents and complex flow conditions.

[0023] The multimodal perception module includes a visual acquisition unit, a 3D point cloud acquisition unit, and a velocity measurement unit. Each unit is fixed to the superstructure of the unmanned surface vessel platform via rigid brackets and is calibrated and aligned in the spatial coordinate system.

[0024] The visual acquisition unit preferably includes at least one high-resolution RGB camera. This RGB camera supports minimum illumination of 0.002 lux for color and 0.0002 lux for monochrome; it is equipped with a motorized zoom lens with a horizontal field of view covering 110 to 45 degrees; and it supports a wide dynamic range adjustment of 120 dB and strong light suppression. The lens of the visual acquisition unit is configured to face the main observation direction during the unmanned surface vessel's backward movement (i.e., towards the tidal bore), and is used to acquire video image data streams containing the two-dimensional boundary of the tidal bore, water texture, and color distribution characteristics.

[0025] The 3D point cloud acquisition unit preferably includes a multi-line obstacle avoidance lidar. This lidar is preferably 32-channel, with a 360-degree omnidirectional horizontal field of view and a scanning speed configured at 10Hz or 20Hz. To address the high-density water mist characteristic of the scene, the lidar is configured to support a dual-echo data output mode, meaning it can simultaneously output the first echo data penetrating the water mist and the second echo data reflected from the solid water wall. The lidar is installed higher than the visual acquisition unit, ensuring that its 150-meter effective detection radius can fully cover the 200-300 meter channel cross-sectional width, used to acquire the 3D spatial coordinates and height characteristics of the tidal bore.

[0026] In addition, the multimodal sensing module also includes a marine Class A Automatic Identification System (AIS) and a weather station. The AIS system is configured to receive real-time static information and dynamic trajectory data of vessels in the surrounding waters; the weather station is configured to collect real-time wind speed, wind direction, and atmospheric pressure data.

[0027] The communication and edge computing modules are electrically connected to the propulsion and power module and the multimodal sensing module, respectively. The communication component adopts a multi-link redundancy architecture, including a 5G / 4G cellular communication router and a satellite broadband terminal (supporting BeiDou and Iridium links). The communication component is configured to ensure continuous transmission and reception of control commands and sensing data for the unmanned surface vessel (USV) under conditions of bridge obstruction or severe convective weather. The edge computing component has a built-in time synchronization main control board and data processing chip. The edge computing component is configured to receive the timestamp input from the multimodal sensing module and achieve microsecond-level hard synchronization of RGB video streams, LiDAR point clouds, and millimeter-wave radar data through PPS or gPTP protocols. The edge computing component stores an adaptive fusion program and a risk weight calculation model, configured to perform local real-time extraction of features from high-turbidity images and dual-echo point clouds, dynamic weight allocation, depth mutation identification, and shock pressure array calculation, and output track adjustment commands to the propulsion and power module, while simultaneously broadcasting hazard avoidance guidance paths to surrounding vessels through the communication component.

[0028] With the hardware and computing power support of the aforementioned unmanned surface vessel (USV) cruise system architecture, this embodiment provides a USV cruise method based on the fusion of lidar and vision. This method is executed in real time by an edge computing component.

[0029] S1. Dynamic tracking of the tidal bore and synchronous acquisition of multimodal raw data Because the Qiantang River tidal bore is accompanied by extremely complex strong eddies and high-energy turbulence zones, traditional tailing or lateral monitoring methods are prone to causing the detection vessel to lose control and capsize. This step establishes an unmanned surface vessel's forward interception and synchronous retreat operational posture and absolute benchmark for data acquisition, thus avoiding the risk of turbulence-induced disasters from a physical spatial perspective.

[0030] The S11 channel centerline is pre-deployed, and the two-dimensional parametric curve of the channel centerline in the target waterway is obtained in advance. And the predicted arrival time of the tidal bore, output by the astronomical tide prediction model. and the expected propagation direction vector .

[0031] To avoid the impact of nonlinear tidal surge disruption caused by the abrupt decrease in water depth at the edge of the shoal on the unmanned surface vessel, edge computing components are used in... ,in Commands are issued to the propulsion and power modules at the preset maneuver lead time. Through the coordinated operation of the main thruster and the auxiliary side thrusters, the absolute coordinates of the unmanned surface vessel are determined. Control and maintain On the trajectory, among them, and These are the x and y coordinates of the unmanned surface vessel at time t. Simultaneously, the bow angle of the unmanned surface vessel is adjusted. To make it satisfy the constraints: This constraint forces the unmanned surface vessel to adopt a catamaran structure with its front facing the direction of the tidal bore, i.e., a head-on posture, to maximize the use of the high lateral stability of the catamaran hull and resist the strong gusts of wind and water shear forces generated by the tidal bore's front pushing the air.

[0032] S12 Dynamic Back-off Navigation Control: When the high-frequency underlying hardware interface of the 3D point cloud acquisition unit detects continuous depth abrupt changes ahead and the distance reaches a preset perception threshold, the system immediately triggers the dynamic back-off mechanism.

[0033] Considering that the multi-source fusion algorithm requires a fixed edge computing power and time period, in order to ensure the system's zero-delay obstacle avoidance response and the platform's absolute physical safety when extreme tidal surges approach, the edge computing component has constructed a time-series dual-loop control architecture that couples high-frequency raw perception with low-frequency fusion feedback at the control level.

[0034] Real-time calculation of dynamic safety distance function based on current tidal physical conditions : In the formula, Minimum fall protection distance to ensure absolute physical safety of the unmanned surface vessel platform; To ensure a safety margin for water spray, considering the combined effects of water wall turbulence and gusts of wind, The value range is set as This coefficient characterizes the threat radius of the water wall roll-up to the deck equipment of the unmanned surface vessel, thus ensuring that the dynamic safety distance can adaptively expand as the destructive force of the tide is nonlinearly amplified; This refers to the height of the tidal bore in the time-series feedforward. To avoid sensing failure caused by computational lag in the fusion algorithm, A time-series segmented update strategy is adopted: during the initial system acquisition phase, The value is the preset theoretical height of the astronomical tide. After the subsequent adaptive fusion algorithm completes the calculation of at least one data cycle, It is updated in real time to the high-precision average tidal head height output by the fusion algorithm in the previous time series.

[0035] During the retreat process, the target retreat speed command of the unmanned surface vessel is calculated and output in real time. : In the formula, and These are the proportional gain and derivative gain parameters for the backshift control, respectively. The thrust compensation sensitivity of the propulsion system to spatial distance deviations is determined to overcome the hydrostatic resistance and inertia of the catamaran hull in water, and its preferred calibration range is as follows: ; This is the system damping term, used to suppress backflip velocity overshoot caused by the instantaneous acceleration of the tidal surge or high-frequency noise from sensors, preventing dangerous longitudinal oscillations of the unmanned surface vessel in front of the tidal bore. Its preferred calibration range is... . For observation distance, this parameter is directly extracted from... In the raw point cloud data of the first echo of the instantaneous lidar, the spatial Euclidean distance of the nearest reflection cluster within the local field of view is taken as the origin, with the bow of the unmanned vessel as the origin. This represents the time-feedback tidal surge propagation velocity. In the initial stage, its value is determined by... The first-order time derivative is roughly estimated by combining the unmanned surface vessel's own inertial navigation velocity; after entering steady state, its value is overwritten in real time by the high-precision instantaneous propulsion velocity output by the fusion algorithm in the previous time series.

[0036] This application adopts a timing dual-loop control architecture that uses the aforementioned fusion of low-level and high-level data for calibration. In extreme scenarios such as the Qiantang River tidal bore, which combines extremely high propulsion speed and extremely strong destructive power, the traditional single-layer perception control architecture faces an inherent contradiction between algorithm accuracy and computation time. If the high-precision visual point cloud fusion is waited for before the retreat command is output, the computation delay will directly cause the unmanned surface vessel to be in a risky area.

[0037] This embodiment resolves this contradiction through a dual-ring architecture: on the one hand, the high-frequency bottom-level detection (0Hz) directly calls the one-dimensional time-of-flight (ToF) distance data of the lidar, bypassing computationally intensive nodes such as image semantic segmentation and 3D reconstruction, achieving zero-latency retreat obstacle avoidance and establishing the physical safety baseline of the unmanned surface vessel; on the other hand, the low-frequency top-level detection (Hz) feeds back the real water wall height and velocity obtained by high-precision fusion of the previous time series to the control feedforward term, effectively correcting the problem of excessive retreat and loss of field of view caused by the bottom-level radar misjudging the high-density water mist at the leading edge as a real tidal wave.

[0038] Through this mechanism, unmanned surface vessels can not only remain invincible at the forefront of turbulent tidal bores, but also establish a stable and positive observation benchmark for facing the tidal bore while maintaining an extremely close dynamic safety buffer distance. This fundamentally overcomes the data loss caused by the limitations of traditional tailing or shore-based fixed-point monitoring methods due to angle limitations or being overturned by turbulent currents.

[0039] S13 Multimodal raw data synchronous acquisition: During the process of maintaining the above-mentioned synchronous retreat dynamic balance, the edge computing component uses the hardware-level triggering mechanism of the time synchronization main control board to synchronize the unified global timestamp. Here, we extract and bind the original multimodal data across three dimensions: The visual acquisition unit is in The current frame image data output at each moment The image data fully maps the two-dimensional wave crest contours of the tidal wall, the color texture of the water with high turbidity, and the optical features such as surface water droplet obstruction.

[0040] Acquiring multi-line obstacle avoidance lidar in A collection of 3D point clouds that completes an omnidirectional scan at any time To address the strong interference from the high-density water mist accompanying the Qiantang River tidal bore, the lidar was configured in dual-echo mode, and each cloud data point was defined as a multi-dimensional feature vector. : in, This represents the total number of points in the point cloud for that frame. For the first The three-dimensional spatial coordinates of a point cloud data point in the local coordinate system of the lidar; This is the characteristic value of the laser reflection intensity at that point; This is used as an identifier for the discrete echo index. When... At that time, it represents the first echo reflected from the high-density water mist splashing in front; when At this point, the laser pulse is represented as the second echo reflected from the high-density solid water wall behind it after penetrating the water mist.

[0041] Further extract the pre-calibrated homogeneous transformation matrix of the extrinsic parameters. ,Will The coordinate system is strictly mapped and aligned to In the visual camera coordinate system.

[0042] Synchronous extraction of shipborne AIS system The set of ship broadcast data received and parsed in the surrounding waters within its adjacent preset time window. : in, The total number of target vessels within the current effective communication and detection radius of the unmanned surface vessel; the first The state feature vector of a target ship is defined as .in The unique identifier for the target vessel. and The absolute geographical coordinates of the ship's current location are latitude and longitude. For ship speed, For the ship's course, This refers to the static tonnage or real-time draft of a ship.

[0043] S2. Data Feature-Driven Adaptive Fusion of LiDAR and Vision After obtaining the unified timestamp After processing the multimodal raw data, considering that the Qiantang River tidal bore is often accompanied by high-density splashing water mist, backlight glare on the water surface, and lens occlusion caused by strong water vapor, a single sensor is prone to feature failure under certain physical transients. To prevent erroneous features from contaminating the final 3D reconstruction results, the edge computing component first performs dynamic confidence assessment of the raw features to establish the dynamic weights of multi-source data in the fusion process.

[0044] S21 Dynamic evaluation of confidence level of original features: This step independently calculates the optical confidence score of the current frame visual image and the spatial physical confidence score of the 3D point cloud, quantifying the effective observation quality of each sensor under the current extreme environment.

[0045] Visual confidence Dynamic calculation: In response to the loss of high-frequency features caused by splashing water droplets adhering to the lens during the retreat and interception of unmanned surface vessels, namely local blurring and illumination distortion caused by high albedo of the water surface, the system constructs visual confidence based on the image edge gradient variance and contour continuity index.

[0046] Specifically, the edge computing component will collect the current frame color image data. Convert to single-channel grayscale image matrix The gradient magnitude matrix of the entire image is extracted using a second-order spatial differential operator. The gradient variance at the image edges is defined. for: in, and These are the pixel width and height of the image, respectively; coordinates The Laplace gradient response value at point; This represents the statistical average of the gradient across the entire image. When the lens is obscured by a large area of ​​water droplets or is in a state of severe overexposure and flare, high-frequency textures in the image are smoothed out or swallowed. A sudden, non-physical drop will occur. The system will then map and output the degree of optical distortion based on this. : in, This is the preset gradient reference constant under unobstructed, sunny conditions.

[0047] Furthermore, to evaluate the semantic extraction quality of the physical morphology of the tidal bore in the image, a pre-trained lightweight semantic segmentation network is invoked. The process is performed to output the binarized profile curve of the two-dimensional tidal wave crest. A true tidal surge crest should exhibit a continuous, laterally extending physical extension. If... The presence of numerous breakpoints or discrete patches indicates that the current visual feature extraction is in an unreliable and fragmented state. (Statistics) The total number of non-connected contour segments in the middle and the Horizontal pixel span of each contour line segment Calculate the degree of contour continuity : in, It is a non-continuous decay penalty constant that imposes an exponential penalty on the visual failure phenomenon of the image fragmenting into numerous isolated patches.

[0048] Taking into account the two factors mentioned above, the system outputs the dynamic visual confidence score of the current frame. : in, and These are the corresponding normalized weight coefficients, and they satisfy... .

[0049] Point cloud confidence Dynamic calculations are used to address the high-density cloud-like water vapor formed by the tidal bore's advance pushing air forward in the Qiantang River, which traditional single-echo lidar easily misinterprets as a solid wall of water. This application fully utilizes 3D point cloud datasets. The dual-echo feature in the data is used to quantify the penetration effectiveness and surface smoothness of the point cloud within the region of interest.

[0050] The edge computing component is based on the low-level coarse observation distance extracted in step S12 above. In three-dimensional space, select a bounding box containing the region of interest that surrounds the main body of the tide. Extracting the falling A subset of point clouds within the data is used to statistically characterize the first echo energy of water mist interference. With the second echo energy characterizing the solid water wall and : in, This represents the characteristic value of laser reflection intensity for the corresponding point cloud. If... This indicates that most of the laser pulse's energy is dissipated in the splashing water mist at the leading edge, and the actual water wall is not effectively detected. Based on this, the system calculates the dual-echo energy ratio. : in, The fraction should be a very small positive real number to prevent the denominator from being zero. The critical energy transmission ratio threshold; is the gain coefficient of the Sigmoid mapping function, used for steep confidence rejection of point cloud clusters with low energy penetration.

[0051] Furthermore, a realistic solid water wall should have a relatively smooth three-dimensional curved surface with gradual transitions, while suspended water mist particles would cause irregular diffuse reflection of the laser. Therefore, [the text abruptly ends here]. Each point cloud data point within Perform principal component analysis on the covariance matrix to obtain its local surface normal vector. Definition point Spatial nearest neighbor is (Include Calculate the local normal vector consistency factor (nearest neighbor points). : in, bounding box for region of interest Total number of points contained within; This represents the absolute inner product of the normal vectors of two adjacent points. When a point cloud hits a chaotic splash of water, the expected value of the inner product of the normal vectors drops sharply, thus objectively reflecting the decrease in the quality of the point cloud surface. It also considers the physical characteristics of energy transmission and spatial geometric distribution, outputting the dynamic point cloud confidence score for the current frame. : In the formula, and These are the normalized weighting coefficients for energy mapping and surface quality, respectively, and satisfy the following conditions: .

[0052] S22 Multi-source feature confidence verification: After obtaining the independent confidence of the current frame, considering the extreme water background of the wide and highly turbid Qiantang River, if high-computational-power-consuming deep clustering and pixel-level region growing are directly performed on multimodal data, it will not only cause the computing power of the edge computing components to be overloaded, but also cause the sensors with high confidence to be misled by the erroneous priors of the sensors with low confidence.

[0053] To avoid falling into misguided guidance, this invention constructs a mutually verified guidance method. Based on S21, the edge computing component immediately calculates the confidence difference. : according to Within the numerical range, the system adaptively switches to the following three constraint and guidance decision states: State A (Radar Dominant State): When ,in When the preset difference threshold is a positive real number, that is... Significantly higher than This indicates that when the unmanned surface vessel (USV) encounters the tidal bore, the violent impact of the surging waves on its bow causes a large amount of water spray and droplets to adhere to the lens surface of the visual acquisition unit, or that it is at a strong backlight angle on the river surface. This physical occlusion and illumination distortion directly lead to a sharp decrease in the image edge gradient variance in step S21, i.e. Furthermore, the extracted contours are extremely fragmented. Thus The laser pulse emitted by the multi-line obstacle avoidance lidar successfully penetrated the surface water vapor, characterizing the second echo energy of the solid water wall. Dominant, and normal vector consistency factor Stability makes Maintaining a high level. Under this condition, it is determined that the output of the visual network has fallen into the semantic incompleteness caused by the water droplet blind zone. The edge computing component strictly relies on the data from the LiDAR, and the system switches to the correction branch of unidirectional vision guided by the LiDAR.

[0054] State B (Visual Dominance): When At that time, that is Significantly higher than This indicates that the tidal bore front pushes the air, creating an extremely dense and thick barrier of suspended water mist. At this point, most of the energy of the laser radar pulse is dissipated and reflected into the spray of water mist from the front, i.e., the energy of the first echo and... This leads to a sharp drop in energy transmission ratio, and the point cloud exhibits an extremely divergent and chaotic diffuse reflection distribution in space, i.e., a local normal vector consistency factor. The bottoming out directly led to This triggers a steep rejection mechanism. Conversely, the visual acquisition unit can clearly capture the highly contrasting optical texture boundaries between turbid, sedimentary water and waves, enabling... Maintaining extremely high reliability. Under this operating condition, the system determines the clusters of three-dimensional water wall entities extracted by the lidar in step S22. In reality, it's a pseudo-water wall composed of dense water mist; directly using it would lead to severe overshoot in obstacle avoidance and impact prediction. The edge computing component strictly adheres to the continuous physical contours extracted by the high-reliability visual acquisition unit, and the system integrates the fog-breaking branch of the visually guided unidirectional lidar.

[0055] State C (Dual-source equilibrium state): When and and When both exceed the system's minimum safe operating threshold, it is determined that there is neither severe water droplet obstruction nor blindingly dense water mist. The physical characteristics of the two sources are equally matched, and the system enters a fusion branch guided by asymmetric timing.

[0056] S23 State-Driven Feature Guidance and 3D Information Extraction: Based on the decision status output by S22, the edge computing component allocates computing resources and strictly executes the corresponding guided extraction as needed: For state C (dual-source equilibrium state), perform asymmetric timing bidirectional guidance: Under these operating conditions, both source features are highly reliable. To achieve focused computational resources and physical error prevention, the system establishes a sequential execution logic of first visual dimensionality reduction, followed by radar error prevention: Activate unidirectional guided dimensionality reduction from RGB image to 3D point cloud. The edge computing component extracts the binarized contour curve of the 2D tidal wave crest generated in step S21. Calculate the smallest two-dimensional rectangular bounding box that can completely enclose it. : in, and These are the pixel coordinates of the image; , , , is the boundary extremum of the enclosing rectangle.

[0057] To avoid the 3D point cloud acquisition unit performing invalid 3D reconstruction calculations in the entire space, the pre-calibrated camera intrinsic parameter matrix of the vision acquisition unit is extracted. Combined with the homogeneous transformation matrix of the external parameters A physical visual cone is constructed by inverse projection in the three-dimensional local coordinate system of the lidar. .

[0058] For the current frame's 3D point cloud set any point in Extend it to homogeneous coordinates Perform spatial dimensionality reduction filtering constraint equations: in, This is the scale-depth normalization factor; For the first The perspective pixel coordinates of a point cloud data projected onto a two-dimensional image plane.

[0059] Forced exclusion of edge computing components does not meet the requirements and All redundant point cloud data under constraints are retained, except for the projection falling within the constraints. The internal high-confidence point cloud set constitutes the frustum dimensionality reduction subset. Clustering algorithms for solid water walls are strictly constrained to only be used when... Internal operation. This cross-domain dimensionality reduction mechanism reduces the effective computational load of 3D laser point clouds to less than one-third of the original, completely avoiding the waste of ineffective computing power on the sky behind the tide, the water surface, and the deck of the unmanned vessel itself.

[0060] To address the semantic illusion problem where visual algorithms easily mistake a distant, static riverbank for a dynamic tide, a unidirectional projection from a 3D point cloud to an RGB image is employed. Considering that a real, solid tidal wall would exhibit a significant relative depth difference in absolute 3D space at its leading edge, an edge computing component calculates a dimensionality-reduced subset of the view frustum. Two point clouds adjacent to each other in the mid-scan azimuth angle and Euclidean distance relative to the origin of the unmanned surface vessel's coordinates and The system defines a continuous depth mutation operator. : When the scanning line appears ,in To distinguish between gently rippling water surfaces and rigid, vertical water walls, a depth threshold is defined. When the number of consecutive occurrences reaches the confidence clustering scale, the region is immediately identified as a real, existing water wall, and the three-dimensional geometric center point of this cluster is calculated. .

[0061] The edge computing component will determine the geometric center point of the 3D water wall. Forced dimensionality reduction and projection back to the current frame's two-dimensional color image data In the process, obtain precise target pixel coordinates. : in, This is the mapping scale coefficient.

[0062] The system will extract the two-dimensional pixel coordinates As an absolute physical constraint, it is forcibly input into the lightweight semantic segmentation network and defined as the unique high-priority absolute seed point for the growth of the two-dimensional tidal body region. This operation, at the algorithm's underlying level, forcibly avoids the erroneous path of visual neural networks starting feature clustering from high-contrast mudflats at the far end of the image. It ensures that pixel-level semantic segmentation is strictly anchored within the real three-dimensional physical boundary of the tidal body established by the LiDAR, thus fundamentally eliminating visual ranging misjudgments under complex terrain from the architectural level.

[0063] For state A (radar-dominated state), unidirectional lidar guidance and visual correction are performed: in this condition, visual features are unreliable. The system immediately suspends the conventional visual semantic segmentation network to save computing power.

[0064] The edge computing component first extracts high-confidence continuous depth mutation features in three-dimensional space to identify real three-dimensional water wall entity clusters. And calculate its three-dimensional geometric center point. Extracting the reduced-dimensional subset of the view frustum. Three-dimensional water wall entity clusters locked by deep mutation of inner diameter Calculate the estimated tidal bore height using lidar. : in, For the first cluster The absolute coordinates of a point cloud data point in the vertical elevation direction.

[0065] Because the visual image is obscured by water droplets, the gradient variance at the image edges is reduced. The size is so small that the tide outline cannot be extracted by relying on the texture of the image itself. Therefore, the system will use the three-dimensional geometric center point. By combining the pre-calibrated camera intrinsic parameter matrix and extrinsic parameter transformation matrix and projecting them onto a 2D image, the absolute physical coordinates can be obtained. .by As the core, a depth-space prior weight graph is generated. And overlay it on top of the original blurred image: in, These are the two-dimensional coordinates of the pixels in the image; and Mapped from 3D clusters The two-dimensional spatial broadening coefficient of the physical boundary. Based on this prior weight map, the system forcibly enhances the local edge gradient features of the blurred image, outputting an enhanced gradient feature map. : in, This represents the second-order differential gradient value of the original blurred image. The preset gain coefficient for the radar.

[0066] Simultaneously, the system forcibly lowers the image edge gradient determination threshold in S21 and defines a modified dynamic region growing threshold. : in, The region growth threshold is based on the unobstructed state; the absolute physical coordinates of the radar are used to forcibly enhance the feature map nodes in the visual network that are blurred by water droplets, forcing the visual algorithm to only... Within the neighborhood, extreme region growth is performed along the physical contour with an extremely low growth threshold. The final instantaneous tidal head height output of the system is strictly calculated using lidar. The primary benchmark.

[0067] For state B (visually dominant state), perform RGB one-way guidance and radar forced filtering: Under this operating condition, the edge computing component strictly relies on high-reliability visual images to extract continuous and reliable two-dimensional tidal wave crest bounding boxes. The system then maps this data to three-dimensional space using an inverse perspective projection matrix to obtain a virtual interface. A depth-distance function based on a two-dimensional viewpoint is established within the system. Used to characterize from perspective coordinates The emitted rays reach the virtual interface The absolute physical distance in space.

[0068] Specifically, two-dimensional bounding box The lower edge pixels physically correspond precisely to the intersection of the leading edge of the tidal wall and the calm river surface below. The edge computing component extracts the maximum vertical pixel coordinates of the lower edge of this bounding box. .

[0069] The system retrieves the pre-calibrated physical installation height of the vision acquisition unit. (i.e., the absolute vertical height from the camera's optical center to the water surface), calculate the absolute depth of the unified virtual light curtain. : In the formula, This refers to the longitudinal focal length parameter in the camera's intrinsic parameters. Here are the coordinates of the longitudinal optical center principal point. This extremely simplified formula, using the geometric principle of similar triangles, instantly calculates the longitudinal physical distance of the tidal wall.

[0070] Since the solid water wall in the attack posture is approximately a positive cross-section, the bounding box... All perspective coordinates within The depth-distance function is uniformly and approximately assigned to the absolute depth of the virtual light curtain: For any point in a 3D point cloud set Calculate its actual physical distance from the origin. And its perspective projection is applied to the image pixel system to obtain the corresponding coordinates. .

[0071] The system determines that all diffuse point clouds in front of the virtual interface meet the conditions. All point data points are water mist noise. Based on this visual prior, the edge computing component sends instructions to the LiDAR driver module to construct a spatially segmented dynamic filtering model. Specifically, it forcibly raises the first echo rejection threshold falling within the virtual light curtain space. : In the formula, Based on the water mist reflection filtering threshold, all first echoes are forcibly discarded. and reflection intensity The noise level is reduced. Simultaneously, the predetermined depth behind the light curtain is reduced. The second echo reflection intensity reception threshold within That is, for satisfying Point cloud execution: in, As the physical reflection threshold, this one-way constraint mechanism forces the lidar to penetrate the leading pseudo-water mist wall and capture the secondary echo of the real water body within the visually specified absolute physical area.

[0072] Subsequently, the target pixel coordinates are extracted. The set of two-dimensional tidal semantic masks output by growing absolute seed points through the above-mentioned limiting regions. By using inverse perspective transformation, the estimated value of the visual tidal bore height is calculated. : In the formula, The depth compensation coefficient for the current frame scale obtained after radar-corrected ranging; This represents the Z-axis elevation component of the spatial vector. The final instantaneous tidal head height output of the system is strictly calculated based on high-reliability visual analysis. The primary benchmark.

[0073] After completing the state-driven feature guidance and extraction in step S23, considering that the width of the core navigable channel of the Qiantang River reaches 200 to 300 meters, the tidal bore does not present an ideal straight line on this cross section. Instead, due to the non-uniform distribution of underwater erosion and sedimentation topography, it is very easy to form cross tides or local water wall collapses. Therefore, obtaining only a single scalar tidal head height or center point coordinates is far from sufficient to meet the needs of extrapolating the instantaneous impact pressure of objects on the water surface.

[0074] Therefore, the edge computing component uses the aforementioned decision states (states A, B, and C) as constraints, targeting the lateral physical coordinates spanning the entire width of the channel. ,in With the core navigable channel width as the origin, located at the channel centerline, a corresponding full-section fusion model is established, outputting a continuous curve with tidal head height distribution spanning the entire channel width. With instantaneous propulsion velocity distribution curve .

[0075] First, through gridded resampling, the discrete features extracted in S23 are uniformly mapped to the horizontal coordinate axis. Above, unfused visual initial state height curves are generated respectively. relative to the initial state height curve of the point cloud For different decision states, the edge computing component performs the following fusion solution: In state C, and All are within the high confidence range. To balance the ultra-high lateral resolution of vision with the absolute depth accuracy of LiDAR, and to avoid system response delays caused by computationally expensive iterative solutions, a lateral coordinate system was used... At each discrete grid point, a point-by-point weighted average based on dynamic confidence is performed to obtain the full-section initial state fused height array. : although While combining the characteristics of both sources, the array inevitably contains spatially punctuated, needle-like discontinuous abrupt changes due to the presence of high-frequency white noise from the sensors. Considering that tidal bores are macroscopic fluids, subject to the physical constraints of surface tension and gravitational wave dynamics, the surface of its water wall must satisfy absolute hydrodynamic spatial continuity in a cross-section of 200 to 300 meters.

[0076] To address this, the edge computing component further introduces a one-dimensional smoothing kernel along the channel cross section. Spatial convolution smoothing is performed on the initial fusion array to output the final high-precision fusion height distribution curve of the current frame. : in, Represents discrete spatial convolution operation; For spatial convolution integral variables; The radius of the local sliding window is based on the maximum lateral physical span of the local "cross tide" in the Qiantang River channel, and is preferably set to 5 meters, which forcibly constrains the local influence boundary of the smoothing operation. This is the fluid continuity smoothing coefficient.

[0077] For state A (radar-dominant state), under this condition, the visual signature is globally unreliable, and it is only verified through the radar center point in step S23. After forcibly reducing the gradient threshold, the corrected limiting visual height was extracted within the local neighborhood. Therefore, visual data cannot be used for global variational methods, and spatial decay weights must be introduced.

[0078] extract Projected coordinates on the cross section of three-dimensional physical space Construct a Gaussian decay confidence field centered on this prior anchor point. : in, The effective physical confidence radius for vision. Based on this, the fusion height curve for output state A is: in, To prevent extremely small real numbers with a denominator of zero, this fusion model ensures that even far from the anchor point... In the local area, the curve height is absolutely dominated by the LiDAR; while near the local anchor point, the high-frequency texture undulations retained after visual correction are extracted, realizing the safe inclusion of the high-confidence mode on the low-confidence correction mode.

[0079] For state B (visual-dominated state), under this condition, there is a high-density water mist ahead. After the lidar's rejection threshold is forcibly increased in S23, a large number of diffuse reflection point clouds are discarded. This results in the secondary echo of the physical water wall obtained through the pseudo-water mist wall exhibiting a sparse, discontinuous, discrete state in the lateral space, which is defined as a sparse absolute depth anchor point set. At this point, high-reliability visual acquisition It possesses excellent relative continuity, but may exhibit an overall drift deviation in absolute height during perspective transformations.

[0080] The edge computing component employs a radial basis function interpolation model, utilizing sparse radar anchor points to perform absolute-scale deformation calibration on continuous visual height curves. The deformation compensation field is then calculated. : in, The influence radius constant of the basis functions; coefficient vector By solving, the condition is satisfied at each radar anchor point. The system obtains the linear equations. The system outputs the height curve after deformation calibration: Instantaneous propulsion speed Spatiotemporal dynamics solution: After outputting the full-section tidal bore height curve for the current frame, the edge computing component needs to further calculate the instantaneous propulsion velocity distribution field of the tidal bore. This is considering that the unmanned surface vessel is currently moving at a backward speed. Execute forward dynamic navigation, meaning the unmanned surface vessel's own reference coordinate system is in dynamic retreat, extracting the longitudinal advancement array of the full-section tidal front physical boundary generated during the fusion process under a unified absolute geographic coordinate system. To overcome the transient jitter caused by single-frame time difference, the edge computing component introduces a spatiotemporal exponential moving average dynamic filtering model that considers inertial hysteresis to calculate the instantaneous propulsion velocity. : in, The time interval between two consecutive multimodal synchronous samplings; To update the weights.

[0081] S3. Impact pressure prediction incorporating tidal crest characteristics After obtaining the spatiotemporal physical field of the tide surface spanning the width of the channel in the current frame, in order to solve the problem that the destructive force of tidal surge cannot be accurately measured by the single dimension of tide height in maritime supervision, the edge computing component extrapolates the absolute impact pressure field of each area of ​​the channel in the future period.

[0082] S31 Tidal Wave Propulsion Trajectory and Characteristics Deduction: Edge computing components can be configured with an advance warning timeframe, i.e., the time span for future predictions. The current lateral height distribution curve is based on the output of step S24. With instantaneous propulsion velocity distribution curve Calculate the lateral coordinates of each point on the tidal bore curve. In the future The expected absolute longitudinal advance position : In the short-term forecast window Within the system, the tidal bore maintains the current hydrodynamic energy conservation, and the system directly maps the current elevation and flow velocity to the future evolution position, that is, defines the prediction point at the future moment. Expected tidal height at the location Expected speed of advancement .

[0083] S32 Space Impact Pressure Array Calculation: Considering the extremely high levels of suspended sediment in the Qiantang River during its high tide, directly using the standard pure water density to calculate the dynamic water pressure would lead to a serious underestimation of the destructive force.

[0084] The density of high-turbidity sediment fluid is introduced based on shipborne water quality sensors or locally pre-set hydrological parameters. : in, The density is the standard freshwater density; The nominal density of suspended sediment particles; This represents the real-time sediment concentration.

[0085] Edge computing components utilize a superposition model of dynamic and hydrostatic pressure in a two-dimensional absolute geographic space system. Below, the instantaneous impact pressure array when the future tidal bore sweeps across the channel area is calculated. : in, This is the acceleration due to gravity. The array transforms the three-dimensional ranging and velocity data obtained from multi-source sensing fusion into a pressure index array characterizing physical destructive forces.

[0086] S4. Construction of Dynamic Risk Weighting Diagram and Guidance for Ship Safety and Risk Avoidance This step couples the physical impact pressure array output by S3 with the static properties of surrounding vessels, avoiding the drawbacks of the traditional one-size-fits-all approach to maritime early warning. By finding the relatively safe waters with the lowest local pressure, personalized risk avoidance guidance broadcasts are achieved.

[0087] S41 Vulnerability Assessment of Target Ships: The edge computing component retrieves the set of surrounding ship broadcast data synchronously collected in step S13. For the first [unit] within the detection radius The target vessel was analyzed, including its static deadweight tonnage. State feature vector .

[0088] Since ships of different tonnages exhibit significant differences in their capsizing resistance thresholds to the same tidal surge pressure, a vulnerability coefficient based on deadweight tonnage is constructed. : In the formula, This serves as a reference tonnage for disaster resistance load capacity. This is the tonnage attenuation constant. The disaster vulnerability coefficient ensures that smaller ships are given a higher disaster vulnerability when facing the same water pressure.

[0089] S42 Dynamic Risk Weighting Graph Construction and Minimum Pressure Safe Zone Optimization: The derived instantaneous impact pressure array Vulnerability coefficient of a specific target vessel Perform linear coupling to generate a unique vessel specific to the target. Two-dimensional risk weight diagram : in, The preset theoretical peak value of extreme impact pressure is used to normalize the risk value to... Interval.

[0090] After generating a dedicated risk weight map, the edge computing component targets the vessel. Current absolute latitude and longitude coordinates Based on its effective maneuver radius Internally, a global minimum search is performed to find the optimal risk-avoidance coordinate point with the lowest cumulative risk value. : This coordinate point corresponds to a local low-pressure trough area formed in the tidal bore cross section due to energy attenuation or interference from intersecting tides, i.e., a location with relatively low impact pressure.

[0091] S43 Risk Aversion Guidance and Data Broadcast Release: Locking in the optimal avoidance coordinates for each target vessel. Then, the edge computing component generates a data set including the target point's latitude and longitude, and the estimated time of tidal arrival. And the highest level of maritime evacuation orders.

[0092] The system transmits the aforementioned hazard avoidance commands point-to-point to the corresponding target vessels via the VHF data link or AIS message broadcast interface in the communication and edge computing module. The shipborne terminal guides the unmanned surface vessel (USV) to enter a safe area with less impact pressure before the arrival of the tidal bore. At the same time, the USV maintains a forward-moving dynamic retreating navigation posture, continuously updates and releases the latest frame of risk weight map data, and completely completes the system-level business loop from multi-source perception fusion to maritime risk avoidance and scheduling.

[0093] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0094] In this specification, the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the descriptions of the embodiments described later are relatively simple, and relevant parts can be referred to the descriptions of the foregoing embodiments.

[0095] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for unmanned surface vessel (USV) cruising based on lidar and vision fusion, characterized in that, Includes the following steps: Perform multimodal raw data synchronous acquisition to obtain the current frame visual image, 3D point cloud set and ship broadcast data in the surrounding waters; calculate the visual confidence of the current frame visual image and the point cloud confidence of the 3D point cloud set, calculate the confidence difference, and adaptively switch to the corresponding constraint and guidance decision state according to the numerical area of ​​the confidence difference. Based on the decision state, the feature guidance and three-dimensional information extraction driven by the execution state output a continuous curve of the tidal head height distribution and an instantaneous propulsion speed distribution curve spanning the width of the channel. Based on the continuous curve of tidal head height distribution and the instantaneous propulsion velocity distribution curve, the tidal head propulsion trajectory and characteristics are deduced, and the instantaneous impact pressure array is calculated by combining the introduced high turbidity sediment fluid density. Find the optimal risk avoidance coordinates with the lowest cumulative risk value, generate and send maritime risk avoidance commands to the target vessel.

2. The unmanned surface vessel (USV) cruising method based on lidar and vision fusion as described in claim 1, characterized in that, The specific steps for calculating the visual confidence of the current frame's visual image and the point cloud confidence of the 3D point cloud set are as follows: Convert the current frame visual image into a single-channel grayscale image matrix. The second-order spatial differential operator is applied to extract the gradient magnitude matrix of the entire image, and the gradient variance of the image edges is calculated. And map the degree of optical distortion output. : in, This is a preset gradient baseline constant; A lightweight semantic segmentation network is invoked to process the current frame's visual image, outputting a binarized contour curve of the two-dimensional tidal wave peak. Extract the total number of unconnected contour segments in the binarized contour curve. Passing the exam Horizontal pixel span of each contour line segment Calculate the degree of contour continuity : The width of the image in pixels. The constant is the discontinuous decay penalty constant; and based on and Overall output visual confidence level ; In 3D space, a bounding box representing the region of interest is extracted. A subset of point clouds falling within this bounding box is then extracted, and the first echo energy, representing water mist interference, is statistically analyzed. With the second echo energy characterizing the solid water wall and Calculate the dual-echo energy ratio : It is a minimal positive real number. This is the critical energy transmission ratio threshold. The gain coefficient is used; principal component analysis of the covariance matrix is ​​performed on each point cloud data point within the bounding box to obtain the local surface normal vector. And calculate the local normal vector consistency factor in the spatial nearest neighborhood. : This represents the total number of points within the bounding box. The number of nearest neighbors; and based on and Overall output point cloud confidence .

3. The unmanned surface vessel (USV) cruising method based on lidar and vision fusion as described in claim 2, characterized in that, Based on the numerical range of confidence level differences, the system adaptively selects the corresponding constraint and guidance decision states, specifically including: Calculate confidence difference ,according to Entering three decision states: when When the radar is in a dominant state, the system switches to the correction branch of the lidar's unidirectional guided vision. when When the system is determined to be in a vision-dominated state, it switches to the fog-breaking branch of the vision-guided unidirectional lidar. when and and When all values ​​exceed the system's minimum safe operating threshold, the system is determined to be in a dual-source equilibrium state, and the fusion branch with asymmetric timing bidirectional guidance is initiated; among them, This is a preset critical threshold for difference.

4. The unmanned surface vessel (USV) cruising method based on lidar and vision fusion as described in claim 3, characterized in that, When entering the asymmetric temporally bidirectional guided fusion branch, cross-modal space dimensionality reduction and feature extraction are performed: the smallest two-dimensional rectangular bounding box that can completely enclose the binarized contour curve is calculated. Combined with the pre-calibrated camera intrinsic parameter matrix Homogeneous transformation matrix with external parameters Homogeneous coordinates of any point in a 3D point cloud set Perform spatial dimensionality reduction filtering constraint equations: in, As the scale-depth normalization factor, Perspective pixel coordinates; only the projection falling into the pixel coordinates are retained. The internal point cloud set constitutes a frustum dimensionality reduction subset. ; In the dimensionality reduction subset of the frustum In this process, the Euclidean distance difference between two adjacent point clouds is calculated as a continuous depth mutation operator. When the scanning line appears When the depth exceeds the rigidity threshold, lock the 3D water wall entity cluster and calculate its 3D spatial geometric center point. And force a dimensionality reduction projection back to the current frame's 2D image to obtain the target pixel coordinates. It is used as the sole high-priority absolute seed point for the growth of the two-dimensional tidal body region.

5. The unmanned surface vessel (USV) cruising method based on lidar and vision fusion as described in claim 3, characterized in that, When switching to the correction branch of unidirectional vision guidance from the LiDAR, image enhancement and visual correction are performed: Extract the 3D water wall entity clusters locked by deep mutation and calculate their 3D geometric center points, then project them onto a 2D image to obtain their absolute physical coordinates. ;by As the core, a depth-space prior weight graph is generated. And superimposed on the original blurred image: These are the two-dimensional coordinates of the image pixels. and This represents the two-dimensional spatial broadening coefficient; based on the prior weight map, local edge gradient features are forcibly enhanced, and the enhanced gradient feature map is output. : In the formula, This represents the second-order differential gradient value of the original blurred image. The gain coefficient is defined by forcibly enhancing feature map nodes in the visual network using absolute physical coordinates, and the modified dynamic region growth threshold is defined accordingly. This forces visual algorithms to only... Limit region growth within the neighborhood: The growth threshold for the base region.

6. The unmanned surface vessel (USV) cruising method based on lidar and vision fusion as described in claim 3, characterized in that, When switching to the fog-breaking branch of the vision-guided unidirectional LiDAR, point cloud filtering and inverse perspective mapping are performed: extracting the maximum vertical pixel coordinates of the lower edge of the 2D tide crest bounding box. Combined with the physical height of the installation Longitudinal focal length parameters and the coordinates of the longitudinal optical center principal point Calculate the absolute depth of a uniform virtual light curtain. : and all perspective coordinates within the bounding box The depth-distance function is uniformly assigned as ; For any point cloud, determine the true physical distance from the origin. The point data contains water mist noise, so the threshold for removing the first echo is increased. Simultaneously reduce the predetermined depth behind the light curtain. The second echo reflection intensity reception threshold within : Based on the basic water mist reflection filtration threshold, The entity reflection threshold is used; subsequently, a set of two-dimensional semantic masks for the main tidal subject is extracted from the output of the extreme region growing. The visual tidal bore height estimate is obtained by using inverse perspective transformation. : This is the scale-depth compensation coefficient. This represents the Z-axis elevation component of a spatial vector.

7. The unmanned surface vessel (USV) cruising method based on lidar and vision fusion as described in claim 3, characterized in that, When entering the dual-source equalization state, in the horizontal coordinate system At each discrete grid point, perform point-by-point weighted averaging based on dynamic confidence to obtain the initial state fused height array. A one-dimensional smooth kernel was introduced along the cross section of the waterway. Perform discrete spatial convolution operations to output a high-precision fused height distribution curve. : For spatial convolution integral variables, Let the radius be the local sliding window radius. This is the fluid continuity smoothing coefficient.

8. The unmanned surface vessel (USV) cruising method based on lidar and vision fusion as described in claim 1, characterized in that, The calculation of the instantaneous impact pressure array specifically includes: The density of high-turbidity sediment fluid is introduced based on pre-set hydrological empirical parameters. : In the formula, For standard freshwater density, The nominal density of suspended sediment particles. To determine real-time sediment concentration, a superposition model of hydrodynamic and hydrostatic pressure is used in a two-dimensional absolute geographic space system. The instantaneous impact pressure array is generated by calculation. : In the formula, It is the acceleration due to gravity. and respectively extrapolating to future moments The expected speed of advance and the expected height of the tidal bore.

9. The unmanned surface vessel (USV) cruising method based on lidar and vision fusion as described in claim 8, characterized in that, The generation of a two-dimensional risk weight diagram specific to the target vessel and the optimization of the optimal avoidance coordinates specifically include: constructing a vulnerability coefficient based on deadweight tonnage. In the formula As a benchmark for disaster resistance load capacity, The tonnage attenuation constant is used to represent the instantaneous impact pressure array. With disaster vulnerability coefficient Linear coupling is performed to generate a two-dimensional risk weight graph. : The preset theoretical peak value of extreme impact pressure; based on the target ship's current absolute latitude and longitude coordinates, within its effective maneuver radius. Internally, a global minimum search is performed to find the optimal risk-avoidance coordinates with the lowest cumulative risk value. : 。 10. An unmanned surface vessel (USV) cruise system based on lidar and vision fusion, used to execute the USV cruise method based on lidar and vision fusion as described in claim 1, characterized in that, The system includes: Confidence calculation module: Performs synchronous acquisition of multimodal raw data, obtains the current frame visual image, 3D point cloud set and ship broadcast data in the surrounding waters; calculates the visual confidence of the current frame visual image and the point cloud confidence of the 3D point cloud set, calculates the confidence difference, and adaptively switches to the corresponding constraint and guidance decision state according to the numerical area of ​​the confidence difference. Tidal crest feature calculation module: Based on decision state, execution state-driven feature guidance and 3D information extraction, outputting a continuous curve of tidal crest height distribution and instantaneous propulsion speed distribution curve spanning the width of the channel; Impact pressure calculation module: Based on the continuous curve of tidal head height distribution and the instantaneous propulsion velocity distribution curve, it performs tidal head propulsion trajectory and characteristic deduction, and calculates the instantaneous impact pressure array by combining the introduced high turbidity sediment fluid density; Guidance module: Finds the optimal risk avoidance coordinates with the lowest cumulative risk value, generates and sends maritime risk avoidance commands to the target vessel.

Images