Ship digital channel perception method and system based on multi-source heterogeneous sensing

Abstract

The application belongs to a ship digital channel perception method and system based on multi-source heterogeneous sensing, comprising a preprocessing module, a feature enhancement extraction module, a coupling situation dynamic deduction module and an output module, relying on a millimeter wave radar, binocular vision, an environment monitoring device and a ship attitude sensor to build a multi-physical field perception network, carrying out joint modeling for electromagnetic fields, optical fields and hydrodynamic fields, completing space-time alignment, physical constraint fusion and channel situation dynamic deduction for multi-source heterogeneous data, realizing high real-time and high-reliability channel environment perception, and providing accurate environment data support for a ship berthing and unberthing intelligent auxiliary system.

Description

Technical Field

[0001] This application belongs to the field of intelligent navigation of ships and environmental perception of port waters, specifically involving a digital waterway perception method and system for ships based on multi-source heterogeneous sensing. Background Technology

[0002] When ships enter and leave port, and berth and unberth operations, the complex environment of port channels, including hydrology, meteorology, and obstacles, directly affects navigation safety. Current digital channel sensing for ships mostly uses single sensors to collect data, employing radar or vision equipment to detect targets separately, or relying on monitoring equipment to obtain hydrological and meteorological information. This approach fails to achieve comprehensive data integration and fusion analysis across the entire channel. Existing technologies have several shortcomings: radar and vision equipment are prone to false detections, missed detections, and blind spots in rain, fog, backlight, and low-light conditions; hydrological and meteorological equipment can only collect localized data, making it difficult to correlate channel geometry and obstacle information; various data are processed independently, without considering the coupling laws of hydrodynamic fields, electromagnetic wave fields, and optical fields. Wind and current can alter ship attitude and sensor observation benchmarks, while rain and fog simultaneously interfere with electromagnetic wave propagation and optical imaging. Due to the lack of multi-physics coupling modeling, existing solutions suffer from low modeling accuracy and large sensing delays, failing to meet the accuracy and real-time requirements of ship berthing and unberthing operations. Therefore, the industry urgently needs a multi-sensor fusion method for digital channel sensing of ships to improve the accuracy, reliability, and real-time performance of sensing. Summary of the Invention

[0003] To address the aforementioned issues, this application proposes a multi-physics coupled digital waterway multi-source perception method for ships, applicable to ship entry and exit from ports and berthing / unberthing operations. This method enables high-precision, high-real-time perception of the waterway environment, providing auxiliary decision-making for ship navigation safety. The technical solution is as follows: A digital navigation channel perception method for ships based on multi-source heterogeneous sensing includes the following steps: S1. Spatiotemporal alignment and coupling preprocessing of multi-source heterogeneous sensing data; S2. Enhanced extraction of waterway environmental features through multiphysics coupling; S3. Dynamic simulation of ship-environment coupling situation; S4. Output of waterway safety risk assessment and perception results.

[0004] Preferably, the multi-source heterogeneous data includes three-dimensional point cloud data output by millimeter-wave radar, stereo image pairs output by binocular vision system, hydrological and meteorological parameters collected by water environment monitoring sensors, and ship motion attitude data. The millimeter-wave radar outputs three-dimensional point cloud data, expressed as: ; in, For a moment A collection of three-dimensional point clouds acquired by radar; Indicates the first The spatial coordinates of a radar point cloud target point in the sensor coordinate system; The echo reflection intensity of the point cloud of the point radar; The total number of valid target points in the radar point cloud; The three-dimensional coordinates of obstacles obtained from radar point clouds and binocular vision reconstruction are uniformly transformed into a right-handed Cartesian coordinate system with the ship's center of gravity as the origin. The coordinate transformation formula is as follows: ; in, To unify coordinates under the ship coordinate system; These are the spatial coordinates in the sensor's original coordinate system. This is the rotation matrix from the sensor coordinate system to the ship coordinate system; This is the corresponding translation vector.

[0005] Preferably, a single point is calculated with its neighbors. Average spatial distance with standard deviation When the average distance of a single point satisfies ,in Using an empirical threshold, the point is determined to be a noise outlier and is removed. For the left image With the image on the right Epipolar correction is performed, a dense disparity map is calculated using a stereo matching algorithm, and depth information is inversely calculated. Mismatched pixels in disparity abrupt changes and texture loss areas are removed. Finally, median filtering is used to remove isolated noise points, and the denoised point cloud data is output. With enhanced visual feature maps .

[0006] Preferably, the denoised point cloud Project onto a voxel grid and calculate the mean radar reflection intensity within each voxel. Simultaneously, the enhanced visual feature map Projected onto the same voxel grid, calculate the average grayscale value of visual pixels within each voxel. and variance ; For each non-empty voxel Construct a multidimensional feature vector: ; in, voxels The corresponding multidimensional feature vector; This represents the average reflection intensity of the radar point cloud within the voxel; These represent the mean and variance of the visual pixel grayscale values ​​falling into the voxel, respectively. For the corresponding spatial location, the hydrological and meteorological monitoring parameters are wind speed, current velocity, and wave height, respectively. ; For a moment The multi-source fusion feature matrix serves as the unified input for subsequent multiphysics modeling.

[0007] Preferably, the multi-source fusion feature matrix Construct three independent processing channels: Hydrodynamic field channel: Input velocity vector Flow angle and wave height ; Electromagnetic wave field channel: Input radar reflection intensity distribution ; Optical field channel: Input binocular visual texture features , And depth information obtained by inverse calculation from the disparity map; Dilated convolutional kernels with different dilation rates are used to perform multi-scale scanning of the feature matrix to capture environmental features at different levels, from local details to global trends. The formula for calculating the output feature map of a single channel is as follows: ; in, Indicates the category of physical field; The expansion rate; For physical fields In expansion rate The feature map output is below; These are the coordinates of the current convolution kernel center in the 3D grid; For the first Each physical field channel in the convolution kernel Convolution weights at three upsampling offset positions; To correspond to the perceptual feature function of the physical field, each physical field channel can output a set of feature maps containing local details and macroscopic trends. .

[0008] Preferably, all multi-scale feature maps from the hydrodynamic field, electromagnetic wave field, and optical field are stitched together along the channel dimension to form a fused tensor after multi-physics feature stitching. : ; An attention mechanism is introduced to perform weighted filtering of the fused features, and the importance weight of each feature channel is calculated: ; ; In the formula, Attention weights For feature maps.

[0009] Preferably, step S3 involves dynamic simulation of the ship-environment coupled situation: S31. Environmental disturbance intensity assessment: With sliding time window Units for feature maps Perform analysis to generate adaptive correction coefficients. and : ; ; in, As a regulating factor; The rate of change of flow velocity over time; Wave energy spectral density The proportion of high-frequency components; The change in the number of dynamic obstacles detected by radar and vision over time; S32. An improved Kalman filter framework is introduced to modify the traditional state transition equation. The state update formula is as follows: ; in, This represents the current channel situation vector. This is an estimate of the state at the previous moment; The current time is represented by the multiphysics feature map. Extracted core feature vectors; S33. Continuously iterate the above state update process to obtain the waterway environment situation feature vector sequence. This provides a stable and reliable state input for subsequent risk assessment.

[0010] Preferably, the output of step S4, waterway safety risk assessment and perception results, is as follows: S41. Spatiotemporal feature splicing and fusion: The spatial feature vector at the current moment With time-series situation feature vector Perform channel concatenation to construct a joint feature vector: ; in, The spatiotemporal fusion total feature vector; spatial feature vector By analyzing multiphysics feature maps Temporal situation feature vectors obtained by performing global average pooling and feature flattening operations. By analyzing the situational state vector Obtained by performing time-series feature encoding; S42. Feature Mapping and Dimensionality Reduction in Fully Connected Layers: Connecting High-Dimensional Joint Feature Vectors The input is a multi-layer fully connected neural network, which uses a non-linear activation function to reconstruct feature weights and remove redundant information, outputting a refined core feature vector. ; S43. Use the softmax function on the core feature vectors. After normalization, calculate the probability distribution of the current waterway belonging to each preset risk level: ; in, Number of risk level categories; For the first The network output score corresponding to the risk level; The probability corresponding to the risk level S44. Risk Assessment and Perception Results Output: According to... Determine the primary risk level and complete a secondary verification in conjunction with port safety control thresholds.

[0011] A ship digital waterway perception system based on multi-source heterogeneous sensing includes a preprocessing module, a feature enhancement and extraction module, a coupled situational dynamic inference module, and an output module. Preprocessing module: unifies the spatiotemporal reference of various sensor data, completes noise suppression processing, generates standardized datasets, and provides effective input for subsequent multiphysics modeling; Feature enhancement and extraction module: Sets up three parallel data processing channels for hydrodynamic field, electromagnetic wave field and optical field to complete multi-scale feature extraction and cross-physical field feature fusion; Coupled situation dynamic simulation: Using a sliding time window as the statistical unit, adaptive correction coefficients are generated by combining the rate of change of flow velocity, wave energy distribution, and dynamic obstacle quantity changes. The channel situation state vector is recursively updated under the Kalman filter framework. Output module: Calculates the probability distribution corresponding to each risk level of the waterway using network output data, determines the current risk level according to the principle of maximum probability, and completes secondary verification in conjunction with port safety management regulations. When the waterway is determined to be in a high-risk state, it triggers a graded audible and visual warning through the bridge human-machine interface.

[0012] Preferably, it also includes a multi-source heterogeneous sensing module: integrating millimeter-wave radar, binocular vision system, hydrological environment monitoring sensor, and ship motion attitude sensor, responsible for collecting raw environmental data and ship status data; Control module: Employs an industrial control computer to perform data processing and logic control.

[0013] Human-computer interaction and decision output module: includes a dashboard display screen, an audible and visual alarm device, and an external communication interface, enabling data display, risk warning, and linkage with external devices.

[0014] Compared with the prior art, the beneficial effects of this application are as follows: 1. Overcome the shortcomings of single sensors in terms of weak perception robustness under complex weather and lighting conditions, and the tendency to misjudge or miss targets.

[0015] 2. To address the problem that multi-source heterogeneous sensor data lacks a unified spatiotemporal reference, fails to achieve deep fusion by combining physical field constraints, and thus cannot fully characterize the environmental state of the entire waterway.

[0016] 3. To address the problem of the disconnect between the channel perception results and the ship's six-degree-of-freedom motion attitude and hydrodynamic environment, resulting in insufficient system adaptability to operating conditions.

[0017] 4. To address the technical problem of significant channel perception delays during ship berthing and departure phases, which prevents millisecond-level real-time safety decision-making. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the overall architecture. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0020] A digital navigation channel perception method for ships based on multi-source heterogeneous sensing includes the following steps: S1. Spatiotemporal alignment and coupling preprocessing of multi-source heterogeneous sensing data: This step is used to address issues such as inconsistent sampling frequencies, spatial coordinate systems, and data dimensions among different sensors, as well as interference from complex environmental noise. It completes the spatiotemporal registration, denoising, and preliminary fusion of multi-source data to generate a unified and standardized dataset.

[0021] Step 1: Unifying the spatiotemporal reference of multi-source data: The system simultaneously collects three types of core sensor data: The first type is 3D point cloud data output by millimeter-wave radar, expressed as: ; in, For a moment A collection of three-dimensional point clouds acquired by radar; Indicates the first The spatial coordinates of a radar point cloud target point in the sensor coordinate system; The echo reflection intensity of the radar point cloud at that point; This represents the total number of valid target points in the radar point cloud.

[0022] The second type is the stereo image pair output by a binocular vision system, with the image from the left camera denoted as... The image from the right camera is recorded as The typical image resolution is 1920×1080, and the frame rate is no less than 30fps.

[0023] The third category is time-series data output by aquatic environmental monitoring equipment, including wind speed. Wind angle Flow rate Flow angle and wave height Parameters such as sampling frequency are generally not lower than 1Hz.

[0024] In terms of time synchronization, the system uses the ship navigation PPS second pulse signal as a unified time reference and adds a standard timestamp to all data. To address the differences in sampling frequencies of radar (10~20Hz), vision (30Hz), and environmental monitoring (1Hz), linear interpolation is used to complete resampling, unify the time axis of all data, and eliminate perception delay and data jitter.

[0025] Regarding the unification of spatial references, the extrinsic parameter matrix between the radar and the binocular vision system is obtained in advance through offline joint calibration. And the extrinsic parameter matrix between the vision system and the ship coordinate system. .in, This is the rotational extrinsic parameter matrix between the radar and the binocular vision system; The translational extrinsic parameter vector between the radar and the binocular vision system; This is the rotation matrix between the binocular vision system and the ship's coordinate system; This is the translation vector between the binocular vision system and the ship's coordinate system. In actual operation, the system uniformly transforms the three-dimensional coordinates of obstacles obtained from radar point clouds and binocular vision reconstruction to a right-handed Cartesian coordinate system with the ship's center of gravity as the origin. The coordinate transformation formula is: in, To unify coordinates under the ship coordinate system; These are the spatial coordinates in the sensor's original coordinate system. This is the rotation matrix from the sensor coordinate system to the ship coordinate system; This is the corresponding translation vector.

[0026] Through the above processing, the system obtains a fused dataset with unified spatiotemporal reference.

[0027] Step 2: Environmental noise suppression and interference compensation: To address false radar scattering points generated by rain and fog, a statistical filtering algorithm is used: calculating the distance between a single point and its neighbors. (Preferred) Average spatial distance with standard deviation When the average distance of a single point satisfies ,in For empirical threshold (preferred) The point was identified as a noise outlier and removed.

[0028] To address the issues of mismatching in backlight and noise at night in binocular vision images, the left and right images were first processed. and Epipolar correction is performed, a dense disparity map is calculated using a stereo matching algorithm, and depth information is inversely calculated. Mismatched pixels in disparity abrupt changes and texture loss areas are removed. Finally, median filtering is used to remove isolated noise points, and the denoised point cloud data is output. With enhanced visual feature maps .

[0029] Step 3: Multi-source heterogeneous data fusion: To further unify the representation of geometric, textural, and hydrological information, the system projects the processed radar point cloud and visual features onto a ship-centric 3D spatial mesh voxel model. The voxel size can be set according to the channel scale, preferably set to [value missing]. .

[0030] Denoising the point cloud Project onto a voxel grid and calculate the mean radar reflection intensity within each voxel. Simultaneously, the enhanced visual feature map Projected onto the same voxel grid, calculate the average grayscale value of visual pixels within each voxel. and variance .

[0031] For each non-empty voxel The system constructs a multidimensional feature vector: ; in, voxels The corresponding multidimensional feature vector; This represents the average reflection intensity of the radar point cloud within the voxel; These represent the mean and variance of the visual pixel grayscale values ​​falling into the voxel, respectively. These are the hydrological and meteorological monitoring parameters for the corresponding spatial locations.

[0032] ; The system will use the feature vector of a single voxel By spatial index Organize. Iterate through the current time. All non-empty voxels (assuming to be) (The system generates 1000 features), and arranges their corresponding multidimensional feature vectors sequentially. Thus, the system obtains a structured multidimensional waterway environment feature matrix. For a moment The multi-source fusion feature matrix serves as the unified input for subsequent multiphysics modeling.

[0033] S2. Enhanced extraction of waterway environment features through multiphysics coupling: This step, based on the fusion feature matrix, completes the multi-physics feature extraction and fusion through three parallel channels, enabling accurate identification of the waterway environment and obstacles.

[0034] Step 1: Construction of Multiphysics Feature Channels: The system targets the input multi-source fusion feature matrix Construct three independent processing channels: (1) Hydrodynamic field channel: Input velocity vector Flow angle and wave height It is used to characterize the impact of water motion on ship maneuverability.

[0035] (2) Electromagnetic wave field channel: Input radar reflection intensity distribution It is used to characterize the geometric contours and material properties of obstacles.

[0036] (3) Optical field channel: Input binocular visual texture features , And the depth information obtained by inverse calculation from the disparity map is used to characterize the spatial morphology of the channel boundary and static facilities.

[0037] Step 2: Parallel computation of multi-scale features: Within each physics channel, the system employs dilated convolution kernels with varying dilation rates to perform multi-scale scanning of the feature matrix, capturing environmental features at different levels, from local details to global trends. The formula for calculating the output feature map of a single channel is as follows: ; in, Indicates the category of physical field; For the expansion rate, we can take... Equal discrete values; For physical fields In expansion rate The feature map output is below; For the first The physical field channel in the first Convolution weights at each sampling location; This refers to the perceptual characteristic function of the corresponding physical field (such as flow velocity field, reflection intensity field, or texture field).

[0038] Through the multi-scale scanning described above, each physical field channel can output a set of feature maps containing local details and macroscopic trends. .

[0039] Step 3: Cross-physics feature fusion and screening: The system stitches together all multi-scale feature maps from hydrodynamic, electromagnetic, and optical fields along the channel dimension to form a fused tensor of multi-physics feature stitching. : ; Subsequently, an attention mechanism is introduced to perform weighted filtering of the fused features, calculating the importance weight of each feature channel: ; in, For the first The scores for each feature channel can be predicted by a lightweight fully connected subnetwork. These are the normalized weighting coefficients.

[0040] ; In the formula, This indicates the attention weight. With the corresponding first The feature channels are weighted and summed (or multiplied channel by channel) to complete the "adaptive filtering" of features. This indicates a "noise suppression" operation, which can be ReLU, Sigmoid, or a specific filtering function used to remove invalid features and retain significant environmental features. Finally, the system outputs a channel environmental feature map after weighted fusion and noise suppression. This map comprehensively reflects the waterway environment under the coupling effect of multiple physics fields.

[0041] S3. Dynamic simulation of ship-environment coupling situation: This step combines ship dynamics models with Kalman filtering algorithms to achieve adaptive updates of the waterway situation and establish a data link between environmental perception and ship motion status.

[0042] Step 1: Environmental disturbance intensity assessment: The system uses a sliding time window (For example Using units of ) for feature maps An analysis was conducted, and the following factors were considered to generate adaptive correction coefficients. and : (1) Rate of change of flow velocity over time ; (2) Wave energy spectral density High frequency component proportion ; (3) Rate of change of the number of dynamic obstacles detected by radar and vision over time .

[0043] The coefficient is calculated as follows: ; ; in, It is a regulating factor.

[0044] Based on the above evaluation results, the system generates adaptive correction coefficients. Environmental feature enhancement coefficient It is used to dynamically adjust the state update strategy.

[0045] Step 2: Adaptive state update calculation: The system introduces an improved Kalman filter framework to modify the traditional state transition equation, and its state update formula is as follows: ; in, The current channel situation vector includes at least the channel boundary confidence, obstacle distribution probability field, and hydrological disturbance intensity level. This is an estimate of the state at the previous moment; The current time is represented by the multiphysics feature map. Extracted core feature vectors; The weighting coefficients are adaptively adjusted according to the intensity of environmental disturbances, satisfying... And it increases when the environment changes drastically. Increase when the environment is stable .

[0046] Through the aforementioned adaptive update mechanism, the system can respond promptly to environmental changes during the switching of operating conditions such as berthing and unberthing, while suppressing the cumulative error caused by steady-state noise.

[0047] Step 3: Situation and Trend Output: By continuously iterating through the above state update process, the system obtains a smooth, continuous, and physically meaningful sequence of waterway environmental situation feature vectors. This provides a stable and reliable state input for subsequent risk assessment.

[0048] S4. Output of waterway safety risk assessment and perception results: This step is the output stage of the entire perception method. By performing spatial-temporal dual-dimensional fusion and risk quantification on the situational characteristics, it achieves classification and trend prediction of the current waterway status and outputs safety prompts that can be used for driving assistance.

[0049] Step 1: Spatiotemporal feature splicing and fusion: The system will use the spatial feature vector at the current moment. (Source: Multiphysics Feature Map) (deep convolutional features) and temporal situation feature vector (Source: Situational state vector) Perform channel concatenation to construct a joint feature vector: ; in, This is the spatiotemporal fusion overall feature vector. This joint feature vector simultaneously contains the instantaneous spatial structure information and historical evolution trend information of the waterway environment, overcoming the problem that the representation ability of a single physical field or a single temporal feature is insufficient.

[0050] Step 2: Feature mapping and dimensionality reduction of fully connected layers: The above-mentioned high-dimensional joint features The input is a multi-layer fully connected neural network, which uses a non-linear activation function to reconstruct feature weights and remove redundant information, outputting a refined core feature vector. The network training objective is to minimize the cross-entropy loss function to ensure good separability of the feature space.

[0051] Step 3: Safety Risk Prediction Calculation: The system uses the softmax function to optimize the core feature vectors. After normalization, calculate the probability distribution of the current waterway belonging to each preset risk level: ; in, The number of risk level categories (e.g., four categories: safe, low risk, medium risk, and high risk); For the first The network output score corresponding to the risk level; This represents the probability corresponding to the risk level.

[0052] Step 4: Risk Assessment and Perception Results Output: according to The primary risk level is determined, and a secondary verification is performed based on port safety control thresholds (maximum permissible current velocity, wave height, safe encounter distance, etc.). If a high-risk level is determined, the system triggers a graded audible and visual warning on the bridge, and simultaneously overlays channel conditions, obstacle information, and risk alerts onto the electronic chart and AR display terminal to provide decision-making support for the crew.

[0053] Model training and deployment methods Model training phase: We collected measured port data under different seasons and weather conditions to construct a training sample library containing radar point clouds, binocular images, hydrological data, and ship attitude data. Using differential GPS trajectories, manually labeled channel boundaries, and measured obstacle distributions as ground truth labels, we jointly trained all parameters, including coordinate transformation matrices, attention weights, and Kalman filter gains. We iteratively optimized the model using the gradient backpropagation algorithm until its performance converged on the test set, thus completing model solidification.

[0054] On-site online operation phase: In real-world ship applications, the system accesses various sensor data streams via standard Ethernet, eliminating the need for manual on-site parameter adjustments. Relying on embedded high-performance computing units, it automatically executes end-to-end algorithms, from raw data preprocessing, spatiotemporal registration, multi-physics feature calculation to situational simulation, outputting standardized waterway perception results and risk levels, achieving all-weather, unmanned adaptive perception.

[0055] Implementation Results Explanation: Verified through real-world testing of the entire process of ship entry, exit, berthing, and unberthing at ports, this invention demonstrates significantly superior anti-interference capabilities and sensing stability compared to traditional single-sensor solutions under complex conditions such as rain, fog, low light, and strong sea states. The invention significantly improves the accuracy of channel boundary extraction and the detection rate of low-speed dynamic navigational obstructions. The system operates continuously for extended periods without data drift, and the overall response latency meets the millisecond-level real-time requirements for ship berthing operations. This invention can accurately capture subtle changes in waterways and potential navigation risks under the coupled effects of wind, waves, and currents, possessing excellent engineering applicability and industry promotion value.

[0056] A ship digital waterway perception system based on multi-source heterogeneous sensing includes a preprocessing module, a feature enhancement and extraction module, a coupled situational dynamic inference module, and an output module. The modules interact with each other via Ethernet or CAN bus. Preprocessing module: unifies the spatiotemporal reference of various sensor data, completes noise suppression processing, generates standardized datasets, and provides effective input for subsequent multiphysics modeling; Feature enhancement and extraction module: Sets up three parallel data processing channels for hydrodynamic field, electromagnetic wave field and optical field to complete multi-scale feature extraction and cross-physical field feature fusion; Coupled situation dynamic simulation: Using a sliding time window as the statistical unit, adaptive correction coefficients are generated by combining the rate of change of flow velocity, wave energy distribution, and dynamic obstacle quantity changes. The channel situation state vector is recursively updated under the Kalman filter framework. Output module: Calculates the probability distribution corresponding to each risk level of the waterway using network output data, determines the current risk level according to the principle of maximum probability, and completes secondary verification in conjunction with port safety management regulations. When the waterway is determined to be in a high-risk state, it triggers a graded audible and visual warning through the bridge human-machine interface.

[0057] Multi-source heterogeneous sensing module: integrates millimeter-wave radar, binocular vision system, hydrological environment monitoring sensor, and ship motion attitude sensor, responsible for collecting raw environmental data and ship status data.

[0058] Control module: It adopts an industrial control computer and is equipped with algorithms such as spatiotemporal alignment, multiphysics field fusion, and situational simulation to complete data processing and logic control.

[0059] Human-computer interaction and decision output module: includes a dashboard display screen, an audible and visual alarm device, and an external communication interface, enabling data display, risk warning, and linkage with external devices.

[0060] The attached diagram shows a three-layer architecture, from top to bottom: a multi-source heterogeneous perception layer, a data processing and fusion layer, and a situational analysis and decision-making layer. Each layer sequentially completes data flow and computation.

[0061] The multi-source heterogeneous sensing layer is the top layer of the architecture, mainly including millimeter-wave radar, binocular vision system, water environment monitoring equipment, and ship navigation system. It is responsible for collecting raw sensor data on waterway environment and ship movement, providing data input sources for this system.

[0062] The data processing and fusion layer is located in the middle of the architecture. It sequentially performs multi-source data spatiotemporal alignment and coupling preprocessing, multi-physics channel feature enhancement and extraction, and ship-environment coupled situational dynamic simulation, thus completing multi-source data parsing, feature fusion and state update.

[0063] The situation simulation and decision-making layer is located at the bottom of the architecture. Through spatiotemporal feature fusion and quantitative assessment of safety risks, it outputs waterway environmental situation maps and graded early warning signals, and pushes them to the bridge human-machine interaction equipment.

[0064] An adaptive feedback loop is set on the right side of the architecture to adjust the operating parameters of the data processing layer in reverse order based on the output of the decision layer, forming a closed-loop optimization structure.

[0065] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for digital navigation channel perception of ships based on multi-source heterogeneous sensing, characterized in that, Includes the following steps: S1. Spatiotemporal alignment and coupling preprocessing of multi-source heterogeneous sensing data; S2. Enhanced extraction of waterway environmental features through multiphysics coupling; S3. Dynamic simulation of ship-environment coupling situation; S4. Output of waterway safety risk assessment and perception results.

2. The ship digital waterway perception method based on multi-source heterogeneous sensing according to claim 1, characterized in that, Multi-source heterogeneous data includes 3D point cloud data output by millimeter-wave radar, stereo image pairs output by binocular vision system, hydrological and meteorological parameters collected by marine environment monitoring sensors, and ship motion and attitude data. The millimeter-wave radar outputs three-dimensional point cloud data, expressed as: ； in, For a moment A collection of three-dimensional point clouds acquired by radar; Indicates the first The spatial coordinates of a radar point cloud target point in the sensor coordinate system; The echo reflection intensity of the point cloud of the point radar; The total number of valid target points in the radar point cloud; The three-dimensional coordinates of obstacles obtained from radar point clouds and binocular vision reconstruction are uniformly transformed into a right-handed Cartesian coordinate system with the ship's center of gravity as the origin. The coordinate transformation formula is as follows: ； in, To unify coordinates under the ship coordinate system; These are the spatial coordinates in the sensor's original coordinate system. This is the rotation matrix from the sensor coordinate system to the ship coordinate system; This is the corresponding translation vector.

3. The ship digital waterway perception method based on multi-source heterogeneous sensing according to claim 1, characterized in that, Calculate a single point and its neighbors Average spatial distance with standard deviation When the average distance of a single point satisfies ,in Using an empirical threshold, the point is determined to be a noise outlier and is removed. For the left image With the image on the right Epipolar correction is performed, a dense disparity map is calculated using a stereo matching algorithm, and depth information is inversely calculated. Mismatched pixels in disparity abrupt changes and texture loss areas are removed. Finally, median filtering is used to remove isolated noise points, and the denoised point cloud data is output. With enhanced visual feature maps .

4. The ship digital waterway perception method based on multi-source heterogeneous sensing according to claim 3, characterized in that, Denoising the point cloud Project onto a voxel grid and calculate the mean radar reflection intensity within each voxel. Simultaneously, the enhanced visual feature map Projected onto the same voxel grid, calculate the average grayscale value of visual pixels within each voxel. and variance ; For each non-empty voxel Construct a multidimensional feature vector: ； in, voxels The corresponding multidimensional feature vector; This represents the average reflection intensity of the radar point cloud within a voxel; These represent the mean and variance of the visual pixel grayscale values ​​falling into the voxel, respectively. These represent wind speed, current speed, and wave height, respectively. ； For a moment The multi-source fusion feature matrix serves as the unified input for subsequent multiphysics modeling.

5. The ship digital waterway perception method based on multi-source heterogeneous sensing according to claim 1, characterized in that, Multi-source fusion feature matrix Construct three independent processing channels: Hydrodynamic field channel: Input velocity vector Flow angle and wave height ; Electromagnetic wave field channel: Input radar reflection intensity distribution ; Optical field channel: Input binocular visual texture features , And depth information obtained by inverse calculation from the disparity map; Dilated convolutional kernels with different dilation rates are used to perform multi-scale scanning of the feature matrix to capture environmental features at different levels, from local details to global trends. The formula for calculating the output feature map of a single channel is as follows: ； in, Indicates the category of physical field; The expansion rate; For physical fields In expansion rate The feature map output is below; These are the coordinates of the current convolution kernel center in the 3D grid; For the first Each physical field channel in the convolution kernel Convolution weights at three upsampling offset positions; To correspond to the perceptual feature function of the physical field, each physical field channel can output a set of feature maps containing local details and macroscopic trends. .

6. The ship digital waterway perception method based on multi-source heterogeneous sensing according to claim 1, characterized in that, All multi-scale feature maps from hydrodynamic, electromagnetic, and optical fields are stitched together along the channel dimension to form a fused tensor of multi-physics feature stitching. : ； An attention mechanism is introduced to perform weighted filtering of the fused features, and the importance weight of each feature channel is calculated: ； ； In the formula, Attention weights For feature maps.

7. The ship digital waterway perception method based on multi-source heterogeneous sensing according to claim 6, characterized in that, Step S3: Dynamic simulation of the ship-environment coupled situation: S31. Environmental disturbance intensity assessment: With sliding time window Units for feature maps Perform analysis to generate adaptive correction coefficients. and : ； ； in, As a regulating factor; The rate of change of flow velocity over time; Wave energy spectral density The proportion of high-frequency components; The change in the number of dynamic obstacles detected by radar and vision over time; S32. An improved Kalman filter framework is introduced to modify the traditional state transition equation. The state update formula is as follows: ； in, This represents the current channel situation vector. This is an estimate of the state at the previous moment; The current time is represented by the multiphysics feature map. Extracted core feature vectors; S33. Continuously iterate the above state update process to obtain the waterway environment situation feature vector sequence. This provides a stable and reliable state input for subsequent risk assessment.

8. The ship digital waterway perception method based on multi-source heterogeneous sensing according to claim 1, characterized in that, Step S4: Output of waterway safety risk assessment and perception results: S41. Spatiotemporal feature splicing and fusion: The spatial feature vector at the current moment With time-series situation feature vector Perform channel concatenation to construct a joint feature vector: ； in, The spatiotemporal fusion total feature vector; spatial feature vector By analyzing multiphysics feature maps Temporal situation feature vectors obtained by performing global average pooling and feature flattening operations. By analyzing the situational state vector Obtained by performing time-series feature encoding; S42. Feature Mapping and Dimensionality Reduction in Fully Connected Layers: Connecting High-Dimensional Joint Feature Vectors The input is a multi-layer fully connected neural network, which uses a non-linear activation function to reconstruct feature weights and remove redundant information, outputting a refined core feature vector. ; S43. Use the softmax function on the core feature vectors. After normalization, calculate the probability distribution of the current waterway belonging to each preset risk level: ； in, Number of risk level categories; For the first The network output score corresponding to the risk level; The probability corresponding to the risk level S44. Risk Assessment and Perception Results Output: According to... Determine the primary risk level and complete a secondary verification in conjunction with port safety control thresholds.

9. A ship digital waterway perception system based on multi-source heterogeneous sensing, characterized in that, It includes a preprocessing module, a feature enhancement and extraction module, a coupled situation dynamic inference module, and an output module; Preprocessing module: unifies the spatiotemporal reference of various sensor data, completes noise suppression processing, generates standardized datasets, and provides effective input for subsequent multiphysics modeling; Feature enhancement and extraction module: Sets up three parallel data processing channels for hydrodynamic field, electromagnetic wave field and optical field to complete multi-scale feature extraction and cross-physical field feature fusion; Coupled situation dynamic simulation: Using a sliding time window as the statistical unit, adaptive correction coefficients are generated by combining the rate of change of flow velocity, wave energy distribution, and dynamic obstacle quantity changes. The channel situation state vector is recursively updated under the Kalman filter framework. Output module: Calculates the probability distribution corresponding to each risk level of the waterway using network output data, determines the current risk level according to the principle of maximum probability, and completes secondary verification in conjunction with port safety management regulations. When the waterway is determined to be in a high-risk state, it triggers a graded audible and visual warning through the bridge human-machine interface.

10. The ship digital waterway perception system based on multi-source heterogeneous sensing according to claim 9, characterized in that, It also includes a multi-source heterogeneous sensing module: integrating millimeter-wave radar, binocular vision system, hydrological environment monitoring sensor, and ship motion attitude sensor, responsible for collecting raw environmental data and ship status data; Control module: Employs an industrial control computer to perform data processing and logic control; Human-computer interaction and decision output module: includes a dashboard display screen, an audible and visual alarm device, and an external communication interface, enabling data display, risk warning, and linkage with external devices.

Images