Underwater acoustic three-dimensional situation real-time visualization system for complex underwater environment

Abstract

This invention discloses a real-time underwater acoustic 3D situation visualization system for complex underwater environments, belonging to the field of underwater acoustic data processing technology. The system includes: a multimodal heterogeneous data acquisition module for simultaneously acquiring sonar data, hydrological environmental parameters, and platform attitude information; an adaptive data preprocessing module for dynamically adjusting the sonar data filtering strategy based on hydrological environmental parameters, while suppressing environmental noise and compensating for propagation distortion caused by changes in sound velocity, integrating multi-source data; a multi-source data fusion and 3D reconstruction module for unifying the preprocessed multi-source data into a global coordinate system by combining platform attitude information, and generating a 3D situation scene containing target geometric features, motion state, and hydrological environmental elements through heterogeneous data feature point matching and deep fusion; and a human-computer interaction and visualization module for providing a task-oriented adaptive operating interface.

Description

Technical Field

[0001] This invention belongs to the field of underwater acoustic data processing technology, specifically relating to a real-time visualization system for three-dimensional underwater acoustic situation in complex underwater environments. Background Technology

[0002] Underwater acoustic monitoring and control systems are core platforms for underwater target detection, identification, and tracking, and are widely used in marine resource exploration and underwater operations. With the increasing complexity of underwater exploration missions, higher demands are being placed on the comprehensive perception and intuitive presentation of the underwater situation.

[0003] Currently, radar, avionics systems, and ship navigation systems have developed mature technological systems in information visualization, data fusion, and intelligent decision support. However, directly transferring these technologies to the field of underwater acoustics faces significant challenges: the underwater environment is complex and variable, and sound wave propagation is greatly affected by factors such as temperature, salinity, and depth, resulting in severe multipath effects and background noise interference; underwater acoustic channel bandwidth is limited, data transmission rate is low, and there is a large delay; underwater target motion has three-dimensional spatial characteristics, and traditional two-dimensional display methods are unable to fully present key information such as target depth and pitch.

[0004] Existing underwater 3D visualization solutions mostly use a single sonar data source. For example, although phased array 3D acoustic camera sonar systems can acquire 3D point cloud data, the data stability is insufficient in complex hydrological environments. Some solutions use dual sonars arranged orthogonally to recover elevation information, but they lack effective fusion of multi-source heterogeneous data, resulting in poor visualization effects. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a real-time underwater acoustic three-dimensional situational awareness system for complex underwater environments, which achieves efficient situational awareness and intuitive presentation of complex underwater environments through multimodal heterogeneous data fusion and adaptive three-dimensional scene reconstruction.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A real-time underwater acoustic 3D situational awareness system for complex underwater environments includes: A multimodal heterogeneous data acquisition module is configured on an underwater mobile platform and a preset fixed observation node to synchronously acquire sonar data, hydrological environmental parameters and platform attitude information; The adaptive data preprocessing module is used to dynamically adjust the filtering strategy of sonar data according to hydrological environmental parameters, while suppressing environmental noise and compensating for propagation distortion caused by changes in sound speed, and integrating multi-source data. The multi-source data fusion and 3D reconstruction module is used to combine the platform's attitude information to unify the preprocessed multi-source data into the global coordinate system, and generate a 3D situation scene containing target geometric features, motion state and hydrological environment elements through heterogeneous data feature point matching and deep fusion. The human-computer interaction and visualization module provides a task-oriented adaptive operating interface to achieve multi-dimensional interactive presentation of three-dimensional situational scenarios and multi-platform collaborative situational display.

[0007] Furthermore, the multimodal heterogeneous data acquisition module includes a first multibeam sonar and a second multibeam sonar arranged orthogonally; The first multibeam sonar is arranged horizontally to obtain azimuth and range information, and the second multibeam sonar is arranged vertically to obtain elevation information. The detection areas of the first and second multibeam sonars need to overlap to maximize data correlation.

[0008] Furthermore, the adaptive data preprocessing module includes a hydrological parameter analysis unit and a dynamic filtering unit; The hydrological parameter analysis unit is used to calculate the current sound velocity profile based on real-time collected hydrological environmental parameters. The calculation method uses the Mackenzie formula; where, S represents water temperature, D represents water salinity, and D represents water depth. The dynamic filtering unit is used to adaptively adjust the passband characteristics of the filter according to the sound velocity profile, and to perform differentiated processing on the echo signals of different distance segments to compensate for the distortion caused by the bending of the sound ray. The formulas for adaptively adjusting the filter's transmission center frequency and bandwidth based on propagation distance and sound speed gradient are as follows: In the formula, and This indicates the adaptively adjusted transmission center frequency and transmission bandwidth. and This represents the initial transmit center frequency and initial transmit bandwidth before adaptive adjustment. Indicates the initial reference speed of sound. Indicates the transmission distance. This represents the average speed of sound at a propagation distance r. Represents the sound speed gradient The modulus, This indicates the preset adjustment coefficient.

[0009] Furthermore, the multi-source data fusion and 3D reconstruction module includes a spatiotemporal alignment unit, a feature extraction and matching unit, and a 3D scene construction unit; The spatiotemporal alignment unit is used to align the preprocessed multi-source data to the same time section based on the acquisition timestamp of each acquired data and the corresponding platform attitude information, and then transform it to the global coordinate system. The feature extraction and matching unit is used to extract geometric feature points from sonar point cloud data in multi-source data, and when the system is equipped with optical sensors, it extracts texture features from synchronously acquired optical images and establishes the correlation between heterogeneous features. The 3D scene construction unit is used to construct fused data based on multi-source data after spatiotemporal alignment and feature matching. An improved ray casting algorithm is used to adaptively sample the fused data at intervals. A continuous 3D field distribution is reconstructed through trilinear interpolation to generate a 3D situation scene containing target geometric features, motion state, and hydrological environmental elements. Specifically, the 3D scene construction unit maps optical textures to the 3D geometric surface reconstructed by sonar according to the correlation between heterogeneous features output by the feature extraction and matching unit, or optimizes the fusion weights of multi-source data based on the correlation between heterogeneous features to enhance the geometric accuracy and texture realism of the 3D situation scene.

[0010] Furthermore, the improved ray casting algorithm involves voxel partitioning of the fused data, identifying invalid voxels and skipping their internal resampling calculations; and adaptively adjusting the sampling step size based on the sound intensity gradient, increasing sampling density in regions of rapid sound intensity change and increasing the sampling step size in flat regions; specifically including: A multi-level density occupancy grid is constructed, discretizing the continuous space into an L-layer voxel grid. The corresponding level is selected for skipping in the blank space based on the distance between the sampling point and the viewpoint. The formula for calculating the adaptive sampling step size is as follows: In the formula, Indicates the adaptive sampling step size. This represents the adaptive sampling step size factor. This indicates the pre-defined base sampling step size; In the formula, and These represent the maximum and minimum sampling step size factors, respectively. This indicates the preset adjustment coefficient. Represents the sound intensity gradient The modulus, This represents the maximum sound intensity value in the current scene.

[0011] Furthermore, the human-computer interaction and visualization module supports multi-level situational display, including a macro-level global view and a micro-level target detail view; it also adopts a task-oriented interface design, automatically adjusting the interface layout and functional control priority according to the current task stage; and it supports multi-modal interaction methods such as voice, gesture and touch.

[0012] Furthermore, a real-time underwater acoustic 3D situational awareness system for complex underwater environments also includes an intelligent decision support module; wherein, the intelligent decision support module includes: The target automatic recognition unit uses a multi-scale convolutional neural network to detect and classify targets in a three-dimensional situational scene, and combines the characteristics of the target's motion state and historical trajectory to evaluate its identity confidence. The threat assessment unit calculates the threat level of a target and generates a threat situation map based on the target type, target motion parameters, distance information, hydrological cover conditions, and confidence assessment results. The target type is obtained from the classification results of the automatic target identification unit, the target motion parameters, distance information, and hydrological cover conditions are extracted from the three-dimensional situation scene, and the confidence assessment results are obtained from the identity confidence assessment of the automatic target identification unit. The decision-making simulation unit, based on the threat situation map and a pre-set decision rule base, predicts the future movement trend of the target and recommends corresponding decisions.

[0013] Furthermore, the multi-scale convolutional neural network of the target automatic recognition unit is a joint neural network based on three-dimensional fusion features; The three-dimensional fusion features include: spatial features of the target region extracted by a three-dimensional convolutional neural network, spectral features obtained by fusing Mel frequency scale features and Mel frequency cepstral coefficients extracted by a two-dimensional convolutional neural network, and motion features composed of target motion parameters. The three features are concatenated and then input into a fully connected network for classification and recognition.

[0014] Furthermore, the threat assessment unit employs an assessment algorithm based on multi-criteria fuzzy comprehensive evaluation. It fuzzifies the target type, target motion parameters, distance information, hydrological concealment conditions, and confidence assessment results into three fuzzy sets: low, medium, and high. The comprehensive threat value is then obtained through Mamdani fuzzy inference and the centroid method for defuzzification and mapped to a discrete threat level. The membership function in the fuzzy inference process uses a trapezoidal function, and the centroid method calculation formula is as follows: In the formula, This represents the overall threat value. This indicates the possible range of values ​​for the threat value. This indicates a comprehensive fuzzy output.

[0015] Furthermore, when predicting the future movement trend of the target, the decision-making inference unit uses an outlier-resistant Kalman filter algorithm to achieve target tracking, and constructs a test statistic through a chi-square test. ,in For the innovation vector, For the new information covariance; when Exceeding the set threshold Time determination of observation value If an observation is an outlier, the observation noise covariance will be adaptively adjusted or the observation will be skipped; the specific handling strategy is as follows: if Update according to standard Kalman filtering; if Then take , Perform a weighted update; if If the observation is not found, then skip that observation and only perform state prediction.

[0016] The beneficial effects of this invention are as follows: 1. Improve the reliability of situational awareness in complex environments: By fusing multimodal heterogeneous data and integrating information from multiple sources such as sonar, hydrology, and attitude, the problem of unstable data from a single sonar under complex hydrological conditions is overcome, and all-round perception of underwater targets is achieved. 2. Achieve efficient 3D scene reconstruction and visualization: By adopting an improved ray casting algorithm and using adaptive sampling and invalid voxel skipping mechanism, rendering efficiency is significantly improved while ensuring reconstruction accuracy, thus meeting the needs of real-time visualization; 3. Enhance intelligent decision support capabilities: Introduce deep learning and decision rule base to achieve automatic target identification and threat assessment, provide operators with real-time decision suggestions, and effectively improve the response speed and decision support level in high target density environments; 4. Optimize human-computer interaction experience: Adopt a task-oriented adaptive interface and multimodal interaction methods to simplify operation complexity and reduce operator cognitive load.

[0017] Other advantages, objectives, and features of the invention will be set forth in the following description and will be apparent to those skilled in the art in some respects, or may be learned by practice of the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0018] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the system modules of a real-time underwater acoustic three-dimensional situation visualization system for complex underwater environments, as described in an embodiment of the present invention. Figure 2 This is a schematic diagram of a unit module of a multi-source data fusion and three-dimensional reconstruction module in a real-time underwater acoustic three-dimensional situation visualization system for complex underwater environments, as described in an embodiment of the present invention. Figure 3 This is a schematic diagram of the overall modules of a real-time underwater acoustic three-dimensional situational awareness system for complex underwater environments, as described in an embodiment of the present invention. Detailed Implementation

[0020] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0021] like Figure 1 , Figure 2 and Figure 3 As shown, this invention proposes a real-time underwater acoustic three-dimensional situational awareness system for complex underwater environments, comprising: A multimodal heterogeneous data acquisition module is configured on an underwater mobile platform and a preset fixed observation node to synchronously acquire sonar data, hydrological environmental parameters and platform attitude information; The adaptive data preprocessing module is used to dynamically adjust the filtering strategy of sonar data according to hydrological environmental parameters, while suppressing environmental noise and compensating for propagation distortion caused by changes in sound speed, and integrating multi-source data. The multi-source data fusion and 3D reconstruction module is used to combine the platform's attitude information to unify the preprocessed multi-source data into the global coordinate system, and generate a 3D situation scene containing target geometric features, motion state and hydrological environment elements through heterogeneous data feature point matching and deep fusion. The intelligent decision support module is used to automatically classify and assess the threat of targets in a 3D scene based on a deep learning network, and generate real-time decision suggestions in combination with a preset decision rule base. The human-computer interaction and visualization module is used to provide a task-oriented adaptive operation interface to complete the multi-dimensional interactive presentation of three-dimensional situational scenes and the collaborative situational display across multiple platforms; The working principle and beneficial effects of the above technical solution are as follows: This invention constructs a complete underwater situational awareness and visualization processing workflow through collaborative acquisition of multi-source heterogeneous data, adaptive preprocessing, intelligent fusion reconstruction, and decision support. The system consists of a multi-modal heterogeneous data acquisition module, an adaptive data preprocessing module, a multi-source data fusion and 3D reconstruction module, an intelligent decision support module, a human-computer interaction and visualization module, and a distributed collaborative interface module. These modules work collaboratively to achieve end-to-end real-time processing from raw data to a 3D situational scenario and intelligent decision suggestions.

[0022] Specifically, in the data acquisition and preprocessing stage, a multimodal heterogeneous data acquisition module is configured on an underwater mobile platform (such as an autonomous underwater vehicle) and pre-set fixed observation nodes to simultaneously acquire multibeam forward-looking sonar data, side-scan sonar data, temperature, salinity, and depth (TDM) hydrographic data, and underwater platform attitude data. This module employs an orthogonally arranged first and second multibeam sonar: the first multibeam sonar is horizontally positioned to acquire the target's azimuth and range information, while the second multibeam sonar is vertically positioned to acquire the target's pitch angle information. The detection areas of the two sonars overlap to maximize data correlation, thereby obtaining complete three-dimensional spatial information of the target. Simultaneously, the TDM sensor acquires water temperature, salinity, and depth data in real time, and the fiber optic gyroscope provides the underwater vehicle's attitude angles (roll, pitch, yaw), angular velocity, and acceleration information, providing necessary input for subsequent processing.

[0023] The collected raw data is sent to the adaptive data preprocessing module. This module first uses the hydrological parameter analysis unit to calculate the current sound velocity profile c(T,S,D) based on real-time temperature, salinity, and depth data using the Mackenzie formula, and then constructs a continuous sound velocity-depth curve c(z) through spline interpolation. Subsequently, the dynamic filtering unit segments the detection distance according to the sound velocity curve and uses bandpass filters with different center frequencies and bandwidths to differentiate the echo signals from different distance segments: a wide-bandwidth filter is used to preserve details in the near-distance segment, while a narrow-band filter is used to suppress noise in the far-distance segment. The filter parameters are adaptively adjusted according to the propagation distance r and the sound velocity gradient, and the calculation formula is as follows: In the formula, and This indicates the adaptively adjusted transmission center frequency and transmission bandwidth. and This represents the initial transmit center frequency and initial transmit bandwidth before adaptive adjustment. Indicates the initial reference speed of sound. Indicates the transmission distance. This represents the average speed of sound at a propagation distance r. Represents the sound speed gradient The modulus, This indicates the preset adjustment coefficient.

[0024] In this application, one preferred parameter selection for the Mackenzie formula used for calculation is: Since the Mackenzie formula is an empirical formula, the coefficients in the formula already include the corresponding unit conversions; Furthermore, in the formula, the constant term represents the reference sound speed under standard conditions, the first-order temperature term represents the linear effect of a 1°C change in temperature on the sound speed, the second-order and third-order temperature terms represent the nonlinear effect of temperature on the sound speed, the salinity term represents the effect of salinity deviating from 35‰ on the sound speed, the first-order depth term represents the linear effect of increased depth (pressure) on the sound speed, the second-order depth term represents the nonlinear effect of depth on the sound speed, the temperature-salinity coupling term represents the coupled effect of temperature and salinity on the sound speed, and the temperature-depth coupling term represents the coupled effect of temperature and depth on the sound speed. The Mackenzie formula described above enables precise control of the current sound speed profile.

[0025] Simultaneously, based on the law of sound ray bending (Snell's Law), the theoretical propagation path and time delay from the sound source to the target are calculated using a ray tracing algorithm. This is compared with the actual measured time delay, and the target position is iteratively corrected to compensate for propagation distortion caused by changes in sound speed. Through this processing, environmental noise is effectively suppressed and sound speed distortion is corrected, resulting in high-quality multi-source data.

[0026] In the multi-source data fusion and 3D reconstruction stage, the preprocessed multi-source data enters the multi-source data fusion and 3D reconstruction module. This module consists of a spatiotemporal alignment unit, a feature extraction and matching unit, and a 3D scene construction unit.

[0027] The spatiotemporal alignment unit employs a motion compensation algorithm based on adaptive Kalman filtering to unify the data from various sensors to the same spatiotemporal reference. This algorithm uses a state vector containing position, velocity, and acceleration. Based on, Through the state transition matrix (Constant acceleration model) for state prediction, and utilize innovation and Kalman gain Adaptive estimation process noise covariance and observation noise covariance : , By using adaptive filtering, the system compensates for positional shifts caused by platform motion in real time, achieving precise spatiotemporal alignment of multi-source data and transforming the data to a global coordinate system.

[0028] The feature extraction and matching unit extracts geometric feature points (such as corner points and edge points) from sonar point cloud data. If the system is equipped with an optical sensor and the water visibility meets the imaging conditions, it extracts texture features such as SIFT from synchronously acquired high-definition optical images and establishes the correlation between heterogeneous features. The nearest neighbor algorithm is used to match feature pairs, and high-confidence matching pairs are selected for subsequent fusion.

[0029] The 3D scene construction unit constructs fused volumetric data based on multi-source data after spatiotemporal alignment and feature matching. First, the detection area is discretized into a regular 3D voxel grid. Then, the measurements from each sensor (such as the sound intensity of multibeam sonar, the echo intensity of side-scan sonar, and hydrological parameters) are mapped to corresponding voxels according to their spatial location. For sparse regions, radial basis function interpolation is used to fill the gaps, forming a continuous volumetric data field. During the fusion process, enhancement processing is achieved using feature matching results: the texture information of the optical image is projected onto the 3D geometric surface reconstructed from the sonar through feature correspondence, so that the final generated scene simultaneously possesses the geometric accuracy of the sonar and the texture realism of the optical image. The fusion weights of the sonar data and optical data in voxel assignment are dynamically adjusted according to the confidence level of feature matching. Using feature correspondence as a constraint, the sonar point cloud is locally fine-tuned by minimizing reprojection error to correct geometric distortions caused by sound velocity errors or platform jitter. If the system is not equipped with an optical sensor or the visibility of the water body does not meet the imaging conditions, then three-dimensional reconstruction is performed only based on sonar data. In this case, the fusion weight of the voxel assignment is determined according to the confidence level of the sonar measurement value. The confidence level is calculated by comprehensively considering factors such as beam directivity, propagation distance, and signal-to-noise ratio.

[0030] Subsequently, an improved ray casting algorithm was used to render the fused volume data. This algorithm first divides the 3D volume data into voxels, then uses an adaptive thresholding method to identify and mark invalid voxels based on the local signal-to-noise ratio, constructing a multi-level density occupancy grid to achieve fast empty space jumping. During sampling, the sound intensity gradient is used... The sampling step size is adaptively adjusted, and the specific calculation formula is as follows: In the formula, Indicates the adaptive sampling step size. This represents the adaptive sampling step size factor. This indicates the pre-set base sampling step size. and These represent the maximum and minimum sampling step size factors, respectively. This indicates the preset adjustment coefficient. Represents the sound intensity gradient The modulus, This represents the maximum sound intensity value in the current scene. The sound intensity gradient ∇I is preferably calculated using the central difference method.

[0031] In areas of drastic sound intensity change (such as target edges), sampling is densified, while in flat areas, the sampling step size is increased. The sampling points are trilinearly interpolated to obtain sound intensity values, which are then mapped to color and opacity according to the transfer function. Pixel colors are synthesized along the light rays from front to back, ultimately generating a 3D situational scene that includes terrain, target, and underwater sound propagation characteristics.

[0032] During the intelligent decision support phase, the generated 3D situational awareness scenario is simultaneously transmitted to the intelligent decision support module. This module consists of an automatic target identification unit, a threat assessment unit, and a decision inference unit.

[0033] The automatic target recognition unit employs a joint neural network based on 3D fusion features to detect and classify targets. This network extracts three types of features: spatial features of the target region extracted by the 3D convolutional neural network. (512-dimensional); spectral features obtained by fusing Mel frequency scale features (MFSC) and Mel frequency cepstral coefficients (MFCC) extracted by a two-dimensional convolutional neural network. (256 dimensions); Motion characteristics consisting of target velocity, acceleration, turning rate, trajectory curvature, etc. (12-dimensional). The three are spliced ​​together to form a 780-dimensional fusion feature, which is then input into a fully connected network for classification and recognition. The target's motion features and historical trajectory are combined to evaluate its identity confidence.

[0034] It is worth noting that there are many ways to assess identity confidence by combining target movement characteristics and historical trajectories in existing technologies. Any of these methods can be used for assessment depending on the actual situation, and will not be elaborated here.

[0035] The threat assessment unit employs an assessment algorithm based on multi-criteria fuzzy comprehensive evaluation. Threat factors such as target type, distance, speed, heading, hydrographic concealment, and maneuverability are fuzzified into three fuzzy sets: low, medium, and high. The comprehensive threat value is obtained through Mamdani fuzzy inference and the centroid method for defuzzification. This is then mapped to discrete threat levels to generate a threat situation map.

[0036] The decision-making unit employs an outlier-resistant Kalman filter algorithm to achieve target tracking and future motion trend prediction. This algorithm constructs a test statistic using the chi-square test. ,in For the innovation vector, For the new information covariance; when Exceeding the set threshold Time determination of observation value If an observation is an outlier, the observation noise covariance will be adaptively adjusted or the observation will be skipped; the specific handling strategy is as follows: if Update according to standard Kalman filtering; if Then take , Perform a weighted update; if If the observation is not found, skip it and only perform state prediction. Based on a pre-defined decision rule base, recommend appropriate decisions (such as avoidance or tracking). The method for recommending appropriate decisions uses rule matching and mapping, matching the current target movement trend, threat level, and other situational information with rule conditions in the pre-defined decision rule base, and outputting the corresponding response strategy through association mapping.

[0037] In the human-computer interaction and visualization stages, the human-computer interaction and visualization module provides a task-oriented, adaptive operating interface that supports multi-dimensional interactive presentation of 3D scenes. This module supports multi-level situational awareness: the macro-level global view uses a top-down perspective to display underwater topographic contours, sonar coverage sectors, multi-target distribution, and movement trajectories; the micro-level target detail view uses a following perspective to display the real-time attitude of the selected target in 3D and overlay attribute information. The interface design adopts a task-oriented strategy, automatically adjusting the interface layout and functional control priority according to the current task stage (search, tracking, etc.). It also supports multi-modal interaction methods such as voice, gestures, and touch, adapting to different operating habits under high-pressure environments.

[0038] Finally, to achieve data interconnection and interoperability with the ship's navigation system, the control system of unmanned underwater vehicles, and other navigation platforms, and to improve the reliability of joint operations, a distributed collaborative interface module can be configured. This module adopts the DDS publish / subscribe communication model, defining topics such as platform status, sensor data, local tracking, fused trajectory, and global situation; it uses octree encoding compression for 3D point clouds (compression ratio of over 10:1), and incremental encoding for trajectory data; it uses an NTP-like protocol for cross-platform clock synchronization and calculates one-way delay. and clock deviation Adjust the local clock; adopt a leader-based state replication mechanism to maintain data consistency and ensure the reliability of the distributed system.

[0039] Through the collaborative work of the above five stages, the system proposed in this application realizes a closed loop of the entire process from raw data acquisition, adaptive preprocessing, multi-source fusion 3D reconstruction to intelligent decision support and human-computer interaction, providing real-time, accurate, and intuitive situational awareness and auxiliary decision-making capabilities for complex underwater environments.

[0040] In summary, by simultaneously acquiring multi-source data from multibeam sonar, side-scan sonar, CTD (temperature, salinity, depth) hydrology, and platform attitude using a multimodal heterogeneous data acquisition module, the problems of unstable and incomplete data from single sonar under complex hydrological conditions are overcome. Orthogonally arranged dual-beam sonar simultaneously acquires the target's horizontal azimuth / range and vertical pitch information, providing a complete understanding of the target's three-dimensional spatial features. The adaptive data preprocessing module dynamically adjusts the filtering strategy and compensates for sound velocity distortion based on real-time hydrological parameters, effectively suppressing environmental noise at different distances and correcting target position shifts caused by sound velocity variations. This significantly improves data quality and environmental adaptability, achieving comprehensive and high-precision perception of underwater targets.

[0041] Meanwhile, the 3D scene construction unit employs an improved ray casting algorithm, skipping invalid voxels through multi-level density occupancy grids and adaptively adjusting the sampling step size based on the acoustic intensity gradient. This significantly improves rendering efficiency (reaching over 30fps in actual tests) while ensuring reconstruction accuracy, meeting real-time visualization requirements. The heterogeneous feature associations established by the feature extraction and matching unit are fully utilized in texture mapping and fusion weight optimization, mapping optical textures onto the geometric surfaces reconstructed by sonar, enhancing the geometric accuracy and texture realism of the 3D scene. Trilinear interpolation and adaptive color synthesis techniques ensure the continuity and intuitiveness of scene presentation, providing operators with more realistic and accurate situational awareness.

[0042] The intelligent decision support module transforms raw sensing data into actionable decision information. The automatic target identification unit employs a joint neural network based on 3D fusion features, integrating spatial, spectral, and motion characteristics, achieving a 92.5% recognition accuracy on the test set, particularly suitable for identifying small targets in complex underwater environments. The threat assessment unit uses a multi-criteria fuzzy comprehensive evaluation algorithm, effectively handling the uncertainty and ambiguity of threat factors and outputting an intuitive threat level. The decision inference unit uses an outlier-resistant Kalman filter algorithm, identifying and adaptively processing abnormal observations through chi-square tests to avoid filter divergence, improve tracking robustness, and, combined with a pre-set decision rule base, provides operators with real-time and accurate decision suggestions, effectively enhancing reaction speed and decision support levels in high-target-density environments.

[0043] Furthermore, the human-computer interaction and visualization module adopts a task-oriented adaptive interface design, automatically adjusting the interface layout and functional control priority according to the current task stage (search, tracking, etc.), highlighting key information, and simplifying operational complexity. It supports multi-level situational awareness display, with a macro-level global view and a micro-level target detail view presented collaboratively to meet the needs of different granular information. It also supports multi-modal interaction methods such as voice, gestures, and touch, adapting to different operating habits under high-pressure environments, reducing operator cognitive load, and improving operational convenience and efficiency.

[0044] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made to it in form and detail without departing from the scope defined by the claims of the present invention.

Claims

1. A real-time underwater acoustic three-dimensional situational awareness system for complex underwater environments, characterized in that, include: A multimodal heterogeneous data acquisition module is configured on an underwater mobile platform and a preset fixed observation node to synchronously acquire sonar data, hydrological environmental parameters and platform attitude information; The multimodal heterogeneous data acquisition module includes a first multibeam sonar and a second multibeam sonar arranged orthogonally. The first multi-beam sonar is arranged horizontally to acquire azimuth and range information, and the second multi-beam sonar is arranged vertically to acquire elevation information; wherein, the detection areas of the first multi-beam sonar and the second multi-beam sonar need to overlap to maximize the data correlation. The adaptive data preprocessing module is used to dynamically adjust the filtering strategy of sonar data according to hydrological environmental parameters, while suppressing environmental noise and compensating for propagation distortion caused by changes in sound speed, and integrating multi-source data. The adaptive data preprocessing module includes a hydrological parameter analysis unit and a dynamic filtering unit; The hydrological parameter analysis unit is used to calculate the current sound velocity profile c(T,S,D) based on the real-time collected hydrological environmental parameters. The calculation method adopts the Mackenzie formula; where T represents water temperature, S represents water salinity, and D represents water depth. The dynamic filtering unit is used to adaptively adjust the passband characteristics of the filter according to the sound velocity profile, and to perform differentiated processing on the echo signals of different distance segments to compensate for the distortion caused by the bending of the sound ray. The calculation formula for the adaptive adjustment of the filter's transmission center frequency and bandwidth based on the propagation distance and sound speed gradient is as follows: In the formula, B(r) represents the adaptively adjusted transmission center frequency and transmission bandwidth. and This represents the initial transmit center frequency and initial transmit bandwidth before adaptive adjustment. Let r represent the initial reference speed of sound, r represent the propagation distance, and c(r) represent the average speed of sound at the propagation distance r. Represents the sound speed gradient The modulus, This indicates a pre-set adjustment coefficient; The multi-source data fusion and 3D reconstruction module is used to combine the platform's attitude information to unify the preprocessed multi-source data into the global coordinate system, and generate a 3D situation scene containing target geometric features, motion state and hydrological environment elements through heterogeneous data feature point matching and deep fusion. The human-computer interaction and visualization module provides a task-oriented adaptive operating interface to achieve multi-dimensional interactive presentation of three-dimensional situational scenarios and multi-platform collaborative situational display.

2. The underwater acoustic three-dimensional situational awareness system for complex underwater environments according to claim 1, characterized in that, The multi-source data fusion and 3D reconstruction module includes a spatiotemporal alignment unit, a feature extraction and matching unit, and a 3D scene construction unit; The spatiotemporal alignment unit is used to align the preprocessed multi-source data to the same time section according to the acquisition timestamp of each acquired data and the corresponding platform attitude information, and transform it to the global coordinate system. The feature extraction and matching unit is used to extract geometric feature points from sonar point cloud data in multi-source data, and when the system is equipped with an optical sensor, to extract texture features from synchronously acquired optical images and establish the correlation between heterogeneous features. The 3D scene construction unit is used to construct fused data based on multi-source data after spatiotemporal alignment and feature matching. An improved ray casting algorithm is used to adaptively sample the fused data at intervals. A continuous 3D field distribution is reconstructed through trilinear interpolation to generate a 3D situation scene containing target geometric features, motion state, and hydrological environmental elements. The 3D scene construction unit maps optical textures onto the 3D geometric surface reconstructed by sonar according to the correlation between heterogeneous features output by the feature extraction and matching unit, or optimizes the fusion weights of multi-source data based on the correlation between heterogeneous features to enhance the geometric accuracy and texture realism of the 3D situation scene.

3. The underwater acoustic three-dimensional situational awareness real-time visualization system for complex underwater environments according to claim 2, characterized in that, The improved ray casting algorithm involves voxel partitioning of the fused data, identifying invalid voxels and skipping resampling calculations within them; adaptively adjusting the sampling step size based on the acoustic intensity gradient, increasing sampling density in regions of rapid acoustic intensity change and increasing the sampling step size in flat regions; specifically including: A multi-level density occupancy grid is constructed, discretizing the continuous space into an L-layer voxel grid. The corresponding level is selected for skipping in the blank space based on the distance between the sampling point and the viewpoint. The formula for calculating the adaptive sampling step size is as follows: In the formula, Indicates the adaptive sampling step size. This represents the adaptive sampling step size factor. This indicates the pre-defined base sampling step size; In the formula, and These represent the maximum and minimum sampling step size factors, respectively. This indicates the preset adjustment coefficient. Represents the sound intensity gradient The modulus, This represents the maximum sound intensity value in the current scene.

4. The underwater acoustic three-dimensional situational awareness real-time visualization system for complex underwater environments according to claim 1, characterized in that, The human-computer interaction and visualization module supports multi-level situational display, including a macro-level global view and a micro-level target detail view; it also adopts a task-oriented interface design, automatically adjusting the interface layout and functional control priority according to the current task stage; and it supports multi-modal interaction methods such as voice, gesture and touch.

5. A real-time underwater acoustic three-dimensional situational awareness system for complex underwater environments according to claim 1, characterized in that, It also includes an intelligent decision support module; wherein the intelligent decision support module includes: The target automatic recognition unit uses a multi-scale convolutional neural network to detect and classify targets in a three-dimensional situational scene, and combines the characteristics of the target's motion state and historical trajectory to evaluate its identity confidence. The threat assessment unit calculates the threat level of a target and generates a threat situation map based on the target type, target motion parameters, distance information, hydrological cover conditions, and confidence assessment results. The target type is obtained from the classification results of the automatic target identification unit, the target motion parameters, distance information, and hydrological cover conditions are extracted from the three-dimensional situation scene, and the confidence assessment results are obtained from the identity confidence assessment of the automatic target identification unit. The decision-making simulation unit, based on the threat situation map and a pre-set decision rule base, predicts the future movement trend of the target and recommends corresponding decisions.

6. The underwater acoustic three-dimensional situational awareness real-time visualization system for complex underwater environments according to claim 5, wherein the multi-scale convolutional neural network of the target automatic identification unit is a joint neural network based on three-dimensional fusion features; in, The three-dimensional fusion features include: spatial features of the target region extracted by a three-dimensional convolutional neural network, spectral features obtained by fusing Mel frequency scale features and Mel frequency cepstral coefficients extracted by a two-dimensional convolutional neural network, and motion features composed of target motion parameters. The three features are concatenated and then input into a fully connected network for classification and recognition.

7. A real-time underwater acoustic three-dimensional situational awareness system for complex underwater environments according to claim 5, wherein the threat assessment unit employs an assessment algorithm based on multi-criteria fuzzy comprehensive evaluation, fuzzifying the target type, target motion parameters, distance information, hydrological concealment conditions, and confidence assessment results into three fuzzy sets: low, medium, and high. A comprehensive threat value is obtained through Mamdani fuzzy inference and the centroid method for defuzzification, and mapped to a discrete threat level; wherein, The membership function in the fuzzy inference process adopts the trapezoidal function, and the calculation formula of the centroid method is as follows: In the formula, K represents the overall threat value, and y represents the possible range of values ​​for the threat value. This indicates a comprehensive fuzzy output.

8. The underwater acoustic three-dimensional situational awareness real-time visualization system for complex underwater environments according to claim 5, wherein the decision-making and inference unit uses an outlier-resistant Kalman filter algorithm to track the target when predicting the future movement trend of the target, and constructs a test statistic through a chi-square test. ,in Let S be the innovation vector. k Let T be the new information covariance; when T k Exceeding the set threshold Time determination of observation value T k If an observation is an outlier, the observation noise covariance will be adaptively adjusted or the observation will be skipped; the specific handling strategy is as follows: if Update according to standard Kalman filtering; if Then take , Perform a weighted update; if If the observation is not found, then skip that observation and only perform state prediction.

Images