A multimodal fusion acquisition, computing, and communication integrated underwater acoustic wired dual-mode communication system and method

Abstract

A multimodal fusion underwater acoustic wired dual-mode communication system and method, relating to the field of underwater electrical communication transmission technology, addresses problems such as poor signal transmission reliability, disconnect between acquisition, computation, and communication, and the contradiction between power consumption and real-time performance in complex underwater channel environments. The system adopts a dual-core heterogeneous architecture, including modules for data acquisition, signal synchronization, fusion computation, and communication. The communication module is the core collaborative unit, supporting wired and underwater acoustic dual-mode communication. It employs LDPC-OFDM low-error-rate modulation and error correction technology, combined with a redundant storage module to achieve data temporary storage and automatic retransmission in case of link interruption. Through scheduling by the dual-core heterogeneous main control unit, it achieves full-link collaboration between acquisition, computation, and communication, optimizing transmission link selection and data packaging processes, and improving the real-time performance and stability of underwater signal transmission. It achieves low-error-rate transmission in high-noise environments, adapting to complex underwater communication environments such as marine gas leak monitoring and deep-sea earthquake early warning.

Description

Technical Field

[0001] This invention relates to the field of underwater electrical communication transmission technology, and in particular to a multi-modal fusion acquisition, computing and communication integrated underwater acoustic wired dual-mode communication system and method. It optimizes signal transmission performance in complex underwater channel environments, improves the reliability, real-time performance and efficiency of data transmission, and is applicable to signal transmission fields that require high-precision real-time sensing, such as marine gas leak monitoring, earthquake and geological disaster early warning, and carbon sequestration safety detection. Background Technology

[0002] With the advancement of offshore oil and gas development, deep-sea engineering, and carbon capture and storage (CCS) projects, underwater monitoring tasks require the simultaneous acquisition and transmission of ground acoustic, underwater acoustic, and various sensor data, placing extremely high demands on the reliability, real-time performance, and coordination of electrical signal transmission in complex underwater channel environments. Due to the high noise levels and limited distance in underwater channels, traditional architectures struggle to balance real-time performance and reliability in communication-constrained scenarios. Existing multimodal signal acquisition, processing, and transmission technologies suffer from the following main shortcomings:

[0003] First, most systems are still based on single-mode or loosely coupled multi-sensor architecture. Even when simultaneously acquiring ground acoustic and underwater acoustic signals, they are often recorded separately in parallel channels. There is a lack of a deep fusion model based on a unified time reference and a unified feature space. This structure results in the lack of a unified transmission standard for the acquired data, significant timing misalignment, and excessive redundant data during transmission. This not only significantly increases the transmission load of underwater channels and easily causes transmission errors, but also makes it difficult to support the smooth implementation of cross-modal joint inversion algorithms such as time difference inversion and beam azimuth estimation. Ultimately, this affects the consistency of multimodal data transmission and the usability of the transmitted data.

[0004] Second, the acquisition end and the computing end are generally separated. The typical approach is to rely on the host computer, shore-based platform or cloud for centralized processing. This separated architecture requires long-distance transmission of a large amount of unfiltered raw data, resulting in a large amount of raw data transmission and a long response link. It is impossible to achieve rapid on-site judgment in scenarios with limited communication or high timeliness requirements, and it is even more impossible to guarantee the real-time performance of data transmission. It is difficult to adapt to the core requirements of underwater scenarios for transmission timeliness.

[0005] Third, communication modules often exist as auxiliary functions, either using simple wired interfaces for data download after recovery or serving as uplinks in large-scale arrays. They are not designed in conjunction with the acquisition and computing stages. This disconnected design leads to rigid transmission link selection, making it difficult to form a closed-loop mechanism integrating "acquisition-computation-communication". The mismatch between transmission and acquisition / computation rhythms easily leads to problems such as link interruptions and data loss, seriously affecting the integrity and reliability of transmission.

[0006] Fourth, under long-term deep-sea operation conditions, relying solely on high-performance processors significantly increases overall power consumption. Traditional low-power solutions struggle to handle complex edge computing tasks such as multimodal feature extraction, time-difference inversion, and beamforming, leading to a clear conflict between real-time performance and endurance. This conflict further impacts the transmission process: insufficient edge computing capabilities result in a large influx of invalid data into the transmission link, increasing the transmission burden; while power imbalance prevents the transmission module from operating stably for extended periods, hindering the continuity and stability of underwater transmission and failing to meet the demands of long-term unattended deep-sea operations.

[0007] Against this backdrop, the development of an integrated acquisition-computation-communication system that integrates multimodal signal acquisition, time synchronization, fusion computing, and communication transmission onto a single embedded platform, with electrical signal transmission optimization as the core design focus, strengthens the collaborative design between the communication transmission link and the acquisition and computing links, optimizes underwater signal transmission performance, improves channel adaptability, and adopts optimization technologies such as low bit error rate modulation error correction, link adaptive switching, and redundant retransmission, addresses the core shortcomings of existing technologies such as the disconnect between electrical signal transmission and acquisition, computing, high transmission bit error rate, easy data loss, and high transmission delay, becoming an important development direction in the field of multi-source signal processing and transmission technology. Summary of the Invention

[0008] The primary objective of this invention is to address the problems of modal separation, time asynchrony, computational lag, high power consumption, disconnect between transmission and acquisition / computation, and poor reliability of underwater transmission in existing multi-source signal acquisition and processing systems. This invention provides a multi-modal fusion acquisition, computation, and communication integrated underwater acoustic wired dual-mode communication system and method. By employing a dual-core heterogeneous architecture with different kernel architectures and functional divisions, multi-modal signal acquisition, time synchronization, fusion computation, and communication transmission are integrated into the same embedded platform, enhancing transmission adaptability and enabling fusion sensing and real-time computation of multi-source signals at the acquisition end.

[0009] The second objective of this invention is to provide a multimodal fusion-based integrated data signal processing method. Based on the aforementioned system hardware architecture, it performs synchronous acquisition, preprocessing, time difference estimation, beam azimuth estimation, and fusion positioning of ground acoustic signals and underwater acoustic signals. Combined with power consumption control and communication scheduling, it focuses on optimizing the adaptability, reliability, and real-time performance of the transmission link. It achieves closed-loop collaboration of acquisition, computation, and communication at the hardware, algorithm, and system layers, providing an efficient signal processing and transmission solution for underwater monitoring scenarios.

[0010] To achieve the above-mentioned objectives, the present invention provides the following technical solutions.

[0011] A multimodal fusion acquisition, computing, and communication integrated underwater acoustic wired dual-mode communication system adopts a dual-core heterogeneous collaborative architecture, including a dual-core heterogeneous main control unit, a ground acoustic signal acquisition module, a signal synchronization module, an acoustic transmission and reception module, a fusion computing module, a power management module, a communication module, and a host computer terminal; each module works together to realize the synchronous acquisition, real-time computing, fusion positioning, and transmission of ground acoustic and underwater acoustic multimodal signals;

[0012] The dual-core heterogeneous main control unit adopts a heterogeneous dual-core chip integrating a Cortex-M4 core and a Cortex-A7 core to uniformly complete data fusion, signal analysis, and communication control functions. It realizes task coordination and wake-up notification through a heterogeneous core communication interface. The Cortex-M4 core serves as the MCU core, connecting the monitoring channels of the acoustic emission and reception modules, and normally performs low-power signal monitoring and event triggering. The Cortex-A7 core serves as the MPU core, running the operating system, connecting the ground acoustic acquisition control unit, the fusion computing module, and the communication module, and performing multimodal data fusion computing and communication management. When a valid signal is detected, the MCU wakes up the MPU to execute the fusion computing task, completes the time difference calculation, azimuth estimation, and distance inversion, and transmits the results back to the host computer in real time through the communication module.

[0013] The ground acoustic signal acquisition module includes a three-component seismometer, a high-precision analog-to-digital converter, a ground acoustic acquisition control unit, and a local storage module, used to complete the acquisition, analog-to-digital conversion, data caching, and local storage of ground acoustic signals. The acoustic emission and reception module includes a multi-channel hydrophone array and at least one underwater acoustic transducer, used to realize the sampling and wake-up triggering of underwater acoustic signals. The ground acoustic signal acquisition module and the acoustic emission and reception module work together to complete the synchronous acquisition of multimodal data. The monitoring channel and working channel of the acoustic emission and reception module both adopt dedicated transmission links to ensure that the acquired underwater acoustic signals can be synchronously transmitted to the dual-core heterogeneous main control unit, avoiding data misalignment caused by transmission delay, and providing a guarantee for the synchronous transmission and fusion calculation of multimodal data.

[0014] The signal synchronization module includes a GPS / BeiDou timing unit, a PPS pulse calibration unit, and an RTC clock. The GPS / BeiDou timing unit is used to obtain an absolute time reference. The PPS pulse calibration unit calibrates the RTC clock based on the absolute time reference to establish a unified time reference. The signal synchronization module forms a two-level processing mechanism of "coarse synchronization + spatiotemporal alignment" through a timestamp interpolation correction algorithm to eliminate time deviations in ground acoustic and underwater acoustic multimodal data.

[0015] The fusion computing module runs on a Cortex-A7 core and is used to perform spatiotemporal alignment processing of multimodal data sequentially, including convolutional feature extraction, dynamic time warping, and attention mechanism, based on a unified time reference provided by the signal synchronization module. Then, target localization is achieved through time difference inversion and beam azimuth estimation.

[0016] The communication module supports wired communication and LDPC-OFDM underwater acoustic communication, establishes a connection with the host computer, and is used to monitor signal strength and bit error rate in real time and adaptively switch links. When the link is interrupted, data is cached through the redundant storage module and retransmission is triggered to ensure transmission integrity.

[0017] The power management module supplies power to each module of the system and achieves dynamic energy consumption control through a power state machine and a power switching switch.

[0018] The host computer interacts with the system through a communication module to complete system parameter configuration, real-time monitoring data reception, and visualization display.

[0019] The dual-core heterogeneous main control unit, ground acoustic acquisition and control unit, fusion computing module and communication module achieve closed-loop collaboration of acquisition-computation-communication through shared memory and heterogeneous core communication interface within the dual-core heterogeneous main control unit. The shared memory is used for rapid data interaction between modules, and the heterogeneous core communication interface is used for instruction and data transmission between the Cortex-M4 core and the Cortex-A7 core. The power management module works with the dual-core heterogeneous main control unit to achieve integrated collaborative scheduling of acquisition-computation-communication based on the event processing stage.

[0020] Furthermore, in the dual-core heterogeneous main control unit, the Cortex-M4 core is connected to the power management module through the control pin to maintain a low-power sampling mode; the Cortex-A7 core is connected to the working channel of the acoustic emission and reception module through the IIO interface, and pushes calculation results to the host computer and receives instructions from the host computer through the communication control interface of the communication module.

[0021] The Cortex-M4 core connects to the monitoring channel of the acoustic emission and reception module via the transducer analog input interface, acquiring one channel of coarse underwater acoustic data for coarse synchronization detection and event triggering. The Cortex-M4 core also interacts with the Cortex-A7 core through a heterogeneous communication interface. This interface employs a high-speed transmission protocol to ensure rapid transmission of trigger commands and monitoring data, providing core support for the coordination of acquisition and transmission. The Cortex-A7 core, a high-performance computing core running an embedded operating system, connects to the ground acoustic signal acquisition module, the fusion computing module, and the communication module. It receives ground acoustic and underwater acoustic signals from the ground acoustic signal acquisition module, performs fusion calculations and manages communication, and is responsible for scheduling the transmission of fusion calculation results. It also coordinates the communication module to complete data uploading and command reception, achieving synchronous coordination between computation and transmission. The heterogeneous communication interface, implemented based on the OpenAMP mechanism, is used to transmit the acquisition data buffer pointer, event trigger flag, and power control commands.

[0022] Furthermore, the ground acoustic signal acquisition module's ground acoustic acquisition control unit is connected to the Cortex-A7 core of the dual-core heterogeneous main control unit via a high-speed SPI interface, periodically pushing ground acoustic data blocks to the Cortex-A7 core of the dual-core heterogeneous main control unit; the ground acoustic acquisition control unit uses an STM32U575; the local storage module of the ground acoustic signal acquisition module is connected to the STM32U575 via an SDMMC interface to achieve long-term storage of ground acoustic data; the STM32U575 uses a ping-pong buffer mechanism with low-power DMA to process the sampled data to avoid data loss at high sampling rates. The ground acoustic signal acquisition module is used to realize high-precision analog-to-digital conversion of ground acoustic signals, sampling data buffering, timestamp appending and local storage, and interacts with the Cortex-A7 core via a high-speed SPI interface to avoid ground acoustic data loss at high sampling rates; each component is connected through corresponding interfaces; the ground acoustic analog signal output by the three-component seismometer is input to a ≥24-bit analog-to-digital converter after front-end conditioning, and the analog-to-digital converter is connected to the STM32U575 via a serial interface;

[0023] Furthermore, the acoustic emission and reception module includes a 3-channel hydrophone array and a 1-channel underwater acoustic transducer. Both the monitoring channel and the working channel of the acoustic emission and reception module adopt dedicated transmission links to ensure that the acquired underwater acoustic signals can be synchronously transmitted to the dual-core heterogeneous main control unit, avoiding data misalignment caused by transmission delay, and providing a guarantee for the synchronous transmission and fusion calculation of multimodal data. The monitoring channel of the acoustic emission and reception module outputs a single underwater acoustic monitoring signal through the transducer interface, and the working channel outputs three underwater acoustic working signals through the IIO interface to ensure the synchronization of underwater acoustic signal acquisition and transmission.

[0024] Furthermore, the signal synchronization module is used to establish a unified time reference and correct the time deviation of multimodal data. It is connected to the dual-core heterogeneous main control unit and the ground acoustic signal acquisition module, respectively. Combined with the timestamp interpolation correction algorithm, it eliminates the time deviation caused by sampling delay, ensures the spatiotemporal alignment of ground acoustic and underwater acoustic data, improves the effectiveness of transmitted data, and achieves a time synchronization accuracy of ≤1ms. The time reference maintenance mechanism of the signal synchronization module is as follows: when the system runs independently in the future, the unified time reference is maintained by the RTC clock, and the time is adjusted once every hour by the PPS pulse to ensure that the time synchronization accuracy always meets the requirement of ≤1ms during long-term operation.

[0025] Furthermore, the fusion computing module runs on the Cortex-A7 core of the dual-core heterogeneous main control unit and is connected to the communication module via a data bus. The fusion computing module is used to perform positioning algorithms on the synchronized ground acoustic and underwater acoustic data and output spatial coordinate results. Before performing time difference inversion and beam azimuth estimation, the fusion computing module first performs spatiotemporal alignment processing on the ground acoustic and underwater acoustic data. The calculation results of the fusion computing module are quickly transmitted to the communication module via a high-speed data bus, realizing seamless connection between calculation and transmission and shortening the result transmission delay.

[0026] Furthermore, the parameters of the fusion computing module are configurable, including the cross-correlation window length, beam scanning angle range, alignment time window length for spatiotemporal alignment processing, DTW path constraint bandwidth, and attention weighting parameters; the fusion computing module has a built-in array direction vector adaptive generation algorithm, supports 2-8 channel underwater acoustic array configurations, and DTW cumulative distance matrix. satisfy:

[0027]

[0028] in, Represents the i-th underwater acoustic feature vector With the j-th ground acoustic feature vector The characteristic distance between them Indicates in Minimum cumulative distance at, This represents the cumulative distance between the previous underwater acoustic feature and the current ground acoustic feature. This indicates the cumulative distance between the current underwater acoustic feature and the previous ground acoustic feature. This represents the cumulative distance between the previous underwater acoustic feature and the previous ground acoustic feature; the alignment path is obtained based on the minimum cumulative distance, and the aligned feature sequence is output;

[0029] Attention mechanism computation satisfies:

[0030]

[0031] in, Attention(·) represents the weighted output feature of the attention mechanism. These are the query matrix, key matrix, and value matrix, respectively. This indicates that the query matrix and the transpose of the key matrix are multiplied together, where T represents the transpose. Denotes the scaling factor, where Let K be the dimension of the key matrix; The softmax activation function is represented; the attention output is used to generate the weighted aligned ground acoustic signal or fused features, which serve as the input for subsequent time difference inversion and beam azimuth estimation; the scanning angle range (-180°~180°) of the beam azimuth estimation can be configured via upper-computer commands to adapt to different monitoring scenario requirements; the alignment time window length, DTW path constraint bandwidth, and attention weighting parameters of the spatiotemporal alignment processing can be configured via upper-computer commands to achieve coordinated control of transmission and computation.

[0032] Furthermore, the communication module supports both wired and underwater acoustic communication. Wired communication is used for high-speed data download and parameter configuration after equipment recovery, and the wired communication interface can use Ethernet to achieve data interaction with the host computer. Underwater acoustic communication is used for status reporting, command issuance, or low-speed data transmission under underwater operating conditions, and the underwater acoustic communication link can use low error rate modulation and error correction technologies such as LDPC-OFDM. A connection is established with the host computer through the underwater acoustic channel or the wired channel to ensure transmission reliability in complex underwater environments.

[0033] Furthermore, the power management module is electrically connected to each module of the system, switching power consumption states according to the "sleep monitoring-sampling-computation-communication-sleep monitoring" mode. The power management module can optimize the power consumption allocation of the communication module, dynamically adjusting the transmission power during the underwater transmission phase, balancing transmission performance and energy consumption control, and ensuring that the system can still stably achieve signal transmission function during long-term underwater operation. The power management module includes a system-level power state machine and a power switching switch. The system-level power state machine is connected to the Cortex-M4 core of the dual-core heterogeneous main control unit through a control interface, and is used to receive event trigger signals to switch power consumption modes. The power switching switch is connected to the dual-core heterogeneous main control unit, the ground acoustic signal acquisition module, and the communication module through control pins, dynamically switching the power supply path in a "sampling-computation-communication-sleep" cycle, thereby reducing energy consumption while ensuring transmission reliability.

[0034] Furthermore, the host computer is a surface-based or shore-based device and its supporting software, used to complete system parameter configuration, task issuance, data reception, and result display. The host computer and the communication module adopt a bidirectional transmission link, and parameter configuration instructions and task issuance instructions can be quickly transmitted to various modules of the system. The multimodal data and fused positioning results collected by the system can be transmitted to the host computer in real time for visualization and storage, realizing reliable bidirectional data transmission. The host computer is indirectly connected to the dual-core heterogeneous main control unit through the communication module, receiving real-time monitoring data and sending control instructions for sampling rate adjustment and algorithm mode switching, reducing the amount of data transmitted, improving transmission efficiency, and adapting to underwater low-bandwidth transmission scenarios.

[0035] Furthermore, the system also includes a redundant storage module, which includes an SD card and a NAND Flash chip, for local storage backup and pre-transmission cache backup of important event data. A real-time backup + periodic verification mechanism is used to achieve dual backup. The important event data includes raw signal data with energy exceeding the threshold, time difference inversion results, beam azimuth estimation results, and spatial positioning coordinates. CRC32 verification is used every hour to verify the data consistency of the two storage media.

[0036] Furthermore, the system also includes an extended sensing interface module, which is located on the Cortex-M4 core of the dual-core heterogeneous main control unit and connected to external extended sensors through the IO / IIC interface of the Cortex-M4 core. The acquired data is preprocessed by the Cortex-M4 core and then used in the system fusion calculation. The extended sensing interface module is connected to external extended sensors through the IO / IIC interface of the Cortex-M4. The external extended sensors include at least a temperature and humidity sensor, a water depth and pressure sensor, and an attitude sensor. The Cortex-M4 acquires data from the external extended sensors and sends it to the Cortex-A7 through a virtual channel to achieve synchronous processing and transmission with other data, thereby supporting the on-demand access of other modal sensors to achieve multi-physics signal acquisition.

[0037] A multimodal fusion acquisition, computing, and communication integrated underwater acoustic wired dual-mode communication method specifically includes the following steps:

[0038] Step S1: System Initialization and Clock Configuration: The power management module is started to complete the hardware power-on reset. The UTC time and PPS pulse signal are obtained through the GPS / BeiDou timing unit of the signal synchronization module. The RTC clock of the dual-core heterogeneous main control unit and the ground sound acquisition control unit is calibrated by the PPS pulse to establish a unified reference clock. The sampling frequency is configured and the heterogeneous communication interface and external storage buffer are initialized. The external storage buffer is a buffer area in the volatile memory of the dual-core heterogeneous main control unit, which is used to temporarily store multimodal data to be processed or transmitted, and serves as a data exchange buffer for heterogeneous communication.

[0039] Step S2: Multimodal synchronous acquisition and trigger monitoring: The Cortex-M4 core of the dual-core heterogeneous main control unit normally acquires the monitoring channel signals of the acoustic emission and reception modules to determine event triggers, and the ground acoustic signal acquisition module responds synchronously in standby mode; when the monitoring signal energy exceeds the threshold, a trigger flag and timestamp are generated to wake up the Cortex-A7 core of the dual-core heterogeneous main control unit and the ground acoustic signal acquisition module, and start multi-channel ground acoustic and underwater acoustic synchronous sampling;

[0040] Step S3: Signal Preprocessing and Buffer Management: The ground acoustic signal acquisition module performs bandpass filtering, detrending, and gain normalization on the ground acoustic signal and writes it into a circular buffer. The Cortex-A7 core performs in-band filtering and gain control on the underwater acoustic signal. The Cortex-A7 core packages the signal buffer address, data length, and timestamp into a structured frame and passes it to the fusion computing thread through the shared memory inside the dual-core heterogeneous main control unit. After receiving the structured frame, the fusion computing thread first performs coarse synchronization based on the timestamp, and then sequentially performs spatiotemporal alignment processing of convolutional feature extraction, dynamic time warping, and attention mechanism to obtain the aligned ground acoustic sequence and underwater acoustic sequence, providing input for subsequent time difference estimation.

[0041] Step S4: Time difference estimation and distance inversion: Calculate the cross-correlation function of the ground acoustic channel and the underwater acoustic monitoring channel signals, determine the time difference, and combine the ground acoustic propagation speed and the underwater acoustic propagation speed to calculate the horizontal distance from the event to the monitoring node;

[0042] Step S5: Beam azimuth estimation: Construct a covariance matrix from the multi-channel signals of the hydrophone array, use a beamformer to calculate the azimuth power spectrum, and determine the incident azimuth of the target event;

[0043] Step S6: Event spatial localization and result packaging: Combining the latitude and longitude of the monitoring node, the horizontal distance obtained in step S4, and the incident azimuth angle obtained in step S5, the geographic latitude and longitude coordinates of the target event are calculated through spherical geometry, and the coordinates, event intensity, and frequency band characteristics are packaged into a structured result frame.

[0044] Step S7: Integrated Coordinated Scheduling and Communication Transmission of Acquisition, Computation, and Communication: The power management module switches power consumption modes between sampling, computation, communication, and sleep states according to the event processing stage. During communication transmission, the link status is monitored in real time. When the link is interrupted, the structured result frame is stored in the redundant storage module. After the link is restored, it is automatically retransmitted. After the transmission is completed, the transmission status is fed back to the Cortex-A7 core. When the transmission is completed and there is no new trigger, the high-power module is turned off and the system returns to the standby monitoring state, realizing closed-loop coordination of acquisition, computation, and communication.

[0045] In step S2, the specific steps of the multimodal synchronous acquisition and trigger monitoring are as follows: The Cortex-M4 core, as the MCU core, operates in low-power normal mode, continuously acquiring discrete signals from a single underwater acoustic monitoring channel for noise background estimation and event trigger judgment; simultaneously, the ground acoustic signal acquisition module is in standby mode, maintaining a responsive state with the synchronous trigger line of the analog-to-digital converter; when the envelope energy or short-time energy of the trigger monitoring channel exceeds a preset threshold, the MCU generates an event trigger flag and records the trigger timestamp, waking up the Cortex-A7 core and the ground acoustic signal acquisition module through the heterogeneous core communication interface, enabling the analog-to-digital converter to start synchronous sampling of multi-channel ground acoustic and underwater acoustic signals.

[0046] In step S3, the specific steps of signal preprocessing and buffer management are as follows: the ground acoustic signal acquisition module performs bandpass filtering, detrending, and gain normalization on the acquired multi-channel ground acoustic discrete signals, and writes the preprocessed data into a circular buffer by channel; the dual-core heterogeneous main control unit performs bandpass filtering and gain control on the underwater acoustic multi-channel discrete signals of the same frequency band to ensure the comparability of ground acoustic signals and underwater acoustic signals in terms of frequency band and amplitude; after preprocessing, the MCU packages the buffer address, data length, and timestamp information of the two types of signals into a structured frame, and passes it to the fusion calculation thread at the MPU end through shared memory; after coarse synchronization, the fusion calculation thread at the MPU end performs feature-level spatiotemporal alignment on the two types of signals, and then proceeds to time difference estimation and beam azimuth estimation.

[0047] In step S7, the specific steps of the integrated collaborative scheduling and communication transmission of acquisition-computation-communication can be as follows: During the communication transmission process, the power management module switches the power supply path between four states of "sampling, computation, communication, and sleep" according to the needs of the event processing stage: before triggering, the MCU core maintains a low-power monitoring mode; during the execution of steps S4 to S6, the MPU and high-performance memory are turned on; after the results are packaged, the LDPC-OFDM underwater acoustic communication interface or wired communication interface of the communication module is started to send the structured result frame to the host computer; after the transmission is completed and it is confirmed that there is no new trigger, the high-power module is turned off, so that the system returns to the standby monitoring state dominated by the MCU, realizing the closed-loop collaboration of acquisition, computation, and communication.

[0048] Furthermore, in near-shore scenarios (distance from shore base ≤ 500m), wired communication interface transmission is initiated, while in deep-sea scenarios (distance from shore base > 500m), LDPC-OFDM underwater acoustic communication submodule transmission is initiated. After transmission is completed and no new triggers are confirmed, the high-power module is shut down, and the system returns to the MCU-controlled standby monitoring state.

[0049] The working principle of this invention is as follows:

[0050] During the "acquisition" phase, when the system is in monitoring mode, the Cortex-M4 core, acting as the MCU core, maintains low-power monitoring, detecting changes in ambient sound energy and characteristics in real time through one underwater acoustic monitoring channel. The MCU uses a low sampling rate (100Hz–1000Hz, preferably around 1000Hz) and simple features (such as energy thresholds, envelope changes, or short-term correlations) to determine the presence of potential events. Once a threshold is exceeded or a set pattern is met, a valid event is considered to exist. At this time, the MCU sends an interrupt signal to the Cortex-A7 core via the OpenAMP heterogeneous core communication interface, and simultaneously notifies the ground acoustic signal acquisition module to lock the ground acoustic data buffer within the current time window.

[0051] After being woken up, the MPU first completes state recovery and power-on of peripherals, then starts the local ADC channel to acquire data from the three underwater acoustic working channels. Simultaneously, it reads the three-channel ground acoustic sampling data corresponding to the time period of the event from the ground acoustic signal acquisition module via the serial bus, and maps both types of data to the same timestamp coordinate system. Relying on prior GPS timing and PPS calibration, the system can ensure coarse alignment of the underwater acoustic and ground acoustic signals in terms of sampling time. Subsequently, the MPU performs spatiotemporal alignment processing on the underwater acoustic and ground acoustic data in the fusion calculation thread. The aligned signal segments are used for subsequent time difference inversion and beam azimuth estimation.

[0052] In the calculation phase, the MPU preprocesses the received multimodal data, including DC removal, bandpass filtering, normalization, and sliding window framing. Then, it sequentially performs time difference inversion, beam azimuth estimation, and fusion positioning. In the time difference inversion phase, the arrival time difference between underwater acoustic and ground acoustic signals is obtained using band-limited cross-correlation and phase difference estimation algorithms, and the distance between the event and the observation point is calculated by combining the propagation speeds of the two media. In the beam azimuth estimation phase, the MVDR beamforming algorithm is used to perform angle scanning on the multi-channel underwater acoustic signals to obtain the incident azimuth angle. In the fusion decision phase, based on the distance and azimuth, the spatial coordinates of the event are solved using spherical geometry, combined with the latitude and longitude of the monitoring nodes provided by BeiDou or inertial navigation. Both the time difference inversion and beam azimuth estimation are performed based on the aforementioned spatiotemporally aligned signal segments to reduce the impact of alignment errors on distance and azimuth calculations.

[0053] During the "communication" phase, the MPU packages the calculated distance, azimuth, and spatial location results into structured data frames, writes them to the communication buffer, and, depending on the current working environment, the communication management module selects either an LDPC-OFDM underwater acoustic link or an Ethernet link for uplink transmission. After receiving the results, the host computer can display the ground and underwater acoustic waveforms, time difference curves, beam power spectra, and event location information in real time on the visualization interface, while also recording information such as equipment operating status and remaining battery power.

[0054] Throughout the process, the host computer can also send control commands via the downlink, such as adjusting the sampling rate, modifying the filter bandwidth and trigger threshold, switching algorithm modes, or upgrading firmware. After receiving and parsing the control commands, the MPU forwards the acquisition-related parameters to the MCU and the ground acoustic signal acquisition module, thereby realizing the reverse scheduling from "connection" to "acquisition" and "computation," forming a complete closed-loop working principle.

[0055] Compared with the prior art, the present invention has the following significant advantages and technical effects:

[0056] 1. Multimodal Fusion Positioning and Integrated Collaboration: A single platform enables synchronous acquisition, fusion calculation, and reliable transmission of ground acoustic and underwater acoustic signals. Three-dimensional positioning is achieved through time difference inversion and beam azimuth estimation. A three-level synchronization mechanism—GPS / BeiDou time synchronization, PPS pulse calibration, and timestamp interpolation correction—is employed, achieving a multimodal time synchronization accuracy of ≤1ms. In typical seabed environments, millisecond-level time difference resolution and 0.1° azimuth accuracy can be achieved. The beam scanning range is -180° to 180° with a step size of 0.1°. The covariance matrix is ​​calculated using a 1024-point time average. Combined with LDPC-OFDM modulation error correction, the underwater communication distance is ≥1 km, effectively overcoming the limitations of single-mode signals.

[0057] 2. Integrated low-power acquisition, computing, and communication: Optimized power allocation relies on the division of labor and collaboration of dual-core heterogeneous main control units. The MCU core is only responsible for one channel of underwater acoustic monitoring and wake-up, which greatly reduces energy consumption. The MPU core and the ground acoustic acquisition module work together to complete the fusion calculation. The overall energy consumption is reduced by about 50% compared with the traditional architecture, and it can operate stably in the deep sea for a long time. Through a unified state machine and data flow control, the closed-loop collaboration of the entire acquisition, computing, and communication chain is realized, which takes into account both real-time performance and long-term stability.

[0058] 3. Highly modular and scalable: The transmission link and interface adopt a standardized modular design to adapt to different transmission scenarios; it supports multi-channel parallel sampling and multi-modal sensor (temperature, pressure, etc.) access; the parameters of the fusion calculation module can be dynamically configured; it has a built-in array direction vector adaptive algorithm; it supports flexible adaptation of 2 to 8 channel underwater acoustic arrays to cover different monitoring needs.

[0059] 4. Stable communication and wide adaptability: Adopting LDPC-OFDM modulation error correction technology, the bit error rate is ≤3.3×10⁻⁶ under high noise environment. -4 Underwater communication distance ≥ 1 km; supports wired and underwater acoustic dual-mode communication switching, adaptable to all working conditions of underwater monitoring and land recovery, and improves transmission reliability.

[0060] 5. Reliable data storage and wide application: The addition of an SD card + NAND Flash redundant storage module, combined with real-time backup, periodic verification and low-power DMA ping-pong buffer technology, avoids data loss under high sampling rates and solves the problem of data loss due to transmission link interruption; it can be widely used in multiple scenarios such as marine gas leak monitoring and carbon sequestration safety assessment, providing key technical support. Attached Figure Description

[0061] Figure 1 This is a schematic diagram of the overall hardware structure of the system of the present invention.

[0062] Figure 2 This is a schematic diagram of the system architecture design of the present invention, which uses STM32MP157 as the main controller.

[0063] Figure 3 This is a schematic diagram of the system architecture of the ground acoustic signal acquisition module of the present invention.

[0064] Figure 4 This is a schematic diagram of the overall operation framework of the multimodal acquisition-computation-communication integrated system of the present invention.

[0065] Figure 5 This is a schematic diagram of the underwater acoustic-ground acoustic-navigation multi-mode data fusion decision model described in this invention.

[0066] Figure 6 This is a schematic diagram of the judgment logic flow of the MCU-side software part of the present invention.

[0067] Figure 7 This is a schematic diagram of the task scheduling and multi-mode data fusion detection process at the MPU end of the present invention.

[0068] Figure 8 This is a schematic diagram of the overall architecture and functional layers of the main control software of this invention.

[0069] Figure 9 This is a schematic diagram of the MPU initialization process of the present invention. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of this invention clearer, the following embodiments will be used in conjunction with the accompanying drawings to further illustrate the invention. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. Aspects not described in detail can be addressed using conventional methods in the art.

[0071] The specific implementation of this invention targets typical high-risk underwater monitoring applications such as seabed gas leak monitoring and seabed seismic signal detection. The system, through hardware and software co-design, achieves synchronous acquisition, real-time calculation, and result uploading of multi-modal underwater acoustic and ground acoustic signals. It constructs an intelligent monitoring system integrating data acquisition, signal analysis, event judgment, and azimuth positioning, addressing issues such as the separation of acquisition and computation, signal transmission delay, and high energy consumption in traditional systems. This system is suitable for the long-term unattended signal transmission requirements in the deep sea. In this invention, the "MCU" refers to the Cortex-M4 processing unit running bare-metal programs within the dual-core heterogeneous main control unit; the "MPU" refers to the Cortex-A7 processing unit running OpenSTLinux within the dual-core heterogeneous main control unit.

[0072] I. System Overall Architecture

[0073] like Figure 1 As shown, the system adopts a dual-core heterogeneous collaborative architecture. The dual-core heterogeneous main control unit is an STM32MP157, integrating a Cortex-A7 core and a Cortex-M4 core. The Cortex-A7 core runs a Linux system and is used to perform data fusion, signal analysis, and communication management tasks on the Linux side. The Cortex-M4 core is used to perform low-power monitoring, event triggering, peripheral management, and wake-up control tasks. The Cortex-A7 core and the Cortex-M4 core communicate internally through a virtual channel, which is based on the OpenAMP mechanism to realize task collaboration and wake-up notification.

[0074] The ground acoustic signal acquisition module includes a three-component seismograph MTSS-2003, a ≥24-bit analog-to-digital converter ADS1283, an STM32U575, a GPS / BeiDou timing unit, and a local storage module (SD card). The analog signal output from the MTSS-2003 is differentially amplified and bandpass filtered before being input to the ADS1283 for analog-to-digital conversion. The ADS1283 connects to the STM32U575 via an SPI interface, and the STM32U575 performs sampling control, data buffering, and timestamp appending. The STM32U575 connects to the local storage module (SD card) via an SDMMC interface to write sampled data and record timestamps at preset intervals, enabling local storage and retrieval. The STM32U575 also connects to the STM32MP157 via a serial interface, which in one implementation includes UART or SPI, for sending cached data or feature fragments to the master control side to support real-time analysis and event processing.

[0075] The GPS / BeiDou timing unit outputs UTC time and PPS pulse to the STM32U575 via the UART interface. The STM32U575, in conjunction with the RTC clock, maintains a unified time reference, enabling it to attach a timestamp corresponding to the unified time reference when generating sampling data frames, thereby providing a basis for the spatiotemporal alignment of ground acoustic data and underwater acoustic data.

[0076] The acoustic emission and reception module includes a three-channel hydrophone array, a one-channel underwater acoustic transducer, and a corresponding sampling interface. The three-channel hydrophone array is connected to the Cortex-A7 core via an IIO interface for real-time sampling of underwater acoustic signals and subsequent azimuth and intensity-related analysis and calculations. The system also includes a one-channel underwater acoustic transducer, which is connected to the main control side via an IIO interface. The transducer is continuously monitored by the Cortex-M4 core, and upon detecting a valid acoustic signal, sends a wake-up interrupt to the Cortex-A7 via OpenAMP, thereby initiating the Linux-side processing flow and subsequent multimodal fusion calculation and communication tasks.

[0077] The system also includes other extended peripherals, which in one implementation include temperature and humidity sensors, water depth and pressure sensors, and attitude sensors. These other extended peripherals are connected to the Cortex-M4 core via IO / IIC interfaces to provide environmental and attitude assistance information and participate in event judgment and power consumption scheduling.

[0078] The host computer and the system communicate via a wireless link for parameter configuration and data reception. In one implementation, the wireless link can be a Wi-Fi link.

[0079] The power management module (battery pack + DC / DC + power supply switch) provides power to the ground acoustic signal acquisition module, the dual-core heterogeneous main control unit, and the acoustic emission and reception module. It also works with the Cortex-M4 core's standby strategy and the Cortex-A7 core's computing / communication tasks to switch between standby and working states, thereby reducing the average power consumption during long-term standby.

[0080] Through the aforementioned hardware architecture, the system completes multi-channel high-precision sampling of ground acoustic and underwater acoustic signals on the acquisition side, and achieves cross-modal spatiotemporal alignment with the help of time synchronization and timestamp mechanisms. At the same time, it completes local caching and storage recycling on the ground acoustic signal acquisition module side, thereby providing a data foundation for subsequent fusion computing and communication transmission processes.

[0081] Based on this overall structure, the following will combine Figure 2 The system architecture of MP157 and Figure 3 The architecture of the ground acoustic acquisition control unit (STM32U575) and the ground acoustic signal sampling link in the ground acoustic signal acquisition module are further explained.

[0082] like Figure 2 As shown, in the system architecture controlled by STM32MP157, the STM32MP157 internally includes a Cortex-A7 and a Cortex-M4, which communicate bidirectionally via a virtual channel. A 100Mbps wired network port is used to build the network environment and mount the system; this port connects to the STM32MP157 via a PHY driver and a TFTP / NFS environment. The Wi-Fi module includes USB Wi-Fi and SDIO Wi-Fi; the USB Wi-Fi is used for OTG services, and the SDIO... WIFI is used for network connectivity. The WIFI module is connected to the STM32MP157 via a WIFI driver. The STM32U575 is connected to the STM32MP157 via a serial port driver. The attitude sensor MPU6050, temperature and humidity sensor DHT11, and water depth and pressure sensor M10 are connected to the STM32MP157 via a serial port driver. The STM32MP157 is connected to the LED beacon unhooking and leveling motor via a PWM driver. The STM32MP157 is connected to the acoustic emission and reception module via an IIO driver and underwater acoustic transceiver control I / O port. The acoustic emission and reception module includes an ADC device (3-channel hydrophone array) and a DAC device (1-channel underwater acoustic transducer).

[0083] like Figure 3As shown, the system architecture of the ground acoustic signal acquisition module consists of a three-component seismograph MTSS-2003, an ADS1283, an STM32U575, an RTC clock, a GPS / BeiDou timing unit, an SD card storage module, and a communication interface with the main controller MP157. The three-component seismograph MTSS-2003 is used to acquire ground vibration signals in three orthogonal directions. Its analog output is pre-conditioned and then input to the ADS1283. The ADS1283 is a 31-bit high-precision analog-to-digital converter chip. In this embodiment, a 2:1 differential input method is used to achieve high-precision sampling of the three-component ground acoustic signals. The sampling clock is provided by the low-power timer LPTIM inside the STM32U575, used for periodic acquisition at a sampling rate of approximately 1000Hz. The digital ground acoustic data acquired by the ADS1283 is transferred to the internal storage area of ​​the STM32U575 via the SPI1 interface in conjunction with the low-power DMA controller LPDMA. The STM32U575 operates in a background autonomous management mode, performing cache management and timestamp marking on the acquired data. The RTC clock continuously keeps time for approximately 136 years, and its current time is periodically written to the STM32U575 to mark each data block with an absolute timestamp. To support long-term offline storage, the STM32U575 connects to a 64GB SD card via the SDMMC interface, and manages it using the FatFs file system in software. This SD card stores the raw data and timestamps from the three-component seismograph MTSS-2003, and supports software switching and hot-swapping between the MCU and USB, facilitating data retrieval and analysis after experiments.

[0084] For timing and positioning, the STM32U575 connects to the GPS / BeiDou timing unit via the USART2 interface. The UTC time and latitude / longitude information output by this module are used to correct the RTC clock and provide station coordinates for subsequent spherical geometric positioning. The STM32U575 establishes a bidirectional communication link with the MP157 main controller via the LPUART1 interface and LPDMA (Low-Power Direct Memory Access). This link is used to send ground acoustic data summaries, data integrity markers, and status information to the main controller, and to receive configuration commands and wake-up control commands from the main controller. Figure 3 With the coordinated operation of the modules shown, the STM32U575 can complete high-precision acquisition, time calibration and local storage of three-component ground acoustic signals under low power consumption conditions, and maintain reliable communication with the host controller.

[0085] II. System Operation Flow

[0086] like Figure 4As shown, the overall operating framework of the multi-modal fusion-based underwater acoustic-wired dual-mode collaborative communication system of this invention consists of a ground acoustic signal acquisition module (STM32U575) and a dual-core heterogeneous main control unit (STM32MP157). Figure 4 The left side shows the ground acoustic signal acquisition and local storage process on the STM32U575 side. Figure 4 The right side shows the on-call and fusion processing flow of the STM32MP157. The two work together through a high-speed interface and a heterogeneous core communication interface to achieve closed-loop operation of data acquisition, computing and communication.

[0087] 1. On the STM32U575 side, the system enters "Initialization" from "Start," completing the clock source configuration, ADC configuration, and SPI configuration. Then, the "GPS Timing and Positioning" step is executed, acquiring absolute time and location coordinates through an external GPS module and writing the timing result to the on-chip RTC clock as a unified reference for subsequent data timestamps. After initialization, the STM32U575 enters the "LPTIM Cyclic Task" driven by the low-power timer LPTIM (Low-Power Timer): within each timing cycle, "SPI+LPDMA Read ADC Data" is triggered, using the SPI interface and the low-power direct memory access controller LPDMA to read the ground acoustic digital signals output by the ADS1283 in batches via DMA; simultaneously, the "Get RTC Clock Time" step is executed to read the current RTC clock value. Subsequently, in the "Data + Time Stored in Ping-Pong Buffer" step, the ground acoustic data for this cycle and the corresponding timestamp are written to a double-buffered ping-pong buffer to support parallel acquisition and transmission / storage.

[0088] Within the main "while loop," when the STM32U575 is not performing data transfer, the processor enters the low-power background automatic mode LPBAM via "entering LPBAM (STOP2) mode," and operates in STOP2 low-power standby mode, retaining only a few peripherals such as LPTIM and LPDMA to run, thus achieving low-power periodic sampling of the ground acoustic signal. The system... Figure 3 The "Buffer Full?" decision node determines the ping-pong buffer status: if the buffer is not full, it continues in LPBAM (STOP2) mode; if the buffer is full, it exits the low-power state, stores the data in the SD card via "SDMMC+FatFs", and then selects "LPUART+LPDMA transmission" as needed. Specifically, the SDMMC interface, in conjunction with the FatFs file system, writes data blocks to the SD card for local storage; LPUART, combined with LPDMA, sends the buffered data to the main controller. After data storage or transmission is complete, the system returns to the while loop and re-enters LPBAM (STOP2) mode.

[0089] 2. On the STM32MP157 side, Figure 4 The right half describes the main control unit's duty and fusion processing flow. After the system powers on, the MCU core enters the "MCU duty, repeat process" state, during which it continuously "receives transducer underwater acoustic signals" and monitors one underwater acoustic monitoring channel in real time. In the "coarse synchronization detection" step, the MCU core performs coarse synchronization judgment on the monitoring channel: if no abnormality is detected, it follows the "no wake-up signal" path and continues to enter the "MCU duty, repeat process" state; if a wake-up signal is detected, it determines that there is a suspicious event, generates a wake-up request, and executes "wake up MPU" through the heterogeneous core communication interface, switching the STM32MP157's Cortex-A7 core from sleep mode to working mode. After the MPU is woken up, it executes the "acquire ground acoustic data" step, reading ground acoustic data from the ping-pong buffer of the STM32U575 through the high-speed interface; and simultaneously executes the "acquire three-channel hydrophone data" step, reading underwater acoustic array data from the three-channel hydrophone acquisition channels; after completing synchronous acquisition, the system enters the "multi-mode data fusion detection target" step, completing detection and positioning according to the algorithm. When it is necessary to actively report to the deck aircraft or the host system, the system uses the DAC output modulated signal to drive the underwater acoustic transducer in the "DAC output + transducer transmission to deck aircraft" step to transmit event information or control signals to the deck aircraft. After the fusion calculation and data transmission are completed, the MPU shuts down the high-power peripherals and re-enters sleep mode through the "MPU sleep, MCU resumes duty" step, and the MCU core continues to execute "MCU on duty, repeat process".

[0090] III. Core Algorithm and Fusion Mechanism

[0091] This invention employs a unified hardware clock reference, ensuring the consistency of sampling time between ground acoustic signals and underwater acoustic signals through synchronous triggering and timestamp correction mechanisms. Before equipment deployment, the system uses an external GPS module and PPS pulses to uniformly synchronize the local clocks of the STM32U575 and STM32MP157, and then their respective RTC clocks maintain the time reference. During the acquisition and buffering process, both ground acoustic and underwater acoustic data frames are accompanied by absolute timestamps, achieving a one-to-one correspondence between cross-modal sampling data.

[0092] The ground acoustic signal and underwater acoustic signal are respectively bandpass filtered and feature enhanced before being input into the fusion calculation module for joint analysis. Furthermore, the feature enhancement includes energy normalization and sliding window framing, providing a unified time window for subsequent spatiotemporal alignment.

[0093] The fusion computing module performs time difference inversion, beam azimuth estimation, and fusion positioning calculations based on time synchronization: the system estimates the event distance based on the time difference of arrival of underwater acoustic / ground acoustic signals, obtains the azimuth angle by scanning the multi-channel underwater acoustic signals based on MVDR beamforming, and solves the spatial location of the event through spherical geometry by combining BeiDou positioning or inertial navigation data, thereby realizing the latitude and longitude positioning of the gas leak point or seismic source. Furthermore, before performing the time difference inversion and beam azimuth estimation, the fusion computing module first performs spatiotemporal alignment processing on the underwater acoustic signals and the ground acoustic signals.

[0094] The spatiotemporal alignment process includes: performing convolutional neural network feature extraction on the underwater acoustic signal and the ground acoustic signal respectively to obtain a feature order of uniform dimension. and Then, Dynamic Time Warping (DTW) is used to calculate the optimal matching path for the two types of features, and the effective arrival segments of the two types of signals are aligned accordingly. The convolutional neural network (CNN) is used to extract local time-frequency features of underwater and ground acoustic signals. The underwater and ground acoustic signals are input into two CNN networks with identical structures but independent parameters. Each network receives a single-channel time-frequency map as input, with a map size of 100×256, corresponding to the number of time steps and frequency points. The network first passes through a first convolutional layer containing 32 3×3 filters, using a stride of 1 for convolution calculation, followed by the introduction of nonlinearity through the ReLU activation function. Next, it enters a first pooling layer, using 2×2 max pooling with a stride of 2 to reduce the feature dimension. The signal then enters a second convolutional layer containing 64 3×3 filters, also using a stride of 1 and the ReLU activation function. Finally, a global average pooling layer is used for feature aggregation, ultimately outputting a 128-dimensional feature vector. The output feature vectors of the two independent CNNs are used as input to the DTW. The DTW cumulative distance matrix is ​​then used. satisfy:

[0095]

[0096] in, Represents the i-th underwater acoustic feature vector With the j-th ground acoustic feature vector The characteristic distance between them Indicates in Minimum cumulative distance at, This represents the cumulative distance between the previous underwater acoustic feature and the current ground acoustic feature. This indicates the cumulative distance between the current underwater acoustic feature and the previous ground acoustic feature. This represents the cumulative distance between the previous underwater acoustic feature and the previous ground acoustic feature; the alignment path is obtained based on the minimum cumulative distance, and the aligned feature sequence is output.

[0097] The spatiotemporal alignment process introduces an attention mechanism based on the DTW alignment result, performing weighted fusion of key aligned segments, the calculation of which satisfies:

[0098]

[0099] in, Attention (·) represents the weighted output feature of the attention mechanism. These are the query matrix, key matrix, and value matrix, respectively. This indicates that the query matrix and the transpose of the key matrix are multiplied together, where T represents the transpose. Denotes the scaling factor, where Let K be the dimension of the key matrix; This represents the softmax activation function; the attention output is used to generate a weighted aligned ground acoustic signal or fused features, which serve as input for subsequent time difference inversion and beam azimuth estimation.

[0100] The spatiotemporal alignment model M0 is used to further align underwater acoustic and ground acoustic signals based on timestamp alignment. M0 first extracts convolutional features from the two types of signals, then obtains the optimal matching path through DTW and completes nonlinear alignment, and finally weights the key aligned segments through an attention mechanism to output the aligned signal segments.

[0101] The specific steps for time difference estimation and distance inversion are as follows: calculate the cross-correlation function of the signals from the ground acoustic channel and the underwater acoustic monitoring channel, determine the time difference, and combine the ground acoustic propagation speed and the underwater acoustic propagation speed to calculate the horizontal distance from the event to the monitoring node;

[0102] The first of a certain ground acoustic channel The amplitude of the discrete ground acoustic signal at each sampling point is denoted as . , will the corresponding position The amplitude of the discrete underwater acoustic signal at each sampling point is denoted as . ,in Integer sample point index, The sampling frequency is (Unit: Hz); First, calculate the normalized cross-correlation function of the two signals. Where k is an integer lazy index; For time delay (in seconds). ; For different time delays The cross-correlation function values ​​of ground sound and underwater sound are obtained by searching. The optimal delay index is obtained by finding the maximum cross-correlation magnitude within the range. This allows us to obtain the time difference between the arrival times of ground sounds and water sounds. The speed of sound propagation on the ground is known. (Unit: m / s) and the speed of sound propagation in water (Unit: m / s) Calculate the horizontal distance from the event to the monitoring node based on the difference in propagation time of the same event in two types of media. (Unit: m): Using this distance parameter, the radial distance estimate of the target event relative to the monitoring node is obtained.

[0103] The specific steps for beam azimuth estimation are as follows: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] The discrete signal of the path channel at each sampling time Composition of complex vectors Construct the covariance matrix (scanning angle range -180°~180°, step size 0.1°, covariance matrix calculated using 1024-point time averaging):

[0104]

[0105] in, H represents the expectation operator, and H represents the conjugate transpose operation; This represents the complex sampling vector of the underwater acoustic array at the nth sampling time. , Denotes the complex field, M×1 represents It is an M-row, 1-column column vector, where M is the number of channels in the underwater acoustic array, each element corresponds to the complex signal value of one channel at that moment, and 1 indicates the number of columns in the complex sampling vector;

[0106] set up azimuth of incidence The array direction vector (in radians) is the weight vector of a minimum variance distortionless response beamformer. Corresponding to the incident azimuth angle of the scan The beam output power spectrum below is By traversing the incident azimuth angle within the preset azimuth search range. , find The incident azimuth angle that yields the maximum value , incident azimuth angle As the estimated incident azimuth of the target event; where, It is a normalization factor. H is the inverse of the covariance matrix R, and H represents the conjugate transpose operation.

[0107] The specific steps for event spatial localization and result packaging are as follows: Record the geographical latitude of the monitoring node as... (Unit: radians), the geographical longitude of the monitoring node is... (Unit: radians), the average radius of the Earth is (Unit: meters); Horizontal distance from the event to the monitoring node With incident azimuth angle Given the information, a local Earth surface approximation is used to increment the latitude of the target event. Longitude increment of the target event They are represented as follows:

[0108]

[0109]

[0110] The geographical latitude of the event location Geographical longitude of the event location They are respectively:

[0111]

[0112]

[0113] This yields the latitude and longitude coordinates of the target event in the global coordinate system, and the horizontal distance from the event to the monitoring node. azimuth of incidence Geographic latitude of the event location Geographical longitude of the event location The event intensity, frequency band characteristics, and other parameters are encapsulated into structured result frames and written into the transmission buffer.

[0114] This invention's underwater acoustic-ground acoustic-navigation multi-mode data fusion decision model is based on the Undersea Seismic Observation System Array (OBS array), and cascades the spatiotemporal alignment model, beamforming model M1, time difference inversion model M2, and spherical geometric positioning model G. For example... Figure 5 As shown, after receiving the signal, the OBS array first inputs a beamforming model M1. The beamforming model M1 uses the MVDR beamforming algorithm to perform spatial filtering and directional scanning on the multi-channel underwater acoustic data, and outputs the signal source azimuth angle Y1. Y1 represents the incident azimuth angle of the signal source relative to the OBS array coordinate system, and its unit is degrees or radians.

[0115] Simultaneously, the system utilizes the underwater / underwater acoustic arrival time difference signals obtained in the aforementioned time difference inversion stage and inputs them into the time difference inversion model M2. Based on the arrival time differences of underwater and ground acoustic signals and the propagation speeds of the two media, the time difference inversion model M2 calculates the propagation distance from the event to the monitoring node and outputs the signal source distance Y2. Y2 represents the distance from the target event to the OBS node, in meters. Y1 and Y2 together characterize the target's azimuth and distance information in polar coordinates.

[0116] In addition, the system also obtains OBS latitude and longitude data (provided by the Beidou module) from the navigation module. This OBS latitude and longitude data, along with the azimuth angle Y1 and distance Y2 of the signal source, are input into the spherical geometric positioning model G. Based on spherical geometric relationships or the spherical cosine theorem, the spherical geometric positioning model G converts the station's latitude and longitude + distance Y2 + azimuth angle Y1 into the spatial coordinates of the target point, ultimately obtaining the three-dimensional positioning result of the target point's spatial coordinates. This three-dimensional positioning result can be represented by the target's latitude, longitude, and depth, or by local three-dimensional rectangular coordinates, providing input for subsequent event interpretation and visualization.

[0117] IV. Integrated Collaborative Circulation Mechanism for Acquisition, Computation, and Communication

[0118] To achieve efficient fusion and real-time transmission of multimodal signals, this invention constructs an integrated data flow mechanism of acquisition, calculation, and communication at the hardware, data, and control layers, enabling acquisition triggering, fusion calculation, and result feedback to operate collaboratively under the same closed-loop logic.

[0119] In the hardware flow path, the ground acoustic signal is sampled and buffered by the ADS1283 controlled by the STM32U575. After the trigger condition is met, it is sent to the MPU of the STM32MP157 through the high-speed interface. The three underwater acoustic working channels are synchronously sampled by the IIO interface on the MPU side for subsequent time difference inversion and beam azimuth estimation.

[0120] In terms of control flow, the MCU core inside the STM32MP157 normally operates in low-power mode and monitors the monitoring channel. When a valid event is detected, the MCU sends a wake-up and trigger command to the MPU and requests the STM32U575 to lock the data buffer of the corresponding time window, thereby completing the triggering and data alignment of the "acquisition → computation". The data alignment of the "acquisition → computation" includes two levels of processing: coarse timestamp synchronization and spatiotemporal alignment. The spatiotemporal alignment processing is completed in the fusion computing thread on the MPU side.

[0121] Based on duty triggering, data alignment, and power consumption scheduling, the system forms a closed-loop operating logic between data acquisition, fusion computing, and result feedback, with the corresponding data and control flows... Figure 4 , Figure 6 and Figure 7 The details are given separately.

[0122] V. Low power consumption design

[0123] This invention supports multi-level energy-saving strategies for power consumption management. When the system does not detect a valid signal, the MPU is in sleep mode, and only the MCU operates in low-frequency sampling mode. When an event signal is detected, the MCU immediately wakes up the MPU to collect data and execute the fusion algorithm. After the calculation is completed, the system automatically enters sleep mode. Through the above strategy, the system achieves a balance between real-time performance and energy efficiency, enabling it to operate autonomously for extended periods in outdoor or deep-sea environments. The power consumption control logic is uniformly managed by the power management module, dynamically switching the power supply path according to the "sampling-calculation-communication-sleep" work cycle to extend battery life and improve overall system stability.

[0124] 1) MCU shift procedure:

[0125] like Figure 6 As shown, after the MCU initializes, it enters the main loop and reads the latest underwater acoustic monitoring data from the ADC connected to the monitoring channel through the "read acquired data to DMA buffer" step, and writes it to the DMA buffer. Then, the MCU performs a "coarse synchronization check to see if a signal is detected" on the monitoring data in the buffer. Based on the energy threshold and coarse synchronization criteria, it determines whether there is a suspected event signal. If not, it is considered that no valid event has been detected, and the system continues to the next round of data acquisition and detection; if so, the MCU enters the event handling branch.

[0126] Upon detecting a valid signal, the MCU packages the event trigger flag and rough energy information through the "MCU virtual serial port transmission" step and sends it to the Linux system on the main control side via the virtual serial port interface. Simultaneously, the MCU executes the "send wake-up message to MPU" step, sending a wake-up request to the MPU via the OpenAMP heterogeneous communication interface or interrupt line, causing the MPU to switch from sleep mode to working mode. After the MPU completes the multimodal fusion calculation and provides the event level, the MCU confirms whether the node has received the warning information calculated by the MPU. If not, the MCU returns to the main loop to continue the monitoring task; if so, the MCU enters the reporting branch and, through the "send warning information to the host computer" step, packages the warning level, event time, and brief characteristic parameters calculated by the MPU and sends them to the host computer via the established communication link, achieving real-time warning of the target event.

[0127] To improve safety and reliability during long-term monitoring, the MCU software sets a higher-priority health monitoring branch in the main loop. Figure 6In this process, the branch is represented by "priority" and "check sensor data for abnormalities every 2 hours". The MCU records the running time through a timer and reads data from key sensors such as temperature, pressure, and attitude every 2 hours, comparing it with preset safety thresholds. If the result is "no" (no abnormality), it returns to the main loop and continues to execute DMA acquisition and coarse synchronization detection of the monitoring channel. If the result is "yes" (abnormality), it immediately enters the fault handling branch, executing the steps of "sending alarm information to the host computer and unhooking and floating, controlling the beacon to indicate the position". It sends alarm information to the host computer through the communication link, controls the release of the mechanical unhooking mechanism to release the observation node to the water surface, and simultaneously illuminates or drives the beacon to indicate the current position, making it easier for the working vessel to find and recover the equipment in time. The health monitoring branch has high priority. Once triggered, it will be executed before the ordinary event detection process, ensuring that the equipment can protect itself in time and report the fault status under abnormal operating conditions.

[0128] 2. MPU scheduling process:

[0129] like Figure 7 The diagram illustrates the task scheduling and sleep switching process of the MPU during this work cycle. The task scheduling of the MPU includes two parts: "MPU idle monitoring task" and "MPU multi-mode data fusion underwater target detection". Figure 7 The left image shows the MPU idle monitoring task, and the right image shows the workflow. Figure 7 The figure shows the workflow of the MPU multi-mode data fusion underwater target detection task, with the two being connected by an idle timer.

[0130] In the MPU idle monitoring task, the system initially enters a low-power sleep state. When the MCU wakes up the main controller via an interrupt signal, the MPU enters the "MPU interrupt wake-up" step. After waking up, the MPU executes the step of "reading underwater acoustic data collected by the three ADCs from the IIO buffer and reading ground acoustic data." It reads underwater acoustic data from the three hydrophone channels from the IIO buffer and simultaneously reads three ground acoustic data transmitted from the STM32U575 from shared memory. Then, it enters the "idle detection" step, which determines whether the MPU should continue operating based on whether there are pending fusion tasks, whether the host computer has issued new instructions, and the amount of data in the buffer.

[0131] If it is determined that there are still pending tasks or new fusion computing requests, the idle time of the MPU is accumulated through the "idle timer" step. If the idle timer has not yet reached the preset threshold, and the "idle timer expired?" judgment node returns "No", the MPU returns to continue executing the fusion detection process or waits for new data blocks. If no new trigger requests are received within a certain period of time, the idle timer reaches the preset threshold, and the "idle timer expired?" judgment node returns "Yes", then the MPU sends a notification to the MCU through the "Notify MCU: MPU will enter sleep" step, indicating that the current fusion task has been completed and there are no new tasks in the short term. After the notification is completed, the MPU executes the "MPU enters sleep" step, shuts down high-power peripherals, and returns to low-power standby state, with the MCU continuing to perform the on-duty monitoring task.

[0132] In the underwater target detection task using MPU multi-mode data fusion, after the MPU is woken up and detects valid event data or instructions from the host computer, it first simultaneously executes the steps of "reading 3 channels of underwater acoustic signals," "reading 3 channels of ground acoustic signals," and "reading sensor information." It reads underwater acoustic signals from the three hydrophone channels, reads the corresponding three channels of ground acoustic signals from the STM32U575 terminal, and simultaneously reads environmental sensor information such as temperature, pressure, and attitude. Then, it enters the "synchronization" step, performing spatiotemporal alignment and frame synchronization of the underwater acoustic and ground acoustic data under a unified clock reference. After synchronization, the MPU executes the "beamforming orientation" step, calling the aforementioned MVDR beamforming algorithm to perform directional scanning on the three-channel underwater acoustic array data and solve for the incident azimuth of the sound source. Next, in the "calculating the time difference between underwater and ground acoustic signals for ranging" step, the horizontal distance between the sound source and the monitoring node is calculated based on the time difference between the ground and underwater acoustic signals. Subsequently, in the "calculating the latitude and longitude of the underwater target" step, the longitude and latitude coordinates of the underwater target are solved based on a spherical geometric model, combining the latitude and longitude information of the monitoring node with the calculated distance and azimuth. Specifically, after reading the underwater and ground acoustic data, the MPU first performs spatiotemporal alignment processing, then calculates the time difference, azimuth, and solves for the latitude and longitude.

[0133] In the "Detection Complete?" judgment node, if the multi-mode data of the current event has been processed and there are no new data blocks to be fused, the judgment result is "Yes". The system then enters the "Store Calculation Results and Send Calculation Results to MCU" step, packaging the horizontal distance, azimuth, latitude and longitude, and related characteristic parameters from the event to the monitoring node into the result cache area, and sending it to the MCU and host computer through the heterogeneous communication interface; at the same time, it executes "Return to Idle Timer", returning to the MPU idle monitoring task flow on the left, waiting for new events or instructions. When the "Detection Complete?" judgment result is "No", it means that there are still data blocks or multiple targets to be processed. The "Idle Timer Cleared to 0" operation is executed, resetting the idle timer, and the MPU returns to the aforementioned data reading and fusion calculation steps until the entire detection task is completed.

[0134] VI. Software Architecture and Algorithm Implementation

[0135] The software portion of this invention employs a multi-threaded mechanism to ensure parallel execution of the system in the acquisition, calculation, and communication stages, thereby improving overall real-time performance. It includes a data acquisition thread, a signal processing thread, a power consumption control thread, a heterogeneous communication management thread, and a communication protocol stack thread.

[0136] like Figure 8 As shown in the embodiment of the present invention, the main control software of the integrated underwater acoustic-wired dual-mode collaborative communication system based on multimodal fusion adopts a bottom-up layered structure design, which includes a driver layer, an OS layer, an interface layer, and a service application layer. Data and control information are transmitted between the layers through clearly defined interfaces, enabling the coordinated operation of acoustic signal processing and duty services.

[0137] The driver layer includes a power management module, a memory management module, a first ADC driver module, an IIO driver module, a UART driver module, an IIC driver module, a second ADC driver module, a DAC driver module, and an RPMsg driver module. The power management module controls the power-on, power-off, and power mode switching of the main control chip and peripherals; the memory management module manages the allocation and reclamation of on-chip and external memory; the first ADC driver module drives the analog-to-digital converter (ADC) for ground acoustic acquisition; the second ADC driver module drives the ADC for underwater acoustic acquisition; the IIO driver module provides a multi-channel industrial I / O acquisition interface; the UART driver module is used for serial data transmission and reception; the IIC driver module configures and reads various sensor and peripheral registers; the DAC driver module controls the output calibration or control signals of the digital-to-analog converter; and the RPMsg driver module encapsulates and parses protocol message frames between the main control chip and external devices.

[0138] In the OS layer, the Cortex-A7 core of the main control chip runs the OpenSTLinux operating system, labeled "MPU OpenSTLinux" in the diagram; the Cortex-M4 core runs the bare-metal program, labeled "MCU Bare-Metal Program" in the diagram; the two exchange data and transmit commands through the "OpenAMP heterogeneous core communication framework". The OpenSTLinux side is mainly responsible for complex multi-threaded task scheduling, file system, and network protocol stack functions; the bare-metal program side is responsible for acquisition control and fast response tasks with higher real-time requirements. Through the OpenAMP heterogeneous core communication framework, acquisition data buffer pointers, event trigger flags, and power control instructions can be transmitted between the Cortex-A7 core and the Cortex-M4 core.

[0139] Within the interface layer, there is an "Interface Implementation for Signal Processing Computation Library" module, which provides a unified encapsulation of the underlying signal processing computation library. This module includes an "FIR Filtering" module, an "FFT / IFFT" module, and a "Matrix Operations" module. The "FIR Filtering" module provides interfaces for various finite impulse response filters for bandpass filtering and denoising of ground acoustic and underwater acoustic signals; the "FFT / IFFT" module provides interfaces for Fast Fourier Transform and Inverse Transform for spectrum analysis, channel characteristic estimation, and modulation / demodulation; and the "Matrix Operations" module provides vector and matrix multiplication and addition operations for implementing algorithms such as beamforming and covariance matrix solving.

[0140] In the service application layer, the left side contains the "Acoustic Signal Processing" module, which is used to complete the acquisition and processing of multimodal acoustic data. Specifically, the "Underwater Acoustic Signal Acquisition" module acquires raw underwater acoustic data from the hydrophone channel; the "Read Ground Acoustic Data" module reads ground acoustic data converted by ADS1283 from the ground acoustic acquisition link; the "Signal Synchronization" module performs spatiotemporal alignment and frame synchronization of ground acoustic and underwater acoustic data; the "Channel Coding and Decoding" module performs channel coding on the data to be transmitted and decoding on the received data; the "Modulation and Demodulation" module implements modulation and demodulation of the underwater acoustic communication link; and the "Multimodal Data Fusion Underwater Target Detection" module executes a multimodal fusion algorithm based on the above data to detect and identify underwater target events. In addition to timestamp alignment, the signal synchronization module also provides cross-modal data within the same event time window to the multimodal data fusion underwater target detection module and triggers spatiotemporal alignment processing using DTW and attention mechanisms.

[0141] The right side of the service application layer contains the "Duty Service" module, which maintains necessary monitoring and control during low-power duty mode. Specifically, the "Control Beacon" module controls the activation and deactivation of acoustic beacons when needed; the "Control Transceiver Transducer" module controls the switching of operating modes for the acoustic emission and reception modules; the "Signal Acquisition" module performs signal acquisition on the monitoring channels at a lower sampling rate during duty mode; the "Signal Transmission" module sends brief status information when an event is detected or a request is received from the host computer; the "Periodic Sensor Data Check" module periodically reads sensor data such as temperature and pressure and determines if the system is operating abnormally; the "Coarse Synchronization Detection Wake-up Signal" module generates a wake-up signal based on the coarse synchronization detection results of the monitoring channels; and the "Cross-Core Wake-up" module transmits the wake-up command to another core via a cross-core interface, completing cross-core wake-up.

[0142] The signal processing thread executes the following core algorithms: (1) a spatiotemporal alignment algorithm for feature-level alignment of ground acoustic and underwater acoustic signals, which includes convolutional feature extraction, DTW path matching and attention weighting; (2) a band-limited cross-correlation algorithm for estimating the time difference between ground acoustic and underwater acoustic signals; (3) a windowed fast Fourier transform (FFT) for extracting the spectral features and phase information of the signal; (4) a phase difference estimation algorithm for improving the accuracy of time difference calculation; (5) a MVDR beamforming algorithm for realizing the azimuth estimation of multi-channel underwater acoustic signals; and (6) a fusion positioning algorithm for calculating event coordinates by combining time difference, direction angle and external positioning data.

[0143] During algorithm execution, the system first performs buffer synchronization based on data frame timestamps, then combines DTW alignment path and attention weighting for residual alignment to ensure consistency of ground acoustic and underwater acoustic signals along both the time and feature axes. In signal arrival detection, the system employs energy thresholding and a sliding window strategy to improve response to weak signals and sudden events. In beam azimuth scanning, the system can automatically generate direction vector matrices based on different array structures, achieving universal adaptation to various underwater acoustic arrays.

[0144] The system supports online hot-switching and version updates of algorithms; it offloads some low-power tasks (such as coarse synchronization detection and energy monitoring) to the MCU for execution, while retaining high-intensity computational tasks at the MPU, thereby achieving optimal allocation of hardware and software resources. When the host computer issues new algorithm modules or parameter configurations, the system can complete hot-switching without interrupting operation.

[0145] like Figure 9As shown, after the MPU is powered on, it first completes "system initialization," followed by setting the "MPU wake-up source." The wake-up source can be from MCU interrupts, timer interrupts, or external communication interrupts. After the wake-up source is configured, the system enters "virtual serial port initialization," used for subsequent log and configuration command transmission. Next, it "loads the MCU firmware" to the specified storage area. Once the firmware is loaded and ready, the system "creates a task," and after the task is ready, it completes "operating system startup," entering the subsequent running phase.

[0146] VII. Communication Module and Host Computer

[0147] The communication module includes an LDPC-OFDM underwater acoustic communication submodule and an Ethernet communication submodule. The two are uniformly scheduled and coordinated through a communication management thread to ensure the stability of the communication link and efficient utilization of resources.

[0148] The link selection adopts a scenario-adaptive strategy: in deep water environments, the water LDPC-OFDM acoustic link is used first for data transmission, relying on its anti-multipath and low bit error rate characteristics to achieve long-distance data transmission; in laboratory debugging and near-shore testing environments, the Ethernet interface is switched to achieve high-speed data interaction to improve debugging efficiency and data download speed.

[0149] When the underwater acoustic link or Ethernet link is temporarily unavailable, the dual-core heterogeneous master control unit (MPU) writes the fusion calculation results and their corresponding timestamps and key characteristic parameters into the local SD card in the form of an event log for redundant storage. After the communication link is restored, the communication management thread triggers the historical data retransmission process, filters and retransmits the events that have not been uploaded according to the timestamp, thereby ensuring real-time transmission capability while avoiding the loss of important monitoring data.

[0150] The system can also output the processing results via Ethernet, serial port or underwater acoustic channel according to different application scenarios, realizing flexible data transmission configuration.

[0151] The host computer features full-process real-time data visualization and remote control capabilities: In terms of data visualization, it can display ground and underwater acoustic waveforms, time difference curves, beam scanning results, and spatial positioning coordinates in real time; in terms of parameter configuration, users can adjust system parameters such as sampling frequency, bandpass filter bandwidth, algorithm threshold, and power consumption mode through the interface; in terms of data management, the host computer integrates historical data playback, waveform scaling, and annotation functions, facilitating later analysis and comparative research. Through the downlink communication link, the host computer can remotely control system start / stop, parameter configuration, algorithm updates, and data recording strategies, realizing a closed-loop operation system for the entire process of acquisition, calculation, and communication.

[0152] VIII. System Performance and Application Effects

[0153] The system of this invention adopts a hardware and software collaborative solution, designing the acquisition, fusion computing and communication control as a closed-loop process, and achieving a comprehensive balance of low power consumption, high precision and high real-time performance through time synchronization link, power state machine and dual-mode communication link.

[0154] To verify the technical effects of this invention in terms of "synchronization accuracy, positioning accuracy, low power consumption performance, and communication reliability," this embodiment sets up four types of verification tests: the first is time synchronization and cross-modal spatiotemporal alignment test; the second is time difference estimation and azimuth / positioning accuracy test; the third is power consumption test and endurance calculation test; and the fourth is communication link reliability and data integrity test. The system hardware consists of a dual-core heterogeneous main control unit (STM32MP157, including Cortex-A7 and Cortex-M4 cores) and a ground acoustic signal acquisition module (STM32U575). The underwater acoustic side includes three hydrophone working channels and a single monitoring channel. The working channels are used for subsequent azimuth and time difference calculations via an IIO interface.

[0155] In the time synchronization and cross-modal spatiotemporal alignment test, the GPS / BeiDou timing unit outputs UTC time and PPS pulse to the STM32U575 via UART. The STM32U575, in conjunction with the RTC clock, maintains a unified time reference and adds a corresponding timestamp when generating the sampling data frame, which is used to provide a unified alignment basis for ground acoustic data and underwater acoustic data.

[0156] On the main control side, the system uses a "timestamp interpolation correction algorithm" to eliminate the time deviation caused by the difference in sampling links of multimodal data, forming a two-level alignment process of "coarse synchronization + spatiotemporal alignment", thereby supporting subsequent time difference inversion and fusion judgment.

[0157] The experimental verification index is based on the cross-modal time synchronization error. The synchronization capability given in the manual is a time synchronization accuracy of ≤1ms. This high-precision synchronization provides core support for the subsequent transmission process, ensuring that the ground acoustic and underwater acoustic fusion data are consistent in timing during transmission, avoiding data failure due to time misalignment, and thus ensuring that the subsequent positioning and discrimination algorithms can achieve accurate calculation based on the effective transmitted data. Based on this, the system can achieve millisecond-level time difference resolution and meet the time accuracy requirements of the subsequent positioning and discrimination algorithms.

[0158] In the orientation and positioning accuracy test, the system synchronously sampled three working channels on the underwater acoustic side and completed time difference estimation and orientation estimation in the fusion computing module. Time difference estimation used a normalized cross-correlation function to search for the optimal delay index to obtain the time difference, and combined with the difference between ground acoustic propagation speed and underwater acoustic propagation speed to invert the horizontal distance from the target event to the monitoring node, achieving a quantitative output of "time difference - distance". This quantitative result was encapsulated in a structured frame format by the communication module and efficiently transmitted to the host computer via an adaptive link. During transmission, relying on time synchronization characteristics, the real-time performance and accuracy of the positioning data were ensured, providing reliable transmission support for visualization and subsequent decision-making on the host computer.

[0159] Azimuth estimation is performed within the beamforming framework. The system supports 2- to 8-channel array configurations, and the beam scanning angle range (-180° to 180°) and cross-correlation window length can be configured by the host computer to adapt to different sea states and array scales. No on-site debugging is required to adapt to different sea states and array scales, reducing transmission link debugging costs. Simultaneously, configuration commands are sent through a low-error transmission link, ensuring accurate synchronization of configuration parameters and guaranteeing the consistency and reliability of azimuth estimation results.

[0160] Based on the configurable algorithm and parameters, the typical effect given in the manual is 0.1° azimuth accuracy, which can be used for scenarios such as seabed target azimuth indication, gas leak sound source pointing, and auxiliary determination of the arrival direction of seismic waves.

[0161] In the low-power performance test, the system uses a power state machine as its core, switching the power supply path cyclically from "sleep monitoring - sampling - calculation - communication - sleep monitoring". When no valid signal is detected, the MPU is in sleep mode, and only the MCU operates normally in low-frequency sampling mode. After an event is detected, the MCU wakes up the MPU to enter data acquisition and fusion calculation, and automatically returns to sleep mode after completion. This mechanism reduces the average power consumption during long-term operation. This low-power mechanism not only extends the system's battery life but also ensures the long-term stable operation of the transmission module, avoiding transmission link interruptions due to power imbalance, ensuring the continuous and stable transmission of fusion calculation results and monitoring data, and reducing the transmission failure rate.

[0162] Power consumption quantification is presented using the method of "normal power consumption of the control group vs. normal power consumption of this system": For example, taking the normal power consumption of a traditional single-core FPGA architecture of ≈1000mW as a control, the normal power consumption of this system under the low-power monitoring strategy is ≈500mW. Then the overall energy consumption reduction percentage is: (1000mW-500mW) / 1000mW×100%=50%. Therefore, under the same battery capacity, the battery life is approximately inversely proportional to the power consumption, which can achieve approximately doubling of the battery life and an overall power consumption reduction of approximately 50%. This can support the long-term uninterrupted operation of the transmission module, achieving continuous monitoring and stable data transmission for several months, and adapting to the transmission needs of unmanned deep-sea scenarios.

[0163] In communication reliability tests, the system supports dual-mode switching between Ethernet and underwater acoustic communication links. In near-shore scenarios, the Ethernet link is prioritized for high-speed data transmission, while in deep-sea scenarios, it switches to an LDPC-OFDM modulated underwater acoustic communication link. This modulation technology effectively resists underwater multipath interference and noise, achieving a bit error rate of ≤3.3×10⁻⁶ in high-noise environments through bit error correction. -4 With low bit error rate transmission, underwater communication distance can reach more than 1 kilometer, ensuring the adaptability and reliability of transmission links in different scenarios, and realizing efficient transmission of monitoring data and fusion results.

[0164] Regarding data integrity, the system can be configured with redundant storage modules (SD card + NAND Flash) to perform real-time dual writing of important event data, and adopts a "real-time backup + periodic verification mechanism" to ensure data consistency under long-term offline conditions and link interruption conditions. At the same time, the auxiliary control side uses a low-power DMA ping-pong buffer to avoid data loss under high sampling rates, thus combining communication reliability and storage reliability into an engineering effect of "results can be uploaded and original data can be traced". When the transmission link is restored, the system automatically triggers the redundant data retransmission mechanism to completely transmit the data temporarily stored during the link interruption to the host computer, completely solving the data loss problem caused by transmission interruption.

[0165] Some existing low-power OBS solutions can achieve low-power recording and storage with an overall power consumption of <0.3W. However, their processing side mainly completes sampling, filtering, storage and basic communication. They lack a transmission mechanism designed in conjunction with the acquisition and calculation links, and cannot achieve efficient transmission of "acquisition-computation-communication" closed-loop collaboration and cross-modal fusion data. This makes it difficult to meet the transmission requirements of high-precision real-time monitoring scenarios.

[0166] The above embodiments are merely preferred embodiments of the present invention and should not be considered as limiting the scope of the present invention. All equivalent variations and improvements made within the scope of the present invention should still fall within the patent coverage of the present invention.

Claims

1. A multimodal fusion acquisition, computing, and communication integrated underwater acoustic wired dual-mode communication system, characterized in that, It adopts a dual-core heterogeneous collaborative architecture, with closed-loop collaboration of acquisition, computing, and communication as the core design logic. It includes a dual-core heterogeneous main control unit, a ground acoustic signal acquisition module, an acoustic emission and reception module, a signal synchronization module, a fusion computing module, a power management module, a communication module, and a redundant storage module. Each module realizes data interaction and task collaboration through shared memory and heterogeneous communication interface, adapting to complex underwater communication environments. The dual-core heterogeneous main control unit adopts a heterogeneous dual-core chip integrating an MCU core and an MPU core. Through the heterogeneous core communication interface, the two cores achieve task coordination and wake-up notification, forming a hierarchical low-power scheduling logic: the MCU core normally runs in low-power mode, performing signal monitoring and event trigger judgment of the acoustic emission and reception modules; the MPU core runs the operating system, performing multimodal data fusion calculation and communication management; when the MCU core detects a valid signal, it wakes up the MPU core to perform the fusion calculation task. After the calculation is completed, the MPU core triggers the MCU core to resume the low-power monitoring mode. The ground acoustic signal acquisition module works in conjunction with the acoustic emission and reception module to achieve synchronous acquisition of ground acoustic and underwater acoustic multimodal data: the ground acoustic signal acquisition module completes the acquisition, analog-to-digital conversion and buffer storage of ground acoustic signals, and the acoustic emission and reception module realizes the sampling and wake-up triggering of underwater acoustic signals. Both modules are uniformly scheduled by the dual-core heterogeneous main control unit. The signal synchronization module includes a GPS / BeiDou timing unit, a PPS pulse calibration unit, and an RTC clock. The GPS / BeiDou timing unit acquires an absolute time reference, and the PPS pulse calibration unit calibrates the RTC clock based on the absolute time reference to establish a unified time reference. A two-level processing mechanism of coarse synchronization and spatiotemporal alignment is formed through a timestamp interpolation correction algorithm to eliminate the time deviation of ground acoustic and underwater acoustic multimodal data. The fusion computing module is electrically connected to the MPU kernel and runs on the basis of the MPU kernel. Based on the unified time reference provided by the signal synchronization module, it uses dynamic time warping combined with attention mechanism to perform spatiotemporal alignment processing on ground acoustic and underwater acoustic multimodal data, and then achieves target positioning through time difference inversion and beam azimuth estimation. The communication module, as the core unit for closed-loop coordination of acquisition, calculation, and communication, supports wired and underwater acoustic dual-mode communication. It adopts LDPC-OFDM low bit error rate modulation and error correction technology, monitors the link signal strength and bit error rate in real time, and adaptively switches the communication link. The communication module feeds back the real-time link status to the dual-core heterogeneous main control unit, which dynamically adjusts the acquisition frequency of the ground acoustic signal acquisition module, the sampling parameters of the acoustic emission and reception modules, and the calculation granularity of the fusion calculation module to achieve closed-loop coordination of acquisition, calculation, and communication. When the link is interrupted, the communication module stores the data to be transmitted in the redundant storage module for temporary storage, and automatically retransmits it according to the timestamp after the link is restored. The power management module supplies power to each module of the system. In cooperation with the dual-core heterogeneous main control unit, it dynamically switches the power supply path based on the event processing stage and the link status fed back by the communication module, so as to realize integrated energy consumption scheduling of data acquisition, computing and communication. The redundant storage module is used for temporary storage of data to be transmitted and backup of important event data when the link is interrupted, ensuring the integrity of data transmission.

2. The multimodal fusion acquisition, computing, and communication integrated underwater acoustic wired dual-mode communication system according to claim 1, characterized in that, The dual-core heterogeneous main control unit uses an STM32MP157 chip, where the MCU core is a Cortex-M4 core and the MPU core is a Cortex-A7 core. The Cortex-M4 core is connected to the power management module through a control pin to maintain a low-power sampling mode. It is also connected to the monitoring channel of the acoustic emission and reception module to collect single-channel underwater acoustic coarse sampling data for event triggering judgment and periodically checks sensor data. When an anomaly is detected, it triggers unhooking and buoyancy indication. The Cortex-A7 core is connected to the working channel of the acoustic emission and reception module through the IIO interface and interacts with the host computer through the communication module.

3. The multimodal fusion acquisition, computing, and communication integrated underwater acoustic wired dual-mode communication system according to claim 1, characterized in that, The ground acoustic signal acquisition module includes a three-component seismometer, a high-precision analog-to-digital converter of no less than 24 bits, a ground acoustic acquisition control unit, and a local storage module. The ground acoustic analog signal output by the three-component seismometer is input to the analog-to-digital converter after being differentially amplified and bandpass filtered at the front end. The analog-to-digital converter is connected to the ground acoustic acquisition and control unit through the SPI interface. The ground acoustic acquisition and control unit connects to the MPU core via the LPUART interface and a low-power LPDMA controller, periodically pushing ground acoustic data blocks to the MPU core, and connects to the local storage module via the SDMMC interface to achieve long-term storage of ground acoustic data; the ground acoustic acquisition and control unit uses a low-power DMA ping-pong buffer mechanism to process the sampling data to avoid data loss at high sampling rates. The acoustic transmitting and receiving module includes a multi-channel hydrophone array and at least one underwater acoustic transducer. The multi-channel hydrophone array is configured with three channels. The monitoring channel and the working channel of the acoustic transmitting and receiving module use a dedicated transmission link. The monitoring channel outputs a single underwater acoustic monitoring signal, and the working channel outputs three underwater acoustic working signals to ensure the synchronization of underwater acoustic signal acquisition and transmission. The underwater acoustic transducer is connected to the MCU core and is continuously monitored by the MCU core to trigger the wake-up mechanism.

4. The multimodal fusion acquisition, computing, and communication integrated underwater acoustic wired dual-mode communication system according to claim 1, characterized in that, The GPS / BeiDou timing unit of the signal synchronization module outputs UTC time and PPS pulses through the UART interface to obtain an absolute time reference; the PPS pulse calibration unit calibrates the RTC clock based on the absolute time reference to establish a unified time reference; the timestamp interpolation correction algorithm adopts a linear interpolation method, based on the absolute time reference of GPS / BeiDou timing, and calculates the corrected timestamp of the deviation data by interpolating adjacent valid timestamps, with a correction accuracy ≤0.1ms; when the system operates independently, the unified time reference is maintained by the RTC clock, and the system is calibrated once per hour by using PPS pulses.

5. The multimodal fusion acquisition, computing, and communication integrated underwater acoustic wired dual-mode communication system according to claim 1, characterized in that, The parameters of the fusion computing module are configurable, including the cross-correlation window length, beam scanning angle range, alignment time window length for spatiotemporal alignment processing, DTW path constraint bandwidth, and attention weighting parameters. The fusion computing module incorporates an array direction vector adaptive generation algorithm, supports 2-8 channel underwater acoustic array configurations, and DTW cumulative distance matrix. satisfy: in, Represents the i-th underwater acoustic feature vector With the j-th ground acoustic feature vector The characteristic distance between them Indicates in Minimum cumulative distance at, This represents the cumulative distance between the previous underwater acoustic feature and the current ground acoustic feature. This indicates the cumulative distance between the current underwater acoustic feature and the previous ground acoustic feature. This represents the cumulative distance between the previous underwater acoustic feature and the previous ground acoustic feature; the alignment path is obtained based on the minimum cumulative distance, and the aligned feature sequence is output; Attention mechanism computation satisfies: Where Attention(·) represents the weighted output feature of the attention mechanism. These are the query matrix, key matrix, and value matrix, respectively. This indicates that the query matrix and the transpose of the key matrix are multiplied together, where T represents the transpose. Denotes the scaling factor, where Let K be the dimension of the key matrix; This represents the softmax activation function; the attention output is used to generate a weighted aligned ground acoustic signal or fused features, which serve as input for subsequent time difference inversion and beam azimuth estimation.

6. A multimodal fusion acquisition, computing, and communication integrated underwater acoustic wired dual-mode communication method, characterized in that, The method, employing the system according to any one of claims 1-5, comprises the following steps: S1: System initialization and clock configuration: The power management module completes power-on reset, the signal synchronization module obtains the absolute time reference through GPS / BeiDou time synchronization, calibrates the RTC clock to establish a unified reference clock, configures sampling parameters and initializes the heterogeneous communication interface and storage buffer. S2: Multimodal synchronous acquisition and trigger monitoring: The MCU core normally monitors the monitoring channel signals of the acoustic emission and reception modules at low power, and performs event trigger judgment and system anomaly detection; the ground acoustic signal acquisition module is in standby and responds to synchronous trigger; when the monitoring signal energy exceeds the preset threshold, a trigger flag and timestamp are generated, the MPU core and the ground acoustic signal acquisition module are woken up, and multi-channel ground acoustic and underwater acoustic synchronous sampling is started; When a system anomaly is detected, fault self-protection and alarm are triggered; S3: Signal preprocessing and buffer management: The ground acoustic signal acquisition module performs filtering, detrending, and normalization processing on the ground acoustic signal and buffers it; The MPU kernel performs bandpass filtering and gain control on the underwater acoustic signal, packages the buffer address, data length and timestamp into a structured frame, performs coarse synchronization and spatiotemporal alignment processing, and obtains the aligned ground acoustic sequence and underwater acoustic sequence. S4: Time difference estimation and distance inversion: Calculate the cross-correlation function of the signals from the ground acoustic channel and the underwater acoustic monitoring channel to determine the time difference of arrival. Combine the ground acoustic propagation speed and the underwater acoustic propagation speed to invert the horizontal distance from the event to the monitoring node. S5: Beam azimuth estimation: Construct a covariance matrix for the multi-channel signals of the hydrophone array, and use a minimum variance distortionless response beamformer to scan within a preset angle range to determine the incident azimuth of the target event; S6: Event Spatial Location and Result Packaging: Combining the latitude and longitude, horizontal distance and incident azimuth of the monitoring node, the geographic coordinates of the target event are calculated through spherical geometry, and the coordinates, event intensity, frequency band characteristics and equipment status are packaged into a structured result frame; S7: Integrated collaborative scheduling and communication transmission of data acquisition, computing, and communication: The power management module dynamically switches power consumption modes according to the event processing stage; the communication module monitors the link status in real time. If the link is normal, it transmits data in real time. If the link is interrupted, the results are stored in the redundant storage module and automatically retransmitted after the link is restored. When the transmission is complete and there are no new triggers, the high-power module is turned off, and the system returns to the low-power standby monitoring state dominated by the MCU, realizing closed-loop coordination of data acquisition, calculation, and communication.

7. The multimodal fusion acquisition, computing, and communication integrated underwater acoustic wired dual-mode communication method according to claim 6, characterized in that, In step S2, the specific steps of the multimodal synchronous acquisition and trigger monitoring are as follows: The Cortex-M4 core, as the MCU core, operates in low-power normal mode, continuously acquiring discrete signals from a single underwater acoustic monitoring channel for noise background estimation and event trigger judgment; simultaneously, the ground acoustic signal acquisition module is in standby mode, maintaining a responsive state with the synchronous trigger line of the analog-to-digital converter; when the envelope energy or short-time energy of the trigger monitoring channel exceeds a preset threshold, the MCU generates an event trigger flag and records the trigger timestamp, waking up the Cortex-A7 core and the ground acoustic signal acquisition module through the heterogeneous core communication interface, enabling the analog-to-digital converter to start synchronous sampling of multi-channel ground acoustic and underwater acoustic signals.

8. The multimodal fusion acquisition, computing, and communication integrated underwater acoustic wired dual-mode communication method according to claim 6, characterized in that, In step S4, the specific steps of time difference estimation and distance inversion are as follows: The first... The amplitude of the discrete ground acoustic signal at each sampling point is denoted as . , will the corresponding position The amplitude of the discrete underwater acoustic signal at each sampling point is denoted as . ,in Integer sample point index, The sampling frequency is Calculate the normalized cross-correlation function of the two signals: Where k is an integer delayed index; For time delay, ; For different time delays The cross-correlation function values ​​of ground sound and underwater sound are obtained by searching. The optimal delay index is obtained by finding the maximum cross-correlation magnitude within the range. This allows us to obtain the time difference between the arrival times of ground sounds and water sounds. The speed of sound propagation on the ground is known. With the speed of sound propagation in water Based on the difference in propagation time of the same event in two types of media, the horizontal distance from the event to the monitoring node is calculated. The radial distance estimate of the target event relative to the monitoring node is obtained using this horizontal distance parameter.

9. A multimodal fusion acquisition, computing, and communication integrated underwater acoustic wired dual-mode communication method according to claim 6, characterized in that, In step S5, the specific steps for beam azimuth estimation are as follows: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] The discrete signal of the path channel at each sampling time Composition of complex vectors Construct the covariance matrix: ， in, H represents the expectation operator, and H represents the conjugate transpose operation; This represents the complex sampling vector of the underwater acoustic array at the nth sampling time. , Denotes the complex field, M×1 represents It is an M-row, 1-column column vector, where M is the number of channels in the underwater acoustic array, each element corresponds to the complex signal value of one channel at that moment, and 1 indicates the number of columns in the complex sampling vector; set up azimuth of incidence The array direction vector is obtained by using the weight vector of a minimum variance distortionless response beamformer. Corresponding to the incident azimuth angle of the scan The beam output power spectrum below is By traversing the incident azimuth angle within the preset azimuth search range. , find The incident azimuth angle that yields the maximum value , incident azimuth angle As the estimated incident azimuth of the target event; where, It is a normalization factor. H is the inverse of the covariance matrix R, and H represents the conjugate transpose operation.

10. A multimodal fusion acquisition, computing, and communication integrated underwater acoustic wired dual-mode communication method according to claim 6, characterized in that... In step S7, the specific steps of the integrated collaborative scheduling and communication transmission of acquisition, calculation, and communication are as follows: The power management module switches the power supply path between the four states of "sampling, calculation, communication, and sleep" according to the needs of the event processing stage: before triggering, the MCU core maintains the low-power monitoring mode; during the execution of steps S4 to S6, the MPU and high-performance memory are turned on; after the results are packaged, the communication management thread adaptively selects the Ethernet or LDPC-OFDM underwater acoustic link according to the scenario and sends the structured result frame to the host computer or shore-based server; After the transmission is completed and no new triggers are confirmed, the high-power module is turned off and the system returns to the standby monitoring state dominated by the MCU core. When the link is interrupted, the structured result frame is stored in the redundant storage module, and after the link is restored, it is filtered and retransmitted according to the timestamp.

Images