Sea area signal investigation method and device based on deep learning

Abstract

The invention belongs to the technical field of underwater acoustic communication reconnaissance and artificial intelligence edge computing, and discloses a sea area signal reconnaissance method and device based on deep learning. According to the method, a constructed lightweight LeNet-5 hierarchical convolutional neural network model is embedded and deployed into an STM32H743 microcontroller through an X-Cube-AI tool to process underwater acoustic signals, automatic identification and classified modulation are performed on underwater signals of active sonar signals and non-cooperative communication signals in real time, and the underwater signals are used for analysis of recovered sea area information. According to the method, deep-level features can be automatically learned from the time-frequency graph, better robustness is achieved for noise and a complex marine environment, and the recognition accuracy is higher.

Description

Technical Field

[0001] This invention belongs to the field of underwater acoustic communication reconnaissance and artificial intelligence edge computing technology, and particularly relates to a method and device for marine signal reconnaissance based on deep learning. Background Technology

[0002] With the increasing demands for marine resource development and maritime security, marine signal reconnaissance technology is playing an increasingly important role in national defense, marine resource exploration, and marine environmental monitoring. Marine signal reconnaissance primarily focuses on the acquisition, processing, and identification of underwater sonar and communication signals, providing technical support for marine monitoring and target identification. Traditional underwater signal identification methods mainly rely on algorithms based on feature extraction and pattern recognition, such as calculating signal characteristics like spectrum, zero-crossing rate, and instantaneous frequency, and combining these with classifiers like support vector machines or decision trees. While these methods may be effective in specific, stable environments, their recognition performance heavily depends on the completeness and accuracy of manually designed features. For non-cooperative signals with low signal-to-noise ratios in complex and variable marine environments, their generalization ability and robustness are often insufficient, leading to a significant decrease in recognition accuracy.

[0003] In recent years, deep learning, especially convolutional neural networks, has achieved great success in image and speech recognition. Its powerful end-to-end feature learning capabilities have provided new ideas for underwater acoustic signal recognition. Existing research has attempted to use the time-frequency diagrams of underwater acoustic signals (e.g., obtained through short-time Fourier transform) as input and utilize deep learning models for automatic classification. However, deploying large-scale deep learning models on embedded front-end devices (such as microcontrollers placed in buoys or submersibles) with extremely limited computing, storage, and power resources presents significant challenges. Existing general-purpose deep learning models typically have high computational complexity and a large number of parameters, making real-time inference difficult to achieve on embedded microcontrollers such as STM32. Sending data back to the cloud for processing fails to meet real-time requirements and increases system power consumption and communication complexity. Therefore, there is an urgent need for a method and device capable of long-term, real-time, and accurate automatic identification of underwater acoustic signals. This method should employ a hierarchical processing strategy for different signal types to improve computational efficiency and operate efficiently on resource-constrained embedded devices, providing immediate intelligence support for underwater non-cooperative target reconnaissance. Summary of the Invention

[0004] To overcome the problems existing in related technologies, the present invention discloses a method and apparatus for marine signal reconnaissance based on deep learning. To address the technical problems of existing underwater reconnaissance systems lacking self-contained hydrophones capable of long-term, real-time, and accurate automatic identification of underwater acoustic signals, failing to provide immediate intelligence support for underwater non-cooperative target reconnaissance, having low computational efficiency of traditional neural networks, and lacking robustness of traditional methods based on artificial features in noisy and complex marine environments, the present invention provides a method and apparatus for marine signal reconnaissance based on deep learning.

[0005] The technical solution is as follows: A deep learning-based marine signal reconnaissance method utilizes a lightweight LeNet-5 hierarchical convolutional neural network model, embedded into an STM32H743 microcontroller using the X-Cube-AI tool, to process underwater acoustic signals. This allows for real-time automatic identification, classification, and modulation of underwater signals, including both active sonar and non-cooperative communication signals, which are then used for post-recovery marine information analysis. Specifically, the method includes the following steps: S101, LA sub-network judgment: Based on the spectral concentration and feature saliency of the signal, the input signal is initially judged and divided into a first type of signal and a second type of signal; wherein, the first type of signal includes active sonar signals and synchronization signals with more salient features and easier to identify; the second type of signal is non-cooperative communication signal; S102, LA sub-network processing, identifies the first type of signal identified in the first step, and performs secondary discrimination through LD sub-network, including: first, based on the characteristic of the fixed modulation mode of active sonar signals, the active sonar signals are identified first; then, the identified synchronization signals are confirmed, and when the synchronization signals are confirmed, a feedback control command is generated and fed back to the STM32H743 microcontroller for continuous monitoring. S103, LC sub-network processing, performs modulation identification and classification on the second type of signal identified in the first step, namely the non-cooperative communication signal.

[0006] Furthermore, constructing a lightweight LeNet-5 hierarchical convolutional neural network model includes the following steps: (1) Use the simulation software MATLAB to generate five non-cooperative communication signals (CW, LFM, 2FSK, 4FSK, BPSK) and HFM synchronization signal to simulate the actual received signals; use data augmentation technology to expand the data set and obtain a data sample set. (2) Import more than a dozen active sonar frequency modulation signals from the currently known database; (3) The processed time-frequency map is received through the input layer and used as the input of the lightweight LeNet-5 hierarchical convolutional neural network; (4) Construct a lightweight LeNet-5 hierarchical convolutional neural network model architecture; use Python language to build the TensorFlow / Keras library based on the TensorFlow / Keras framework, define the LA sub-network as a binary classification model, and define the LB, LC, and LD sub-networks as multi-class classification models; implement the hierarchical architecture using custom conditional judgment functions; (5) Lightweight design of LA, LB, LC and LD subnetworks; (6) Through the SE attention module, the feature channels are converted into scalars by global average pooling to obtain the weight of each feature channel. After the scalar is processed by two fully connected layers, it is used as the input of the ReLU activation function and outputs a weight vector with the same number of channels, i.e. attention weights. The feature channel weights are adjusted during training. (7) Global average pooling is used to perform average pooling on each feature map to extract feature information describing the global features; (8) The extracted feature information is classified by the output layer Softmax function to obtain the probability distribution of five types of modulation signals. The category with the highest probability is selected as the recognition result after decision-making. (9) Training: For the lightweight LeNet-5 layered convolutional neural network model constructed by the host computer, the parameters of the lightweight LeNet-5 layered convolutional neural network model are trained by unsupervised training stage and fine-tuning stage, the conditional probability distribution of each layer is obtained, and the parameters of different sub-networks are adjusted to realize the adaptive adjustment of the parameters of the lightweight LeNet-5 layered convolutional neural network.

[0007] The data sample set obtained in step (1) specifically includes: Non-cooperative signal dataset: Five non-cooperative communication signals, namely CW, LFM, 2FSK, 4FSK and BPSK, were generated using the simulation software MATLAB to simulate the signals actually received. Active sonar signal dataset, utilizing more than a dozen active sonar frequency-modulated signals from a known database; Synchronization signal dataset: HFM signals are commonly used for synchronization signals in underwater acoustic communication. Therefore, multiple sets of HFM signals are generated using the simulation software MATLAB to simulate the actual received signals. The signal is subjected to a Short Time Fourier Transform (STFT) to generate a time-frequency plot. Then, SpecAugment data augmentation technology is used to expand the data by shifting the time dimension and masking the time-frequency dimension, thereby increasing the dataset size and obtaining a complete dataset sample set.

[0008] Furthermore, in step (4), the LeNet-5 hierarchical convolutional neural network model is lightweighted by: using depthwise separable convolutions in the convolutional layers to reduce the number of model parameters and computational complexity; using a global average pooling module instead of flattening to prevent overfitting; and using the ReLU function instead of the Sigmoid function as the activation function to introduce non-linear features and avoid gradient vanishing. Step (5) involves lightweighting the LA, LB, LC, and LD sub-networks by: using depthwise separable convolution to calculate multiple feature maps of the time-frequency map separately in a single channel, then using a 1*1 point convolution and performing operations on the channel to complete the convolution calculation, reducing the upper and lower dimensions of the feature map by two pixels each; after halving the feature map size through the second pooling layer, performing depthwise separable convolution again, and then halving the feature map size again through the fourth pooling layer.

[0009] Furthermore, the constructed lightweight LeNet-5 hierarchical convolutional neural network model is embedded and deployed into the STM32H743 microcontroller using the X-Cube-AI tool, including the following steps: (a) Using the X-Cube-AI tool deployed in STM32CubeMX, the trained lightweight LeNet-5 hierarchical convolutional neural network model was imported into the embedded microcontroller system to verify the model compatibility and resource requirements of the lightweight LeNet-5 hierarchical convolutional neural network model. (b) Using the code generation function of the X-Cube-AI tool, the lightweight LeNet-5 hierarchical convolutional neural network model was converted into pure C code; (c) The tool generates the core files network.c and network_data.c. Open both files in the Keil MDK development environment. (d) Open the converted project with Keil MDK, compile and modify the code, view the code information with the serial port debugging assistant, and burn the project to the STM32H743 microcontroller connected to the PC to complete the embedded deployment of the lightweight LeNet-5 hierarchical convolutional neural network model. (e) Return to CubeMX, click Validate on target, and run the verification program on the board. If it works, it has been successfully deployed on the STM32H743 microcontroller.

[0010] Another objective of this invention is to provide a deep learning-based marine signal reconnaissance device, comprising a core processor module, an AD signal acquisition module, an underwater filtering circuit module, a serial communication module, a lightweight marine signal processing module, a power management module, and an SD storage module; The AD signal acquisition module, underwater filtering circuit module, serial communication module, lightweight marine signal processing module, power management module, and SD storage module are all embedded in the core processor module, which uses a microcontroller based on STM32H743. Specifically, all modules are installed in the hydrophone enclosure; the AD signal acquisition module first acquires and converts the marine signals, then filters out noise through the underwater filter circuit module before inputting them into the lightweight marine signal processing module for processing and classification; finally, the results are input into the SD storage module; the serial communication module is used for communication between the STM32H743 microcontroller and each module, and the power management module is used for the normal power supply of each module during operation.

[0011] Furthermore, the core processor module is based on the Arm® Cortex®-M7 core and has a built-in FPU; it supports the deployment of lightweight LeNet-5 hierarchical convolutional neural network models through the X-Cube-AI tool; The STM32H743 microprocessor acquires underwater acoustic signals through its internal ADC acquisition module; filters out noise through an underwater filtering circuit module; and processes the underwater acoustic signals using a short-time Fourier transform to obtain a time-frequency diagram; specifically including: (i) The ADC acquisition module samples the signal using the ADC built into the STM32H743 microprocessor. The sampling rate is set to 96kHz, and dual buffering technology is used to obtain the real-time underwater acoustic signal input. (ii) Use an underwater filter circuit module to filter out low-frequency and high-frequency noise from the input signal to obtain a noise-removed underwater acoustic signal; (iii) Using CMSIS-DSP library functions, the energy decision statistic of the signal is calculated by taking the modulus of the received signal and calculating the sum of squares. The statistic is then compared with the decision threshold to determine whether the signal is valid. (iv) Normalize the amplitude characteristics of the detected signal to a standard range and then window the signal; (v) Use DSP library functions to perform Fast Fourier Transform (FFT) to obtain a complex array; (vi) Use DSP library functions to calculate the amplitude of the complex sequence, obtain the amplitude information at each frequency, and output the time-frequency diagram of the signal; (vii) After completing the STFT calculation of the signal, a one-dimensional array with a length of 512 is obtained. The data size is rearranged and converted into an output size of 321×70, which is the same as the input size of the lightweight LeNet-5 hierarchical convolutional neural network model.

[0012] Furthermore, the underwater filter circuit module employs an operational amplifier to construct a bandpass filter composed of a cascaded high-pass filter designed by a fourth-order Butterworth and a low-pass filter designed by an eighth-order Butterworth. The AD signal acquisition module specifically includes an ADC acquisition module and a bias circuit module. The ADC acquisition module uses the built-in ADC function of the STM32H743 to perform analog-to-digital conversion on the signal. It adopts dual-buffering technology, where data is read from one buffer while data is written to the other buffer, enabling real-time data processing. The bias circuit module provides an operating point for bias DC component adjustment and deploys an RC DC blocking circuit to adjust the signal. The SD storage module uses an SD card module for data storage, and a TF card slot is used to fix the SD card module and provide electrical connection. Parallel data transmission is performed through the communication protocol of SD mode. The power management module is powered by an external battery and uses an integrated step-down module for step-down output. The serial communication module uses a universal asynchronous transceiver as the communication protocol and adopts the RS232 serial communication interface standard. It uses a MAX232 level conversion chip to convert between logic level and RS232 standard level, enabling serial communication between the core processor module, AD signal acquisition module, underwater filter circuit module, lightweight marine signal processing module, power management module, SD storage module and PC.

[0013] Furthermore, the lightweight LeNet-5 hierarchical convolutional neural network model of the lightweight marine signal processing module is embedded and deployed into the STM32H743 microcontroller through the X-Cube-AI tool to perform hierarchical processing of underwater acoustic signal features in the marine area; when the module is deployed in the marine area for underwater acoustic signal processing, it automatically identifies, classifies and modulates underwater signals including active sonar signals, synchronization signals and non-cooperative communication signals in real time, and uses them for marine information analysis after recovery.

[0014] Furthermore, the SD storage module specifically includes an SD card module, an external memory expansion module, and a memory management module; The SD card module uses a large SD card interface and is driven by SDMMC for massive data storage. The external memory expansion module uses synchronous dynamic random access memory (DRAM), which is initialized and its address allocated by an external storage controller to store data generated during system operations. The memory management module uses distributed loading technology, which modifies the storage allocation scheme specified in the distributed loading file to divide the execution image file into partitions and then maps them to the corresponding physical storage addresses. The power management module specifically includes an integrated buck module and a linear regulator. The integrated buck module reduces the high input voltage of the external battery to an intermediate voltage level, while the linear regulator further reduces the voltage while reducing output voltage fluctuations and noise.

[0015] Combining all the above technical solutions, the beneficial effects of this invention are as follows: This invention enables self-contained hydrophones to automatically identify underwater acoustic signals in a long-term, real-time, and accurate manner, providing immediate intelligence support for underwater non-cooperative target reconnaissance; the hierarchical network design significantly improves signal processing efficiency and system real-time response capability, and performs hierarchical processing according to signal characteristics (such as spectral concentration), prioritizing the screening and processing of easily identifiable signals (such as active sonar and synchronization signals), and concentrating computing resources on more complex non-cooperative communication signals; compared with traditional methods based on artificial features, the deep learning-based method can automatically learn deep features from time-frequency maps, has better robustness to noise and complex marine environments, and has a higher identification accuracy; the device integrates signal acquisition, filtering, intelligent identification, and storage, has a compact structure, low power consumption, and is suitable for deployment on front-end platforms such as buoys and underwater vehicles that require long-term, autonomous reconnaissance of the sea area. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure; Figure 1 This is a flowchart of the underwater acoustic signal processing using a lightweight LeNet-5 hierarchical convolutional neural network model provided in an embodiment of the present invention; Figure 2 This is a flowchart of a deep learning-based marine signal reconnaissance method provided in an embodiment of the present invention; Figure 3 This is a circuit diagram of the ADC acquisition module provided in an embodiment of the present invention; Figure 4 This is a circuit diagram of the SD card interface provided in an embodiment of the present invention; Figure 5 This is a flowchart of the signal processing algorithm provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the lightweight LeNet-5 hierarchical convolutional neural network architecture provided in an embodiment of the present invention. Detailed Implementation

[0017] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0018] Example 1: The deep learning-based marine signal reconnaissance device includes a core processor module, an AD signal acquisition module, an underwater filtering circuit module, a serial communication module, a lightweight marine signal processing module, a power management module, and an SD storage module.

[0019] For example, the core processor module uses an STM32H743 microcontroller, based on the Arm® Cortex®-M7 core, with a built-in FPU. This core processor module supports the deployment of neural network models through the X-Cube-AI tool, meeting real-time signal processing requirements.

[0020] All functional modules are embedded in a microcontroller based on the STM32H743, and are housed within a hydrophone enclosure designed to withstand high voltage and external influences. The AD signal acquisition module first acquires and converts the marine signal. After noise is filtered out by an underwater filtering circuit, the signal is input to a lightweight marine signal processing module for processing and classification. Finally, the results are input to an SD storage module. A serial communication module is used for communication between the STM32H743 microcontroller and each module, while a power management module ensures adequate power supply to all modules during operation.

[0021] For example, the underwater filter circuit module uses an LT6233 operational amplifier to construct a bandpass filter composed of a high-pass filter designed by a fourth-order Butterworth and a low-pass filter designed by an eighth-order Butterworth. For example, the AD signal acquisition module specifically includes an ADC acquisition module and a bias circuit module; the ADC acquisition module, such as... Figure 3 As shown, the STM32H743's built-in ADC function is used to perform analog-to-digital conversion on the signal. A dual-buffering technique is employed, where data is read from one buffer while data is written to the other, enabling real-time data processing. The bias circuit module uses an AD8031 to provide an operating point for adjusting the bias DC component, and an RC DC blocking circuit is deployed to achieve appropriate signal adjustment.

[0022] For example, the SD storage module uses an SD card module for data storage, uses a TF card slot to fix the SD card module and provides an electrical connection for parallel data transmission via the SD mode communication protocol; Specifically, it includes an SD card module, an external memory expansion module, and a memory management module; The SD card module uses a large SD card interface (i.e., a camera card; TF cards cannot be inserted directly and require an adapter), and is driven by SDMC. This SD card interface can meet the needs of massive data storage. The circuitry of the SD card interface is as follows: Figure 4 As shown; the external memory expansion module uses the IS42S3280J from ISSI as a synchronous dynamic random access memory (DRAM), which is initialized and its address allocated by an external storage controller to store data generated during system operations; the memory management module employs a distributed loading technique, which modifies the storage allocation scheme specified in the distributed loading file to divide the executable image file into specific partitions and then maps them to corresponding physical storage addresses. When the embedded system restarts or resets, it initializes the processor's internal memory and allocates physical addresses. The corresponding code and data are copied to the runtime address according to the rules set by the storage allocation scheme, achieving efficient memory utilization.

[0023] For example, the power management module uses an external battery for power supply and an integrated buck module for voltage reduction output. Specifically, this includes an integrated buck module LTM8033 and a linear regulator TPS7A47. The integrated buck module reduces the higher input voltage of the external battery to an intermediate voltage level, while the linear regulator further reduces output voltage fluctuations and noise while further reducing the voltage.

[0024] The power management module has an input voltage range of 5V-9V, providing 5V power to the STM32H743 microcontroller and underwater filter circuit module, and 3.3V power to the serial port and SD storage module. For example, the serial communication module uses a universal asynchronous transceiver as the communication protocol and adopts the RS232 serial communication interface standard. It uses a MAX232 level conversion chip to convert between logic levels and RS232 standard levels, thereby enabling serial communication between the core processor module, AD signal acquisition module, underwater filter circuit module, lightweight marine signal processing module, power management module, SD storage module and the PC.

[0025] For example, the lightweight LeNet-5 hierarchical convolutional neural network model of the lightweight marine signal processing module is embedded and deployed into the STM32H743 microcontroller through the X-Cube-AI tool to perform hierarchical processing of underwater acoustic signal features in the marine area; when the module is deployed in the marine area for underwater acoustic signal processing, it automatically identifies, classifies and modulates underwater signals including active sonar signals, synchronization signals and non-cooperative communication signals in real time, and uses them for marine information analysis after recovery.

[0026] Example 2: The deep learning-based marine signal reconnaissance method utilizes a lightweight LeNet-5 hierarchical convolutional neural network model, which is embedded into an STM32H743 microcontroller using the X-Cube-AI tool to process underwater acoustic signals. It automatically identifies, classifies, and modulates underwater signals, including active sonar signals and non-cooperative communication signals, in real time, and uses this information for analysis of recovered marine information. like Figure 1 As shown, the underwater acoustic signal processing using the constructed lightweight LeNet-5 hierarchical convolutional neural network model includes: S101, LA sub-network judgment: Based on the spectral concentration and feature saliency of the signal, the input signal is initially judged and divided into a first type of signal and a second type of signal; wherein, the first type of signal includes active sonar signals and synchronization signals with more salient features and easier to identify; the second type of signal is non-cooperative communication signal; S102, LA sub-network processing, identifies the first type of signal identified in the first step, and performs secondary discrimination through LD sub-network, including: first, based on the characteristic of the fixed modulation mode of active sonar signals, the active sonar signals are identified first; then, the identified synchronization signals are confirmed, and when the synchronization signals are confirmed, a feedback control command is generated and fed back to the STM32H743 microcontroller for continuous monitoring. S103, LC sub-network processing, performs modulation identification and classification on the second type of signal identified in the first step, namely the non-cooperative communication signal.

[0027] Example 3, exemplary, the deep learning-based maritime signal reconnaissance method of the present invention will be further described in detail below with reference to specific technical features.

[0028] like Figure 2 As shown, the deep learning-based method for maritime signal reconnaissance includes the following steps: S1, Construction of a deep learning-based marine signal reconnaissance device; the deep learning-based marine signal reconnaissance device includes a core processor module, an AD signal acquisition module, an underwater filtering circuit module, a serial communication module, a lightweight marine signal processing module, a power management module, and an SD storage module.

[0029] S2, based on a deep learning-based marine signal reconnaissance device, performs signal acquisition and processing. It uses the internal ADC of the STM32H743 microprocessor to acquire underwater acoustic signals; filters out noise through a constructed underwater filtering circuit module; and processes the underwater acoustic signals through short-time Fourier transform to obtain a time-frequency diagram. Specifically, the steps include the following: (i) The ADC acquisition module samples the signal using the ADC built into the STM32H743, with the sampling rate set to 96kHz and using double buffering technology to obtain the real-time underwater acoustic signal input. (ii) Use an underwater filter circuit module to filter out low-frequency and high-frequency noise from the input signal to obtain a noise-removed underwater acoustic signal; (iii) Using CMSIS-DSP library functions, the energy decision statistic of the signal is calculated by taking the modulus of the received signal and calculating the sum of squares. The statistic is then compared with the decision threshold to determine whether the signal is valid.

[0030] ; ; In the formula, Assuming "no signal", Assuming there is a signal, For the received signal, It is noise.

[0031] Under constant false alarm conditions, i.e., when only noise exists, the detection probability and false alarm probability are calculated using the following formulas: ; ; ; ; ; In the formula, This represents the probability of a false alarm. For detection probability, As a threshold for judgment, The number of sampling points. For signal-to-noise ratio, Let Variance be the noise variance. The variance of the signal. Let be the right-tail probability function of the standard normal distribution. It is the inverse function of the right-tail probability function of the standard normal distribution.

[0032] (iv) Normalize the amplitude characteristics of the detected signal to a standard range, window the signal, select Hanning window as the window function, select window length of 1024, select 900 overlapping samples, and select sampling rate of 96k.

[0033] (v) Perform a Fast Fourier Transform (FFT) using DSP library functions to obtain a complex array, and calculate using the following formula: ; In the formula, Given a discrete signal sequence as input. For window functions at time points Add windows. For the length of the window, For time indexing, For frequency index, This is the kernel function for the Fourier transform.

[0034] (vi) Use DSP library functions to calculate the amplitude of the complex sequence, obtain the amplitude information at each frequency, and output the time-frequency diagram of the signal using the following formula: ; In the formula, For the first The amplitude value of each frequency component, For the first Complex results for each frequency component, For complex numbers The real part, For complex numbers The imaginary part.

[0035] (vii) After completing the STFT calculation of the signal, a one-dimensional array with a length of 512 is obtained. The data size is rearranged and converted into 321×70, which is the same as the output and neural network input size.

[0036] S3, Lightweight LeNet-5 Hierarchical Convolutional Neural Network Model Construction and Training; such as Figure 5 As shown, the specific steps include the following: (1) Use the simulation software MATLAB to generate five non-cooperative communication signals (CW, LFM, 2FSK, 4FSK, BPSK) and HFM synchronization signal to simulate the actual received signals; use data augmentation technology to expand the data set and obtain a data sample set. For example, obtaining a data sample set specifically includes: Non-cooperative signal dataset: Five non-cooperative communication signals, namely CW, LFM, 2FSK, 4FSK and BPSK, were generated using the simulation software MATLAB to simulate the signals actually received.

[0037] Active sonar signal dataset: Utilizing dozens of active sonar frequency-modulated signals from currently known databases.

[0038] Synchronization signal dataset: HFM signals are commonly used for synchronization in underwater acoustic communication. Therefore, multiple sets of HFM signals are generated using the simulation software MATLAB to simulate the actual received signals. The signal is subjected to short-time Fourier transform (STFT) to generate a time-frequency plot. Then, SpecAugment data augmentation technology is used to shift the time dimension and mask the time-frequency dimension to expand the data and obtain a complete dataset sample set. (2) Import more than a dozen active sonar frequency modulation signals from the currently known database.

[0039] (3) The processed time-frequency map is received through the input layer and used as the input of the lightweight LeNet-5 hierarchical convolutional neural network; (4) Constructing a lightweight LeNet-5 hierarchical convolutional neural network model architecture. Using Python, a TensorFlow / Keras library was built based on the TensorFlow / Keras framework. The LA sub-network was defined as a binary classification model, and the LB, LC, and LD sub-networks were defined as multi-class classification models. A custom conditional judgment function was used to implement the hierarchical architecture. The principle of the lightweight LeNet-5 hierarchical convolutional neural network architecture is as follows: Figure 5 As shown.

[0040] For example, such as Figure 6 As shown, the lightweight LeNet-5 hierarchical convolutional neural network model processes signals hierarchically according to the characteristics of marine signals. Because active sonar signals and synchronization signals have a more concentrated spectrum and are easier to identify than non-cooperative communication signals, the lightweight LeNet-5 hierarchical convolutional neural network model prioritizes the identification and classification of active sonar signals and synchronization signals, and then classifies the remaining signals as non-cooperative communication signals. Active sonar signals are characterized by fewer categories and fixed frequency modulation methods; therefore, among active sonar signals and synchronization signals, the lightweight LeNet-5 hierarchical convolutional neural network model prioritizes matching and classifying active sonar signals. Finally, the synchronization signal is identified and confirmed.

[0041] Another example is the process of hierarchical processing (processing underwater acoustic signals) based on marine signal characteristics of the lightweight LeNet-5 hierarchical convolutional neural network model, which includes the following steps: 1) The LA subnetwork determines whether the input signal is an active sonar signal or a synchronization signal. If so, it switches to the LB subnetwork; otherwise, it switches to the LC subnetwork. 2) If the signal is an active sonar signal or a synchronization signal, it is further identified and classified by the LB sub-network; if it is identified as a synchronization signal, it is fed back to the microcontroller to continuously monitor subsequent signals; if it is an active sonar signal, it is transferred to the LD sub-network. 3) If the signal is a non-cooperative communication signal, it will be identified and classified by the LC subnetwork; 4) The LD subnetwork performs precise matching and identification of active sonar signals.

[0042] For example, in step (4), a lightweight LeNet-5 hierarchical convolutional neural network model is constructed, including: performing network hierarchical activation processing to handle complex underwater acoustic signal types through hierarchical design; using depthwise separable convolution in the convolutional layers to reduce the number of model parameters and computational complexity; using a global average pooling module instead of flattening to prevent overfitting; and using the ReLU function instead of the Sigmoid function as the activation function to introduce nonlinear features and avoid gradient vanishing.

[0043] The ReLU activation function expression is: ; In the formula, The output of the activation function, The input values ​​for the activation function. To take the maximum value between 0 and the input value; Furthermore, the expression for the cross-entropy loss function in the lightweight LeNet-5 hierarchical convolutional neural network model is as follows: ; In the formula, For the number of categories, and Each indicates its position. The true value at its position (1 for true, 0 for false). This represents the predicted probability value for its location; Furthermore, the expression for batch gradient descent in the lightweight LeNet-5 hierarchical convolutional neural network model is as follows: ; In the formula, For parameters, For learning rate, For gradient.

[0044] (5) Lightweight design of LA, LB, LC and LD sub-networks is carried out. Depth-separable convolution is adopted to calculate multiple feature maps of the time-frequency map separately in separate channels. Then, point convolution with size 1*1 is used to perform operations on the channels to complete the convolution calculation, reducing the upper and lower dimensions of the feature map by two pixels. After halving the feature map size through the second pooling layer, depth-separable convolution is performed again, and the feature map size is halved again through the fourth pooling layer. (6) Through the SE attention module, the feature channels are converted into scalars by global average pooling to obtain the weight of each feature channel. After the scalar is processed by two fully connected layers, it is used as the input of the ReLU activation function and outputs a weight vector with the same number of channels, i.e. attention weights. The feature channel weights are adjusted during training. (7) Global average pooling is used to perform average pooling on each feature map to extract feature information describing the global features; (8) The extracted feature information is classified by the output layer Softmax function to obtain the probability distribution of five types of modulation signals. The category with the highest probability is selected as the recognition result after decision-making.

[0045] (9) Training: For the lightweight LeNet-5 layered convolutional neural network model constructed by the host computer, the parameters of the lightweight LeNet-5 layered convolutional neural network model are trained by unsupervised training stage and fine-tuning stage, the conditional probability distribution of each layer is obtained, and the parameters of different sub-networks are adjusted to realize the adaptive adjustment of the parameters of the lightweight LeNet-5 layered convolutional neural network.

[0046] S4, lightweight LeNet-5 hierarchical convolutional neural network model embedded deployment; includes the following steps: (a) Using the X-Cube-AI tool deployed in STM32CubeMX, the trained lightweight LeNet-5 hierarchical convolutional neural network model was imported into the embedded microcontroller system to verify the model compatibility and resource requirements of the lightweight LeNet-5 hierarchical convolutional neural network model. (b) Using the code generation function of the X-Cube-AI tool, the lightweight LeNet-5 hierarchical convolutional neural network model was converted into pure C code; (c) The tool generates the core files network.c and network_data.c. Open both files in the Keil MDK development environment. (d) Open the converted project with Keil MDK, compile and modify the code, view the code information with the serial port debugging assistant, and burn the project to the STM32H743 microcontroller connected to the PC to complete the embedded deployment of the lightweight LeNet-5 hierarchical convolutional neural network model. (e) Return to CubeMX, click Validate on target, and run the verification program on the board. If it works, the model has been successfully deployed in the MCU.

[0047] S5 utilizes the lightweight LeNet-5 hierarchical convolutional neural network model to complete the real-time classification and recognition of marine signals, and stores the recognition results in the SD storage module, thereby realizing real-time monitoring of marine signals and completing the construction of a marine reconnaissance system.

[0048] This invention conducted comprehensive power consumption and real-time performance verification tests on a maritime reconnaissance system. The results show that the system successfully met the expected performance requirements in both aspects. To verify the system's recognition function, experiments were conducted using simulated signals and signals from a real underwater acoustic channel. Experimental results demonstrate that the system can effectively recognize real-time input communication signals. Furthermore, it provides a new design approach for signal processing and recognition in the field of underwater acoustic communication.

[0049] Application example. Deep learning-based methods for maritime signal reconnaissance, such as... Figure 5 ,include: Step 1, Signal Acquisition and Preprocessing: The AD signal acquisition module embedded in the STM32H743 microcontroller is used to acquire marine signals in real time for real-time analysis of marine information.

[0050] Step 2, Signal Filtering: The underwater filter circuit module is used to filter noise, which improves the recognition accuracy while ensuring signal quality.

[0051] Step 3, Signal Detection and Feature Extraction: The presence of the signal is determined by the energy detection method, and the effective signal segment is subjected to short-time Fourier transform (STFT) to generate a time-frequency diagram; Step 4: Construct and train a lightweight LeNet-5 hierarchical convolutional neural network model; a) Dataset Construction and Preprocessing: First, a dataset sample for training the network is constructed. Commonly used underwater signals in the sea area can be divided into three categories: synchronization signals, active sonar signals, and non-cooperative communication signals. Non-cooperative communication signals such as CW, LFM, 2FSK, 4FSK, and BPSK, as well as HFM synchronization signals, are generated using the simulation software MATLAB. Then, dozens of active sonar frequency-modulated signals from the currently known database are added and subjected to Short-Time Fourier Transform (STFT) to generate a time-frequency diagram. Finally, SpecAugment technology is used to expand the dataset, forming a complete dataset.

[0052] b) Construction and Training of a Lightweight LeNet-5 Hierarchical Convolutional Neural Network Model: Unlike traditional neural networks that learn to classify all signals, this invention processes signals hierarchically according to their characteristics. Using Python, a TensorFlow / Keras library is built, defining the LA sub-network as a binary classification model and the LB, LC, and LD sub-networks as multi-class classification models. A custom conditional judgment function is used to implement the hierarchical architecture, and training is performed using a corresponding signal database to complete feature recognition and real-time hierarchical classification of signals.

[0053] Step 5, Deployment of the lightweight LeNet-5 hierarchical convolutional neural network model: Using the X-Cube-AI tool, the trained lightweight LeNet-5 hierarchical convolutional neural network model is imported into the STM32H743 microcontroller, and the compatibility and resource requirements of the lightweight LeNet-5 hierarchical convolutional neural network model are verified. Step 6, Pattern Recognition and Result Output: The time-frequency map generated after signal preprocessing is input into a lightweight convolutional neural network model deployed on an STM32H743 microcontroller. This lightweight LeNet-5 hierarchical convolutional neural network model will complete the real-time classification and recognition of marine signals and store the recognition results to an SD card, realizing real-time monitoring of marine signals.

[0054] In maritime and ship detection applications, the optimized lightweight LeNet-5 hierarchical convolutional neural network model can be used and downloaded to the embedded STM32H743 microprocessor to process real-time acquired maritime signal data online, thereby obtaining information on the presence of ships in the sea area, which plays an important role in maritime monitoring.

[0055] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention and within the spirit and principles of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A deep learning-based method for maritime signal reconnaissance, characterized in that, This method utilizes a lightweight LeNet-5 hierarchical convolutional neural network model, embedded into an STM32H743 microcontroller using the X-Cube-AI tool, to process underwater acoustic signals. It automatically identifies, classifies, and modulates underwater signals (active sonar signals and non-cooperative communication signals) in real time, and then uses this information for post-recovery marine information analysis. Specifically, it includes the following steps: S101, LA sub-network judgment: Based on the spectral concentration and feature saliency of the signal, the input signal is initially judged and divided into a first type of signal and a second type of signal; wherein, the first type of signal includes active sonar signals and synchronization signals with more salient features and easier to identify; the second type of signal is non-cooperative communication signal; S102, LA sub-network processing, identifies the first type of signal identified in the first step, and performs secondary discrimination through LD sub-network, including: first, based on the characteristic of the fixed modulation mode of active sonar signals, the active sonar signals are identified first; then, the identified synchronization signals are confirmed, and when the synchronization signals are confirmed, a feedback control command is generated and fed back to the STM32H743 microcontroller for continuous monitoring. S103, LC sub-network processing, performs modulation identification and classification on the second type of signal identified in the first step, namely the non-cooperative communication signal.

2. The deep learning-based maritime signal reconnaissance method according to claim 1, characterized in that, Building a lightweight LeNet-5 hierarchical convolutional neural network model involves the following steps: (1) Use the simulation software MATLAB to generate five non-cooperative communication signals (CW, LFM, 2FSK, 4FSK, BPSK) and HFM synchronization signal to simulate the actual received signals; use data augmentation technology to expand the data set and obtain a data sample set. (2) Import more than a dozen active sonar frequency modulation signals from the currently known database; (3) The processed time-frequency map is received through the input layer and used as the input of the lightweight LeNet-5 hierarchical convolutional neural network; (4) Construct a lightweight LeNet-5 hierarchical convolutional neural network model architecture; use Python language to build the TensorFlow / Keras library based on the TensorFlow / Keras framework, define the LA sub-network as a binary classification model, and define the LB, LC, and LD sub-networks as multi-class classification models; implement the hierarchical architecture using custom conditional judgment functions; (5) Lightweight design of LA, LB, LC and LD subnetworks; (6) Through the SE attention module, the feature channels are converted into scalars by global average pooling to obtain the weight of each feature channel. After the scalar is processed by two fully connected layers, it is used as the input of the ReLU activation function and outputs a weight vector with the same number of channels, i.e. attention weights. The feature channel weights are adjusted during training. (7) Global average pooling is used to perform average pooling on each feature map to extract feature information describing the global features; (8) The extracted feature information is classified by the output layer Softmax function to obtain the probability distribution of five types of modulation signals. The category with the highest probability is selected as the recognition result after decision-making. (9) Training: For the lightweight LeNet-5 layered convolutional neural network model constructed by the host computer, the parameters of the lightweight LeNet-5 layered convolutional neural network model are trained by unsupervised training stage and fine-tuning stage, the conditional probability distribution of each layer is obtained, and the parameters of different sub-networks are adjusted to realize the adaptive adjustment of the parameters of the lightweight LeNet-5 layered convolutional neural network.

3. The deep learning-based maritime signal reconnaissance method according to claim 2, characterized in that, The data sample set obtained in step (1) specifically includes: Non-cooperative signal dataset: Five non-cooperative communication signals, namely CW, LFM, 2FSK, 4FSK and BPSK, were generated using the simulation software MATLAB to simulate the signals actually received. Active sonar signal dataset, utilizing more than a dozen active sonar frequency-modulated signals from a known database; Synchronization signal dataset: HFM signals are commonly used for synchronization signals in underwater acoustic communication. Therefore, multiple sets of HFM signals are generated using the simulation software MATLAB to simulate the actual received signals. The signal is subjected to a Short Time Fourier Transform (STFT) to generate a time-frequency plot. Then, SpecAugment data augmentation technology is used to expand the data by shifting the time dimension and masking the time-frequency dimension, thereby increasing the dataset size and obtaining a complete dataset sample set.

4. The deep learning-based maritime signal reconnaissance method according to claim 2, characterized in that, In step (4), the lightweight LeNet-5 hierarchical convolutional neural network model is implemented by: using depthwise separable convolutions in the convolutional layers to reduce the number of model parameters and computational complexity; replacing flattening with global average pooling to prevent overfitting; and using the ReLU function instead of the Sigmoid function as the activation function to introduce non-linear features and avoid gradient vanishing. Step (5) involves lightweighting the LA, LB, LC, and LD sub-networks by: using depthwise separable convolution to calculate multiple feature maps of the time-frequency map separately in a single channel, then using a 1*1 point convolution and performing operations on the channel to complete the convolution calculation, reducing the upper and lower dimensions of the feature map by two pixels each; after halving the feature map size through the second pooling layer, performing depthwise separable convolution again, and then halving the feature map size again through the fourth pooling layer.

5. The deep learning-based maritime signal reconnaissance method according to claim 1, characterized in that, The lightweight LeNet-5 hierarchical convolutional neural network model was constructed and embedded into an STM32H743 microcontroller using the X-Cube-AI tool, including the following steps: (a) Using the X-Cube-AI tool deployed in STM32CubeMX, the trained lightweight LeNet-5 hierarchical convolutional neural network model was imported into the embedded microcontroller system to verify the model compatibility and resource requirements of the lightweight LeNet-5 hierarchical convolutional neural network model. (b) Using the code generation function of the X-Cube-AI tool, the lightweight LeNet-5 hierarchical convolutional neural network model was converted into pure C code; (c) The tool generates the core files network.c and network_data.c. Open both files in the Keil MDK development environment. (d) Open the converted project with Keil MDK, compile and modify the code, view the code information with the serial port debugging assistant, and burn the project to the STM32H743 microcontroller connected to the PC to complete the embedded deployment of the lightweight LeNet-5 hierarchical convolutional neural network model. (e) Return to CubeMX, click Validate on target, and run the verification program on the board. If it works, it has been successfully deployed on the STM32H743 microcontroller.

6. A deep learning-based maritime signal reconnaissance device, characterized in that, The device for implementing the deep learning-based marine signal reconnaissance method as described in any one of claims 1-5 includes a core processor module, an AD signal acquisition module, an underwater filtering circuit module, a serial communication module, a lightweight marine signal processing module, a power management module, and an SD storage module. The AD signal acquisition module, underwater filtering circuit module, serial communication module, lightweight marine signal processing module, power management module, and SD storage module are all embedded in the core processor module, which uses a microcontroller based on STM32H743. Specifically, all modules are installed in the hydrophone enclosure; the AD signal acquisition module first acquires and converts the marine signals, then filters out noise through the underwater filter circuit module before inputting them into the lightweight marine signal processing module for processing and classification; finally, the results are input into the SD storage module; the serial communication module is used for communication between the STM32H743 microcontroller and each module, and the power management module is used for the normal power supply of each module during operation.

7. The deep learning-based maritime signal reconnaissance device according to claim 6, characterized in that, The core processor module is based on the Arm® Cortex®-M7 core and has a built-in FPU; it supports the deployment of lightweight LeNet-5 hierarchical convolutional neural network models through the X-Cube-AI tool. The STM32H743 microprocessor acquires underwater acoustic signals through its internal ADC acquisition module. Noise is filtered out using an underwater filter circuit module; the underwater acoustic signal is processed using a short-time Fourier transform to obtain a time-frequency diagram; specifically including: (i) The ADC acquisition module samples the signal using the ADC built into the STM32H743 microprocessor. The sampling rate is set to 96kHz, and dual buffering technology is used to obtain the real-time underwater acoustic signal input. (ii) Use an underwater filter circuit module to filter out low-frequency and high-frequency noise from the input signal to obtain a noise-removed underwater acoustic signal; (iii) Using CMSIS-DSP library functions, the energy decision statistic of the signal is calculated by taking the modulus of the received signal and calculating the sum of squares. The statistic is then compared with the decision threshold to determine whether the signal is valid. (iv) Normalize the amplitude characteristics of the detected signal to a standard range and then window the signal; (v) Use DSP library functions to perform Fast Fourier Transform (FFT) to obtain a complex array; (vi) Use DSP library functions to calculate the amplitude of the complex sequence, obtain the amplitude information at each frequency, and output the time-frequency diagram of the signal; (vii) After completing the STFT calculation of the signal, a one-dimensional array with a length of 512 is obtained. The data size is rearranged and converted into an output size of 321×70, which is the same as the input size of the lightweight LeNet-5 hierarchical convolutional neural network model.

8. The deep learning-based maritime signal reconnaissance device according to claim 6, characterized in that, The underwater filter circuit module uses an operational amplifier to construct a bandpass filter composed of a high-pass filter designed by a fourth-order Butterworth and a low-pass filter designed by an eighth-order Butterworth. The AD signal acquisition module specifically includes an ADC acquisition module and a bias circuit module. The ADC acquisition module uses the built-in ADC function of the STM32H743 to perform analog-to-digital conversion on the signal. It adopts dual-buffering technology, where data is read from one buffer while data is written to the other buffer, enabling real-time data processing. The bias circuit module provides an operating point for bias DC component adjustment and deploys an RC DC blocking circuit to adjust the signal. The SD storage module uses an SD card module for data storage, and a TF card slot is used to fix the SD card module and provide electrical connection. Parallel data transmission is performed through the communication protocol of SD mode. The power management module is powered by an external battery and uses an integrated step-down module for step-down output. The serial communication module uses a universal asynchronous transceiver as the communication protocol and adopts the RS232 serial communication interface standard. It uses a MAX232 level conversion chip to convert between logic level and RS232 standard level, enabling serial communication between the core processor module, AD signal acquisition module, underwater filter circuit module, lightweight marine signal processing module, power management module, SD storage module and PC.

9. The deep learning-based maritime signal reconnaissance device according to claim 6, characterized in that, The lightweight LeNet-5 hierarchical convolutional neural network model of the lightweight marine signal processing module is embedded and deployed into the STM32H743 microcontroller through the X-Cube-AI tool to perform hierarchical processing of underwater acoustic signal characteristics in the marine area. When the module is deployed in the marine area for underwater acoustic signal processing, it automatically identifies, classifies and modulates underwater signals, including active sonar signals, synchronization signals and non-cooperative communication signals, in real time, and uses them for marine information analysis after recovery.

10. The deep learning-based maritime signal reconnaissance device according to claim 6, characterized in that, The SD storage module specifically includes an SD card module, an external memory expansion module, and a memory management module; The SD card module uses a large SD card interface and is driven by SDMMC for massive data storage. The external memory expansion module uses synchronous dynamic random access memory (DRAM). The DRAM is initialized and its address is allocated by an external storage controller to store data generated during system operations. The memory management module uses a distributed loading technique, which modifies the storage allocation scheme specified in the distributed loading file to divide the execution image file into partitions and then maps them to the corresponding physical storage addresses. The power management module specifically includes an integrated buck module and a linear regulator. The integrated buck module reduces the high input voltage of the external battery to an intermediate voltage level, while the linear regulator further reduces the voltage while reducing output voltage fluctuations and noise.

Images