Method for acoustic underwater target detection

Abstract

A method for acoustic underwater target detection is disclosed, comprising the following step: a) applying a model (22) trained by means of machine learning to a representation (28) of sonar data, wherein the representation of the sonar data is calculated on the basis of a mode decomposition, a spectrogram with a logarithmic representation, in particular the log-mel scale, the gammaton frequency scale or the cepstrum, or has a frequency spectrum, wherein an algorithm of the model (22) comprises a convolutional artificial neural network [CNN] or an artificial neural network with transformer technology.

Description

[0001] The invention relates to underwater acoustic target recognition (UATR) based on a model trained by machine learning.

[0002] Underwater Acoustic Target Recognition (UATR) describes techniques for classifying, categorizing, and identifying unknown surface vessels or submarines through the analysis of their acoustic signals. Obtaining knowledge about the nature of the source of underwater noise has applications in maritime security, maritime traffic monitoring and management, navigation, surveillance and detection, and the protection of marine life. In many areas, remote sensing is primarily conducted in the visual or radar range. However, most modalities are unable to penetrate the water column, making them unsuitable for underwater remote sensing.

[0003] In contrast, acoustic waves can propagate over long distances underwater. To date, the classification of unknown seagoing vessels is mostly performed manually by experienced SONAR operators. Powerful SONAR systems are capable of detecting, locating, and differentiating subtle noise sources. Accompanied by the increasing number of seagoing vessels due to advancing globalization, the number of targets that SONAR operators must identify, manage, and classify has steadily increased. Manual evaluation is no longer practical from a purely time and cost perspective. Initial automated approaches utilized hand-crafted features and traditional statistical analyses.

[0004] The development of automated sonar processing methods has been predominantly focused on the human operator. Consequently, the focus in the past has been on understandable and interpretable, usually time- and frequency-based features obtained through STFT, LOFAR, and DEMON analysis. These feature representations are typically combined with simple statistical models to classify or categorize a received acoustic signal. Statistical models have been shown to work adequately in data-sparse applications and are less prone to overfitting. They are relatively simple and offer understandable parameters and operating principles and are often used as decision-support systems for human operators. A support vector machine (SVM) is typically used as the classifier.The SVM is a versatile supervised model primarily used for regression and classification tasks. The main goal of the SVM is to find an optimal decision boundary (or hyperplane) that separates the input data points with a maximum margin. This margin is based on the Euclidean distance between the data point and the decision boundary.

[0005] The Gaussian Mixture Model (GMM) is another commonly used classification model in UATR. The GMM is a probabilistic model that assumes that the input samples are generated from a mixture of normal distributions. The main goal is to determine the distribution parameters, such as mean and variance, to model the underlying mixed Gaussian distribution. Therefore, the GMM is widely used in supervised and unsupervised clustering and classification scenarios. GMMs are extremely flexible and can model a wide range of complex data distributions. Furthermore, due to their probabilistic nature, GMMs can model uncertainty and thus provide confidence levels.

[0006] The presented statistical and feature extraction methods demonstrate that the classification and categorization of unknown ships based solely on the emitted sounds is feasible. Classical approaches require only a handful of data samples for training and are less prone to overfitting, while still being interpretable due to their purely mathematical and statistical nature. However, it has been shown that a rather sophisticated and highly technical feature extraction procedure is required to extract meaningful patterns from the data. Feature extraction is of great importance and requires a high level of expertise and extensive manual labor. Due to the limited model capacity and the highly compressed information in the features, handcrafted features combined with shallow classifiers are unable to extract deep features from the data.In reality, the properties of underwater acoustics are unable to satisfy the assumptions made during feature creation, resulting in insufficient generalization capabilities across different recording conditions. Furthermore, the approaches often lack robustness.

[0007] Therefore, there is a great need for reliable, automatic classification systems.

[0008] CN 000118051831 A discloses a method for acoustic underwater target identification based on a cooperative CNN-Transformer model.

[0009] Z. Yi, L. Pingzheng, X. Shuidong, Y. Qiong, M. Yanxin and L. Mengqi, “Multiresolution Convolutional Neural Network for Underwater Acoustic Target Recognition,” 2021 IEEE 6th International Conference on Signal and Image Processing (ICSIP), Nanjing, China, 2021, pp. 846-850, doi: 10.1109 / ICSIP52628.2021.9688815 discloses a CNN for underwater acoustic target recognition.

[0010] The object of the present invention is therefore to create an improved concept for underwater target detection.

[0011] This problem is solved by the subject matter of the independent patent claims. Further advantageous embodiments are the subject matter of the dependent patent claims.

[0012] Embodiments show a method for acoustic underwater target detection with the following steps: a) applying a model trained by machine learning to a representation of sonar data, wherein the representation of the sonar data is calculated based on a mode decomposition, a spectrogram with a logarithmic representation, for example the mel scale or the cepstrum, or has a frequency spectrum; and b) wherein an algorithm of the model comprises a convolutional artificial neural network [CNN] or an artificial neural network with transformer technology.

[0013] The goal of the model is to classify unknown watercraft, such as ships or underwater vehicles, such as submarines, based on their passive sonar signature. In particular, classification is performed exclusively based on the passive sonar signature.

[0014] The machine learning of some algorithms, such as convolutional neural networks (CNN) and artificial neural networks with transformer technology, is also called “deep learning.”

[0015] Traditionally, UATR systems consist of independent feature extraction and classification processes. Due to the complexity of the oceanic environment, the extraction of meaningful features is a crucial aspect of the ship classification process.

[0016] Digitized acoustic signals are constructed as one-dimensional signals describing amplitude fluctuations over time. The raw waveform itself contains the highest possible information content of the received signal. Various waveform-based features, such as the zero-crossing rate and the waveform structure itself, can be used in acoustic processing. However, features based exclusively on the pure temporal waveform are difficult to describe and understand, so derived features are advantageously used. Derived features include, for example, spectral features (i.e., a frequency spectrum) or a spectrogram extracted from the original signal.

[0017] The primary tool for extracting spectral components is the Fourier transform (FT). One property of spectral features that is often presented as a drawback is their inability to capture temporal context. However, this is usually not a problem, as watercraft noise can often be considered stationary.

[0018] If the temporal context is to be captured, two-dimensional time-frequency representations, so-called spectrograms, can be used. The most basic time-frequency representation is the short-time Fourier transform (STFT). The STFT is a temporally segmented Fourier transform (FT) over short, consecutive time frames. Furthermore, it is possible to exploit the harmonic behavior of the spectral components by extracting the cepstrum. The cepstrum can capture periodic structures within the spectrogram and is obtained by the inverse Fourier transform of the logarithmic spectrogram.

[0019] Since most of the energy is concentrated at low frequencies, it may be useful to focus on extracting the characteristic narrowband line spectrum generated by the ship's engines. A common approach is the extraction of the Low Frequency Analysis Recording (LOFAR) spectrogram. In contrast to narrowband detection, Detection of Envelope Modulation On Noise (DEMON) focuses on extracting the modulation frequencies of the broadband noise caused by propeller cavitation. DEMON analysis is therefore used to extract specific propeller characteristics such as the number of propellers, the number of blades per propeller, etc. DEMON or LOFAR spectrograms are therefore particularly suitable as additional, complementary features.

[0020] In addition to the cepstrum, there are other features that use a logarithmic representation. One example is features based on the mel scale, such as the log-mel spectrogram or the mel frequency coefficients (MFCC) derived from it. Since humans do not perceive sound linearly, the logarithmic mel scale is often used to mimic human auditory perception, for example, in the form of a log-mel spectrogram (LMS). The mel scale emphasizes lower frequencies over higher frequencies. Typically, a low-dimensional representation that captures the periodic cepstral properties of the mel spectrogram is extracted using mel frequency cepstral coefficients (MFCC).

[0021] In addition to MFCC, a similar feature representation uses the gammatone frequency scale instead of the mel scale to formulate the gammaton spectrogram (GST). Analogous to the mel approach, the gammatone frequency cepstral coefficients (GFCC) can be extracted from the GST. Both MFCC and GFCC use a series of predefined filters with different bandwidths and amplitudes to emphasize different spectral regions of an input signal. In contrast to MFCC, the gammatone filter bank uses smooth filters, which allow for greater overlap and thus better correlation between the independent filters. The constant Q transform (CQT) is another time / frequency representation closely related to the log-mel spectrogram. Similarly, the CQT provides geometrically spaced frequency filters whose resolution varies logarithmically.It has its origins in music processing and therefore aims at equal resolutions across all octaves.

[0022] All spectrum- or cepstrum-based approaches are capable of capturing energy, frequency, and temporal information. The fundamentals of these feature representations are based on the Fourier transform, which describes a complex wave structure as the sum of superimposed sine waves of infinite length. This periodic property of the sine wave leads to insufficient temporal localization capability. Furthermore, this property is emphasized by the fixed window length of the STFT, in which low and high frequencies are equally resolved in time.

[0023] The wavelet transform (WT) enables better localization in both time and frequency, allowing for more precise analysis of non-stationary signals. Thus, the conflict between low- and high-frequency resolution is avoided by choosing an appropriate window length. The wavelet transform adapts to the signal by using shorter window lengths for high frequencies and shorter window lengths for low frequencies, resulting in a more effective representation of a broad spectrum of signal dynamics. This is particularly advantageous for analyzing signals with transient components and varying frequencies. However, the wavelet transform still requires a priori information to select an appropriate mother wavelet, making it difficult to handle.

[0024] Furthermore, it is possible to use mode analysis, for example, as part of the Hilbert-Huang transform (HHT), to analyze underwater acoustic signals. HHT is an adaptive approach that extracts subtle oscillator-like features from a signal. It focuses only on a specific group of frequencies that have a high energy response. The main strengths of HHT lie in the analysis of nonlinear and nonstationary signals. It uses empirical mode decomposition (EMD) to decompose a signal into eigenmode functions (IMFs). Frequency-time analysis is performed by applying a Hilbert transform to the mode functions. HHT is preferred to WT when instantaneous frequencies are of great interest, since WT assumes a certain degree of frequency stability over the time span of the wavelet.

[0025] The choice of the various time-frequency representations mentioned above depends heavily on the assumption of whether the ship noise is considered stationary or non-stationary. This assumption is not easy to make, as it strongly depends on the specific application. If ships are constantly moving and far away, the noise is more likely to be stationary, whereas strong deceleration and acceleration movements and processes such as engine starts, etc., are more likely to imply the presence of a time-varying signal.

[0026] In addition to the actual feature extraction, generating the representation of the sonar data can include further signal processing steps, for example, smoothing the sonar data. The representation of the sonar data is a multi-channel representation and comprises several channels with features. This means that the multi-channel representation can comprise or consist of any combination of the mode decomposition of the sonar signal, the spectrogram with a logarithmic representation of the sonar signal, the frequency spectrum of the sonar signal, and further features of the sonar signal. Further features of the sonar signal can comprise any selection of temporal derivatives, for example the first and / or second temporal derivative of the sonar signal, or of photometric representations such as chroma and / or contrast and / or tone grid and / or zero-crossing rate.

[0027] Machine learning-based approaches, particularly deep learning, have been shown to improve the generalization capabilities of underwater acoustic target detection while delivering robust results. Among deep learning approaches, convolutional neural networks (CNNs) and transformer-based artificial neural networks have been shown to deliver excellent results. The field of deep learning is a rapidly developing research area with numerous applications across diverse domains. However, SONAR processing has traditionally been a domain of statistical analysis, which is why machine learning for watercraft classification is still in its infancy.

[0028] Furthermore, among the deep learning approaches, the convolutional artificial neural network and the artificial neural network with transformers have each been shown to deliver very good results. It is also possible to combine several algorithms, such as the two mentioned above, for example, running them in parallel or sequentially. This can result in submodels whose results can be merged, for example, by another model to obtain the final classification.

[0029] Deep neural networks have been shown to be more effective at capturing deep hierarchical feature representations than shallow networks using SONAR data. A "deep" network can be defined as one with at least 3, preferably at least 5, more preferably at least 8, or at least 15 hidden layers. Despite the performance improvement, problems such as the gradient descent problem complicate the training process of deep neural networks. This can be remedied by using a CNN with a long-short-term memory neural network (LSTM), which reduces the influence of gradient descent during the training process of conventional recurrent neural networks. This means that the algorithm can be based on a combination of a convolutional artificial neural network (CNN) with a long-short-term memory neural network (LSTM).

[0030] In exemplary embodiments, the algorithm has a residual neural network [ResNet] as its backbone. In particular, ResNet has a deeper architecture than ResNet 20, especially ResNet 30 or ResNet 40.

[0031] It has also been shown that the use of a backbone for the deep learning model in the UATR domain is advantageous. A residual neural network (ResNet) has proven to be a suitable backbone. Deeper architectures such as ResNet50 have proven advantageous in extracting semantically meaningful features compared to shallower architectures such as ResNet18. With appropriate expertise invested in feature selection, even smaller architectures such as ResNet10 are capable of achieving comparable results to those of the ResNet18 architecture.

[0032] Embodiments show that step a0) is performed before step a). a0): Training the model with a data set containing a target to be detected at least twice, each with different environmental conditions. The environmental conditions preferably include wind, rain, (water) temperature, diving depth or hydrostatic pressure, salinity, type of waterbed, sounds from living organisms, or turbulence in the water. The target to be detected is further included in the data set multiple times with at least three varying environmental conditions. This creates a data set with a certain degree of variability.

[0033] Preferably, a data set for each target to be detected has at least 5, preferably at least 10, more preferably at least 15 or at least 20 different environmental properties (permutations of the environmental properties), in particular caused by at least 2, preferably at least 4, more preferably at least 6 different environmental conditions. An environmental condition can be subdivided into different environmental properties. Wind as an environmental condition can be entered into the levels of the Beaufort scale or an alternative scale. One environmental property can be present per scale value. This means that a different environmental property exists if the target to be detected is recorded or detected with wind in two different Beaufort levels. Rain can be viewed as binary (yes / no), so that two environmental properties are present here. However, it is also possible to subdivide the rain even further. The water temperature, for example, canin 5-degree increments between 5° and 35°. The diving depth can be divided into reasonable sections depending on the area of ​​operation. For example, salinity can be divided into 3 different levels. The same applies to the type of seabed. Sounds from living organisms can be divided into constant sounds, such as schools of fish or crab sounds, and time-varying sounds, such as whale or dolphin songs. The various environmental properties now result in a multitude of permutations. The more permutations of the combinations of environmental properties a data set contains for a target, the better the training for that target.

[0034] The main goal of UATR is to distinguish targets by analyzing the acoustic noise emitted. Unfortunately, the received acoustic signal, especially its spectral properties, is influenced not only by the design of the target vessel (e.g., vessel type) and operating conditions, but also by various environmental influences. Therefore, for good generalization, it is important to use a dataset with the greatest possible variability for training. Emitted ship noise follows a complex generation mechanism. Ship noise is typically predominantly of mechanical, propeller, and hydrodynamic origin. This means that ship noise is predominantly composed of these three main sources.

[0035] Mechanical noise describes the noise generated by the propulsion system, engines, transmission, and other mechanical actuators on or within the vessel. Propeller noise describes the modulation of the broadband cavitation noise caused by propeller rotation. Hydrodynamic noise arises from interfacial interactions between the vessel's hull and the marine environment. The composition of these noise sources results in unique acoustic fingerprints of watercraft.

[0036] A sound wave propagating through water can be subject to dynamic effects that alter the properties of the emitted wave, as well as additive effects, such as background noise and sounds from other sources. The nature and extent of these effects depend largely on the specific environment, geographical region, time of year, and the design of the target and receiving platform. High-level influences are further subdivided into intermediate influences. Some of the characteristics are controllable, while others are uncontrollable. When creating a dataset, one would influence the controllable parameters, while the uncontrollable parameters cannot be influenced or can only be influenced indirectly. The individual effects are described below.

[0037] An ideal dataset for the field of underwater acoustics should account for sufficient variability across all influences to ensure adequate complexity coverage. Like any mechanical wave that transports energy through a medium, the key quantities describing underwater sound waves are amplitude, frequency, and phase. Amplitude is an umbrella term for the received pressure within a specific range. Like any physical wave, underwater acoustic waves are subject to interference, attenuation effects, and frequency and phase shifts. Despite the fundamental similarity, many approaches from classical acoustics are not applicable to underwater acoustics, primarily due to the inhomogeneous nature of the oceanic environment. Underwater acoustic signals are subject to various propagation and transmission losses, as well as additive effects such as ambient noise.Regular inhomogeneities such as time-varying underwater and surface channels, as well as antiwaveguides such as shadow zones, are caused by different sound velocity profiles. The sound velocity profile typically depends on water temperature, salinity, and pressure, and is therefore dependent on the geographical region and time. Transmission loss is the reduction in received signal strength as a wave moves through and interacts with a medium.

[0038] The received pressure is influenced by many different effects. First and foremost, any mechanical wave is absorbed by the propagation medium, with some of the energy being absorbed in the form of heat as the wave interacts with the molecules. This effect increases with increasing propagation distances. Second, transmission losses such as scattering, geometric scattering, and reflections reduce the amplitude. Scattering losses describe the redistribution of energy when acoustic signals scatter off particles, bubbles, and other irregularities in the water column. Because acoustic waves propagate omnidirectionally, geometric losses occur where the energy is scattered spherically. In the specific case of an underwater acoustic channel, the energy is distributed cylindrically. Reflection losses occur at the boundaries of media with different propagation velocities, such as the seabed or the water surface.This leads to a signal attenuation, as portions of the energy are either transmitted or reflected. Finally, mechanical properties and operating conditions of the transducer can also alter the radiated amplitude.

[0039] All of the aforementioned effects contribute to the decrease in signal strength. In general, underwater acoustics are considered a low signal-to-noise ratio (SNR) scenario. Real-world transmission losses are mainly comparable to spherical scattering. Spherical scattering exhibits a loss of r -2decreasing intensity and transmission losses of 20 log(r), where r denotes the distance from the source. Furthermore, the propagation of underwater sound signals is usually accompanied by several frequency shift effects. Primarily, relative dynamic movements between the signal transmitter and receiver cause frequency shifts due to the Doppler effect. Second, vessel operating conditions such as engine speed, gear selection, etc., as well as general ship conditions, contribute differently to the radiated frequency spectrum. The dissipation and absorption of signal energy is typically frequency-dependent, with high frequencies being more attenuated compared to lower frequencies. Reflection, scattering, and refraction of an incident wavefront at interfaces cause frequency-dependent generations of secondary wavefronts that overlap with the original wave.Finally, additive effects such as ambient noise contribute additional frequency components to the spectrum of a received signal. Furthermore, the frequency-dependent attenuation of sound waves varies with the water temperature.

[0040] Finally, the received phase of an underwater acoustic signal can be distorted or altered by various effects. Acoustic turbulence, such as local inhomogeneities in the water column, causes fluctuations in the speed and direction of the sound wave. Boundary reflections from the seabed, the water surface, or submerged objects can cause phase shifts. Dynamic properties such as the movement of the transmitter and receiver also alter the phase component. Finally, acoustic signals can take multiple propagation paths due to reflection, refraction, and scattering at the surface and seabed.

[0041] This results in different paths and thus distances that the acoustic signal can travel from the transmitter to the receiver, which in turn leads to phase shifts. Multipath propagation is particularly prevalent in shallow water regions and can add significant temporal complexity to the signal. Finally, the characteristics of the transducer and receiver also influence the received phase due to design, assembly, and processing decisions. In addition to the preceding losses and variations caused by wave propagation in water, various additive contributions from ambient noise influence the SNR and specificity of a received signal.

[0042] Ambient noise can be roughly divided into the following eight categories. At low frequencies (0.1–10 Hz), turbulence noise is present, which is caused by changes in the ocean, atmosphere, and geophysical noise sources such as underwater volcanic eruptions and earthquakes. In the frequency band between 10 and 300 Hz, most noise is caused by long-distance shipping traffic.

[0043] The significant increase in commercial shipping traffic and the poor attenuation of sound at these frequencies create a continuous background noise. The main sources of ship noise are the propulsion engines, propeller interactions in the water, and flow noise caused by water movement against the ship's hull. This type of noise is of greatest interest to the UATR. The state of the sea itself is also an important noise contributor, with many mechanisms. Chief among these is the breaking of waves at the sea surface and the associated cavitation noise of collapsing bubbles. This effect is strongly related to wind conditions. Other weather conditions, such as rain, also generate broadband noise at the surface, typically in the range between 1 kHz and 5 kHz.In addition, biological noise generated by marine fauna can have a significant impact on noise levels in certain regions and seasonal periods. Furthermore, the random motion of atoms and molecules due to thermal energy, also known as molecular agitation or thermal noise, generates a noise spectrum in the range between 100 kHz and 1 MHz. Finally, uncontrollable, human-induced noise contributions, such as seabed mining / drilling, active SONAR systems, etc., can be present across a very broad frequency spectrum. All of the aforementioned noise sources contribute to the isotropic noise level in varying degrees.

[0044] Despite the many different influences on a propagating underwater acoustic wave, some key characteristics of ship-emitted noise are evident. Radiated ship noise consists of a broadband, continuous spectrum with discrete, narrowband frequency components. Propeller cavitation and the ship's flow noise generate the broadband component. The ship's internal machinery and propulsion systems are the main contributors to pronounced, narrowband spectral lines. While target information is predominantly found in low frequencies, especially below 5 kHz, the high-frequency components also contain target information. Most known approaches focus on determining either the low-frequency line spectrum or the high-frequency modulation caused by the target's propeller.One approach to achieving better classification is to use a multi-channel representation of the sonar data. For example, one channel may contain a feature that better emphasizes the low-frequency line spectra, while another channel may contain a feature that better emphasizes the high-frequency modulation. The model can thus perform the classification for both vessel characteristics, thus achieving a better classification result.

[0045] While varying ship operating conditions can lead to temporal irregularities, ship noise is generally considered to be static for a short time, particularly in the range of 10-30 ms, sometimes up to 100 ms. Nevertheless, short-term fluctuations, such as engine start-ups, deceleration and acceleration of target vessels, etc., can provide useful ship-specific information.

[0046] As previously mentioned, the UATR region is considered a low-SNR scenario, where many of the influences described in this section reduce the effective amplitude of a propagating acoustic wave. Despite the distinct characteristics that can be used to categorize unknown ships, ship-radiated noise is still subject to strong intra-class variance, where targets within the same category can vary significantly in terms of speed, tonnage, propulsion system, and propeller characteristics.

[0047] In some embodiments, training can be supervised (supervised learning). In particular, training can then be performed using artificially generated training data. Artificially generated training data has the advantage that the events to be detected in the data can be directly labeled with the correct classification result. While this is possible with real sonar data, it is very complex. Especially when using an artificial neural network with transformer technology, very large amounts of data are required for training. Manually labeling the sonar data is very time-consuming.

[0048] Alternatively, training can be performed unsupervised (unsupervised learning). In particular, training can then be carried out primarily with real training data. With unsupervised learning, the model is trained to classify the training data into different classes. Similar data then belong to the same class. The classes can then be manually labeled. This almost completely reduces the labeling effort.

[0049] As mentioned above, ship noise is composed of narrowband and wideband frequency components. Capturing wideband frequency relationships contrasts with the local feature extraction of CNNs. The use of large convolution kernels and very deep networks can overcome this problem, but at the cost of computationally expensive (time / computational power) and difficult training. It is advantageous to use kernels of different sizes in parallel to extract features at different resolutions to increase robustness when detecting complex wideband targets. For example, a basic block module can be used based on separate fields of the input spectrum, with an MLP (Multi Layer Perceptron) capturing the globally distributed features across these fields.Furthermore, several differently sized dilated kernels can be used to enlarge the receptive field without significantly increasing the computational effort.

[0050] In addition to model-driven solutions, it is also possible to expand the receptive field from the perspective of the data. For example, three different, similar spectrograms with different frequency resolutions can be fused as input for the model in a multi-channel array (multi-channel representation). This means that the representation is a multi-channel representation, with three channels of the multi-channel representation having similar spectrograms with different frequency resolutions. All spectrograms mentioned in this disclosure are suitable as spectrogram types.

[0051] When using a convolutional artificial neural network in conjunction with a transformer, attention mechanisms can be introduced into the CNNs to extract globally distributed features in the frequency domain. This makes it possible to introduce channel-wise attention modules capable of highlighting dominant frequency bands that contain the most useful information. These attention modules can compensate for the limitations of shallow networks. This means that in the convolutional artificial neural network, channel-wise attention modules are used to highlight dominant frequency bands that contain the most useful information.

[0052] It is also possible to expand the receptive field of the CNN by modeling the received time sequence using recurrent neural networks (RNNs). In this case, it is advantageous to use features distributed across the time domain rather than the frequency domain. Long-Short-Term Memory Neural Networks (LSTMs) are particularly suitable, as they reduce the influence of gradient descent during the training process of conventional recurrent neural networks. Temporally correlated features can be effectively extracted from time-frequency representations by introducing recursion. These temporally correlated features can also be extracted from the time-domain signal.

[0053] Furthermore, to overcome the small size of CNNs, it is possible to circumvent the use of convolutional operators by using purely attention-based approaches, e.g., using a transformer. These purely attention-based approaches demonstrate that comparable results can be achieved without the use of convolutional operators. In contrast to the attention-focused channels mentioned above, the transformer utilizes self-attention, which effectively correlates all input tokens, e.g., the frequency bands of a spectrogram. For example, a vision transformer (ViT) can be used. Transformer-based approaches can rely on a self-attention mechanism, which has a quadratic computational complexity of O(N2D), where N is the sequence length and D is the dimension of the feature vector.However, it is also possible to use an additive attention mechanism with linear complexity O(ND) to reduce the computational effort.

[0054] Automatic feature encoding can be achieved through layer-by-layer unsupervised training. Target detection was achieved by passing the encoded feature vector through a backpropagation network. It is also possible to use unsupervised pre-training approaches in a combination of CNN and LSTM to extract meaningful features without the need for annotated data. Furthermore, it is possible to use a contrastive learning approach. For example, a supervised version of SimCLR, a popular contrastive learning approach from the field of computer vision that focuses on self-supervised learning, can be used as a contrastive learning approach. These contrastive approaches enable superior intra-class accumulation and inter-class discrimination.

[0055] Furthermore, a computer program is disclosed, comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method.

[0056] Similarly, a data processing unit for acoustic underwater target detection is disclosed, wherein the data processing unit is configured to perform the following steps: a) applying a model trained by machine learning to a representation of sonar data, wherein the representation of the sonar data is calculated based on a mel scale or has a frequency spectrum; b) wherein an algorithm of the model comprises a convolutional artificial neural network [CNN] or an artificial neural network with transformer technology. Furthermore, the data processing unit can also perform the further steps of the method.

[0057] In principle, real sonar data obtained during training trips, training maneuvers, or in use is suitable as training data. The data should preferably be classified, i.e. labeled, in advance, for example by a trained sonar operator. This means that the actual classification is assigned to the events in the sonar data. This enables supervised learning of the model. Synthetic data, i.e. data generated using technical means, is also suitable for training the model. Synthetic data can be used in particular when the volume of real training data is insufficient for training. In this case, a combination of real and synthetic data can preferably be used for training. The algorithm is also referred to as the architecture. It is also possible to train the model unsupervised.

[0058] Preferred embodiments of the present invention are explained below with reference to the accompanying drawings. It shows: Fig. 1: a schematic block diagram of a data processing unit for acoustic underwater target detection.

[0059] Before exemplary embodiments of the present invention are explained in more detail below with reference to the drawings, it is pointed out that identical, functionally equivalent or equivalent elements, objects and / or structures in the different figures are provided with the same reference numerals, so that the description of these elements shown in different exemplary embodiments is interchangeable or can be applied to one another.

[0060] Fig.1 shows a schematic block diagram of a data processing unit 20 for acoustic underwater target detection. The data processing unit 20, for example a computer, comprises a model 22 and a first interface 24 for obtaining training data. The training data 24 can be converted into a representation thereof, for example a spectrogram, by means of a module 26. Based on the representation 28 of the training data, the model 22 can be trained using machine learning. Furthermore, the data processing unit 20 comprises a second interface 30 for obtaining current sonar data. The current sonar data is also converted into a representation 28 thereof by means of the module 26. Based on the representation of the current sonar data, the model 22 can perform underwater target detection after training and output corresponding watercraft, for example ships or submarines, as targets 32.The term “underwater target detection” refers to the fact that the classification is carried out based on data from the underwater sound transducers of a sonar system, i.e. sensors that are located underwater.

[0061] It should be noted that the training and the actual classification based on current sonar data can also be carried out on two different data processing units.

[0062] The disclosed (water) sound transducers are designed for use underwater, particularly in the sea. The sound transducers can convert water sound into an electrical signal (e.g., voltage or current) corresponding to the sound pressure, the (received) water sound signal. Furthermore, it is possible for the sound transducers to convert an applied electrical voltage into water sound. The electrical voltage can follow a predetermined pattern and then be referred to as the (to be transmitted) sonar signal, while the water sound resulting from the sonar signal to be transmitted is referred to as the (transmitted) sonar signal. Examples of sonar signals are a chirp (frequency-modulated signal) or, as a special case of chirp, a sweep (linear frequency-modulated signal). The sound transducers can therefore be used as water sound receivers and / or as water sound transmitters.The sound transducers can contain a piezoelectric material, such as a piezoceramic, as their sensor material. A plurality of waterborne sound transducers or one or more waterborne sound transducers in conjunction with a signal processing unit can be referred to as a sonar system. The sound transducers can be used for (active and / or passive) sonar (sound navigation and ranging). The sound transducers are preferably not suitable for medical applications or are not used for medical applications. Likewise, the sound transducers are preferably not suitable for ultrasonic testing of materials or are not suitable for this purpose.

[0063] Although some aspects have been described in connection with a device, it is understood that these aspects also represent a description of the corresponding method, so that a block or component of a device can also be understood as a corresponding method step or as a feature of a method step. Similarly, aspects described in connection with or as a method step also represent a description of a corresponding block, detail, or feature of a corresponding device.

[0064] Depending on the implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be carried out using a digital storage medium, for example a magnetic or optical memory, on which electronically readable control signals, e.g. a computer program, are stored, which can interact or interact with a programmable computer system (CPU and / or GPU) in such a way that the respective method is carried out. For this reason, the digital storage medium should be computer-readable. Embodiments can therefore comprise a data carrier having electronically readable control signals capable of interacting with a programmable computer system in such a way that one of the methods described herein is carried out.

[0065] In some embodiments, a programmable logic device (e.g., a field-programmable gate array, an FPGA) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field-programmable gate array may interact with a microprocessor to perform any of the methods described herein. In general, in some embodiments, the methods are performed by any hardware device. This may be general-purpose hardware such as a computer processor (CPU) or a graphics processing unit (GPU), or method-specific hardware such as an ASIC. Distributed execution across the CPU and GPU is also possible.

[0066] The above-described embodiments are merely illustrative of the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be apparent to others skilled in the art. Therefore, it is intended that the invention be limited only by the scope of the following claims and not by the specific details presented in the description and explanation of the embodiments herein. List of reference symbols: 20 Data processing unit 22 Model 24 first interface for receiving training data 26 Module for creating a representation of the training data or current sonar data 28 Representation of training data or current sonar data 30 second interfaces for receiving current sonar data 32 underwater targets

Claims

[1] A method for acoustic underwater target detection comprising the following step: a) applying a model (22) trained by machine learning to a representation (28) of sonar data, wherein the representation of the sonar data is calculated on the basis of a mode decomposition, a spectrogram with a logarithmic representation, in particular the log-mel scale, the gammaton frequency scale or the cepstrum, or has a frequency spectrum; - wherein an algorithm of the model (22) comprises a convolutional artificial neural network [CNN] or an artificial neural network with transformer technology, wherein the representation (28) of sonar data is a multi-channel representation and the calculation - is calculated based on the mode decomposition and the spectrogram with a logarithmic representation; or - is calculated based on the mode decomposition and has the frequency spectrum; or - is calculated based on the spectrogram with a logarithmic representation and has the frequency spectrum. [2] Method according to claim 1, wherein the convolutional artificial neural network [CNN] with transformer technology is used as the algorithm. [3] The method of claim 2, wherein the convolutional artificial neural network includes channel-wise attention modules to emphasize dominant frequency bands containing the most useful information. [4] Method according to one of the preceding claims, wherein the algorithm has a residual neural network [ResNet] as a backbone. [5] Method according to claim 4, wherein the ResNet has a deeper architecture than ResNet 20, in particular ResNet 30 or ResNet 40. [6] Method according to one of the preceding claims, with step a0) before step a): a0): Training the model (22) with a dataset containing a target to be recognized at least twice, each with different environmental conditions. [7] Method according to claim 6, - where the environmental conditions include wind, rain, (water) temperature, diving depth or hydrostatic pressure, salinity, type of water bed, noise from living organisms or turbulence in the water; - where the target to be detected is contained in the data set multiple times with at least three varying environmental conditions. [8] Method according to claim 6 or 7, wherein the training is supervised. [9] Method according to claim 8, wherein the training is carried out predominantly with artificially generated training data. [10] Method according to claim 6 or 7, wherein the training is unsupervised. [11] Method according to claim 10, wherein the training is carried out predominantly with real training data. [12] Method according to one of the preceding claims, wherein the algorithm is based on a combination of the convolutional artificial neural network with a recurrent neural network. [13] Method according to claim 12, wherein a long-short-term memory neural network (LSTM) is used as the recurrent neural network. [14] Method according to one of the preceding claims, wherein the algorithm is based on the convolutional artificial neural network, wherein the convolutional artificial neural network uses different sized kernels in parallel. [15] Method according to one of the preceding claims, wherein three channels of the multi-channel representation have similar spectrograms with different frequency resolutions. [16] A computer program comprising instructions which, when executed by a computer, cause the computer to carry out the method according to any one of the preceding claims. [17] Data processing unit (20) for acoustic underwater target detection, wherein the data processing unit (20) is designed to carry out the following step: a) applying a model (22) trained by machine learning to a representation (28) of sonar data, wherein the representation of the sonar data is calculated on the basis of a mel scale or has a frequency spectrum; - wherein an algorithm of the model comprises a convolutional artificial neural network [CNN] or an artificial neural network with transformer technology, wherein the representation (28) of sonar data is a multi-channel representation and the calculation - is calculated based on the mode decomposition and the spectrogram with a logarithmic representation; or - is calculated based on the mode decomposition and has the frequency spectrum; or - is calculated based on the spectrogram with a logarithmic representation and has the frequency spectrum.

Images