Method for an acoustic underwater target detection

Abstract

The invention relates to a method for an acoustic underwater target detection, having the following step: a) applying a model (22), which is trained by means of machine learning, to a representation (28) of sonar data, the representation of the sonar data being calculated on the basis of a mode decomposition, a spectrogram with a logarithmic representation, in particular the log-mel scale, the gamma tone frequency scale, or the cepstrum, or having a frequency spectrum, and an algorithm of the model (22) comprising a convolutional artificial neural network [CNN] or an artificial neural network with transformer technology.

Description

[0001] Methods for acoustic underwater target detection

[0002] Description

[0003] The invention relates to underwater acoustic target recognition (UATR) based on a model trained using machine learning.

[0004] Underwater Acoustic Target Recognition (UATR) describes techniques for classifying, categorizing, and identifying unknown surface vessels or submarines by analyzing their acoustic signals. Gaining knowledge about the nature of underwater noise sources is applied in maritime security, including monitoring and management of maritime traffic, navigation, surveillance and detection, and the protection of human life. In many areas, remote sensing is primarily conducted using visual or radar methods. However, most of these modalities are unable to penetrate the water column and are therefore unsuitable for underwater remote sensing.

[0005] In contrast, acoustic waves can propagate over long distances underwater. To this day, the classification of unknown seagoing vessels is mostly carried out manually by experienced SONAR operators. High-performance SONAR systems are capable of detecting, locating, and differentiating subtle noise sources. With the increasing number of seagoing vessels due to ongoing globalization, the number of targets that SONAR operators must identify, manage, and classify has steadily risen. Manual evaluation is no longer practical from a purely time- and cost-oriented perspective. Early automated approaches used hand-drawn features and classic statistical analyses.

[0006] The development of automated sonar processing methods has been predominantly geared towards human operators. Consequently, the focus in the past has been on understandable and interpretable, mostly time- and frequency-based features obtained through STFT, LOFAR, and DEMON analysis. These feature representations are typically combined with simple statistical models to classify or categorize a received acoustic signal. Statistical models have proven adequate in data-poor applications and are less prone to overfitting. They are relatively simple, offer understandable parameters and operating principles, and are often used as supportive decision systems for human operators. A Support Vector Machine (SVM) is commonly employed as the classifier.The SVM is a versatile supervised model primarily used for regression and classification tasks. The main goal of the SVM is to find an optimal decision boundary (or hyperplane) that separates the input data points with maximum margin. This margin is based on the Euclidean distance between the data point and the decision boundary.

[0007] The Gaussian Mixture Model (GMM) is another commonly used classification model in UATR. The GMM is a probabilistic model that assumes input samples are generated from a mixture of normal distributions. Its primary goal is to determine distribution parameters such as mean and variance to model the underlying mixed Gaussian distribution. Therefore, the GMM is frequently used in both supervised and unsupervised clustering and classification scenarios. GMMs are highly flexible and can model a wide range of complex data distributions. Furthermore, due to their probabilistic nature, GMMs can model uncertainty and thus provide confidence levels.

[0008] The presented statistical and feature extraction methods demonstrate that classifying and categorizing unknown ships based solely on their emitted sounds is possible. Classical approaches require only a handful of data samples for training and are less prone to overfitting, while still remaining interpretable due to their purely mathematical and statistical nature. However, it has become apparent that a rather sophisticated and highly technical feature extraction process is necessary to extract meaningful patterns from the data. Feature extraction is of paramount importance and demands a high level of expertise and extensive manual work. Due to limited model capacity and the highly compressed information contained within the features, hand-crafted features, in combination with shallow classifiers, are unable to extract deep features from the data.In reality, the properties of underwater acoustics are unable to fulfill the assumptions made when developing the characteristics, leading to insufficient generalizability under different recording conditions. Furthermore, the approaches often exhibit insufficient robustness.

[0009] Therefore, there is a great need for reliable, automatic classification systems.

[0010] The object of the present invention is therefore to create an improved concept for underwater target detection.

[0011] The problem is solved by the subject matter of the independent patent claims. Further advantageous embodiments are the subject matter of the dependent patent claims.

[0012] Exemplary embodiments show a method for acoustic underwater target detection comprising the following steps: a) applying a machine learning-trained model to a representation of sonar data, wherein the representation of the sonar data is calculated based on a mode decomposition, a spectrogram with a logarithmic plot, for example the Mel scale or the cepstrum, or has a frequency spectrum; and b) wherein an algorithm of the model comprises a convolutional artificial neural network [CNN] or an artificial neural network with transformer technology.

[0013] The aim of the model is to classify unknown watercraft, such as ships or underwater vehicles like submarines, based on their passive sonar signature. The classification is based exclusively on the passive sonar signature.

[0014] The machine learning of some algorithms, such as convolutional neural networks (CNNs) and artificial neural networks with Transformer technology, is also referred to as "deep learning".

[0015] Traditionally, UATR systems consist of independent feature extraction and classification processes. Due to the complexity of the oceanic environment, the extraction of meaningful features is a crucial aspect of the ship classification process.

[0016] Digitized acoustic signals are constructed as one-dimensional signals that describe amplitude fluctuations over time. The raw waveform itself contains the highest possible information content of the received signal. Various waveform-based features, such as the zero-crossing rate and the waveform structure itself, can be used in acoustic processing. However, features based solely on the raw temporal waveform are difficult to describe and understand, so derived features are advantageously used. Derived features include, for example, spectral features (i.e., a frequency spectrum) or a spectrogram extracted from the original signal.

[0017] The primary tool for extracting spectral components is the Fourier transform (FT). A characteristic of spectral features, often cited as a drawback, is their inability to represent the temporal context. However, this is usually not a major issue, as sounds from watercraft can often be considered stationary.

[0018] If the temporal context needs to be represented, two-dimensional time-frequency representations, known as spectrograms, can be used. The most fundamental time-frequency representation is the short-time Fourier transform (STFT). The STFT is a time-segmented Fourier transform (FT) over short, successive time frames. Furthermore, it is possible to exploit the harmonic behavior of the spectral components by extracting the cepstrum. The cepstrum can detect periodic structures within the spectrogram and is obtained by the inverse Fourier transform of the logarithmic spectrogram.

[0019] Since most of the energy is concentrated at low frequencies, it can be advantageous to focus on extracting the characteristic narrowband line spectrum generated by the ship's own machinery. A common approach is the extraction of the Low Frequency Analysis Recording (LOFAR) spectrogram. In contrast to narrowband detection, Detection of Envelope Modulation On Noise (DEMON) focuses on extracting the modulation frequencies of the broadband noise caused by propeller cavitation. DEMON analysis is therefore used to extract specific propeller characteristics such as the number of propellers, the number of blades per propeller, and so on. DEMON or LOFAR spectrograms are thus particularly suitable as additional, complementary features.

[0020] Besides the cepstrum, other features also use a logarithmic representation. Examples include features based on the Mel scale, such as the log-Mel spectrogram or the derived Mel Frequency Coefficients (MFCC). Since humans do not perceive sound linearly, the logarithmic Mel scale is often used to mimic human hearing, for example, in the form of a log-Mel spectrogram (LMS). The Mel scale emphasizes lower frequencies over higher ones. A low-dimensional representation that captures the periodic cepstral properties of the Mel spectrogram is usually extracted using Mel Frequency Cepstral Coefficients (MFCC).

[0021] In addition to the MFCC, a similar feature representation uses the gamma-tone frequency scale, rather than the Mel scale, to formulate the gamma-tone spectrogram (GST). Analogous to the Mel approach, the gamma-tone frequency cepstral coefficients (GFCC) can be extracted from the GST. Both MFCC and GFCC employ a set of predefined filters with varying bandwidths and amplitudes to emphasize different spectral ranges of an input signal. Unlike the MFCC, the gamma-tone filter bank uses smooth filters, which allow for greater overlap and thus better correlation between the independent filters. The constant Q transform (CQT) is another time / frequency representation closely related to the log-Mel spectrogram. Similarly, the CQT provides geometrically spaced frequency filters with logarithmically varying resolution.It originates in music processing and therefore aims for the same resolution across all octaves.

[0022] All spectrum- or cepstrum-based approaches are capable of capturing energy, frequency, and time information. The foundations of these feature representations rest on the Fourier transform, which describes a complex wave structure as a sum of superimposed sine waves of infinite length. This periodic property of the sine wave leads to insufficient temporal localization capability. Furthermore, this property is emphasized by the fixed window length of the STFT, in which low and high frequencies are resolved equally over time.

[0023] The wavelet transform (WT) enables better localization in both time and frequency, allowing for more precise analysis of non-stationary signals. It avoids the conflict between low and high frequency resolution by selecting an appropriate window length. The wavelet transform adapts to the signal by using shorter window lengths for high frequencies and shorter window lengths for low frequencies, resulting in a more effective representation of a wide range of signal dynamics. This is particularly advantageous for analyzing signals with transient components and varying frequencies. However, the wavelet transform still requires a priori information for selecting a suitable parent wavelet, making it somewhat complex to use.

[0024] Furthermore, it is possible to use mode analysis, for example as part of the Hilbert-Huang Transform (HHT), to analyze underwater acoustic signals. HHT is an adaptive approach that extracts subtle oscillator-like features from a signal. It focuses only on a specific group of frequencies that exhibit a high energy response. HHT's main strengths lie in the analysis of nonlinear and non-stationary signals. It uses Empirical Mode Decomposition (EMD) to decompose a signal into eigenmode functions (IMFs). Frequency-time analysis is performed by applying a Hilbert transform to the mode functions. HHT is preferred to Wavelet Transform (WT) when instantaneous frequencies are of great interest, as WT requires a certain degree of frequency stability over the time span of the wavelet.

[0025] The choice of the various aforementioned time-frequency representations depends heavily on the assumption of whether the ship noise is considered stationary or non-stationary. This assumption is not easy to make, as it strongly depends on the specific application. If constantly moving and distant ships are being studied, the noise is more likely to be stationary, while strong deceleration and acceleration movements, as well as processes like engine starts, are more likely to imply the presence of a time-varying signal.

[0026] Generating the sonar data representation can include further signal processing steps in addition to the actual feature extraction, such as smoothing the sonar data. The sonar data representation can also be multi-channel and comprise several channels with features. That is, the multi-channel representation can include or consist of any combination of the sonar signal's mode decomposition, the spectrogram with a logarithmic plot of the sonar signal, the sonar signal's frequency spectrum, and other sonar signal features. These additional sonar signal features can include any selection of time derivatives, such as the first and / or second time derivative of the sonar signal, or photometric representations such as chroma and / or contrast and / or tone grid and / or zero-crossing rate.

[0027] Approaches based on machine learning, particularly deep learning, have been shown to improve the generalization capabilities of underwater acoustic target detection while delivering robust results. Specifically, among deep learning approaches, convolutional neural networks (CNNs) and artificial neural networks using transformer technology have demonstrated excellent results. Deep learning is a rapidly developing field of research with numerous applications across various disciplines. However, SONAR processing is traditionally a statistical analysis area, which is why machine learning for vessel classification is still in its infancy.

[0028] Furthermore, it has been shown that among the deep learning approaches, the convolutional artificial neural network and the artificial neural network with transformers have each delivered very good results. It is also possible to combine several algorithms, such as the two mentioned above, i.e., to run them in parallel or sequentially. This allows for the creation of sub-models whose results can be fused with another model, for example, to obtain the final classification.

[0029] Deep neural networks have been shown to be more effective than shallow networks using SONAR data at capturing deep hierarchical feature representations. A network can be considered "deep" if it has at least 3, preferably at least 5, and preferably at least 8 or at least 15 hidden layers. Despite the performance increase, problems such as gradient fall complicate the training process of deep neural networks. This can be mitigated by using a convolutional neural network (CNN) with a long-short-term memory neural network (LSTM), which reduces the impact of gradient fall-off during the training process of typical recurrent neural networks. That is, the algorithm can be based on a combination of the convolutional neural network (CNN) and a long-short-term memory neural network (LSTM).

[0030] In exemplary implementations, the algorithm uses a residual neural network [ResNet] as its backbone. In particular, the ResNet has a deeper architecture than ResNet 20, especially ResNet 30 or ResNet 40. Furthermore, it has been found that using a backbone for the deep learning model in the UATR domain is advantageous. A residual neural network (ResNet) has proven suitable as the backbone. Deeper architectures, such as ResNet 50, have proven advantageous compared to shallower architectures, such as ResNet 18, in extracting semantically meaningful features. If appropriate expertise is invested in feature selection, even smaller architectures, such as ResNet, are capable of achieving comparable results to, for example, the ResNetW architecture.

[0031] Exemplary embodiments show that step aO) is performed before step a). aO): Training the model with a dataset containing a target to be recognized at least twice, each time with different environmental conditions. Preferably, the environmental conditions include wind, rain, (water) temperature, immersion depth or hydrostatic pressure, salinity, type of seabed, sounds from living organisms, or turbulence in the water. Furthermore, the target to be recognized is included multiple times in the dataset with at least three varying environmental conditions. This results in a dataset with a certain degree of variability.

[0032] Preferably, a data set for each detectable target has at least 5, preferably at least 10, more preferably at least 15 or at least 20 different environmental properties (permutations of environmental properties), in particular caused by at least 2, preferably at least 4, more preferably at least 6 different environmental conditions. An environmental condition can be subdivided into different environmental properties. Wind as an environmental condition can be entered into the levels of the Beaufort scale or an alternative scale. One environmental property can be present per scale value. That is, a different environmental property exists if the detectable target is recorded or detected with wind at two different Beaufort levels. Rain can be considered binary (yes / no), so that two environmental properties are present. However, it is also possible to subdivide rain even further. Water temperature, for example, can be...The angle of inclination can be divided into 5-degree increments between 5° and 35°. Diving depth can be further subdivided into reasonable sections depending on the operational area. Salinity, for example, can be categorized into three different levels. The same applies to the type of seabed. Sounds from living organisms can be divided into constant sounds, such as schools of fish or crab sounds, and time-varying sounds, such as whale or dolphin songs. A multitude of permutations arise from the various environmental properties. The more permutations of combinations of environmental properties a dataset contains for a given target, the more effective the training for that target.

[0033] The primary objective of UATR is to differentiate targets by analyzing their emitted acoustic noise. Unfortunately, the received acoustic signal, particularly its spectral characteristics, is influenced not only by the target vessel's design (e.g., ship type) and operating conditions, but also by various environmental factors. Therefore, for accurate generalization, it is crucial to use a training dataset with the greatest possible variability. Emitted ship noise follows a complex generation mechanism. Ship noise is typically predominantly mechanical, propeller, and hydrodynamic in origin. That is, ship noise is primarily composed of these three main sources.

[0034] Mechanical noise describes the sounds generated by the propulsion system, machinery, transmission, and other mechanical actuators on or inside the vessel. Propeller noise describes the modulation of the broadband cavitation noise caused by propeller rotation. Hydrodynamic noise arises from interface interactions between the vessel's hull and the marine environment. The combination of these noise sources results in unique acoustic fingerprints for each vessel.

[0035] A sound wave propagating through water can be subject to both dynamic effects that alter the properties of the emitted wave and additive effects such as background noise and sounds from other sources. The nature and extent of these effects depend heavily on the specific environment, the geographical region, the season, and the design of the target and receiving platforms. These high-level influences are further subdivided into intermediate influences. Some of these characteristics are controllable, while others are not. When creating a dataset, one could influence the controllable parameters, while the non-controllable parameters can only be influenced indirectly or not at all. The individual effects are described below.

[0036] An ideal dataset for underwater acoustics should account for sufficient variability in all influencing factors to ensure adequate complexity coverage. Like any mechanical wave that transports energy through a medium, the most important quantities for describing underwater sound waves are amplitude, frequency, and phase. Amplitude is a general term for the pressure received in a given area. Analogous to any physical wave, underwater acoustic waves are subject to interference, attenuation effects, and frequency and phase shifts. Despite the fundamental similarities, many approaches from classical acoustics are not applicable to underwater acoustics, primarily due to the inhomogeneous nature of the oceanic environment. Underwater acoustic signals are subject to various propagation and transmission losses, as well as additive effects such as ambient noise.Regular inhomogeneities, such as time-varying underwater and surface channels, as well as antiwaveguides like shadow zones, are caused by differing sound velocity profiles. The sound velocity profile typically depends on water temperature, salinity, and pressure, and is therefore dependent on the geographical region and time. Transmission loss refers to the reduction in received signal strength as a wave travels through and interacts with a medium.

[0037] The received pressure is influenced by many different effects. Primarily, every mechanical wave is absorbed by the propagation medium, with some of the energy being absorbed as heat when the wave interacts with molecules. This effect increases with greater propagation distances. Secondly, transmission losses such as scattering, geometric scattering, and reflections reduce the amplitude. Scattering losses describe the redistribution of energy when acoustic signals scatter by particles, bubbles, and other irregularities in the water column. Since acoustic waves propagate omnidirectionally, geometric losses occur where the energy is scattered spherically. In the specific case of an underwater acoustic channel, the energy is distributed cylindrically. Reflection losses occur at the boundaries of media with different propagation velocities, such as the seabed or the water surface.This leads to a weakening of the signals, as portions of the energy are either transmitted or reflected. Finally, the mechanical properties and operating conditions of the transducer can also alter the radiated amplitude.

[0038] All of the aforementioned effects contribute to the decrease in signal strength. Underwater acoustics is generally considered a low signal-to-noise ratio (SNR) scenario. The actual transmission losses are mainly comparable to spherical scattering. Spherical scattering exhibits a characteristic with r 2Decreasing intensity and transmission losses of 20 log(r), where r is the distance from the source, are observed. Furthermore, the propagation of underwater sound signals is typically accompanied by several frequency shift effects. Primarily, relative dynamic movements between the signal source and receiver cause frequency shifts due to the Doppler effect. Secondly, ship operating conditions such as engine speed, gear selection, etc., as well as general ship conditions, contribute to the radiated frequency spectrum to varying degrees. The dissipation and absorption of signal energy is typically frequency-dependent, with higher frequencies being attenuated more than lower frequencies. Reflection, scattering, and refraction of an incident wavefront at interfaces cause frequency-dependent generations of secondary wavefronts that superimpose on the original wave.Finally, additive effects such as ambient noise contribute additional frequency components to the spectrum of a received signal. Furthermore, the frequency-dependent attenuation of sound waves varies with the water temperature.

[0039] Finally, the received phase of an underwater acoustic signal can be distorted or altered by various effects. Acoustic turbulence, such as local inhomogeneities in the water column, causes fluctuations in the speed and direction of the sound wave. Boundary reflections at the seabed, the surface, or submerged objects can cause phase shifts. Dynamic characteristics, such as the movement of the transmitter and receiver, also alter the phase component. Finally, acoustic signals can take multiple propagation paths due to reflection, refraction, and scattering at the surface and seabed.

[0040] This results in different paths and thus distances that the acoustic signal can travel from the transmitter to the receiver, which in turn leads to phase shifts. Multipath propagation occurs particularly in shallow water regions and can add considerable temporal complexity to the signal. Finally, the characteristics of the transducer and the receiver also influence the received phase due to design, assembly, and processing decisions. In addition to the aforementioned losses and changes caused by wave propagation in water, various additive contributions from ambient noise affect the signal-to-noise ratio (SNR) and the specificity of a received signal.

[0041] Ambient noise can be broadly divided into the following eight categories. At low frequencies (0.1–10 Hz), turbulence noise is present, resulting from changes caused by the ocean, atmosphere, and geophysical noise sources such as underwater volcanic eruptions and earthquakes. In the frequency band between 10 and 300 Hz, most noise is caused by long-distance shipping. The significant increase in commercial shipping traffic and the low attenuation of sound at these frequencies create a continuous background noise. The main sources of ship noise are the propulsion engines, propeller interactions in the water, and flow noise caused by water movement against the ship's hull. This type of noise is of paramount interest to the UATR. The state of the sea itself is also a significant source of noise, involving many mechanisms.This includes, above all, the breaking of waves at the sea surface and the associated cavitation noise of collapsing bubbles. This effect is strongly related to wind conditions. Other weather conditions, such as rain, also generate broadband noise at the surface, typically in the range of 1 kHz to 5 kHz. In addition, biological noise generated by marine fauna can have a significant impact on the noise level in certain regions and during certain seasonal periods. Furthermore, the random movement of atoms and molecules due to thermal energy, also known as molecular agitation or thermal noise, generates a noise spectrum in the range of 100 kHz to 1 MHz. Finally, uncontrollable, human-caused noise contributions, such as...

[0042] Seabed mining / drilling, active SONAR systems, etc., can be present across a very broad frequency spectrum. All of the aforementioned noise sources contribute to the isotropic noise level in different ways.

[0043] Despite the many different influences on a propagating underwater acoustic wave, some key characteristics of ship-emitted noise are evident. Ship-radiated noise consists of a broadband, continuous spectrum with discrete, narrowband frequency components. Propeller cavitation and the ship's flow noise generate the broadband component. The ship's internal machinery and propulsion systems are the primary sources of distinct, narrowband spectral lines. While target information is predominantly found in low frequencies, particularly below 5 kHz, the high-frequency components also contain target information. Most known approaches focus either on determining the low-frequency line spectrum or the high-frequency modulation caused by the target's propeller.One approach to achieving better classification is the use of a multi-channel representation of the sonar data. For example, one channel could contain a feature that better highlights the low-frequency line spectra, while another channel could contain a feature that better highlights the high-frequency modulation. The model can thus perform the classification for both characteristics of vessels, resulting in a better classification outcome. While varying operating conditions of ships can lead to temporal irregularities, ship sounds are generally considered to be static for short periods, particularly in the range of 10–30 ms, sometimes up to 100 ms. Nevertheless, short-term fluctuations, such as engine start-up, deceleration, and acceleration of the target vessels, can provide useful ship-specific information.

[0044] As previously mentioned, the UATR area is considered a low SNR scenario, where many of the factors described in this section reduce the effective amplitude of a propagating acoustic wave. Despite the distinct characteristics that can be used to categorize unknown vessels, ship-radiated noise is still subject to significant class-specific variance, with targets of the same category varying considerably in terms of speed, tonnage, propulsion system, and propeller characteristics.

[0045] In some implementations, training can be supervised (supervised learning). Specifically, training can then be performed using artificially generated training data. Artificially generated training data has the advantage that the events to be detected within the data can be directly labeled with the correct classification result. While this is possible with real sonar data, it is very time-consuming. Especially when using an artificial neural network with transformer technology, very large datasets are required for training. Manually labeling the sonar data in such cases is extremely time-consuming.

[0046] Alternatively, training can be unsupervised. In particular, this allows the training to be performed primarily with real-world training data. In unsupervised learning, the model is trained to classify the training data into different categories. Similar data then belong to the same category. These categories can then be labeled manually. This almost completely reduces the computational effort required by the modeler. As mentioned earlier, ship noise consists of narrowband and broadband frequency components. Capturing broadband frequency relationships contrasts with the local feature extraction of CNNs. Using large convolutional kernels and very deep networks can overcome this problem, but at the cost of computationally intensive (time / power) calculations and difficult training.It is advantageous here to use cores of different sizes in parallel to extract features with varying resolutions, thereby increasing robustness when acquiring complex broadband targets. For example, a basic block module based on separate fields of the input spectrum can be used, with a multi-layer perceptron (MLP) capturing the globally distributed features across these fields. Furthermore, multiple dilated cores of different sizes can be used to increase the receptive field without significantly increasing the computational cost.

[0047] In addition to model-driven solutions, it is also possible to expand the receptive field from the perspective of the data. For example, three different, similar spectrograms with different frequency resolutions can be fused as input for the model in a multi-channel array (multi-channel representation). That is, the representation is a multi-channel representation, where three channels of the multi-channel representation exhibit similar spectrograms with different frequency resolutions. All spectrogram types mentioned in this disclosure are suitable.

[0048] When using a convolutional artificial neural network (CNN) in conjunction with a transformer, attentional mechanisms can be introduced into the CNN to extract globally distributed features in the frequency domain. This allows for the introduction of channel-wise attentional modules capable of highlighting dominant frequency bands containing the most useful information. These attentional modules can overcome the limitations of flat networks. Similarly, the receptive field of the CNN can be expanded by modeling the received time sequence using recurrent neural networks (RNNs). In this case, it is advantageous to use features distributed across the time domain rather than the frequency domain.Long-short-term memory neural networks (LSTMs) are particularly suitable, as they reduce the influence of gradient decay during the training process of conventional recurrent neural networks. Time-correlated features can be effectively extracted from time-frequency plots by introducing recursion. These time-correlated features can also be extracted from the time-domain signal.

[0049] Furthermore, to overcome the small size of CNNs, it is possible to circumvent the use of convolution operators by employing purely attention-based approaches, such as a transformer. These purely attention-based approaches demonstrate that comparable results can be achieved without the use of convolution operators. In contrast to the attention-heavy channels mentioned above, the transformer utilizes self-attention, which effectively correlates all input tokens, such as the frequency bands of a spectrogram. For example, a Vision Transformer (ViT) can be used. The transformer-based approaches can rely on a self-attention mechanism with a quadratic computational complexity of O(N²D), where N is the sequence length and D is the dimension of the feature vector.However, it is also possible to use an additive attention mechanism with linear complexity O(ND) to reduce the computational effort.

[0050] Automatic feature coding can be achieved through layer-by-layer unsupervised training. Target recognition was performed by passing the encoded feature vector through a backpropagation network. Similarly, unsupervised pre-training approaches combining CNN and LSTM can be used to extract meaningful features without requiring annotated data. Furthermore, a contrastive learning approach can be employed. For example, a supervised version of SimCLR, a popular contrastive learning approach from the field of image processing that focuses on self-supervised learning, can be used. These contrastive approaches enable superior intra-class accumulation and inter-class separability.

[0051] Furthermore, a computer program is disclosed, comprising instructions which, when the program is executed by a computer, cause it to perform the procedure.

[0052] Analogously, a data processing unit for acoustic underwater target detection is disclosed, wherein the data processing unit is configured to perform the following steps: a) applying a machine learning-trained model to a representation of sonar data, wherein the representation of the sonar data is calculated based on a Mel scale or has a frequency spectrum; b) wherein an algorithm of the model comprises a convolutional artificial neural network [CNN] or an artificial neural network with transformer technology. Furthermore, the data processing unit can also perform the further steps of the procedure.

[0053] In principle, real sonar data acquired during training runs, training maneuvers, or in the field are suitable as training data. Ideally, the data should be pre-classified, i.e., labeled, for example, by a trained sonar operator. This means that the events in the sonar data are assigned the actual classification. This enables supervised learning of the model. Synthetic data, i.e., data generated using technical means, are also suitable for training the model. Synthetic data can be used particularly when the amount of real training data is insufficient. In this case, a combination of real and synthetic data is preferably used for training. The algorithm is also referred to as the architecture. Furthermore, it is possible to train the model unsupervised.

[0054] Preferred embodiments of the present invention are explained below with reference to the accompanying drawing. It shows: Fig. 1: a schematic block diagram of a data processing unit for acoustic underwater target detection.

[0055] Before exemplary embodiments of the present invention are explained in detail below with reference to the drawings, it should be noted that identical, functionally equivalent or equivalent elements, objects and / or structures in the different figures are provided with the same reference numerals, so that the description of these elements shown in different exemplary embodiments is interchangeable or can be applied to one another.

[0056] Fig. 1 shows a schematic block diagram of a data processing unit 20 for acoustic underwater target detection. The data processing unit 20, for example a computer, comprises a model 22 and a first interface 24 for receiving training data. The training data 24 can be converted into a representation of the data, for example a spectrogram, by means of a module 26. Based on the representation 28 of the training data, the model 22 can be trained by machine learning. Furthermore, the data processing unit 20 comprises a second interface 30 for receiving current sonar data. The current sonar data is also converted into a representation 28 of the same by means of the module 26. Based on the representation of the current sonar data, the model 22 can perform underwater target detection after training and output corresponding watercraft, for example ships or submarines, as targets 32.The term "underwater target detection" refers to the fact that the classification is carried out based on data from underwater sound transducers of a sonar system, i.e. sensors that are located underwater.

[0057] It should be noted that the training and the actual classification based on current sonar data can also be carried out on two different data processing units.

[0058] The disclosed (underwater) sound transducers are designed for use underwater, particularly in the sea. The transducers can convert underwater sound into an electrical signal (e.g., voltage or current) corresponding to the sound pressure, the (received) underwater sound signal. Furthermore, it is possible for the transducers to convert an applied electrical voltage into underwater sound. The electrical voltage can follow a predefined pattern and then be referred to as the (transmitted) sonar signal, while the underwater sound resulting from the sonar signal to be transmitted is referred to as the (transmitted) sonar signal. Examples of sonar signals are a chirp (frequency-modulated signal) or, as a special case of the chirp, a sweep (linearly frequency-modulated signal). The transducers can therefore be used as underwater sound receivers and / or underwater sound transmitters.The transducers can be made of a piezoelectric material, such as a piezoceramic, to serve as the sensor material. A plurality of underwater transducers, or one or more underwater transducers in conjunction with a signal processing unit, can be referred to as a sonar system. The transducers can be used for (active and / or passive) sonar (sound navigation and ranging). The transducers are preferably not suitable for, or are not used for, medical applications. Likewise, the transducers are preferably not used for, or are not suitable for, the ultrasonic testing of materials.

[0059] Although some aspects have been described in connection with a device, it is understood that these aspects also constitute a description of the corresponding process, so that a block or component of a device can also be understood as a corresponding process step or as a feature of a process step. Similarly, aspects described in connection with or as a process step also constitute a description of a corresponding block, detail, or feature of a corresponding device.

[0060] Depending on the implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be carried out using a digital storage medium, for example, a magnetic or optical storage medium, on which electronically readable control signals, e.g., a computer program, are stored. These signals can interact with a programmable computer system (CPU and / or GPU) in such a way that the respective method is carried out. For this reason, the digital storage medium should be computer-readable. Embodiments can therefore include a data carrier that has electronically readable control signals capable of interacting with a programmable computer system in such a way that one of the methods described herein is carried out.

[0061] In some embodiments, a programmable logic device (for example, a field-programmable gate array, an FPGA) can be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field-programmable gate array can interact with a microprocessor to perform one of the methods described herein. Generally, in some embodiments, the methods are performed by any hardware device. This can be general-purpose hardware such as a computer processor (CPU) or a graphics processing unit (GPU), or hardware specific to the method, such as an ASIC. Distributed execution across the CPU and GPU is also possible.

[0062] The embodiments described above merely illustrate the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be obvious to other people skilled in the art. Therefore, it is intended that the invention be limited only by the scope of protection set forth in the following claims and not by the specific details presented herein by way of description and explanation of the embodiments. List of reference numerals:

[0063] 20 Data processing units

[0064] 22 Model 24 first interface for receiving training data

[0065] 26 Module for creating a representation of the training data or current sonar data

[0066] 28 Representation of training data or current sonar data

[0067] 30 Second interface for receiving current sonar data; 32 Underwater targets

Claims

Patent claims 1. Method for acoustic underwater target detection comprising the following step: a) Applying a machine learning trained model (22) to a representation (28) of sonar data, wherein the representation of the sonar data is calculated on the basis of a mode decomposition, a spectrogram with a logarithmic representation, in particular the log-mel scale, the gamma tone frequency scale or the cepstrum, or has a frequency spectrum; - wherein an algorithm of model (22) includes a convolutional artificial neural network [CNN] or an artificial neural network using Transformer technology.

2. Method according to claim 1, wherein the representation (28) of sonar data is a multi-channel representation and the calculation - calculated based on the mode decomposition and the spectrogram with a logarithmic representation; or - is calculated based on the mode decomposition and exhibits the frequency spectrum; or - is calculated based on the spectrogram with a logarithmic representation and exhibits the frequency spectrum.

3. Method according to one of the preceding claims, wherein the convolutional artificial neural network [CNN] with Transformer technology is used as the algorithm.

4. Method according to claim 3, wherein the convolutional artificial neural network includes channel-wise attention modules to highlight dominant frequency bands that contain the most useful information.

5. Method according to one of the preceding claims, wherein the algorithm comprises a residual neural network [ResNet] as a backbone.

6. Method according to claim 5, wherein the ResNet has a deeper architecture than ResNet 20, in particular ResNet 30 or ResNet 40.

7. Method according to one of the preceding claims, with step aO) prior to step a): aO): Training the model (22) with a data set containing a target to be recognized at least twice with different environmental conditions.

8. Method according to claim 7, - where environmental conditions include wind, rain, (water) temperature, diving depth or hydrostatic pressure, salinity, type of seabed, sounds of living beings or turbulence in the water; - where the target to be identified is included multiple times in the dataset with at least three varying environmental conditions.

9. Method according to claim 7 or 8, wherein the training is supervised.

10. Method according to claim 9, wherein the training is predominantly carried out with artificially generated training data.

11. Method according to claim 7 or 8, wherein the training is unsupervised.

12. Method according to claim 11, wherein the training is predominantly carried out with real training data.

13. Method according to one of the preceding claims, wherein the algorithm is based on a combination of the convolutional artificial neural network with a recurrent neural network.

14. Method according to claim 13, wherein a long-short term memory neural network [LSTM] is used as the recurrent neural network.

15. Method according to one of the preceding claims, wherein the algorithm is based on the convolutional artificial neural network, wherein the convolutional artificial neural network uses kernels of different sizes in parallel.

16. Method according to one of the preceding claims, wherein the representation is a multi-channel representation, wherein three channels of the multi-channel representation have similar spectrograms with different frequency resolutions.

17. Computer program comprising instructions which, when the program is executed by a computer, cause it to execute the method according to one of the preceding claims.

18. Data processing unit (20) for acoustic underwater target detection, wherein the data processing unit (20) is configured to perform the following step: a) Applying a machine learning-trained model (22) to a representation (28) of sonar data, wherein the representation of the sonar data is calculated on the basis of a Mel scale or has a frequency spectrum; - where one algorithm of the model includes a convolutional artificial neural network [CNN] or an artificial neural network using Transformer technology.

Images