Method for determining features for classifying vehicles in a maritime environment
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2026-02-05
- Publication Date
- 2026-08-13
Smart Images
Figure EP2026053056_13082026_PF_FP_ABST
Abstract
Description
[0001] Methods for determining characteristics for classifying vehicles in the maritime environment
[0002] Description
[0003] The invention relates to the generation of features for classifying vehicles in a maritime environment. The vehicles are located in a maritime environment. This means, for example, watercraft (e.g., ships, submarines, each manned or unmanned) or aircraft over water or on the coast. For improved readability, the term "vehicle" will be used only hereafter.
[0004] Traditionally, watercraft are classified by sonar operators, i.e., people. These operators use both auditory sonar signals (i.e., they listen to the signals) and various visual representations of the sonar signals. LOFAR analysis (LOFAR: Low Frequency Analysis and Recording) and / or DEMON analysis (DEMON: Demodulation Envelope Modulation On Noise) of the sonar signals, especially directional sonar signals, have proven to be valuable visual aids.
[0005] However, the classification of watercraft by sonar operators is not of consistent quality. Human error plays a role not only in the classification itself, but also in the settings of parameters such as audio playback or the visual display of sonar signals. This can lead to failures in the detection of watercraft altogether. Initial trials have shown that a machine learning-trained model based on LOFAR and DEMON features delivers acceptable results. However, there is a desire to improve these results.
[0006] The object of the present invention is therefore to create an improved concept for the classification of vehicles in the maritime environment. This object is achieved by the subject matter of the independent claims. Further advantageous embodiments are the subject matter of the dependent claims.
[0007] A method for determining characteristics for classifying vessels in the maritime environment using a deep learning model is disclosed. A deep model is based on a multitude of hidden layers. Artificial neural networks are commonly used as deep models. Examples include artificial neural networks based on transformer technology, convolutional artificial neural networks, and / or recurrent neural networks (RNNs). Classification refers to the differentiation of vessels into different classes. For example, a Type 212A submarine can be identified as such and distinguished from submarines of other classes. This identification also allows for the differentiation of essentially identical vessels within each class.The model provides a set of instructions for processing the sensor data. The result is an output signal corresponding to the sensor data input to the model; this signal is called the feature, or usually a multitude of features, the feature vector.
[0008] The procedure, in step a), involves providing a large number of sensor data packets comprising or consisting of active sonar or radar data, with each sensor data packet being assigned to a first category and a second category. Sensor data sets are then created from these packets, with each sensor data set containing a pair of sensor data packets from the first category and the second category. That is, each pair of sensor data packets consists of one sensor data packet from the first category and one sensor data packet from the second category. A sensor data packet is defined as a segment of the sensor signals. This segmentation can be achieved, for example, by applying a window to the sensor signal. Active sonar data are reflections of a known transmitted sonar signal, often referred to as a ping.Reflections from the vehicle can be classified based on the vehicle-specific smearing effect on its extended shape. In other words, the signal travel times to the reflection points on the vehicle differ, resulting in vehicle-specific reflections. The same applies analogously to a received reflection of a transmitted radar signal. Active sonar also includes bi-static and multi-static techniques. However, incorporating bi-static and multi-static methods typically increases the size, i.e., the amount, of training data. This is because the specific received reflections need to be recorded and trained not just for a single transmitter-receiver position, but for a multitude of possible transmitter and receiver positions in space.Furthermore, it is advantageous for both conventional active sonar and bistatic or multistatic methods to consider different vehicle orientations, i.e., to train the deep model accordingly. This allows the detection of features that are only meaningful for specific orientations of the vehicle relative to the transmitter and / or receiver (sonar or radar). These specific reflections can also be referred to as the active signature.
[0009] Advantageously, each sensor data packet is used multiple times in different categories. A sensor data packet can therefore be assigned to the first and second categories multiple times. However, care should be taken to ensure that, in particular, the sensor data packet pairs of the first and second categories are used only once within an epoch (su). The sensor data can be real, recorded sensor data. For example, sensor data from publicly accessible databases can be used. It is also possible to use artificially generated sensor data. Advantageously, the sensor data, i.e., also the sensor data packets, are time series, i.e., digitized audio files or time series of the received radar signals.
[0010] The first and second sensor data packets of the sensor dataset each exhibit different reflections from the same vehicle to be classified. The sensor data packets of the first category can be referred to as "positive examples." The sensor data packets of the second category can be referred to as "anchors." For example, the sensor data packets of the first category can be obtained by processing the sensor data packets of the second category or the associated sensor data. Specifically, the sensor data packets are obtained within the same sensor dataset through this processing. The processing can involve shifting the section of the sensor signal and / or adding noise or other interference. This processing process is also referred to as augmentation.
[0011] In step b), the deep model is trained with a large number of sensor datasets to reduce, and in particular minimize, the difference between the model's output values using the sensor data packages of the first and second categories, and to increase, and in particular maximize, the difference between the model's output values using the sensor data packages of the second category and one or more sensor data packages from different categories. Such training is also known as "contrastive learning." Furthermore, it is not necessary to label the sensor data packages if they are assigned to categories. Such training is also known as "self-supervised learning." An iteration is defined as training with a predetermined number of sensor datasets, or optionally a single dataset, before the model is optimized.The data sets that make up the predetermined number of sensor data sets are also referred to as a "batch".
[0012] The sensor data packets of one or more sensor data packets different from the sensor data set can also be referred to as third-category sensor data packets or as additional sensor data packets. To reduce the required amount of data, these can be sensor data packets that are already assigned to the first and / or second category in other sensor data sets. However, it is also possible to use sensor data packets that are not used in other sensor data sets. Advantageously, the sensor data packets of the predetermined number of sensor data sets are presented to the model consecutively, i.e., one after the other, and the output values are stored in each case. A mapping of the output values to the corresponding input values, i.e., the sensor data packets, is possible. Optionally, the sensor data packets of the first and second categories can be presented alternately; however, it is not possible, for example, to...It is also possible to first present the sensor data packets of the first category of data records of the predetermined number of data records (i.e., the data records of a batch) and then the sensor data packets of the second category of these data records.
[0013] In principle, it is also possible, and encompassed by the term "training the model," to train two or, as shown in Fig. 3, even three identical models in parallel. This saves time during model training but requires significantly more computing power.
[0014] The sensor data packets in the first and second categories each exhibit reflections from different vehicles or simply noise compared to the other sensor data packets. The other sensor data packets can be described as "negative examples".
[0015] To reduce the difference between the model's output values using the first and second category sensor data packets, and to increase the difference between the model's output values using the second category sensor data packets and one or more sensor data packets from different categories, it is possible to determine a mathematical distance between the output values that is maximized or minimized, respectively. For example, algorithms based on cosine similarity or the InfoNCE loss function can be used as the mathematical distance. The Adam W algorithm, for instance, can be used as the optimization algorithm.
[0016] One idea is to train a deep model to generate features for the automatic classification of vessels in maritime environments based on active sonar or radar data. Vessel classification based on active signals, whether radar or active sonar, is currently unknown. Only classification based on passive sonar data is known. This is primarily because the differences in the distortion of the active signal due to reflection from different vessels are invisible to humans and cannot be physically represented. However, the deep model generates features to which humans cannot assign physical properties. Experiments have shown that very good classification results can be achieved using these features in conjunction with any machine learning-trained model. Furthermore, it is now possible, for example, to...For a submarine operating at periscope depth and equipped with radar, or for a ship, it is possible to classify aircraft using radar over long distances.
[0017] In exemplary embodiments, the model features a convolutional neural network (CNN), in particular one or more one-dimensional residual networks (1d-ResNets). 1d-ResNets are particularly suitable for processing time series and have achieved very good results in this case as well. Specifically, a cascade of 1d-ResNets is formed. The cascade does not have to be continuous and can be interrupted, for example, by averaging values or other (computational) operations.
[0018] Exemplary embodiments show that the model incorporates downsampling. Downsampling consolidates feature vectors, meaning that important information is emphasized and unimportant information is reduced or completely eliminated. Preferably, downsampling is also performed using one or more one-dimensional residual networks. In particular, an initial [O-th] set of feature vectors can be constructed using a one-dimensional residual network. Based on this initial set of feature vectors, further sets of feature vectors can be determined, with the number of feature vectors being increased at each stage by means of N-stage downsampling and the values per feature vector being reduced. Advantageously, the number of feature vectors in the N-th set is reduced to a predetermined number of feature vectors using one or more one-dimensional residual networks.In some implementation examples, the number of values per feature vector in the hidden layers can be reduced to one value per feature vector by averaging if the number of values per feature vector before averaging, for example after downsampling, is greater than one. Thus, the feature vectors are available in the desired single-column or single-row vector format.
[0019] In further embodiments, the method includes step c) classifying the vehicle in the maritime environment using the features obtained in step b). In particular, a machine learning-trained model is used for classification.
[0020] In one embodiment, a mixed dataset of sonar or radar data from various maneuvers can be used for the (consecutive) training of the deep-sea model. The resulting files, i.e., the input data, advantageously each have the same length, i.e., the same number of samples. The model can be trained on this combined dataset in multiple training runs. A run is called an "epoch." Since the combined dataset would consume a significant amount of memory in the processing unit, it is divided into "batches." A batch contains a multitude of sensor datasets. A GPU (Graphics Processing Unit) or a CPU (Central Processing Unit) is preferably suitable as the processing unit.
[0021] A batch size of, for example, 128 means that the model is run twice (once with the sensor data packages of the first category (positive vector) and once with the sensor data packages of the second category (anchor vector)) to obtain two sets of 128 feature vectors, each with K values. Now, the individual sensor data sets, i.e., each pair of sensor data packages (anchor / positive) from this batch, can be examined. The distance between the two output values belonging to the sensor data package of the first category and the sensor data package of the second category of the pair of sensor data packages is reduced, for example, using the information loss algorithm. The remaining sensor data packages of the second category and / or the first category, in this case 127, can be used as negative vectors, i.e., one or more sensor data packages different from the sensor data set.The distance is determined for all sensor data sets in the batch. The results can be averaged. The averaged results are referred to as the loss. The loss is backpropagated through the network to create a gradient for each model weight, indicating how that weight would need to change to minimize the loss. The weight change is advantageously performed by an optimizer, such as the AdamW algorithm. That is, after each batch, the model is optimized once until the epoch is complete. This process can be repeated for a large number of epochs.
[0022] Analogous to the method, a computer program comprises instructions which, when the program is executed by a computer, cause it to execute the method disclosed.
[0023] Furthermore, a sensor system for classifying vehicles in a maritime environment is disclosed. The sensor system comprises an active sonar system for generating active sonar data or a radar system for generating radar data, as well as a signal processing unit. The signal processing unit can perform the following steps:
[0024] a) Processing a representation of the active sonar data or the radar data with a deep model trained using the method according to any one of claims 1 to 12 or the computer program according to claim 13 to obtain a plurality of features;
[0025] b) Processing the features with a machine learning-trained model to classify the known vehicles.
[0026] The signals can be represented, for example, by digitization and / or conventional signal processing of the (digitized) active sonar signals or radar signals. In exemplary embodiments, the signal processing unit is configured to perform step a) direction formation of the signals (28) to obtain a direction signal (33) for each direction considered, wherein step a) and step b) are performed for different direction signals (33) at each viewing time, in particular for each direction signal (33), in order to obtain the direction in which the vehicles (22) to be classified are located. In this case, the signal representation also includes beams (direction signals) that have been obtained by direction formation. Step a) and step b) can be performed for different direction signals at each viewing time, in particular for each direction signal, in order to obtain the direction in which the vehicles to be classified are located.This enables, for example, the tracking of a vehicle.
[0027] Vehicle classification can be performed using a model, i.e., an artificial intelligence, based on sensor data (especially direction signals). The model can optionally comprise multiple, and in particular, different, sub-models. A (sub-)model describes the architecture of one or more algorithms. The model can be optimized through training by adjusting weights, i.e., parameters. Most common, well-known algorithms are suitable as models, especially artificial neural networks, Bayesian classifiers, support vector machines (SVMs), and linear regression.
[0028] Similarly, deep learning approaches such as artificial neural networks based on transformer technology and / or convolutional artificial neural networks can be used. If the model comprises several sub-models for classifying the vehicles, similar algorithms (e.g., different artificial neural networks) as well as different algorithms (e.g., an artificial neural network and linear regression) can be combined.
[0029] In the following, a distinction is made between the "model" for classifying the vehicles and the "deep model" for determining the characteristics, up to the brief description of the figures.
[0030] In particular, it has been shown that no special algorithms or algorithm architectures are needed to create the vehicle classification model. It is sufficient to train commercially available algorithms with appropriate data. Such algorithms are also known as COTS (Components Off-The-Shelf). It has been found that the structure—for example, the number of neurons or layers in artificial neural networks—is irrelevant for training the model, as long as a minimum level of complexity is achieved. Models that perform well in public benchmarks are particularly suitable. Furthermore, it has been shown that even simple feedforward neural networks, especially those with few layers, are suitable for vehicle classification when using the features from the deep model.
[0031] Preferably, at least one (partial) model is implemented as an artificial neural network. This artificial neural network can, for example, be based on Transformer technology and / or be or include a convolutional artificial neural network. However, a classic feedforward neural network, such as a multilayer perceptron (MLP), is preferred. It has been found that a feedforward neural network is sufficient for classifying vehicles with the features from the deep model. More complex and thus computationally intensive models can therefore be avoided.
[0032] If a single model without sub-models is used, it will preferentially classify all vehicles. If multiple sub-models are used, each sub-model can, for example, classify one vehicle.
[0033] Real, recorded sensor data (especially signals or direction signals) can be used to train the model or sub-models. However, it is also possible to generate artificial sensor data.
[0034] Artificial (i.e., synthetic) training data is suitable. Artificial data can be used particularly when the amount of real-world training data is insufficient. Preferably, a combination of real and artificial data can be used for training. This means that artificial data can be supplemented with real sensor data. Real sensor data can be acquired during training runs, training maneuvers, or in the field. Preferably, the real data is pre-labeled, for example, by a trained sonar operator. This means that the actual vehicles in the sensor data are identified beforehand. This is called labeling. This enables supervised learning of the model. Manual labeling is not necessary for the artificial sensor data, as it is known where (i.e., at which bearing and at what time) an object was simulated and where only noise is present.The labeling can therefore be automated.
[0035] Furthermore, a computer program comprising instructions that, when executed by a computer, cause it to perform the method using active sonar or radar data is disclosed. The computer program can, for example, receive the active sonar or radar data and perform the entire sensor signal processing, or, for example, perform only the signal processing relevant to the invention. In particular, the model as a sequence of instructions can be implemented in a computer program.
[0036] Preferred embodiments of the present invention are explained below with reference to the accompanying drawings. These show:
[0037] Fig. 1: a schematic representation of a sensor system with a model for classifying vehicles;
[0038] Fig. 2: a schematic representation of the model in an exemplary embodiment;
[0039] Fig. 3 shows a schematic representation of a block diagram of a method for determining features for classifying vehicles using sensor data analysis with a deep model;
[0040] Fig. 4 shows a schematic representation of a block diagram of the model from Fig. 3. Before exemplary embodiments of the present invention are explained in detail below with reference to the drawings, it should be noted that identical, functionally equivalent or equivalent elements, objects and / or structures in the different figures are provided with the same reference numerals, so that the description of these elements shown in different exemplary embodiments is interchangeable or can be applied to each other.
[0041] Fig. 1 shows a schematic diagram of a sensor system 20 for classifying known vehicles 22 in a maritime environment. The sensor system comprises an active sonar system 24 for generating active sonar data 28 or, alternatively, a radar system for generating radar data. It should be noted that the signal processing of the active sonar data is described in the figures as an example. The signal processing of the radar data, however, is analogous. The active sonar system 24 emits underwater sound 26 and receives its reflections 26'. Furthermore, the sensor system comprises a signal processing unit 30 for processing the active sonar data 28.
[0042] Fig. 2 shows a schematic diagram of the signal processing unit 30 in an exemplary embodiment. The signal processing unit includes an optional beamforming module 32 for directional shaping of the active sonar data in order to obtain a directional signal 33 for each direction considered. Furthermore, the signal processing unit includes a feature generation module 34 for processing a representation of the active sonar data 28, in particular the directional signals, with a deep model to obtain a plurality of feature signals 35. The features 35 are fed to a machine learning-trained model 36, in particular its input nodes 38. Advantageously, the number of input nodes 38 corresponds to the number of features 35.
[0043] Furthermore, model 36 features a large number of output nodes (here, 5 output nodes). This allows, for example, the classification of 5 different vehicles or the classification of 4 different vehicles and the detection of other vehicles. The number of layers and the number of nodes per layer can be chosen arbitrarily. The model is illustrated as a multi-layer perceptron as an example.
[0044] Figure 3 shows a schematic block diagram of three models 36a, 36b, and 36c illustrating a method for determining features for classifying vehicles using the deep model. Each model forms a convolutional artificial neural network (CNN). It should be noted again that a consecutive training process, optimized particularly with regard to the required computing power and training data, using a model based on batches of sensor data sets, has already been described above. To better illustrate the optimization principle, parallel training with three identical models, each with the same weights, is shown.
[0045] First, sensor data packets 54, 56, and 58 are assigned to a first category (packet 54), a second category (packet 56), and a third category (packet 58), respectively. A sensor data packet 54, 56, each belonging to the first and second categories, and one or more sensor data packets from the third category (58) constitute a sensor data set. A sensor data packet is defined, for example, as a time series of a digitized audio signal with a specific length, such as between 1 ms and 10 s. For the sake of argument, this example assumes that a sensor data set contains 49,152 values (samples).
[0046] For training, a multitude of sensor data sets are presented to the first model 36a, the second model 36b, and the third model 36c. The sensor data packets 54 of the first category form the input data for the first model 36a, the sensor data packets 56 of the second category form the input data for the second model 36b, and the sensor data packets 58 of the third category form the input data for the third model 58. Three identical algorithms 36a, 36b, and 36c are shown as examples. This allows for faster training by processing the sensor data in parallel. However, it is also possible to perform the training sequentially on only one model. The sensor data packets of a sensor data set are then presented to this single model one after the other. An example setup of the deep model could look like this.First, a CNN module 60, preferably a 1D ResNet, is used to build an initial set of feature vectors. In this example, 64 feature vectors, each with 12288 values (64 x 12288), were generated. This is followed by an example downsampling module 62 with a cascade of, for example, four downsampling blocks. Downsampling creates a new set of feature vectors in several (here, four) steps. In this case, it is a set with 1024 feature vectors, each with three values. This new set of feature vectors is processed by another CNN module 64. This additional CNN module 64, purely as an example, has three CNN blocks per model, preferably three 1D ResNet blocks. The size of the feature vectors does not change in this case.This is followed by a module 66 for averaging the values per feature vector before a compression module 68, for example in three stages, outputs a feature vector 70a, 70b, 70c with a predefined size of, for example, 128 features per model.
[0047] It should be noted that the size and number of feature vectors were chosen purely arbitrarily. Any other size and number of feature vectors can also be used. For example, instead of 128 initial features, it is also possible to output 256 features, 512 features, or even a number of features other than a power of two.
[0048] The training is performed with the goal of minimizing, after one iteration (i.e., specifically, after a sensor dataset or batch has been processed by the models), the difference between the output values (i.e., the feature vectors) 70a, 70b of the first and second models, and maximizing the difference between the output values (i.e., the feature vectors) 70b, 70c of the second and third models. Since the models are identical, any trained model can be used as the deep model for feature determination.
[0049] Fig. 4 shows a schematic block diagram of the deep model 72.
[0050] The deep model 72 can run in the feature generation module 34 (see Fig. 2) and be operated using sensor data packets 74, in particular direction-generated sensor data, to obtain a feature vector 70 for the detection and classification of vehicles. Thus, classification can be performed in real time, i.e., within a few seconds, in particular less than 10 seconds after the sensor data is recorded.
[0051] A Python implementation of the deep model with K output features might look like this:
[0052] class Waveform Encoder(torch.nn. Module):
[0053] def _ init _ (seif, output_features=K):
[0054] great (WaveformEncoder, soap). _init_()
[0055] self.down = torch. nn. Sequential
[0056] torch. nn.Conv1d(1 , 64, kernel_size = 8, padding=2, stride=4, bias=False), # CNN module 60
[0057] DownsampleBlock(in_channels = 64, out_channels = 128,kernel_size=13, downsample_factor = 8),
[0058] DownsampleBlock(in_channels = 128, out_channels = 256,kernel_size=13, downsample_factor = 8),
[0059] DownsampleBlock(in_channels = 256, out_channels = 512, kernel_size=13, downsample_factor = 8),
[0060] DownsampleBlock(in_channels = 512, out_channels = 1024, kernel_size=13, downsample_factor = 8),
[0061] # Downsamplingmodul 62
[0062] ResBlock(1024, kernel_size=3),
[0063] ResBlock(1024, kernel_size=3),
[0064] ResBlock(1024, kernel_size=3),
[0065] # weiteres CNN-Modul 64
[0066] torch.nn.AdaptiveAvgPool1d(1)
[0067] # Modul zur Mittelung 66)
[0068] between_features1 = int(( 1024-output_features)*(2 / 3)+output_features) between_features2 = int((1024-output_features)*(1 / 3)+output_features) # Kompressionsmodul 68
[0069] self.compress = torch. nn. Sequential
[0070] torch. nn.Conv1d(in_channels = 1024,out_channels = between_features1 , kernel_size=1 , padding=0),
[0071] torch.nn.LeakyReLU(0.2),
[0072] torch. nn.Conv1d(in_channels = between_features1 , out_channels = between_features2, kernel_size=1 , padding=0),
[0073] torch.nn.LeakyReLU(0.2),
[0074] torch. nn.Conv1d(in_channels = between_features2, out_channels = output_features, kernel_size=1 , padding=0),
[0075] def forward(self, x):
[0076] return self.compress(self.down(x)).squeeze()
[0077] dass DownsampleBlock(torch.nn. Module):
[0078] def init (seif, in_channels, out_channels, kernel_size, downsample_factor): super(DownsampleBlock, seif). init ()
[0079] self.resblocks = torch, nn. Sequential
[0080] ResBlock(in_channels, kernel_size),
[0081] ResBlock(in_channels, kernel_size),
[0082] ResBlock(in_channels, kernel_size),
[0083] ResBlock(in_channels, kernel_size),
[0084] )
[0085] self.down = torch. nn. Sequential
[0086] torch. nn.Conv1d(in_channels, out_channels, kernel_size= 1, padding=0, bias=True),torch. nn.AvgPool1d(downsample_factor)
[0087] )
[0088] def forward(self, x):
[0089] x = self.resblocks(x)
[0090] return self.down(x)
[0091] dass ResBlock(torch.nn. Module):
[0092] def init (self, channels, kernel_size):
[0093] super(ResBlock, seif). init ()
[0094] self.resblock = torch. nn. Sequential
[0095] torch. nn.Conv1d(channels, channels, kernel_size= kernel_size, padding=(kernel_size-1 ) / / 2, bias=False),
[0096] torch.nn.LeakyReLU(0.2),
[0097] torch. nn.Conv1d(channels, channels, kernel_size= kernel_size, padding=(kernel_size-1 ) / / 2, bias=False),
[0098] torch.nn.LeakyReLU(0.2),
[0099] )
[0100] def forward(self,x):
[0101] return self.resblock(x) + x
[0102] Although some aspects have been described in connection with a device, it is understood that these aspects also constitute a description of the corresponding method, so that a block or component of a device is also to be understood as a corresponding method step or as a feature of a method step. Analogously, aspects described in connection with or as a method step also constitute a description of a corresponding block, detail, or feature of a corresponding device. Depending on the implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be carried out using a digital storage medium, for example, a magnetic or optical storage medium, on which electronically readable control signals, e.g.,A computer program is stored that can interact with a programmable computer system (CPU and / or GPU) in such a way that the respective method is carried out. For this reason, the digital storage medium should be computer-readable. Exemplary embodiments may therefore include a data carrier that has electronically readable control signals capable of interacting with a programmable computer system in such a way that one of the methods described herein is carried out.
[0103] In some embodiments, a programmable logic device (for example, a field-programmable gate array, an FPGA) can be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field-programmable gate array can interact with a microprocessor to perform one of the methods described herein. Generally, in some embodiments, the methods are performed by any hardware device. This can be general-purpose hardware such as a computer processor (CPU) or a graphics processing unit (GPU), or hardware specific to the method, such as an ASIC. Distributed execution across the CPU and GPU is also possible.
[0104] The embodiments described above merely illustrate the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be obvious to other people skilled in the art. Therefore, it is intended that the invention be limited only by the scope of protection of the following claims and not by the specific details presented herein by way of description and explanation of the embodiments. Reference numerals:
[0105] 20 sensor system
[0106] 22 vehicles
[0107] 24 Active sonar system
[0108] 26 Water sound
[0109] 26' Reflections of water sound
[0110] 28 active sonar data
[0111] 30 Signal processing unit
[0112] 32 Beamforming
[0113] 33 Directional signal
[0114] 34 Character formation
[0115] 35 features / feature vector
[0116] 36 model
[0117] 38 input nodes
[0118] 40 output nodes
[0119] 54 sensor data packets of the first category 56 sensor data packets of the second category 58 sensor data packets of the third category 60 CNN module
[0120] 62 Downsampling module
[0121] 64 additional CNN modules
[0122] 66 Module for averaging
[0123] 68 Compression modulus
[0124] 70 feature vectors
[0125] 72 deep model
[0126] 74 sensor data packets
Claims
Patent claims 1. Method for determining features for classifying vehicles (22) in a maritime environment using a deep learning model (36) with the following steps: a) Providing a plurality of sensor data packets (54, 56, 58) comprising or consisting of active sonar data or radar data, wherein the sensor data packets are each assigned to a first category (54) and a second category (56) and a sensor data set is formed, wherein a sensor data set comprises a pair of sensor data packets (54, 56) of the first category and the second category; b) Training the model (36) with a variety of sensor data sets (54, 56, 58) to reduce the difference between the output values (70a, 70b) of the model using the sensor data packages of the first category and the second category, and to increase the difference between the output values (70b, 70c) of the model using the sensor data packages of the second category and one or more sensor data packages different from the sensor data set.
2. Method according to claim 1, wherein the sensor data packets (54, 56, 58) of a sensor data set or a plurality of sensor data packets are consecutively presented to the model (36) for training in order to obtain the output values.
3. Method according to one of the preceding claims, wherein the model (36) comprises a convolutional artificial neural network [CNN], in particular one or a plurality of one-dimensional residual networks [1d-ResNet].
4. Method according to any of the preceding claims, wherein the model (36) features downsampling.
5. A method according to any one of the preceding claims, wherein the sensor data packets (54, 56, 58) are each available as time series.
6. A method according to any one of the preceding claims, wherein the sensor data packets comprise or are sonar data or radar data.
7. Method according to one of the preceding claims, wherein the sensor data packets (54, 56, 58) of the pair of sensor data packets each contain user data of the same vehicle (22).
8. Method according to one of the preceding claims, wherein the sensor data packets (56) of the second category and the one or more sensor data packets (58) different from the sensor data set each have different user data.
9. Method according to one of the preceding claims, wherein the model (36) is configured to construct an initial [0th] set of feature vectors using a one-dimensional residual network and to determine further sets of feature vectors based on the initial set of feature vectors, wherein the number of feature vectors is increased per stage by means of N-stage downsampling and the values per feature vector are reduced.
10. Method according to claim 9, wherein the number of feature vectors of the N-th set is reduced to a predetermined number of feature vectors by compression with a one-dimensional residual mesh.
11. Method according to one of claims 9 or 10, wherein the number of values per feature vector (35) is reduced by averaging to one value per feature vector (35) if the number of values per feature vector before averaging is greater than one.
12. A method according to any one of the preceding claims, wherein the method comprises step c) classifying a vehicle (22) using the features obtained in step b).
13. A computer program comprising instructions which, when the program is executed by a computer, cause it to execute the method according to any one of claims 1 to 12.
14. Sensor system (20) for classifying known vehicles (22) in a maritime environment with the following features: - an active sonar system for generating active sonar data or a radar system for generating radar data; and - a signal processing unit (30) configured to perform the following steps: a) (34) Processing a representation of the active sonar data or the radar data with a deep model trained using the method according to any one of claims 1 to 12 or the computer program according to claim 13 to obtain a plurality of features (35); b) (36) Processing the features (35) with a machine learning trained model (36) to classify the known vehicles (22).
15. Sensor system (20) according to claim 14, wherein the signal processing unit is configured to perform step aO) (32) direction formation of the signals (28) in order to obtain a direction signal (33) for each direction considered, wherein step a) and step b) are performed for different direction signals (33) per viewing time, in particular each direction signal (33), in order to obtain the direction in which the vehicle (22) to be classified is located.