Generating acoustic training data

Abstract

Training procedure for an artificial neural network with the following steps: - specifying (S1) a target sound by providing a single recording of the target sound, the recording being free of ambient noise, the target sound being an acoustic warning signal, a characteristic machine noise indicating a defect or wear, or an ambient sound to be identified if the ambient sound can be recognized by certain characteristics; - generating (S2) at least one variant of the recording of the target sound by further modifying the recording due to expected environmental effects using computer-aided technical means, such that the neural network is also trained with the variant of the single recording; - Training the artificial neural network with the recording and the variant of the recording, wherein the environmental effects include ambient noise and the recording is modified by combining the target noise with ambient noise, the ambient noise resulting from tire rolling noise, wind noise, precipitation noise, animal sounds, work noise due to construction work and / or green maintenance work and / or the like.

Description

FIELD OF THE INVENTION

[0001] The present invention relates to a training method for an artificial neural network.

[0002] The state of the art is, for example, US 2017 / 0 133 006 A1, SELTZER, Michael L.; YU, Dong; WANG, Yongqiang: An investigation of deep neural networks for noise robust speech recognition. In: 2013 IEEE international conference on acoustics, speech and signal processing. IEEE, 2013. pp. 7398-7402 and SAINATH, Tara N. [et al.]: Learning the speech front-end with raw waveform CLDNNs. In: Sixteenth Annual Conference of the International Speech Communication Association. Revealed in 2015. SUMMARY OF THE INVENTION

[0003] Accordingly, it is provided: - a training method for an artificial neural network comprising the following steps: specifying a target sound by providing a single recording of the target sound, the recording being free of ambient noise; generating at least one variant of the recording of the target sound by modifying the recording due to expected ambient effects; training the artificial neural network with the recording and the variant of the recording.

[0004] Vehicles are land, water or air vehicles with or without propulsion.

[0005] A microphone is a sound transducer that converts airborne sound as alternating sound pressure vibrations into corresponding electrical voltage changes as a microphone signal.

[0006] Noise is a collective term for all auditory sensations, including tones, sounds, tonal mixtures, harmonies, or mixtures of sounds. Periodic or non-periodic vibrational processes are the cause of noise. Ambient vehicle noise is perceptible in the environment surrounding a vehicle.

[0007] Ambient traffic noise refers to all sounds audible near a road. Accordingly, engine noise emitted by vehicles traveling on a road, for example, is considered ambient noise. However, this also includes noise from other road users, such as pedestrians or cyclists, who are in close proximity to the road. This also includes noise emitted by animals, such as the barking of a dog.

[0008] Furthermore, noises from construction sites or lawn maintenance work are also ambient noise from road traffic.

[0009] An artificial neural network (ANN) is, in particular, a network of interconnected artificial neurons simulated in a computer program. The artificial neurons are typically arranged in different layers. The artificial neural network usually comprises an input layer and an output layer, whose neuron output is the only one of the artificial neural network that is visible. Layers lying between the input layer and the output layer are typically referred to as hidden layers. Typically, an architecture or topology of an artificial neural network is first initiated and then trained in a training phase for a specific task or for several tasks in a training phase.

[0010] The term "topology of an ANN" encompasses all aspects of the structure of an ANN. This includes, for example, the number of neurons in the ANN, the distribution of neurons across the individual layers of the ANN, the number of layers of an ANN, the interconnection of the neurons, and the weighting of the interconnection.

[0011] Training the artificial neural network typically involves changing the weight of a connection between two artificial neurons in the artificial neural network. The weight contains information about the strength of consideration given to a neuron's input. Training the artificial neural network can also involve developing new connections between artificial neurons, deleting existing connections between artificial neurons, adjusting the thresholds of the artificial neurons, and / or adding or deleting artificial neurons.

[0012] An example of an artificial neural network is a shallow artificial neural network, which often contains only a single hidden layer between the input and output layers and is therefore relatively easy to train. Another example is a deep artificial neural network, which contains several nested hidden layers of artificial neurons between the input and output layers. The deep artificial neural network enables improved recognition of patterns and complex relationships. For example, the artificial neural network can be a single- or multi-layer feedforward network or a recurrent network. Feedforward networks have neurons that are exclusively fed forward, i.e. a neuron is fed exclusively by higher layers.

[0013] A recurrent network has bidirectionally connected neurons, meaning a neuron is also fed by deeper layers. This allows information from a previous run to be incorporated into a subsequent run of the ANN, creating a memory.

[0014] A training system is a computing unit on which an ANN is trained.

[0015] In this application, training data refers to data pairs of input data (acoustic data) to be processed by the ANN, as well as target result data, such as a reaction to a sound or a decision as to whether a target sound is contained in input data to be determined by the ANN. During training, the ANN is adapted based on a comparison of target result data with the actual result data determined by the ANN, resulting in a training effect. A training data set can contain multiple data pairs of input and output data.

[0016] An ego vehicle is a vehicle that collects and evaluates microphone data.

[0017] The input data fed to the ANN in this application is acoustic data containing the target sound.

[0018] The target sound is a sound to be identified by a trained ANN and, according to the invention, can be an acoustic warning signal or a characteristic machine noise indicating a defect or a sign of wear. However, the target sound can also be an ambient sound to be identified, if the ambient sound can be recognized by certain characteristics.

[0019] An echo occurs when reflections of a sound wave are delayed so much that the sound can be detected as a separate event. A single, separately detected echo has a lower volume than the original.

[0020] The Doppler effect is the temporal compression or expansion of a signal when the distance between transmitter and receiver changes during the signal's duration. The cause is the change in propagation time. This purely kinematic effect occurs in all signals that propagate at a certain speed, usually the speed of light or sound. If the signal propagates in a medium, its state of motion must be taken into account. For periodic signals, the observed frequency increases or decreases. This applies to both pitches and modulation frequencies, e.g., the changing tones of a siren horn. At low speeds in relation to the propagation speed, this ratio also indicates the relative frequency change Δf.

[0021] Sound attenuation is defined as the obstruction of sound propagation by absorbing airborne sound. This sound absorption converts the sound energy into inaudible waves of vibrational energy, thus reducing reflection at a boundary surface. The ability of air to attenuate sound waves is described by its dissipation properties.

[0022] Data descriptions, or labels in technical jargon, are descriptions of training data and the characteristics contained in the training data. For example, a label can be used to assign a cause to specific parts of a sound or parts of a recording.

[0023] Labeled data, or information-tagged data, is a data set that has been tagged with one or more pieces of information. This typically involves using a set of unlabeled data and supplementing it with meaningful information. For example, labels can indicate whether a photo contains a horse or a cow, what words were spoken in an audio recording, what type of action is performed in a video, the topic of a news article, and the general sentiment, etc.

[0024] Labels are often generated by humans, e.g., by people making judgments about a given piece of unlabeled data (e.g., "Does this photo contain a horse or a cow?"). Consequently, labeled data is significantly more expensive than unlabeled raw data.

[0025] Labels can be applied by a human or automatically. Automated labels can be applied, for example, by another trained AI system or by using labeled datasets and combining the labeled dataset with training data. For example, it is conceivable to combine a target sound with various weather sounds, such as rain or gusts of wind.

[0026] Examples of acoustic warning signals include a siren, a Yelp signal, a train whistle, warning sounds at railroad crossings, and the like.

[0027] A siren is generally an acoustic device on vehicles that emits several signal tones of different fundamental frequencies one after the other.

[0028] A specific sequence of sounds is generated to warn other road users and prompt them to give way to the emergency vehicle. In most states, when emergency vehicles use sirens, this is always combined with flashing blue lights.

[0029] The basic idea of ​​the invention is to train an artificial neural network with the recording of a target sound, whereby the recording only records the target sound without any further acoustic interference.

[0030] During training, it is also planned to train the artificial neural network with further, modified recordings of the target sound.

[0031] Artificial neural networks can be trained by typically providing a larger number of data points in a training dataset than the network's adjustable parameters, i.e., its topology. This ensures that an artificial neural network makes a desired prediction without overfitting the neural network. Instead, training data is simply "memorized" and no generalization to the problem occurs. This problem particularly affects deep neural networks (DNNs).

[0032] In acoustic event detection, the neural network must learn numerous target sounds, which are often very similar or have identical characteristics. Such sounds can be of a technical nature and / or environmental origin, for example. Sounds that can be assigned to a specific event are usually altered or distorted by physical effects, such as attenuation in a propagation medium, such as air, superposition with background noise, or the Doppler effect.

[0033] The invention proposes training an artificial neural network with a single recording of a target sound. This target sound must be recorded without any further influences.

[0034] If the term “single recording” is used in this patent application, this means that a target sound is recorded only once, whereby the single recording can be further modified using computer-assisted technical means, so that the neural network is also trained with several variants of the single recording.

[0035] This is to be distinguished from the fact that a single target sound is recorded multiple times in different acoustic environments and the artificial neural network is trained with multiple recordings of a single target sound.

[0036] This allows variants of the single recording to be generated automatically and allows specific characteristics that arise due to physical environmental effects to be precisely adjusted.

[0037] This simplifies and improves the labeling of the training data.

[0038] Advantageous embodiments and further developments emerge from the further subclaims and from the description with reference to the figures of the drawing.

[0039] According to the invention, the environmental effects include ambient noise and the recording is modified by combining or superimposing the target noise with ambient noise.

[0040] According to the invention, the ambient noise results from tire rolling noise, wind noise, precipitation noise, animal noises, work noise due to construction work and / or green maintenance work and / or the like.

[0041] This makes it possible to simulate a specific target sound in a variety of different simulation environments. For example, corresponding ambient sounds can be provided from an existing database without having to be recorded in advance.

[0042] Another advantage of this is that existing ambient noises are usually provided with a data description, so that the data description of the ambient noises can be used to describe the data of the recording variant.

[0043] This eliminates the need to record numerous typical ambient sounds of a target environment, or to play the target sound under appropriate environmental conditions and record the played recordings together with the ambient sounds. Accordingly, the recording effort required to generate training data can be reduced.

[0044] According to a preferred embodiment of the invention, environmental effects are those effects that influence the propagation of sound waves.

[0045] Accordingly, the environmental effects can be designed, for example, as attenuation effects, echo effects, Doppler effects or as various dissipation properties of the air and / or the like.

[0046] This makes it possible to modify the recording based on various expected, for example, weather-dependent, wave propagation properties of air or due to expected reflections from surfaces such as other vehicles, roads, buildings, and / or natural structures. Accordingly, the occurrence of a Doppler effect or various humidity- and temperature-dependent dissipation properties of air can be simulated in a training dataset.

[0047] According to a preferred development of the invention, a training data set is generated that includes the recording as well as a plurality of variations of the recording, wherein the environmental effects are described in an automated data description process such that the description is interpretable for the artificial neural network. In technical jargon, data description is also referred to as labeling. The labeling of the data can thus be automated, and the human processing effort required to describe the data can be reduced.

[0048] It is understood that it is also possible to train the ANN using recordings that do not contain the target sound. This can further improve the ANN's ability to distinguish between the presence and absence of the target sound.

[0049] It is understood that it is advantageous to carry out the method as described above using computer-aided means in the form of a computer program product. TABLE OF CONTENTS OF THE DRAWINGS

[0050] The present invention will be explained in more detail below with reference to the exemplary embodiments shown in the schematic figures of the drawings. In the drawings: Fig. 1 is a schematic block diagram of an embodiment of the invention.

[0051] The accompanying drawings are intended to provide a further understanding of embodiments of the invention. They illustrate embodiments and, in conjunction with the description, serve to explain principles and concepts of the invention. Other embodiments and many of the noted advantages will be apparent upon review of the drawings. Elements of the drawings are not necessarily shown to scale relative to one another.

[0052] In the figures of the drawings, identical, functionally identical and acting elements, features and components are provided with the same reference symbols, unless otherwise stated. DESCRIPTION OF EMBODIMENTS

[0053] Fig. Figure 1 shows a schematic block diagram of a method for training an artificial neural network. In step S1, a target sound is specified by providing a single recording of the target sound, the recording being free of ambient noise. In step S2, at least one variant of the recording of the target sound is generated by modifying the recording based on expected ambient effects. In step S3, the artificial neural network is trained using a recording and the variant of the recording. Reference symbol S1-S3 process steps

Claims

[1] Training procedure for an artificial neural network with the following steps: - specifying (S1) a target sound by providing a single recording of the target sound, the recording being free of ambient noise, the target sound being an acoustic warning signal, a characteristic machine noise indicating a defect or wear, or an ambient sound to be identified if the ambient sound can be recognized by certain characteristics; - generating (S2) at least one variant of the recording of the target sound by further modifying the recording due to expected environmental effects using computer-aided technical means, such that the neural network is also trained with the variant of the single recording; - Training the artificial neural network with the recording and the variant of the recording, wherein the environmental effects include ambient noise and the recording is modified by combining the target noise with ambient noise, the ambient noise resulting from tire rolling noise, wind noise, precipitation noise, animal sounds, work noise due to construction work and / or green maintenance work and / or the like. [2] A training method for an artificial neural network according to any one of the preceding claims, wherein the environmental effects influence a propagation of sound waves. [3] Training method for an artificial neural network according to claim 2, wherein the environmental effects are formed as damping effects, echo effects, Doppler effects, various dissipation properties of the air and / or the like. [4] Training method for training an artificial neural network according to one of the preceding claims, wherein a training data set is generated which comprises the recording and a plurality of variants of the recording, wherein the environmental effects are described in an automated data description method such that the description is interpretable for the artificial neural network. [5] A computer program product adapted to be loaded into a memory of a computer and comprising software code portions for carrying out the steps of the method according to any one of the preceding claims 1-4 when the computer program product is running on the computer.

Images