Audio masking of speakers

Abstract

The present disclosure relates to a method for masking a speech signal in a zone-based audio system, comprising: capturing a speech signal to be masked in an audio zone; determining a time point relevant for speech intelligibility within the speech signal; generating a distraction signal for the determined time point, wherein the distraction signal is adapted to the speech signal with respect to a spectral characteristic and / or its energy; and outputting the distraction signal as a masking signal at the determined time point in another audio zone.

Description

[0001] The present disclosure relates to generating a masking signal for speech in a zone-based audio system.

[0002] Modern means of communication and their ever-increasing coverage enable communication to take place almost anywhere, for example in the form of telephone conversations. In public spaces, other people can often overhear such conversations and understand their content. This is particularly problematic when they involve confidential private or business conversations. Such a scenario exists in public transport, such as trains or airplanes, but also in private vehicles, such as taxis or rented limousines. In these cases, other people are located alongside the speaker in fixed locations, for example in assigned seats. Such seats often have an associated audio system or at least components thereof.For example, speakers for individual playback of audio content can be provided in these seats, for example integrated into headrests, which is also referred to as a zone-based audio system.

[0003] In addition to telephone conversations, the problem of unwanted eavesdropping can also occur in conversations between people. For example, two passengers in the back of a taxi may be discussing a confidential topic where the driver is not permitted to listen in.

[0004] It is known from the state of the art that unwanted eavesdropping can be reduced by playing loud noise. However, this increases the noise level for everyone involved and is perceived as an unpleasant disturbance that can also affect attention and reaction time, which is particularly undesirable in traffic.

[0005] DE 10 2014 214052 A1 describes a method and a corresponding device for generating a shielded listening zone within a vehicle. A method for masking a target speech signal is described. The method comprises determining the target speech signal, which is reproduced in a first listening zone of a vehicle, and generating a first masking sound signal as a function of the determined target speech signal. The method also comprises binauralizing the first masking sound signal to generate a binaural masking sound signal. Furthermore, the method comprises reproducing the binaural masking sound signal via at least two loudspeakers in a second listening zone of the vehicle, in which the target speech signal is to be masked.

[0006] US 2012 / 016665 A1 discloses a masking signal generation device, wherein a CPU analyzes the utterance rate of a received audio signal. The CPU then copies the received audio signal into a plurality of audio signals and performs the following processing for each of the audio signals. Namely, the CPU divides each of the audio signals into frames based on a frame length determined based on the utterance rate. A reversal process is performed on each of the frames to replace a waveform of the frame with an inverted waveform, and a windowing process is performed to achieve a smooth connection between the frames. The CPU then randomly rearranges the order of the frames and mixes the multiple audio signals to generate a masking audio signal.

[0007] In "Aircraft noise and speech intelligibility in an outdoor living space", a partial loudness model is presented that predicts the loudness of a signal in the presence of a masking sound signal, taking into account masking across spectral bands and the effects of temporal masking of time-varying noise.

[0008] This document deals with the technical task of generating a masking signal in a zone-based audio system that reduces unwanted eavesdropping of a conversation while not causing any unpleasant disturbance.

[0009] The problem is solved by the features of the independent claims. Advantageous embodiments are described in the dependent claims.

[0010] The invention is described in the appended set of claims.

[0011] According to a first aspect, a method for masking a speech signal in a zone-based audio system is disclosed. The method comprises detecting a speech signal to be masked in an audio zone, e.g., using one or more conveniently placed microphones, which can be arranged, for example, in a seat's headrest. The speech signal can originate from the local speaker of a telephone conversation or belong to a conversation between people present. The detected speech signal is then transformed into spectral bands, which can be done, for example, using an FFT and MEL filters. The method further comprises swapping spectral values ​​of at least two spectral bands, thereby changing the spectral structure of the speech signal without changing its overall energy content. A (as broadband as possible) noise signal is then generated based on the swapped spectral values.Although the generated noise signal bears a certain resemblance to the spectrum of the speech signal, it does not completely match it because the spectral structure of the speech signal is no longer fully preserved due to the band swapping. A noise signal of this kind, with a spectrum similar to but not identical to the speech signal, is well suited as a masking signal for the speech signal. It should also be noted that any number of bands can be swapped (e.g., all of them), with more bands swapped resulting in more variation in the noise spectrum. Finally, the noise signal is output as a masking signal in another audio zone with the lowest possible energy input. This makes it more difficult for a person located at the listening location to overhear the conversation by reducing speech intelligibility for that person.

[0012] Generating a noise signal based on the swapped spectral values ​​can involve generating a broadband noise signal, e.g., using a noise generator, and transforming the generated noise signal into the frequency domain. Furthermore, multiplying the frequency representation of the noise signal by a frequency representation of the speech signal can be performed, taking the swapped spectral values ​​into account. Multiplication in the frequency domain generates a noise spectrum that essentially corresponds to that of the speech signal after swapping the spectral bands—i.e., it is similar to, but not identical to, the speech spectrum. A similar effect can also be achieved by convolution in the time domain.

[0013] The frequency representation of the speech signal can be generated by interpolating the spectral values ​​of the bands (for example, in the mel range) after swapping the spectral values. This interpolation generates the required frequency support values ​​for multiplication with the noise spectrum from the (relatively few) spectral values ​​of the bands.

[0014] The method may further comprise estimating a background noise spectrum (preferably at the monitoring location) and comparing spectral values ​​of the speech signal with the background noise spectrum. The comparison of the spectral values ​​preferably (but not necessarily) takes place in the range of the spectral bands (e.g., mel bands), which means that the background noise spectrum must also be represented in the spectral bands. Furthermore, only spectral values ​​of the speech signal that are greater than the corresponding spectral values ​​of the background noise spectrum (or are in a predetermined ratio thereto) can be considered for the further procedure (e.g., the above-mentioned interpolation). Spectral components of the speech signal that are already masked by the background noise need not be considered for generating the masking signal and can be masked (e.g., by setting them to zero).Background noise can be taken into account both before and after swapping spectral values. In the former case, the spectral bands to be compared still match exactly, and the background noise is correctly accounted for. In the latter case, swapping bands and masking low-energy bands in the speech signal introduces additional variation into the noise spectrum, which can lead to increased masking. This enables a masking signal adapted to the background or environment, which can be output with the least possible energy input into the listener's audio zone.

[0015] The transformation of the acquired speech signal into spectral bands can be performed for blocks of the speech signal and using a MEL filter bank. Optionally, it is possible to perform temporal smoothing of the spectral values ​​for the MEL bands, e.g., in the form of a moving average.

[0016] In a further embodiment of the invention, the noise signal can be spatially represented during output by means of multi-channel (i.e. at least 2-channel) reproduction. For this purpose, a multi-channel representation of the masking signal can be generated, which enables spatial reproduction of the masking signal. For 2-channel systems, this can preferably be done by multiplying with binaural spectra of an acoustic transfer function. The spatial reproduction increases the effect of the masking signal on concealing speech at the monitoring location, particularly if the noise signal in the other audio zone is output spatially in such a way that it appears to originate from the direction of the speaker of the speech signal to be masked.

[0017] In addition to the masking signal described above, which is based on a broadband noise signal matched to the speech signal, a further component can be generated for the masking signal, which are output together to the listener in the second audio zone. To this end, the method can comprise determining a point in time in the speech signal that is relevant for speech intelligibility (e.g., the presence of consonants in the speech signal) and generating a suitable distraction signal for the specific point in time. The distraction signal can then be output at the specific point in time as a further masking signal in the other audio zone, thereby providing additional, selective concealment (masking) of the conversation content during speech onsets. Since the distraction signal is only output at specific relevant points in time, it does not significantly increase the overall sound level and does not lead to any significant impairment.

[0018] The time point relevant for speech intelligibility can be determined based on extreme values ​​(e.g., local maxima, onsets) of a spectral function of the speech signal. The spectral function is determined based on the summation of spectral values ​​along the frequency axis. The spectral values ​​can be previously smoothed in temporal and / or frequency directions. After summation of the spectral values ​​along the frequency axis, the sum values ​​can optionally be logarithmized. To generate local maxima for the detection of relevant time points, the (optionally logarithmized) sum values ​​can be temporally differentiated.

[0019] Furthermore, the time points relevant for speech intelligibility can be verified based on speech signal parameters such as zero-crossing rate, short-term energy, and / or spectral center of gravity. It is also possible to consider temporal constraints on extreme values, for example, requiring them to have a specified minimum temporal separation.

[0020] The deflection signal for a specific point in time can then be randomly selected from a set of predefined deflection signals. These can be stored in a memory for selection. It has proven advantageous if the deflection signal is adapted to the speech signal in terms of its spectral characteristics and / or its energy. In this way, the spectral center of gravity of the deflection signal can be adapted to the spectral center of gravity of the corresponding speech section at the specific point in time, e.g., by means of single-sideband modulation. A speech section with a high spectral center of gravity can thus be masked with a deflection signal with a similarly high spectral center of gravity (possibly even with the same spectral center of gravity), which leads to greater masking effectiveness.The energy of the distraction signal can also be adjusted to the energy of the speech section in order to avoid creating a masking signal that is too loud and excessively disturbing.

[0021] In a further embodiment of the invention, the distraction signal can be represented during output by means of a multi-channel spatial reproduction, preferably by multiplication with binaural spectra of an acoustic transfer function, thereby generating a multi-channel (at least 2-channel) representation of the distraction signal, which enables a spatial reproduction of the distraction signal. The spatial reproduction increases the effect of the distraction signal on masking speech at the listening location, particularly if the distraction signal in the other audio zone is output spatially in such a way that it appears to originate from a random direction and / or close to the head of the listener in the other audio zone. This spatialization reduces the distinguishability of speech and distraction signals and makes monitoring the speech signal more difficult due to the deflection signal, and the energy for the distraction signal can thus be reduced.

[0022] The processing of the speech signal described above and the generation of a masking signal are preferably performed in the digital domain. This requires steps not described in detail, such as analog-to-digital conversion and digital-to-analog conversion, which will, however, be obvious to those skilled in the art after studying the present disclosure. Furthermore, the above method can be implemented entirely or partially by means of a programmable device, which in particular comprises a digital signal processor and the required analog-to-digital converters.

[0023] According to a further aspect of the invention, a device for generating a masking signal in a zone-based audio system is proposed, which device receives a speech signal to be masked and generates the masking signal based on the speech signal. The device comprises means for transforming the detected speech signal into spectral bands; means for swapping spectral values ​​of at least two spectral bands; and means for generating a noise signal as a masking signal based on the swapped spectral values.

[0024] The above embodiments of the method described therein can also be applied to this device. Thus, the device can further comprise: means for determining a time point in the speech signal relevant for speech intelligibility; means for generating a deflection signal for the relevant time point; and means for adding the noise signal and the deflection signal and outputting the sum signal as a masking signal.

[0025] In a further embodiment of the device, it also comprises means for generating a multi-channel representation of the masking signal, which enables a spatial reproduction of the masking signal.

[0026] According to a further aspect of the invention, a zone-based audio system with a plurality of audio zones is disclosed, wherein at least one audio zone has a microphone for detecting a speech signal and another audio zone has at least one loudspeaker. The microphone and loudspeaker can be arranged in the headrests of vehicle occupant seats. It is also possible for both audio zones to have a microphone and loudspeaker. The audio system has a device for generating a masking signal, as shown above, which receives a speech signal from a microphone in one audio zone and sends the masking signal to the loudspeaker(s) in the other audio zone.

[0027] A further aspect of the present disclosure relates to the above-described generation of a deflection signal as a masking signal independent of the aforementioned noise signal. A corresponding method for masking a speech signal in a zone-based audio system comprises: detecting a speech signal to be masked in one audio zone; determining a point in time in the speech signal that is relevant for speech intelligibility; generating a deflection signal for the specific point in time, wherein the deflection signal can be adapted to the speech signal with regard to a spectral characteristic and / or its energy; and outputting the deflection signal at the specific point in time as a masking signal in the other audio zone. The possible embodiments of the method correspond to the embodiments described above in combination with the generated noise signal.

[0028] A corresponding device for generating a deflection signal as a masking signal in a zone-based audio system is also disclosed. The device receives a speech signal to be masked and generates the masking signal based on the speech signal. It comprises means for determining a time point in the speech signal relevant for speech intelligibility; means for generating a deflection signal for the relevant time point, wherein the deflection signal can be adapted to the speech signal with regard to a spectral characteristic and / or its energy; and means for outputting the deflection signal as a masking signal. Optionally, means for generating a multi-channel representation of the masking signal, which enables spatial reproduction of the masking signal, can be provided.

[0029] The features described above can be combined in a variety of ways, even if such a combination is not explicitly mentioned. In particular, features described for a method can also be used for a corresponding device, and vice versa.

[0030] In the following, exemplary embodiments of the invention are described in more detail with reference to the schematic drawings. In the drawings: Fig. 1 schematic of an example of a zone-based audio system; Fig. 2 schematic of another example of a zone-based audio system; Fig. 3 schematic of another example of a zone-based audio system with two zones; Fig. 4 schematic of another example of a zone-based audio system with multiple zones; Fig. 5 an example of a block diagram for generating a broadband masking signal for speech concealment; and Fig. 6An example of a block diagram for generating a distraction signal to mask speech.

[0031] The exemplary embodiments described below are non-limiting and are to be understood as purely illustrative. For illustrative purposes, they include additional elements that are not essential to the invention. The scope of protection is to be determined solely by the appended claims.

[0032] The following exemplary embodiments enable vehicle occupants, regardless of their seating position, to conduct undisturbed private conversations, such as telephone calls with other people outside the vehicle. For this purpose, an audio masking signal is generated and played to other vehicle occupants, disrupting their perception of the conversation and making it difficult, and ideally impossible, for unintentional comprehension of the private conversation. This creates a private space for the speaker in which they can conduct private conversations undisturbed, without the risk of other vehicle occupants overhearing confidential information. The conversation could, for example, be a telephone conversation or a conversation between vehicle occupants.In the latter case, there are two speakers who alternately emit speech signals that other passengers should not understand, although the intelligibility of the speech between the two conversation participants should not be impaired.

[0033] Similar scenarios generally arise when people are located in acoustic zones or acoustic environments within a room, each of which is exposed to sound through separate acoustic reproduction devices. Such acoustic zones can exist, for example, in means of transport such as vehicles, trains, buses, airplanes, ferries, etc., in which passengers are seated in seats that are each equipped with acoustic reproduction devices. However, the proposed approach for creating private acoustic zones is not limited to these examples. It can generally be applied to situations in which people are located at specific locations within a room (e.g., in theater or cinema seats) and can be exposed to sound through individual acoustic reproduction devices, and it is possible to capture the speech signals of a speaker whose speech should not be understood by the other people.

[0034] In one embodiment, a zone-based audio system is provided to create private acoustic zones at each passenger seat of a vehicle, or more generally, an acoustic environment. The individual components of the audio system are networked with each other and can interactively exchange information / signals. Figure 1 shows schematically an example of such a zone-based audio system 1. A user or passenger is located at a seat 2 with a neck rest 3, which has two loudspeakers 4 and two microphones 5.

[0035] Such a zone-based audio system has one, preferably at least two, loudspeakers 4 for the active acoustic reproduction of personal and individual audio signals, which should not be perceived or only slightly perceived by the neighboring zones. The loudspeaker(s) 4 can be mounted in the headrest 3, the seat 2 itself, or in the vehicle's roof lining. The loudspeakers have a sufficient acoustic design and can be controlled via appropriate signal processing to minimize the acoustic influence on neighboring zones.

[0036] Furthermore, such an audio zone has the ability to record the speech of the occupant of the primary acoustic zone independently of the adjacent zones and the signals actively reproduced therein. For this purpose, one or more microphones 5 can be integrated into the seat 2 or the neck restraint 3 or mounted in the direct acoustic environment of the zone and the occupant, as shown in Figure 2 is shown schematically. Preferably, the microphones 5 are arranged in such a way that they enable the best possible recording of the speech of the occupant using the telephone. If a microphone can be placed in the immediate vicinity of the speaker's mouth (as the middle microphone in Figure 2), a single microphone is generally sufficient to capture the speaker's audio signals with sufficient quality. For example, the microphone of a telephone headset can be used to record the speech signals. Otherwise, two or more microphones are advantageous for capturing the speech in order to capture it better and, above all, more precisely using digital signal processing, as explained below.

[0037] The speaker's audio zone can be equipped with appropriate signal processing to record the primary occupant's speech signals as smoothly as possible and without being influenced by the neighboring zones and the ambient noise (wind, rolling noise, ventilation, etc.).

[0038] The voice signal of the vehicle occupant using the telephone is thus recorded at the seat position (either directly by an appropriately arranged microphone or indirectly by means of one or more remote microphones with appropriate signal processing) and separated from any interference signals, such as background noise.

[0039] From this speech signal, a masking signal, also referred to below as a speech obfuscation signal, can be generated for a listening passenger. In exemplary embodiments, a broadband masking signal adapted to the speech to be obfuscated is generated for this passenger. Additionally or alternatively, distraction signals can also be generated at the individual speech onsets within the speech of the primary speaker. These are short interference signals that are output at specific speech sections that are important for speech intelligibility and can also be adapted to the speech to be obfuscated. These distraction signals are output with a temporal overlap with the speech sections relevant for speech intelligibility in order to reduce the information content for the listener and to improve the intelligibility of the speech.to impair their interpretation (information masking) without significantly increasing the overall sound level.

[0040] Adapted to the local acoustic requirements, these masking signals can be played in a spatial manner (multi-channel), creating a spatial perception of the masking signals. This way, eavesdropping at the listening positions can be avoided as best as possible.

[0041] The proposed approach ensures that the overall sound pressure level at the seating positions of the listening passengers increases only minimally and that the annoyance or disturbance (annoyance) of the passengers is not increased or that the local listening comfort is maintained as best as possible, in contrast to an approach in which a loud noise is simply emitted to mask the speech (energetic masking).

[0042] Figure 3illustrates the functionality and basic system structure of an exemplary embodiment for two audio zones. The speech signals of the occupant of the primary acoustic zone I are captured by the microphones 5 of this zone arranged in the headrest 3 of the speaker and subjected to a first digital signal processing A in order to record the speech signals of the primary occupant as interference-free as possible and unaffected by the neighboring zones and the ambient disturbances (wind, rolling noise, ventilation, etc.). Alternatively, the microphone(s) 5 can also be arranged in front of the speaker, as in Figure 2shown, for example, in the rear part of the front passenger's headrest or in the headliner, steering wheel, or dashboard. In the example shown, the person listening is in the seat directly in front of the speaker, but this does not have to be the case, and the person listening can be located anywhere else within the vehicle.

[0043] The speech signals processed in this way are then fed to a second signal processing unit B, which generates suitable speech masking signals so that the speech intelligibility of the listening occupant is reduced. The speech masking signals are then output via the loudspeakers 4' in the second acoustic zone II. These are arranged, for example, in the headrest 3' of the listening occupant in order to achieve the most direct and undisturbed reproduction of the speech masking signals. As already mentioned, a speech masking signal can comprise a broadband masking signal adapted to the speech signal of the primary occupant and / or a distraction signal that is triggered at individual speech onsets. In this way, acoustic zones can be designed to be so private that unwanted eavesdropping beyond the boundary of an acoustic zone is made significantly more difficult.

[0044] In an alternative solution - similar to active noise suppression - the estimated speech signals at the respective listening or microphone location are reduced by actively playing adaptive cancellation signals. However, since the listening position is slightly variable in practice and the listening and microphone locations are a few centimeters apart, only speech signal components up to approximately 1.5 kHz can be actively reduced. However, since speech intelligibility is primarily dominated by consonants and thus signal components with frequencies above 2 kHz, this approach alone is inadequate or at best should be viewed critically. If the adjustment is inadequate (e.g. incorrect adjustment to the head position), the cancellation signals carry precisely the relevant private information and can even amplify it, so that speech intelligibility is increased rather than reduced.In contrast, the proposed approach is less sensitive to the exact head positions of the speaker and the listener and allows for a reduction in speech intelligibility even of higher frequency speech components such as consonants.

[0045] Due to the modularity of the proposed approach, implementations with multiple audio zones are also conceivable, such as in mass transport (rail, aircraft, train) or other application areas (entertainment, cinema, etc.). Figure 4 schematically illustrates such a multi-zone approach using a multi-row vehicle in which six acoustic zones are provided. As before, loudspeakers and microphones are integrated into the passengers' headrests, although the microphones can also be arranged in other positions in front of the respective speakers to provide a favorable arrangement for capturing the speech signals. Similar to Figure 3In this example, it is assumed that the speaker is sitting behind the unwanted listener (here, the driver). However, the speech signals of the speaking passenger can be used in the same way to generate masking or concealment signals for passengers other than the driver, and also for several unwanted listeners. Of course, the speaker can also be located in a different location in the vehicle than in the Figure 4 The approach disclosed here can be generally applied to all scenarios in which a speaker's speech can be captured and generated speech obfuscation signals can be specifically output to the unwanted listener(s).

[0046] As mentioned at the beginning, the speech signals can be a telephone conversation that the speaker has with an external person outside the room in which the acoustic zones are located. Alternatively, the conversation can also be between people in the room, for example, between the person in Figure 4 shown speaker and the passenger to their right. In this case, the zone-based audio system must provide the same signal processing for the second speaker as for the shown speaker, so that their speech is also captured and processed to generate appropriate masking signals for the listener(s). If the two speakers speak alternately, only the current speaker needs to be determined and the masking signals associated with that speaker output. If both speakers speak simultaneously, both masking signals can also be output simultaneously.

[0047] The following describes the required signal processing steps for an example application. In this application, a vehicle passenger sitting in the rear left corner acts as the internal speaker and conducts a telephone conversation with a person outside the vehicle. In addition to the internal speaker's speech, the external speaker's speech (e.g., output from the internal speaker's headrest loudspeaker) can also be used. Far End Speaker Signal ) is captured as speech to be obfuscated. This is retouched or obfuscated for the listening driver in the "front left" listening position. Of course, this is only one possible scenario, and the proposed methods can be generally applied to all possible configurations of the speaker position and listening position.

[0048] The signal estimated by digital signal processing A say yesfor the speech signal to be masked provides the base value for the subsequent generation of the masking or concealment signal. The speech signal to be masked can be the active internal speaker in the vehicle compartment and / or the external speaker outside. The concealment signal can be a broadband masking signal and / or distraction signals. These generated signals ( send two: out LS-Left & LS-Right) are reproduced via the active neck support at the listening position. In exemplary embodiments, both masking signals are generated, added together, and reproduced together in order to have an amplified effect on the listener and impair their intelligibility. The combination of the two masking signals creates a synergistic effect of these signals in reducing speech intelligibility. The continuous broadband masking signal creates a background noise, whereby the volume (energy) of the signal can be reduced compared to the output of only one noise signal, so that a less disruptive effect is achieved. By outputting the distraction signals at suitable positions (speech onsets), the speech intelligibility of these speech sections (e.g.for consonants) without significantly increasing the overall energy of the masking signal and causing additional discomfort to the listener. It has even been shown that the distracting signals are perceived as less unpleasant when presented together with the noise signal.

[0049] Figure 5 shows a schematic block diagram for the generation of a broadband speech-signal-dependent masker. The input signal is the speech signal to be masked. say yes The resulting two-channel output signals (out LS-Left & LS-Right) are sent to the active neckrest at the listening position, superimposed with distraction signals if necessary, and output to the listening person via loudspeakers mounted on / in the neckrest.

[0050] The following describes in detail the signal processing steps for generating a broadband noise signal for speech masking according to one embodiment. It should be noted that not all steps are always required, and some steps may be performed in a different order, as those skilled in the art of digital signal processing will appreciate. Also, some calculations can be performed equally well in the frequency domain or the time domain.

[0051] First, the speech signal say yes transformed into the frequency domain and smoothed both temporally and in frequency direction. For this purpose, the speech signal is first say yesdivided into blocks (for example, 512 samples at a sampling rate of fs = 44.1 kHz are arranged into blocks with a duration of 11.6 ms and 50% overlap). Subsequently, each signal block is transformed into the frequency domain in section 105 using a Fourier transform with NFFT 1 = 1024 points.

[0052] In a further step 110, the Fourier spectra are filtered with a Mel filter bank with M = 24 bands – i.e., the spectra are spectrally compressed by the Mel filter bank. The filter bank can consist of overlapping bands with a triangular frequency response. The center frequencies of the bands are distributed equidistantly across the Mel scale. The lowest frequency band of the filter bank starts at 0 Hz, and the highest frequency band ends at half the sampling rate (fs). For all bands of the filter bank, a short-term energy value (RMS level or specific loudness curves of the individual Mel bands) is calculated for each signal block in section 115 of the block diagram. These short-term energy values ​​are averaged over time in section 120 over MA = 120 blocks in the form of a moving average (120 blocks correspond to approximately 700 ms).

[0053] In exemplary embodiments, these dynamic loudness curves are interchanged in the immediate frequency range (scrambling) in section 125. To do this, the loudness values ​​of the bands are interchanged according to the following table, with the assignment of the "in" band resulting from the corresponding position in the "out" row below. For example, the loudness value of band number 2 is assigned to band number 4, and the value of band 4 is assigned to band 5, whose value is assigned to band 3, etc. This results in the loudness values ​​being interchanged with neighboring or next-but-one bands, i.e., the difference between a mel band and a swapped band in this example is a maximum of two mel bands. Of course, the table shown is only one possible example of band interchange, and other implementations are possible. Band assignment in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 out 1 4 2 5 3 6 7 10 8 11 9 12 13 16 14 17 15 18 19 22 20 23 21 24

[0054] Using the proposed band swapping, the loudness values ​​are "scrambled" to create a certain degree of "disorder" in the distribution of loudness values ​​for a corresponding speech segment, thereby altering the description of its spectral energy or loudness distribution without changing the overall energy or loudness of the speech segment. For example, a particularly pronounced energy content in one band is shifted to another band, or a low energy (loudness) in one band is transformed to an adjacent band. It has been shown that by redistributing the energy into adjacent bands, a particularly effective broadband noise signal can be generated, which reduces the intelligibility of the corresponding speech segment more than without band swapping.By swapping or rotating the sequence of bins of the temporal dynamic responses of the masking bands, the transmission of speech information in the noise signal is avoided. If speech energy were captured in frequency bands (e.g., mel bands as described above) and these temporal energy responses were directly modulated onto a noise signal, also divided into equal frequency bands, the speech content would become audible—even more intelligible if narrow frequency bands were used. This effect is significantly reduced by swapping the loudness values.

[0055] The potentially swapped dynamic loudness curves can be adjusted based on the current background spectra (including all interference noise) in section 130 of the block diagram in order to evaluate background noise and the ambient situation. For this purpose, the background noise is recorded, for example, at the monitoring position and, similar to the speech signal, the background spectra are determined using frequency transformation and temporal and frequency averaging. A microphone arranged at the monitoring position is preferably used for this purpose. Alternatively, microphones arranged elsewhere (but preferably close to the monitoring position) can be used to record the background noise at the monitoring position. Only those bands of the speech signal that lie above the background spectrum need to be taken into account when generating the masking signal.Speech bands whose energy lies below the energy of the corresponding background noise band can be neglected, as they play no role in speech intelligibility or are already masked by background noise. This can be achieved, for example, by setting the loudness value of such speech bands to zero. In other words, if a frequency band is already masked by strong background noise, no additional masking signal is generated in that frequency band. This allows a situation-based decision as to which signal components of the broadband masking noise are introduced to mask speech.

[0056] In section 135, the resulting masking thresholds (frequency axis sampled at 24 frequencies corresponding to the 24 center frequencies of the MEL filter bank) are interpolated at all frequency points of the Fourier transform. This interpolation generates a spectral value for the speech signal for the entire frequency range of the Fourier transform, for example, 1024 values ​​for the aforementioned Fourier transform with NFFT 1 = 1024 points.

[0057] Finally, in section 155, the frequency support points are multiplied pointwise (or convolved in the time domain) with a noise spectrum. This can be obtained by a noise generator (not shown), whose noise signal is analogous to the speech signal. say yesthrough a block segmentation 145 and Fourier transformation 150 with identical dimensions. In this way, a broadband noise signal is generated as a masking signal with a similar frequency characteristic (apart from the swapping and zeroing of sections 125 and 130) as the speech signal. Alternatively, the masking signal can also be generated in the time domain by convolving the noise signal with the spectral values ​​of the speech signal processed as described above (see sections 100 to 135), transformed back into the time domain. By switching between the frequency and time domains, different frequency resolutions or time durations can be used in the various processing steps. Alternatively, it is also possible to perform the entire processing in the frequency domain. For each block of the speech signal, a broadband noise spectrum adapted to the speech section of the block is generated.

[0058] In exemplary embodiments, section 160 is followed by spatial processing by point-by-point multiplication of the frequency support points (or convolution in the time domain, see above) with binaural spectra of an acoustic transfer function, which corresponds to the source direction of the speaker (or the dominant direction of the energy center of the speech signal to be masked) from the perspective of the listening person. The source direction of the speaker is known from the spatial arrangement of the acoustic zones. In the Figure 4 In the example shown, the speaker's source direction is directly behind the listening person. In embodiments with spatial alignment of the masking signal, multi-channel playback (e.g., using two loudspeakers) is required. Otherwise, single-channel playback is sufficient, preferably using two loudspeakers arranged in the listening person's neckrest.

[0059] The broadband masking signal can thus be reproduced spatially and adapted to the target direction of the direct signal or the prominently perceived direction of the speaker. Due to binaural loudness addition, masking is significantly improved with lower level excesses of the masking noise.

[0060] In Section 165, an inverse transformation (IFFT) of the two resulting spectra (per block) into the time domain is performed, followed by a superposition of the blocks using the overlap-add method (see Section 170). It should be noted that for spatial reproduction, a multi-channel signal is created, which can be reproduced, for example, in stereo. If the previous steps have already been performed in the time domain, the inverse transformation and superposition of the blocks are obviously unnecessary.

[0061] The resulting time signals are sent to the respective active neck support of the listener. In embodiments where distraction signals are also generated, the masking signals can be summed with the distraction signals before being output via the neck support's loudspeakers.

[0062] As already mentioned, signal processing can be performed partially in the frequency domain or in the time domain, although it is also possible to perform the entire processing in the frequency domain. The specific values ​​mentioned above are only examples of possible configurations and can be changed in many ways. For example, a frequency resolution of the FFT transform with fewer than 1024 points or a division of the MEL filters with more or fewer than 24 filters is possible. It is also possible for the frequency transformation of the noise signal to be performed with a different block size and / or FFT configuration than that of the speech signal. In this case, the interpolation in section 135 would have to be adjusted accordingly to generate suitable frequency values.In a further variation, the masking noises calculated block by block are first transformed back into the time domain after interpolation and then converted back into the frequency domain to take spatialization into account—possibly with a different spectral resolution. Those skilled in the art will recognize such variations of the inventive method for generating a broadband speech-signal-dependent masking signal after studying the present disclosure.

[0063] In exemplary embodiments, instead of masking noise, short-duration distraction signals are used, which are temporally and / or frequency-matched to sections of the speech signal that are particularly relevant for intelligibility. An example of generating such distraction signals is described below. Figure 6shows a schematic example of a block diagram for generating speech-signal-dependent distraction signals. The distraction of the listener occurs at signal-dependent, defined points in time. For this purpose, the critical points in time (ti,distract ) are determined based on three information parameters in the speech signal: spectral centroid "SC" (roughly corresponds to pitch), short-term energy "RMS" (roughly corresponds to loudness), and number of zero crossings "ZCR" (to distinguish between speech signal and background noise).

[0064] A series of pre-selected distraction signals (e.g. bird calls, chirps, ...) with associated parameters (SC and RMS), determined through additional preliminary analyses, are stored in a digital memory. Suitable distraction signals preferably have the following properties: Firstly, they are natural signals that are familiar to the listener from other situations / from everyday life and are therefore not related to the signal and context to be masked. Furthermore, they are characterized by the fact that they represent acoustically distinctive signals of short duration and have the broadest possible spectrum. Other examples of such signals are the sound of dripping water or water waves or brief gusts of wind. The distraction signals are usually longer than the relevant speech segments (e.g. consonants) and mask them completely.It is also possible to store deflection signals of different lengths and select them to match the duration of the current critical time.

[0065] A distraction signal is selected and adjusted in time and frequency to the current speech segment. The adjusted distraction signal can then be played back to the listener from a virtual spatial position. For spatialization (BRTF), short impulse responses (256 points) can be used to simulate the outer ear transfer function, ensuring that these distraction signals are localized as close and present as possible to the listener's head, thus achieving a strong distraction effect. Multi-channel playback (e.g., stereo) is required for spatial playback.

[0066] The following describes in detail the signal processing steps for generating discrete, spatially distributed, short deflection signals according to one embodiment. It should be noted that not all steps are always required, and some steps may be performed in a different order, as those skilled in the art will appreciate. Also, some calculations can be performed equally well in the frequency domain or the time domain. Some of the processing steps correspond to those for generating broadband masking signals and therefore need not be repeated in embodiments that use both signal types to mask speech.

[0067] In section 200, the speech signal say yes divided into blocks (BlockLength = 512 samples, fs = 44.1kHz) with a duration of 11.6 ms and 50% overlap (HopSize = 256) (see section 100).

[0068] From these blocks XBuffer n (m), where n = block index and m = time sample, the number of zero-crossing rates (ZCR) per signal block is determined in section 205. This can be done using the following formula: ZCR n = 0.5 ∗ ∑ m = 1 m BlockLength − 1 sgn XBuffer n m + 1 − sgn XBuffer n m

[0069] In section 210, each signal block is subjected to a Fourier transform with NFFT 2 = 1024 points (see section 105).

[0070] From these spectra S(k,n) with k = frequency index and n = block index, two further parameters are calculated in sections 215 and 220: the short-time energy (RMS) and the spectral centroid (SC): RMS n = ∑ k ‖ S k n ‖ 2 SC n = ∑ k = 1 NFFT 2 2 + 1 k ∗ S k n ∑ k = 1 NFFT 2 2 + 1 S k n

[0071] The short-term energy RMS and zero-crossing rate (ZCR) curves can also be filtered using signal-dependent thresholds, and ranges that do not meet these thresholds can be masked out (e.g., set to zero). The thresholds can, for example, be selected so that a certain percentage of the signal values ​​lie above or below them.

[0072] Each spectrum is spectrally smoothed in both directions in section 225 using a recursive 1st order discrete-time filter: H(z) = Bs(z) / As(z), where Bs = 0.3 and As(z) = 1 - (Bs-1)*z -1< (= acausal, zero-phase 2nd order filter).

[0073] The resulting spectra are time-smoothed in section 230 using a recursive first-order discrete-time filter: H(z) = Bt(z) / At(z), where Bt = 0.3 and At(z) = 1 - (Bt-1)*z -1<.

[0074] To detect sections of the speech signal relevant to speech intelligibility (onsets) (onset detection), an onset detection function is first determined in section 235. For this purpose, the spectrally and temporally averaged spectra are summed along the frequency axis. The resulting signal is logarithmized and time-differentiated, with negative values ​​set to zero. Regularization (e.g., adding a small number to all frequency support points) can be performed prior to logarithmization to avoid zero values.

[0075] This onset detection function is examined for local maxima, which must be separated by at least a specified number of blocks. The maxima thus found can be further filtered using a signal-dependent threshold, leaving only particularly pronounced maxima. Local maxima of the onset detection function determined in this way are candidates for perceptually relevant sections of the speech signal, which are to be selectively disrupted using a distraction signal.

[0076] In exemplary embodiments, the maxima of the onset detection function determined in this way are checked for plausibility in section 240 via a logic unit using the parameters: ZCR, RMS, and SC. Only if these values ​​​​lie within a defined range are these maxima considered relevant, critical times. ten,distractThis can be done, for example, by specifying that at the times of determined maxima of the onset detection function, the values ​​of RMS, SC and / or ZCR must meet certain logical conditions (e.g. RMS>X1; X2 <SC<X3; ZCR> X4 with predetermined threshold values ​​for X1 to X4). In exemplary embodiments, for example, only those maxima are considered that lie in time periods that satisfy the above-mentioned filter conditions for RMS and ZCR (i.e., do not lie in masked areas). The condition that ZCR and RMS must simultaneously satisfy certain threshold conditions can also be used to filter the curve of SC by retaining the values ​​of SC when the threshold conditions are met and interpolating or extrapolating intermediate values, thereby creating the function SC int.

[0077] At the determined times ten,distractOne deflection signal is randomly selected from a selection of N digitally stored deflection signals (using section 245) in a memory 250. Additional metadata for these deflection signals is found in the memory 250: SC and RMS values.

[0078] The selected deflection signal is divided into blocks in section 255 (see above with BlockLength = 2 and Hopsize = BlockLength = 2 or Overlap = 0) and then subjected to a 2-point Fourier transformation using NFFT in section 260. The parameters of this frequency transformation can be different and independent of the above description for the speech signal to be masked. Alternatively, the frequency representation of a deflection signal could also be stored directly in the frequency domain.

[0079] The resulting spectra can be viewed in section 265 depending on the signal sig est at the respective time ten,distract based on the SCParameter ratios in the frequency range (e.g., by single-sideband modulation) and / or based on the RMS parameter ratios in the gain can be adjusted. For this purpose, the ratio of the spectral centers of gravity SC of the respective speech signal section at an onset time ten,distract and the corresponding deflection signal, and the frequency position of the deflection signal is adjusted so that it matches that of the speech signal as closely as possible. This can be achieved by determining the value of the function SC int of the interpolated spectral center of gravity at an onset time SC int ( ten,distract ) is compared with the SC value of the selected deflection signal and a detuning parameter is determined, whereby positive values ​​of the detuning parameter mean an increase in the pitch of the deflection signal by means of single-sideband modulation and negative values ​​lead to a lowering of the pitch.

[0080] The energy (RMS) of the distraction signal is also adjusted to the energy of the speech signal portion, achieving a predetermined energy ratio for the distraction signal to the speech signal. Due to their high effectiveness in reducing speech intelligibility, the distraction signals can be played at a low volume, so that the overall sound pressure level at the seats of the listening passengers increases only minimally, preventing any annoyance or disturbance to passengers and maintaining the best possible listening comfort.

[0081] In embodiments, the resulting modified spectra of the deflection signals are determined depending on a random direction selection t i , distractTime in section 270 is spatially variable by a binaural spatial transfer function (BRTF) by point-wise multiplication of the frequency support points (or convolution in the time domain) of the corresponding spectra. For this purpose, a direction is randomly selected for a deflection signal in section 275. Memory 280 contains binaural spatial transfer functions (BRTF) matching the possible directions. As already explained above for the masking noise, the spatialization can be carried out in the frequency or time domain. In the time domain, a convolution with the impulse response of a selected outer ear transfer function is carried out for this purpose. The spatialization of the deflection signals is preferably carried out in such a way that the distraction signals are localized by the listener as close and present as possible to the head, so that they achieve a strong distraction effect. For spatial reproduction, a multi-channel (e.g.in stereo) playback is required, otherwise single-channel playback would be sufficient, but this would preferably also be achieved via two loudspeakers integrated in the headrest.

[0082] In the case of spatialization of the deflection signal in the frequency domain, the convolution results are transformed back to the time domain using an inverse Fourier transform (IFFT) with NFFT 2 points in Section 285. The inversely transformed time blocks are combined in Section 290 using the overlap-add method to form a time signal in a sequential and value-correct manner. If the previous steps have already been performed in the time domain, the inverse transformation and the superposition of the blocks are, of course, unnecessary.

[0083] The resulting time signals are sent to the respective active neck restraint of the listener. In embodiments where masking noise signals are also generated, the masking signals can be summed with the distraction signals before being output via the neck restraint's loudspeakers.

[0084] The speech-signal-adapted distraction signal generates randomly spatially distributed stimulator / trigger information and masks the speech target signal in an improved manner without significant permanent signal levels.

[0085] As already mentioned, the signal processing can be carried out partially in the frequency domain or in the time domain. The specific values ​​mentioned above are only examples of a possible configuration of the frequency transformation and can be changed in many ways. In one possible variation, the energy- and frequency-adjusted spectra (see section 265) are first transformed back into the time domain and then brought back into the frequency domain in order to take spatialization into account there—possibly with a different spectral resolution. However, it is also possible to carry out the entire processing in the frequency domain. Those skilled in the field of digital signal processing will recognize such variations of the inventive procedure for generating speech-signal-dependent distraction signals after studying the present disclosure.

[0086] In exemplary embodiments, both obfuscation signals – broadband masking noise and distraction signals – are summed before output and reproduced together. The masking noise, which is preferably perceived from the direction of the speaker, generates a broadband noise signal adapted to the spectral properties of the respective speech section, onto which short distraction signals are selectively superimposed (in terms of both time and frequency) at particularly relevant locations. These distraction signals are perceived spatially close to the head and lead to a particularly effective reduction in speech intelligibility, even when reproduced at low volume or energy. However, by combining them with the broadband masking noise, the brief switching on and off of the distraction signals is perceived as less disturbing or impairing.The overall sound pressure level at the seating positions of the listening passengers increases only minimally and the annoyance or impairment of the passengers is not increased or the local hearing comfort is maintained as best as possible.

[0087] The above description of exemplary embodiments includes a large number of details that are not essential to the invention defined by the claims. The description of the exemplary embodiments serves to understand the invention and is purely illustrative and should be understood without limiting the scope of protection. Those skilled in the art will recognize that the described elements and their technical effects can be combined with one another in various ways to create further exemplary embodiments covered by the claims. Furthermore, the described technical features can be used in devices and methods, for example embodied by programmable devices. They can be implemented in particular by hardware elements or by software. As is known, digital signal processing is preferably implemented by specially designed signal processors.Communication between individual components of the described device can be wired (e.g., via a bus system) or wireless (e.g., via Bluetooth or Wi-Fi). Protection is also expressly claimed for a computer-implemented implementation and the associated program or machine code in the form of data storage media or in a downloadable representation.

Claims

1. A method for masking a voice signal in a zone-based audio system, comprising: capturing a voice signal to be masked in an audio zone; determining a point in time relevant for speech intelligibility in the voice signal; generating a deflection signal for the determined point in time, wherein the deflection signal is adapted to the voice signal with regard to a spectral characteristic and / or its energy; and outputting the deflection signal as a masking signal at the determined point in time in another audio zone, the method being characterized in that it comprises the following method steps: transforming the captured voice signal into spectral bands; swapping spectral values of at least two spectral bands; generating a noise signal based on the swapped spectral values; and outputting the noise signal as a further masking signal for the voice signal in the other audio zone.

2. The method of claim 1, wherein the point in time relevant for speech intelligibility is determined on the basis of extreme values of a spectral function of the voice signal, wherein the spectral function is determined based on an addition of, optionally averaged, spectral values over the frequency axis.

3. The method of claim 1 or 2, wherein the point in time relevant for speech intelligibility is verified on the basis of parameters of the voice signal, such as zero crossing rate, short-term energy and / or spectral centroid.

4. The method of any one of claims 1 to 3, wherein the deflection signal for the determined point in time is randomly selected from a set of predetermined deflection signals.

5. The method of any one of claims 1 to 4, wherein the deflection signal is spatially represented in the other audio zone during output by means of a multi-channel playback, preferably by multiplication with binaural spectra of an acoustic transfer function.

6. The method of claim 5, wherein the deflection signal is spatially output in the other audio zone such that it appears to originate from a random direction and / or close to the head of a listener in the other audio zone.

7. The method of any one of the preceding claims, wherein generating a noise signal based on the swapped spectral values comprises: generating a wideband noise signal; transforming the generated noise signal into the frequency domain; and multiplying the frequency representation of the noise signal by a frequency representation of the voice signal taking into account the swapped spectral values, wherein optionally the frequency representation of the voice signal is generated by an interpolation of the spectral values of the bands after the swapping of spectral values.

8. The method of any one of the preceding claims, further comprising: estimating a background noise spectrum; comparing spectral values of the voice signal with the background noise spectrum; and taking into account only spectral values of the voice signal that are greater than the corresponding spectral values of the background noise spectrum.

9. The method of any one of the preceding claims, wherein the transformation of the captured voice signal into spectral bands is performed for blocks of the voice signal and by means of a Mel filter bank and optionally a temporal smoothing of the spectral values is performed for the Mel bands.

10. The method of any one of the preceding claims, wherein the noise signal is spatially represented in the other audio zone during output by means of a multi-channel playback, preferably by multiplication with binaural spectra of an acoustic transfer function, wherein optionally the noise signal is spatially output in the other audio zone such that it appears to originate from the dominant direction of the speaker of the voice signal to be masked.

11. A device for generating a masking signal in a zone-based audio system, receiving a voice signal to be masked in an audio zone and generating the masking signal based on the voice signal, comprising: means for determining a point in time relevant for speech intelligibility in the voice signal; means for generating a deflection signal for the relevant point in time, wherein the deflection signal is adapted to the voice signal with regard to a spectral characteristic and / or its energy; and means for outputting the deflection signal as a masking signal at the determined point in time in another audio zone, the device characterized by comprising the following means: means for transforming the captured voice signal into spectral bands; means for swapping spectral values of at least two spectral bands; and means for generating a noise signal as a masking signal based on the swapped spectral values; and means for adding the noise signal and the deflection signal and for outputting the sum signal as a masking signal.

12. The device of claim 11, further comprising: means for generating a multi-channel representation of the masking signal enabling a spatial playback of the masking signal.

13. A zone-based audio system comprising a plurality of audio zones, wherein one audio zone comprises at least one microphone for capturing a voice signal and another audio zone comprises at least one loudspeaker, wherein microphone and loudspeaker are preferably arranged in neck supports of seats for occupants of a vehicle, wherein the audio system comprises a device for generating a masking signal in accordance with claim 11 or 12, which receives a voice signal from a microphone of the one audio zone and sends the masking signal to the loudspeaker(s) of the other audio zone.

Images