Method and apparatus for processing an initial audio signal - Patents.com
The method optimizes speech intelligibility in audio and audiovisual media by using neural networks to condition and evaluate audio signals, ensuring minimal impact on the sound scene, thus addressing the challenge of maintaining quality and cost-effectiveness in speech enhancement.
Patent Information
- Application Number
- JP2022573351
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-05-29
- Publication Date
- 2024-11-11
- Estimated Expiration
- 2040-05-29
AI Technical Summary
Existing methods for improving speech intelligibility in audio and audiovisual media often compromise the sound aesthetic by either manually creating alternative mixes or reducing background noise, which is costly and inefficient, or they fail to maintain the original sound quality during automated processing.
A method involving signal conditioning and evaluation using neural networks to optimize speech intelligibility while preserving the sound scene by selecting the best conditioned audio signal based on evaluation criteria such as perceptual similarity and speech intelligibility, allowing for automatic adjustment of audio signals to meet individual preferences.
This approach effectively enhances speech intelligibility while minimizing changes to the original sound quality, providing a cost-effective and automated solution that maintains the intended sound aesthetic.
Smart Images

Figure 0007580495000001 
Figure 0007580495000002 
Figure 0007580495000003
Abstract
Description
[Technical field]
[0001] Embodiments of the present invention relate to methods and corresponding devices for processing an initial audio signal (such as a recording or raw data). Preferred embodiments relate to techniques (methods and algorithms) for improving speech intelligibility and for listening to broadcast audio material. [Background technology]
[0002] When producing and broadcasting audio and audiovisual media (e.g. movies, TV, radio, podcasts, YouTube® videos), it is not always possible to ensure sufficient speech intelligibility in the final sound mix, for example due to the addition of excessive background sounds (music, sound effects, noise in the recording, etc.).
[0003] This is particularly problematic for those with hearing impairments, but improving speech intelligibility can also be beneficial for those with normal hearing or non-native speaker listeners.
[0004] A fundamental problem in producing audio and audiovisual media is that the background signals (music, sound effects, ambience) make up part of the important sound aesthetics of the production, i.e. they cannot be considered as "interference noise" that should be eliminated as much as possible. Therefore, all methods aimed at improving speech intelligibility or reducing listening effort in this application should additionally take into account that the original intended nature of the sound should be altered as little as possible in order to address the high quality requirements and creative aspects of sound production. However, currently there is no single technical method or tool to guarantee an optimal trade-off between good intelligibility and preservation of the sound scene / recording.
[0005] However, there are various technological approaches that can fundamentally result in improved speech intelligibility (or reduced listening effort) in audio and audiovisual media.
[0006] One solution could be for expert sound engineers to manually create alternative audio mixes, so that the end user can freely choose between the original mix and a speech-enhanced mix. The speech-enhanced mix could also be created, for example, by employing hearing loss simulation to ensure that the intended mix is suitable for listeners with the target hearing loss.[1] However, such manual methods would be very cost-prohibitive and inapplicable to the majority of produced audio / audiovisual media.
[0007] As another solution to provide automatic signal enhancement, there are various methods for reducing or eliminating undesired signal parts (eg, interference noise), but these are different from the technical approach of the present invention.
[0008] Improving speech intelligibility by mixed signal interference reduction methods: Such methods aim to process a mixed signal containing both a signal of interest (e.g. speech) and an interfering signal (e.g. background noise) in such a way that as much of the interfering noise as possible is removed, while the signal of interest is theoretically left intact (e.g. the method according to [2]). Since these methods have to estimate the respective portions of the component of interest and the interfering noise components in the mixed signal, they are always based on assumptions about the physical properties of the signal components. Such algorithms are used, for example, in hearing aids and mobile phones, are prior art and are under continuous development.
[0009] In recent years, more and more methods based on machine learning (neuronal networks) have been presented that aim to separate different sources in a mixed signal. Based on large amounts of data, these methods are trained for a specific problem (e.g., separating several speakers in a mix [3]) and can essentially be used to extract speech from atmosphere / music in audiovisual media, thus providing the basis for remixes with improved SNR. In [4] such an approach is presented to give the user the option to adjust the speech to background ratio himself.
[0010] Improved speech intelligibility by pre-processing the speech signal: In some applications, the target signal (e.g. speech) is separate from other signal parts, so it is not a mixed signal as described above, and the method does not need to estimate which signal components correspond to the target and the interference noise. This is the case for example for train station announcements. At the same time, at the signal processing level, the interference noise cannot be affected, i.e. it is not possible to remove or reduce the interference noise (e.g. the noise of a passing train interfering with the intelligibility of the station announcement). For such application scenarios, there are methods to adaptively pre-process the target signal such that its intelligibility is optimized or improved in the currently existing interference noise (e.g. the method of [5]). Such methods use, for example, band-pass filtering, frequency-dependent amplification, time delay and / or dynamic compression of the target signal, and basically become applicable also for audiovisual media when the background noise / ambience should not be (significantly) modified.
[0011] Encoding objects and background noise as separate audio objects: Additionally, methods exist for parametrically encoding information about the object signal when encoding and transmitting the audio signal, allowing the energy of the object signal to be adjusted separately during decoding at the receiver. Increasing the energy of the object of interest (e.g., speech) relative to other audio objects (e.g., ambience) can result in improved speech intelligibility
[11] .
[0012] Detection and level adaptation of speech signals in a mixed signal: Moreover, technical systems exist that identify speech points in a mixed signal and adjust these points with the aim of improving speech intelligibility, for example by increasing their volume. Depending on the adjustment type, this will only improve speech intelligibility if no further interfering noise is simultaneously present in the mixed signal
[12] .
[0013] Lowering channels that do not primarily contain speech: In multi-channel audio signals that are mixed in such a way that one channel (usually center) contains most of the speech information and the other channels (e.g., left / right) contain primarily background noise, one technical solution is to attenuate the non-speech channels by a fixed gain (e.g., by 6 dB) and improve the signal-to-noise ratio in that way (e.g., adapting downmix rules for sound retrieval system (SRS) dialog clarity or surround decoders).
[0014] In such methods, it may happen that background noise parts that are already very low and have practically no negative effect on speech intelligibility are also attenuated. This can lead to a reduction in the overall impression of sound aesthetics, since the ambience intended by the sound engineer is no longer perceptible. To prevent this, US Patent No. 8,577,676 B2 describes a method in which non-speech channels are only attenuated to the effect that a speech intelligibility metric reaches a certain threshold, but no further. Furthermore, US Patent No. 8,577,676 B2 discloses a method in which several frequency-dependent attenuations are calculated, each attenuation having the effect that a speech intelligibility metric reaches a certain threshold. The option that maximizes the loudness of the background noise is then selected from several options. This is based on the assumption that this preserves the original sound properties as best as possible.
[0015] Based thereon, US Patent Application Publication No. 2016 / 0071527 A1 describes a method for not attenuating non-speech channels, or attenuating them less, when, contrary to common assumptions, they also contain relevant speech information and thus lowering them may adversely affect intelligibility. This document also includes a method in which multiple frequency-dependent attenuations are calculated and the one that maximizes the loudness of the background noise is selected (again, based on the assumption that this preserves the original sound properties as best as possible).
[0016] Both US patent documents describe in their independent claims very specific methods (e.g., scaling the drop coefficient with the probability of speech occurrence) that are not necessary for the invention described herein, and therefore the invention can be realized without using the techniques disclosed in US Patent No. 8,577,676 B2 and US Patent Application Publication No. 2016 / 0071527 A1.
[0017] US Patent No. 8,195,454 B2 describes a method for detecting parts in an audio signal where speech occurs by using voice activity detection (VAD). One or several parameters are then modified for these parts (e.g. dynamic range control, dynamic equalization, spectral sharpening, frequency transposition, speech extraction, noise reduction, or other speech enhancement actions) so that a speech intelligibility metric (e.g. speech intelligibility index (SII), [6]) is either maximized or raised above a desired threshold. In this case, hearing loss or the preferences of the listener or noise in the listening environment may also be taken into account.
[0018] US Patent No. 8,271,276 B1 describes adapting the loudness or level of speech segments with an amplification factor that depends on the preceding time segment. This is not relevant to the core of the invention described herein, and would only be relevant if the invention described herein simply changed the loudness or level of a segment identified as speech according to the preceding segment. No adaptation of the audio signal beyond the amplification of the speech segment is included, such as source separation, background noise reduction, spectral variation, dynamic compression, etc. Therefore, the steps disclosed in US Patent No. 8,271,276 B1 are also not adversely affected. [Prior art documents] [Patent documents]
[0019] [Patent Document 1] U.S. Pat. No. 8,577,676 [Patent Document 2] US Patent Application Publication No. 2016 / 0071527 [Patent Document 3] U.S. Pat. No. 8,195,454 [Patent Document 4] U.S. Pat. No. 8,271,276 Summary of the Invention [Problem to be solved by the invention]
[0020] The object of the present invention is to provide a concept which allows to improve the trade-off between (speech) intelligibility and preservation of the sound scene. [Means for solving the problem]
[0021] This object is solved by the subject matter of the independent claims.
[0022] One embodiment of the present invention provides a method for processing an initial audio signal including a target portion (e.g., a speech portion) and a sub-portion (e.g., an ambient noise portion), comprising the following four steps: 1. receiving an initial audio signal; 2. adjusting the received initial audio signal by using a first signal conditioner to obtain a first conditioned audio signal, and adjusting the received initial audio signal by using a second signal conditioner to obtain a second conditioned audio signal; 3. evaluating the first adjusted audio signal against the evaluation criterion to obtain a first evaluation value representative of a satisfaction degree of the evaluation criterion, and evaluating the second adjusted audio signal against the evaluation criterion to obtain a second evaluation value representative of a satisfaction degree of the evaluation criterion; 4. Selecting the first or second adjusted audio signal depending on the respective first or second evaluation value. Includes.
[0023] According to embodiments, the evaluation criteria can be one or more of the group including perceptual similarity, speech intelligibility, loudness, sound pattern, and spatiality. It should be noted that the selecting step can be based on a plurality of independent first and second evaluation values representing independent evaluation criteria according to embodiments. The evaluation criteria and in particular the selecting step may depend on a so-called optimization object. Thus, according to embodiments, the method includes a step of receiving information on an optimization object defining personal preferences, and the evaluation criteria depend on the optimization object, or the adjusting step and / or the evaluating step and / or the selecting step depend on the optimization object, or the weighting of the independent first and second evaluation values representing independent evaluation criteria for the selecting step depends on the optimization object.
[0024] For example, if the optimization target is a combination of two factors, such as optimal speech intelligibility and acceptable perceptual similarity between the initial audio signal and the adjusted audio signal, weighting for selection may be performed. For example, these two criteria, i.e., speech intelligibility and perceptual similarity, may be evaluated separately, thereby determining respective evaluation values for the evaluation criteria, and then selection is made based on the weighted evaluation values. The weighting depends on the optimization target, which in turn may be set by personal preferences.
[0025] According to embodiments, the adapting, evaluating and selecting steps may be performed by using neuro-neural networks / artificial intelligence.
[0026] According to a preferred embodiment, it is assumed that the speech intelligibility improves in a sufficient way with two or more used adjusters. Expressed from another perspective, this means that only adjusters that allow a sufficiently high improvement in speech intelligibility or that output signals for which speech intelligibility is sufficient are taken into account. In a next step, a selection between the differently adjusted signals is made. For this selection, perceptual similarity is used as evaluation criterion, which makes it possible to carry out steps 3 and 4 (see method above) as follows: 3. comparing the received initial audio signal with the first adjusted audio signal to obtain a first perceptual similarity value representative of a perceptual similarity between the initial audio signal and the first adjusted audio signal, and comparing the received initial audio signal with the second adjusted audio signal to obtain a second perceptual similarity value representative of a perceptual similarity between the initial audio signal and the second adjusted audio signal; 4. The substep of selecting the first or second adjusted audio signal depending on the respective first or second perceptual similarity value.
[0027] According to one embodiment of the invention, the first adjusted audio signal is selected when the first perceptual similarity value is higher than the second perceptual similarity value (a high first perceptual similarity value indicates a higher perceptual similarity of the first adjusted audio signal), and conversely, the second adjusted audio signal is selected when the second perceptual similarity value is higher than the first perceptual similarity value (a high second perceptual similarity value indicates a higher perceptual similarity of the second adjusted audio signal). According to further embodiments, instead of perceptual similarity another value may be used, such as a loudness value.
[0028] This adaptation method with the step 3 of comparing and the step 4 of selecting based on the perceptual similarity values can be augmented according to further embodiments by an additional step of evaluating the first and second adjusted signals after step 2 and before step 3 against another optimization criterion, for example against speech intelligibility. As mentioned above, in this case some adjusted signals may not be taken into account, since this first evaluation criterion is not (sufficiently) met, for example when the speech intelligibility is too low. Alternatively, it may be possible that all evaluation criteria are taken into account during the step of selecting unweighted or weighted. This weighting may be selected by the user. According to embodiments, the method further comprises a step of outputting the first or second adjusted audio signal depending on the selection.
[0029] An embodiment of the present invention provides a method, where the target portion is a speech portion of an initial audio signal and the sub-portion is an ambient noise portion of the audio signal.
[0030] The embodiments of the present invention are based on defining that different speech intelligibility options differ in terms of their improvement effect, which depends on several influencing factors, for example on the input audio stream or the input audio scene. The optimal speech intelligibility algorithm may also differ from scene to scene within an audio stream. Therefore, the embodiments of the present invention analyze the different adjustments of the audio signal, in particular with regard to the perceptual similarity between the initial audio signal and the adjusted audio signal, and select the adjuster / adjusted audio signal with the highest perceptual similarity. This system / concept allows for the first time to satisfy both requirements, i.e. to improve the speech intelligibility (or reduce the listening effort) of the initial signal while at the same time affecting the sound aesthetics component as little as possible, as the preservation of this sound aesthetics represents a significant component of user acceptance that has not been taken into account in automated methods so far, and thus represents a considerable reduction in effort and cost compared to non-automated methods that have been used so far to improve intelligibility only as a boundary condition, as well as a large added value for those methods.
[0031] According to one embodiment, the step of outputting the initial audio signal is performed instead of outputting the first or second adjusted audio signal when the respective first or second perceptual similarity value falls below a threshold value, "below" indicating that the adjusted signal is not sufficiently similar to the initial audio signal. This is advantageous because the system allows both automatic testing of the sound mix for speech intelligibility or listening effort, while at the same time ensuring that the overall sound is perceptually altered in an effective way.
[0032] An embodiment of the present invention provides a method, in which the comparing step comprises extracting the first and / or second perceptual similarity values by using a (perceptual) model such as the PEAQ model [8], the POLQA model [9] and / or the PEMO-Q model
[10] . It is noted that PEAQ, POLQA, PEMO-Q are specific models trained to output a perceptual similarity of two audio signals. According to embodiments, the degree of processing is controlled by a further model.
[0033] It should be noted that according to one embodiment, the first and / or second perceptual similarity value depends on physical parameters of the first or second adjusted audio signal, volume levels of the first or second adjusted audio signal, psychoacoustic parameters for the first or second adjusted audio signal, loudness information of the first or second adjusted audio signal, pitch information of the first or second adjusted audio signal, and / or perceived source width information of the first or second adjusted audio signal.
[0034] An embodiment of the present invention provides a method, in which the first and / or second signal conditioner is configured to perform SNR increase (e.g., for the initial audio signal), dynamic compression (e.g., for the initial audio signal), and / or the adjusting step includes increasing the object part, increasing a frequency weighting for the object part, dynamically compressing the object part, decreasing the sub-part, decreasing a frequency weighting for the object part if the initial audio signal includes a separate object part and a separate sub-part, or the adjusting step includes performing separation of the object part and the sub-part if the initial audio signal includes a combined object part and sub-part. Generally, this means that an embodiment of the present invention provides a method, in which the first and / or second adjusted audio signal includes an object part moved to the foreground and a sub-part moved to the background, and / or a speech part as the object part moved to the foreground and ambient noise as the sub-part moved to the background.
[0035] According to one embodiment, the selecting step is performed taking into account one or more further factors, such as the hearing loss grade of the hearing-impaired person, the individual's hearing performance; the individual's frequency-dependent hearing performance; the individual's preferences, and / or the individual's preferences regarding the signal adjustment rate. Similarly, according to embodiments, the adjusting and / or comparing step is performed taking into account one or more factors, such as the hearing loss grade of the hearing-impaired person, the individual's hearing performance; the individual's frequency-dependent hearing performance; the individual's preferences, and / or the individual's preferences regarding the signal adjustment rate. Thus, the selecting, adjusting, and / or comparing steps may also take into account the individual's hearing or individual preferences.
[0036] According to embodiments, models for controlling the processing may be configured with respect to, for example, hearing loss or personal preferences.
[0037] According to one embodiment, the comparing step is performed for the entire initial audio signal and the entire first and second adjusted audio signals, or for target portions of the individual audio signals which are compared with the target portions of the respective first and second adjusted audio signals, or for sub-portions of the initial audio signal which are compared with sub-portions of the first and second adjusted audio signals.
[0038] An embodiment of the present invention provides a method, which further comprises an initial step of analyzing an initial audio portion to determine a speech portion, an initial step of comparing the speech portion with an ambient noise portion to evaluate the speech intelligibility of the initial audio signal, and an initial step of activating a first and / or a second signal conditioner for the adjustment step if the indicative value for the speech intelligibility is below a threshold. It is therefore advantageous that the processing is performed only where speech occurs. In this case, an adjusted sound mix is generated for this speech portion, the sound mix being aimed at meeting or maximizing a certain perceptual metric.
[0039] One embodiment of the present invention provides a method, where the initial audio signal comprises multiple time frames or scenes, and the basic steps are repeated for each time frame or scene.
[0040] According to embodiments, a first time frame may be adapted using a first adjuster, and for a second time frame, another adjuster may be selected. To ensure perceptual continuity, transitions between time frames, or adaptation parts of two time frames, may be inserted. For example, the end of the first time frame and the beginning of the following time frame are adapted in terms of their adaptation performance. For example, some kind of interpolation between the two adaptation methods may be applied. According to further embodiments, the same adjuster may be used for all or several following time frames to allow perceptual continuity. According to further embodiments, adaptation of the time frames may also be performed even if there is no adaptation required, for example from the point of view of intelligibility performance. However, this allows to ensure perceptual similarity between the respective time frames.
[0041] An embodiment of the present invention provides a computer program having a program code for executing, when run on a computer, in accordance with the above method.
[0042] Another embodiment of the present invention provides an apparatus for processing an initial audio signal, the apparatus comprising an interface for receiving the initial audio signal, respective adjusters for processing the initial audio signal to obtain respective adjusted audio signals, an evaluator for performing an evaluation of the respective adjusted audio signals, and a selector for selecting the first or second adjusted audio signal depending on the respective first or second evaluation value.
[0043] Further details are defined by the subject matter of the dependent claims. In the following, embodiments of the invention are discussed in detail with reference to the accompanying figures. [Brief description of the drawings]
[0044] [Figure 1]2 shows a schematic diagram of a method sequence for processing an audio signal to improve the reproduction quality of a target portion, such as a speech portion, of the audio signal according to a base embodiment; [Diagram 2] 1 is a schematic flow chart illustrating an enhanced embodiment. [Diagram 3] 2 is a schematic block diagram of a decoder for processing an audio signal according to one embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0045] Subsequently, embodiments of the present invention will be discussed below with reference to the accompanying figures, in which objects having the same or similar functions are given the same reference numerals.
[0046] 1 shows a schematic flow chart illustrating a method 100 including three steps / step groups 110, 120 and 130. The method 100 has the objective of enabling processing of an initial audio signal AS and can have the result of outputting an adjusted audio signal MOD AS. A possible outcome of the output audio signal MOD AS can be that no processing of the audio signal AS is required, so an assumption is used. In that case, the audio signal and the adjusted audio signal are the same.
[0047] The three basic steps 110 and 120 are construed here as step groups, since the sonar steps 110a, 110b, etc., and 120a, may be performed in parallel or sequentially with one another.
[0048] Within the group of steps 110, the audio signal AS is processed separately by using different adjusters / processing techniques. Here, two exemplary steps of applying a first and a second adjuster are shown, marked by the references 110a, 110b. Both steps can be performed in parallel or sequentially to each other to process the audio signal AS. The audio signal may be, for example, an audio signal including an audio track, which includes two signal parts. For example, the audio track may include a speech signal part (target part) and an ambient noise signal part (sub-part). These two parts are marked by the references AS_TP and AS_SP. In this embodiment, it is assumed that the AS_TP should be extracted from the audio signal AS or should be identified within the audio signal AS in order to amplify this signal part AS_TP to enhance speech intelligibility. This processing can be performed for an audio signal having only one audio track containing the two parts AS_SP and AS_TP, without separating an audio signal AS that contains multiple audio tracks, e.g. one audio track for AS_SP and one audio track for AS_TP.
[0049] As mentioned above, there are several possible adjustments of the audio signal AS that make it entirely possible to improve speech intelligibility, for example by amplifying the AS_TP part or by reducing the AS_SP part. Further examples are lowering non-speech channels, dynamic range control, dynamic equalization, spectral sharpening, frequency transposition, speech extraction, noise reduction, or other speech enhancement actions discussed in the context of the prior art. The efficiency of these adjustments depends on several factors, for example on the recording itself, on the format of the AS (for example a format with only one audio track or a format with multiple audio tracks), or on several other factors. To enable optimal speech intelligibility, at least two signal adjustments are applied to the signal AS. Within a first step 110a, the received initial audio signal AS is adjusted by using a first adjuster to obtain a first adjusted audio signal 1st MOD AS. Independently of step 110a, a second adjustment of the received initial audio signal AS is performed by using a second adjuster to obtain a second adjusted audio signal 2nd MOD AS. For example, the first adjuster may be based on dynamic range control and the second adjuster may be based on spectral sharpening. Of course, other adjusters may be used instead of the first and / or second adjuster or as a third adjuster (not shown), for example based on dynamic equalization, frequency re-transposition, speech extraction, noise reduction or speech enhancement actions, or a combination of such adjusters. All approaches may result in different resulting adjusted audio signals 1st MOD AS and 2nd MOD AS, which may differ in terms of speech intelligibility and in terms of similarity to the initial audio signal AS. These two parameters, or at least one of these two parameters, is evaluated in the next step 120.
[0050] In detail, in step 120a, the first adjusted audio signal 1st MOD AS is compared with the original audio signal AS to find a similarity. Similarly, in step 120b, the second adjusted audio signal 2nd MOD AS is compared with the initial audio signal AS. For the comparison, the entity performing step 120 receives the audio signal AS directly and the first / second MOD AS. The result of this comparison is a first perceptual similarity value and a second perceptual similarity value, respectively. The two values are marked by the symbols 1st PSV and 2nd PSV. Both values represent the perceptual similarity between the respective first / second adjusted audio signal 1st MOD AS, 2nd MOD AS and the initial audio signal AS. Under the assumption that the improvement of speech intelligibility is sufficient, the first or second adjusted audio signal with the first / second PSV showing a higher similarity is selected. This is done by a selecting step 130.
[0051] The result of the selection may be output / transferred according to embodiments, thereby enabling the method 100 to output the respective adjusted audio signal 1st MOD AS or 2nd MOD AS that is most similar to the original signal. As can be seen, the adjusted audio signal MOD AS still comprises two parts AS_SP' and AS_TP'. As indicated by "'" in AS_SP' and AS_TP', both or at least one of the two parts AS_SP' and AS_TP' are adjusted. For example, the amplification for AS_TP' may be increased.
[0052] According to a further embodiment, an enhanced evaluation may be performed within step 120. In this case, it is then further verified whether the adjustments made by the first or second adjuster (see steps 110a and 110b) are sufficient to improve speech intelligibility. For example, it may be analyzed whether the ratio of AS_TP' to AS_SP' is greater than the ratio of AS_TP to AS_SP.
[0053] The above embodiment starts from the assumption that the objective of the method 100 is for the MOD AS to have improved speech intelligibility. According to further embodiments, the objective of the adjustment may be different. For example, the part AS_TP may be another part, generally the target part, to be emphasized within the whole adjusted signal MOD AS. This can be done by emphasizing / amplifying AS_TP' and / or by adjusting AS_SP'.
[0054] Also, the above embodiment of FIG. 1 is discussed in the context of perceptual similarity. It should be noted that this approach can be used more generally for other evaluation criteria. FIG. 1 starts from the assumption that the evaluation criterion is perceptual similarity. However, according to further embodiments, still another evaluation criterion can be used instead or additionally. For example, speech intelligibility can be used as evaluation criterion. In such a case, an evaluation of the first adjusted audio signal 1st MOD AS is performed instead of step 120a, and an evaluation of the second adjusted audio signal 2nd MOD AS is performed in step 120b. The results of these two evaluating steps 120a and 120b are the respective first and second evaluation values. Then, step 130 is performed based on the respective evaluation values.
[0055] Further evaluation criteria can be such as loudness or auditory spaciousness.
[0056] Further embodiments with enhanced functionality are described below with reference to FIG.
[0057] FIG. 2 shows a schematic flow chart that allows processing an audio signal AS that includes two parts: AS_TP (speech S) and AS_SP (ambient noise N). Here, a signal conditioner 11 is used to process the signal AS, so that a selection entity 13 can output an adjusted signal mode AS. In this embodiment, the conditioner performs different adjustments 1, 2, ..., M, which are based on a number of different models to generate three adjusted signals 1st MOD AS, 2nd MOD AS, and M MOD AS. For each signal 1st MOD AS, 2nd MOD AS, and M MOD AS, two parts are illustrated: S1', N1', S2', N2', and SN', NNM'. The output signals 1st MOD AS, 2nd MOD AS, and M MOD AS are evaluated by an evaluator 12 with respect to their perspective similarity to the initial signal AS. Thus, one or more evaluator stages 12 receive the signal AS and the respective adjusted signals 1st MOD AS, 2nd MOD AS and M MOD AS. The output of this evaluator 12 is the respective adjusted signals 1st MOD AS, 2nd MOD AS and M MOD AS together with respective similarity information. Based on this similarity information, a position stage 13 determines which adjusted signal MOD AS to output.
[0058] According to embodiments, the signal AS may be analyzed by an analyzer 21 to determine whether speech is present or not. This decision step is marked 21 if there is no speech present in the initial audio signal AS or there is no signal to adjust. The initial / original audio signal AS is used as is, i.e. without adjustment (see N-MOD AS).
[0059] If speech is present, the second analyzer 22 analyzes whether the speech intelligibility needs to be improved. This decision point is marked with the symbol 22s. If no adjustment is needed, the original signal AS is used as the signal to be output (see N-MOD AS). If adjustment is recommended, the signal conditioner 11 is enabled.
[0060] Based on this structure, an improvement of speech intelligibility in audio and audiovisual media is possible. Here, the sound mix to be processed can be either a complete mix or composed of separate audio tracks or sound objects (e.g., dialogue, music, reverberation, effects). In a first step, the signal is analyzed for the presence of speech (see references 21, 21s). The speech active parts will be further analyzed for physical or psychoacoustic parameters, for example in the form of calculated values of speech intelligibility (e.g., SII) or listening effort, for example based on the method for mixed signals presented in [7] (see references 22, 22s). Based on this evaluation, a decision is made whether the speech intelligibility is sufficient or whether sound adaptation is required, by comparing the parameters with targets or thresholds. If adaptation is not required, sound mixing is performed as usual or the original mix AS is maintained. If adaptation is required, an algorithm will be applied that adjusts the audio tracks or different audio tracks so that the desired intelligibility is obtained. So far, this method is similar to the techniques disclosed in US Pat. No. 8,195,454 B2 and US Pat. No. 8,271,276 B1, but is not limited to the details set forth in claim 1 of each.
[0061] This means that according to embodiments, this concept provides a model-based selection 13 of a sound reduction method beyond the maximization of loudness of non-speech channels as described, for example, in US Pat. No. 8,577,676 B2 and US Pat. Appl. Pub. No. 2016 / 0071527 A1. For the selection, a further model stage 12 is applied, which simulates the perceptual similarity between the original mix AS and the differently modified mixes (1st MOD AS, 2nd MOD AS, M MOD AS) based on physical and / or psychoacoustic parameters. Here, the original mix AS as well as the different types of modified mixes 1st MOD AS, 2nd MOD AS, M MOD AS serve as inputs to the further model stage 12.
[0062] With the goal of preserving the sound scene as best as possible, the method for sound adaptation that obtains the desired intelligibility with the least perceptually noticeable signal adjustment can be selected (see reference numeral 13).
[0063] According to embodiments, perceptual similarity can be measured in an instrumental way; possible models that may be used here are, for example, PEAQ [8], POLQA [9], or Pemo-Q
[10] . Also or additionally, further physical metrics (e.g., level) or psychoacoustic metrics (e.g., loudness, pitch, perceived source width) may be used to assess perceptual similarity.
[0064] Typically, an audio stream includes different scenes that are organized along the time domain. Therefore, according to various embodiments, different sound adaptations can be performed at different times in the audio track AS in order to have the least intrusive perceptual effect. For example, if the speech sound AS_TP and the background noise AS_TP already have clearly different spectra, a simple SNR adaptation can be the best solution, since it maintains the authenticity of the background noise to the best possible effect. If more speakers overlap the target speech sound, other methods (e.g., dynamic compression) to meet the optimization target can be better.
[0065] According to further embodiments, this model-based selection can take into account possible hearing impairments of future listeners of the audio material in the calculations, e.g. in the form of audiograms, personal roundness functions or input of personal sound preferences, whereby speech intelligibility is guaranteed not only for people with normal hearing abilities but also for people with certain forms of hearing impairment (e.g. age-related hearing loss), also taking into account possible individual differences in the perceptual similarity between the original and the processed version.
[0066] The analysis of speech intelligibility and perceptual similarity by the model, and the respective signal processing, may be performed on the entire sound mix, or only on parts of the mix (individual scenes, individual dialogues), or on short time windows along the entire mix, allowing a decision to be made for each window whether sound adaptation has to be performed.
[0067] An example of such a process is illustratively discussed below. i. No sound adaptation: If the analysis of the listening model shows that a sufficiently high speech intelligibility is guaranteed, no further sound adaptation is performed. Alternatively, the following adaptation is performed to avoid perceptual differences between different scenes. Also, an "interpolation" between no processing and the following selected processing may be performed. Both models allow perceptual continuity across different time frames / scenes. For separate audio tracks of dialogue and background noise, the following steps are possible: ii. Adapting the sound signal: Only the audio track of the speech signal is processed to improve speech intelligibility, for example by level increasing, by frequency weighting and / or single-channel or multi-channel dynamic compression. iii. Adapting interference noise: One or some of the speechless audio tracks are processed to improve speech intelligibility, for example by lowering the level, by frequency weighting and / or single-channel or multi-channel dynamic compression. However, if it were self-evident that a complete elimination of background noise would result in improved speech intelligibility, this would be impractical for reasons of sound aesthetics, as the design of music, effects, etc. is also an essential part of creative sound design. iv. Adapting all audio tracks: The audio track of the speech signal and one or some of the other audio tracks are both processed by the method described above for improving speech intelligibility.
[0068] It should be noted that for the adaptation artificial intelligence, for example using neuronal networks, may be used. It should be noted that in an already mixed audio signal (i.e. non-separate audio tracks for speech and background noise), steps ii-iv can also be performed when a source separation method is used beforehand, for example, which separates the mix into speech and one or several background noises. In that case, the improvement of speech intelligibility may consist in, for example, remixing the separate signals at an improved SNR, or adjusting the speech signal and / or the background noise or parts of the background noise by frequency weighting or single-channel or multi-channel dynamic compression. Here again, a sound adaptation will be selected that both improves speech intelligibility as necessary and at the same time preserves the original sound as best as possible. The method for source separation may be applied without an explicit stage for detecting speech activity.
[0069] It should be noted that according to embodiments, the selection of the respective treatment may be performed by using an artificial intelligence / neuron network, which may be used, for example, when there are multiple factors for the selection, such as a perceptual value and a loudness value, or a value that represents a match to an individual's listening preferences.
[0070] Above, it has been discussed that adaptation of scenes can be done even if not necessary, in order to maintain perceptual continuity across different time frames / scenes. According to another variant, adaptation for multiple or all scenes can be selected. Furthermore, it should be noted that certain transitions between different scenes, differently adapted or between adapted and non-adapted scenes, can be integrated in order to maintain perceptual continuity.
[0071] According to embodiments, the evaluation and optimization based on perceptual similarity (see reference numeral 12) may concern the target language, background noise, or a mix of speech and background noise. For example, there may be different thresholds for the perceptual similarity of the processed speech signal, the processed background noise, or the processed mix to the respective original signals, which may not exceed a certain signal adjustment degree for each signal. A further boundary condition may be that the background noise (such as music) does not change perceptually much with respect to the preceding or following time point, otherwise the perceptual continuity will be disturbed, for example, if at the moment when speech is present, the music is turned down too much or its frequency content changes, or the actor's speech may not change much in the movie flow. Such boundary conditions may be considered based on the model described above.
[0072] This may have the effect that the desired intelligibility improvement may not be obtained without too much interference with the perceptual similarity of the speech and / or background noise, where a (possibly configurable) decision stage is allowed to decide which objectives to obtain or trade-offs to find and how to find them.
[0073] Here, the process can be iterative, i.e. the listening model can be examined again after sound adaptation, to verify that the desired speech intelligibility and perceptual similarity to the original has been achieved.
[0074] Processing may be performed for the entire duration of the audio material or only for parts of it (eg scenes, dialogues) (depending on the calculations of the listening model).
[0075] The embodiments can be used for all audio and audiovisual media (film, radio, podcasts, audio renderings in general). Possible commercial applications are e.g. i. An internet-based service where the customer loads their audio material, activates the automated speech intelligibility improvement and downloads the processed signal, which can be extended by a customer-specific selection of the sound adaptation method and the degree of sound adaptation. Such services already exist, but do not use the listening model for sound adaptation for speech intelligibility (see under 2.(V.) above). ii. Software solutions in the form of tools for sound production, for example integrated into digital audio workstations (DAW), allowing the correction of filed or currently produced sound mixes. iii. A testing algorithm for identifying portions of the audio material that do not correspond to the desired speech intelligibility and possibly offering the user the option to select recommended sound adaptation adjustments. iv. Software and / or hardware integrated into end devices on the listener's side of the broadcast chain, such as, for example, sound bars, headphones, television devices or any device that receives streamed audio content.
[0076] The method discussed in the context of Figure 1 or the concept discussed in the context of Figure 2 may be implemented by using a processor, which is illustrated by Figure 3.
[0077] Figure 3 shows a processor 10 in two stages, a signal conditioner 11 and evaluators / selectors 12 and 13. The conditioner receives the audio signal from the interface and performs adjustments based on different models to obtain an adjusted audio signal MOD AS. The evaluator / selector 12 receives the audio signal from the interface and performs adjustments based on different models to obtain an adjusted audio signal MOD AS. The evaluators / selectors 12, 13 evaluate the similarity and based on this information select the signal with the highest or high similarity and improved speech intelligibility that is sufficient to output the MOD AS.
[0078] Of course, these two stages 11, 12 and 13 may be implemented by one processor.
[0079] Although some aspects are described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, with blocks or devices corresponding to method steps or features of method steps. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item, or feature of a corresponding apparatus. Some or all of the method steps may be performed by (or using) a hardware apparatus, such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, some or one or more of the most important method steps may be performed by such an apparatus.
[0080] The encoded audio signal of the present invention may be stored in a digital storage medium or transmitted in a transmission medium, such as a wireless transmission medium or a wired transmission medium such as the Internet.
[0081] Depending on the particular implementation requirements, the embodiments of the present invention can be implemented in hardware or in software. Implementation can be done using digital storage media, such as floppy disks, DVDs, Blu-Ray, CDs, ROMs, PROMs, EPROMs, EEPROMs, or FLASH memories that store electronically readable control signals, which cooperate (or can cooperate) with a programmable computer system to execute the respective methods. Thus, the digital storage media can be computer readable.
[0082] Some embodiments according to the invention include a data carrier having electronically readable control signals capable of cooperating with a programmable computer system to perform one of the methods described herein.
[0083] Generally, embodiments of the present invention can be implemented as a computer program product including program code that operates to perform one of the methods when the computer program product runs on a computer. The program code can, for example, be stored on a machine readable carrier.
[0084] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.
[0085] In other words, therefore, an embodiment of the invention is a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
[0086] A further embodiment of the inventive method is therefore a data carrier (or digital storage medium, or computer readable medium) having recorded thereon a computer program for performing one of the methods described herein. The data carrier, digital storage medium, or recorded medium is typically tangible and / or non-transitory.
[0087] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein, the data stream or the sequence of signals being for example adapted to be transferred via a data communication connection, for example via the Internet.
[0088] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.
[0089] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
[0090] Further embodiments according to the invention include an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, or a memory device. The apparatus or system may, for example, include a file server for transferring the computer program to the receiver.
[0091] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.
[0092] The above-described embodiments merely illustrate the principles of the present invention. It is to be understood that modifications and variations of the devices and details described herein will be apparent to those skilled in the art. It is therefore intended to be limited only by the scope of the appended claims and not by the specific details presented in the description and illustration of the embodiments herein.
[0093] (References) [1] Simon, C. and Fassio, G.,(2012), Optimierung audiovisueller Medien fuer Hoergeschaedigte,In: Fortschritte der Akustik - DAGA 2012,Darmstadt,March 2012 [2] Ephraim, Y. and Malah, D.,(1984),Speech enhancement using a minimum-mean square error short-time spectral amplitude estimator,IEEE Transactions on Acoustics Speech and Signal Processing,32(6):1109-1121 [3] Kolbaek, M., Yu, D., Tan, Z-H., and Jensen, J.,(2017),Multitalker Speech Separation With Utterance-Level Permutation Invariant Training of Deep Recurrent Neural Networks,IEEE Transactions on Audio, Speech and Language Processing,25(10),1901-1913,https: / / doi.org / 10.1109 / TASLP.2017.2726762 [4] Jouni, P., Torcoli, M., Uhle, C., Herre, J., Disch, S., Fuchs, H.,(2019),Source Separation for Enabling Dialogue Enhancement in Object-based Broadcast with MPEG-H,JAES 67,510-521,https: / / doi.org / 10.17743 / jaes.2019.0032 [5] Sauert, B. and Vary, P.,(2012),Near end listening enhancement in the presence of bandpass noises,In: Proc. der ITG-Fachtagung Sprachkommunikation,Braunschweig,September 2012 [6] ANSI S3.5,(1997),Methods for calculation of speech intelligibility index [7] Huber, R., Pusch, A., Moritz, N., Rennies, J., Schepker, H., Meyer, B.T.,(2018),Objective Assessment of a Speech Enhancement Scheme with an Automatic Speech Recognition-Based System,ITG-Fachbericht 282: Speech Communication,10-12, October 2018 in Oldenburg,86-90 [8] ITU-R Recommendation BS.1387: Method for objective measurements of perceived audio quality (PEAQ) [9] ITU-T Recommendation P.863: Perceptual objective listening quality assessment
[10] Huber, R. and Kollmeier, B.,(2006),PEMO-Q - A New Method for Objective Audio Quality Assessment Using a Model of Auditory Perception,IEEE Transactions on Audio, Speech, and Language Processing,14(6),1902-1911
[11] NetMix player of Fraunhofer IIS,http: / / www.iis.fraunhofer.de / de / bf / amm / forschundentw / forschaudiomulti / dialogenhanc.html
[12] https: / / auphonic.com / [Explanation of symbols]
[0094] 11 Signal conditioner 12 Evaluators 13 Selector 21 Analyzer 22 Second Analyzer 100 ways AS Initial Audio Signal AS_TP Speech sound, target part AS_SP Ambient noise, secondary part AS_SP',AS_TP' part MOD AS Modulated audio signal 1st MOD AS, 2nd MOD AS, M MOD AS adjustment signal 1st PSV, 2nd PSV perceptual similarity value
Claims
1. A method (100) for processing an initial audio signal (AS) comprising a target part (AS_TP) and a sub-part (AS_SP), comprising: a. receiving the initial audio signal (AS); b. adjusting (110, 110a) the received initial audio signal (AS) by using a first signal conditioner to obtain a first adjusted audio signal (1st MOD AS); adjusting (110, 110b) the received initial audio signal (AS) by using a second signal conditioner to obtain a second adjusted audio signal (2nd MOD AS); c. evaluating (120, 120a) the first adjusted audio signal against a criterion to obtain a first performance value (1st PSV) representative of a satisfaction degree of the criterion; evaluating (120, 120b) the second adjusted audio signal against the evaluation criteria to obtain a second evaluation value (2nd PSV) representative of a satisfaction degree of the evaluation criterion; d. selecting (130) the first or second adjusted audio signal (1st MOD AS, 2nd MOD AS) depending on the respective first or second evaluation value (1st PSV, 2nd PSV); Including, the selecting step is performed based on a plurality of independent first evaluation values and an independent second evaluation value, or based on at least two independent evaluation criteria; the evaluation criterion is perceptual similarity, and step c a sub-step of comparing (120, 120a) the received initial audio signal (AS) with the first adjusted audio signal (1st MOD AS) to obtain a first perceptual similarity value (1st PSV) as a first evaluation value representative of the perceptual similarity between the initial audio signal (AS) and the first adjusted audio signal (1st MOD AS); a sub-step of comparing (120, 120b) the received initial audio signal (AS) with the second adjusted audio signal (2nd MOD AS) to obtain a second perceptual similarity value (2nd PSV) as a second evaluation value representative of the perceptual similarity between the initial audio signal (AS) and the second adjusted audio signal (2nd MOD AS); A method (100).
2. The evaluation criteria are: - a perceptual similarity represented by first and second perceptual similarity values, the first and second perceptual similarity values representing a perceptual similarity between the first and second adjusted audio signals, respectively, and the initial audio signal (AS); - speech intelligibility in the form of a calculated value of speech intelligibility for comparison with a target or threshold; - loudness, expressed by a loudness value; - sound patterns, - Spatial degree The method (100) of claim 1, in the group comprising:
3. 3. The method (100) of claim 1 or 2, wherein the selection is made based on weighted first and second evaluation values after the at least two independent evaluation criteria are evaluated separately such that a respective first evaluation value representative of a degree of satisfaction for the at least two independent evaluation criteria for the first adjusted audio signal and a respective second evaluation value representative of a degree of satisfaction for the at least two independent evaluation criteria for the second adjusted audio signal are determined.
4. When the first perceptual similarity value (1st PSV) is higher than the second perceptual similarity value (2nd PSV), the first adjusted audio signal (1st MOD AS) is selected to indicate a higher perceptual similarity of the first adjusted audio signal (1st MOD AS); when the second perceptual similarity value (2nd PSV) is higher than the first perceptual similarity value (1st PSV), the second adjusted audio signal (2nd MOD AS) is selected to indicate a higher perceptual similarity of the second adjusted audio signal (2nd MOD AS), 4. The method (100) according to any one of claims 1 to 3.
5. 5. The method (100) according to any one of claims 1 to 4, further comprising the step of outputting the first or second adjusted audio signal (1st MOD AS, 2nd MOD AS) depending on the selection of step d.
6. 4. The method (100) of claim 3, wherein the step of outputting the initial audio signal (AS) is performed instead of outputting the first or second adjusted audio signal (1st MOD AS, 2nd MOD AS) when the respective first or second perceptual similarity value (1st PSV, 2nd PSV) falls below a threshold, whereby the respective first or second adjusted audio signal (1st MOD AS, 2nd MOD AS) is indicated as not sufficiently similar to the initial audio signal (AS).
7. 7. The method (100) of claim 1, wherein the target portion (AS_TP) is a speech portion of the initial audio signal (AS) and the sub-portion (AS_SP) is an ambient noise portion of the initial audio signal (AS).
8. 8. The method (100) according to any one of claims 1 to 7, wherein the first and / or second adjusted audio signal (1st MOD AS, 2nd MOD AS) comprises the object part (AS_TP) moved to the foreground and the sub-part (AS_SP) moved to the background, and / or a speech part as the object part (AS_TP) moved to the foreground and an ambient noise part as the sub-part (AS_SP) moved to the background.
9. 9. The method (100) according to any one of claims 1 to 8, wherein the comparing step comprises the step of extracting the first and / or second evaluation values (1st PSV, 2nd PSV) by using a perceptual model, a PEAQ model, a POLQA model, and / or a PEMO-Q model.
10. 10. The method (100) according to any one of claims 1 to 9, wherein the first and / or second evaluation value (1st PSV, 2nd PSV) depends on physical parameters of the first or second adjusted audio signal (1st MOD AS, 2nd MOD AS), on a volume level of the first or second adjusted audio signal (1st MOD AS, 2nd MOD AS), on psychoacoustic sound parameters for the first or second adjusted audio signal (1st MOD AS, 2nd MOD AS), on loudness information of the first or second adjusted audio signal (1st MOD AS, 2nd MOD AS), on pitch information of the first or second adjusted audio signal (1st MOD AS, 2nd MOD AS) and / or on perceived source width information of the first or second adjusted audio signal (1st MOD AS, 2nd MOD AS).
11. the first and / or second signal conditioner is configured to perform SNR increase, dynamic compression, and / or SNR decrease on the initial audio signal (AS); and / or and / or, if the initial audio signal (AS) comprises a separate target part (AS_TP) and a separate sub-part (AS_SP), the adjusting step comprises: increasing the target part (AS_TP); increasing a frequency weighting for the target part (AS_TP); dynamically compressing the target part (AS_TP); decreasing the sub-part (AS_SP); decreasing a frequency weighting for the sub-part (AS_SP); and / or said adjusting step comprising, if said initial audio signal (AS) comprises a combined target part (AS_TP) and a sub-part (AS_SP), a step of separating said target part (AS_TP) and said sub-part (AS_SP); 11. The method (100) according to any one of claims 1 to 10.
12. The step of selecting (130) may include the following factors: - the grade of hearing loss in a person with hearing loss, - individual hearing performance, - an individual's frequency-dependent hearing performance, - personal preferences, - Personal preferences regarding signal adjustment rates 12. The method (100) according to any one of claims 1 to 11, taking into account one or more of:
13. The adjusting step (110) and / or the comparing step (120) may take into account the following factors: - the grade of hearing loss in a person with hearing loss, - individual hearing performance, - an individual's frequency-dependent hearing performance, - personal preferences, - Personal preferences regarding signal adjustment rates 13. The method (100) according to any one of claims 1 to 12, taking into account one or more of:
14. receiving information regarding optimization objectives defining personal preferences; 14. The method (100) according to any one of claims 1 to 13, wherein the evaluation criterion depends on the optimization object, or the adjusting and / or evaluating and / or selecting steps depend on the optimization object, or a weighting of independent first and second evaluation values representing independent evaluation criteria for the selecting step depends on the optimization object.
15. The step of comparing (120) on the entirety of the initial audio signal (AS) and on the entirety of the first and second adjusted audio signals (1st MOD AS, 2nd MOD AS), and / or for the target portion (AS_TP) of the initial audio signal (AS) and for the target portions (AS_TP) of the first and second adjusted audio signals (1st MOD AS, 2nd MOD AS), respectively; and / or for said sub-portion (AS_SP) of said initial audio signal (AS) and for each sub-portion (AS_SP) of said first and second adjusted audio signals (1st MOD AS, 2nd MOD AS), The method (100) according to any one of claims 1 to 13,
16. the initial audio signal (AS) comprises a number of time frames; Steps a to d are repeated for each time frame; and / or Steps a to d are repeated for time portions or time frames of scenes of the initial audio signal (AS).
16. The method (100) according to any one of claims 1 to 15.
17. Adaptation of the initial audio signal (AS) comprising a plurality of time frames is performed for the time frames for which the adaptation is required and for the other time frames in order to maintain perceptual continuity, or adaptation of the initial audio signal (AS) comprising a plurality of time frames is performed in an interpolated manner for the time frames for which the adaptation is required and for the other time frames in order to maintain perceptual continuity, and / or the adaptation of the first and second subsequent time frames is performed such that a transition between the first and second subsequent time frames is formed to maintain perceptual continuity.
17. The method (100) according to any one of claims 1 to 16.
18. The method further includes an initial step, the initial step being: Analysing (21) an initial audio portion to determine a speech portion; comparing said speech portion with an ambient noise portion to assess speech intelligibility of said initial audio signal (AS); activating the first and / or second signal conditioner for the adjusting step if the indicative value for speech intelligibility is below a threshold value; 18. The method (100) of any one of claims 1 to 17, comprising:
19. A computer program having a program code which, when run on a computer, causes the computer to perform the method according to any one of claims 1 to 18.
20. An apparatus for processing an initial audio signal (AS) comprising a target part (AS_TP) and a sub-part (AS_SP), comprising: an interface for receiving the initial audio signal (AS); a first signal conditioner (11) for adjusting (110) the received initial audio signal (AS) to obtain a first adjusted audio signal (1st MOD AS), and a second signal conditioner (11) for adjusting the received initial audio signal (AS) to obtain a second adjusted audio signal (2nd MOD AS); an evaluator for evaluating (120, 120a) the first adjusted audio signal against an evaluation criterion to obtain a first evaluation value (1st PSV) representative of a satisfaction level of the evaluation criterion, and for evaluating (120, 120b) the second adjusted audio signal against the evaluation criterion to obtain a second evaluation value (2nd PSV) representative of a satisfaction level of the evaluation criterion; a selector (13) for selecting (130) said first or second adjusted audio signal (1st MOD AS, 2nd MOD AS) depending on said first or second evaluation value (1st PSV, 2nd PSV), respectively; Equipped with the selection is based on a plurality of independent first evaluation values and independent second evaluation values, or based on at least two independent evaluation criteria; the evaluation criterion is a perceptual similarity, and the evaluator comparing (120, 120a) the received initial audio signal (AS) with the first adjusted audio signal (1st MOD AS) to obtain a first perceptual similarity value (1st PSV) as a first evaluation value representative of the perceptual similarity between the initial audio signal (AS) and the first adjusted audio signal (1st MOD AS); comparing (120, 120b) the received initial audio signal (AS) with the second adjusted audio signal (2nd MOD AS) to obtain a second perceptual similarity value (2nd PSV) as a second evaluation value representing the perceptual similarity between the initial audio signal (AS) and the second adjusted audio signal (2nd MOD AS); The apparatus is configured so as to
Citation Information
Patent Citations
Component separation method of voice signal, and voice encoding method using this method
JP2000099096A
Noise suppressing device and program
JP2010160246A
Methods for evaluating voice quality, computers, computer programs, and computer program products.
JP2013500498A
Speech intelligibility predictor and applications thereof
US20110224976A1
Method and System for Scaling Ducking of Speech-Relevant Channels in Multi-Channel Audio
US20160071527A1
