Method for adjusting the sound characteristics of an audio system, as well as audio system and device
The audio system uses a sound filter and machine learning to generate and refine audio signals based on user feedback, addressing the limitations of manual parameter adjustment and enhancing sound quality and comfort.
Patent Information
- Application Number
- DE102023005435
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2023-07-06
- Publication Date
- 2024-11-14
- Estimated Expiration
- 2043-07-06
AI Technical Summary
Existing audio systems lack flexibility in personalizing sound characteristics to individual user preferences, often requiring manual adjustment of numerous audio parameters and failing to account for the spectral mix of instruments and voices, leading to a suboptimal listening experience.
An audio system generates multiple filtered audio signals with different parameter settings using a sound filter, allowing users to provide feedback, and adjusts parameters through a learning method to optimize sound characteristics based on user preferences, utilizing machine learning techniques like autoencoders and genetic algorithms to explore and optimize parameter settings.
The system enables personalized sound adjustment tailored to individual tastes, optimizing audio parameters to enhance listening experience by reducing complexity and improving sound quality and comfort.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[0001] The invention relates to a method for adjusting the sound characteristics of an audio system. By applying a sound filter to an audio signal, at least two at least partially different filtered audio signals are generated. The sound filter has different parameter settings (e.g., magnitude response), which result in a different sound characteristic and are determined by at least one audio parameter (e.g., level values for individual frequency ranges).
[0002] The sound or sound characteristics of audio systems are now optimized to be perceived as pleasant by "most people." However, the listening experience is highly individual and varies from person to person. For this reason, most audio systems have equalizer functions that allow different frequency ranges to be manually boosted or cut. Higher-quality systems often also offer the option of adjusting 3D sound to customize a spatial sound experience. In general, however, more setting options, such as compressing the dynamics of the audio signal, also mean a larger number of adjustable audio parameters.
[0003] Manufacturers are increasingly adapting their audio systems through interactions with the user, based on their feedback. During an adaptation process, or a so-called tuning procedure, specific audio content or audio signals are often played, for example, containing the sound recording of a specific musical instrument and / or covering specific frequency ranges in the human-audible spectrum. This allows a person to assess the sound characteristics of these sound samples and thus influence the system settings for different frequency ranges and / or 3D sound adjustments based on their personal taste, without requiring a detailed understanding of the technical aspects.
[0004] Audio systems that offer direct adjustment of different frequencies using equalizers require the user to familiarize themselves with technical details such as frequency responses, audio filters, or frequency ranges of the respective instruments in order to find the optimal setting. Similarly, with other parametric audio signal processing modules, such as dynamic compression, the user must familiarize themselves with the audio parameters and their effects. The more audio parameters—such as the gain of individual frequency bands, frequency band width, dynamic compression rate, and stereo image width—can be manually adjusted, the more complex it becomes to achieve a high-quality sound experience due to the sheer number of adjustment options.
[0005] Today's audio systems, which play various musical instruments and then make appropriate adjustments based on user feedback, therefore have certain limitations. Their flexibility is limited because only one or a few frequency ranges can be adjusted to individual tastes at a time. This often neglects the spectral mix and interplay of all the instruments and voices present in a piece of music, which have a decisive influence on the listening experience. Furthermore, this method only allows for sound personalization based on the music samples provided by the manufacturer. However, different parameter settings for different music genres generally lead to a significantly improved listening experience.
[0006] EP 3048026A1 discloses a method for adapting vehicle function settings or sound settings for a vehicle occupant. The adaptation occurs interactively based on feedback or evaluation provided by the occupant regarding a reproduced sound content in the form of a selection of one of at least two presented options to be compared. The method uses classified personal data to assign the vehicle user to a specific cluster. Thus, there is a statistical correlation between cluster membership and the function settings. The parameter settings used to generate the sound samples are selected from a predefined, finite set of settings. A free, direct optimization of the system's audio parameters does not occur.
[0007] DE 10 2020202372A1 describes a system for adjusting parameter settings for media playback in a vehicle. During the adjustment process, feedback or consent from a vehicle user is taken into account. Here, too, no targeted, direct optimization of the audio parameters is performed.
[0008] US 2020 / 0211 578 A1 describes a system and method for presenting a mixed reality environment to a user. A first audio control is presented within this mixed reality environment, and the user's manipulation of this first audio control is detected. Based on this manipulation, a first value of an audio-related parameter related to speech intelligibility is determined. Subsequently, two or more parameter values for audio processing related to speech intelligibility and based on the first value are determined. First speech audio signals are then received and subjected to audio processing based on the determined parameter values to generate second speech audio signals. These second speech audio signals are then transmitted to a loudspeaker system for presentation to the user for playback.
[0009] EP 1 677 574 A2 describes integrated multimedia signal processing that can eliminate unnecessary signal processors and converters without compromising the functionality of typical home entertainment system components. The integrated multimedia system includes a main player that digitally captures and processes the signals. This main player can adjust the audio signal to achieve a uniform volume across all frequencies by taking into account the varying sensitivity of the human ear to sounds of different frequencies. Furthermore, the main player can also account for perceived differences in volume due to the listener's angle to the loudspeaker by detecting the user's position and making appropriate adjustments.The invention also provides a loudspeaker having embedded performance characteristics or an identifier that enables the system to provide an optimal loudspeaker drive current for a particular system or to determine how that loudspeaker should best be implemented into the integrated system.
[0010] US 2022 / 0 329 941 A1 describes a device for equalizing audio signals using a neural network model. This device consists of at least one memory, machine-readable instructions, and a processor circuit. The functionalities of the device include at least one of the following actions: instantiating or executing machine-readable instructions to train a neural network model; applying a first audio equalization profile to a first audio signal associated with a first media variant; and applying a second audio equalization profile to a second audio signal associated with a second media variant. The processor circuit is responsible for instantiating or executing the machine-readable instructions to train, send, or execute the neural network model.
[0011] The invention is based at least on the object of enabling individual settings in an audio system for users.
[0012] The problem is solved by the subject matter of the independent patent claims. Advantageous developments of the invention are described by the dependent patent claims, the following description, and the figures.
[0013] The invention provides a method for adjusting the sound characteristics of an audio system. A sound characteristic refers to a sound and / or a sound source with specific tonal properties, such as pitch, timbre, dynamic range, and / or volume. The sound characteristic describes a transmission behavior that indicates how an audio signal is modified during processing by the audio system with respect to the aforementioned audio parameters. Thus, a distinction can be made between the sound of the audio signal itself and its modification by the sound characteristics of the audio system (e.g., attenuation of the treble).
[0014] In the method, the audio system carries out the following steps. An audio signal is provided or received in step a), wherein a sound filter of the audio system is applied to the audio signal in step b). This generates at least two further, at least partially different, filtered audio signals, wherein the sound filter each has different parameter settings, which result in a different sound characteristic that is determined by at least one audio parameter. A sound filter, e.g. an equalizer and / or dynamic compressor, can be a filter as an electronic component for sound design and / or for equalizing audio frequencies and / or other frequency-based signals, such as modulated data signals.
[0015] In other words, the sound filter is used to generate two further filtered audio signals from one audio signal, each of which is at least partially different.
[0016] Using a sound filter, unwanted frequencies can be removed and / or reduced from an audio signal. Additionally or alternatively, desired frequencies can be amplified and / or the dynamics and / or spatial perception can be changed. This allows a sound characteristic to be specifically imposed on the audio signal and / or unwanted noise to be eliminated. For example, a low-pass filter can attenuate high frequencies, while a high-pass filter attenuates low frequencies. The at least two filtered audio signals are then reproduced or output in a step c), for example via a loudspeaker, so that a user can listen to them. Regarding this reproduction, a system can receive or obtain feedback or evaluation in a step d). The system can be either a component of or independent of the audio system. The user can thus compare and evaluate the multiple reproduced filtered audio signals.In step e), a new value of the at least one audio parameter is set for each of the at least two sound characteristics using an adjustment function. The adjustment function depends on a current value of the audio parameter and on the feedback and / or a feedback history. The feedback history can, for example, describe the trend detected by the user when deciding between the filtered audio signals. A feedback history can therefore, for example, contain a history of user feedback.
[0017] The adjustment function can comprise a machine learning method, such as from the field of preference learning. In other words, when an audio parameter is adjusted, the adjustment function sets a new value for this parameter. An example of changing an audio parameter can be a bass level, where the bass level can be adjusted, for example, from -12 dB (decibels) to 12 dB. So if the current value of the bass level in one audio example A is 7 dB and in the other audio example B is 9 dB and the user selects audio example B, this user feedback via the adjustment function could be used to set a delta value of, for example, +1 dB to the current parameter value of the audio parameter "bass level" of audio example B, in this example 10 dB, in order to obtain a new audio example A.A function or a so-called "taste function" can therefore be built or constructed based on the feedback from the comparisons (e.g., user avoids settings according to A, user prefers settings according to B). Using the method, a complete and / or higher-dimensional parameter space can be explored, whereby the user's taste can be learned in order to optimize the taste function based on previously given feedback regarding the user's taste (in the sense of a predetermined optimization criterion, e.g., user approval of filtered audio signals of more than a threshold, e.g., more than 80% or 90%). Additionally or alternatively, it can be provided that in a subsequent iteration of the method, only new and / or previously unheard parameter settings are proposed.This allows a parameter value to be generated that is not predefined, but can be freely determined by the delta value and the user's response, without requiring a predefined set of sound characteristics. The method can thus provide one or more new parameter settings through exploration and / or optimization.
[0018] Instead of just a bass level, for example, the audio parameters that could be adjusted additionally or alternatively include the volume and / or a balance that defines the position of an audio signal in the stereo image and / or a phase shift and / or a surround sound and / or a compression and / or a chorus effect and / or a flanger effect and / or a phaser effect and / or a delay effect and / or treble and / or mid-range and / or bass.
[0019] This allows the sound characteristics of an audio signal to be specifically adapted to a user's preferences. Thus, a desired and / or individually adapted sound characteristic can be created and / or developed. In particular, there is no restriction to predefined value combinations of the audio parameters. Furthermore, the invention provides that in step f), a repetition of the aforementioned steps or a repetition of the previously disclosed method is carried out until the feedback meets a predetermined completeness criterion. For example, the user confirms that the parameter settings are to be saved and used in the future. In other words, one or more intended parameter settings of the audio signal are thereby achieved according to a user's individual preferences.The steps can be repeated until a predefined number of iterations, such as in a range of 10 to 20 repetitions, has been performed. Additionally or alternatively, the following further termination criteria can be specified: For example, the user can terminate the experiment or procedure when they no longer perceive any difference between the suggested audio parameter settings. Another termination criterion can be provided by the adjustment function working in the background by determining that a predefined optimum has been reached. This could be done by comparison with neighboring points in the latent space or other optimality criteria. Such a situation can arise when the latent space or parameter space has been sufficiently explored and the taste evaluation function is precise enough to determine that the optimal user taste has been reached.
[0020] The parameter settings are then saved and / or output in step g), and / or the audio system is operated with the final parameter settings in step h). The parameter settings can be saved and / or stored in memory banks of the audio system. In other words, a so-called feedback loop is executed until the desired parameter settings and / or a predefined number of repetitions are set or executed.
[0021] The invention includes further developments which result in additional advantages.
[0022] A further development provides that the adjustment function comprises: a value of at least one so-called latent variable, i.e. a variable that cannot be measured directly but must first be derived with the help of other variables, is changed, and the changed latent variable is converted or transformed into the new value of at least one audio parameter using a decoder network. Dimensionality reduction can therefore be carried out using one or at least one machine learning technique, such as principal component analysis and / or (variational) autoencoder. It is therefore possible to switch to a smaller subspace of the high-dimensional parameter space, which enables coverage of parameter combinations. This helps to accelerate the optimization process or optimization, as it is easier to carry out in lower dimensions.In the context of the autoencoder, "latent variables" are referred to as points in the aforementioned low-dimensional space, the latent space. In other words, a setting of at least one latent variable is made or changed. The changed latent variable is then converted or transformed into the new value of at least one audio parameter using a decoder network or a generative network. By changing the value of at least one latent variable, an individual sound characteristic can be achieved according to the user's preferences.
[0023] The at least one latent variable can also be provided as an abstract description of a plurality of predetermined sound characteristics by analyzing a statistical dependency and / or correlation and / or redundancy of the audio parameters of these predetermined sound characteristics in order to define a redundancy-reduced representation, namely the latent variables. According to an advantageous embodiment, it can be provided that, in order to generate the decoder network, an autoencoder is trained by applying error feedback or so-called backpropagation to training audio parameters. No labels are required for training the autoencoder, as it is assigned to so-called "unsupervised learning." Instead, a representative selection of meaningful parameter settings or audio system settings (i.e.Parameter values) are required, which can be generated using expert knowledge and / or data augmentation. Prior knowledge can be used additionally or alternatively, for example, correlating the gain of neighboring frequency bands to achieve a smooth frequency response. An encoder network can be created that receives the training audio parameters as input and generates a low-dimensional representation and / or a so-called latent space, i.e., a latent space with at least one latent variable.
[0024] The encoder or encoder network can consist of multiple layers of neurons connected by convolutional layers and / or fully connected layers of artificial neurons. The goal of the encoder is to reduce input data, i.e., the training audio parameters, to a compact representation. The input data contains, for example, information about one or more frequency responses and / or audio signals and / or sound characteristics. A decoder network can be created that receives points in latent space as input and / or is configured to reconstruct the original training audio parameters. The decoder network essentially mirrors the structure of the encoder, but in reverse. It can also consist of multiple layers of artificial neurons, which in turn are connected by convolutional layers and / or fully connected layers.As mentioned above, the autoencoder or autoencoder network can be trained by minimizing the error between the reconstructed training audio parameters and the actual training audio parameters. This can be achieved by applying a backpropagation technique, which backpropagates the error and adjusts the network's weights. The mean squared error can be used as the loss function. After the autoencoder is trained, the decoder can be isolated and used as a decoder network. The decoder network can receive points in the latent space as input and generate a reconstruction of the training audio parameters.
[0025] According to the invention, for example, a taste function f(p) can be established, where p are the audio parameters and the taste function returns a rating of these audio parameters (e.g., in the form of points). New parameter values can be suggested, balancing exploration and optimization. These new audio parameters p1 and p2 can be found, for example, by solving the following optimization problem: max_{p1, p2}λ*Exploration(f, p1, p2)+f(p1)+f(p2)
[0026] The taste function f can be maximized. The "exploration" function can, for example, result from the previously created taste model f(p), e.g., when using Bayesian optimization and / or Gaussian processes, from which it can be identified where the control function is uncertain, and / or from the audio parameters (e.g., by their distance from previously tested audio parameters). The parameter λ (lambda) > 0 can be used to weight whether more focus should be placed on exploration or optimization.
[0027] According to a further development, the control function can be designed as a genetic algorithm. A genetic algorithm is an optimization technique that solves optimization problems by combining partial solutions that ultimately lead to a global solution. In a first step, audio parameters can be defined and / or identified. An initial initialization of a so-called initial population can be carried out, whereby an initial population can include several sets of different audio parameters. A random initialization of these sets can be carried out, for example, using a random generation function or a program library from a programming language. The aforementioned genetic algorithm can therefore be used to solve the above optimization problem numerically. In principle, any method for global optimization can be suitable for this.Additionally or alternatively, the choice may depend on further details in the implementation (e.g., dimensionality and / or chosen taste model and / or chosen exploration function).
[0028] An unknown taste function can thus be optimized (in the sense of an optimization criterion) by determining which of at least two parameter sets is the best. The method according to the invention is not fixed in this regard. These can differ even in the choice of function class (e.g., radial basis functions and / or Gaussian processes) for the taste model. The method according to the invention can therefore arrive at a result with a few feedback points, e.g., with a number of 10 to 20. However, more feedback points can also be evaluated. In principle, however, an attempt is made to reduce the number of required comparisons as much as possible during the development of the method.
[0029] One embodiment provides that the parameter settings relating to audio parameters to be set comprise information that describes a temporal and / or spatial context when carrying out the method. In other words, a time period and / or a point in time and / or GNSS data of a current location can be recorded during the method (GNSS - Global Navigation Satellite System, e.g. GPS - Global Positioning System). This can mean a spatial distribution, wherein GPS data points represent various geographical positions and / or are distributed over an area and / or a geographical space. In other words, final parameter settings can comprise time data and / or GPS data. Operation of the audio system with the final parameter settings can be carried out, for example, depending on the time of day and / or location.For example, the user likes to listen to loud rock music while driving past a football stadium, so when such a situation occurs, i.e. driving past a football stadium, the appropriate parameter settings are retrieved and / or output. This means that different parameter settings can be operated depending on the situation. For example, continuous time recording and / or GPS data acquisition can be performed by implementing libraries such as Datetime and / or GeoPy as a function and using this function by regularly calling this function within a control structure, e.g., every 1 to 20 seconds. Parameter settings whose time data and / or GNSS data match the acquired data can then be operated and / or retrieved.
[0030] A further development provides that the at least one audio parameter of the sound filter is normalized and / or adapted to the (actual) audio signal according to a weighting function, such as the Fletcher-Munson curve. With regard to the at least one audio parameter of the sound filter, normalization according to the Fletcher-Munson curve means that the audio parameter is adjusted so that it takes the specific perception of the human ear into account as reliably as possible. The Fletcher-Munson curve shows that at low volumes, certain frequencies are heard less, while at higher volumes these frequencies are heard more amplified. As a result, frequency ranges that are less perceived by the ear at different volumes can be less amplified and / or emphasized, while the frequency ranges that are better perceived by the ear at higher volumes may be amplified.By applying the Fletcher-Munson curve, these differences in perception can be compensated for.
[0031] This approach can thus enable tonal adaptation that takes into account the perception of sound characteristics by the human ear. For example, the frequency spectrum of an audio signal can be logarithmically divided into bands or filter banks based on octaves and / or thirds, with an associated Fletcher-Munson correction factor determined for each band or frequency band according to the loudness. The Fletcher-Munson correction factor can be applied to the loudness or amplitude of the audio signal in the respective frequency band. Thus, a gain factor can be used for the entire audio signal to achieve normalization.The goal is for two different equalizer settings or parameter settings and / or frequency responses to appear equally loud to the user, so that the decision for a parameter setting is made (primarily) based on the frequency content and not on the level. Additionally or alternatively, it may be possible for this normalization to be performed depending on the actual audio content being played, in particular the frequencies present, rather than applying static normalization. This can be particularly advantageous for content with limited frequency ranges such as speech, as the normalization is actually performed based on the perceived loudness of the signal. This allows certain frequency ranges to be attenuated at low volumes and / or amplified at higher volumes. This can be repeated and / or performed for all frequency bands of the audio signal.Additionally or alternatively, the rating function can have a so-called A-, B-, CD-weighting and / or an equal loudness contour.
[0032] According to a further advantageous embodiment, it can be provided that the at least one audio parameter of the sound filter controls a dynamic volume adjustment. By dynamically adjusting the volume, unwanted volume peaks can be reduced and / or the overall volume of the audio signal can be controlled. This can prevent clipping and / or distortion. Furthermore, the listening experience and / or comfort can be improved, particularly when playing audio signals in different environments and / or on different playback devices.
[0033] This can be achieved, for example, using an AGC (Automatic Gain Control) system. An AGC system can regulate the audio signal in real time by adjusting the gain and / or attenuation to achieve a constant volume. The AGC system can use a feedback mechanism to adjust the volume accordingly. Additionally or alternatively, a sound leveling system can be used. This can enable consistent playback by compensating for volume fluctuations. This can improve intelligibility, as raising the volume of quieter sections and / or lowering the volume of louder sections leads to better understanding of spoken information. Excessive volume and / or frequent volume fluctuations can lead to listening fatigue, so dynamic volume adjustment provides a more pleasant listening experience and can reduce fatigue.Furthermore, dynamic volume adjustment can be adapted to different environments, so that, for example, the volume is increased in a noisy environment to emphasize the audio signal, while in a quiet environment it can be reduced to avoid excessive volume. Additionally or alternatively, normalization, as described above, can be performed to control the volume adjustment and / or dynamic compression. This can ensure that the filtered audio signals are perceived to be similarly loud, so that the user decision is made based on the feedback based on dynamics rather than level.
[0034] A further development provides that the provision in step a) includes input via a user interface. The audio signal can transmit various types of information, including speech and / or sound and / or music. By inputting the audio signal via a user interface, the user can use different data formats and / or interact flexibly with the audio system. This can be implemented, for example, as manual input via a screen device and / or as voice input using a speech recognition device.
[0035] According to a further advantageous embodiment, it can be provided that the provision in step a) includes feeding the audio signal into a machine learning model that is trained to automatically determine the music genre of the audio signal, and in step g) the final parameter settings are stored genre-specifically. The machine learning model used for this purpose can be designed, for example, as a convolutional neural network (CNN). CNNs can be suitable for recognizing patterns in data that are present in a spatial arrangement, such as in spectrogram-based representations of audio signals. For the automatic recognition of music genres, a CNN can therefore be used to extract features from the spectrogram-based representations of the audio signal and / or to link these features to the characteristic features of different music genres.The CNN model can then be trained to learn these features and / or make predictions about the genre of the audio signal. The machine learning model used can additionally or alternatively be implemented as a recurrent neural network (RNN) and / or a support vector machine (SVM).
[0036] In this way, the machine learning model enables automatic and / or genre-specific adaptation of the audio signal. This can facilitate the categorization and / or organization of audio signals, e.g., music, in digital libraries and / or the automatic labeling (tagging) of music pieces by genre. Furthermore, automatic genre recognition can avoid tedious manual processes. Furthermore, the machine learning model can ensure consistent adaptation of the audio signal to specific genre requirements, i.e., predetermined parameter settings according to a genre. Predefined parameter settings can be taken from publicly available music datasets, such as the GTZAN Genre Collection dataset, which is used in many research projects and contains predefined parameter settings.It can therefore implement parameter settings that have proven suitable for the identified genre based on a sound quality when assigning an audio signal to that genre. This can result in a uniform sound quality and / or coherent sound characteristics within a specific genre.
[0037] A further development provides that in step d), feedback is received by means of a slider, wherein a guide rail represents a predetermined rating level, wherein each end of the guide rail represents one of the two filtered audio signals, wherein a value is quantified according to the rating level by placing a slider on the guide rail. The rating level can be designed as a numerical scale on which a predetermined value range is defined. For example, a scale from 1 to 10 can be used, where 1 represents the lowest rating and 10 the highest rating. This scale can be divided into smaller intervals, for example in 0.5 increments, to allow for more fine gradations.
[0038] A slider can enable simple and / or intuitive interaction by moving the slider along a linear guide or axis so that the desired option, i.e., feedback related to two filtered audio signals, can be selected. Instead of making a single discrete decision, the slider can be positioned at any point along the axis to achieve a gradual transition between the options consisting of the two filtered audio signals. When moving the slider, it is therefore possible to interpolate between two filtered audio signals and / or between two parameter sets (parameter settings), with the user being able to assess the resulting result by giving or entering feedback. This allows for fine tuning and / or adaptation to individual preferences and / or needs. The slider can be designed in analog and / or digital form. For example,When the slider is positioned fully to the left, feedback is provided for a first filtered audio signal, while when the slider is positioned fully to the right, feedback is provided for a second filtered audio signal. Positioning the slider in the center of the slider allows feedback based on an even mix of the two filtered audio signals and / or an average or mean value of the system parameters or parameter settings.
[0039] A further development provides that in step d), n audio signals are received and / or feedback is provided via a polygonal input field, wherein each corner represents a filtered audio signal, and by placing the slider within the polygonal input field, a value is quantified using barycentric coordinates according to a rating level. By means of the polygonal input field, which can be configured, for example, analog and / or digital, a slider can be positioned within the input field, whereby the ratio and / or weighting of parameter settings of n audio signals can be adjusted. The polygonal input field can therefore be implemented, for example, as an equilateral triangle, wherein each corner of the triangle represents a filtered audio signal. For the number n, n > 2 applies in particular.For example, a first filtered audio signal can be represented at the top corner, a second filtered audio signal at the bottom left corner, and a third filtered audio signal at the bottom right corner. The polygonal input field can therefore be divided into one or more triangles or other methods and / or shapes.
[0040] Feedback can be provided by moving a slider on the triangular input field. When the user moves the slider, barycentric coordinates can be calculated to determine the current value within the input field. The coordinates (u, v, w) can be used to describe the area division of the polygonal input field or triangle, thus quantifying feedback. Based on the barycentric coordinates, barycentric interpolation can be performed, allowing a value to be calculated according to a predefined evaluation level.
[0041] For use cases or application situations that may arise during the method and which are not explicitly described here, it may be provided that, in accordance with the method, an error message and / or a request to enter user feedback is issued and / or a default setting and / or a predetermined initial state is set.
[0042] The invention also includes further developments of the method according to the invention that have features already described in connection with the further developments of the audio system according to the invention. For this reason, the corresponding further developments of the method according to the invention are not described again here.
[0043] A device can comprise the audio system. The device can be configured as a motor vehicle and / or a smartphone and / or a hi-fi system (preferably with internet and / or cloud connectivity for managing and / or processing user profiles and / or for optimization or hyperparameter optimization).
[0044] The motor vehicle may preferably be designed as a motor vehicle, in particular as a passenger car or lorry, or as a passenger bus or motorcycle.
[0045] As a further solution, the invention also encompasses a computer-readable storage medium comprising program code which, when executed by a computer or computer network, causes the computer to carry out an embodiment of the method according to the invention. The storage medium can be provided at least partially as a non-volatile data memory (e.g. as a flash memory and / or as an SSD - solid state drive) and / or at least partially as a volatile data memory (e.g. as a RAM - random access memory). The storage medium can be arranged in the computer or computer network. However, the storage medium can also be operated on the Internet, for example, as a so-called app store server and / or cloud server. The computer or computer network can provide a processor circuit with, for example, at least one microprocessor.The program code may be provided as binary code and / or as assembly code and / or as source code of a programming language (e.g. C) and / or as a program script (e.g. Python).
[0046] As a further solution, the invention includes an audio system having a processor device for carrying out the described method. The processor device can have at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (Field Programmable Gate Array) and / or at least one DSP (Digital Signal Processor). In particular, a CPU (Central Processing Unit), a GPU (Graphical Processing Unit) or an NPU (Neural Processing Unit) can be used as the microprocessor. Furthermore, the processor device can have program instructions which are configured to carry out the embodiment of the method according to the invention when executed by the processor device. The program instructions can be stored in a data memory of the processor device. The processor device can, for example,based on at least one circuit board and / or on at least one SoC (System on Chip).
[0047] The audio system can store a profile of a user that is at least partially based on user data and / or represents the user's musical taste. Furthermore, the audio system can store and / or output parameter settings. If necessary, the user profile can be updated based on newly acquired user data. In addition, the audio system can comprise an image recognition and / or video camera device configured to record a user and / or capture an image and / or a video of the user; a database configured to store previously created user profiles, including user data representing the user's musical taste; a biometric recognition device configured to compare the captured image and / or video with the profiles stored in the database to find a match;an assignment device configured to assign the profile of the identified user to the captured image and / or video in the event of a match, and an updating device configured to update the profile of the identified user, if necessary, based on new captured user data, wherein the user data represents a user's musical taste;
[0048] In addition, the audio system can have a backend system that can collect and / or store preferences and / or genres and / or optimized audio parameters or parameters (p and / or q) and / or optimization histories of multiple users. Multiple audio systems can communicate with the same backend system to identify relationships between users. The backend system can store and / or optimize an algorithm and / or hyperparameters of a dimensionality reduction and / or optimization (in the sense of a predefined optimization criterion) and / or perform initializations depending on the user and / or genre. If users use different audio systems, communication with the backend system can ensure that personal settings and / or user profiles and / or preferences are synchronized between the different audio systems.Users can maintain their individual audio settings on different audio systems without having to adjust them manually.
[0049] The invention also encompasses combinations of the features of the described embodiments. The invention therefore also encompasses implementations that each comprise a combination of the features of several of the described embodiments, unless the embodiments are described as mutually exclusive.
[0050] Exemplary embodiments of the invention are described below. Shown are: Fig. 1 a schematic representation of an embodiment of the method according to the invention; Fig. 2 a sketch to illustrate an implementation of the method according to the invention; Fig. 3 a flowchart of an embodiment of the method according to the invention and Fig. 4 is a sketch illustrating an implementation for performing feedback according to the disclosure.
[0051] The exemplary embodiments explained below are preferred embodiments of the invention. In the exemplary embodiments, the described components of the embodiments each represent individual features of the invention that can be considered independently of one another, each of which also develops the invention independently of one another. Therefore, the disclosure is intended to encompass combinations of the features of the embodiments other than those shown. Furthermore, the described embodiments can also be supplemented by further features of the invention already described.
[0052] In the figures, the same reference symbols designate elements with the same function.
[0053] In Fig. 1 represents the interaction of an encoder E and a decoder D, where both components are part of an autoencoder. The encoder E is the first part of the autoencoder. It can receive an input with multiple sound characteristics (e.g. multiple equalizer curves), in this case audio parameters A of the respective sound characteristic and / or their parameter values for different frequencies, e.g. comprising bass Ba and / or mid-range Mi and / or treble H, and transforms them into a representative representation and / or into one or more latent variables L. If, for example, the mid-range Mi of the input, i.e. of the multiple sound characteristics, remain unchanged, i.e. all have the same or similar mid-range curves and only differ in bass and treble, which is why these mid-ranges are not part of the latent variable L in the latent space or are not taken into account.The purpose of the encoder E is to map the inputs into a low-dimensional space that captures the underlying structure or pattern of the data. The latent variables L generated by the encoder E contain important information about the input and thus represent a compressed representation of the data. Redundancy is reduced compared to the inputs. The decoder D is the second part of the autoencoder. It takes the latent variables L from the encoder E and reconstructs the original input from them. The decoder D attempts to reconstruct the original data as accurately as possible based on the information in the latent variables L. The purpose of the decoder D is therefore to transform the latent variables L back into the original high-dimensional space and generate a reconstruction of the input.According to the present idea, a value of at least one latent variable L can be changed and the changed latent variable L can be converted into the new value of at least one audio parameter A by means of a decoder network D.
[0054] In Fig. 2 and Fig. Figure 3 shows one of the possible implementations of the method. As a starting point, for example, in a step S1, equalizer settings (P, C, S, R, B), each representing an audio genre such as pop, classic, speech, rock, and / or balanced, can serve or be used, which, from the perspective of experts, result in suitable frequency responses H(f) for different audio genres or music genres. Additional features, such as specific delay and amplitude distributions across different loudspeakers, which can be crucial for 3D sound, and / or threshold, ratio (e.g., ratio of input to output level), attack (e.g., response time after the threshold has been exceeded), and / or release (e.g., fade-out time) of a dynamic compressor, can be used. The audio system can generate optimized audio parameter values A or parameter values A for a user in the background, and if applicable.find a specific situation and / or a specific music genre by iteratively suggesting at least two parameter settings or settings in the form of at least two filtered audio signals to the user, as symbolically represented in S5, from which the user selects the best version for them by listening comparison in S6, either by directly selecting the suggested setting or by selecting a combination of settings created by interpolation. Based on the feedback or user feedback, the system can suggest new settings, each with different audio parameter values A, i.e. via steps S2 and / or S3 and / or S4, with the aim of getting to know and / or optimizing the user's taste, i.e. the optimal parameters for them and a recorded situation. The specific parameter values A are not relevant to the user.
[0055] If the parameters A used have a high dimension, it may be useful to perform a dimension reduction as described in S2 and / or S4 in Fig. 2 symbolically depicted (which is marked in the image below by an optional transformation). Thus, a frequency response can be characterized by several hundred to a thousand amplitude-phase pairs. In this case, the subsequent optimization, i.e., changing a value of at least one latent variable L, according to step S3, can be performed in the space of the latent variable L, which has a much smaller dimension, which can significantly reduce the optimization effort and / or optimization time (in the sense of an optimization criterion). As the transformation, i.e., the dimensionality reduction, well-known methods such as principal component analysis (PCA) and / or machine learning (ML)-based approaches such as autoencoders can be used. Furthermore, generative methods such as variational autoencoders [cf. DP Kingma and M. Welling. “An Introduction to Variational Autoencoders.” In: arXiv: 1906.02691v3. URL: https: / / arxiv.org / pdf / 1906.02691.pdf] offer an advantage because they are capable of generating meaningful, realistic inputs that are not present in the data sets used for training. This allows the system to potentially explore personal preferences.
[0056] The data for dimensionality reduction can be obtained through expert knowledge (e.g., meaningful correlations of parameters) and / or through swarm intelligence, where settings made by other users are collected, analyzed, and / or obtained. Furthermore, when selecting the parameter space or latent space, care can be taken to ensure that each setting or selection of a parameter set results in the same loudness of the processed audio signal. This can prevent the optimization of the taste tuning from simply increasing the volume, for example, which can overshadow nuances such as changes in frequency content in human perception. To achieve this, the audio signal processing system can measure the loudness of the output signal or audio signal and, if necessary, correct it by amplification and / or attenuation.
[0057] At the beginning of the optimization process, two different and / or contrasting parameter sets or parameter settings, e.g. two equalizer settings and / or corresponding frequency responses, can be selected. As mentioned above, existing prior knowledge can be used to accelerate the optimization. For example, based on the streamed music, it is possible to find out which music genres a person or user prefers. This information could be used accordingly, for example, to select a search space and / or initialization accordingly and / or thus reduce the number of iterations. These parameters A can be taken directly from the original parameter space, such as delays and / or volumes for different loudspeakers, and / or from the latent variable space, i.e. from latent variables L.If the latter is the case, the selected parameters A can first be converted into the original space (i.e. the non-latent space) using an inverse transformation before they are then made available to a person or user as an audio sample as two (or more) candidates, i.e. as at least two filtered audio signals.
[0058] A concrete example could be that a sample is randomly selected from all possible sensible frequency responses, such as one for pop and / or one for rock music (Pop P or Rock R), with the aim of making a decision between these frequency responses to enable the user to explore and / or optimize their taste. These frequency responses can then be applied to the music tracks selected by the person in question and / or made available as an audio sample. While listening, the person can switch back and forth between the settings to decide which sample sounds better to them. In this case, one could speak of an A / B comparison, for example. The person could use slider 2 to create an overlay between the two settings.If, for example, in this case the control (slider) were located exactly in the middle between the two candidates, the sound that would be heard would be 0.5*setting_1 + 0.5*setting_2, where by changing the latent variable L, the actual audio parameters A or system parameters A and / or the processed audio signals can be combined with each other.
[0059] The decision for the sound or the sound characteristic A or B in the case of an A / B comparison and / or for a mixture of both in the case of a slider 2 can be transmitted back to the optimization algorithm as feedback. The core of the algorithm can be based on the methods of preference learning [cf. J. Fürnkranz and E. Hüllermeier. “Preference Learning and Ranking by Pairwise Comparison”. In: Preference Learning, pp. 65-82, Springer]. This can be done iteratively, taking the feedback into account, using either deterministic [cf. A. Bemporad and D. Piga. “Global optimization based on active preference learning with radial basis functions”. In: Machine Learning 110 (2021), pp. 417-448. URL: https: / / doi.org / 10.1007 / s10994-020-05935-y] and / or probabilistic [cf. W. Chu and Z. Ghahramani. “Preference Leaning with Gaussian Processes”.In: Proceedings of the 22nd International Conference on Machine Learning, Bonn, Germany, 2005], a surrogate function can be created for the preference of the system, which in our case is represented by a person. Thus, after the initial feedback, the algorithm can calculate two new candidates, which are then also provided as a new audio sample.
[0060] After one or more iterations, the algorithm can have collected enough information to reconstruct a surrogate function or setting function for the personal preference. One of the well-known standard optimization methods can then be used to determine the maximum of this function. Additionally or alternatively, a mixture of preference and exploration can be maximized to generate new suggestions. By optimizing the taste function, at least one filtered audio signal can be determined. Alternatively and preferably, this surrogate function can be updated in each iteration and used to suggest the next sets of parameters to be evaluated by the user. The termination of the optimization process, i.e., the repetition of steps b) to e) of the process, by the person themselves can serve as the termination criterion for the feedback iterations.Another possibility is to automatically determine the convergence of the optimization, e.g., when a specified number of repetitions have been performed, and to suggest to the person that the optimization process be terminated. Additionally or alternatively, as already mentioned, these and / or other termination criteria can be specified: For example, the user can terminate the experiment or procedure if they no longer perceive any difference between the suggested audio parameter settings. Another termination criterion can be provided by the adjustment function operating in the background by determining that a predefined optimum has been reached. This could be done by comparing with neighboring points in the latent space or other optimality criteria.Such a situation can occur when the latent parameter space has been sufficiently explored and the taste evaluation function is precise enough to determine that the optimal user taste has been achieved. For example, by maximizing the taste function, a current candidate, i.e., a currently preferred audio signal, can be determined.
[0061] The optimization can be sufficiently converged, for example, if the algorithm proposes candidates in several consecutive iterations that do not differ significantly from each other, for example, if the sound structure of at least two audio signals lies within a specified interval. The option may exist (see Fig. ), data or user data about multiple users (e.g., musical tastes, genres, and more when the system is embedded in another system, e.g., a smartphone or vehicle), as well as their decisions made during optimization and / or final preferences, to be collected and / or stored centrally in a backend system. This can be used to adapt the algorithm and / or hyperparameters of the dimensionality reduction and optimization, with the goal of the process described above finding suitable settings for the respective user with fewer iterations.
[0062] The dimensionality reduction can thus be designed in such a way that the settings of most users are represented and only "meaningful" settings can be generated. This can mean that not all combinations of parameter values result in good-sounding settings. In particular, individual parameters A can be correlated with each other within the optimized user settings, which can be exploited by the dimensionality reduction. In the optimization algorithm, i.e. in the setting function, hyperparameters and / or functions can be optimized for the formation of the replacement model, i.e. for the formation of final parameter settings, as well as the selection of the parameter sets to be evaluated. Furthermore, it is possible that the optimization process, in addition to user feedback, also uses further information such as genre and / or situation (e.g.time of day), and / or takes the user profile into account and / or initialises the settings appropriately based on the settings of similar users, genres and situations and moves close to them in the optimisation process.
[0063] In a special embodiment, according to Fig. 3 in step S10, an audio signal and / or, for example, frequency responses are selected and / or entered via an audio system in step S20. An audio signal processing unit, e.g., an equalizer, can contain audio parameters p1 to pn. In step S30, the at least two filtered audio signals can be reproduced, e.g., via an amplifier and / or loudspeaker. The at least one audio signal can therefore be perceived by a user via sound in a step S40, wherein, optionally, a user profile of the user, which can contain, for example, usage data and / or preferences with regard to their musical taste, can be taken into account in a step S60. Via an interaction system, which can be part of the audio system, the user can switch between suggested parameter settings, i.e., in the form of at least two audio signals to be reproduced, and / or write or enter feedback in a step S50.In step S70, as already mentioned above, a so-called optimization can be performed using a control function, whereby new parameters can be suggested. Additionally or alternatively, a dimensionality reduction can be performed, whereby conversions can be made between parameters p1 to pn and latent variables q1 to qm, where m is less than or equal to n. Additionally or alternatively, it can be anticipated that an adaptation to the user profile and / or a genre and / or a situation will be performed.
[0064] In a backend system, in a step S100, preferences and / or genres of optimized parameters p and q and / or optimization histories of one or more users and / or other user characteristics can be stored. An algorithm and / or hyperparameters of dimensionality reduction and / or optimization can be stored. Initializations can be performed depending on the user and / or genre. Through the optimization process, new parameters A p can be created at S20, allowing a renewed evaluation—i.e., providing feedback with respect to at least two filtered audio signals—to be performed via S30 to S70, with the parameter suggestions being displayed via the interaction system. Additionally or alternatively, a genre and / or situation can be recognized via S10, continuing at S110. From there, optimization can be performed in S70.
[0065] In Fig.Figure 4 symbolically depicts a slider 2 and a polygonal input field 3, with the polygonal input field 3 being implemented as a triangular input field. The slider 2 and / or the polygonal input field 3, with which the user can provide feedback, can each be configured in analog and / or digital form.
[0066] The present invention further relates to embodiments according to the following aspects: Aspect 1 Method for adjusting a sound characteristic of an audio system, comprising the following steps performed by the audio system: - a) Providing an audio signal; - b) applying a sound filter to the audio signal, whereby at least two further, at least partially different filtered audio signals are generated, in that the sound filter each has different parameter settings, which result in a different sound characteristic which is determined by at least one audio parameter A; - c) reproducing at least two filtered audio signals; - d) receiving feedback regarding the playback; - e) for the at least two sound characteristics: setting a new value of the at least one audio parameter A by means of an adjusting function which is dependent on a current value of the audio parameter A and on the feedback and / or a feedback history; - f) repeating steps b) to e) until the feedback meets a predefined completeness criterion and / or until a predefined number of iterations have been performed; - g) saving and / or outputting the final parameter settings; - h) Operating the audio system with the final parameter settings; Aspect 2. Method according to aspect 1, wherein in step e) the adjustment function comprises: a value of at least one latent variable L is changed and the changed latent variable L is converted into the new value of the at least one audio parameter A by means of a decoder network D. Aspect 3. Method according to one of the preceding aspects, wherein, to generate the decoder network D, training of an autoencoder is carried out by applying error feedback to training audio parameters, wherein the training audio parameters are provided with an identification of different frequency responses of different sound characteristics. Aspect 4. Method according to one of the preceding aspects, wherein the control function is designed as a genetic algorithm. Aspect 5. Method according to one of the preceding aspects, wherein the parameter settings comprise audio parameters A to be set concerning information describing a temporal and / or spatial context when carrying out the method. Aspect 6. Method according to one of the preceding aspects, wherein the at least one audio parameter A of the sound filter is normalized according to a weighting function and / or adapted to the at least one audio signal. Aspect 7. Method according to one of the preceding aspects, wherein the at least one audio parameter A of the sound filter controls a dynamic volume adjustment. Aspect 8. The method according to any one of the preceding aspects, wherein the providing in step a) includes input via a user interface. Aspect 9. Method according to one of the preceding aspects, wherein the providing in step a) includes feeding the audio signal into a machine learning model that is trained to automatically determine the music genre of the audio signal, and in step g) the final parameter settings are stored genre-specifically. Aspect 10. Method according to one of the preceding aspects, wherein in step d) the receiving of feedback takes place by means of a slider 2, wherein a guide rail represents a predetermined rating level, wherein each end of the guide rail represents one of the two filtered audio signals, wherein by placing a slider on the guide rail, a value is quantified according to the rating level. Aspect 11. Method according to one of the preceding aspects, wherein in step d) n audio signals are received and / or feedback is provided via a polygonal input field 3, wherein each corner represents a filtered audio signal, and by placing the slider within the polygonal input field 3, a value is quantified using barycentric coordinates according to a rating scale. Aspect 12 Audio system designed to: - to create and / or store a profile of a user based at least in part on user data that represents the user's musical taste, - to save and / or output parameter settings, - to update the user's profile, if necessary, based on new acquired user data, wherein the audio system comprises a processor device having program instructions which, when executed by the processor device, cause it to carry out a method according to any one of the preceding aspects. Aspect 13. A device comprising an audio system according to aspect 12.
[0067] Overall, the examples demonstrate how interactive, automated adjustment of an audio system's settings to user preferences can be provided. List of reference symbols A Audio parameters Ba Basses M Mitten H Heights D decoder E Encoder L latent variable P Pop C Classic S Speech R Rock B Balanced 2 sliders 3 polygonal input field
Claims
[1] A method for adjusting a sound characteristic of an audio system, comprising the following steps performed by the audio system: - a) Providing an audio signal; - b) applying a sound filter to the audio signal, whereby at least two further, at least partially different filtered audio signals are generated, in that the sound filter each has different parameter settings, which result in a different sound characteristic which is determined by at least one audio parameter (A); - c) reproducing at least two filtered audio signals; - d) receiving feedback regarding the playback; - e) for the at least two sound characteristics: setting a new value of the at least one audio parameter (A) by means of an adjusting function which is dependent on a current value of the audio parameter (A) and on the feedback and / or a feedback history; - f) repeating steps b) to e) until the feedback meets a predefined completeness criterion and / or until a predefined number of iterations have been performed; - g) saving and / or outputting the final parameter settings; - h) where • the at least one audio parameter (A) of the sound filter controls a dynamic volume adjustment, whereby unwanted volume peaks are reduced and / or the overall volume of the audio signal is controlled, and the audio system is operated with the final parameter settings, and / or • the parameter settings relating to the audio parameters (A) to be set include information describing a temporal context when carrying out the method, wherein the final parameter settings comprise time data, and the audio system is operated with the final parameter settings depending on the time of day, so that different parameter settings are operated depending on the situation, and / or • the parameter settings relating to audio parameters (A) to be set include information describing a spatial context when carrying out the method, wherein the final parameter settings comprise GPS data, and operation of the audio system is carried out with the final parameter settings depending on the location, so that different parameter settings are operated depending on the situation. [2] Method according to claim 1, wherein in step e) the adjustment function comprises: a value of at least one latent variable (L) is changed and the changed latent variable (L) is converted into the new value of the at least one audio parameter (A) by means of a decoder network (D). [3] Method according to one of the preceding claims, wherein, to generate the decoder network (D), training of an autoencoder is carried out by applying error feedback to training audio parameters, wherein the training audio parameters are provided with an identification of different frequency responses of different sound characteristics. [4] Method according to one of the preceding claims, wherein the setting function is designed as a genetic algorithm. [5] Method according to one of the preceding claims, wherein the at least one audio parameter (A) of the sound filter is normalized according to a weighting function and / or adapted to the at least one audio signal. [6] Method according to one of the preceding claims, wherein the providing in step a) includes an input via a user interface. [7] Method according to one of the preceding claims, wherein the providing in step a) includes feeding the audio signal into a machine learning model which is trained to automatically determine the music genre of the audio signal and in step g) the final parameter settings are stored genre-specifically. [8] Method according to one of the preceding claims, wherein in step d) the receiving of a feedback is carried out by means of a slider (2), wherein a guide rail represents a predetermined evaluation level, wherein each end of the guide rail represents one of the two filtered audio signals, wherein by placing a slider on the guide rail a value is quantified according to the evaluation level. [9] Method according to one of the preceding claims, wherein in step d) n audio signals are received and / or feedback is carried out via a polygonal input field (3), wherein each corner represents a filtered audio signal and by placing the slider within the polygonal input field (3) a value is quantified by means of barycentric coordinates according to a degree of evaluation. [10] Audio system designed to: - to create and / or store a profile of a user based at least in part on user data that represents the user's musical taste, - to save and / or output parameter settings, - to update the user's profile, if necessary, based on new acquired user data, wherein the audio system comprises a processor device having program instructions which, when executed by the processor device, cause it to perform a method according to any one of the preceding claims. [11] Apparatus comprising an audio system according to claim 10.
Citation Information
Patent Citations
Adjusting settings for media playback in a means of transport
DE102020202372A1
Integrated multimedia signal processing system using centralized processing of signals
EP1677574A2
Method and system for assisting a vehicle occupant in tailoring vehicle settings
EP3048026A1
Mixed-reality audio intelligibility control
US20200211578A1
Methods and apparatus for audio equalization based on variant selection
US20220329941A1
