Audio control method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of audio control, and provides an audio control method and device, electronic equipment and a storage medium. The audio control method comprises the steps that a sound effect test is carried out on a target space based on a specified measurement signal to obtain environment response information, and the environment response information comprises pulse response information and information obtained after pulse response is separated; analyzing the environmental response information to obtain acoustic characteristic information of the target space; performing compensation processing on the to-be-processed audio based on the acoustic characteristic information to obtain a regulated audio; regulating and controlling the gains of the multiple sound channels corresponding to the regulated and controlled audio to obtain audio regulated and controlled again; and carrying out multi-channel collaborative optimization processing on the audio after being regulated and controlled again to obtain the audio with a target effect. According to the method, accurate compensation of the acoustic defect of the target space is realized, the problem of crosstalk between sound channels is effectively solved, the positioning and space sense of the sound image is remarkably improved, the overall hearing sense of the audio is greatly improved, and the audio output is continuously optimized.

Description

Technical Field

[0001] The present disclosure relates to the technical field of audio control, and in particular to an audio control method, device, electronic device and storage medium. Background Art

[0002] Audio system, also known as sound system or audio system, is a collection of equipment and technologies for generating, processing, transmitting and playing sound. Traditional audio systems often use preset audio parameters. Since they cannot effectively adapt to the characteristics of different playback environments, the audio quality performance in actual use is significantly different from the ideal state, which in turn leads to obvious peaks and valleys in the low-frequency response, thus affecting the overall listening experience of the audio. Summary of the invention

[0003] The present disclosure provides an audio control method, device, electronic device and storage medium, which realizes accurate compensation for the acoustic defects of the target space by performing compensation processing, gain control processing and multi-channel collaborative optimization processing based on the acoustic characteristic information obtained by analyzing the environmental response information of the sound effect test based on the specified measurement signal, effectively solves the crosstalk problem between channels, significantly improves the positioning and spatial sense of sound and image, thereby greatly improving the overall listening experience of the audio and continuously optimizing the audio output.

[0004] A first aspect of the present disclosure provides an audio control method, comprising:

[0005] Performing a sound effect test on the target space based on the specified measurement signal to obtain environmental response information, wherein the environmental response information includes impulse response information and information after the impulse response is separated;

[0006] Analyzing the environmental response information to obtain acoustic characteristic information of the target space;

[0007] Performing compensation processing on the audio to be processed based on the acoustic characteristic information to obtain regulated audio;

[0008] Adjusting the gains of multiple channels corresponding to the adjusted audio to obtain further adjusted audio;

[0009] The re-adjusted audio is subjected to multi-channel collaborative optimization processing to obtain audio with a target effect.

[0010] A second aspect of the present disclosure provides an audio control device, comprising:

[0011] An acquisition module, configured to perform a sound effect test on a target space based on a specified measurement signal to obtain environmental response information, wherein the environmental response information includes impulse response information and information after the impulse response is separated;

[0012] An analysis module, used to analyze the environmental response information to obtain acoustic characteristic information of the target space;

[0013] A compensation module, used for performing compensation processing on the audio to be processed based on the acoustic characteristic information to obtain regulated audio;

[0014] A control module, used to control the gain of multiple channels corresponding to the regulated audio to obtain the further regulated audio;

[0015] The optimization module is used to perform multi-channel collaborative optimization processing on the audio after the re-adjustment to obtain audio with a target effect.

[0016] A third aspect of the present disclosure provides an electronic device, which includes a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the above audio control method.

[0017] A fourth aspect of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the above-mentioned audio control method.

[0018] In the technical solution provided by the present disclosure, a sound effect test is performed on the target space based on a specified measurement signal to obtain environmental response information, wherein the environmental response information includes impulse response information and information after the impulse response is separated; the environmental response information is analyzed to obtain acoustic characteristic information of the target space; compensation processing is performed on the audio to be processed based on the acoustic characteristic information to obtain regulated audio; the gain of the multi-channels corresponding to the regulated audio is regulated to obtain the audio regulated again; multi-channel collaborative optimization processing is performed on the audio regulated again to obtain audio with the target effect. In the embodiment of the present disclosure, by performing compensation processing, gain regulation processing and multi-channel collaborative optimization processing based on the acoustic characteristic information obtained by analyzing the environmental response information of the sound effect test based on the specified measurement signal, accurate compensation for the acoustic defects of the target space is achieved, the problem of crosstalk between channels is effectively solved, the positioning and spatial sense of the sound and image are significantly improved, and thus the overall listening experience of the audio is greatly improved, and the audio output is continuously optimized. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of an embodiment of an audio control method in an embodiment of the present disclosure;

[0020] Figure 2 It is a schematic diagram of another embodiment of the audio control method in the embodiment of the present disclosure;

[0021] Figure 3 is a schematic diagram of an embodiment of an audio control device in an embodiment of the present disclosure;

[0022] Figure 4 Schematic diagram of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] This embodiment provides an audio control method, device, electronic device and storage medium, which greatly improve the overall listening experience of the audio and continuously optimize the audio output.

[0024] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] This embodiment provides an audio control method, device, electronic device and storage medium, which are mainly used in audio systems.

[0026] It is understandable that the execution subject of the present disclosure may be an audio control device, or a terminal, a server, or an audio system, which is not limited here. This embodiment is described by taking a server as the execution subject as an example.

[0027] For ease of understanding, the specific process of this embodiment is described below. Figure 1 , an embodiment of the audio control method in this embodiment includes:

[0028] 101. Performing a sound effect test on a target space based on a specified measurement signal to obtain environmental response information, wherein the environmental response information includes impulse response information and information after the impulse response is separated;

[0029] Among them, the specified measurement signal can be understood as a preset audio signal for evaluating the performance of the audio system or the acoustic characteristics of the environment, and the specified measurement signal is a measurement signal having, but not limited to, controllability, stability and analyzability. The impulse response information can be understood as the information after the target response information is extracted, and the target response information is the response information fed back by the target space after the sound effect test based on the specified measurement signal and recorded by the specified recording device. The information after the impulse response is separated can be understood as the information after the impulse response information is processed by a preset window separation function (or a preset window separation algorithm). As an example and not a limitation, the target space can be the room where the audio to be processed is played.

[0030] As an example but not limitation, a designated measurement signal is determined; the designated measurement signal is transmitted to the target space by a preset transmitting device; response information based on feedback of the designated measurement signal at a specific position in the target space is collected by a preset recording device, and the response information is extracted to obtain pulse response information; the pulse response information is separated and processed to obtain separated information; and the pulse response information and the separated information are determined as environmental response information.

[0031] 102. Analyze the environmental response information to obtain acoustic characteristic information of the target space;

[0032] As an example but not limitation, at least one acoustic characteristic factor to be analyzed is determined, wherein at least one acoustic characteristic factor is a key factor of spatial acoustic performance; a preset algorithm library is searched based on at least one acoustic characteristic factor by a preset retrieval algorithm to obtain a target algorithm, wherein the retrieval algorithm is used to retrieve or find an algorithm corresponding to the optimal path in the preset algorithm library, for example, the retrieval algorithm may be a Dijkstra algorithm, an A* algorithm, a depth-first search DFS, or a breadth-first search BFS, and the preset algorithm library stores a variety of algorithms (models) for analyzing acoustic characteristic factors, and the target algorithm includes algorithms corresponding to each acoustic characteristic factor; the environmental response information is processed by the target algorithm to obtain the acoustic characteristic information of the target space. By matching the target algorithm based on at least one acoustic characteristic factor, the efficiency and accuracy of the environmental response information analysis are improved, and the accuracy and reliability of the acoustic characteristic information of the target space are improved.

[0033] 103. Perform compensation processing on the audio to be processed based on the acoustic characteristic information to obtain the regulated audio;

[0034] The audio to be processed is audio information corresponding to a target usage scenario, such as music appreciation, movie watching, and gaming.

[0035] As an example but not limitation, a target compensation algorithm corresponding to the acoustic characteristic information is matched, and compensation processing is performed on the processed audio based on the target compensation algorithm to obtain the regulated audio, wherein the target compensation algorithm includes but is not limited to an adaptive equalization algorithm, an environmental mode suppression algorithm (also referred to as an intelligent environmental adaptive processing algorithm, which is used to analyze the environmental information of the target space in real time and dynamically adjust audio parameters), an early reflection optimization algorithm, an automatic gain control algorithm, a wide dynamic range compression algorithm, an equalizer, a noise suppression algorithm, and an echo cancellation algorithm.

[0036] 104. Regulating the gains of the multiple channels corresponding to the regulated audio to obtain the regulated audio;

[0037] As an example but not limitation, a gain adjustment strategy for each channel is determined, wherein specifically, the content of the audio to be processed is obtained, and the gain adjustment strategy for each channel corresponding to the regulated audio is determined based on the content and acoustic characteristic information of the audio to be processed, wherein the content of the gain adjustment strategy includes but is not limited to loudness matching, dynamic range control, spatial balance, spectral balance and scene optimization; based on the gain adjustment strategy of each channel, the gains of multiple channels corresponding to the regulated audio are regulated to obtain the re-regulated audio.

[0038] 105. Perform multi-channel collaborative optimization processing on the audio after the second adjustment to obtain audio with a target effect.

[0039] As an example but not limitation, it is possible to: determine the multi-channel layout of the re-adjusted audio; based on the multi-channel layout, perform channel analysis, adaptive filtering and multi-channel complementary equalization processing on the re-adjusted audio to obtain audio with target effect.

[0040] As an example but not a limitation, after performing channel analysis, adaptive filtering and multi-channel complementary equalization processing on the re-adjusted audio, the processed audio can also be: sound enhancement is performed on the processed audio to obtain enhanced audio; testing is performed based on the enhanced audio, and evaluation is performed based on the test results, and when the evaluation results meet the preset conditions, the enhanced audio is determined as the audio of the target effect. Through testing and evaluation, the reliability of the audio of the target effect can be improved, thereby providing a high-quality audio experience.

[0041] In this embodiment, compensation processing, gain control processing and multi-channel collaborative optimization processing are performed based on the acoustic characteristic information obtained by analyzing the environmental response information of the sound effect test based on the specified measurement signal, so as to achieve accurate compensation for the acoustic defects of the target space, effectively solve the problem of crosstalk between channels, significantly improve the positioning and spatial sense of sound and image, thereby greatly improving the overall listening experience of the audio and continuously optimizing the audio output.

[0042] See also Figure 2 , another embodiment of the audio control method in this embodiment includes:

[0043] 201. Performing a sound effect test on a target space based on a designated measurement signal to obtain environmental response information, wherein the environmental response information includes impulse response information and information after the impulse response is separated;

[0044] In one implementation, when performing an acoustic test on a target space based on a specified measurement signal to obtain environmental response information, response information of multiple measurement points in the target space to the specified measurement signal can be collected through a preset acquisition device array, wherein the multiple measurement points constitute a measurement grid; information is extracted from the response information of the multiple measurement points to obtain impulse response information corresponding to the multiple measurement points; the impulse response information corresponding to the multiple measurement points is separated to obtain direct sound, early reflection and reverberation tail corresponding to the multiple measurement points; the impulse response information, direct sound, early reflection and reverberation tail corresponding to the multiple measurement points are determined as the environmental response information of the target space.

[0045] Wherein, as an example but not limitation, the specified measurement signal is a wide-band swept frequency signal that can cover the full frequency band of 20 Hz-20 kHz. Before collecting the response information of multiple measurement points in the target space to the specified measurement signal through the preset acquisition device array, the specified measurement signal can be generated by a preset exponential swept frequency signal algorithm, wherein the exponential swept frequency signal algorithm can be as follows: x(t)=sin[2πf 1 T(e ∧(t / L) -1 / ln(f 2 / f 1 )],f 1 is the starting frequency, f 2 is the end frequency, T is the sweep time, L = T / ln(f 2 / f 1 ). The specified measurement signal is generated by an exponential frequency sweep signal algorithm, thereby improving the signal-to-noise ratio and time resolution of the specified measurement signal.

[0046] Among them, in one implementation, before collecting the response information of multiple measurement points in the target space to the specified measurement signal through the preset collection device array, the environmental characteristics of the target space can be obtained, wherein the environmental characteristics include but are not limited to geometric structure and acoustic characteristics; according to the environmental characteristics of the target space, determine that multiple measurement points are distributed in different positions of the listening area in the target space, wherein the multiple measurement points form a three-dimensional measurement grid in the X, Y, and Z axis directions, and the three-dimensional measurement grid can cover multiple spatial positions in the target space, so as to collect comprehensive acoustic data. Specifically, it can be: obtain the environmental characteristics of the target space, wherein the environmental characteristics include but are not limited to geometric structure (for example, space shape, space size), environment type, environment use and acoustic characteristics; match the measurement point layout information corresponding to the environmental characteristics from the preset database, wherein the measurement point layout information includes but is not limited to the key position, number and combined shape of the measurement point; determine that multiple measurement points are distributed in different positions of the listening area in the target space based on the measurement point layout information. By determining multiple measurement points of the target space based on the measurement point layout information, the reliability of multiple measurement points is improved, which is conducive to the measurement of spaces adapted to various environments, and thus helps to improve the accuracy control of the audio of the target effect.

[0047] As an example but not a limitation, a high-precision measurement-grade microphone may be used in the preset acquisition device array to ensure the accuracy of the measurement. When the response information of multiple measurement points in the target space to the specified measurement signal is collected by the preset acquisition device array, the response information corresponding to each measurement point in the target space can be collected based on the preset measurement quantity by the preset acquisition device array to obtain the collection information corresponding to the preset measurement quantity of each measurement point; the arithmetic mean of the collection information corresponding to the preset measurement quantity of each measurement point is calculated to obtain the response information of multiple measurement points in the target space to the specified measurement signal. The arithmetic mean of the collection information corresponding to the preset measurement quantity of each measurement point is calculated to improve the signal-to-noise ratio of the response information.

[0048] In one implementation, when information extraction is performed on response information of multiple measurement points to obtain pulse response information corresponding to the multiple measurement points, a preset frequency domain analysis algorithm can be used to perform signal extraction on the response information of the multiple measurement points to obtain pulse response information corresponding to the multiple measurement points, wherein the frequency domain analysis algorithm can be as follows: h(t)=IFFT(H(ω)), H(ω)=Y(ω) / X(ω), H(ω) is the transfer function (frequency response function) corresponding to each measurement point, Y(ω) is the response information of each measurement point, X(ω) is the specified measurement signal, and h(t) is the pulse response information corresponding to each measurement point.

[0049] In one implementation, the impulse response information corresponding to the multiple measurement points can be separated respectively through a preset time window algorithm to obtain the direct sound, early reflection and reverberation tail corresponding to the multiple measurement points, wherein the time window algorithm can be a sliding window algorithm, a window function (for example, Hanning window) and an overlapping window algorithm to reduce spectrum leakage and improve the accuracy of frequency domain analysis.

[0050] By obtaining impulse response information, direct sound, early reflections and reverberation tail based on the specified measurement signal at multiple measurement points in the form of a measurement grid in the target space, it not only simplifies the measurement process, but also helps to improve the accuracy of environmental response information analysis, and helps to improve the audio propagation effect and listening quality.

[0051] In one implementation, before performing a sound effect test on the target space based on a specified measurement signal to obtain environmental response information, it is also possible to: identify the current audio usage scenario and automatically match the control strategy corresponding to the audio usage scenario, wherein the control strategy includes the content of the acoustic characteristic information, the compensation processing content, and the compensation processing flow.

[0052] In one implementation, the current audio usage scenario can be identified by a preset machine learning algorithm, wherein the machine learning algorithm can be a support vector machine (SVM) classification algorithm, specifically as follows: f(x)=sign(Σ(α i *y i *K(x i ,x))+b),α i is the Lagrange multiplier, y i is the category label, K is the kernel function, x i , x is the input audio to be processed, and b is the bias term.

[0053] As an example but not limitation, the audio usage scenario may be music listening, movie watching, and gaming. By adjusting the parameters and algorithms of the audio processing system according to the audio usage scenario, the audio experience in each scenario is optimized to meet the needs and preferences of different users.

[0054] Among them, the control strategy can be understood as performing a sound effect test on the target space based on a specified measurement signal to obtain environmental response information, wherein the environmental response information includes impulse response information and information after the impulse response is separated; analyzing the environmental response information to obtain the acoustic characteristic information of the target space; performing compensation processing on the processed audio based on the acoustic characteristic information to obtain the regulated audio; regulating the gain of the multi-channel corresponding to the regulated audio to obtain the audio after further regulation; performing multi-channel collaborative optimization processing on the audio after further regulation to obtain the specific implementation process of the audio with the target effect.

[0055] As an example but not limitation, the control strategy corresponding to the automatic matching audio usage scenario can be as follows: (1) When the audio usage scenario is music (characteristics: requiring high-fidelity sound quality, strong low-frequency response, and clear mid-high frequencies, and good audio spatial sense and stereo effects), the corresponding control strategy includes room acoustic compensation strategy (adjusting the frequency response to reduce standing waves and mode interference in the room to make the low frequency more balanced), reverberation time control strategy (setting a moderate reverberation time to preserve the natural continuity of the music without causing excessive reverberation tails) and dynamic range processing strategy (using slight dynamic range compression when listening to music to preserve the sound quality of the music). (2) When the audio usage scenario is a movie (characteristics: clear dialogue, rich sound effects, and appropriate low frequencies to enhance immersion, and more sensitive to the spatial positioning and effects of sound, such as surround sound and foreground sound effects), the corresponding control strategies include room acoustic compensation strategy (optimizing frequency response and phase to ensure that dialogue is clear and sound effects have good positioning), reverberation time control strategy (setting a longer reverberation time to simulate the surround sound effect of a movie while controlling low-frequency reverberation) and dynamic range processing strategy (using moderate dynamic range compression to balance the dynamic range of dialogue and sound effects to avoid the sound being too abrupt).

[0056] For example, the audio system has developed a touch screen interface where users can intuitively view and adjust the sound field through a 3D room model. By analyzing the audio content and user operations, the audio system can distinguish between music, movie, and game modes with 95% accuracy and automatically apply the corresponding optimized settings. The audio system observes that users often increase low-frequency output when watching action movies, so when an action scene is recognized, the low-frequency response is automatically increased by 2dB.

[0057] Intelligent dynamic optimization is achieved through intelligent scene recognition and matching of control strategies.

[0058] In one implementation, the audio control method further includes: displaying the sound field distribution of the audio to be processed and / or the sound field distribution of the audio of the target effect, and the comparison result through a preset three-dimensional (3D) spatial acoustic visualization tool, wherein the comparison result is the result of the comparison between the real-time frequency response curve and the optimization effect. The three-dimensional intuitive visualization of the three-dimensional spatial acoustic visualization tool ensures high performance while also ensuring ease of use and scalability, thereby greatly improving the user experience.

[0059] In one implementation, the audio control method further includes: in response to a touch operation of the three-dimensional space acoustic visualization tool, obtaining adjustment data, and adjusting the sound field of the audio to be processed and / or the audio of the target effect based on the adjustment data. Through the intuitive control of the three-dimensional space acoustic visualization tool, high performance is ensured while also ensuring ease of use and scalability, thereby greatly improving the user experience.

[0060] In one implementation, the audio control method further includes: in response to the optimization instruction, obtaining adjustment operation data based on the user interface; and optimizing parameters based on the adjustment operation data by using a preset reinforcement learning algorithm. As an example but not a limitation, the reinforcement learning algorithm may be as follows: Q(s,a)←Q(s,a)+α*[r+γ*max a 'Q(s',a')-Q(s,a)], s is the state, a is the action, α is the learning rate, r is the reward, and γ is the discount factor. The preset reinforcement learning algorithm is used to optimize parameters based on the adjusted operation data, which can continuously optimize system parameters to meet user preferences, thus realizing user preference learning and personalized dynamic optimization.

[0061] 202. Analyze the environmental response information to obtain acoustic characteristic information of the target space;

[0062] In one implementation, when analyzing environmental response information to obtain acoustic characteristic information of the target space, the following can be done: spectral analysis can be performed on impulse response information in the environmental response information to obtain spectral analysis results, wherein the spectral analysis results include a frequency response curve and a power spectrum density; environmental mode parameters of the target space are identified based on the power spectrum density, wherein the environmental mode parameters include modal density and modal overlap; reverberation times corresponding to multiple measurement points are calculated based on reverberation tails corresponding to the multiple measurement points; early reflections and direct sounds corresponding to the multiple measurement points are analyzed to obtain early reflection analysis results corresponding to the multiple measurement points; and the spectral analysis results, the environmental mode parameters of the target space, and the reverberation times and early reflection analysis results corresponding to the multiple measurement points are determined as the acoustic characteristic information of the target space.

[0063] In one implementation, when performing spectrum analysis on the impulse response information in the environmental response information and obtaining the spectrum analysis result, the following can be done: based on the impulse response information of each measurement point in the environmental response information, a frequency response curve of each measurement point is calculated, wherein the frequency response curve is used to present the response of the audio system at different frequencies; the frequency response curve is smoothed to obtain a smoothed frequency response curve; based on the smoothed frequency response curve, a power spectrum density is calculated (the square of the amplitude of the smoothed frequency response curve is normalized to obtain the power spectrum density); and the frequency response curve and the power spectrum density are determined as the spectrum analysis result.

[0064] As an example but not a limitation, the frequency response curve can be smoothed based on a one-third octave smoothing algorithm to obtain a smoothed frequency response curve to reduce the impact of small fluctuations; based on a preset power spectrum density estimation formula, the smoothed frequency response curve is calculated to obtain a power spectrum density, wherein the power spectrum density is used to indicate the distribution of the signal power of the environmental response information at different frequencies, and the power spectrum density estimation formula is as follows: PSD(ω)=|FFT(h(t)) 2 / N, PSD(ω) is the power spectral density, h(t) is the frequency response curve after smoothing, and N is the FFT length.

[0065] In one implementation, the modal density is used to indicate the number of modes within a unit frequency range, and the modal overlap is used to indicate the degree of overlap of different modes in frequency. When identifying the environmental mode parameters of the target space based on the power spectrum density, the following steps can be performed: statistically analyzing the power spectrum density to obtain the peak value of the power spectrum density, wherein the peak value of the power spectrum density corresponds to the main mode frequency; calculating the number of modes within a specific frequency range based on the peak value of the power spectrum density, wherein the number of modes within the specific frequency range is associated with the geometric size of the target space; calculating the modal density based on the number of modes within the specific frequency range and a preset calculation formula, wherein the preset calculation formula can be as follows: D(f)=N / Δf, N=(4πV / c 3 )*f 2 , D(f) is the modal density, N is the number of modes within a specific frequency range, Δf is the width of the frequency range, V is the volume of the target space, c is the speed of sound, and f is the frequency; the frequency response information of the target space is analyzed to determine the bandwidth of each mode of the target space; based on the bandwidth of each mode of the target space, the frequency range of different modes of the target space is plotted on the spectrum corresponding to the impulse response information to obtain the target spectrum; based on the target spectrum, the overlapping area of ​​different modes of the target space is calculated; the overlapping area of ​​different modes in the target spectrum and the total spectrum area of ​​all modes are calculated through the overlapping area of ​​different modes; the modal overlap is calculated based on the overlapping area of ​​different modes in the target spectrum and the total spectrum area of ​​all modes. Specifically, the modal overlap can be calculated by dividing the overlapping area of ​​different modes in the target spectrum by the total spectrum area of ​​all modes to obtain the modal overlap; the modal density and modal overlap are determined as the environmental mode parameters of the target space. By calculating the environmental mode parameters of the modal density and modal overlap, it is helpful to have a preliminary understanding of the environmental mode of the target space, which can reflect the degree of interaction between modes and the complexity of the dynamic response of the audio system, and help to more accurately analyze the acoustic characteristics of the target space.

[0066] In one implementation, when calculating the reverberation time corresponding to the multiple measurement points based on the reverberation tails corresponding to the multiple measurement points, the reverberation tails corresponding to the multiple measurement points can be calculated by a preset Schroeder integration algorithm to obtain the reverberation time corresponding to the multiple measurement points, wherein the Schroeder integration algorithm is as follows: EDC(t) = ∫[t→∞]h 2 (τ)dτ, RT60 = time when EDC(t) drops by 60dB, EDC(t) is the energy decay curve, t is the lower limit time of the integral, h 2 (τ) is the impulse response information, dτ is the small increment of the integral variable τ, and RT60 is the reverberation time corresponding to each measurement point. By calculating the reverberation time by frequency band, the attenuation characteristics of different frequencies can be effectively analyzed, which improves the reliability of the acoustic characteristic information.

[0067] In one implementation, when analyzing the early reflections and direct sounds corresponding to a plurality of measurement points, respectively, and obtaining the early reflection analysis results corresponding to the plurality of measurement points, the following can be done: the early reflections and direct sounds corresponding to each measurement point are respectively detected by a preset threshold detection algorithm to obtain the reflection peak value and the direct sound peak value corresponding to each measurement point, wherein the reflection peak value is the peak value of the early reflection, and the direct sound peak value is the peak value of the direct sound; the time delay of the early reflection relative to the direct sound is calculated based on the distance difference between the reflection peak value and the direct sound peak value corresponding to each measurement point; the total sound energy in the entire measurement period (including the direct sound and all reflected sounds) is calculated; the direct sound energy is calculated based on the direct sound peak value. Energy; calculate the sum of reflected energy, where specifically, for each reflection peak, determine a window around the center moment of each peak, calculate the square sum of the signal in each window, obtain the energy of a single reflection peak, add the energies of all reflection peaks, and obtain the sum of reflected energy; divide the direct sound energy by the sum of reflected energy to obtain a first energy ratio, divide the sum of reflected energy by the total sound energy to obtain a second energy ratio, and use the first energy ratio and the second energy ratio as the energy ratio of early reflection to direct sound; determine the time delay of early reflection corresponding to multiple measurement points relative to direct sound and the energy ratio of early reflection to direct sound as the early reflection analysis results corresponding to multiple measurement points. By calculating the time delay of early reflection corresponding to multiple measurement points relative to direct sound and the energy ratio of early reflection to direct sound, the efficiency and accuracy of early reflection analysis results are improved, which can effectively improve the spatial sense of audio and enhance the listener's sense of immersion.

[0068] By analyzing the environmental response information, we can obtain acoustic characteristic information including spectrum analysis results, environmental mode parameters of the target space, and reverberation time and early reflection analysis results corresponding to multiple measurement points. This improves the accuracy of the acoustic characteristic analysis of the target space, helps to effectively overcome the negative impact of the target space acoustics on the sound quality, and can effectively adapt to the unique acoustic environment of different spaces.

[0069] 203. Create an adaptive equalization model based on the frequency response curve in the acoustic characteristic information, wherein the adaptive equalization model is used for band-limited adaptive filtering;

[0070] In one implementation, when creating an adaptive equalization model based on the frequency response curve in the acoustic characteristic information, the following can be done: the frequency response curve in the acoustic characteristic information can be adjusted by specifying a standard curve to obtain a target response curve; a parameter equalization model can be constructed based on the target response curve; and the parameters of the parameter equalization model can be adjusted by a preset adaptive algorithm to obtain an adaptive equalization model.

[0071] Among them, the preset standard curve can be understood as a standard curve or reference curve in the industry that helps to design and calibrate audio systems to achieve a more natural or specific listening effect, that is, it can also be understood as an idealized frequency response curve with flatness, bandwidth coverage, phase consistency and compliance with industry standards, for example, Harman Target Curves, Diffuse Field Equalization (DFE) Curve, Equal Loudness Contours and Audio Engineering Society Standard (AES Standard) curve.

[0072] As an example but not limitation, before adjusting the frequency response curve in the acoustic characteristic information by specifying a standard curve to obtain a target response curve, you can: obtain the usage scenario and adjustment target of the audio to be processed; determine the corresponding standard curve based on the usage scenario and adjustment target of the audio to be processed to obtain the specified standard curve. When adjusting the frequency response curve in the acoustic characteristic information by specifying a standard curve to obtain a target response curve, you can: compare and analyze the specified standard curve and the frequency response curve in the acoustic characteristic information to obtain the difference; adjust the frequency response curve in the acoustic characteristic information based on the difference by a preset calibration tool to obtain the target response curve. By adjusting the frequency response curve in the acoustic characteristic information, the diffuse sound field response of the listening position can be taken into account.

[0073] In one implementation, when constructing a parameter equalization model based on a target response curve, the parameter equalization model can be constructed by using a target response curve and an IIR filter group in a second-order filter cascade structure, wherein the parameter equalization model can implement the following algorithm: H(s) = (s 2 +A*ω 0 / Q*s+ω 0 2 ) / (s 2 +ω 0 / Q*s+ω 0 2 ), where s is a complex frequency domain variable, s 2 It reflects the influence of the system on the second-order derivative of the time domain signal, A is the gain, ω 0 is the center frequency in the target response curve, Q is the quality factor, and accurate frequency response correction is achieved by adopting an IIR filter group. The smoothness of the phase response is ensured by adopting a second-order filter cascade structure.

[0074] As an example but not a limitation, the adaptive algorithm may be a minimum mean square error (LMS) algorithm, and the adaptive algorithm is specifically as follows: W(k+1)=W(k)+μ*X(k)*E(k), W(k) is the filter coefficient, μ is the step size, X(k) is the frequency response corresponding to the target response curve, and E(k) is the error signal. The parameters of the parameter equalization model are adjusted by the minimum mean square error (LMS) algorithm to achieve band-limited adaptive filtering and avoid overcorrection.

[0075] By adjusting the frequency response curve in the acoustic characteristic information through a specified standard curve and adjusting the parameters of the parametric equalization model through a preset adaptive algorithm, the reliability and accuracy of the adaptive equalization model are improved, which helps to output better audio effects and meet the usage requirements in various scenarios (various complex environments).

[0076] 204. Create an adaptive modal control model, wherein the adaptive modal control model is used for adaptive dynamic control environment mode;

[0077] In one implementation, when creating an adaptive modal control model, the following steps may be performed: identifying a mode type of a target space; setting a filter corresponding to the mode type; and adjusting parameters of the filter corresponding to the mode type to obtain an adaptive modal control model.

[0078] In one implementation, when identifying the pattern type of the target space, a calculation result may be obtained by performing calculation using a preset pattern frequency algorithm, wherein the pattern frequency algorithm is specifically as follows: c is, nx, ny, nz are the mode numbers, Lx, Ly, Lz are the room dimensions; the mode type of the target space is determined based on the calculation result, where the mode type includes axial mode, tangential mode and oblique mode. By identifying the mode type of the target space, detailed information is provided for subsequent suppression and optimization. As an example but not limitation, when the calculation result f nx,ny,nz There is only one non-zero pattern number (f nx,0,0 , f 0,ny,0 , f 0,0,nz ), the mode type of the target space is the axial mode; when the calculation result f nx,ny,nz There are two non-zero pattern ordinals (f nx,ny,0 , f nx,0,nz , f 0,ny,nz ), the mode type of the target space is the tangential mode; when the calculation result f nx,ny,nz When there are three non-zero pattern ordinals in , the pattern type of the target space is an oblique pattern.

[0079] In one implementation, when setting the filter corresponding to the mode type, narrow-band notch filters can be set for the axial mode, tangential mode and oblique mode in the mode type, respectively, wherein the narrow-band notch filter can realize: H(z) = (1-2r*cosω 0 *z -1 +r 2 *z -2 ) / (1-2*cosω 0 *z -1 +z -2 ), r is the control bandwidth, ω 0 is the center frequency, z is the complex variable in the Z transform, representing the frequency domain characteristics of the discrete-time system; the quality factor value and depth of each narrowband notch filter are adjusted to achieve the best suppression effect.

[0080] In one implementation, when adjusting the parameters of the filter corresponding to the mode type to obtain the adaptive modal control model, the parameters of the filter corresponding to the mode type can be adjusted by a recursive least squares (RLS) algorithm to obtain the adaptive modal control model, wherein the recursive least squares (RLS) algorithm is specifically as follows: K(n) = [λ -1 *P(n-1)*x(n)] / (1+λ -1 *x∧T(n)*P(n-1)*x(n)), e(n)=d(n)-w∧T(n)*x(n), w(n)=w(n-1)+K(n)*e(n), P(n)=λ -1 *P(n-1)-λ -1*K(n)*x∧T(n)*P(n-1), K(n) is the gain vector, which controls the amplitude of weight update, λ is the forgetting factor, which controls the influence of historical data on the current iteration, P(n) is the error covariance matrix, which measures the error and uncertainty of the filter, x(n) is the input signal, the signal entering the filter, T(n) is the transpose of the vector x at the time point n, e(n) is the error signal, the difference between the expected signal and the filter output, d(n) is the expected signal, the ideal value that the system wants the filter to output, and w(n) is the filter weight, which determines how the filter processes the input signal. Through the recursive least squares (RLS) algorithm, it can cope with the fluctuation of low-frequency response under environmental changes, especially the low-frequency resonance caused by changes in spatial state (such as opening doors and moving furniture), and make more precise and faster real-time dynamic adjustments to the spatial mode of the low-frequency band, monitor the changes in low-frequency response in real time, and adapt to the state changes of the target space.

[0081] By setting the corresponding filter according to the mode type of the target space and performing adaptive modal control on the filter, it is possible to cope with low-frequency response fluctuations caused by environmental changes, and make more precise and faster real-time dynamic adjustments to the spatial mode of the low-frequency band, thereby improving the flexibility of audio control and maintaining the flatness of the low-frequency response.

[0082] 205. Optimizing the early reflection analysis result in the acoustic characteristic information to obtain optimized early reflection information;

[0083] In one implementation, when optimizing the early reflection analysis results in the acoustic characteristic information to obtain the optimized early reflection information, the following can be done: based on the early reflection analysis results in the acoustic characteristic information, calculate the delay time and energy attenuation value of each early reflection path; perform time alignment processing on the early reflection based on the delay time, and perform phase correction on the impulse response information to obtain processed information; obtain target sound field characteristics, wherein the target sound field characteristics are ideal environmental acoustic characteristics; based on the energy attenuation value and the target sound field characteristics, adjust the reflection energy in the early reflection analysis results to obtain adjusted reflection energy; and determine the processed information and the adjusted reflection energy as the optimized early reflection information.

[0084] In one implementation, when calculating the delay time and energy attenuation value of each early reflection path based on the early reflection analysis results in the acoustic characteristic information, it is possible to: construct an early reflection path model based on the geometric information of the target space and the response information of multiple measurement points in the target space to a specified measurement signal; calculate the delay time and energy attenuation value of each early reflection path through the early reflection path model, specifically, perform sound ray tracing through the early reflection path model, and calculate the total path length of the sound ray of each early reflection path, divide the total path length of the sound ray of each early reflection path by the speed of sound to obtain the delay time of each early reflection path, calculate the continuous product of the sound absorption coefficients of all reflecting surfaces along the early reflection path, and calculate the energy attenuation value of each early reflection path based on the continuous product of the sound absorption coefficients of all reflecting surfaces along the early reflection path.

[0085] Among them, the early reflection path model can be used based on the mirror source algorithm to calculate the mirror position of the sound source for each reflecting surface and trace the sound line path to achieve sound line tracking through the early reflection path model. Specifically, the structure of the target space (including the length, width, height of the target space and the position of the reflecting surface) is measured, the position of the sound source receiver is set, the mirror sound source position of each reflecting surface is calculated, the path from the mirror sound source to the receiver is traced, the path length, time delay and energy attenuation are calculated, multiple reflections are repeatedly processed based on a preset number of reflections, the impulse responses of all reflected sounds are synthesized, the final room impulse response is generated, and convolved with the input signal (impulse response information) to simulate the sound propagation effect in the target space.

[0086] In one implementation, when the early reflections are time-aligned based on the delay time and the impulse response information is phase-corrected to obtain the processed information, an FIR filter can be set, wherein the FIR filter is specifically as follows: h(n)=sinc(nD)*w(n), n is the time point in the delay time, D is the fractional delay, which represents the delay between the early reflected sound and the direct sound, and w(n) is the window function; the early reflections are time-aligned by the FIR filter. The early reflections are time-aligned by the FIR filter, so that the early reflected sound and the direct sound are precisely synchronized in time. The impulse response information is extracted to obtain phase information; phase compensation is performed based on the phase information by a preset all-pass filter to adjust the phase of the corresponding frequency component. Phase correction is performed by the all-pass filter to optimize the sound image positioning.

[0087] Among them, the target sound field characteristics are ideal environmental acoustic characteristics. The target sound field characteristics are determined through reference standards, measurements, simulations or user needs, and define ideal spatial acoustic characteristics, such as reverberation time, energy distribution and diffusivity of early reflections.

[0088] In one implementation, when the reflected energy in the early reflection analysis result is adjusted based on the energy attenuation value and the target sound field characteristic to obtain the adjusted reflected energy, the energy of each early reflection in the early reflection analysis result is adjusted based on the energy attenuation value, the target sound field characteristic and the reflection energy optimization algorithm to make it meet the target energy to obtain the adjusted reflected energy, wherein the reflection energy optimization algorithm is specifically as follows: minimize∑(E i -T i ) 2 ,subject to∑E i =E total , E i is the energy of the i-th early reflection, T i is the target energy, E total The total reflected energy is adjusted based on the energy attenuation value, target sound field characteristics and reflection energy optimization algorithm to achieve a controllable diffuse sound field and precise acoustic control in the target space, thus optimizing the clarity, spatial sense and positioning of the audio sound.

[0089] Through time alignment, phase correction and reflected energy adjustment, a controllable diffuse sound field and precise acoustic control in the target space are achieved, which optimizes the clarity, spatial sense and positioning of the audio sound, creates a natural diffuse sound field, and significantly improves the stability and spatial sense of the sound image.

[0090] 206. Adjust the reverberation time in the acoustic characteristic information to obtain an adjusted reverberation time;

[0091] In one implementation, when adjusting the reverberation time in the acoustic characteristic information to obtain the adjusted reverberation time, the following steps may be performed: determining an ideal reverberation time according to a scene type; constructing a multi-channel feedback delay network, wherein the multi-channel feedback delay network is used to simulate natural reverberation; and adjusting the reverberation time in the acoustic characteristic information based on the ideal reverberation time and the multi-channel feedback delay network to obtain the adjusted reverberation time.

[0092] As an example but not a limitation, the scene type of the target space and the content type of the audio to be processed can be obtained, and the ideal reverberation time is determined according to the scene type of the target space, the content type of the audio to be processed and a preset calculation function, wherein the ideal reverberation time considers the optimal reverberation time ratio of different frequency bands, wherein the calculation function is specifically as follows: RT o pt=k*(V / S) (1 / 3) , k is a coefficient determined according to the purpose of the target space (scene type), V is the volume of the target space, and S is the total surface area of ​​the target space.

[0093] As an example but not a limitation, the constructed multi-channel feedback delay network may be specifically as follows: y(n)=b T *z(n)+d*x(n), z(n+1)=A*[D*z(n)+c*x(n)], z(n) is the state vector of the feedback delay network, which represents the signal input after passing through several delay units, b, d, c are gain coefficients, x(n) is the input signal, that is, the data to be reverberated, A is the feedback matrix, and D is the delay matrix. By constructing a multi-channel feedback delay network, frequency-dependent reverberation attenuation is achieved, simulating the characteristics of a real space.

[0094] In one implementation, when adjusting the reverberation time in the acoustic characteristic information based on the ideal reverberation time and the multi-channel feedback delay network to obtain the adjusted reverberation time, the following can be done: analyzing the actual reverberation characteristics of the target space in real time, wherein the actual reverberation characteristics may be the actual reverberation time, estimating the reverberation time from the impulse response information by a preset reverberation time estimation algorithm to obtain the actual reverberation characteristics of the target space, wherein the reverberation time estimation algorithm is specifically as follows: RT60 = argmax_RTp(h|RT), h is the impulse response information, and p(h|RT) is the probability of observing h at a given reverberation time RT; adjusting the artificial reverberation parameters of the multi-channel feedback delay network based on the ideal reverberation time and the actual reverberation characteristics of the target space to obtain the adjusted reverberation time. By adapting the reverberation parameters to the adaptive conditions, the natural reverberation is compensated or enhanced.

[0095] By adjusting the reverberation time in the acoustic characteristic information based on the ideal reverberation time and the multi-channel feedback delay network, the reverberation time is controlled, the natural reverberation and real space characteristics are simulated, and the fullness of the low-frequency response is maintained.

[0096] 207. The audio to be processed is regulated and processed by using an adaptive equalization model, an adaptive modal control model, the optimized early reflection information and the adjusted reverberation time to obtain regulated audio;

[0097] By creating an adaptive equalization model, an adaptive modal control model, early reflection optimization and reverberation time adaptive control, the reliability and accuracy of the target acoustic characteristic information are improved, which is conducive to effectively overcoming the negative impact of spatial acoustics on sound quality, so that close to ideal sound reproduction effects can be obtained in different environments.

[0098] 208. Regulating the gain of the multi-channels corresponding to the regulated audio to obtain the re-regulated audio;

[0099] In one implementation, after adjusting the gain of the multi-channels corresponding to the adjusted audio to obtain the re-adjusted audio, it is also possible to: obtain noise prediction information of the target space, wherein the noise prediction information is used to indicate the noise level in different time periods; based on the noise prediction information, adjust the compression ratio and compression threshold of the multi-channels corresponding to the adjusted audio to obtain the adjusted multi-channels; and perform masking-based gain control on the adjusted audio based on the adjusted multi-channels to obtain the re-adjusted audio.

[0100] In one implementation, when obtaining the noise prediction information of the target space, the background noise of the target space can be identified by using a preset long-time average spectrum algorithm, wherein the long-time average spectrum algorithm is specifically as follows: LTAS(f) = (1 / M)*∑|X m (f)| 2 , M is the total number of frames, X m (f) is the short-time Fourier transform of the mth frame; a noise model is established based on the background noise of the target space; the noise level at different time periods is predicted by the noise model to obtain the noise prediction information of the target space.

[0101] In one implementation, the compression ratio is used to indicate the degree of compression after the input signal exceeds the threshold, and defines the input-output relationship of the signal when the signal exceeds the set threshold. The higher the compression ratio, the stronger the signal is compressed. The compression threshold is used to indicate the trigger point of dynamic range compression and defines when the signal starts to be compressed. When the compression ratio and compression threshold of the multi-channel corresponding to the regulated audio are adjusted based on the noise prediction information to obtain the adjusted multi-channel, the compression ratio and compression threshold of the multi-channel corresponding to the regulated audio can be adjusted by a preset multi-band dynamic range compression algorithm and noise prediction information to obtain the adjusted multi-channel, wherein the multi-band dynamic range compression algorithm is specifically as follows: y(n)=x(n)*G(n), G(n)=min(1,(T / |x(n)|) (1-1 / R) ), y(n) is the output signal after dynamic range compression, G(n) is the gain (or attenuation) coefficient of the dynamic range compressor, which is used to control the amplitude of the signal, T is the compression threshold, |x(n)| is the signal amplitude, and R is the compression ratio; when the input signal amplitude |x(n)| exceeds the compression threshold T, the compressor starts to work and reduces the increment of the input signal. When the input signal amplitude |x(n)| is less than or equal to the compression threshold T, the original state is maintained without compression. Through the preset multi-band dynamic range compression algorithm and noise prediction information, the compression ratio and compression threshold of the multi-channel corresponding to the regulated audio are adjusted, which can control the amplitude of the signal to keep it within a reasonable range, control the signal with large intensity changes, reduce its dynamic range, prevent the signal from being too strong or too weak, and ensure auditory comfort, which is very important especially in the case of high spatial noise.

[0102] In one implementation, when performing masking-based gain control on the regulated audio based on the adjusted multi-channel pair to obtain the re-regulated audio, the regulated audio based on the adjusted multi-channel pair can be processed by a preset masking-based gain control algorithm to obtain the re-regulated audio, wherein the masking-based gain control algorithm is as follows: G_opt = argmax_G (SNR_perceptual (G) - α*|G - G_prev|), SNR_perceptual (G) is the perceived signal-to-noise ratio, α is the smoothing factor, G_prev is the gain of the previous frame, and G is the gain of the current frame. By processing the regulated audio based on the adjusted multi-channel pair through the preset masking-based gain control algorithm, the automatic volume adjustment is realized, the appropriate ratio of signal to noise is maintained, the auditory masking effect is considered, and the gain adjustment strategy is optimized.

[0103] By adjusting the compression ratio and compression threshold of the multi-channel corresponding to the controlled audio based on noise prediction information, and performing masking-based gain control on the controlled audio based on the adjusted multi-channel, multi-band compression is achieved, the spectral balance of the signal is protected, automatic volume adjustment is achieved, the appropriate ratio of signal to noise is maintained, the auditory masking effect is taken into account, the gain adjustment strategy is optimized, and it is ensured that the dynamic range of the audio is fully displayed in a quiet environment while avoiding excessive sound stimulation.

[0104] 209. Perform multi-channel collaborative optimization processing on the audio after the second adjustment to obtain audio with a target effect.

[0105] In one implementation, when performing multi-channel collaborative optimization processing on the re-adjusted audio to obtain audio with a target effect, the following can be done: analyze the channel cross-influence characteristics between the adjusted multi-channels; perform phase compensation and amplitude compensation on the re-adjusted audio based on the channel cross-influence characteristics to obtain a balanced sound effect; and process the equalized sound effect based on a preset adaptive cross-cancellation algorithm to obtain audio with a target effect.

[0106] In one implementation, when analyzing the channel cross-effect characteristics between the adjusted multi-channels, the following steps may be performed: measuring and analyzing the acoustic coupling characteristics between the adjusted channels; establishing a multi-channel interaction model based on the acoustic coupling characteristics; and analyzing the channel cross-effect characteristics between the adjusted multi-channels based on a multi-channel transfer function matrix through the multi-channel interaction model, wherein the multi-channel transfer function matrix is ​​specifically as follows: H = [H 11 H 12 ...H 1N ;? H 21 H 22 ...H 2N;? ...? H N1 H N2 ...H NN ], H ij is the transfer function from channel j to position i.

[0107] In one implementation, when performing phase compensation and amplitude compensation on the re-adjusted audio based on the channel cross-influence characteristics to obtain the equalized sound effect, a multi-channel joint equalizer can be created based on the channel cross-influence characteristics, wherein the multi-channel joint equalizer can achieve: o pt=(H H *H+λI) -1 *H H , H is the multi-channel transfer function matrix used in the channel cross-effect characteristics, λ is the regularization parameter, I is the unit matrix, and is the weight calculation formula for the multi-channel Wiener filter. It is a matrix with diagonal elements of 1 and the remaining elements of 0, and its size is the same as H H The same is used for regularization; the phase compensation and amplitude compensation of the re-adjusted audio are performed through the multi-channel joint equalizer to obtain the balanced sound effect. The phase and amplitude compensation between channels are realized through the multi-channel joint equalizer.

[0108] As an example and not limitation, the adaptive cross-cancellation algorithm may be: i (n+1)=w i (n)+μ*e(n)*x i (n) / (||x(n)|| 2 +δ), w i (n) is the coefficient of the i-th filter, μ is the step size, e(n) is the error signal, x i (n) is the input signal of each channel, and δ is a small positive number.

[0109] By performing phase compensation and amplitude compensation on the re-adjusted audio based on the channel cross-influence characteristics, and processing the equalized sound effect based on a preset adaptive cross-cancellation algorithm, phase and amplitude compensation between channels are achieved, adaptive interference cancellation between channels is achieved, negative interference between channels is minimized, comb filtering effects caused by crosstalk are reduced, and the clarity of dialogues in the audio and the separation of the front sound field are improved.

[0110] In one implementation, after performing multi-channel collaborative optimization processing on the re-adjusted audio to obtain audio with a target effect, it is also possible to: obtain analysis data, and update the mechanism and / or optimize the strategy of the audio control method based on the analysis data, wherein the analysis data includes performance test data and user evaluation data.

[0111] In one implementation, the performance test data includes but is not limited to a comprehensive score of acoustic parameters. The performance test data may be obtained as follows: a comprehensive score of acoustic parameters is calculated by an acoustic parameter comprehensive scoring algorithm, wherein the acoustic parameter comprehensive scoring algorithm is specifically as follows: Score = w1*F_flatness+w2*T_60_deviation+w3*C_50+w4*D_50+w5*LF, w1, w2, w3, w4, w5 are weight coefficients, F_flatness is frequency response flatness, T_60_deviation is reverberation time deviation, C_50 is clarity, D_50 is definition, and LF is lateral energy score.

[0112] In one implementation, the user evaluation data includes data obtained after double-blind testing by professional audio engineers and ordinary users, as well as overall sound quality performance data of the system evaluated using the MUSHRA method.

[0113] In one implementation, the cloud big data can be self-optimized based on the analysis data, or the algorithm performance can be improved by remote firmware update based on the analysis data (mechanism update and / or strategy optimization of the audio control method), wherein the cloud big data self-optimization based on the analysis data can be implemented by a parameter optimization algorithm based on swarm intelligence, and the parameter optimization algorithm based on swarm intelligence is as follows: i (t+1)=p i (t)+v i (t+1), v i (t+1)=w*v i (t)+c1*r1*(p best -p i (t))+c2*r2*(g best -p i (t)), p i (t+1) is the parameter vector, v i (t+1) is the velocity vector, w, c1, c2 are control parameters, r1, r2 are random numbers, p best is the individual optimal solution, p i (t) is the individual optimization strategy, g best It is the global optimal solution.

[0114] For example, in 50 different home environments, the system improved the frequency response flatness to within ±3dB in 90% of the scenarios, and the reverberation time control error was less than 10%. In the MUSHRA test, the system received an average score of 85 points, significantly higher than the 65 points in the unoptimized state. Through cloud data analysis, the system identified that the system 3In view of the general trend of weak low-frequency response in rooms with large volume, we updated the optimization strategy for the corresponding room types and further improved the system performance.

[0115] Through rigorous performance testing and user evaluation, it has demonstrated excellent performance and stability. Through cloud-based big data analysis and remote update mechanisms, it can continuously optimize performance and make customized adjustments to the special needs of different types of spaces, greatly improving the adaptability and long-term value of the audio system.

[0116] This audio control method uses advanced spatial acoustic measurement technology and combines artificial intelligence algorithms to accurately analyze the acoustic characteristics of the room (including frequency response, reverberation time, early reflection, room mode, etc.), introduces adaptive filtering and intelligent equalization technology, can automatically adjust audio parameters according to the analysis results, and realizes accurate compensation for spatial acoustic defects. The innovative room mode suppression algorithm effectively improves the low-frequency response. In addition, real-time adaptive technology is integrated to dynamically respond to changes in room acoustic characteristics (such as opening and closing doors and windows, movement of people, etc.) and continuously optimize audio output. The automatic adjustment function brings significant sound quality improvement and convenience to users. It can effectively overcome the negative impact of room acoustics on sound quality, so that close to ideal sound playback effects can be obtained in different environments. The intelligent and automated features allow users to easily obtain professional-level audio adjustment effects without professional knowledge. By accurately controlling the frequency response and phase characteristics, the system significantly improves the clarity, positioning and spatial sense of the sound. The dynamic adaptive function ensures that the best sound quality can be maintained when the room usage status changes. The system's room mode processing technology greatly improves the low-frequency performance and provides a more balanced and powerful low-frequency effect. Through innovative algorithm design and system architecture, it comprehensively solves many challenges faced by traditional audio systems in actual room environments. It not only accurately analyzes and compensates for the impact of room acoustic characteristics on sound quality, but also realizes intelligent dynamic optimization, providing strong technical support for users to obtain the best audio experience in various environments. The adaptive learning ability and real-time optimization characteristics enable it to adapt to different acoustic environments, audio content and user preferences, representing a major leap in home audio system technology. Through rigorous performance testing and user evaluation, the system has demonstrated excellent performance and stability, and is expected to be widely used in the high-end audio market, and points the way for the development of future smart home audio technology.

[0117] In the present embodiment, compensation processing, gain control processing and multi-channel collaborative optimization processing are performed based on the acoustic characteristic information obtained by analyzing the environmental response information based on the sound effect test of the specified measurement signal, so that accurate compensation for the acoustic defects of the target space is achieved, the crosstalk problem between channels is effectively solved, and the positioning and spatial sense of sound and image are significantly improved, thereby greatly improving the overall listening experience of the audio. It can adapt to different listening environments and sound source types, continuously optimize audio output, and ensure the best listening experience in various environments.

[0118] The audio control method in this embodiment is described above. The audio control device in this embodiment is described below. Figure 3 In this embodiment, an audio control device includes:

[0119] An acquisition module 301 is used to perform a sound effect test on a target space based on a specified measurement signal to obtain environmental response information, wherein the environmental response information includes impulse response information and information after the impulse response is separated;

[0120] An analysis module 302 is used to analyze the environmental response information to obtain acoustic characteristic information of the target space;

[0121] A compensation module 303, configured to perform compensation processing on the audio to be processed based on the acoustic characteristic information to obtain a regulated audio;

[0122] A control module 304 is used to control the gain of multiple channels corresponding to the regulated audio to obtain the regulated audio;

[0123] The optimization module 305 is used to perform multi-channel collaborative optimization processing on the audio after re-adjustment to obtain audio with target effect.

[0124] In this embodiment, compensation processing, gain control processing and multi-channel collaborative optimization processing are performed based on the acoustic characteristic information obtained by analyzing the environmental response information of the sound effect test based on the specified measurement signal, so as to achieve accurate compensation for the acoustic defects of the target space, effectively solve the problem of crosstalk between channels, significantly improve the positioning and spatial sense of sound and image, thereby greatly improving the overall listening experience of the audio and continuously optimizing the audio output.

[0125] Optionally, the acquisition module 301 may also be specifically used for:

[0126] Collect response information of multiple measurement points in the target space to a specified measurement signal by presetting an array of acquisition devices, wherein the multiple measurement points constitute a measurement grid;

[0127] Extracting information from the response information of multiple measurement points respectively to obtain impulse response information corresponding to the multiple measurement points;

[0128] The impulse response information corresponding to the multiple measurement points is separated respectively, so as to obtain the direct sound, early reflection and reverberation tail sound corresponding to the multiple measurement points respectively;

[0129] The impulse response information, direct sound, early reflection and reverberation tail corresponding to the multiple measurement points are determined as the environmental response information of the target space.

[0130] Optionally, the analysis module 302 may also be specifically used for:

[0131] Performing spectrum analysis on the impulse response information in the environmental response information to obtain spectrum analysis results, wherein the spectrum analysis results include a frequency response curve and a power spectrum density;

[0132] Identifying environmental mode parameters of the target space based on power spectrum density, wherein the environmental mode parameters include modal density and modal overlap;

[0133] Calculate the reverberation time corresponding to the plurality of measurement points based on the reverberation tails corresponding to the plurality of measurement points;

[0134] Analyze the early reflections and direct sounds corresponding to the multiple measurement points to obtain the early reflection analysis results corresponding to the multiple measurement points;

[0135] The spectrum analysis results, the environmental mode parameters of the target space, and the reverberation time and early reflection analysis results corresponding to a plurality of measurement points are determined as the acoustic characteristic information of the target space.

[0136] Optionally, the compensation module 303 may also be specifically used for:

[0137] Creating an adaptive equalization model based on a frequency response curve in the acoustic characteristic information, wherein the adaptive equalization model is used for band-limited adaptive filtering;

[0138] Creating an adaptive modal control model, wherein the adaptive modal control model is used for adaptive dynamic control environment mode;

[0139] Optimizing the early reflection analysis results in the acoustic characteristic information to obtain optimized early reflection information;

[0140] Adjusting the reverberation time in the acoustic characteristic information to obtain an adjusted reverberation time;

[0141] The audio to be processed is regulated and processed through an adaptive equalization model, an adaptive modal control model, optimized early reflection information and adjusted reverberation time to obtain the regulated audio.

[0142] Optionally, the compensation module 303 may also be specifically used for:

[0143] The frequency response curve in the acoustic characteristic information is adjusted by specifying a standard curve to obtain a target response curve;

[0144] Construct parameter equalization model based on target response curve;

[0145] The parameters of the parameter equalization model are adjusted by a preset adaptive algorithm to obtain an adaptive equalization model.

[0146] Optionally, the compensation module 303 may also be specifically used for:

[0147] Identify the pattern type of the target space;

[0148] Set the filter corresponding to the mode type;

[0149] The parameters of the filter corresponding to the mode type are adjusted to obtain an adaptive modal control model.

[0150] Optionally, the compensation module 303 may also be specifically used for:

[0151] Based on the early reflection analysis results in the acoustic characteristic information, the delay time and energy attenuation value of each early reflection path are calculated;

[0152] Performing time alignment processing on the early reflection based on the delay time and performing phase correction on the impulse response information to obtain processed information;

[0153] Acquiring target sound field characteristics, wherein the target sound field characteristics are ideal environmental acoustic characteristics;

[0154] Based on the energy attenuation value and the target sound field characteristics, the reflection energy in the early reflection analysis result is adjusted to obtain the adjusted reflection energy;

[0155] The processed information and the adjusted reflection energy are determined as optimized early reflection information.

[0156] Optionally, the compensation module 303 may also be specifically used for:

[0157] Determine the ideal reverberation time based on the scene type;

[0158] Constructing a multi-channel feedback delay network, wherein the multi-channel feedback delay network is used to simulate natural reverberation;

[0159] Based on the ideal reverberation time and the multi-channel feedback delay network, the reverberation time in the acoustic characteristic information is adjusted to obtain the adjusted reverberation time.

[0160] Optionally, the control module 304 may also be specifically used for:

[0161] Acquiring noise prediction information of the target space, wherein the noise prediction information is used to indicate noise levels at different time periods;

[0162] Based on the noise prediction information, the compression ratio and the compression threshold of the multi-channel corresponding to the regulated audio are adjusted to obtain an adjusted multi-channel;

[0163] The masking-based gain control is performed on the regulated audio based on the adjusted multi-channel to obtain the re-regulated audio.

[0164] Optionally, the optimization module 305 may also be specifically used for:

[0165] Analyze the cross-channel influence characteristics between the adjusted multi-channels;

[0166] Based on the cross-channel influence characteristics, phase compensation and amplitude compensation are performed on the re-adjusted audio to obtain a balanced sound effect;

[0167] The equalized sound effects are processed based on a preset adaptive cross-cancellation algorithm to obtain audio with the target effect.

[0168] Optionally, the audio control device further comprises:

[0169] The identification and matching module 306 is used to identify the current audio usage scenario and automatically match the control strategy corresponding to the audio usage scenario, wherein the control strategy includes the content of the acoustic characteristic information, the compensation processing content, and the compensation processing flow.

[0170] Optionally, the audio control device further comprises:

[0171] The update optimization module 307 is used to obtain analysis data and perform mechanism update and / or strategy optimization of the audio control method based on the analysis data, wherein the analysis data includes performance test data and user evaluation data.

[0172] The functional implementation of each module and each unit in the above audio control device corresponds to each step in the above audio control method embodiment, and their functions and implementation processes are not repeated here one by one.

[0173] This embodiment also provides an electronic device, including a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the above audio control method. The electronic device can be a server or a terminal device.

[0174] See also Figure 4 As shown, the electronic device includes a processor 400 and a memory 401 . The memory 401 stores machine executable instructions that can be executed by the processor 400 . The processor 400 executes the machine executable instructions to implement the above audio control method.

[0175] Further, Figure 4 The electronic device shown further includes a bus 402 and a communication interface 403 , and the processor 400 , the communication interface 403 and the memory 401 are connected via the bus 402 .

[0176] The memory 401 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 403 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 402 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0177] The processor 400 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 400. The above processor 400 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in this embodiment can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with this embodiment can be directly embodied as a hardware decoding processor to execute, or a combination of hardware and software modules in the decoding processor to execute. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 401, and the processor 400 reads the information in the memory 401 and completes the steps of the audio control method in combination with its hardware.

[0178] The present disclosure also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. A computer program is stored in the computer-readable storage medium. When the computer program runs on a computer, the computer executes the steps of the audio control method.

[0179] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0180] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several computer programs to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0181] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. An audio control method, characterized in that: The audio control method comprises: Performing a sound effect test on the target space based on the specified measurement signal to obtain environmental response information, wherein the environmental response information includes impulse response information and information after the impulse response is separated; Analyzing the environmental response information to obtain acoustic characteristic information of the target space; Performing compensation processing on the audio to be processed based on the acoustic characteristic information to obtain regulated audio; Adjusting the gains of multiple channels corresponding to the adjusted audio to obtain further adjusted audio; The re-adjusted audio is subjected to multi-channel collaborative optimization processing to obtain audio with a target effect.

2. The audio control method according to claim 1, characterized in that: The step of performing a sound effect test on the target space based on the specified measurement signal to obtain environmental response information includes: Collecting response information of a plurality of measurement points in a target space to a specified measurement signal by presetting an array of acquisition devices, wherein the plurality of measurement points constitute a measurement grid; Extracting information from the response information of the plurality of measurement points respectively to obtain impulse response information corresponding to the plurality of measurement points respectively; Separating the impulse response information corresponding to the multiple measurement points respectively to obtain the direct sound, early reflection and reverberation tail sound corresponding to the multiple measurement points respectively; The impulse response information, direct sound, early reflection and reverberation tail sound respectively corresponding to the multiple measurement points are determined as the environmental response information of the target space.

3. The audio control method according to claim 2, characterized in that: The analyzing the environmental response information to obtain the acoustic characteristic information of the target space includes: Performing spectrum analysis on the impulse response information in the environmental response information to obtain spectrum analysis results, wherein the spectrum analysis results include a frequency response curve and a power spectrum density; Identifying environmental mode parameters of the target space based on the power spectrum density, wherein the environmental mode parameters include modal density and modal overlap; Calculating the reverberation time corresponding to the plurality of measurement points respectively based on the reverberation tails corresponding to the plurality of measurement points respectively; Analyze the early reflections and direct sounds respectively corresponding to the multiple measurement points to obtain early reflection analysis results respectively corresponding to the multiple measurement points; The spectrum analysis result, the environmental mode parameters of the target space, and the reverberation time and early reflection analysis results respectively corresponding to the plurality of measurement points are determined as the acoustic characteristic information of the target space.

4. The audio control method according to claim 3, characterized in that: The compensating the audio to be processed based on the acoustic characteristic information to obtain the regulated audio includes: Creating an adaptive equalization model based on a frequency response curve in the acoustic characteristic information, wherein the adaptive equalization model is used for band-limited adaptive filtering; Creating an adaptive modal control model, wherein the adaptive modal control model is used for adaptive dynamic control environment mode; Optimizing the early reflection analysis result in the acoustic characteristic information to obtain optimized early reflection information; Adjusting the reverberation time in the acoustic characteristic information to obtain an adjusted reverberation time; The audio to be processed is regulated and processed by using the adaptive equalization model, the adaptive modal control model, the optimized early reflection information and the adjusted reverberation time to obtain the regulated audio.

5. The audio control method according to claim 4, characterized in that: The step of creating an adaptive equalization model based on a frequency response curve in the acoustic characteristic information includes: The frequency response curve in the acoustic characteristic information is adjusted by specifying a standard curve to obtain a target response curve; constructing a parameter balancing model based on the target response curve; The parameters of the parameter equalization model are adjusted by a preset adaptive algorithm to obtain an adaptive equalization model.

6. The audio control method according to claim 4, characterized in that: The step of creating an adaptive modal control model comprises: identifying a pattern type of the target space; Set the filter corresponding to the mode type; The parameters of the filter corresponding to the mode type are adjusted to obtain an adaptive modal control model.

7. The audio control method according to claim 4, characterized in that: The optimizing the early reflection analysis result in the acoustic characteristic information to obtain optimized early reflection information includes: Calculating the delay time and energy attenuation value of each early reflection path based on the early reflection analysis result in the acoustic characteristic information; Performing time alignment processing on the early reflection based on the delay time and performing phase correction on the impulse response information to obtain processed information; Acquiring target sound field characteristics, wherein the target sound field characteristics are ideal environmental acoustic characteristics; Based on the energy attenuation value and the target sound field characteristic, adjusting the reflection energy in the early reflection analysis result to obtain adjusted reflection energy; The processed information and the adjusted reflection energy are determined as optimized early reflection information.

8. The audio control method according to claim 4, characterized in that: The adjusting the reverberation time in the acoustic characteristic information to obtain the adjusted reverberation time includes: Determine the ideal reverberation time based on the scene type; Constructing a multi-channel feedback delay network, wherein the multi-channel feedback delay network is used to simulate natural reverberation; Based on the ideal reverberation time and the multi-channel feedback delay network, the reverberation time in the acoustic characteristic information is adjusted to obtain an adjusted reverberation time.

9. The audio control method according to claim 1, characterized in that: The step of adjusting the gains of multiple channels corresponding to the adjusted audio to obtain the further adjusted audio includes: Acquiring noise prediction information of the target space, wherein the noise prediction information is used to indicate noise levels in different time periods; Based on the noise prediction information, adjusting the compression ratio and compression threshold of the multi-channels corresponding to the regulated audio to obtain an adjusted multi-channel; Based on the adjusted multi-channel, masking-based gain control is performed on the regulated audio to obtain re-regulated audio.

10. The audio control method according to claim 9, characterized in that: The performing multi-channel collaborative optimization processing on the re-regulated audio to obtain audio with a target effect includes: Analyzing the channel cross-influence characteristics between the adjusted multiple channels; Performing phase compensation and amplitude compensation on the re-regulated audio based on the channel cross-influence characteristics to obtain a balanced sound effect; The equalized sound effect is processed based on a preset adaptive cross-cancellation algorithm to obtain audio with a target effect.

11. The audio control method according to any one of claims 1 to 10, characterized in that: Before performing the sound effect test on the target space based on the specified measurement signal to obtain the environmental response information, the method further includes: Identify the current audio usage scenario and automatically match the control strategy corresponding to the audio usage scenario, wherein the control strategy includes the content of acoustic characteristic information, compensation processing content, and compensation processing flow.

12. The audio control method according to any one of claims 1 to 10, characterized in that: After performing multi-channel collaborative optimization processing on the re-regulated audio to obtain audio with a target effect, the method further includes: Acquire analysis data, and perform mechanism update and / or strategy optimization of the audio control method based on the analysis data, wherein the analysis data includes performance test data and user evaluation data.

13. An audio control device, characterized in that: The audio control device comprises: An acquisition module, configured to perform a sound effect test on a target space based on a specified measurement signal to obtain environmental response information, wherein the environmental response information includes impulse response information and information after the impulse response is separated; An analysis module, used to analyze the environmental response information to obtain acoustic characteristic information of the target space; A compensation module, used for performing compensation processing on the audio to be processed based on the acoustic characteristic information to obtain regulated audio; A control module, used to control the gain of multiple channels corresponding to the regulated audio to obtain the further regulated audio; The optimization module is used to perform multi-channel collaborative optimization processing on the audio after the re-adjustment to obtain audio with a target effect.

14. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the audio control method according to any one of claims 1 to 12.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the audio control method according to any one of claims 1 to 12 is implemented.

Images