Real-time dynamic sound mixer parameter adjusting method, device, equipment and medium

Abstract

The invention relates to a real-time dynamic sound mixer parameter adjusting method and device, equipment and a medium. The method comprises the following steps: acquiring an input audio signal, output audio characteristic information and feedback audio data; the method comprises the following steps: performing time-frequency transformation on an input audio signal, extracting a multi-track phase feature and a multi-track amplitude feature, and generating a sound mixing initial parameter; generating a sound mixing target parameter based on the output audio characteristic information and the input audio signal; dynamically correcting the sound mixing target parameter according to the feedback audio data to obtain a dynamic sound mixing target parameter; and adjusting real-time dynamic sound mixer parameters by combining the sound mixing initial parameters and the dynamic sound mixing target parameters. By adopting the method, multi-parameter collaborative optimization can be realized by fusing the input audio signal, the output characteristics and the feedback data through feature extraction, target parameter generation and dynamic correction, so that the sound mixing output quality, the sound mixing complex environment adaptability and the personalized auditory experience can be remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of audio processing and speaker control, and particularly relates to a method, device, equipment and medium for adjusting real-time dynamic mixer parameters. Background Technique

[0002] With the continuous development of audio processing technology, the audio processing technology industry has experienced a transformation from analog signal processing to digitalization. Digital mixers have become the mainstream in the market, and their powerful processing capabilities and flexible controllability provide more creative space for music producers. The application of intelligent technology enables mixers to automatically analyze and process audio signals, improving the mixing efficiency and quality.

[0003] In traditional technologies, audio mixing usually involves a one-time parameter setting by professional sound engineers during the content creation stage. Professional sound engineers manually adjust the parameters of effectors such as equalizers, compressors, and reverbs according to the characteristics of the audio content and the characteristics of the target playback device to achieve the best sound quality effect. However, in the modern audio processing field, with the continuous improvement of users' requirements for audio quality and the diversification of audio output scenarios, traditional mixing methods are difficult to meet the complex and changing needs.

[0004] In the prior art with the authorization announcement number CN118945568B, by obtaining the audio signal and the user's mixing selection information, the overall mixing numerical information is determined; targeted sound generation parameters are generated for each region; the real-time sound field monitoring data of each partition is obtained, and the mixing parameters of each track are adaptively adjusted according to the optimization feedback. It can comprehensively improve the efficiency and quality of automatic tuning. However, this technical solution still has deficiencies in aspects such as complex sound field adaptability, hardware non-linear compensation, and personalized adaptation. Summary of the Invention

[0005] Based on this, it is necessary to provide a method, device, equipment and medium for adjusting real-time dynamic mixer parameters that can adapt to different playback conditions in real time, dynamically optimize mixing parameters, and meet the personalized needs of users, so as to achieve high-quality and immersive audio output and significantly improve the intelligence of audio processing.

[0006] In the first aspect, the present application provides a method for adjusting real-time dynamic mixer parameters, including:

[0007] Obtain the input audio signal, output audio characteristic information, and feedback audio data, where the output audio characteristic information is used to characterize the target effect of the preset output audio;

[0008] Perform time-frequency transformation on the input audio signal, extract multi-track phase characteristics and multi-track amplitude characteristics, and generate initial mixing parameters;

[0009] Generate mixing target parameters based on output audio characteristic information and input audio signals;

[0010] Dynamically correct the mixing target parameters according to the feedback audio data to obtain dynamic mixing target parameters;

[0011] Adjust the real-time dynamic mixer parameters by combining the initial mixing parameters and the dynamic mixing target parameters.

[0012] In one embodiment, the output audio characteristic information includes output audio mode information, hardware response characteristic information, and environmental response characteristic information. Generating mixing target parameters based on the output audio characteristic information and the input audio signals includes:

[0013] Perform output audio mode matching on the input audio signals, combine the input audio signals and the output audio mode information to generate initial parameters of the effectors, and obtain initial dynamic mixing target parameters;

[0014] Generate hardware compensation target parameters based on the hardware response characteristic information;

[0015] Generate environmental compensation target parameters based on the environmental response characteristic information;

[0016] Correct the initial dynamic mixing target parameters according to the hardware compensation target parameters and the environmental compensation parameters to obtain the mixing target parameters.

[0017] In one embodiment, the output audio mode information includes user personalized preference information and basic mode information. Performing output audio mode matching on the input audio signals, and combining the input audio signals and the output audio mode information to generate initial parameters of the effectors, and obtaining the initial dynamic mixing target parameters includes:

[0018] Input the user personalized preference information and the basic mode information into the audio mode generation model to generate comprehensive output audio mode information;

[0019] Perform output audio mode matching on the input audio signals and the comprehensive output audio mode information to identify effector selection information;

[0020] Generate initial parameters of the effectors based on the effector selection information by combining the input audio signals and the comprehensive output audio mode information;

[0021] Integrate the initial parameters of the effectors to generate initial dynamic mixing target parameters.

[0022] In one embodiment, dynamically correcting the mixing target parameters according to the feedback audio data to obtain the dynamic mixing target parameters includes:

[0023] Preprocess the feedback audio data to generate enhanced feedback audio data;

[0024] Perform audio mode analysis, hardware response analysis, and environmental response analysis on enhanced feedback audio data to generate performance metric error information and an error value corresponding to the performance metric error information. The performance metric error information includes mode error component information, hardware error component information, and environmental error component information;

[0025] If the error value is greater than a preset error threshold, identify the highest error component information in the performance metric error information. The highest error component information is the error component information with the largest error among the mode error component information, hardware error component information, and environmental error component information;

[0026] Dynamically correct the corresponding target parameter based on the highest error component information;

[0027] Among them, the target parameter corresponding to the mode error component information is the initial dynamic mixing target parameter, the target parameter corresponding to the hardware error component information is the hardware compensation target parameter, and the target parameter corresponding to the environmental error component information is the environmental compensation target parameter.

[0028] In one embodiment, dynamically correcting the mixing target parameter according to the feedback audio data to obtain the dynamic mixing target parameter further includes:

[0029] Input the feedback audio data into the psychoacoustic characteristic evaluation model to generate a psychoacoustic evaluation index value;

[0030] Compare and judge the psychoacoustic evaluation index value with a preset psychoacoustic index threshold to obtain a psychoacoustic evaluation result;

[0031] If the psychoacoustic evaluation result is that the psychoacoustic evaluation index value is lower than the psychoacoustic index threshold, match the psychoacoustic optimization format model based on the difference between the psychoacoustic evaluation index value and the psychoacoustic index threshold;

[0032] Correct the mixing target parameter based on the psychoacoustic optimization format model to obtain the dynamic mixing target parameter.

[0033] In one embodiment, performing time-frequency transformation on the input audio signal and extracting multi-track phase features and multi-track amplitude features to generate initial mixing parameters includes:

[0034] Preprocess the input audio signal to obtain the input audio short-time frame signal;

[0035] Perform short-time Fourier transform on the input audio short-time frame signal to construct an input audio time-frequency transformation model;

[0036] Perform multi-track phase consistency analysis and multi-track amplitude energy analysis on the input audio time-frequency transformation model, and extract multi-track phase features and multi-track amplitude features to generate multi-track phase feature information and multi-track amplitude feature information;

[0037] Generate the initial parameters of the input matching effector based on the multi-track phase feature information and the multi-track amplitude feature information, and generate the initial mixing parameters according to the initial parameters of the input matching effector.

[0038] In one embodiment, the method for adjusting the parameters of the real-time dynamic mixer further includes:

[0039] Optimize the output audio characteristic information based on the difference between the dynamic mixing target parameters and the mixing target parameters.

[0040] In a second aspect, the present application further provides a device for adjusting the parameters of a real-time dynamic mixer, including:

[0041] A data management module, configured to obtain an input audio signal, output audio characteristic information, and feedback audio data;

[0042] An initial mixing parameter generation module, configured to perform time-frequency transformation on the input audio signal, extract multi-track phase features and multi-track amplitude features, and generate initial mixing parameters;

[0043] A mixing target parameter generation module, configured to generate mixing target parameters based on the output audio characteristic information and the input audio signal;

[0044] A mixing target parameter correction module, configured to dynamically correct the mixing target parameters according to the feedback audio data to obtain dynamic mixing target parameters;

[0045] A dynamic mixing parameter generation module, configured to adjust the parameters of the real-time dynamic mixer by combining the initial mixing parameters and the dynamic mixing target parameters.

[0046] In a third aspect, the present application further provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the steps of the method according to any one of the first aspects of the present application are implemented.

[0047] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method according to any one of the first aspects of the present application are implemented.

[0048] The above method, device, computer device, and storage medium for adjusting the parameters of the real-time dynamic mixer can dynamically adjust the mixer parameters in real time by fusing the input audio, output device characteristics, and real-time feedback data, and then dynamically generate and correct the multi-track phase and amplitude parameters, so as to achieve environment-adaptive high-precision frequency response compensation and real-time closed-loop optimization, and significantly improve the sound quality consistency and sound field localization accuracy in complex scenarios. Description of the Drawings

[0049] To more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the accompanying drawings required for the description of the embodiments or related technologies. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0050] Figure 1 A flowchart of a method for adjusting real-time dynamic mixer parameters provided by an embodiment of the present application;

[0051] Figure 2 A flowchart of a method for generating mixed target parameters provided by an embodiment of the present application;

[0052] Figure 3 A flowchart of a method for generating initial dynamic mixed target parameters provided by an embodiment of the present application;

[0053] Figure 4 A flowchart of a method for generating dynamic mixed target parameters provided by an embodiment of the present application;

[0054] Figure 5 Another flowchart of a method for generating dynamic mixed target parameters provided by an embodiment of the present application;

[0055] Figure 6 A flowchart of a method for generating initial mixed parameters provided by an embodiment of the present application;

[0056] Figure 7 Another flowchart of a method for adjusting real-time dynamic mixer parameters provided by an embodiment of the present application;

[0057] Figure 8 A structural diagram of a device for adjusting real-time dynamic mixer parameters provided by an embodiment of the present application. Detailed implementation manners

[0058] In order to make the purpose, technical solutions and advantages of the present application clearer, the following further details the present application in combination with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0059] In one embodiment, as Figure 1As shown, a method for adjusting real-time dynamic mixer parameters is provided. In this embodiment, the method is exemplified by being applied to a mixer terminal. It can be understood that the method can also be applied to a mixer server, and can also be applied to a system including a mixer terminal and a mixer server, and is implemented through the interaction between the mixer terminal and the mixer server. Among them, the mixer terminal can be a mixer physical terminal or a mixer virtual terminal. In this embodiment, the method includes the following steps:

[0060] Step S101, obtain an input audio signal, output audio characteristic information, and feedback audio data.

[0061] Specifically, the mixer terminal can, in response to obtaining an audio playback instruction, obtain the input audio signal from a data storage device. The data storage device can be carried on the mixer terminal or can be an independent data storage device connected to the mixer terminal through a data communication channel. The mixer terminal can, based on the obtained audio playback instruction, combine the mixing processing effect analysis model to match the output audio characteristic information. The mixing processing effect analysis model can be a preset pattern matching algorithm, and the output audio characteristic information is used to characterize the target effect of the preset output audio. The mixer terminal can obtain the feedback audio data based on an audio feedback sensor, and the feedback audio data can include an audio feedback sensor spatial position tag.

[0062] Furthermore, the mixing processing effect analysis model can be, but is not limited to, an artificial intelligence model.

[0063] Step S102, perform time-frequency transformation on the input audio signal, extract multi-track phase features and multi-track amplitude features, and generate initial mixing parameters.

[0064] Optionally, the mixer terminal can, but is not limited to, perform time-frequency transformation on the input audio signal based on Fourier transform and wavelet transform to extract multi-track phase features and multi-track amplitude features.

[0065] Optionally, the mixer terminal can match the initial mixing parameters based on the extracted multi-track phase features and multi-track amplitude features according to a preset initial mixing parameter setting rule to generate initial mixing parameters. The initial mixing parameters can be used to characterize the recommended reference values of the mixing parameters determined by the time-frequency characteristics of the input audio signal.

[0066] Optionally, the initial mixing parameters can, but are not limited to, include track volume, equalizer (EQ) initial range parameters, filter selection parameters, dynamic range control initial range parameters, audio alignment parameters, and noise gate (Noise Gate) initial range parameters.

[0067] Step S103, generate target mixing parameters based on the output audio characteristic information and the input audio signal.

[0068] Optionally, the mixer terminal may generate initial static mixing target parameters of the mixer by combining the input audio signal and its time-frequency characteristics based on the output audio characteristic information used to characterize the target effect of the preset output audio. The mixer may perform mixing processing on the input audio signal based on the mixing target parameters to generate an output audio signal.

[0069] Optionally, the mixing target parameters may include, but are not limited to, equalizer target parameters, pan target parameters, filter setting target parameters, dynamic processor target parameters, reverb target parameters, delay target parameters, phaser target parameters, and spatial effects target parameters.

[0070] Step S104: Dynamically correct the mixing target parameters according to the feedback audio data to obtain dynamic mixing target parameters.

[0071] Optionally, the mixer terminal may perform time-frequency feature analysis on the feedback audio data acquired by the audio feedback sensor, and perform error threshold comparison analysis on the feature analysis result of the feedback audio data and the target effect characterized by the output audio characteristic information. When the error exceeds the preset threshold, the initial static mixing target parameters of the mixer are dynamically corrected based on the error to obtain dynamic mixing target parameters.

[0072] Step S105: Adjust the real-time dynamic mixer parameters by combining the initial mixing parameters and the dynamic mixing target parameters.

[0073] Optionally, the mixer terminal may, on the basis of the initial mixing parameters, superimpose the dynamic mixing target parameters according to the preset weighted linear adjustment rule to generate and adjust the real-time dynamic mixer parameters.

[0074] In the above real-time dynamic mixer parameter adjustment method, by performing time-frequency transformation on the input audio signal and extracting multi-track phase characteristics and multi-track amplitude characteristics, the spatial sense and stereo effect of the audio, as well as the loudness and energy distribution of the audio, can be accurately analyzed; by dynamically correcting the mixing target parameters according to the feedback audio data, the changes in the audio signal can be adapted in real time, avoiding the problems of mixing imbalance and interference caused by fixed parameter settings, thereby further optimizing the mixing quality and enabling the audio to maintain a good auditory effect in different scenarios; by automatically generating the initial mixing parameters, the efficiency of the mixing work can be improved; by adjusting the real-time dynamic mixer parameters by combining the initial mixing parameters and the dynamic mixing target parameters, the mixer can quickly adapt to different types, styles, and sources of audio signals, reducing the repeated debugging process caused by audio characteristic differences.

[0075] Schematically, the above real-time dynamic mixer parameter adjustment method, through feature extraction and dynamic correction, can promote the deep integration of audio processing technology with fields such as artificial intelligence and signal processing. Furthermore, it can provide new ideas and directions for the development of music production technology, promote the development of music production towards a more intelligent direction, and help improve the technical level and work efficiency of the entire music production industry.

[0076] In one optional embodiment, the output audio characteristic information includes output audio mode information, hardware response characteristic information, and environmental response characteristic information. As Figure 2 shown, generating the mixing target parameters based on the output audio characteristic information and the input audio signal includes:

[0077] Step S201, perform output audio mode matching on the input audio signal, combine the input audio signal and the output audio mode information to generate the initial parameters of the effect processor, and obtain the initial dynamic mixing target parameters.

[0078] Step S202, generate the hardware compensation target parameters based on the hardware response characteristic information.

[0079] Step S203, generate the environmental compensation target parameters based on the environmental response characteristic information.

[0080] Step S204, correct the initial dynamic mixing target parameters according to the hardware compensation target parameters and the environmental compensation parameters to obtain the mixing target parameters.

[0081] In the above real-time dynamic mixer parameter adjustment method, by subdividing the output audio characteristic information into output audio mode information, hardware response characteristic information, and environmental response characteristic information, and generating corresponding compensation parameters for these information respectively, the generation of the mixing target parameters can be made more accurate and comprehensive; by further correcting according to the hardware compensation target parameters and the environmental compensation target parameters on the basis of generating the initial dynamic mixing target parameters, the mixing parameters can be adjusted in real time, and thus the changes of the audio signal and the differences of different playback environments can be adapted, further optimizing the mixing effect and improving the flexibility and adaptability of the mixing method.

[0082] In one optional embodiment, the output audio mode information includes user personalized preference information and basic mode information. As Figure 3 shown, performing output audio mode matching on the input audio signal, combining the input audio signal and the output audio mode information to generate the initial parameters of the effect processor, and obtaining the initial dynamic mixing target parameters includes:

[0083] Step S301, input the user personalized preference information and the basic mode information into the audio mode generation model to generate the comprehensive output audio mode information.

[0084] In step S302, perform output audio mode matching on the input audio signal and the comprehensive output audio mode information to identify the effector selection information.

[0085] In step S303, generate initial effector parameters based on the effector selection information in combination with the input audio signal and the comprehensive output audio mode information.

[0086] In step S304, integrate the initial effector parameters to generate initial dynamic mixing target parameters.

[0087] In the above real-time dynamic mixer parameter adjustment method, by combining the user's personalized preference information with the basic mode information to generate the comprehensive output audio mode information, it is possible to fully consider the user's preference characteristics, summarize the user's usage habits, and enhance the user's sense of control and participation in audio processing; by inputting the user's personalized preference information and the basic mode information into the audio mode generation model to generate the comprehensive output audio mode information, and then matching the input audio signal and the comprehensive output audio mode information, it is possible to more accurately identify the effector selection information, and then be able to generate initial effector parameters that better meet the actual needs, thereby improving the accuracy and adaptability of mixing.

[0088] In one alternative embodiment, as Figure 4 shown, dynamically correct the mixing target parameters according to the feedback audio data to obtain the dynamic mixing target parameters, including:

[0089] In step S401, preprocess the feedback audio data to generate enhanced feedback audio data.

[0090] In step S402, perform audio mode analysis, hardware response analysis, and environmental response analysis on the enhanced feedback audio data to generate performance index error information and the error value corresponding to the performance index error information.

[0091] Specifically, the performance index error information may include mode error component information, hardware error component information, and environmental error component information.

[0092] In step S403, if the error value is greater than the preset error threshold, identify the highest error component information in the performance index error information.

[0093] Specifically, the highest error component information is the error component information with the largest error among the mode error component information, hardware error component information, and environmental error component information.

[0094] In step S404, dynamically correct the corresponding target parameters based on the highest error component information.

[0095] Specifically, the target parameter corresponding to the pattern error component information is the initial dynamic mixing target parameter, the target parameter corresponding to the hardware error component information is the hardware compensation target parameter, and the target parameter corresponding to the environmental error component information is the environmental compensation target parameter.

[0096] In the above real-time dynamic mixer parameter adjustment method, by intelligently identifying the highest error component information and dynamically correcting the corresponding target parameters according to different error component information, the most suitable adjustment strategy can be automatically selected according to different scenarios, improving the adjustment efficiency of the mixer parameters, enabling the mixing system to better adapt to various complex situations, and improving the flexibility and adaptability of mixing.

[0097] In one alternative embodiment, as Figure 5 shown, dynamically correcting the mixing target parameter according to the feedback audio data to obtain the dynamic mixing target parameter further includes:

[0098] Step S501: Input the feedback audio data into the psychoacoustic characteristic evaluation model to generate a psychoacoustic evaluation index value.

[0099] Step S502: Compare and judge the psychoacoustic evaluation index value with a preset psychoacoustic index threshold to obtain a psychoacoustic evaluation result.

[0100] Step S503: If the psychoacoustic evaluation result is that the psychoacoustic evaluation index value is lower than the psychoacoustic index threshold, match the psychoacoustic optimization format model based on the difference between the psychoacoustic evaluation index value and the psychoacoustic index threshold.

[0101] Step S504: Correct the mixing target parameter based on the psychoacoustic optimization format model to obtain the dynamic mixing target parameter.

[0102] In the above real-time dynamic mixer parameter adjustment method, by inputting the feedback audio data into the psychoacoustic characteristic evaluation model to generate a psychoacoustic evaluation index value, the audio quality can be evaluated from the perspective of auditory perception; through the psychoacoustic characteristic evaluation model and the psychoacoustic optimization format model, the psychoacoustic characteristics of the feedback audio data can be automatically learned and analyzed, and the mixing parameters can be dynamically adjusted according to the evaluation result, thereby ensuring that the mixing effect reaches the target state in auditory perception and providing users with a high-quality audio experience.

[0103] In one alternative embodiment, as Figure 6 shown, performing time-frequency transformation on the input audio signal, extracting multi-track phase characteristics and multi-track amplitude characteristics, and generating the initial mixing parameters includes:

[0104] Step S601: Preprocess the input audio signal to obtain the input audio short-time frame signal.

[0105] Step S602: Perform a short-time Fourier transform on the input audio short-time frame signal to construct an input audio time-frequency transformation model.

[0106] Illustratively, the short-time Fourier transform (STFT) is a signal processing technique used to analyze the frequency components of non-stationary signals as they change over time. It obtains the frequency information of the signal at different time points by performing a Fourier transform on the signal within each short-time window, and is suitable for analyzing non-stationary signals that change over time.

[0107] Step S603: Perform multi-track phase consistency analysis and multi-track amplitude energy analysis on the input audio time-frequency transformation model, extract multi-track phase features and multi-track amplitude features, and generate multi-track phase feature information and multi-track amplitude feature information.

[0108] Step S604: Generate initial parameters for the input matching effector based on the multi-track phase feature information and multi-track amplitude feature information, and generate initial mixing parameters according to the initial parameters of the input matching effector.

[0109] In one optional embodiment, the real-time dynamic mixer parameter adjustment method further includes:

[0110] Optimize the output audio characteristic information based on the difference between the dynamic mixing target parameters and the mixing target parameters.

[0111] In an exemplary embodiment of the present application, as Figure 7 shown, the real-time dynamic mixer parameter adjustment method may include the following steps:

[0112] Step S701: Obtain the input audio signal, output audio characteristic information, and feedback audio data.

[0113] Step S702: Perform time-frequency transformation on the input audio signal, extract multi-track phase features and multi-track amplitude features, and generate initial mixing parameters.

[0114] Step S703: Perform output audio mode matching on the input audio signal, combine the input audio signal and output audio mode information to generate initial parameters for the effector, and obtain the initial dynamic mixing target parameters.

[0115] Step S704: Generate hardware compensation target parameters based on the hardware response characteristic information.

[0116] Step S705: Generate environmental compensation target parameters based on the environmental response characteristic information.

[0117] Step S706: Correct the initial dynamic mixing target parameters according to the hardware compensation target parameters and the environmental compensation parameters to obtain the mixing target parameters.

[0118] Step S707: Preprocess the feedback audio data to generate enhanced feedback audio data.

[0119] Step S708: Perform audio mode analysis, hardware response analysis, and environmental response analysis on the enhanced feedback audio data to generate performance index error information and the error values corresponding to the performance index error information.

[0120] Step S709: If the error value is greater than the preset error threshold, identify the highest error component information in the performance index error information.

[0121] Step S710: Dynamically correct the corresponding target parameters based on the highest error component information to obtain the dynamic mixing target parameters.

[0122] Step S711: Adjust the real-time dynamic mixer parameters by combining the initial mixing parameters and the dynamic mixing target parameters.

[0123] In the above real-time dynamic mixer parameter adjustment method, through comprehensive audio feature extraction and multi-dimensional parameter generation and correction, it can ensure that the mixing effect adapts to different conditions and improve the mixing efficiency; through model-driven parameter adjustment and real-time optimization and dynamic adjustment, it can ensure the optimal state of the mixing parameters, enhance the intelligent level of mixing, and optimize the user experience; through user-friendly design and high-quality audio output, it can provide a stable and high-quality audio experience, thereby promoting the development of music production technology and driving the progress of intelligent mixing technology.

[0124] It should be understood that although the steps in the flowcharts involved in the above embodiments are shown in sequence according to the arrows, these steps do not necessarily need to be executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily need to be executed at the same moment, but can be executed at different moments. The execution order of these steps or stages does not necessarily need to be sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0125] Based on the same inventive concept, an embodiment of the present application further provides a real-time dynamic mixer parameter adjustment device for implementing the real-time dynamic mixer parameter adjustment method involved above. The solution provided by this device for solving problems is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the real-time dynamic mixer parameter adjustment device provided below can refer to the limitations on the real-time dynamic mixer parameter adjustment method in the above text, and will not be elaborated here.

[0126] In an exemplary embodiment, as Figure 8 shown, a real-time dynamic mixer parameter adjustment device 800 is provided, including:

[0127] A data management module 801, which can be used to obtain an input audio signal, output audio characteristic information, and feedback audio data.

[0128] A mixing initial parameter generation module 802, which can be used to perform time-frequency transformation on the input audio signal, extract multi-track phase characteristics and multi-track amplitude characteristics, and generate mixing initial parameters.

[0129] A mixing target parameter generation module 803, which can be used to generate mixing target parameters based on the output audio characteristic information and the input audio signal.

[0130] A mixing target parameter correction module 804, which can be used to dynamically correct the mixing target parameters according to the feedback audio data to obtain dynamic mixing target parameters.

[0131] A dynamic mixing parameter generation module 805, which can be used to adjust the real-time dynamic mixer parameters by combining the mixing initial parameters and the dynamic mixing target parameters.

[0132] In one optional embodiment, the mixing target parameter generation module 803 can also be used to:

[0133] Perform output audio mode matching on the input audio signal, combine the input audio signal and the output audio mode information to generate initial parameters of the effecter, and obtain initial dynamic mixing target parameters.

[0134] Generate hardware compensation target parameters based on the hardware response characteristic information.

[0135] Generate environment compensation target parameters based on the environment response characteristic information.

[0136] Correct the initial dynamic mixing target parameters according to the hardware compensation target parameters and the environment supplement parameters to obtain the mixing target parameters.

[0137] In one optional embodiment, the mixing target parameter generation module 803 can also be used to:

[0138] Input user personalized preference information and basic mode information into an audio mode generation model to generate comprehensive output audio mode information.

[0139] Perform output audio mode matching on the input audio signal and the comprehensive output audio mode information to identify effector selection information.

[0140] Based on the effector selection information, combine the input audio signal and the comprehensive output audio mode information to generate initial effector parameters.

[0141] Integrate the initial effector parameters to generate initial dynamic mixing target parameters.

[0142] In one optional embodiment, the mixing target parameter correction module 804 can also be used for:

[0143] Preprocess the feedback audio data to generate enhanced feedback audio data.

[0144] Perform audio mode analysis, hardware response analysis, and environmental response analysis on the enhanced feedback audio data to generate performance metric error information and the error values corresponding to the performance metric error information. The performance metric error information includes mode error component information, hardware error component information, and environmental error component information.

[0145] If the error value is greater than a preset error threshold, identify the highest error component information in the performance metric error information. The highest error component information is the error component information with the largest error among the mode error component information, hardware error component information, and environmental error component information.

[0146] Dynamically correct the corresponding target parameters based on the highest error component information.

[0147] In one optional embodiment, the mixing target parameter correction module 804 can also be used for:

[0148] Input the feedback audio data into a psychoacoustic characteristic evaluation model to generate psychoacoustic evaluation metric values.

[0149] Compare and judge the psychoacoustic evaluation metric values with a preset psychoacoustic metric threshold to obtain a psychoacoustic evaluation result.

[0150] If the psychoacoustic evaluation result is that the psychoacoustic evaluation metric value is lower than the psychoacoustic metric threshold, match a psychoacoustic optimization format model based on the difference between the psychoacoustic evaluation metric value and the psychoacoustic metric threshold.

[0151] Correct the mixing target parameters based on the psychoacoustic optimization format model to obtain dynamic mixing target parameters.

[0152] In one optional embodiment, the mixing initial parameter generation module 802 can also be used for:

[0153] Preprocess the input audio signal to obtain the short-time frame signal of the input audio.

[0154] Perform a short-time Fourier transform on the short-time frame signal of the input audio to construct the time-frequency transformation model of the input audio.

[0155] Perform multi-track phase consistency analysis and multi-track amplitude energy analysis on the time-frequency transformation model of the input audio, extract multi-track phase features and multi-track amplitude features, and generate multi-track phase feature information and multi-track amplitude feature information.

[0156] Generate the initial parameters of the input matching effector based on the multi-track phase feature information and multi-track amplitude feature information, and generate the initial parameters of the mixing based on the initial parameters of the input matching effector.

[0157] In one alternative embodiment, the real-time dynamic mixer parameter adjustment device 800 can also be used for:

[0158] Optimize the output audio characteristic information based on the difference between the dynamic mixing target parameters and the mixing target parameters.

[0159] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of a real-time dynamic mixer parameter adjustment method as described above are implemented.

[0160] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0161] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial descriptions of the method embodiments. The device embodiments described above are only illustrative. The components described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present disclosure solution. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0162] The above embodiments only represent several implementation manners of the embodiments of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the embodiments of the application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the embodiments of the present application, several deformations and improvements can still be made, and these all belong to the protection scope of the embodiments of the present application.

Claims

1. A method for adjusting real-time dynamic mixer parameters, characterized in that, The method includes: Obtaining an input audio signal, output audio characteristic information, and feedback audio data, where the output audio characteristic information is used to characterize the target effect of a preset output audio; Performing time-frequency transformation on the input audio signal, extracting multi-track phase features and multi-track amplitude features, and generating initial mixing parameters; Generating target mixing parameters based on the output audio characteristic information and the input audio signal; Dynamically correcting the target mixing parameters according to the feedback audio data to obtain dynamic target mixing parameters; Adjusting the real-time dynamic mixer parameters by combining the initial mixing parameters and the dynamic target mixing parameters.

2. The method according to claim 1, characterized in that, The output audio characteristic information includes output audio mode information, hardware response characteristic information, and environmental response characteristic information. The generating of the target mixing parameters based on the output audio characteristic information and the input audio signal includes: Performing output audio mode matching on the input audio signal, combining the input audio signal and the output audio mode information to generate initial parameters of an effector, and obtaining initial dynamic target mixing parameters; Generating hardware compensation target parameters based on the hardware response characteristic information; Generating environmental compensation target parameters based on the environmental response characteristic information; Correcting the initial dynamic target mixing parameters according to the hardware compensation target parameters and the environmental compensation parameters to obtain the target mixing parameters.

3. The method according to claim 2, characterized in that The output audio mode information includes user personalized preference information and basic mode information. The performing of output audio mode matching on the input audio signal, combining the input audio signal and the output audio mode information to generate initial parameters of an effector, and obtaining initial dynamic target mixing parameters includes: Inputting the user personalized preference information and the basic mode information into an audio mode generation model to generate comprehensive output audio mode information; Performing output audio mode matching on the input audio signal and the comprehensive output audio mode information to identify effector selection information; Generating the initial parameters of the effector based on the effector selection information, the input audio signal, and the comprehensive output audio mode information; Integrating the initial parameters of the effector to generate the initial dynamic target mixing parameters.

4. The method according to claim 2, wherein The dynamically correcting the target mixing parameters according to the feedback audio data to obtain dynamic target mixing parameters includes: Performing preprocessing on the feedback audio data to generate enhanced feedback audio data; Performing audio mode analysis, hardware response analysis, and environmental response analysis on the enhanced feedback audio data to generate performance index error information and an error value corresponding to the performance index error information. The performance index error information includes mode error component information, hardware error component information, and environmental error component information; If the error value is greater than a preset error threshold, identifying the highest error component information in the performance index error information, where the highest error component information is the error component information with the largest error among the mode error component information, the hardware error component information, and the environmental error component information; Dynamically correcting the corresponding target parameters based on the highest error component information. Among them, the target parameter corresponding to the pattern error component information is the initial dynamic mixing target parameter, the target parameter corresponding to the hardware error component information is the hardware compensation target parameter, and the target parameter corresponding to the environmental error component information is the environmental compensation target parameter.

5. The method according to claim 4, characterized in that, The dynamically correcting the mixing target parameter according to the feedback audio data to obtain a dynamic mixing target parameter further includes: Inputting the feedback audio data into a psychoacoustic characteristic evaluation model to generate a psychoacoustic evaluation index value; Comparing and judging the psychoacoustic evaluation index value with a preset psychoacoustic index threshold to obtain a psychoacoustic evaluation result; If the psychoacoustic evaluation result is that the psychoacoustic evaluation index value is lower than the psychoacoustic index threshold, matching a psychoacoustic optimization format model based on the difference between the psychoacoustic evaluation index value and the psychoacoustic index threshold; Correcting the mixing target parameter based on the psychoacoustic optimization format model to obtain the dynamic mixing target parameter.

6. The method according to claim 1, characterized in that, The performing time-frequency transformation on the input audio signal, extracting multi-track phase features and multi-track amplitude features, and generating an initial mixing parameter includes: Preprocessing the input audio signal to obtain an input audio short-time frame signal; Performing a short-time Fourier transform on the input audio short-time frame signal to construct an input audio time-frequency transformation model; Performing multi-track phase consistency analysis and multi-track amplitude energy analysis on the input audio time-frequency transformation model, extracting multi-track phase features and multi-track amplitude features, and generating multi-track phase feature information and multi-track amplitude feature information; Generating an initial parameter of an input matching effector based on the multi-track phase feature information and the multi-track amplitude feature information, and generating the initial mixing parameter according to the input to match the initial parameter of the input matching effector.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Optimizing the output audio characteristic information based on the difference between the dynamic mixing target parameter and the mixing target parameter.

8. A real-time dynamic mixer parameter adjustment device, characterized in that The device includes: A data management module, configured to obtain an input audio signal, output audio characteristic information, and feedback audio data; An initial mixing parameter generation module, configured to perform time-frequency transformation on the input audio signal, extract multi-track phase features and multi-track amplitude features, and generate an initial mixing parameter; A mixing target parameter generation module, configured to generate a mixing target parameter based on the output audio characteristic information and the input audio signal; A mixing target parameter correction module, configured to dynamically correct the mixing target parameter according to the feedback audio data to obtain a dynamic mixing target parameter; A dynamic mixing parameter generation module, configured to adjust the real-time dynamic mixer parameter by combining the initial mixing parameter and the dynamic mixing target parameter.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.

Images