Sound effect parameter adjustment model training method and related device
By training a sound effect parameter adjustment model and using a neural network model to automatically adjust the sound effect processor parameters, the problem of differences in processing results between different sound effect processors is solved, and efficient and consistent adjustment of sound effect parameters is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING DAJIA INTERNET INFORMATION TECH CO LTD
- Filing Date
- 2023-04-11
- Publication Date
- 2026-07-24
AI Technical Summary
Differences in processing results between different audio processors lead to inconsistent audio experiences for singers and audiences, and existing technologies make manual adjustments inefficient.
By training a sound effect parameter adjustment model, a neural network model is used to automatically adjust the parameters to make them consistent based on the similarity of the processing results of different sound effect processors.
It achieves consistency in output results across different audio processors and improves the efficiency of adjusting audio parameters.
Smart Images
Figure CN116564252B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computer technology, and more specifically, to a training method for a sound effect parameter adjustment model, a sound effect parameter adjustment method, a training device for a sound effect parameter adjustment model, a sound effect parameter adjustment device, an electronic device, and a computer-readable storage medium. Background Technology
[0002] With the rapid development of audio processing technology, audio post-processing has been widely used in the production of entertainment works such as songs, movies, variety shows, and short videos.
[0003] Because audio processors come in various implementations, in some scenarios, it's necessary to align the processing results of different processors to ensure consistent output. For example, in an online virtual karaoke room, a singer might choose to process their voice using a specific audio processor before sending it to the audience, while simultaneously monitoring the processed audio through in-ear monitors. In this case, the audio processors used by the in-ear monitors and the audience often differ. Without processor audio adaptation, there will be a significant difference between the audio the singer is listening to and the audio received by the audience, hindering a consistent perception of the performance between the singer and the audience.
[0004] In related technologies, the parameters of one sound processor are usually adjusted manually to make its effect similar to that of another processor, but this method is inefficient. Summary of the Invention
[0005] This disclosure provides a training method for a sound effect parameter adjustment model, a sound effect parameter adjustment method, a training device for a sound effect parameter adjustment model, a sound effect parameter adjustment device, an electronic device, and a computer-readable storage medium. This method can make the processing results of different sound effect processors for the same audio basically consistent, realize adaptive adaptation of the output results of different sound effect processors, and improve the efficiency of sound effect parameter adjustment.
[0006] This disclosure provides a training method for a sound effect parameter adjustment model. The method includes: acquiring sample audio and first sound effect parameters; inputting the first sound effect parameters into a sound effect parameter adjustment model to be trained to obtain second sound effect parameters; inputting the sample audio into a first sound effect processor configured with the first sound effect parameters to obtain a first processing result of the sample audio; and inputting the sample audio into a second sound effect processor configured with the second sound effect processing parameters to obtain a second processing result of the sample audio; and training the sound effect parameter adjustment model to be trained based on the approximation between the first processing result and the second processing result to obtain a trained sound effect parameter adjustment model.
[0007] In some exemplary embodiments of this disclosure, training the sound effect parameter adjustment model to be trained based on the degree of approximation between the first processing result and the second processing result to obtain a trained sound effect parameter adjustment model includes: determining the degree of approximation between the first processing result and the second processing result; if the degree of approximation is greater than a preset threshold, adjusting the model parameters of the sound effect parameter adjustment model to be trained until the degree of approximation between the adjusted second processing result obtained based on the adjusted sound effect parameter adjustment model and the first sound effect processing result is less than or equal to the preset threshold, and determining the adjusted initial sound effect parameter adjustment model as the trained sound effect parameter adjustment model.
[0008] In some exemplary embodiments of this disclosure, when the approximation degree is greater than a preset threshold, the model parameters of the sound effect parameter adjustment model to be trained are adjusted until the approximation degree between the adjusted second processing result obtained according to the adjusted sound effect parameter adjustment model and the first sound effect processing result is less than or equal to the preset threshold, and the adjusted initial sound effect parameter adjustment model is determined as the trained sound effect parameter adjustment model. This includes: when the approximation degree is greater than the preset threshold, adjusting the model parameters of the sound effect parameter adjustment model to be trained; inputting the first sound effect parameters into the adjusted sound effect parameter adjustment model to obtain the adjusted second sound effect parameters; inputting the sample audio into a second sound effect processor configured with the adjusted second sound effect parameters to obtain the adjusted second processing result; and when the approximation degree between the first processing result and the adjusted second processing result is less than or equal to the preset threshold, determining the adjusted sound effect parameter adjustment model as the trained sound effect parameter adjustment model.
[0009] In some exemplary embodiments of this disclosure, obtaining sample audio includes: obtaining sample singing data and sample speech data; adjusting the reverberation parameter value and equalization parameter value of the sample singing data, and adjusting the reverberation parameter value and equalization parameter value of the sample speech data; and generating the sample audio based on the sample singing data, the sample speech data, the adjusted sample singing data, and the adjusted sample speech data.
[0010] In some exemplary embodiments of this disclosure, the degree of approximation includes at least one of waveform distance, spectral distance, and signal-to-noise ratio between the first processing result and the second processing result.
[0011] This disclosure provides a method for adjusting sound effect parameters, including: acquiring a first sound effect parameter of an audio to be processed and a first sound effect processor; inputting the first sound effect parameter into a trained sound effect parameter adjustment model to obtain a second sound effect parameter; wherein the trained sound effect parameter adjustment model is trained according to any of the above methods; and inputting the audio to be processed into a second sound effect processor configured with the second sound effect parameter to obtain a sound effect processing result of the audio to be processed.
[0012] This disclosure provides a training apparatus for a sound effect parameter adjustment model, comprising: an acquisition module configured to acquire sample audio and a first sound effect parameter; an acquisition module configured to input the first sound effect parameter into a sound effect parameter adjustment model to be trained to obtain a second sound effect parameter; the acquisition module is further configured to input the sample audio into a first sound effect processor configured with the first sound effect parameter to obtain a first processing result of the sample audio; and input the sample audio into a second sound effect processor configured with the second sound effect parameter to obtain a second processing result of the sample audio; and a training module configured to train the sound effect parameter adjustment model to be trained based on the approximation between the first processing result and the second processing result to obtain a trained sound effect parameter adjustment model.
[0013] This disclosure provides a sound effect parameter adjustment device, comprising: an acquisition module configured to acquire a first sound effect parameter of an audio to be processed and a first sound effect processor; and an obtaining module configured to input the first sound effect parameter into a trained sound effect parameter adjustment model to obtain a second sound effect parameter; wherein the trained sound effect parameter adjustment model is trained according to any of the above methods; the obtaining module is further configured to input the audio to be processed into a second sound effect processor configured with the second sound effect parameter to obtain a sound effect processing result of the audio to be processed.
[0014] This disclosure provides an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to implement a training method for a sound effect parameter adjustment model as described above, or a sound effect parameter adjustment method as described above.
[0015] This disclosure provides a computer-readable storage medium that, when executed by a processor of an electronic device, enables the electronic device to perform a training method for a sound effect parameter adjustment model as described above, or a sound effect parameter adjustment method as described above.
[0016] This disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements a training method for a sound effect parameter adjustment model as described above, or a sound effect parameter adjustment method as described above.
[0017] The method for training a sound effect parameter adjustment model provided in this embodiment involves inputting a first sound effect parameter into the sound effect parameter adjustment model to be trained to obtain a second sound effect parameter; inputting sample audio into a first sound effect processor configured with the first sound effect parameter and a second sound effect processor configured with the second sound effect parameter to obtain a first processing result and a second processing result of the sample audio; training the sound effect parameter adjustment model to be trained based on the approximation between the first processing result and the second processing result; and thereby automatically outputting the second sound effect parameter of the second sound effect processor according to the first sound effect parameter of the first sound effect processor, so that the processing results of different sound effect processors on the same audio are basically consistent, realizing adaptive adaptation of the output results of different sound effect processors, and improving the efficiency of sound effect parameter adjustment.
[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.
[0020] Figure 1 A schematic diagram of an exemplary system architecture for which the training method or the sound effect parameter adjustment method of the sound effect parameter adjustment model can be applied according to the embodiments of this disclosure is shown.
[0021] Figure 2 This is a flowchart illustrating a training method for a sound effect parameter adjustment model according to an exemplary embodiment.
[0022] Figure 3 This is a schematic diagram illustrating the training and application process of a sound effect parameter adjustment model according to an exemplary embodiment.
[0023] Figure 4 This is a flowchart illustrating another training method for a sound effect parameter adjustment model according to an exemplary embodiment.
[0024] Figure 5 This is a flowchart illustrating a method for adjusting sound effect parameters according to an exemplary embodiment.
[0025] Figure 6 This is a block diagram illustrating a training apparatus for a sound effect parameter adjustment model according to an exemplary embodiment.
[0026] Figure 7 This is a block diagram illustrating a sound effect parameter adjustment device according to an exemplary embodiment.
[0027] Figure 8 This is a schematic diagram illustrating the structure of an electronic device suitable for implementing exemplary embodiments of the present disclosure, according to an exemplary embodiment. Detailed Implementation
[0028] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.
[0029] The features, structures, or characteristics described in this disclosure can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more specific details omitted, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.
[0030] The accompanying drawings are merely illustrative of this disclosure, and the same reference numerals in the drawings denote the same or similar parts, thus omitting repeated descriptions of them. Some block diagrams shown in the drawings do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in at least one hardware module or integrated circuit, or in different network and / or processor devices and / or microcontroller devices.
[0031] The flowchart shown in the accompanying drawings is merely illustrative and does not necessarily include all content and steps, nor does it require execution in the described order. For example, some steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.
[0032] In this specification, the terms “a,” “an,” “the,” “the,” and “at least one” are used to indicate the presence of at least one element / component / etc.; the terms “comprising,” “including,” and “having” are used to indicate an open-ended inclusion and to mean that there may be other elements / components / etc. in addition to the listed elements / components / etc.; the terms “first,” “second,” and “third,” etc., are used only as markings and are not a limitation on the number of objects.
[0033] Figure 1 A schematic diagram of an exemplary system architecture for which the training method or the sound effect parameter adjustment method of the sound effect parameter adjustment model can be applied according to the embodiments of this disclosure is shown.
[0034] like Figure 1 As shown, the system architecture may include server 101, network 102, terminal device 103, terminal device 104, and terminal device 105. Network 102 serves as the medium for providing a communication link between terminal device 103, terminal device 104, or terminal device 105 and server 101. Network 102 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.
[0035] Server 101 can be a server that provides various services, such as a back-end management server that supports the devices operated by users using terminal devices 103, 104, or 105. The back-end management server can analyze and process received requests and other data, and feed back the processing results to terminal devices 103, 104, or 105.
[0036] Terminal devices 103, 104, and 105 can be smartphones, tablets, laptops, desktop computers, smart speakers, wearable smart devices, virtual reality devices, augmented reality devices, etc., but are not limited to these.
[0037] Terminal devices 103, 104, and 105 can be configured with different sound effect processors. The following description takes terminal device 103 as having a first sound effect processor and terminal device 104 as having a second sound effect processor as an example. The first sound effect processor and the second sound effect processor have different sound effect processing parameters, but this disclosure is not limited to this.
[0038] The sound effect parameter adjustment method provided in this disclosure can be applied to online virtual karaoke scenarios. In online virtual karaoke scenarios, the sound effect processors used by singers and audiences may differ. For example, the sound effect processor used by the singer can be used as the first sound effect processor, and the sound effect processor used by the audience can be used as the second sound effect processor. Through the sound effect parameter adjustment method provided in this disclosure, the processing results of the first sound effect processor and the processing results of the second sound effect processor can be automatically aligned.
[0039] In this embodiment of the disclosure, server 101 can: acquire sample audio and first sound effect parameters; input the first sound effect parameters into a sound effect parameter adjustment model to be trained to obtain second sound effect parameters; input the sample audio into a first sound effect processor configured with the first sound effect parameters to obtain a first processing result of the sample audio; and input the sample audio into a second sound effect processor configured with the second sound effect processing parameters to obtain a second processing result of the sample audio; and train the sound effect parameter adjustment model to be trained based on the approximation between the first processing result and the second processing result to obtain a trained sound effect parameter adjustment model.
[0040] In this embodiment of the present disclosure, server 101 can obtain the audio to be processed and the first sound effect parameters of the first sound effect processor from terminal device 103; input the first sound effect parameters into the sound effect parameter adjustment model trained by the above method to obtain the second sound effect parameters; use the second sound effect parameters to configure the second sound effect processor of terminal device 104, input the audio to be processed into the second sound effect processor, and obtain the sound effect processing result of the audio to be processed.
[0041] It should be understood that Figure 1 The number of terminal devices 103, 104, 105, network 102, and server 101 in the diagram is merely illustrative. Server 101 can be a single physical server, a server cluster consisting of multiple servers, or a cloud server. Depending on actual needs, it can have any number of terminal devices, networks, and servers.
[0042] The training steps of the sound effect parameter adjustment model in the exemplary embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings and embodiments. The method provided in the embodiments of this disclosure can be executed by any electronic device, such as the one described above. Figure 1 The server and / or terminal equipment in the process, but this disclosure does not limit this.
[0043] Figure 2 This is a flowchart illustrating a training method for a sound effect parameter adjustment model according to an exemplary embodiment.
[0044] like Figure 2As shown, the method provided in this disclosure embodiment may include the following steps.
[0045] In step S210, the sample audio and the first sound effect parameters are obtained.
[0046] In this embodiment of the disclosure, the first sound effect parameter refers to the sound effect processing parameter of the first sound effect processor, and the second sound effect parameter refers to the sound effect processing parameter of the second sound effect processor. The first sound effect processor and the second sound effect processor are different sound effect processors. Different sound effect processors have different numbers of parameters, parameter value ranges, parameter meanings, etc. The configuration of the parameters determines the effect of the sound effect processor on the sound effect processing. The first sound effect processor and the second sound effect processor can process the sample audio separately to obtain the sound effect processing result of the sample audio.
[0047] In this embodiment of the disclosure, the first sound effect processor can be used as the target sound effect processor, and the second sound effect processor can be used as the fitting sound effect processor. The training task of the sound effect parameter adjustment model is to use a neural network model to fit the output sound effect of the second sound effect processor to the output sound effect of the first sound effect processor.
[0048] The sound effect parameters in this embodiment may include, but are not limited to: parameters of different frequency bands of the equalizer, reverberation time of the reverberation effect, wet / dry ratio, distortion, room size, early reverberation time, reverberation bandwidth, etc.
[0049] In this embodiment of the disclosure, the first sound effect parameter may include manually adjusted parameters, randomly fine-tuned parameters, and completely random parameters. The manually adjusted parameters refer to the adjustment of parameters by the sound engineer or experienced audio staff based on the first sound effect processor to achieve some high-quality sound effects under the sound effect processor. The randomly fine-tuned parameters are some details that are randomly adjusted on the basis of the manually adjusted parameters to increase the diversity of data, prevent the neural network from overfitting, and enhance its generalization ability. The completely random parameters refer to data augmentation that does not rely on human intervention at all, in order to increase the diversity of data.
[0050] In this embodiment of the disclosure, the proportion of manually adjusted parameters, randomly fine-tuned parameters, and completely random parameters to the first sound effect parameters can be set according to the actual situation. For example, manually adjusted parameters account for 10%, randomly fine-tuned parameters account for 80%, and completely random parameters account for 10%. The number of manually adjusted parameters, randomly fine-tuned parameters, and completely random parameters can be set according to the actual situation. For example, the total number of manually adjusted parameters, randomly fine-tuned parameters, and completely random parameters is more than 1,000 sets.
[0051] In this embodiment of the disclosure, the sample audio may include sample singing data and sample speech data. The proportion of sample singing data and sample speech data in the sample audio can be set according to the actual situation, for example, sample singing data accounts for 50% and sample speech data accounts for 50%. The amount of sample singing data and sample speech data can be set according to the actual situation, for example, the total amount of sample singing data and sample speech data is more than 100 hours.
[0052] In an exemplary embodiment, acquiring sample audio may include: acquiring sample singing data and sample speech data; adjusting the reverberation parameter value and equalization parameter value of the sample singing data, and adjusting the reverberation parameter value and equalization parameter value of the sample speech data; and generating sample audio based on the sample singing data, sample speech data, adjusted sample singing data, and adjusted sample speech data.
[0053] In this embodiment, sample singing data and sample speech data can be augmented to add reverberation and equalization to simulate data collected using different acquisition devices in different environments. Specifically, the reverberation and equalization parameters of the sample singing data can be adjusted to obtain adjusted sample singing data; the reverberation and equalization parameters of the sample speech data can be adjusted to obtain adjusted sample speech data; the sample singing data, sample speech data, adjusted sample singing data, and adjusted sample speech data are used as the aforementioned sample audio and input to the first sound effect processor and the second sound effect processor, respectively.
[0054] In step S220, the first sound effect parameters are input into the sound effect parameter adjustment model to be trained to obtain the second sound effect parameters.
[0055] refer to Figure 3 The first sound effect parameter is represented by parameter A, the second sound effect parameter is represented by parameter B, the first sound effect processor is represented by sound effect processor A, the second sound effect processor is represented by sound effect processor B, the first processing result is represented by output audio A, and the second processing result is represented by output audio B. The sound effect parameter adjustment model to be trained can adopt any neural network model, such as CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), or a combination of multiple structures. This disclosure does not limit this.
[0056] Continue to refer to Figure 3The solid line represents the part that is needed in both the training and application phases of the sound effect parameter adjustment model, while the dashed line represents the part that is needed only in the training phase. Input parameter A into the neural network model and output parameter B. Use parameter A to configure sound effect processor A and use parameter B to configure audio processor B.
[0057] In step S230, the sample audio is input to a first sound effect processor configured with a first sound effect parameter to obtain a first processing result of the sample audio; and the sample audio is input to a second sound effect processor configured with a second sound effect parameter to obtain a second processing result of the sample audio.
[0058] Continue to refer to Figure 3 The sample audio is input to the sound effect processor A configured with parameter A to obtain the output audio A; the sample audio is input to the sound effect processor B configured with parameter B to obtain the output audio B.
[0059] In step S240, based on the degree of approximation between the first processing result and the second processing result, the sound effect parameter adjustment model to be trained is trained to obtain the trained sound effect parameter adjustment model.
[0060] In this embodiment of the disclosure, the model parameters of the sound effect parameter adjustment model to be trained are adjusted according to the degree of approximation between the first processing result and the second processing result, so that the degree of approximation between the first processing result and the second processing result is less than or equal to a preset threshold, thereby obtaining the trained sound effect parameter adjustment model.
[0061] In an exemplary embodiment, the degree of approximation includes at least one of waveform distance, spectral distance, and signal-to-noise ratio between the first processing result and the second processing result.
[0062] refer to Figure 3 The sample audio is input into sound effect processors A and B, which are configured with a pair of parameters A and B respectively, and processed to obtain a pair of processing results, "output sound effect A and output sound effect B". The loss function of output sound effect A and output sound effect B is calculated. The loss function can be defined as a quantity describing the degree of approximation between output sound effect A and output sound effect B, such as time-domain waveform distance, spectral distance, signal-to-noise ratio. The loss function and gradient descent method are used to backpropagate the neural network model, and finally the optimization goal of gradually approximating the output sound effect A and output sound effect B of sound effect processors A and B is achieved.
[0063] In an exemplary embodiment, training a sound effect parameter adjustment model to be trained based on the degree of approximation between the first processing result and the second processing result to obtain a trained sound effect parameter adjustment model includes: determining the degree of approximation between the first processing result and the second processing result; if the degree of approximation is greater than a preset threshold, adjusting the model parameters of the sound effect parameter adjustment model to be trained until the degree of approximation between the adjusted second processing result and the first sound effect processing result obtained based on the adjusted sound effect parameter adjustment model is less than or equal to the preset threshold, and determining the adjusted initial sound effect parameter adjustment model as the trained sound effect parameter adjustment model.
[0064] Continue to refer to Figure 3 The approximation between output audio A and output audio B can be calculated using a loss function. If the approximation is greater than a preset threshold, the model parameters of the neural network model to be trained are adjusted. Parameter A is input into the adjusted neural network model to obtain the adjusted parameters (e.g., represented by parameter B'). The sound effect processor B is configured using the adjusted parameter B', and the sample audio is input into the sound effect processor B to obtain the adjusted output audio (e.g., represented by output audio B'). The approximation between output audio A and adjusted output audio B' is calculated again using the loss function. If the approximation is still greater than a preset threshold, the model parameters of the neural network model are adjusted using the above method. If the approximation is less than or equal to the preset threshold, the adjusted neural network model is determined as the sound effect parameter adjustment model.
[0065] The method for training a sound effect parameter adjustment model provided in this embodiment involves inputting a first sound effect parameter into the sound effect parameter adjustment model to be trained to obtain a second sound effect parameter; inputting sample audio into a first sound effect processor configured with the first sound effect parameter and a second sound effect processor configured with the second sound effect parameter to obtain a first processing result and a second processing result of the sample audio; training the sound effect parameter adjustment model to be trained based on the approximation between the first processing result and the second processing result; and thereby automatically outputting the second sound effect parameter of the second sound effect processor according to the first sound effect parameter of the first sound effect processor, so that the processing results of different sound effect processors on the same audio are basically consistent, realizing adaptive adaptation of the output results of different sound effect processors, and improving the efficiency of sound effect parameter adjustment.
[0066] Figure 4 This is a flowchart illustrating another training method for a sound effect parameter adjustment model according to an exemplary embodiment. Figure 4The document describes the following steps: "When the approximation is greater than a preset threshold, adjust the model parameters of the sound effect parameter adjustment model to be trained until the approximation between the adjusted second processing result and the first sound effect processing result obtained from the adjusted sound effect parameter adjustment model is less than or equal to the preset threshold, and determine the adjusted initial sound effect parameter adjustment model as the trained sound effect parameter adjustment model."
[0067] In step S410, if the approximation is greater than a preset threshold, the model parameters of the sound effect parameters to be trained are adjusted to adjust the model parameters of the model.
[0068] In this embodiment of the disclosure, the preset threshold can be set according to the actual situation, and this disclosure does not limit it.
[0069] In step S420, the first sound effect parameter is input into the adjusted sound effect parameter adjustment model to obtain the adjusted second sound effect parameter.
[0070] In this embodiment of the disclosure, the second sound processor is reconfigured using the adjusted second sound effect parameters.
[0071] In step S430, the sample audio is input to a second sound effect processor configured with adjusted second sound effect parameters to obtain the adjusted second processing result.
[0072] In this embodiment of the disclosure, the sample audio is re-inputted to the reconfigured second sound effect processor to obtain the adjusted second processing result.
[0073] In step S440, if the approximation between the first processing result and the adjusted second processing result is less than or equal to a preset threshold, the adjusted sound effect parameter adjustment model is determined as the trained sound effect parameter adjustment model.
[0074] In this embodiment of the disclosure, the relationship between the approximation degree between the first processing result and the adjusted second processing result and the preset threshold is further determined. If the approximation degree is still greater than the preset threshold, steps S410 to S430 are repeated. If the approximation degree is less than or equal to the preset threshold, it indicates that the sound effect parameter adjustment model has been trained and the adjusted sound effect parameter adjustment model is determined as the trained sound effect parameter adjustment model.
[0075] The method for training a sound effect parameter adjustment model provided in this embodiment adjusts the model parameters of the sound effect parameter adjustment model to be trained when the approximation between the first processing result and the second processing result is greater than a preset threshold; the first sound effect parameters are input into the adjusted sound effect parameter adjustment model to obtain the adjusted second sound effect parameters; sample audio is input into a second sound effect processor configured with the adjusted second sound effect parameters to obtain the adjusted second processing result; the relationship between the approximation between the first processing result and the adjusted second processing result and the preset threshold is further determined; if the approximation is still greater than the preset threshold, the model parameters are further adjusted; if the approximation is less than or equal to the preset threshold, it is indicated that the sound effect parameter adjustment model has been trained successfully, and the adjusted sound effect parameter adjustment model is determined as the trained sound effect parameter adjustment model; this method can improve the accuracy of sound effect parameter adjustment model training.
[0076] Figure 5 This is a flowchart illustrating a method for adjusting sound effect parameters according to an exemplary embodiment. Figure 5 The application process of the sound effect parameter adjustment model after training it using the method provided in the above embodiments is shown.
[0077] In step S510, the audio to be processed and the first sound effect parameters of the first sound effect processor are obtained.
[0078] In step S520, the first sound effect parameter is input into the trained sound effect parameter adjustment model to obtain the second sound effect parameter; wherein the trained sound effect parameter adjustment model is obtained by training according to the above embodiment.
[0079] refer to Figure 3 In the model application phase, parameter A is input into the trained neural network model, and parameter B is output; parameter B is used to configure the sound effect processor B.
[0080] In step S530, the audio to be processed is input to a second audio processor configured with second audio parameters to obtain the audio processing result.
[0081] refer to Figure 3 In the model application stage, the audio to be processed is input into the audio effect processor B configured with parameter B to obtain the audio effect processing result of the audio to be processed, which is similar to the audio effect processing result of the audio to be processed is obtained by inputting the audio to be processed into the audio effect processor A configured with parameter A.
[0082] The sound effect parameter adjustment method provided in this embodiment inputs a first sound effect parameter into a trained sound effect parameter adjustment model to obtain a second sound effect parameter; the audio to be processed is input into a second sound effect processor configured with the second sound effect parameter, and the sound effect processing result of the audio to be processed is basically consistent with the sound effect processing result of the first sound effect processor configured with the first sound effect parameter. This method can make the processing results of different sound effect processors for the same audio to be processed basically consistent, realize the adaptive adaptation of the output results of different sound effect processors, and improve the efficiency of sound effect parameter adjustment.
[0083] It should also be understood that the above is only to help those skilled in the art better understand the embodiments of this disclosure, and is not intended to limit the scope of the embodiments of this disclosure. Those skilled in the art can obviously make various equivalent modifications or changes based on the examples given above. For example, some steps in the above methods may be unnecessary, or new steps may be added, etc. Alternatively, any combination of any two or more of the above embodiments may be used. Such modifications, changes, or combinations also fall within the scope of the embodiments of this disclosure.
[0084] It should also be understood that the above description of the embodiments of this disclosure focuses on highlighting the differences between the various embodiments. Similarities or differences not mentioned can be referred to each other, and for the sake of brevity, they will not be repeated here.
[0085] It should also be understood that the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.
[0086] It should also be understood that, in the various embodiments of this disclosure, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions between different embodiments are consistent and can be referenced by each other, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.
[0087] The foregoing section detailed an example of a training method for the sound effect parameter adjustment model provided in this disclosure. It is understood that, in order to implement the aforementioned functions, a computer device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.
[0088] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.
[0089] Figure 6 This is a block diagram illustrating a training apparatus for a sound effect parameter adjustment model according to an exemplary embodiment. (Reference) Figure 6 The device 600 may include an acquisition module 610, an acquisition module 620, and a training module 630.
[0090] The acquisition module 610 is configured to acquire sample audio and a first sound effect parameter; the acquisition module 620 is configured to input the first sound effect parameter into a sound effect parameter adjustment model to be trained to obtain a second sound effect parameter; the acquisition module 620 is further configured to input the sample audio into a first sound effect processor configured with the first sound effect parameter to obtain a first processing result of the sample audio; and input the sample audio into a second sound effect processor configured with the second sound effect processing parameter to obtain a second processing result of the sample audio; the training module 630 is configured to train the sound effect parameter adjustment model to be trained based on the approximation between the first processing result and the second processing result to obtain a trained sound effect parameter adjustment model.
[0091] In some exemplary embodiments of this disclosure, the training module 630 is configured to perform: determining the degree of approximation between the first processing result and the second processing result; if the degree of approximation is greater than a preset threshold, adjusting the model parameters of the sound effect parameter adjustment model to be trained until the degree of approximation between the adjusted second processing result obtained according to the adjusted sound effect parameter adjustment model and the first sound effect processing result is less than or equal to the preset threshold, and determining the adjusted sound effect parameter adjustment model as the sound effect parameter adjustment model that has been trained.
[0092] In some exemplary embodiments of this disclosure, the training module 630 is configured to perform: adjusting the model parameters of the sound effect parameter adjustment model to be trained when the approximation degree is greater than a preset threshold; inputting the first sound effect parameters into the adjusted sound effect parameter adjustment model to obtain the adjusted second sound effect parameters; inputting the sample audio into a second sound effect processor configured with the adjusted second sound effect parameters to obtain the adjusted second processing result; and determining the adjusted sound effect parameter adjustment model as the trained sound effect parameter adjustment model when the approximation degree between the first processing result and the adjusted second processing result is less than or equal to the preset threshold.
[0093] In some exemplary embodiments of this disclosure, the acquisition module 610 is configured to perform: acquiring sample singing data and sample speech data; adjusting the reverberation parameter value and equalization parameter value of the sample singing data, and adjusting the reverberation parameter value and equalization parameter value of the sample speech data; and generating the sample audio based on the sample singing data, the sample speech data, the adjusted sample singing data, and the adjusted sample speech data.
[0094] In some exemplary embodiments of this disclosure, the degree of approximation includes at least one of waveform distance, spectral distance, and signal-to-noise ratio between the first processing result and the second processing result.
[0095] Figure 7 This is a block diagram illustrating a sound effect parameter adjustment device according to an exemplary embodiment. (Refer to...) Figure 7 The device 700 may include an acquisition module 710 and an acquisition module 720.
[0096] The acquisition module 710 is configured to acquire the audio to be processed and the first sound effect parameters of the first sound effect processor; the acquisition module 720 is configured to input the first sound effect parameters into a trained sound effect parameter adjustment model to obtain the second sound effect parameters, wherein the trained sound effect parameter adjustment model is trained in the above embodiment; the acquisition module 720 is further configured to input the audio to be processed into a second sound effect processor configured with the second sound effect parameters to obtain the sound effect processing result of the audio to be processed.
[0097] It should be noted that the block diagrams shown in the above figures are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor terminal devices and / or microcontroller terminal devices.
[0098] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0099] The following reference Figure 8 To describe an electronic device 800 according to such an embodiment of the present disclosure. Figure 8 The electronic device 800 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.
[0100] like Figure 8As shown, the electronic device 800 is presented in the form of a general-purpose computing device. The components of the electronic device 800 may include, but are not limited to: at least one processing unit 810, at least one storage unit 820, a bus 830 connecting different system components (including storage unit 820 and processing unit 810), and a display unit 840.
[0101] The storage unit stores program code, which can be executed by the processing unit 810 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 810 can perform actions such as... Figure 2 The steps shown are as follows.
[0102] For example, electronic devices can achieve such Figure 2 The steps shown.
[0103] Storage unit 820 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 821 and / or cache memory 822, and may further include a read-only memory (ROM) 823.
[0104] The storage unit 820 may also include a program / utility 824 having a set (at least one) of program modules 825, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.
[0105] Bus 830 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.
[0106] Electronic device 800 can also communicate with one or more external devices 870 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 800, and / or with any device that enables electronic device 800 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 850. Furthermore, electronic device 800 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 860. As shown, network adapter 860 communicates with other modules of electronic device 800 via bus 830. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0107] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.
[0108] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions that can be executed by a processor of the device to perform the described method. Optionally, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.
[0109] In an exemplary embodiment, a computer program product is also provided, including a computer program / instructions, which, when executed by a processor, implement the training method for the sound effect parameter adjustment model in the above embodiments.
[0110] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.
[0111] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.
Claims
1. A training method for a sound effect parameter adjustment model, characterized in that, include: Obtain the sample audio and the first sound effect parameters; The first sound effect parameter is input into the sound effect parameter adjustment model to be trained to obtain the second sound effect parameter; The sample audio is input to a first sound effect processor configured with the first sound effect parameter to obtain a first processing result of the sample audio; and the sample audio is input to a second sound effect processor configured with the second sound effect parameter to obtain a second processing result of the sample audio; Based on the degree of approximation between the first processing result and the second processing result, the sound effect parameter adjustment model to be trained is trained to obtain the trained sound effect parameter adjustment model; The first sound effect processor and the second sound effect processor are different sound effect processors; the first sound effect processor is the target sound effect processor, and the second sound effect processor is the fitting sound effect processor. The training task of the sound effect parameter adjustment model is to use a neural network model to fit the output sound effect of the second sound effect processor to the output sound effect of the first sound effect processor.
2. The method according to claim 1, characterized in that, Based on the degree of approximation between the first processing result and the second processing result, the sound effect parameter adjustment model to be trained is trained to obtain the trained sound effect parameter adjustment model, including: Determine the degree of approximation between the first processing result and the second processing result; If the approximation is greater than a preset threshold, the model parameters of the sound effect parameter adjustment model to be trained are adjusted until the approximation between the adjusted second processing result obtained according to the adjusted sound effect parameter adjustment model and the first processing result is less than or equal to the preset threshold. The adjusted sound effect parameter adjustment model is then determined as the sound effect parameter adjustment model that has been trained.
3. The method according to claim 2, characterized in that, If the approximation degree is greater than a preset threshold, the model parameters of the sound effect parameter adjustment model to be trained are adjusted until the approximation degree between the adjusted second processing result obtained according to the adjusted sound effect parameter adjustment model and the first processing result is less than or equal to the preset threshold. The adjusted sound effect parameter adjustment model is then determined as the trained sound effect parameter adjustment model, including: If the approximation is greater than a preset threshold, the model parameters of the training sound effect parameters are adjusted. The first sound effect parameter is input into the adjusted sound effect parameter adjustment model to obtain the adjusted second sound effect parameter. The sample audio is input to a second sound effect processor configured with the adjusted second sound effect parameters to obtain the adjusted second processing result; If the approximation between the first processing result and the adjusted second processing result is less than or equal to the preset threshold, the adjusted sound effect parameter adjustment model is determined as the trained sound effect parameter adjustment model.
4. The method according to any one of claims 1-3, characterized in that, Obtain sample audio, including: Acquire sample singing data and sample speech data; Adjust the reverberation and equalization parameter values of the sample singing data, and adjust the reverberation and equalization parameter values of the sample speech data; The sample audio is generated based on the sample singing data, the sample speech data, the adjusted sample singing data, and the adjusted sample speech data.
5. The method according to any one of claims 1-3, characterized in that, The degree of approximation includes at least one of waveform distance, spectral distance, and signal-to-noise ratio between the first processing result and the second processing result.
6. A method for adjusting sound effect parameters, characterized in that, include: Obtain the audio to be processed and the first sound effect parameters of the first sound effect processor; The first sound effect parameter is input into the trained sound effect parameter adjustment model to obtain the second sound effect parameter; wherein the trained sound effect parameter adjustment model is trained according to any one of claims 1-5. The audio to be processed is input to a second audio processor configured with the second audio effect parameters to obtain the audio effect processing result.
7. A training device for a sound effect parameter adjustment model, characterized in that, include: The acquisition module is configured to acquire sample audio and first sound effect parameters; The module is configured to input the first sound effect parameters into the sound effect parameter adjustment model to be trained, and obtain the second sound effect parameters; The obtaining module is further configured to input the sample audio to a first sound effect processor configured with the first sound effect parameter to obtain a first processing result of the sample audio; and to input the sample audio to a second sound effect processor configured with the second sound effect parameter to obtain a second processing result of the sample audio; The training module is configured to train the sound effect parameter adjustment model to be trained based on the degree of approximation between the first processing result and the second processing result, so as to obtain the trained sound effect parameter adjustment model. The first sound effect processor and the second sound effect processor are different sound effect processors; the first sound effect processor is the target sound effect processor, and the second sound effect processor is the fitting sound effect processor. The training task of the sound effect parameter adjustment model is to use a neural network model to fit the output sound effect of the second sound effect processor to the output sound effect of the first sound effect processor.
8. A sound effect parameter adjustment device, characterized in that, include: The acquisition module is configured to acquire the audio to be processed and the first sound effect parameters of the first sound effect processor. The module is configured to input the first sound effect parameter into a trained sound effect parameter adjustment model to obtain the second sound effect parameter; wherein the trained sound effect parameter adjustment model is trained according to the method of any one of claims 1-5. The obtaining module is further configured to input the audio to be processed to a second sound effect processor configured with the second sound effect parameters, and obtain the sound effect processing result of the audio to be processed.
9. An electronic device, characterized in that, include: Processor; Memory used to store the processor's executable instructions; The processor is configured to execute the executable instructions to implement the training method of the sound effect parameter adjustment model as described in any one of claims 1 to 5, or the sound effect parameter adjustment method as described in claim 6.
10. A computer-readable storage medium, wherein instructions in the computer-readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform a training method for a sound effect parameter adjustment model as described in any one of claims 1 to 5, or a sound effect parameter adjustment method as described in claim 6.