Audio sound effect generalization tuning method, medium, device and program product

Abstract

The invention provides an audio sound effect generalization tuning method, a medium, equipment and a program product, and relates to the field of audio processing, and the method comprises the steps: when a tuning instruction for a target audio is received, recognizing the tuning instruction to obtain user evaluation information of the target audio, and determining a sound effect parameter vector of the target audio; inputting the user evaluation information into a sound effect mapping model to obtain sound effect adjustment parameters corresponding to the user evaluation information; and performing generalization tuning on the sound effect parameter vector by using the sound effect adjustment parameter. Through application of the method and the device, the user can use the text information to carry out intelligent tone tuning on the audio according to own demands and preferences, the sound effect quality and the individuation degree are improved, the sound effect demands of different scenes and audiences can be met, and higher flexibility and innovativeness are brought to the fields of music making and audio processing.

Description

Technical Field

[0001] The present application relates to the field of audio processing, and in particular to a method, medium, device and program product for generalized tuning of audio sound effects. Background Art

[0002] With the popularity of streaming music platforms, music has entered the lives of more and more people. In audio processing, generalized tuning involves adjusting the overtone components in the audio signal. Overtones are an important part of the timbre. By enhancing or weakening the frequency and amplitude of certain overtones, the characteristics and expressiveness of the timbre can be changed. However, the current tuning of audio depends on the adjustment of professionals. When the music playback platform does not provide music that has been generalized and tuned by professionals, it is often difficult to meet the music usage needs of users.

[0003] For example, there may be more subjective tuning schemes depending on the music playing scenario and personal preferences. For example, in a car scenario, users may prefer to enhance the low frequency of the music, while some users may prefer music that focuses on the human voice. Summary of the invention

[0004] The purpose of this application is to provide a generalized tuning method for audio sound effects, a computer-readable storage medium, an electronic device and a computer program product, which can adjust the sound effects according to user needs and realize the personalization of the audio sound effects.

[0005] In order to solve the above technical problems, the present application provides a generalized tuning method for audio sound effects, and the specific technical solution is as follows:

[0006] When a tuning instruction for a target audio is received, identifying the tuning instruction to obtain user comment information on the target audio, and determining a sound effect parameter vector of the target audio;

[0007] Inputting the user comment information into a sound effect mapping model to obtain a sound effect adjustment parameter corresponding to the user comment information;

[0008] The sound effect adjustment parameter is applied to perform generalized tuning on the sound effect parameter vector.

[0009] Optionally, identifying the tuning instruction to obtain user comment information on the target audio includes:

[0010] parsing the tuning instruction to obtain text features;

[0011] Semantic information corresponding to the text feature is identified, and the semantic information is used as user comment information on the target audio.

[0012] Optionally, identifying the tuning instruction to obtain user comment information on the target audio includes:

[0013] If the tuning instruction includes quantization information, the target audio is identified based on the quantization information group to obtain an objective sound effect evaluation result of the sound effect evaluation; the objective sound effect evaluation result is used to guide objective sound effect tuning;

[0014] If the tuning instruction includes non-quantitative information, the target audio is identified based on the non-quantitative information group to obtain a subjective sound effect evaluation result of the sound effect evaluation; the subjective sound effect evaluation result is used to guide subjective sound effect tuning.

[0015] Optionally, before inputting the user comment information into the sound effect mapping model, the method further includes:

[0016] Acquire an audio combination; the audio combination includes standard audio and processed audio; the processed audio is obtained by processing the standard audio through an audio effect processing link;

[0017] Acquire annotation information after annotating the audio combination; the annotation information includes non-quantitative annotation information obtained based on the annotation of the non-quantitative information group, and quantitative annotation information obtained based on the annotation of the quantitative information group;

[0018] An association analysis is performed on the annotation information and the corresponding audio combination to generate the sound effect mapping model including the audio processing mapping relationship between the annotation information and the audio combination.

[0019] Optionally, performing association analysis on the annotation information and the corresponding audio combination to generate the sound effect mapping model including the audio processing mapping relationship between the annotation information and the audio combination includes:

[0020] Extracting the sound effect processing feature corresponding to the sound effect processing link in the audio combination;

[0021] Extracting text features from the annotation information;

[0022] Matching the text feature with the annotation information to determine an association rule between the sound effect processing feature and the annotation information; the association rule is used to indicate the text feature matched by the sound effect processing feature;

[0023] The audio combination, the annotation information and the association rule are maintained by using a custom data structure to obtain a sound effect mapping model.

[0024] Optionally, obtaining the quantitative annotation information in the annotation information includes:

[0025] Using a sound effect multidimensional scoring model to evaluate the sound effect of the audio combination with reference to the quantitative information group to obtain quantitative annotation information;

[0026] The sound effect multi-dimensional scoring model is generated as follows:

[0027] Obtaining evaluation results of several sound effect dimension parameter effects included in the audio combination;

[0028] The sound effect multi-dimensional scoring model is generated based on the review results and the review methods corresponding to the sound effects in each dimension.

[0029] Optionally, inputting the user comment information into a sound effect mapping model to obtain a sound effect adjustment parameter corresponding to the user comment information includes:

[0030] Determining the sound effect review information of multiple dimensions included in the user review information;

[0031] Determine the sound effect adjustment method corresponding to the sound effect review information of each dimension;

[0032] Convert each of the sound effect adjustment methods into a corresponding sound effect adjustment vector;

[0033] The sound effect adjustment vectors corresponding to the sound effects of each dimension are combined to obtain a sound effect adjustment parameter vector.

[0034] Optionally, if the tuning instruction includes audio playback environment information, after obtaining the tuning instruction, the method further includes:

[0035] Determining scene sound effect information adapted to the target audio playback environment information;

[0036] Accordingly, after inputting the user comment information into the sound effect mapping model to obtain the sound effect adjustment parameter corresponding to the user comment information, the method further includes:

[0037] Modifying the sound effect adjustment parameter according to the scene sound effect information to obtain a sound effect modification parameter;

[0038] Correspondingly, applying the sound effect adjustment parameter to perform generalized tuning on the sound effect parameter vector includes:

[0039] The sound effect modification parameter is applied to perform generalized tuning on the sound effect parameter vector.

[0040] Optionally, after applying the sound effect adjustment parameter to perform generalized tuning on the sound effect parameter vector, the method further includes:

[0041] Acquiring evaluation data of the target audio;

[0042] Using the evaluation data as public user review information results;

[0043] Inputting the public user review information result into the sound effect mapping model to obtain a second sound effect adjustment parameter;

[0044] The sound effect parameter vector is generalized and tuned according to the second sound effect adjustment parameter.

[0045] The present application also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method for generalizing audio sound effects tuning as described above are implemented.

[0046] The present application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, the steps of the audio sound effect generalization tuning method as described above are implemented.

[0047] The present application also provides a computer program product, including a computer program, which, when executed, implements the steps of the method for generalizing audio sound effects tuning as described above.

[0048] The present application provides a method for generalizing audio sound effects tuning, comprising: when a tuning instruction for a target audio is received, identifying the tuning instruction to obtain user comment information on the target audio, and determining a sound effect parameter vector of the target audio; inputting the user comment information into a sound effect mapping model to obtain sound effect adjustment parameters corresponding to the user comment information; and applying the sound effect adjustment parameters to perform generalized tuning on the sound effect parameter vector.

[0049] After obtaining the tuning instructions for the audio, the application determines the user comment information and the sound effect parameter vector of the audio corresponding to the tuning instructions, thereby converting the user comment information into corresponding sound effect adjustment parameters. The user can intelligently tune the audio according to his or her needs and preferences by issuing instruction information, thereby improving the sound quality and personalization, helping to meet the sound effect needs of different scenarios and audiences, and bringing greater flexibility and innovation to the fields of music production and audio processing.

[0050] The present application also provides a computer-readable storage medium, an electronic device, and a computer program product, which have the above-mentioned beneficial effects and are not described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0052] Figure 1 A flowchart of a method for generalizing audio sound effects tuning provided by an embodiment of the present application;

[0053] Figure 2 A schematic diagram of the process of generating an audio pair provided in an embodiment of the present application;

[0054] Figure 3 A schematic diagram of the audio to text annotation processing process provided in an embodiment of the present application;

[0055] Figure 4 A five-dimensional schematic diagram of the multi-dimensional sound effect scoring model provided in an embodiment of the present application;

[0056] Figure 5 A schematic diagram of the model application structure provided in the embodiment of the present application;

[0057] Figure 6 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0059] Generalized tuning refers to an audio processing technology that aims to adapt audio devices or systems to different acoustic environments and hearing preferences. Generalized tuning improves the overall sound quality experience and achieves the best hearing effect by expanding the target population and usage scenarios of the audio system so that it is not limited to specific sound settings or user configurations.

[0060] See also Figure 1 , Figure 1 A flowchart of a method for generalizing audio sound effects tuning provided by an embodiment of the present application, the method comprising:

[0061] S101: when receiving a tuning instruction for a target audio, identifying the tuning instruction to obtain user comment information on the target audio, and determining a sound effect parameter vector of the target audio;

[0062] S102: inputting the user comment information into a sound effect mapping model to obtain a sound effect adjustment parameter corresponding to the user comment information;

[0063] S103: Apply the sound effect adjustment parameter to perform generalized tuning on the sound effect parameter vector.

[0064] First, obtain the tuning instructions. There is no limitation on how to obtain the tuning instructions, and the tuning instructions may include direct tuning instructions and indirect tuning instructions. Direct tuning instructions refer to instructions whose instruction information contains parameters related to sound effect tuning, including but not limited to tuning information in text form input by the user, or tuning instructions that can be converted into other formats containing text information after processing, such as voice tuning instructions containing user comment information. In a feasible practical application scenario, the user's voice instructions can be obtained through a sound pickup device such as a smart speaker, and the voice instructions can be parsed to obtain tuning instructions in text format.

[0065] Indirect tuning instructions are instructions that do not contain parameters related to sound effect tuning, but can be converted into direct tuning instructions through instruction recognition. For example, indirect tuning instructions may include audio playback environment information, mood information, or music preferences. After semantic recognition of the indirect tuning instructions, they are converted into corresponding direct tuning instructions. The conversion process can be defined in advance to determine the conversion relationship between various types of indirect tuning instructions and direct tuning instructions, and different types of tuning instructions may have different conversion relationships. For example, for audio playback environment information, it may include the conversion relationship between weather information and direct tuning instructions, or the conversion relationship between the playback environment and direct tuning instructions; for mood information, the conversion relationship between mood information and direct tuning instructions may be adopted.

[0066] It is easy to understand that if the tuning instructions in text format are used, in order to achieve generalized tuning of sound effects, corresponding text content recognition needs to be performed. The tuning instructions can be parsed to obtain text features, and then by identifying the semantic information corresponding to the text features, the semantic information can be used as user comment information on the target audio.

[0067] There is no limitation on how to perform text content recognition. Pre-trained text recognition models such as BERT (Bidirectional Encoder Representations from Transformers), GPT (Generative Pre-trained Transformer), and ELMo (Embeddings from Language Models) can be used to understand the text content of the tuning instructions to determine the user comment information contained therein.

[0068] In step S102, in addition to determining the user comment information contained in the tuning instruction, it is also necessary to determine the current sound effect parameter vector corresponding to the target audio. It is easy to understand that there is a corresponding target audio for the tuning instruction. The target audio may be included in the tuning instruction, or it may be determined based on the instruction acquisition time of the tuning instruction. In a feasible implementation, the target audio played at the same time may be determined based on the acquisition time of the tuning instruction. For example, when an audio device such as a smart speaker plays music, if a tuning instruction is obtained, it can be determined that the target audio is the currently playing music. The determined target audio should at least include a unique identification feature, such as the name of the target audio, singer information, album, etc. There is no limitation on how to obtain the sound effect parameter vector. After determining the target audio, it can be obtained from a local audio database or online.

[0069] The sound effect parameter vector of the target audio refers to the current sound effect parameter vector of the target audio. The sound effect parameters in vector form make it easier to directly perform generalized sound effect tuning on the target audio and improve tuning efficiency. The user review information includes the user's subjective evaluation of the target audio, such as review information of parameters such as the current audio, loudness, and timbre. The sound effect parameter vector is the objective parameter of the audio commented on by the user, mainly including timbre, number of channels, sound quality, loudness, and dynamics, etc. Dynamics refers to the change in loudness during the playback of the target audio, such as gradually increasing or decreasing. It should be noted that the target audio targeted by this application is not limited to a certain music song, and can be any piece of audio, or a long audio containing several audios, that is, audio of any length and type for the user.

[0070] When the user comment information is input into the sound effect mapping model to obtain the sound effect adjustment parameters corresponding to the user comment information, the following process can be used:

[0071] The first step is to determine the sound effect review information of multiple dimensions contained in the user review information;

[0072] Step 2: Determine the sound effect adjustment method corresponding to the sound effect review information in each dimension;

[0073] Step 3: convert each of the sound effect adjustment methods into a corresponding sound effect adjustment vector;

[0074] Step 4: Combine the sound effect adjustment vectors corresponding to the sound effects in each dimension to obtain a sound effect adjustment parameter vector.

[0075] First, the sound effect evaluation information of each dimension is determined, and then the corresponding sound effect adjustment method is determined. For example, if the sound effect evaluation information of the reverberation dimension is too small, the corresponding sound effect adjustment method is to increase the reverberation.

[0076] The sound effect adjustment parameter can be represented as a multidimensional vector, where each component of the multidimensional vector represents a sound effect parameter. For example, suppose there are the following sound effect parameters: volume, low frequency gain, mid-frequency gain, high frequency gain, reverberation depth, and delay time. Then the sound effect adjustment parameter vector can be represented as:

[0077] [Volume, low frequency gain, middle frequency gain, high frequency gain, reverb depth, delay time];

[0078] In practice, the specific values ​​of each component will vary depending on how the sound effect is adjusted. For example, if you need to increase the volume and adjust the equalizer settings to enhance the low-frequency effect, while adding some reverb and delay, the sound effect adjustment parameter vector can be:

[0079] [0.75,0.1,0,-0.2,0.5,0.3];

[0080] At this time, 0.75 can represent that the volume is increased to 75%, 0.1 represents that the low frequency gain is increased by 10%, the mid-frequency remains unchanged (0), the high frequency is reduced by 20% (-0.2), the reverberation depth is set to 50% (0.5), and the delay time is set to 0.3 seconds. It should be noted that the units of the sound effect adjustment vectors corresponding to sound effects of different dimensions may be different.

[0081] It should be noted that the dimensions and components of the sound effect adjustment parameter vector may vary depending on the specific sound effect processing system or software. In addition, the value range in the vector may also vary depending on the system, and can be set by those skilled in the art according to the vector corresponding to the sound effect adjustment vector.

[0082] After obtaining the tuning instruction of the audio, the embodiment of the present application determines the user comment information and the sound effect parameter vector of the audio corresponding to the tuning instruction, thereby converting the user comment information into the corresponding sound effect parameter vector. The user can use the text information to intelligently tune the audio according to his or her needs and preferences, thereby improving the sound effect quality and personalization, helping to meet the sound effect needs of different scenarios and listeners, and bringing higher flexibility and innovation to the fields of music production and audio processing.

[0083] Based on the above embodiment, in a feasible implementation, the user comment information may include at least one of quantitative information and non-quantitative information. At this time, when identifying the tuning instruction and obtaining the user comment information on the target audio, if the tuning instruction includes quantitative information, the target audio can be identified based on the quantitative information group to obtain an objective sound effect comment result of the sound effect comment, and the objective sound effect comment result is used to guide the objective sound effect tuning.

[0084] Quantitative information refers to the subjective description of the quantitative information of sound effects, which is strongly related to the sound effects, including but not limited to loudness, equalization, compression, delay, reverberation, etc. Quantitative information can be labeled based on a multi-dimensional scoring model for sound effects through questionnaires or question-and-answer methods. At this time, the objective sound effect evaluation results are characterized by instructions in the tuning instructions that are directly related to the sound effect adjustment. At this time, the sound effects of the audio can be directly affected by adjusting the quantitative information. For quantitative information, a questionnaire can be used to obtain the user's subjective evaluation. For example, ask the user: What is the reverberation of this song like? If the answer received is: The reverberation of this song is too small and can be adjusted to be much larger, then the objective sound effect evaluation effect that can be obtained at this time can be that the reverberation of the song is small. When the sound effect is adjusted based on the objective sound effect evaluation results, the reverberation of the song can be enhanced.

[0085] If the tuning instruction includes non-quantitative information, the target audio may be identified based on the non-quantitative information group to obtain a subjective sound effect evaluation result of the sound effect evaluation.

[0086] The non-quantitative information is weakly related to the sound effects, and is mainly music style information, such as the music emotion type, music genre, whether the music contains human voice, etc. Accordingly, when the target audio is identified based on the non-quantitative information group and a subjective sound effect evaluation result of the sound effect evaluation is obtained, the subjective sound effect evaluation result may include the evaluation content of the music style information, and when the subjective sound effect tuning is guided based on the subjective sound effect evaluation effect, adjustments can be made according to the music style.

[0087] The quantitative information group and the non-quantitative information group described above are used as references when determining the user review information, that is, judging whether the user review information contains information corresponding to the sound effect type in the group.

[0088] Before determining the quantitative information and non-quantitative information, the types of quantitative information and non-quantitative information to be determined can be determined by annotation. Specifically, the audio combination can be obtained first, and the standard audio and the processed audio in the audio combination can be annotated in pairs to obtain the annotation information. The annotation information includes the non-quantitative information group and the quantitative information group of the audio combination. Figure 2 , Figure 2This is a schematic diagram of the generation process of the audio pair provided in the embodiment of the present application. The audio combination consists of standard audio (SQ, StereoQuality) and processed audio (PA, Professional Audio). Standard audio usually refers to the original audio file that has not been processed or compressed. Standard audio can provide the purest and closest to the original sound quality experience. The processed audio refers to the audio file obtained after a series of sound effect processing. Among them, the processed audio can be obtained by the standard audio through the sound effect processing link. The sound effect processing link is composed of several sound effect processing processes, such as loudness, equalization, compression, delay, reverberation, etc. In the sound effect processing link, the sound effect parameter vector can be applied. By adjusting the sound effect parameter vector corresponding to each part of the sound effect processing process, the processed audio can be obtained.

[0089] Thereafter, see Figure 3 , Figure 3 The present invention provides a schematic diagram of the text annotation processing process of audio pairs provided in the embodiment of the present invention, in which the audio combination formed by the standard audio and the processed audio is annotated with text, and the audio non-quantitative information and the sound effect quantitative information are obtained by the annotation. The annotation information after the audio combination is annotated is obtained. Specifically, the annotation personnel can refer to the standard audio to obtain the annotation information of the processed audio. The annotation information is divided into two parts, namely the non-quantitative information and the quantitative information of the audio combination.

[0090] Finally, the annotation information and the corresponding audio combination are analyzed for association, and a sound effect mapping model including the audio processing mapping relationship between the annotation information and the audio combination is generated. Specifically, the following steps may be included:

[0091] The first step is to extract the sound effect processing features corresponding to the sound effect processing link in the audio combination;

[0092] Step 2: extracting text features from the annotation information;

[0093] Step 3: Match the text feature with the annotation information to determine an association rule between the sound effect processing feature and the annotation information; the association rule is used to indicate the text feature matched by the sound effect processing feature;

[0094] Step 4: Use a custom data structure to maintain the audio combination, the annotation information and the association rule to obtain a sound effect mapping model.

[0095] When performing association analysis, the standard audio and the corresponding processed audio in the audio combination are compared, and the difference in the text features of the annotation information is used to feedback the difference in the sound effects between the processed audio obtained after the standard audio is processed through the sound effect processing link, that is, the difference between the sound effect processing features corresponding to the sound effect processing link. By analyzing and summarizing several audio combinations, the audio processing mapping relationship between the annotation information and the audio combination is determined, that is, the association rules between the sound effect processing link contained in the audio combination and the corresponding annotation information, and the sound effect mapping model is obtained.

[0096] It is easy to understand that the text features and corresponding semantic information obtained by the pre-trained text recognition model can include objective sound effect evaluation results and subjective sound effect evaluation results.

[0097] Based on the above embodiments, in a feasible implementation, when obtaining the sound effect quantification information, the sound effect multi-dimensional scoring model can be applied to the audio combination with reference to the quantification information group to evaluate the sound effect effect, and obtain the quantitative annotation information. By using the sound effect multi-dimensional scoring model, the sound effect processing link used by the audio combination can be directly mapped to the corresponding sound effect parameters, thereby simplifying the sound effect generalization tuning process and improving the efficiency of sound effect generalization tuning.

[0098] The generation method of the multidimensional sound effect scoring model is not specifically limited here. In a feasible implementation, the review results of the parameter effects of several sound effect dimensions contained in the audio combination can be obtained, so as to generate the multidimensional sound effect scoring model based on the review results and the review methods corresponding to the sound effects of each dimension. After that, the multidimensional sound effect scoring model can be directly collected and applied to refer to the quantitative information group to annotate the quantitative information of the sound effect of the audio combination. The review method used to obtain the review results is not limited here, and review methods such as user subjective scoring and user questionnaires can be used.

[0099] Each dimension in the multi-dimensional sound effect scoring model corresponds to some important parameters in the sound effect. Figure 4 As shown, Figure 4 The five-dimensional schematic diagram of the multi-dimensional sound effect scoring model provided in the embodiment of the present application lists some of the sound effect dimensions, including sound quality, balance, reverberation, dynamics, and loudness. For each dimension of any audio combination, the multi-dimensional sound effect scoring model will be used to ask questions and collect user comment information in the form of a questionnaire.

[0100] For the quantitative information in the user review information, the quantitative information group can be used to compare the quantitative information and directly determine the corresponding sound effect adjustment parameters.

[0101] For non-quantitative information in user review information, such as the adjustment of the emotional type of audio, it usually includes the adjustment of at least one sound effect. Taking the adjustment of the emotional type of audio as an example, if the audio is changed from light to sad, it can be achieved by comprehensively adjusting multiple sound effect parameters such as rhythm, harmony, melody, timbre, dynamics and reverberation. In terms of rhythm, slowing down the rhythm can extend the duration of the notes, making the music seem slower and heavier, thus creating a sense of sadness. In terms of harmony, using minor keys instead of major keys, introducing more minor chords or lowering the tonality of the music usually makes the music sound more melancholy and sad. In terms of melody, by simplifying the melody line, reducing jumping notes, and using more descending and continuous bass melodies, the music can be made heavier and sadder. In terms of timbre, choosing warm and soft timbres, such as string instruments and pianos, can reduce the bright and cheerful feeling and increase the sad atmosphere. In terms of loudness, the dynamic range of the music can be reduced and the overall volume can be lowered, which can make the music seem more subtle and restrained, thus creating a sad mood. Reverb adjustment can make music sound more spacious, melodious, and sad by increasing the reverberation effect to simulate the effect of playing in a larger space (such as a church or cave).

[0102] Therefore, the non-quantitative information of the user review information can also be converted into sound effect review information containing several dimensions.

[0103] From this, it can be seen that the embodiments of the present application can not only process quantized text information directly related to sound effect parameters, but also process non-quantized information weakly related to sound effect parameters, thereby expanding the application scope of sound effect tuning and meeting more types of sound effect tuning needs.

[0104] On the basis of the above embodiments, as a preferred embodiment, after obtaining the tuning instruction, if the tuning instruction is an indirect tuning instruction, such as audio playback environment information, the scene sound effect information adapted to the audio playback environment information can be determined, so that the sound effect adjustment parameters are corrected according to the scene sound effect information to obtain the sound effect correction parameters, so as to apply the sound effect correction parameters to perform generalized tuning on the sound effect parameter vector.

[0105] The audio playback environment information may include space information, weather information, etc. For example, if a user prefers to listen to cheerful music on cloudy days, based on the sound effect adjustment parameters obtained in the above embodiment, the sound effect adjustment parameters may be further modified according to the scene sound effect information to obtain the sound effect correction parameters, thereby performing generalized tuning. In addition, if the user comment information of the user tuning instruction is invalid comment information, that is, it has no relevance to the sound effect and only includes the audio playback environment information, then the sound effect adjustment parameters corresponding to the scene sound effect information may be directly determined, thereby performing generalized tuning. Of course, the user can preset the sound effect adjustment parameters under different environments, so that after detecting the keywords of the audio playback environment information, the adapted sound effect adjustment parameters are directly applied for generalized tuning. For example, different sound effect adjustment parameters adapted to open environments and bathrooms.

[0106] On the basis of the above embodiment, as a preferred embodiment, after applying the sound effect adjustment parameters to generalize and tune the sound effect parameter vector, the evaluation data of the target audio can also be obtained; the evaluation data is used as the result of the public user review information, and the public user review information result is input into the sound effect mapping model to obtain the second sound effect adjustment parameter, and the sound effect parameter vector is generalized and tuned according to the second sound effect adjustment parameter.

[0107] If the target audio is a song, on the basis of obtaining the sound effect parameter vector of the song, relevant comments and other information of the song can also be extracted. This embodiment limits the evaluation data to be obtained after the sound effect adjustment parameter is applied to the sound effect parameter vector for generalization tuning. In other embodiments of the present application, the evaluation data can be obtained synchronously with the sound effect parameter vector. After the sound effect adjustment parameter is applied to the sound effect parameter vector for generalization tuning, the sound effect parameter vector is further generalized and tuned based on the second sound effect adjustment parameter corresponding to the result of the public user comment information.

[0108] If the comments on a song include other users' suggestions for adjusting the song's sound effects, further tuning can be done based on the sound effect adjustment suggestions to meet richer tuning needs and achieve better listening needs.

[0109] In the actual application process of this application, see Figure 5 , Figure 5 The schematic diagram of the model application structure provided in the embodiment of the present application shows that the acquired text is used as a tuning instruction and input into a pre-generated sound effect parameter model. The sound effect parameter generation model consists of two parts: one is a text understanding module that processes the text to obtain text features, and the other is a sound effect mapping module that maps the text features obtained by the text understanding module to sound effect parameters, and finally obtains a sound effect adjustment parameter vector. The text understanding module can use a pre-trained text understanding model, and the final model can input any text (Chinese) to obtain a corresponding sound effect adjustment parameter vector.

[0110] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed, the steps of the generalized tuning method for sound effects provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0111] The present application also provides a computer program product, including a computer program, which, when executed, implements the steps of the audio sound effect generalization tuning method as described in the above embodiment.

[0112] The present application also provides an electronic device, see Figure 6 , a structural diagram of an electronic device provided in an embodiment of the present application, such as Figure 6 As shown, a processor 610 and a memory 620 may be included.

[0113] Among them, the processor 610 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 610 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 610 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 610 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 610 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0114] The memory 620 may include one or more computer-readable storage media, which may be non-transitory. The memory 620 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 620 is at least used to store the following computer program 621, wherein, after the computer program is loaded and executed by the processor 610, it can implement the relevant steps in the method performed by the electronic device side disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 620 may also include an operating system 622 and data 623, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 622 may include Windows, Linux, Android, etc.

[0115] In some embodiments, the electronic device may further include a display screen 630 , an input / output interface 640 , a communication interface 650 , a sensor 660 , a power source 670 , and a communication bus 680 .

[0116] certainly, Figure 6 The structure of the electronic device shown does not constitute a limitation on the electronic device in the embodiments of the present application. In actual applications, the electronic device may include Figure 6 More or fewer components than shown, or combinations of certain components.

[0117] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0118] Specific examples are used herein to illustrate the principles and implementation methods of the present application, and the description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the present application.

[0119] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A method for generalizing audio sound effects, characterized in that: include: When a tuning instruction for a target audio is received, identifying the tuning instruction to obtain user comment information on the target audio, and determining a sound effect parameter vector of the target audio; Inputting the user comment information into a sound effect mapping model to obtain a sound effect adjustment parameter corresponding to the user comment information; The sound effect adjustment parameter is applied to perform generalized tuning on the sound effect parameter vector.

2. The method for generalizing sound effects according to claim 1, characterized in that: Identifying the tuning instruction to obtain user comment information on the target audio includes: parsing the tuning instruction to obtain text features; Semantic information corresponding to the text feature is identified, and the semantic information is used as user comment information on the target audio.

3. The method for generalizing sound effects according to claim 1, characterized in that: The user review information includes at least one of an objective sound effect review result and a subjective sound effect review result; the identifying the tuning instruction to obtain the user review information on the target audio includes: If the tuning instruction includes quantization information, the target audio is identified based on the quantization information group to obtain an objective sound effect evaluation result of the sound effect evaluation; the objective sound effect evaluation result is used to guide the objective sound effect tuning; If the tuning instruction includes non-quantitative information, the target audio is identified based on the non-quantitative information group to obtain a subjective sound effect evaluation result of the sound effect evaluation; the subjective sound effect evaluation effect is used to guide subjective sound effect tuning.

4. The method for generalizing sound effects according to claim 3, characterized in that: Before inputting the user comment information into the sound effect mapping model, the method further includes: Acquire an audio combination; the audio combination includes standard audio and processed audio; the processed audio is obtained by processing the standard audio through an audio effect processing link; Acquire annotation information after annotating the audio combination; the annotation information includes non-quantitative annotation information obtained based on the annotation of the non-quantitative information group, and quantitative annotation information obtained based on the annotation of the quantitative information group; An association analysis is performed on the annotation information and the corresponding audio combination to generate the sound effect mapping model including the audio processing mapping relationship between the annotation information and the audio combination.

5. The method for generalizing sound effects according to claim 4, characterized in that: Performing correlation analysis on the annotation information and the corresponding audio combination to generate the sound effect mapping model including the audio processing mapping relationship between the annotation information and the audio combination includes: Extracting the sound effect processing feature corresponding to the sound effect processing link in the audio combination; Extracting text features from the annotation information; Matching the text feature with the annotation information to determine an association rule between the sound effect processing feature and the annotation information; the association rule is used to indicate the text feature matched by the sound effect processing feature; The audio combination, the annotation information and the association rule are maintained by using a custom data structure to obtain a sound effect mapping model.

6. The method for generalizing sound effects according to claim 4, characterized in that: Acquiring the quantitative annotation information in the annotation information includes: Using a sound effect multidimensional scoring model to evaluate the sound effect of the audio combination with reference to the quantitative information group to obtain quantitative annotation information; The sound effect multi-dimensional scoring model is generated as follows: Obtaining evaluation results of several sound effect dimension parameter effects included in the audio combination; The sound effect multi-dimensional scoring model is generated based on the review results and the review methods corresponding to the sound effects in each dimension.

7. The method for generalizing sound effects according to claim 1, characterized in that: Inputting the user comment information into the sound effect mapping model to obtain the sound effect adjustment parameters corresponding to the user comment information includes: Determining the sound effect review information of multiple dimensions included in the user review information; Determine the sound effect adjustment method corresponding to the sound effect review information of each dimension; Convert each of the sound effect adjustment methods into a corresponding sound effect adjustment vector; The sound effect adjustment vectors corresponding to the sound effects of each dimension are combined to obtain a sound effect adjustment parameter vector.

8. The generalized sound effect tuning method according to any one of claims 1 to 7, characterized in that: If the tuning instruction includes audio playing environment information, after obtaining the tuning instruction, the method further includes: Determining scene sound effect information adapted to the target audio playback environment information; After inputting the user comment information into the sound effect mapping model to obtain the sound effect adjustment parameter corresponding to the user comment information, the method further includes: Modifying the sound effect adjustment parameter according to the scene sound effect information to obtain a sound effect modification parameter; The applying the sound effect adjustment parameter to perform generalized tuning on the sound effect parameter vector comprises: The sound effect modification parameter is applied to perform generalized tuning on the sound effect parameter vector.

9. The generalized sound effect tuning method according to any one of claims 1 to 7, characterized in that: After applying the sound effect adjustment parameter to perform generalized tuning on the sound effect parameter vector, the method further includes: Acquiring evaluation data of the target audio; Using the evaluation data as public user review information results; Inputting the public user review information result into the sound effect mapping model to obtain a second sound effect adjustment parameter; The sound effect parameter vector is generalized and tuned according to the second sound effect adjustment parameter.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for generalizing the audio sound effects tuning as described in any one of claims 1 to 9 are implemented.

11. An electronic device, characterized in that: It comprises a memory and a processor, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, it implements the steps of the audio sound effect generalization tuning method as described in any one of claims 1 to 9.

12. A computer program product, characterized in that It comprises a computer program, which, when executed, implements the steps of the method for generalizing the audio sound effects tuning as claimed in any one of claims 1 to 9.

Images