Audio processing method and device

Abstract

The invention discloses an audio processing method and equipment. The audio processing method at least comprises the following steps: acquiring a first audio to be output by an application program at the current moment; identifying a second audio and a background audio in the first audio at the current moment; wherein the second audio comprises a target audio generated by simulating a target object; processing a target audio in the second audio at the current moment to obtain a third audio at the current moment; outputting the background audio and a third audio at the current moment; wherein the target audio before processing has first audio intensity, the target audio after processing has second audio intensity, and the second audio intensity is greater than the first audio intensity.

Description

Technical Field

[0001] The present application relates to the technical field of audio processing, and is related to but not limited to audio processing methods and devices. Background Art

[0002] With the continuous development of audio technology, various common audio can be adjusted. For example, you can set the parameters of the equalization mode when playing sound in the system settings of the device to adjust the audio. You can also set the audio parameters in an application to adjust the audio of this application.

[0003] In practice, sometimes it is necessary to adjust a specific audio in the audio being played. For example, in a shooting game, in order to clearly perceive the position of the enemy, it is necessary to enhance the footsteps or gunshots. The solutions in the related art can only adjust the audio as a whole, and cannot enhance a specific audio (such as footsteps or gunshots) in the audio being played. Summary of the invention

[0004] The embodiment of the present application provides an audio processing method and device. The technical solution of the present application is implemented as follows:

[0005] In a first aspect, the present application provides an audio processing method, the method comprising:

[0006] Get the first audio to be output by the application at the current moment;

[0007] Identify the second audio and background audio in the first audio at the current moment; wherein the second audio includes the target audio generated by the simulated target object;

[0008] Processing the target audio in the second audio at the current moment to obtain the third audio at the current moment;

[0009] Output the background audio and the third audio at the current moment;

[0010] The target audio before processing has a first audio intensity, and the target audio after processing has a second audio intensity, and the second audio intensity is greater than the first audio intensity.

[0011] In a second aspect, the present application provides an audio processing device, the device comprising:

[0012] An acquisition unit, used for acquiring a first audio to be output by the application program at a current moment;

[0013] A recognition unit, configured to recognize a second audio and a background audio in the first audio at a current moment; wherein the second audio includes a target audio generated by simulating a target object;

[0014] A processing unit, configured to process the target audio in the second audio at a current moment to obtain a third audio at a current moment;

[0015] An output unit, used for outputting background audio and a third audio at the current moment;

[0016] The target audio before processing has a first audio intensity, and the target audio after processing has a second audio intensity, and the second audio intensity is greater than the first audio intensity.

[0017] In a third aspect, the present application provides an electronic device, the electronic device comprising a first interface, a processor, and an output interface;

[0018] The first interface is used to: obtain the first audio to be output by the application at the current moment;

[0019] The processor is used to execute: identifying the second audio and the background audio in the first audio at the current moment; processing the target audio in the second audio at the current moment to obtain the third audio at the current moment; wherein the second audio includes the target audio generated by the simulated target object; the target audio before processing has a first audio intensity, and the target audio after processing has a second audio intensity, and the second audio intensity is greater than the first audio intensity;

[0020] The output interface is used to output background audio and the third audio at the current moment.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the audio processing method provided in the first aspect is implemented.

[0022] In a fifth aspect, the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the audio processing method provided in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A schematic diagram of a first optional flow chart of the audio processing method provided in an embodiment of the present application;

[0024] Figure 2 A second optional flow chart of the audio processing method provided in the embodiment of the present application;

[0025] Figure 3 A third optional flow chart of the audio processing method provided in the embodiment of the present application;

[0026] Figure 4 A fourth optional flow chart of the audio processing method provided in the embodiment of the present application;

[0027] Figure 5 A fifth optional flow chart of the audio processing method provided in the embodiment of the present application;

[0028] Figure 6 A sixth optional flow chart of the audio processing method provided in the embodiment of the present application;

[0029] Figure 7 An optional flowchart of the audio processing process provided in the embodiment of the present application;

[0030] Figure 8 An optional schematic diagram of a detection process provided in an embodiment of the present application;

[0031] Fig. 9 An optional schematic diagram of a sound enhancement process provided in an embodiment of the present application;

[0032] Fig.10 An optional schematic diagram of a filtering process provided in an embodiment of the present application;

[0033] Fig.11 A schematic diagram of an optional structure of an audio processing device provided in an embodiment of the present application;

[0034] Fig.12 An optional structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the specific technical solution of the application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0036] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0037] In the following description, the terms "first\second\third" are used only as examples to distinguish different objects, and do not represent a specific order for the objects, nor do they have a limitation on the order of precedence. It is understandable that "first\second\third" can be interchanged with a specific order or order of precedence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0038] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0039] The embodiments of the present application may provide an audio processing method, device, equipment, storage medium and computer program product. In practical applications, the audio processing method may be implemented by an audio processing device, and each functional entity in the audio processing device may be collaboratively implemented by hardware resources of the electronic device, such as computing resources such as a processor, and communication resources (such as for supporting various communication methods such as optical cables and cellular).

[0040] In a first aspect, an embodiment of the present application provides an audio processing method, which is applied to an electronic device.

[0041] The specific type of electronic device is not limited in the present application embodiment and can be configured according to actual needs. For example, the electronic device may include but is not limited to any of the following: a mobile phone, a tablet computer, a smart wearable device (such as a smart watch, smart glasses, etc.) or a computer, etc.

[0042] refer to Figure 1 The audio processing method may include but is not limited to Figure 1 S101 to S104 shown.

[0043] S101. Obtain the first audio to be output by the application at the current moment.

[0044] The application program here is not specifically limited in the embodiment of the present application, and can be configured according to actual needs. For example, the application program can be an application program for shooting games, or an application program for fighting games, etc.

[0045] The first audio refers to the audio currently to be output by the application.

[0046] The first audio here may include: target audio, the first audio may also include environmental audio (rain sound, thunder sound, etc.) The first audio may also include background audio.

[0047] Target audio refers to the audio that needs to be enhanced. Here, the target audio can be configured according to actual needs. For example, the target audio can include but is not limited to at least one of the following: gunshots, footsteps, explosions, etc.

[0048] S101 may be implemented as follows: the electronic device calls the first audio at the current moment to be output by the application program through the audio output interface.

[0049] S102: Identify the second audio and the background audio in the first audio at the current moment.

[0050] The second audio includes a target audio generated by simulating a target object, which may be an audio simulating gunshots or an audio simulating footsteps in a game application.

[0051] The embodiment of the present application does not limit the specific identification method, and can be configured according to actual needs. For example, it can be identified by a binary classification model. Of course, it can also be identified by other algorithms (such as Mel spectrum energy algorithm), which will not be listed here one by one.

[0052] The second audio refers to the audio in the first audio except the background audio, and may also be called content audio.

[0053] In a possible implementation, the first audio is input into a neural network binary classification model, and the first audio is recognized by the neural network binary classification model to obtain the second audio and background audio.

[0054] The neural network binary classification model may be a pre-trained neural network for identifying background audio and non-background audio. The specific training process is not described in detail here.

[0055] S103: Process the target audio in the second audio at the current moment to obtain the third audio at the current moment.

[0056] The target audio before processing has a first audio intensity, and the target audio after processing has a second audio intensity, and the second audio intensity is greater than the first audio intensity.

[0057] The second audio includes the target audio before processing, and the third audio includes the target audio after processing. The second audio and the third audio here are a general term. For audio at different times, there will be second audio at different times and third audio at different times.

[0058] The second audio intensity can be configured based on actual needs. For example, the second audio intensity can be configured based on the type of target audio. For different types of target audio, the same or different second audio intensities can be configured. For example, one second audio intensity can be configured for gunshots, and another second audio intensity can be configured for footsteps. In practice, different second audio intensities can be configured for different types of gunshots (pistols, sniper rifles, etc.), and different second audio intensities can be configured for different types of footsteps (walking, running, etc.).

[0059] The embodiment of the present application does not limit the processing method of the target audio in the second audio at the current moment, and can be configured according to actual needs. In the process of processing the second audio, the purpose is to enhance the target audio and keep other audio unchanged.

[0060] S104: Output the background audio and the third audio at the current moment.

[0061] The third audio and the background audio at the current moment are output. Here, the background audio and the third audio can be output through their original output channels.

[0062] It should be noted that, in a possible implementation, when the application is a shooting game, the target audio that needs to be enhanced may be the target audio of enemies or teammates in the game, that is, other objects other than oneself.

[0063] An embodiment of the present application provides an audio processing method, which includes: obtaining a first audio at a current moment to be output by an application; identifying a second audio and background audio in the first audio at the current moment; wherein the second audio includes a target audio generated by a simulated target object; processing the target audio in the second audio at the current moment to obtain a third audio at the current moment; outputting the background audio and the third audio at the current moment; wherein the target audio before processing has a first audio intensity, and the target audio after processing has a second audio intensity, and the second audio intensity is greater than the first audio intensity.

[0064] It can be seen that in practice, the target audio and other audio in the second audio are generally integrated together, and the target audio cannot be directly enhanced. In this embodiment, the second audio can be processed so that the target audio is enhanced from the first audio strength to the second audio strength, and other audio in the second audio is kept as unchanged as possible, thereby achieving the enhancement of the target audio in the second audio.

[0065] The audio processing method provided by this embodiment is described below through Example 1.

[0066] Example 1: Assume that the application is a shooting game application. The user plays the game on the mobile phone based on different operations. During the game, the audio needs to be output continuously. In different scenes, the output audio is different. In this embodiment, it is necessary to enhance the gunshots to be output of each object (enemy or teammate) in the game so that the user can determine the position of the enemy or teammate based on the gunshots.

[0067] The mobile phone obtains the first audio to be output at the current moment through the audio output interface. Assume that the first audio includes background audio, gunshots, rain, etc. First, the first audio is recognized to obtain the second audio (gunshots, rain) and background audio, and then the second audio is processed so that the gunshot in the second audio is enhanced from the first audio intensity to the second audio intensity, and the audio such as rain is kept at the original audio intensity as much as possible, so as to highlight the gunshot.

[0068] In this way, after the background audio and the third audio (gunshot and rain sound of the second audio intensity) are output, the user can clearly perceive the gunshot, so as to further perceive the position of the enemy or teammate who issued the gunshot.

[0069] In another embodiment, reference Figure 2 The audio processing method may include but is not limited to Figure 2 S201 to S206 shown.

[0070] S201. Obtain the first audio to be output by the application at the current moment.

[0071] The implementation of S201 may refer to the description of obtaining the first audio to be output by the application at the current moment in S101, which will not be described in detail here.

[0072] S202: Identify the second audio and the background audio in the first audio at the current moment.

[0073] The implementation of S202 may refer to the description of identifying the second audio and the background audio in the first audio at the current moment in S102, which will not be described in detail here.

[0074] S203. Perform feature extraction on the second audio at the current moment by using the first model to obtain audio features of the second audio at the current moment.

[0075] The first model is used for feature extraction, that is, converting audio information into feature vectors. The embodiment of the present application does not limit the specific structure of the first model, and can be configured according to actual needs. For example, the first model can be a neural network model or a large language model.

[0076] Here, the first model is only used for feature extraction, that is, which audio information is included in the second audio, and which audio features will be included in the audio features of the second audio after the first model extracts them.

[0077] For example, when the second audio includes the target audio, the audio features of the second audio include the audio features of the target audio. When the second audio includes the target audio and the ambient audio, the audio features of the second audio include the audio features of the target audio and the audio features of the ambient audio. Since the first model only has a feature extraction function but no recognition function, the audio features of the target audio and the audio features of the ambient audio here are actually fusion features.

[0078] S204: Determine a target enhancement parameter based at least on the audio features of the second audio at the current moment.

[0079] The target enhancement parameter is used to enhance the target audio in the second audio at the current moment from the first audio intensity to the second audio intensity. Exemplarily, the target enhancement parameter here may be a parameter in an equalization filter.

[0080] In a possible implementation, the target enhancement parameter is determined based on the audio features of the second audio at the current moment.

[0081] In another possible implementation, the target enhancement parameter is determined based on the audio features of the second audio at the current moment and the historical second audio and the historical third audio.

[0082] The types of target enhancement parameters are not limited here and can be configured according to actual needs. For example, the target enhancement parameters may include: center frequency (Frequency), gain (Gain) and quality factor (Q value). The target enhancement parameters may also include: slope and cutoff frequency, etc.

[0083] S205: Process the second audio at the current moment based on the target enhancement parameter to obtain the third audio at the current moment.

[0084] The electronic device configures an equalizing filter based on the target enhancement parameter, and processes the second audio through the equalizing filter to enhance the target audio in the second audio from the first audio intensity to the second audio intensity, while keeping other audio as unchanged as possible, thereby obtaining the third audio.

[0085] S206: Output the background audio and the third audio at the current moment.

[0086] The implementation of S206 may refer to the description of outputting the background audio and the third audio at the current moment in S104, which will not be described in detail here.

[0087] In this embodiment, the audio features of the second audio are first extracted, and then the target enhancement parameters are determined based on the audio features of the second audio. Since the target enhancement parameters are used to enhance the target audio in the second audio at the current moment from the first audio intensity to the second audio intensity, the target audio can be enhanced by processing the second audio based on the target enhancement parameters.

[0088] Next, the process of determining the target enhancement parameter based on at least the audio feature of the second audio at the current moment in S204 is described. The process may include but is not limited to the following method 1 or method 2.

[0089] Method 1: Determine the target enhancement parameters based on the prediction model;

[0090] Method 2: Determine the target enhancement parameters based on the matching degree.

[0091] Next, the process of determining the target enhancement parameters based on the prediction model in method 1 is described.

[0092] refer to Figure 3 As shown in the content, the audio processing method based on this process may include but is not limited to the following S301 to S307.

[0093] S301, obtaining the first audio to be output by the application at the current moment.

[0094] The implementation of S301 may refer to the description of obtaining the first audio to be output by the application at the current moment in S101, which will not be described in detail here.

[0095] S302: Identify the second audio and the background audio in the first audio at the current moment.

[0096] The implementation of S302 may refer to the description of identifying the second audio and the background audio in the first audio at the current moment in S102, which will not be described in detail here.

[0097] S303: extract features of the second audio at the current moment by using the first model to obtain audio features of the second audio at the current moment.

[0098] The implementation of S303 may refer to S203 in which the first model is used to extract features of the second audio at the current moment to obtain a description of the audio features of the second audio at the current moment, which will not be described in detail here.

[0099] S304: Input the audio features of the second audio at the current moment into the prediction model.

[0100] The audio features of the second audio here are the output of the first model, and the output of the first model is input into the prediction model. Therefore, in practice, the first model and the prediction model can be two independent models, or the prediction model and the first model can be integrated into one model, and feature extraction and prediction are achieved through the one model.

[0101] S305: Process the audio features through the prediction model, and output a target enhancement parameter for enhancing the target audio in the second audio at the current moment.

[0102] The prediction model is used to predict enhancement parameters corresponding to the audio features for enhancing the target audio in the second audio at the current moment.

[0103] The audio features input to the prediction model are the audio features of the second audio. The prediction model processes the audio features of the second audio and outputs a target enhancement parameter for enhancing the target audio in the second audio at the current moment.

[0104] The prediction model is a trained model used to predict enhancement parameters for enhancing the target audio in the second audio at the current moment corresponding to the audio feature. Different enhancement parameters are obtained for different audio features.

[0105] The embodiment of the present application does not limit the type of prediction model, and can be configured according to actual needs. For example, the prediction model can be a neural network model.

[0106] In this embodiment, the target enhancement parameter is determined by a prediction model, and the target enhancement parameter determined by the prediction model has the characteristic of high accuracy.

[0107] S306: Process the second audio at the current moment based on the target enhancement parameter to obtain the third audio at the current moment.

[0108] The implementation of S306 may refer to the description of processing the second audio at the current moment based on the target enhancement parameter in S205 to obtain the third audio at the current moment, which will not be described in detail here.

[0109] S307: Output the background audio and the third audio at the current moment.

[0110] The implementation of S307 may refer to the description of outputting the background audio and the third audio at the current moment in S104, which will not be described in detail here.

[0111] The prediction model is described below.

[0112] In a possible implementation, the prediction model is trained based on the following multiple sets of training data.

[0113] Multiple sets of training data include: audio features of target audio and first enhancement parameters, fused audio features of target audio and ambient audio and second enhancement parameters, fused audio features of target audio and scene audio and third enhancement parameters, fused audio features of target audio, ambient audio and scene audio and fourth enhancement parameters.

[0114] The first enhancement parameter is a reference enhancement parameter determined by a user for pure target audio.

[0115] The second enhancement parameter is a reference enhancement parameter determined by a user for the target audio and the ambient audio.

[0116] The third enhancement parameter is a reference enhancement parameter determined by the user for the target audio and the scene audio.

[0117] The fourth enhancement parameter is a reference enhancement parameter determined by the user for the target audio, scene audio, and environment.

[0118] The environment here may include but is not limited to various game environments, such as rain, thunder, sunny days, snowy days, storms, typhoons, sandstorms, etc. The scene here may include but is not limited to various game scenes, such as deserts, grasslands, mud, prairie, lakes, mountains, swamps, etc.

[0119] Furthermore, the target sounds can be classified in detail, and different reference enhancement parameters can be configured for different types of target sounds to train the model, so that the prediction model can predict different target enhancement parameters for different types of target sounds. For example, for different types of gunshots (pistols, sniper rifles, etc.), the prediction model can predict the target enhancement parameters corresponding to pistols and the target enhancement parameters corresponding to sniper rifles.

[0120] In this way, the trained prediction model can predict different enhancement parameters when the first audio includes different audio types.

[0121] In practice, training can also be performed using at least one of the above-mentioned multiple sets of training data. For example, when the training data only includes the audio features of the target audio and the first enhancement parameter, the target enhancement parameter predicted by the prediction model obtained through training is only applicable to the enhancement in the target audio scene. If the second audio also includes environmental audio, etc., the influence of the environmental audio on the enhancement parameter cannot be identified, and the target enhancement parameter adapted to the environment cannot be obtained.

[0122] The prediction model trained by the above-mentioned multiple sets of training data can achieve: when the second audio includes the target audio, the target enhancement parameters applicable to the target audio can be predicted; when the second audio includes the target audio and the scene audio, the target enhancement parameters applicable to the target audio and the scene audio can be predicted; when the second audio includes the target audio and the ambient audio, the target enhancement parameters applicable to the target audio and the ambient audio can be predicted; when the second audio includes the target audio, the scene audio and the ambient audio, the target enhancement parameters applicable to the target audio, the scene audio and the ambient audio can be predicted. In this way, different target enhancement parameters can be predicted for different scenes and different environments.

[0123] The embodiments of the present application do not limit the specific training process and can be configured according to actual needs. For example, the training process may include: multiple sets of training data may be input into the prediction model, the prediction model outputs enhancement parameters corresponding to the training data after being processed by the process, the loss is determined based on the enhancement parameters and the corresponding reference enhancement parameters, and the parameters of the prediction model are reversely adjusted based on the loss until the loss converges, thereby obtaining a trained prediction model.

[0124] In one possible implementation, the first model is a local large language model deployed on an electronic device running an application, the audio features of the second audio are embedding vectors of the large language model, and the embedding vector of the large language model at the current moment is input into the prediction model, wherein the embedding vector includes association information characterizing the target audio features, and the prediction model is used to output target enhancement parameters based on the association information.

[0125] The associated information here refers to environmental information, scene information, and so on.

[0126] Next, the process of determining the target enhancement parameter based on the matching degree in method 2 is described.

[0127] refer to Figure 4 As shown in the content, the audio processing method based on this process may include but is not limited to the following S401 to S407.

[0128] S401, obtaining the first audio to be output by the application at the current moment.

[0129] The implementation of S401 may refer to the description of obtaining the first audio to be output by the application at the current moment in S101, which will not be described in detail here.

[0130] S402: Identify the second audio and the background audio in the first audio at the current moment.

[0131] The implementation of S402 may refer to the description of identifying the second audio and the background audio in the first audio at the current moment in S102, which will not be described in detail here.

[0132] S403: extract features of the second audio at the current moment by using the first model to obtain audio features of the second audio at the current moment.

[0133] The implementation of S403 may refer to S203 in which the first model is used to extract features of the second audio at the current moment to obtain a description of the audio features of the second audio at the current moment, which will not be described in detail here.

[0134] S404: Determine a matching degree between an audio feature of the second audio at the current moment and a plurality of groups of reference audio features.

[0135] In advance, multiple common situations and reference enhancement parameters for each common situation are determined. The multiple common situations are embodied by audio features, that is, each situation corresponds to an audio feature of a second audio (reference audio feature), and each audio of the second audio corresponds to a reference enhancement parameter.

[0136] The electronic device determines the matching degree between the audio feature of the second audio at the current moment and the audio feature of the second audio in each case respectively, and obtains multiple matching degrees.

[0137] The embodiment of the present application does not limit the specific algorithm for determining the matching degree, and can be configured according to actual needs. For example, the matching degree can be determined according to the cosine similarity of the features or the adjacency algorithm.

[0138] The embodiment of the present application does not limit the determination method of the reference enhancement parameter in each case, and can be configured according to actual conditions. For example, it can be obtained according to actual calibration values, or it can be an empirical value, or a value obtained by an algorithm, etc.

[0139] S405: Determine the reference enhancement parameter corresponding to the reference audio feature with the highest matching degree as the target enhancement parameter.

[0140] The reference enhancement parameter is used to enhance the target audio in the second audio corresponding to the reference audio feature from the first audio intensity to the second audio intensity.

[0141] The reference enhancement parameter here is also used to enhance the target audio in the second audio corresponding to the reference audio feature from the first audio intensity to the second audio intensity.

[0142] For example, assuming that the target audio is a gunshot, 50 groups of reference enhancement parameters are configured for the second audio in the following common situation, wherein the first 12 groups of reference enhancement parameters can be described in the following situations 1 to 12.

[0143] Case 1: the second audio includes: gunshot; correspondingly, the reference audio feature 1 can represent the audio feature of the gunshot; the corresponding reference enhancement parameter is the reference enhancement parameter 1;

[0144] Case 2: The second audio includes: a pistol sound; correspondingly, the reference audio feature 2 can represent the audio feature of the pistol sound; and the corresponding reference enhancement parameter is the reference enhancement parameter 2;

[0145] Case 3: the second audio includes: a sniping sound; correspondingly, the reference audio feature 3 can represent the audio feature of the sniping sound; and the corresponding reference enhancement parameter is the reference enhancement parameter 3;

[0146] Case 4: the second audio includes: gunshot and rain sound; correspondingly, reference audio feature 4 can represent the audio features of the gunshot and rain sound; the corresponding reference enhancement parameter is reference enhancement parameter 4;

[0147] Case 5: The second audio includes: gunshot and thunder; correspondingly, the reference audio feature 5 can represent the audio features of the gunshot and thunder; the corresponding reference enhancement parameter is the reference enhancement parameter 5;

[0148] Case 6: The second audio includes: gunshot and wind sound; correspondingly, the reference audio feature 6 can represent the audio features of the gunshot and thunder sound; the corresponding reference enhancement parameter is the reference enhancement parameter 6;

[0149] Case 7, the second audio includes: a gunshot in a grass scene; correspondingly, the reference audio feature 7 can represent the audio feature of the gunshot in the grass scene; the corresponding reference enhancement parameter is the reference enhancement parameter 7;

[0150] Case 8, the second audio includes: a gunshot in a desert scene; correspondingly, the reference audio feature 8 can represent the audio feature of the gunshot in the desert scene; the corresponding reference enhancement parameter is the reference enhancement parameter 8;

[0151] Case 9, the second audio includes: gunshot and rain sound in a desert scene; correspondingly, reference audio feature 9 can represent the audio features of the gunshot and rain sound in the desert scene; the corresponding reference enhancement parameter is reference enhancement parameter 9;

[0152] Case 10: The second audio includes: gunshot and rain sound in a grass scene; correspondingly, the reference audio feature 10 can represent the audio features of the gunshot and rain sound in the grass scene; the corresponding reference enhancement parameter is the reference enhancement parameter 10;

[0153] Case 11: the second audio includes: gunshot and wind sound in a grass scene; correspondingly, the reference audio feature 11 can represent the audio features of the gunshot and rain sound in the grass scene; the corresponding reference enhancement parameter is the reference enhancement parameter 11;

[0154] Case 12: The second audio includes: gunshots and wind sounds in a desert scene; correspondingly, the reference audio feature 12 can represent the audio features of the gunshots and wind sounds in the desert scene; the corresponding reference enhancement parameter is the reference enhancement parameter 12.

[0155] The electronic device matches the audio features of the second audio at the current moment with 50 features including reference audio feature 1, reference audio feature 2, reference audio feature 3, etc., to obtain 50 matching degrees, and determines the reference enhancement parameter corresponding to the reference audio feature with the highest matching degree among the 50 matching degrees as the target enhancement parameter.

[0156] The implementation method based on matching degree has the characteristics of simple implementation logic, convenient implementation process and fast response.

[0157] S406: Process the second audio at the current moment based on the target enhancement parameter to obtain the third audio at the current moment.

[0158] The implementation of S406 may refer to the description of processing the second audio at the current moment based on the target enhancement parameter in S205 to obtain the third audio at the current moment, which will not be described in detail here.

[0159] S407: Output the background audio and the third audio at the current moment.

[0160] The implementation of S407 may refer to the description of outputting the background audio and the third audio at the current moment in S104, which will not be described in detail here.

[0161] In this way, the target enhancement parameters can be quickly determined by the matching method. When the target audio is enhanced, it has the characteristic of fast response because the model does not need to implement a complex processing process.

[0162] It should be noted that the second audio may also be directly associated with the reference enhancement parameter. In this way, when determining the target enhancement parameter, the matching degree between the second audios is directly determined, that is, the second audio at the current moment is matched with each historical second audio in the parameter library to obtain multiple matching degrees, and the reference enhancement parameter corresponding to the second audio with the highest matching degree is determined as the target enhancement parameter.

[0163] Next, the process of processing the target audio in the second audio at the current moment in S103 to obtain the third audio at the current moment is described.

[0164] The process may include but is not limited to implementation 1 or implementation 2 described below.

[0165] Implementation 1: Determine the third audio at the current moment based on the second audio at the current moment, the second audio and the third audio at the previous moment, the second audio and the third audio at the previous two moments, and the target enhancement parameter.

[0166] Implementation 2: Determine the third audio at the current moment based on the second audio at the current moment and the target enhancement parameter.

[0167] Next, the process of determining the third audio at the current moment based on the second audio at the current moment, the second audio and the third audio at the previous moment, the second audio and the third audio at the previous two moments, and the target enhancement parameter in implementation 1 is described.

[0168] refer to Figure 5 As shown in the content, the audio processing method based on implementation 1 may include but is not limited to the following S501 to 505.

[0169] S501: Obtain the first audio to be output by the application at the current moment.

[0170] The implementation of S501 may refer to the description of obtaining the first audio to be output by the application at the current moment in S101, which will not be described in detail here.

[0171] S502: Identify the second audio and the background audio in the first audio at the current moment.

[0172] The implementation of S502 may refer to the description of identifying the second audio and the background audio in the first audio at the current moment in S102, which will not be described in detail here.

[0173] S503: When the target enhancement parameter includes a first parameter, a second parameter, a third parameter, a fourth parameter and a fifth parameter, obtain the first parameter, the second parameter, the third parameter, the fourth parameter and the fifth parameter.

[0174] Among them, the first parameter is the input weight parameter used to enhance the target audio in the second audio; the second parameter is the weight parameter of the second audio at the previous moment, the third parameter is the weight parameter of the third audio at the previous moment, the fourth parameter is the weight parameter of the second audio at the previous two moments, and the fifth parameter is the weight parameter of the third audio at the previous two moments.

[0175] In practice, when determining the third audio at the current moment, the input and output of the previous two moments (the second audio and the third audio at the previous two moments) can be combined. In this way, the third audio at the current moment obtained can avoid stuttering and has good sound quality.

[0176] After obtaining the target enhancement parameters, the electronic device directly obtains the first parameter, the second parameter, the third parameter, the fourth parameter and the fifth parameter in the target enhancement parameters.

[0177] S504. Perform weighted sum processing on the first parameter, the second parameter, the third parameter, the fourth parameter and the fifth parameter, and the second audio at the current moment, the second audio and the third audio at the previous moment, and the second audio and the third audio at the previous two moments, so as to obtain the third audio at the current moment.

[0178] The weighted sum can be effectively fused. The weight coefficients, i.e., the first parameter, the second parameter, the third parameter, the fourth parameter, and the fifth parameter, are target enhancement parameters obtained based on the model. If the parameters need to be adjusted, the training process of the model can be adjusted. Of course, the enhancement parameters output by the model can also be manually adjusted directly.

[0179] S505: Output the background audio and the third audio at the current moment.

[0180] The implementation of S505 may refer to the description of outputting the background audio and the third audio at the current moment in S104, which will not be described in detail here.

[0181] In this embodiment, the third audio output at the current moment is combined with the second audio at the current moment, the second audio and third audio at the previous moment, and the second audio and third audio at the previous two moments. In this way, the third audio at the current moment can be integrated with the output of the previous moment and the previous two moments, avoiding the audio stuttering caused by too large a difference, so that the output is smoother and more fluent.

[0182] For implementation 2, the third audio at the current moment is determined based on the second audio at the current moment and the target enhancement parameter. In implementation 2, the target enhancement parameter is directly input into the equalization filter, and the second audio at the current moment is processed by the equalization filter to obtain the third audio at the current moment.

[0183] It can be seen that the implementation logic of implementation 2 is simple, the third audio at the current moment can be obtained quickly, and the response is fast.

[0184] Next, the process of processing the target audio in the second audio at the current moment in S103 to obtain the third audio at the current moment is described from the perspective of frequency.

[0185] When the target audio frequency belongs to multiple frequency bands, refer to Figure 6 As shown in the content, the process may include but is not limited to the following S601 to S606.

[0186] S601: Obtain the first audio to be output by the application at the current moment.

[0187] The implementation of S601 may refer to the description of obtaining the first audio to be output by the application at the current moment in S101, which will not be described in detail here.

[0188] S602: Identify the second audio and the background audio in the first audio at the current moment.

[0189] The implementation of S602 may refer to the description of identifying the second audio and the background audio in the first audio at the current moment in S102, which will not be described in detail here.

[0190] S603: For each frequency band of the multiple frequency bands, determine a target enhancement parameter for each frequency band.

[0191] The target enhancement parameter of each frequency band is used to enhance the target audio in the second audio at the current moment from the first audio intensity to the second audio intensity in the frequency band.

[0192] Here, one frequency band can be realized by one filter, and the target enhancement parameter of one frequency band is the filtering parameter of the filter.

[0193] Specifically, it includes several frequency bands, which are determined by the frequency of the defined target audio.

[0194] For determining the target enhancement parameter of each frequency band, the corresponding method is similar, and the specific method can refer to the above description of determining the target enhancement parameter, which will not be described here one by one.

[0195] S604: Process the second audio at the current moment respectively based on at least the target enhancement parameter of each frequency band to obtain the third audio at the current moment in each frequency band.

[0196] Among them, for each frequency band, the second audio of each frequency band at the current moment is processed to obtain the third audio of each frequency band at the current moment. The process is similar. For details, please refer to the above description of obtaining the third audio at the moment, which will not be repeated here.

[0197] S605: Superimpose the third audio at the current moment in each frequency band to obtain the third audio at the current moment.

[0198] The superposition here is based on the frequency correspondence. If the frequencies do not overlap, it is equivalent to splicing.

[0199] S606: Output the background audio and the third audio at the current moment.

[0200] The implementation of S606 may refer to the description of outputting the background audio and the third audio at the current moment in S104, which will not be described in detail here.

[0201] In this embodiment, the target audio can be enhanced through multiple frequency bands, which is applicable to various sound ranges and various types of target audio enhancement and has the characteristics of a wide range of applications.

[0202] Below, taking an application program as a shooting game as an example, the audio processing process provided in the embodiment of the present application is explained.

[0203] Currently, the sounds of gunfire and footsteps (equivalent to the above-mentioned target audio) in shooting games on tablets are relatively small, resulting in users being unable to clearly perceive the enemy's position.

[0204] The solutions in the related art include: A. Adjusting the volume of the overall music and sound effects in the game, but the disadvantage is that it cannot enhance the sound of gunfire and footsteps. B. Setting the equalization mode for sound playback in the device system settings, but the disadvantage is that it will process all played sounds, such as music, instead of specifically targeting games, and some tablets do not have an equalization mode option for sound playback, so it cannot be implemented.

[0205] In simple terms, the audio processing process can refer to Figure 7 The content shown includes:

[0206] S701. Input audio stream.

[0207] S702: Detection is performed using a background sound detection algorithm.

[0208] S703. Directly output the background sound.

[0209] S704: Input the target sound (gunshot or footsteps) for enhancement.

[0210] S705: Enhance the target sound using a sound enhancement algorithm.

[0211] S706: Output audio stream.

[0212] Next, the background sound detection process is described.

[0213] By training a neural network-based binary classification model, it is determined whether the current sound is background sound or target sound (equivalent to the second audio mentioned above).

[0214] For example, the detection process can refer to Figure 8 As shown in the figure, the input audio 801 is input into the neural network binary classification model 802 for recognition to obtain the judgment result 803. The neural network binary classification model 802 may include: a convolution layer 8021, an activation layer 8022, a pooling layer 8023, a semantic segmentation layer 8024, a global average pooling 8025, and a fully connected layer 8026.

[0215] The judgment result 803 may include background sound 8031 ​​or target sound 8032 .

[0216] Next, the sound enhancement process is described.

[0217] refer to Fig. 9 As shown in the figure, the process may include: inputting audio (e.g., a game sound dataset) into a local AI big model 901, the local AI big model 901 performs speech understanding 902 on the input audio to obtain an embedding vector 903, and the local AI big model 901 performs natural language processing 904 on the embedding vector 903 to obtain an output result 905. At the same time, the local AI big model 901 inputs the embedding vector 903 into a parameter prediction model 906, and obtains an output enhancement parameter 907 through processing by the parameter prediction model 906.

[0218] The local large model is fine-tuned and trained using the recorded game sound data, so that the original large model, which only has the ability to understand speech, can generate EQ filter parameters for sound enhancement. This parameter is input into the filter, thereby achieving the enhancement of the target sound using the filter.

[0219] Combined with the big model: The algorithm uses the embedding vector output by the speech recognition module in the big model as the input of the subsequent parameter prediction model. This vector contains the environmental information and semantic information of the audio. If someone is speaking, it also includes gender information, emotional information, etc.

[0220] The combination of the vector output by the large model and the parameter prediction model: Use the speech recognition module of the large model to extract features and use the features as the input of the parameter prediction model.

[0221] Due to the rich information contained in the vector output by the large model, the parameter prediction model can make predictions more accurate. The model mainly uses the environmental information, audio content information, scene category information, etc. in the vector, so that it can generate different enhancement parameters according to different scenes and different contents. For example, different types of enhancement parameters can be generated according to gunshots and footsteps, and different parameters can be generated according to different types of gunshots.

[0222] Apply the enhancement parameters output by the parameter prediction model to the filter and filter the game sound to enhance the game sound. Fig.10 As shown in the figure, the filtering process may include: inputting data of an input buffer, that is, input audio stream 1101, into filter 1102, and obtaining output audio stream 1003 after the filter performs processing. x represents the current input sampling point x(t), x1 represents the sampling point x(t-1) at the previous moment, and x2 represents the sampling point x(t-2) at the second moment before. Similarly, y represents the output at the current moment after filtering, y1 represents the output at the previous moment, and y2 represents the output at the previous two moments.

[0223] If the predicted weights are a0, a1, a2, b1, b2, the calculation can refer to the following formula 1.

[0224] y=a0×x+a1×x1+a2×x2+b1×y1+b2×y2 Formula 1;

[0225] In formula 1,

[0226] x represents the current input sampling point x(t), x1 represents the sampling point x(t-1) at the previous moment, and x2 represents the sampling point x(t-2) at the second moment before. Similarly, y represents the output at the current moment after filtering, y1 represents the output at the previous moment, and y2 represents the output at the previous two moments.

[0227] Next, the filter slides back to process the new input samples.

[0228] In a second aspect, in order to implement the above processing method, an audio processing device of an embodiment of the present application is deployed in a user terminal device. Fig.11 The structure of the audio processing device 110 is described with reference to the schematic diagram of the structure of the audio processing device 110 shown in FIG.

[0229] like Fig.11 As shown, the audio processing device 110 includes: an acquisition unit 1101, a recognition unit 1102, a processing unit 1103 and an output unit 1104. Among them:

[0230] An acquiring unit 1101 is used to acquire a first audio to be outputted by the application program at a current moment;

[0231] The recognition unit 1102 is used to recognize the second audio and the background audio in the first audio at the current moment; wherein the second audio includes the target audio generated by the simulated target object;

[0232] The processing unit 1103 is used to process the target audio in the second audio at the current moment to obtain the third audio at the current moment;

[0233] The output unit 1104 is used to output the background audio and the third audio at the current moment;

[0234] The target audio before processing has a first audio intensity, and the target audio after processing has a second audio intensity, and the second audio intensity is greater than the first audio intensity.

[0235] In some embodiments, the processing unit 1103 is further configured to:

[0236] The first model is used to extract features of the second audio at the current moment to obtain audio features of the second audio at the current moment; a target enhancement parameter is determined based at least on the audio features of the second audio at the current moment; the target enhancement parameter is used to enhance the target audio in the second audio at the current moment from the first audio intensity to the second audio intensity; the second audio at the current moment is processed based on the target enhancement parameter to obtain a third audio at the current moment.

[0237] In some embodiments, the processing unit 1103 is further configured to:

[0238] The audio features of the second audio at the current moment are input into the prediction model; the audio features are processed by the prediction model, and target enhancement parameters for enhancing the target audio in the second audio at the current moment are output; the prediction model is used to predict the enhancement parameters for enhancing the target audio in the second audio at the current moment corresponding to the audio features.

[0239] In some embodiments, the processing unit 1103 is further configured to:

[0240] The prediction model is trained based on the following multiple sets of training data: the multiple sets of training data include: audio features of the target audio and a first enhancement parameter, fused audio features of the target audio and ambient audio and a second enhancement parameter, fused audio features of the target audio and scene audio and a third enhancement parameter, fused audio features of the target audio, ambient audio and scene audio and a fourth enhancement parameter.

[0241] In some embodiments, the first model is a local large language model deployed on an electronic device running an application, the audio features of the second audio are the embedding vectors of the large language model, and the embedding vectors of the large language model at the current moment are input into the prediction model, wherein the embedding vector includes associated information characterizing the target audio features, and the prediction model is used to output target enhancement parameters based on the associated information.

[0242] In some embodiments, the processing unit 1103 is further configured to:

[0243] Determine the degree of match between the audio features of the second audio at the current moment and multiple groups of reference audio features; determine the reference enhancement parameters corresponding to the reference audio features with the highest degree of match as target enhancement parameters; wherein the reference enhancement parameters are used to enhance the target audio in the second audio corresponding to the reference audio features from the first audio intensity to the second audio intensity.

[0244] In some embodiments, the processing unit 1103 is further configured to:

[0245] Determine the third audio at the current moment based on the second audio at the current moment, the second audio and the third audio at the previous moment, the second audio and the third audio at the previous two moments, and the target enhancement parameter;

[0246] Alternatively, based on the second audio at the current moment and the target enhancement parameter, the third audio at the current moment is determined.

[0247] In some embodiments, the processing unit 1103 is further configured to:

[0248] When the target enhancement parameter includes a first parameter, a second parameter, a third parameter, a fourth parameter and a fifth parameter, the first parameter, the second parameter, the third parameter, the fourth parameter and the fifth parameter are obtained; the first parameter, the second parameter, the third parameter, the fourth parameter and the fifth parameter are weighted and processed correspondingly with the second audio at the current moment, the second audio and the third audio at the previous moment, and the second audio and the third audio at the previous two moments, so as to obtain the third audio at the current moment; wherein the first parameter is an input weight parameter for enhancing the target audio in the second audio; the second parameter is the weight parameter of the second audio at the previous moment, the third parameter is the weight parameter of the third audio at the previous moment, the fourth parameter is the weight parameter of the second audio at the previous two moments, and the fifth parameter is the weight parameter of the third audio at the previous two moments.

[0249] In some embodiments, when the frequency of the target audio belongs to multiple frequency bands, the processing unit 1103 is further configured to:

[0250] For each of the multiple frequency bands, a target enhancement parameter of each frequency band is determined; the target enhancement parameter of each frequency band is used to enhance the target audio in the second audio at the current moment from the first audio intensity to the second audio intensity in the frequency band; the second audio at the current moment is processed respectively based on at least the target enhancement parameter of each frequency band to obtain the third audio at the current moment in each frequency band; the third audio at the current moment in each frequency band is superimposed to obtain the third audio at the current moment.

[0251] It should be noted that the audio processing device provided in the embodiment of the present application includes the various units included, which can be implemented by a processor in an electronic device; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU, Central Processing Unit), a microprocessor (MPU, MicroProcessor Unit), a digital signal processor (DSP, Digital Signa l Processor) or a field programmable gate array (FPGA, Field-Programmable Gate Array), etc.

[0252] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0253] It should be noted that in the embodiment of the present application, if the above-mentioned processing method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the audio processing method of each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0254] In a third aspect, to implement the above-mentioned processing method, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the audio processing memory stores a computer program that can be run on the processor, and the audio processing processor implements the audio processing method provided in the above-mentioned embodiment when executing the audio processing program.

[0255] In some embodiments, the electronic device includes a first interface, a processor, and an output interface;

[0256] The first interface is used to: obtain the first audio to be output by the application at the current moment;

[0257] The processor is used to execute: identifying the second audio and the background audio in the first audio at the current moment; processing the target audio in the second audio at the current moment to obtain the third audio at the current moment; wherein the second audio includes the target audio generated by the simulated target object; the target audio before processing has a first audio intensity, and the target audio after processing has a second audio intensity, and the second audio intensity is greater than the first audio intensity;

[0258] The output interface is used to output background audio and the third audio at the current moment.

[0259] Combine the following Fig.12 The electronic device 120 shown is used to illustrate a structural diagram of the electronic device.

[0260] In one example, if Fig.12 As shown, the electronic device 120 includes: a processor 1201, at least one communication bus 1202, an interface 1203, at least one external communication interface 1204 and a memory 1205. The communication bus 1202 is configured to achieve connection and communication between these components. The interface 1203 can be connected to an external display screen, and the external communication interface 1204 can include a standard wired interface and a wireless interface.

[0261] The memory 1205 is configured to store instructions and applications executable by the processor 1201, and can also cache data to be processed or processed by the processor 1201 and various modules in the electronic device (for example, image data, audio data, voice communication data, and video communication data), which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0262] In a fourth aspect, an embodiment of the present application provides a storage medium, that is, a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the audio processing method provided in the first aspect of the above embodiment is implemented.

[0263] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the audio processing method provided in the first aspect of the above embodiment.

[0264] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0265] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in some embodiments" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The above-mentioned sequence numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments.

[0266] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0267] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the audio processing unit is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0268] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0269] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0270] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0271] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the audio processing method of each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0272] The above audio processing is only an implementation method of this application, but the protection scope of this application is not limited to this. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be included in the protection scope of this application. Therefore, the protection scope of this application should be based on the protection scope of the audio processing claim.

Claims

1. A method for processing audio, the method comprising: Get the first audio to be output by the application at the current moment; Identify the second audio and background audio in the first audio at the current moment; wherein the second audio includes the target audio generated by simulating the target object; Processing the target audio in the second audio at the current moment to obtain a third audio at the current moment; Outputting the background audio and the third audio at the current moment; The target audio before processing has a first audio intensity, and the target audio after processing has a second audio intensity, and the second audio intensity is greater than the first audio intensity.

2. The method according to claim 1, wherein the processing the target audio in the second audio at the current moment to obtain the third audio at the current moment comprises: Extracting features of the second audio at the current moment by using the first model to obtain audio features of the second audio at the current moment; Determining a target enhancement parameter based at least on the audio feature of the second audio at the current moment; The target enhancement parameter is used to enhance the target audio in the second audio at the current moment from the first audio intensity to the second audio intensity; The second audio at the current moment is processed based on the target enhancement parameter to obtain a third audio at the current moment.

3. The method according to claim 2, wherein determining the target enhancement parameter based at least on the audio feature of the second audio at the current moment comprises: Inputting the audio features of the second audio at the current moment into the prediction model; The audio features are processed by a prediction model to output target enhancement parameters for enhancing the target audio in the second audio at the current moment; the prediction model is used to predict enhancement parameters corresponding to the audio features for enhancing the target audio in the second audio at the current moment.

4. According to the method of claim 3, the prediction model is trained based on the following multiple sets of training data: The multiple sets of training data include: Audio features of the target audio and a first enhancement parameter, fused audio features of the target audio and ambient audio and a second enhancement parameter, fused audio features of the target audio and scene audio and a third enhancement parameter, fused audio features of the target audio, ambient audio and scene audio and a fourth enhancement parameter.

5. The method according to claim 3, The first model is a local large language model deployed on the electronic device running the application, the audio features of the second audio are the embedding vectors of the large language model, the embedding vectors of the large language model at the current moment are input into the prediction model, wherein the embedding vectors include associated information characterizing the target audio features, and the prediction model is used to output the target enhancement parameters based on the associated information.

6. The method according to claim 2, wherein determining the target enhancement parameter based at least on the audio feature of the second audio at the current moment comprises: Determining a matching degree between the audio feature of the second audio at the current moment and the multiple groups of reference audio features; Determining the reference enhancement parameter corresponding to the reference audio feature with the highest matching degree as the target enhancement parameter; The reference enhancement parameter is used to enhance the target audio in the second audio corresponding to the reference audio feature from the first audio intensity to the second audio intensity.

7. The method according to any one of claims 1 to 6, wherein the step of processing the target audio in the second audio at the current moment to obtain the third audio at the current moment comprises: Determine the third audio at the current moment based on the second audio at the current moment, the second audio and the third audio at the previous moment, the second audio and the third audio at the previous two moments, and the target enhancement parameter; or, Based on the second audio at the current moment and the target enhancement parameter, a third audio at the current moment is determined.

8. The method according to claim 7, wherein determining the third audio at the current moment based on the second audio at the current moment, the second audio and the third audio at the previous moment, the second audio and the third audio at the previous two moments, and the target enhancement parameter comprises: In a case where the target enhancement parameter includes a first parameter, a second parameter, a third parameter, a fourth parameter and a fifth parameter, acquiring the first parameter, the second parameter, the third parameter, the fourth parameter and the fifth parameter; Perform weighted sum processing on the first parameter, the second parameter, the third parameter, the fourth parameter and the fifth parameter, and the second audio at the current moment, the second audio and the third audio at the previous moment, and the second audio and the third audio at the previous two moments, so as to obtain the third audio at the current moment; Among them, the first parameter is an input weight parameter for enhancing the target audio in the second audio; the second parameter is the weight parameter of the second audio at the previous moment, the third parameter is the weight parameter of the third audio at the previous moment, the fourth parameter is the weight parameter of the second audio at the previous two moments, and the fifth parameter is the weight parameter of the third audio at the previous two moments.

9. According to the method according to any one of claims 1 to 5, when the frequency of the target audio belongs to multiple frequency bands, the processing of the target audio in the second audio at the current moment to obtain the third audio at the current moment comprises: For each of the plurality of frequency bands, determining a target enhancement parameter for each of the frequency bands; The target enhancement parameter of each frequency band is used to enhance the target audio in the second audio at the current moment from the first audio intensity to the second audio intensity in the frequency band; Processing the second audio at the current moment respectively based on at least the target enhancement parameter of each frequency band to obtain the third audio at the current moment in each frequency band; The third audio at the current moment in each frequency band is superimposed to obtain the third audio at the current moment.

10. An electronic device, comprising a first interface, a processor and an output interface; The first interface is used to: obtain the first audio to be output by the application at the current moment; The processor is configured to execute: identifying the second audio and background audio in the first audio at a current moment; The target audio in the second audio at the current moment is processed to obtain the third audio at the current moment; wherein, The second audio includes a target audio generated by simulating a target object; The target audio before processing has a first audio intensity, and the target audio after processing has a second audio intensity, and the second audio intensity is greater than the first audio intensity; The output interface is used to output the background audio and the third audio at the current moment.

Images