Audio data processing method and device, computer device and program

Abstract

In an embodiment of the present application, a voice data processing method and apparatus, and a computer device and program are disclosed, which include the steps of: obtaining, in a gaming voice mode, a signal processing result associated with a first pre-processing signal policy in an application layer of a business application (S101); based on the signal processing result, in the application layer, controlling the on / off state of a second optimization component in a second pre-processing signal policy in a terminal system layer, or controlling the on / off state of the second optimization component in the first pre-processing signal policy, wherein the first optimization component turned on in the first pre-processing signal policy is different from the second optimization component turned on in the second pre-processing signal policy (S102); and, when uplink voice data in a gaming voice mode of a first user is obtained, performing voice optimization on the uplink voice data based on the first optimization component turned on and the second optimization component turned on (S103).

Description

[Technical field]

[0001] This application claims priority to a Chinese patent application bearing application number 202110088769.3 and entitled "Audio data processing method, device, apparatus and readable storage medium" filed with the China Patent Office on January 22, 2021, the entire contents of which are incorporated herein by reference.

[0002] The present application relates to the field of computer technology, and in particular to a method and apparatus for processing audio data, and a computer device and program. [Background technology]

[0003] Currently, a user (e.g., user A) using a mobile terminal (also called a mobile terminal) can make a system call with another user (e.g., user B) through the system call mode. For example, user A can make a system call (i.e., make a phone call) with user B through the above-mentioned system call mode in a telephone call scenario.

[0004] Therefore, when the user A runs a game application (e.g., game application X) on a mobile terminal and makes a system call (i.e., makes a phone call) with user B, the application layer of the game application X often needs to share the system call mode of the terminal system layer of the mobile terminal. Based on this, when the mobile terminal indiscriminately turns on each signal processing unit (i.e., each voice optimization component) in the voice pre-processing scheme in the system call mode, the application layer and the terminal system layer can perform voice optimization on the collected voice of the user A by the signal processing unit of the same function type (i.e., the voice optimization component having the same function). As a result, there exists a phenomenon that the voice optimization component having the same function is repeatedly executed, which can increase the system overhead, and also can reduce the effect of voice optimization, since the voice optimization component having the same function is repeatedly executed, which can cause problems such as sound quality damage due to multiple overlapping processes. Summary of the Invention [Problem to be solved by the invention]

[0005] An object of the embodiments of the present application is to provide an audio data processing method and apparatus, a computer device, and a program that can improve the audio optimization effect in a game scenario. [Means for solving the problem]

[0006] According to one aspect of an embodiment of the present application, there is provided a method for processing audio data, the method being executed by a computer device, the method comprising: In a gaming audio mode, obtain a signal processing result associated with a first pre-signal processing policy in an application layer of a business (also referred to as a service) application, in which the first pre-signal processing policy includes at least one first optimization component; According to the signal processing result, in the application layer, control the on-off (open / close) state of a second optimization component in a second pre-processing policy in the terminal system layer, or control the on-off state of a first optimization component in a first pre-processing policy, in which the first optimization component turned on in the first pre-processing policy is different from the second optimization component turned on in the second pre-processing policy; and The method includes a step of acquiring uplink audio data in a game audio mode of a first user corresponding to a business application, and performing audio optimization on the uplink audio data in the game audio mode based on a first optimization component that is turned on in a first pre-processing signal policy and a second optimization component that is turned on in a second pre-processing signal policy.

[0007] According to one aspect of an embodiment of the present application, there is provided a method for processing audio data, the method being executed by a computer device, the method comprising: In a gaming audio mode, obtain a signal processing result associated with a first pre-processing policy in an application layer of a business application, where the first pre-processing policy includes at least one first optimization component; and The step includes controlling, in the application layer, an on-off state of a second optimization component in a second pre-processing policy in a terminal system layer based on the signal processing result, wherein the second pre-processing policy includes at least one second optimization component.

[0008] According to one aspect of an embodiment of the present application, there is provided a method for processing audio data, the method being executed by a computer device, the method comprising: In a gaming audio mode, obtain a signal processing result associated with a first pre-processing policy in an application layer of a business application, where the first pre-processing policy includes at least one first optimization component; and The present invention includes a step of controlling an on / off state of a second optimization component in a second pre-processing policy in a terminal system layer based on the signal processing result, or controlling an on / off state of a first optimization component in the first pre-processing policy, in which the first optimization component turned on in the first pre-processing policy is different from the second optimization component turned on in the second pre-processing policy.

[0009] According to one aspect of an embodiment of the present application, there is provided an audio data processing apparatus, the apparatus comprising: a processing result acquisition module for acquiring, in a game audio mode, a signal processing result associated with a first pre-processing policy in an application layer of a business application, in which the first pre-processing policy includes at least one first optimization component; a component control module, at the application layer, for controlling an on-off state of a second optimization component in a second pre-processing policy in the terminal system layer or controlling an on-off state of a first optimization component in a first pre-processing policy according to the signal processing result, where the first optimization component turned on in the first pre-processing policy is different from the second optimization component turned on in the second pre-processing policy; and The system includes a voice optimization module that acquires uplink voice data in a game voice mode of a first user corresponding to a business application, and performs voice optimization on the uplink voice data in the game voice mode based on a first optimization component that is turned on in a first pre-processing signal policy and a second optimization component that is turned on in a second pre-processing signal policy.

[0010] According to one aspect of an embodiment of the present application, there is provided an audio data processing apparatus, the apparatus comprising: a processing result acquisition module for acquiring, in a gaming audio mode, a signal processing result associated with a first front-end signal processing policy in an application layer of a business application, the first front-end signal processing policy including at least one first optimization component; and and a component control module, at the application layer, for controlling an on-off state of a second optimization component in a second front-end signal processing policy in a terminal system layer based on the signal processing result, wherein the second front-end signal processing policy includes at least one second optimization component.

[0011] According to one aspect of an embodiment of the present application, there is provided an audio data processing apparatus, the apparatus comprising: a processing result acquisition module for acquiring, in a gaming audio mode, a signal processing result associated with a first front-end signal processing policy in an application layer of a business application, the first front-end signal processing policy including at least one first optimization component; and The present invention includes a component control module that controls an on / off state of a second optimization component in a second pre-processing policy in a terminal system layer based on the signal processing result, or controls an on / off state of a first optimization component in the first pre-processing policy, wherein the first optimization component turned on in the first pre-processing policy is different from the second optimization component turned on in the second pre-processing policy.

[0012] According to one aspect of an embodiment of the present application, a computing device is provided, the computing device including a processor and a memory device, The processor is connected to a memory device, which stores a computer program, which when executed by the processor, causes the computing device to perform the methods provided in the embodiments of the present application.

[0013] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, which has a computer program stored therein, and the computer program is loaded and executed by a processor to cause a computing device having the processor to perform a method provided in the embodiment of the present application.

[0014] According to one aspect of the embodiments of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions, the computer instructions being stored in a computer readable storage medium, a processor of a computing device reading the computer instructions from the computer readable storage medium, and the processor executing the computer instructions, thereby causing the computing device to perform a method provided in the embodiments of the present application. Effect of the Invention

[0015] In an embodiment of the present application, in a game voice mode, a computer device (e.g., a mobile terminal) can control the on and off of a second optimization component (i.e., a voice optimization component in the second pre-pre ... In addition, when the computing device obtains the uplink audio data of the first user in the game audio mode, it can quickly perform audio optimization on the uplink audio data in the game audio mode based on the first optimization component and the second optimization component having different functions, thereby improving the audio optimization effect in the game scenario while reducing the loss of sound quality. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a diagram showing the configuration of a network architecture provided in an embodiment of the present application. [Diagram 2] FIG. 2 is a diagram showing the division of business modes provided in an embodiment of the present application. [Diagram 3]1 is a flowchart of audio data processing provided in an embodiment of the present application. [Figure 4] FIG. 2 is a diagram showing a scenario of voice interaction in a game scenario provided in an embodiment of the present application. [Diagram 5] 1 is a flowchart of a voice data processing method provided in an embodiment of the present application. [Figure 6] FIG. 2 illustrates a test list scenario provided in an embodiment of the present application. [Figure 7] FIG. 2 illustrates a scenario for determining an optimal signal processing policy associated with a sound quality parameter, as provided in an embodiment of the present application; [Figure 8] FIG. 2 illustrates a scenario for controlling the on and off of each audio optimization component in an audio pre-processing scheme provided in an embodiment of the present application. [Figure 9] FIG. 2 is a diagram showing a voice data processing method provided in an embodiment of the present application. [Figure 10] FIG. 2 illustrates a scenario of a resource setting interface provided in an embodiment of the present application. [Figure 11] 4 is a flowchart for providing double-talk services in different types of languages ​​provided in an embodiment of the present application; [Figure 12] 4 is a flowchart of another audio data processing method provided in an embodiment of the present application. [Figure 13] 4 is a flowchart of another audio data processing method provided in an embodiment of the present application. [Figure 14] FIG. 1 is a diagram showing the configuration of an audio data processing device provided in an embodiment of the present application. [Figure 15] FIG. 2 is a diagram showing the configuration of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Each embodiment provided in this application may be realized alone or in any combination to form a new embodiment, all of which fall within the scope of protection of this application.

[0018] Before describing the embodiments of the present application, some technical terms related to the present application will be introduced first.

[0019] 1. Game Voice Mode: This is a voice mode parallel to the media mode and the call mode, which is provided by the terminal system according to the voice needs and characteristics in game application scenarios.

[0020] 2.Sampling Rate: Sampling rate, also called sampling frequency, is the number of samples taken from a continuous signal per second to form a discrete signal, and is measured in Hertz (Hz). The higher the sampling rate, the more accurate the data. Commonly used sampling rates are 8khz, 16khz, 44.1khz, and 48khz.

[0021] 3. Bits of Samples: Bits of Samples is the value of a sample, a parameter for evaluating the wave (change) of an audio signal, and the number of binary bits of a digital audio signal used by a sound card when collecting and playing an audio file. Commonly used bit sizes of samples are 8 bits, 16 bits, and 32 bits, and the bit size of a sample is usually 16 bits on mobile phone platforms.

[0022] 4. Number of Channels: Number of channels is also called channel number, that is, the number of audio channels, and is usually related to hardware devices. Common channel numbers are mono and dual channel (stereo), mono audio can only come from one speaker, dual channel audio can come from two speakers, and generally, the division of labor between the two channels, left and right, can make the spatial effect feel better.

[0023] 5. Noise Suppression: Audio data collected via audio collection tools usually contains not only useful audio data such as human voices and music, but also unnecessary noise data such as environmental sounds. Noise suppression is a technology that removes or reduces the impact of noise on the overall effect of the audio as much as possible based on the characteristics of the audio data.

[0024] 6. Automatic Gain Control: This is an automatic control method for automatically adjusting the gain of an amplifier circuit according to the signal strength, and is mainly used to increase the signal strength of valid voice data.

[0025] 7. Acoustic Echo Cancellation: Echo refers to the sound reflected or repeated by sound waves, or the sound that is collected by the receiving end and retransmitted after the voice signal is transmitted and played over the network, and reaches the speaker. Removing these sounds using signal processing algorithms or devices is called echo cancellation.

[0026] 8. Dynamic Range Compression: Dynamic range compression is a dynamic range control that dynamically adjusts the output amplitude of the audio. For example, when the volume is high, the volume is appropriately suppressed, and when the volume is low, the volume is appropriately increased, so that the volume can always be controlled within one appropriate range. It is usually used to control the output power of the audio so that the speaker does not distort the sound and the sound can be heard clearly even when playing at a low volume.

[0027] 9. Front-end Processing (also called pre-processing): Audio front-end processing technology is a technology that processes the original audio data before encoding and transmission, so that the processed audio signal can better reflect the essential characteristics of the audio. Audio front-end processing technology usually mainly includes technologies such as noise suppression, echo cancellation, and automatic gain control.

[0028] Please refer to Fig. 1, which is a configuration diagram of a network architecture provided in an embodiment of the present application. As shown in Fig. 1, the network architecture may include a business server 2000 and a group of user terminals.

[0029] The user terminal group may include one or more user terminals, and the number of user terminals is not limited here. As shown in Fig. 1, the multiple user terminals here may specifically include user terminal 3000a, user terminal 3000b, user terminal 3000c, ..., user terminal 3000n. As shown in Fig. 1, each of user terminals 3000a, ..., user terminal 3000n is connected to business server 2000 via a network, so that each user terminal in the user terminal group can perform data interaction with business server 2000 via the network connection.

[0030] The business server 2000 shown in FIG. 1 may be an independent physical server, or may be a server group or distributed system consisting of multiple physical servers, and may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), big data, and artificial intelligence platforms.

[0031] For ease of understanding, in the embodiment of the present application, one user terminal (e.g., the user terminal used by user A) may be selected as the target user terminal from the group of user terminals shown in Fig. 1. For example, in the embodiment of the present application, the user terminal 3000a shown in Fig. 1 may be selected as the target user terminal, and the target user terminal may be integrated with a business application having a voice data processing function (e.g., a voice data collection and playback function). Among them, the business application may specifically include an application client having a voice data collection and playback function, such as an entertainment client (e.g., a game client), a social client, an office client, a live broadcast client, etc. Among them, the target user terminal (e.g., the user terminal 3000a) may specifically include a mobile terminal having a voice data processing function, such as a smartphone, a tablet computer, a notebook computer, a wearable device, etc. Among them, in the embodiments of the present application, application types corresponding to entertainment clients (e.g., game clients) may be collectively referred to as game types, and application types corresponding to social clients (e.g., clients such as QQ and WeChat), office clients (e.g., enterprise clients), live broadcast clients, etc. may be collectively referred to as non-game types.

[0032] As can be understood, for a user terminal (e.g., the aforementioned target user terminal) on which the above-mentioned business application is executed, different business modes can be adaptively selected based on different application types of the business application, thereby performing different types of voice interaction business in different business scenarios.

[0033] As shown in FIG. 2, the business modes here may specifically include a system media mode 21a (also referred to as a “media mode”), a system call mode 21b (also referred to as a “voice call mode” or a “call mode”), and a game voice mode 21c.

[0034] As can be seen, the target user terminal can set the business mode of the business application to the system media mode 21a by default when the user (i.e., the first user) does not need voice double talk (i.e., does not need to perform voice interaction). Optionally, the target user terminal can further adaptively select different business modes according to different application types of the business application by intelligently identifying the application type of the business application that needs to perform voice interaction when the user (i.e., the first user) needs double talk (i.e., needs to perform voice interaction). For example, when the application type of the business application is a game type, the business mode of the business application is set to the game voice mode 21c, and when the application type of the business application is a non-game type, the business mode of the business application is set to the system call mode 21b.

[0035] The system media mode 21a may be used to instruct the target user terminal to play audio data of a music or video program for the current user (ie, the first user).

[0036] The system call mode 21b may be used by the target user terminal in a non-game scenario to instruct the current user (i.e., the first user) to make a system call with another user (i.e., a second user, which may be a user selected by the first user in a business application and requested to make a system call) via the system call mode 21b.

[0037] The game voice mode 21c may be used to instruct the target user terminal to provide a new voice interaction service in a game scenario, for example, in the game voice mode 21c, the user (i.e., the first user) can directly conduct a game voice call with another user (e.g., a third user, which may be a game user in the same game camp (team) as the first user in the game voice mode 21c).

[0038] Unlike chat software, the audio environment in game scenarios is more complex, and both the audio call quality and the media playback sound quality need to be considered. Existing user terminals only provide a system call mode 21b suitable for call scenarios and a system media mode 21a suitable for music playback scenarios, and do not consider scenarios that integrate both, and the application effect is not good in game scenarios. Therefore, how to improve the system media playback sound quality while ensuring the experience of two-way audio communication is the key to improving the game user audio experience. As shown in FIG. 2, in this application, the smart mobile terminal provides a game audio mode 21c parallel to the system call mode 21b and the system media mode 21a, thereby achieving the purpose of optimizing the audio service in the game application scenario. The mobile application can also select the most suitable audio mode according to its own business characteristics and requirements, thereby quickly and efficiently integrating the audio service in the application. The game audio mode 21c is an audio mode applied to game services or game scenarios, and the mode aims to optimize the player's audio experience for the game scenario. In the game audio mode 21c, effective optimization measures are taken for each stage of audio collection, processing, setting, etc. for game application scenarios, so as to provide game players with smooth game audio and high-quality game sound effects.

[0039] It should be noted that, since this application is mainly used in the game audio industry, the audio mode newly proposed in this application is called a "game audio mode". As can be understood, the "game audio mode" can be applied not only to game scenarios, but also to business scenarios having the same or similar audio processing requirements as game scenarios, for example, to all audio business scenarios that need to consider audio call quality and media playback sound quality at the same time, such as live video broadcast scenarios, video conference scenarios, etc., but this application is not limited thereto.

[0040] Taking a game scenario as an example, the sound of the game mainly goes through two stages: a sound data collection stage and a sound data playback stage. Please refer to Figure 3, which is a diagram showing the flow of sound data processing.

[0041] The audio data collection phase involves, in sequence:

[0042] 1. Audio signal collection: Audio is usually input to a mobile phone through a microphone. The microphone is mainly used to convert sound waves into a voltage signal, and then convert the continuous voltage signal into a digital signal that can be processed by a computer by sampling the voltage signal. The main indicators that can affect the quality of the collected audio signal include the sampling rate, the number of bits of the sample, and the number of channels. The higher the sampling rate, the more audio samples are obtained per second, and the higher the quality of the final audio.

[0043] 2. Audio signal front-end processing (pre-processing): Pre-processing the data collected by the microphone to improve the quality of the audio data. The front-end processing process usually includes audio processing algorithms such as echo cancellation, automatic gain control, and noise suppression.

[0044] 3. Encoding: Voice encoding refers to compressing the collected digital voice signal for digital transmission by lowering the transmission code rate.

[0045] 4. Transmission: Transmission refers to transmitting the encoded voice data to a designated voice server via a network so that other users can listen to the user's voice data through the server.

[0046] The audio data playback stage involves, in sequence:

[0047] 5. Receiving voice data: Obtaining other users' voice data from the designated voice server for use in the playback process.

[0048] 6. Decoding: Decoding is the process corresponding to encoding, that is, decoding the received encoded voice data to convert the digital signal into an analog signal.

[0049] 7. Post-processing: After decoding, the audio data may have phenomena such as interruptions in the reproduced audio data due to problems such as packet loss, which may affect the audio playback effect. Therefore, the decoded audio data needs to be adjusted and optimized through a post-processing process.

[0050] 8. Playback: The audio data is played back through devices such as speakers or earphones.

[0051] It can be seen that the target user terminal activates a coordination mechanism between the application layer and the terminal system layer in the game voice mode, and according to the coordination mechanism, can adaptively select one voice optimization component having the same optimization function from among the voice optimization component of the application layer and the voice optimization component of the terminal system layer according to the comparison result of the algorithm (i.e., the signal processing result). In this way, when the application layer and the terminal system layer work in coordination according to the above-mentioned coordination mechanism, in the game scenario, the uplink voice data of the current user (i.e., the above-mentioned first user) collected in real time can be processed with real-time human voice, thereby improving the voice optimization effect of the uplink voice data, and improving the experience of voice interaction between game users.

[0052] For ease of understanding, please refer to FIG. 4, which is a diagram showing a scenario of voice interaction in a game scenario provided in an embodiment of the present application. As can be understood, in the game scenario, the application type of the business application in the user terminal 10a shown in FIG. 4 may be the above-mentioned game type. At this time, the user terminal 10a can switch the business mode of the business application from the system media mode to the game voice mode, so that the user 1 shown in FIG. 4 (i.e., the above-mentioned first user) can make a game voice call with the user 2 shown in FIG. 4 (i.e., the above-mentioned third user) in the game voice mode.

[0053] As can be understood, the user terminal 10a shown in FIG. 4 may be the above-mentioned target user terminal having a voice data processing function. As can be understood, when the user 1 shown in FIG. 4 needs to perform voice interaction with the user terminal 20a corresponding to the user 2 shown in FIG. 4 through the user terminal 10a, the user terminal 10a can perform voice optimization on the collected voice of the user 1 in advance, so that the voice of the user 1 after voice optimization can be transmitted to the user terminal 20a corresponding to the user 2 as a target voice optimization result corresponding to the uplink voice data, so that the user terminal 20a can play the voice of the user 1 after voice optimization through the speaker shown in FIG. 4. Meanwhile, as can be understood, in the embodiment of the present application, the voice of the user 1 collected by the microphone of the user terminal 10a (corresponding to the voice control in the application display interface 100a shown in FIG. 4) can be collectively referred to as a voice uplink signal, that is, the voice frame obtained by performing frequency spectrum analysis on the voice signal collected by the microphone can be collectively referred to as uplink voice data. As can be further understood, in the embodiments of the present application, the voice of user 1 after voice optimization played by the speaker of the user terminal 20a (corresponding to the playback control in the application display interface 200a shown in FIG. 4) may be collectively referred to as a voice downlink signal, i.e., the voice frames of the voice signal propagated to the speaker and played may be collectively referred to as downlink voice data, and similarly, the voice of another user (e.g., user 2) after voice optimization played by the speaker of the user terminal 10a may also be collectively referred to as a voice downlink signal.

[0054] As can be understood, in the game scenario, the user terminal 10a shown in Fig. 4 can collect the voice of the user 1 in real time (i.e., the above-mentioned voice uplink signal) by a microphone, and then obtain uplink voice data corresponding to the voice uplink signal, so that the application layer and the terminal system layer of the above-mentioned business application jointly negotiate to obtain an optimal signal processing policy, for example, the first optimization component turned on in the application layer and the second optimization component turned on in the terminal system, which are jointly negotiated, can perform voice optimization on the uplink voice data of the user 1. Note that the second optimization component here is different from the first optimization component, and the first optimization component turned on in the application layer (i.e., the first optimization component turned on in the first pre-processing policy) has the same optimization function as the second optimization component turned off in the second pre-processing policy, and the second optimization component turned on in the terminal system has the same optimization function as the first optimization component turned off in the above-mentioned first pre-processing policy. It can be understood that in the embodiments of the present application, the voice optimization components in the first pre-processing policy may be collectively referred to as the first optimization components, and the voice optimization components in the second pre-processing policy may be collectively referred to as the second optimization components. In this way, the cooperation mechanism between the application layer and the terminal system layer can effectively ensure that the voice optimization components having the same optimization functions are executed at the application layer or the terminal system layer, so as to effectively avoid the problem of the voice optimization components having the same optimization functions being executed redundantly.

[0055] The voice optimization here refers to the front-end processing process described above, and mainly includes, but is not limited to, acoustic echo cancellation (AEC), noise suppression (NS), and automatic gain control (AGC).

[0056] In the process of echo cancellation (AEC), echo mainly refers to the phenomenon that the voice transmitted by a speaker (e.g., the aforementioned user 1) to another person (e.g., the aforementioned user 2) through his / her communication device (e.g., the aforementioned user terminal 10a) returns to his / her earpiece again. The echo cancellation according to the embodiment of the present application mainly refers to the processing scheme for the target user terminal (e.g., the aforementioned user terminal 10a) to cancel such echo by a certain computing device (e.g., an echo cancellation component).

[0057] In the process of noise suppression (NS), noise mainly refers to the voice signals collected by a target user terminal (e.g., the aforementioned user terminal 10a) that originate from objects other than the speaker (e.g., the aforementioned user 1). On this basis, noise suppression in the embodiments of the present application mainly refers to the processing scheme for the target user terminal (e.g., the aforementioned user terminal 10a) to remove such noise by a certain computing device (e.g., a noise suppression component).

[0058] The process of performing automatic gain control (AGC) refers to a processing scheme in which a target user terminal (e.g., the aforementioned user terminal 10a) uses a predetermined computing device (e.g., a gain control component) to intelligently adjust the energy of a voice signal based on the human hearing sensitivity range, thereby allowing the voice signal to be better perceived.

[0059] It can be understood that, when the user terminal 10a selects to turn on the first optimization component 11 (e.g., echo suppression component) in the first pre-signal processing policy in the application layer according to the comparison result of the algorithm, the second optimization component 21 having the same optimization function as the first optimization component 11 (e.g., echo suppression component) in the terminal system layer needs to be turned off synchronously, and the second optimization component 21 may be an echo suppression component in the second pre-signal processing policy in the terminal system layer that is controlled and turned off by the application layer. This means that in the embodiment of the present application, when the target user terminal collects the voice of the user 1 in the game scenario (i.e., the uplink voice data of the first user) through the microphone in real time, it only needs to execute one voice optimization component with the same optimization function in the application layer or the terminal system layer, thereby ensuring that the function of the voice optimization component with the same optimization function is executed only once, so as to fundamentally solve the problem of the waste of computing resources caused by the function of the voice optimization component with the same optimization function being executed multiple times.

[0060] As can be understood, as shown in FIG. 4, when a user 1 (here, mainly referring to a certain game user, for example, game user A) selects to turn on the voice control shown in FIG. 4 in the application display interface 100a shown in FIG. 4 (i.e., when the voice control is in the off state, the business mode of the business application executed in the user terminal 10a can be the system media mode), the terminal used by the user 1 (for example, the user terminal 10a shown in FIG. 4) can determine the application type of the business application executed in the user terminal 10a, so that the business mode of the business application can be switched from the system media mode to the game voice mode, so that the user terminal 10a can collect and optimize the voice of the user 1 in real time in the game voice mode to obtain the voice of the user 1 after voice optimization shown in FIG. 4. Furthermore, the user terminal 10a can broadcast the voice of the user 1 after voice optimization to other teammates in the camp where the user 1 is located (for example, user 2, the user 2 can be another game user in the same camp as the user 1). In this way, a terminal (e.g., user terminal 20a shown in FIG. 4) used by another teammate (e.g., user 2) in the same camp can play back the received voice of user 1 after voice optimization when turning on the playback control shown in FIG. 4 (e.g., turning on the speaker in the game scenario).

[0061] Optionally, when the above-mentioned user terminal 10a detects that the application type of the above-mentioned business application belongs to a non-game type (e.g., a social type), the user terminal 10a (i.e., the above-mentioned target user terminal) can intelligently switch the business mode of the business application from a system media mode to a system call mode, and perform a second type of voice call business in the system call mode, and the second type of voice call business can be a voice interaction business corresponding to the system call type in the non-game scenario. For example, in a social scenario, the user 1 shown in FIG. 4 can be allowed to send a system call request corresponding to the system call type to the user 2 shown in FIG. 4, so that the terminal corresponding to the user 2 shown in FIG. 4 (i.e., the above-mentioned user terminal 20a) can establish a system communication channel between the user 1 and the user 2 to perform a system call through the system communication channel in response to the system call request (e.g., when the user 2 confirms receipt of the call request of the user 1).

[0062] In addition, for a specific implementation method of controlling the on / off of the second optimization component in the second pre-processing policy in the terminal system layer at the above-mentioned application layer of the target user terminal (for example, the user terminal 10a shown in Figure 4), and performing voice optimization on the uplink voice data of the first user, reference can be made to the embodiments corresponding to the following Figures 5 to 15.

[0063] Referring to FIG. 5, FIG. 5 is a flowchart of a voice data processing method provided in an embodiment of the present application. The method is executed by a computer device, for example, the method may be executed by a user terminal (for example, the above-mentioned target user terminal, which may be the user terminal 10a in the embodiment corresponding to the above-mentioned FIG. 4), or may be executed by a business server (for example, the business server 2000 shown in the above-mentioned FIG. 1), or may be executed by cooperation between the user terminal and the business server. For ease of understanding, the present embodiment will be described taking the method executed by a user terminal as an example. Among them, the voice data processing method may include at least one of the following steps S101 to S103.

[0064] Step S101: In a game audio mode, obtain a signal processing result associated with a first pre-processing policy in an application layer of a business application.

[0065] Specifically, the target user terminal can obtain a sound quality indicator of a business application in a game sound mode, and set sound quality parameters of the business application (the sound quality parameters here may include, but are not limited to, a sound sampling rate and a number of sound channels) according to the sound quality indicator of the business application. In addition, the target user terminal can obtain a terminal type of a terminal to which the business application belongs, and find a test type that matches the terminal type in a test list associated with the business application. When a test type that matches the terminal type is found in the test list, the target user terminal can obtain a first test processing result obtained by adopting a first pre-processing policy and a second test processing result obtained by adopting a second pre-processing policy from the test list according to the sound quality parameters. Wherein, the first pre-processing policy is a pre-processing policy in the application layer of the business application. The second pre-processing policy is a pre-processing policy in a terminal system corresponding to the test terminal type. In addition, the target user terminal can determine an optimal signal processing policy associated with the sound quality parameter from among the first pre-fixed signal processing policy and the second pre-fixed signal processing policy based on the first test processing result and the second test processing result, and can set the determined optimal signal processing policy as the signal processing result associated with the first pre-fixed signal processing policy.

[0066] It can be understood that before performing step S101, the target user terminal can further load the system resource packet of the business application in advance in the target user terminal, so that the system resource data of the business application can be obtained after performing analysis processing on the system resource packet, and thus the target user terminal can initialize the business mode of the business application to the system media mode according to the system resource data after initialization processing after performing initialization processing. It can be understood that in the embodiment of the present application, after completing the initialization processing of the system resource data, the system media mode can be entered by default, so that the application display interface of the business application can be output in the system media mode according to the system resource data after initialization processing, so that the multimedia data (such as video frame data and audio frame data) of the business application can be output in the application display interface. As can be understood, the application display interface may include a voice control for instructing the first user to start a voice interaction task, and in this way, when the first user needs to have a voice interaction with another user, the first user can select to trigger the voice control that is currently in an off state, thereby allowing the target user terminal to automatically detect the application type of the task application that starts the voice interaction task according to the voice-on operation performed by the first user on the voice control.

[0067] As can be understood, when the target user terminal determines that the application type of the business application starting the voice interaction business belongs to a game type, the target user terminal can determine that the current business scenario is a game scenario, and can switch the business mode of the business application executed on the target user terminal from the system media mode to the game voice mode, for example, in the game scenario, generate a first voice call command associated with the game type, and switch the business mode of the business application executed on the target user terminal from the system media mode to the game voice mode according to the first voice call command, and then perform the first type of voice call business in the game voice mode. Wherein, as can be understood, the target user terminal can refine some sound quality parameters associated with the above-mentioned voice double talk needs according to the sound quality indicators of the business application in the game voice mode. For example, the target user terminal can allow the first user (i.e., the user using the target user terminal) to set the voice sampling rate and the number of voice channels corresponding to the target user terminal in the game voice mode.

[0068] Optionally, as can be understood, if the target user terminal determines that the application type of the business application currently initiating the voice interaction business belongs to a non-game type, the target user terminal can determine that the current business scenario is a non-game scenario, and can switch the business mode of the business application executed on the target user terminal from the system media mode to the system voice mode, for example, in the non-game scenario, generate a second voice call command associated with the non-game type, and switch the business mode of the business application executed on the target user terminal from the system media mode to the system voice mode based on the second voice call command, so that in the system voice mode, voice interaction can be performed with another user (e.g., the above-mentioned second user) and the above-mentioned second type of voice call business can be performed.

[0069] Therefore, it can be seen that the game voice mode and the system voice mode provided in the embodiment of the present application are two kinds of business modes for providing different types of voice call services in the above-mentioned voice double-talk scenario. In this way, the target user terminal can determine the application type of the business application that initiates the voice call business, and when the application type is a game type, can intelligently enter the above-mentioned game voice mode to perform the above-mentioned first type of voice call business. Optionally, the target user terminal can also intelligently enter the above-mentioned system voice mode to perform the above-mentioned second type of voice call business when the application type is a non-game type.

[0070] It can be understood that the target user terminal according to the embodiment of the present application includes, but is not limited to, the mobile terminal having the above-mentioned voice data processing function. Therefore, the setting of the voice sampling rate corresponding to the target user terminal according to the embodiment of the present application can mainly include the setting of the uplink sampling rate and the downlink sampling rate of the terminal. In addition, the setting of the number of voice channels of the target user terminal according to the embodiment of the present application mainly refers to the setting of the number of voice channels, for example, the number of channels can be set to dual channels according to the sound quality index of the target user terminal.

[0071] As can be understood, the voice sampling rate (e.g., uplink sampling rate and downlink sampling rate) here may be the number of times that the recording component of the target user terminal samples the voice signal within a unit sampling period. The voice sampling rate may include, but is not limited to, 4kHZ, 8kHZ, and 48kHZ. As can be understood, the magnitude of the voice sampling rate value can be used to reflect the authenticity and naturalness of the user's voice that the recording component can restore.

[0072] For example, in the game voice mode, the first user can perform voice interaction through the voice double talk service provided by the voice interaction system in the target user terminal. That is, when the microphone in the target user terminal collects the voice signal of the first user (i.e., can be used to collect the voice of the first user), in the game voice mode, the microphone can sample and obtain the uplink voice data of the first user in the game voice mode by performing frequency spectrum analysis on the voice signal of the first user based on the above-mentioned uplink sampling rate. At this time, the target user terminal can perform voice optimization on the uplink voice data, and transmit the voice signal of the first user after voice optimization (i.e., the voice of the first user after the above-mentioned voice optimization) to another communication destination (for example, the terminal corresponding to the above-mentioned third user), so that the other communication destination can play the voice of the first user after the above-mentioned voice optimization through their respective speakers. Similarly, the target user terminal may further be used to receive a voice-optimized voice signal of a third user transmitted from another communication destination, so that after performing frequency spectrum analysis on the voice-optimized voice signal of the third user according to the above-mentioned downlink sampling rate, downlink voice data can be obtained for transmission to the speaker of the target user terminal, and thus, when the downlink voice data is played by the speaker of the target user terminal, the voice of the third user after voice optimization can be restored as accurately as possible for the first user.

[0073] For ease of understanding, please refer to FIG. 6, which is a diagram illustrating a scenario of a test list provided in an embodiment of the present application. Wherein, the test terminal type in the test list 301a may be a test type corresponding to one or more test terminals. As can be understood, the test terminal type here may include, but is not limited to, one or more brand models (also referred to as models), and optionally, the test terminal type here may further include the system type, system version, etc. of the terminal environment system corresponding to these models.

[0074] As can be understood, when a developer corresponding to a business application develops a business application having the above-mentioned game audio mode, the developer can first integrate and install the business application into a test terminal corresponding to each known model for testing, thereby testing the performance using multiple types of pre-amplifier signal processing policies (e.g., the first pre-amplifier signal processing policy in the above-mentioned application layer and the second pre-amplifier signal processing policy in the above-mentioned terminal system layer) on the test terminal corresponding to these known models, and obtaining the optimization performance of each voice optimization component in the application layer and the optimization performance of the voice optimization component of the corresponding function in the terminal system layer under specific sound quality parameters of the same known model (i.e., the same test type) through testing.

[0075] For ease of understanding, the present invention takes as an example that the type of the test terminal is a model of a single brand, and n models (where n is a positive integer) under the brand may be test types T1, ..., test types Tn as shown in Fig. 6. For example, test type T1 may be model 1 of brand A, and test type T2 may be model 2 of brand A, and by analogy therewith, test type Tn may be model n of brand A.

[0076] As can be understood, in order to test the optimized performance of each audio test component in the application layer and each audio test component in the terminal system layer under different sound quality parameters of the same model, when the set sound quality parameter is sound quality parameter D1 (for example, uplink sound sampling rate 8 kHz, downlink sound sampling rate 8 kHz, monaural number of left channel, etc.), the developer can use the first pre-processing signal policy in the application layer and the second pre-processing signal policy in the terminal system layer to test and obtain the sound test effect when the sound quality parameter D1 of the test terminal, whose model is test type T1, is used.

[0077] For example, in the process of testing performance, the test processing result obtained by performing test optimization on uplink voice data (e.g., uplink voice data R1 for testing performance) using each voice optimization component in the above-mentioned application layer (e.g., first optimization components such as a first echo cancellation component for performing echo cancellation, a first noise suppression component for performing noise suppression, and a first gain control component for performing gain adjustment) may be the test processing result of the application layer associated with the sound quality parameter D1 shown in Figure 6. Also, the test processing result obtained by performing test optimization on the same uplink voice data R1 using each voice optimization component in the above-mentioned terminal system layer (e.g., second optimization components such as a second echo cancellation component for performing echo cancellation, a second noise suppression component for performing noise suppression, and a first gain control component for performing gain adjustment) may be the test processing result of the terminal system layer associated with the sound quality parameter D1 shown in Figure 6.

[0078] For ease of understanding, in the embodiment of the present application, the test processing result corresponding to the first echo cancellation component in the application layer may be the first test result 31a shown in FIG. 6, and at this time, the voice optimization component having the same optimization function as the above-mentioned first echo cancellation component in the terminal system layer may be the above-mentioned second echo cancellation component, and as shown in FIG. 6, the test processing result obtained after performing echo cancellation on the uplink voice data R1 using the second echo cancellation component may be the second test result 31b shown in FIG. 6.

[0079] Also, for example, the test processing result corresponding to the first noise suppression component in the application layer may be the first test result 32a shown in Figure 6, and at this time, the voice optimization component having the same optimization function as the above-mentioned first noise suppression component in the terminal system layer may be the above-mentioned second noise suppression component, and as shown in Figure 6, the test processing result obtained after performing noise suppression on the uplink voice data R1 using the second noise suppression component may be the second test result 32b shown in Figure 6.

[0080] Also, for example, the test processing result corresponding to the first gain control component in the application layer may be the first test result 33a shown in Fig. 6. At this time, the voice optimization component having the same optimization function as the above-mentioned first gain control component in the terminal system layer may be the above-mentioned second gain control component, and the test processing result obtained after performing gain adjustment on the uplink voice data R1 using the second noise suppression component as shown in Fig. 6 may be the second test result 33b shown in Fig. 6.

[0081] Also, based on this, as shown in FIG. 6, when the set sound quality parameter is sound quality parameter D2 (for example, uplink sound sampling rate 8 kHz, downlink sound sampling rate 16 kHz, monaural number of left channel, etc.), the developer can also use the first pre-processing signal policy in the application layer and the second pre-processing signal policy in the terminal system layer to test and obtain the sound test effect when the sound quality parameter D2 of another test terminal, the model of which is test type Tn.

[0082] For example, in the process of testing another performance, the test processing result obtained by performing test optimization on another uplink voice data (e.g., uplink voice data R2 for testing the performance) using each voice optimization component in the above-mentioned application layer (e.g., first optimization components such as a first echo cancellation component for performing echo cancellation, a first noise suppression component for performing noise suppression, and a first gain control component for performing gain adjustment) may be the test processing result of the application layer associated with the sound quality parameter D2 shown in Figure 6. Also, the test processing result obtained by performing test optimization on the same uplink voice data R2 using each voice optimization component in the above-mentioned terminal system layer (e.g., second optimization components such as a second echo cancellation component for performing echo cancellation, a second noise suppression component for performing noise suppression, and a first gain control component for performing gain adjustment) may be the test processing result of the terminal system layer associated with the sound quality parameter D2 shown in Figure 6.

[0083] Similarly, for ease of understanding, in the embodiment of the present application, the test processing result corresponding to the first echo cancellation component in the application layer may be the first test result 34a shown in FIG. 6, and at this time, the voice optimization component having the same optimization function as the above-mentioned first echo cancellation component in the terminal system layer may be the above-mentioned second echo cancellation component, and as shown in FIG. 6, the test processing result obtained after performing echo cancellation on the uplink voice data R2 using the second echo cancellation component may be the second test result 34b shown in FIG. 6.

[0084] Similarly, the test processing result corresponding to the first noise suppression component in the application layer may be the first test result 35a shown in Figure 6, and at this time, the voice optimization component having the same optimization function as the above-mentioned first noise suppression component in the terminal system layer may be the above-mentioned second noise suppression component, and as shown in Figure 6, the test processing result obtained after using the second noise suppression component to perform noise suppression on the uplink voice data R1 may be the second test result 35b shown in Figure 6.

[0085] Similarly, the test processing result corresponding to the first gain control component in the application layer may be the first test result 36a shown in Fig. 6. At this time, the voice optimization component having the same optimization function as the above-mentioned first gain control component in the terminal system layer may be the above-mentioned second gain control component, and as shown in Fig. 6, the test processing result obtained after using the second gain control component to perform gain adjustment on the uplink voice data R2 may be the second test result 36b shown in Fig. 6.

[0086] In short, the target user terminal can obtain the test processing results under different sound quality parameters of each known model through pre-testing after the performance test is performed on the first optimization component in the application layer and the second optimization component having the same optimization function in the terminal system layer. Then, the developer can construct and obtain the test list 301a shown in FIG. 6 according to the above-mentioned test terminal type, sound quality parameters, the test processing results of the application layer and the test processing results of the terminal system layer. In this way, when the first user needs to perform the above-mentioned voice interaction business with other users in the above-mentioned game voice mode, the test type that matches the terminal type can be quickly and intelligently looked up in the test list 301 according to the terminal type of the terminal to which the business application currently belongs (i.e., the above-mentioned target user terminal). For example, the target user terminal can quickly obtain the first test processing result obtained by adopting the above-mentioned first pre-pre-processing policy and the second test processing result obtained by adopting the above-mentioned second pre-pre-processing policy from the test list 301a based on the sound quality parameters (for example, the sound quality parameters set by the first user are the above-mentioned sound quality parameters D1) set by the current user (i.e., the first user) based on the sound quality index of the business application. As can be understood, at this time, after comparing the test results of the voice optimization components having the same optimization functions, the target user terminal can quickly determine the optimal signal processing policy corresponding to each optimization function under the specific sound quality parameters of the current terminal type (i.e., the current model) from the above-mentioned first pre-pre-processing policy and the second pre-pre-processing policy based on the voice test effect of the voice optimization component having the same optimization function after comparing the test results of the voice optimization components having the same optimization function, and then the determined optimal signal processing policy can be the signal processing result associated with the first pre-pre-processing policy, so as to subsequently execute the below-described step S102.

[0087] For example, when the sound quality parameter set by the first user (i.e., the current user) based on the sound quality index is the sound quality parameter D1 shown in Fig. 6, the first test processing result here may specifically include the first test result 31a corresponding to the first echo cancellation component (i.e., the AEC component in the application layer), the first test result 32a corresponding to the first noise suppression component (i.e., the NS component in the application layer), and the first test result 33a corresponding to the first gain control component (i.e., the AGC component in the application layer). Among them, the second test processing result here may specifically include the second test result 31b corresponding to the second echo cancellation component (i.e., the AEC component in the terminal system layer), the second test result 32b corresponding to the second noise suppression component (i.e., the NS component in the terminal system layer), and the second test result 33b corresponding to the second gain control component (i.e., the AGC component in the terminal system layer).

[0088] Similarly, when the sound quality parameter set by the first user based on the sound quality index is another sound quality parameter shown in Fig. 6 (for example, the above-mentioned sound quality parameter D2), the first test processing result obtained by adopting the first pre-processing signal policy and the second test processing result obtained by adopting the second pre-processing signal policy can be quickly obtained from the above-mentioned test list 301a. Note that detailed explanations of the first test processing result associated with the other sound quality parameters and the second test processing result associated with the other sound quality parameters are omitted here.

[0089] Step S102: Based on the signal processing result, the application layer controls the on / off state of the second optimization component in the second pre-processing policy in the terminal system layer, or controls the on / off state of the first optimization component in the first pre-processing policy.

[0090] Optionally, the application layer controls the on and off of the second optimization component in the second pre-processing policy in the terminal system layer according to the signal processing result, or controls the on and off of the first optimization component in the first pre-processing policy, in which the first pre-processing policy includes at least one first optimization component, and the second pre-processing policy includes at least one second optimization component. In some embodiments, the number of first optimization components in the first pre-processing policy and the number of second optimization components in the second pre-processing policy are the same, for example, three. In addition, each first optimization component in the first pre-processing policy has a second optimization component in the second pre-processing policy with the same optimization function, and correspondingly, each second optimization component in the second pre-processing policy has a first optimization component in the first pre-processing policy with the same optimization function.

[0091] Optionally, the first optimization component turned on in the first pre-processing policy is different from the second optimization component turned on in the second pre-processing policy.

[0092] Optionally, a first optimization component that is turned on in the first pre-precedence signal processing policy has the same optimization function as a voice optimization component that is turned off in the second pre-precedence signal processing policy, and a second optimization component that is turned on in the second pre-precedence signal processing policy has the same optimization function as a first optimization component that is turned off in the first pre-precedence signal processing policy.

[0093] Specifically, the target user terminal determines the second optimization component to be turned on in the second pre-prefix signal processing policy based on the signal processing result, and also determines the second optimization component to be turned off in the second pre-prefix signal processing policy. For example, the target user terminal can activate an interaction mechanism between the application layer and the terminal system layer of the terminal to which the business application belongs based on the above-mentioned signal processing result, and can control the on and off of the second optimization component in the second pre-prefix signal processing policy in the terminal system layer in the application layer based on the interaction mechanism. Also, the target user terminal can set the second optimization component turned off in the second pre-prefix signal processing policy as the first interaction component in the application layer, and turn on the first optimization component having the same optimization function as the first interaction component in the first pre-prefix signal processing policy. Also, the target user terminal can set the second optimization component turned on in the second pre-prefix signal processing policy as the second interaction component in the application layer, and turn off the first optimization component having the same optimization function as the second interaction component in the first pre-prefix signal processing policy.

[0094] As can be understood, the voice optimization algorithm of the first optimization component in the above-mentioned first pre-signal processing policy may include at least one of the following: a first echo cancellation algorithm for performing echo cancellation at the application layer (the first optimization component corresponding to the first echo cancellation algorithm is the above-mentioned first echo cancellation component), a first noise suppression algorithm for performing noise suppression at the application layer (the first optimization component corresponding to the first noise suppression algorithm is the above-mentioned first noise suppression component), and a first gain control algorithm for performing gain adjustment at the application layer (the first optimization component corresponding to the first gain control algorithm is the above-mentioned first gain control component). Similarly, the voice optimization algorithm of the second optimization component in the above-mentioned second pre-signal processing policy may include at least one of the following: a second echo cancellation algorithm for performing echo cancellation at a terminal system layer (the second optimization component corresponding to the second echo cancellation algorithm is the above-mentioned second echo cancellation component), a second noise suppression algorithm for performing noise suppression at a terminal system layer (the second optimization component corresponding to the second noise suppression algorithm is the above-mentioned second noise suppression component), and a second gain control algorithm for performing gain adjustment at a terminal system layer (the second optimization component corresponding to the second gain control algorithm is the above-mentioned second gain control component).

[0095] At this time, the signal processing result obtained by the target user terminal may be obtained by the following steps: obtain a first echo cancellation result corresponding to a first echo cancellation algorithm from the first test processing result, obtain a second echo cancellation result corresponding to a second echo cancellation algorithm from the second test processing result, and select an optimal echo cancellation algorithm from the first echo cancellation algorithm and the second echo cancellation algorithm based on the first echo cancellation result and the second echo cancellation result, and set the optimal echo cancellation algorithm as a first optimal signal processing policy associated with the sound quality parameter. The target user terminal may further obtain a first noise suppression result corresponding to a first noise suppression algorithm from the first test processing result, obtain a second noise suppression result corresponding to a second noise suppression algorithm from the second test processing result, and select an optimal noise suppression algorithm from the first noise suppression algorithm and the second noise suppression algorithm based on the first noise suppression result and the second noise suppression result, and set the optimal noise suppression algorithm as a second optimal signal processing policy associated with the sound quality parameter. Furthermore, the target user terminal can obtain a first gain control result corresponding to the first gain control algorithm from the first test processing result, obtain a second gain control result corresponding to the second gain control algorithm from the second test processing result, and select an optimal gain control algorithm from the first gain control algorithm and the second gain control algorithm based on the first gain control result and the second gain control result, and set the optimal gain control algorithm as a third optimal signal processing policy associated with the sound quality parameter. Furthermore, the target user terminal can set the first optimal signal processing policy, the second optimal signal processing policy, and the third optimal signal processing policy as signal processing results associated with the first pre-conditioning signal processing policy.

[0096] For ease of understanding, please refer to FIG. 7, which is a diagram showing a scenario for determining an optimal signal processing policy associated with a sound quality parameter provided in an embodiment of the present application. Wherein, the first test processing result 401a shown in FIG. 7 may be the application layer test processing result associated with the sound quality parameter D1 in the embodiment corresponding to the above-mentioned FIG. 6 (i.e., the first test processing result associated with the sound quality parameter D1). Wherein, the test result 41a in the first test processing result 401a may be the first test result 31a in the embodiment corresponding to the above-mentioned FIG. 6, that is, the test result 41a shown in FIG. 7 may be the first echo cancellation result corresponding to the first echo cancellation algorithm obtained from the first test processing result 401a. Wherein, the test result 42a in the first test processing result 401a may be the first test result 32a in the embodiment corresponding to the above-mentioned FIG. 6, that is, the test result 42a shown in FIG. 7 may be the first noise suppression result corresponding to the first noise suppression algorithm obtained from the first test processing result 401a. Wherein, the test result 43a in the first test processing result 401a may be the first test result 33a in the embodiment corresponding to the above-mentioned FIG. 6, that is, the test result 43a shown in FIG. 7 may be the first gain control result corresponding to the first gain control algorithm obtained from the first test processing result 401a.

[0097] The second test processing result 401b as shown in Fig. 7 may be the test processing result of the terminal system layer associated with the sound quality parameter D1 in the embodiment corresponding to Fig. 6 described above (i.e., the second test processing result associated with the sound quality parameter D1). Wherein, the test result 41b in the second test processing result 401b may be the second test result 31b in the embodiment corresponding to Fig. 6 described above, that is, the test result 41b shown in Fig. 7 may be the second echo cancellation result corresponding to the second echo cancellation algorithm obtained from the second test processing result 401b. Wherein, the test result 42b in the second test processing result 401b may be the second test result 32b in the embodiment corresponding to Fig. 6 described above, that is, the test result 42b shown in Fig. 7 may be the second noise suppression result corresponding to the second noise suppression algorithm obtained from the second test processing result 401b. Wherein, the test result 43b in the second test processing result 401b may be the second test result 33b in the embodiment corresponding to FIG. 6 described above, that is, the test result 43b shown in FIG. 7 may be the second gain control result corresponding to the second gain control algorithm obtained from the second test processing result 401b.

[0098] As can be seen, the specific process of the target user terminal determining the first optimal signal processing policy based on the first echo cancellation result (for example, the test result 41a shown in the above-mentioned FIG. 7) and the second echo cancellation result (for example, the test result 41b shown in the above-mentioned FIG. 7) can be described as follows: the target user terminal can obtain a first echo cancellation result corresponding to the first echo cancellation algorithm from the first test processing result, and obtain a second echo cancellation result corresponding to the second echo cancellation algorithm from the second test processing result; and the target user terminal can perform a first comparison between the optimization quality corresponding to the first echo cancellation result and the optimization quality corresponding to the second echo cancellation result, and obtain a first comparison result. Wherein, as can be seen, as shown in FIG. 7, the target user terminal can determine the voice test effect of the first optimization component and the second optimization component having the same optimization function based on the test result 41a and the test result 41b. For example, the voice test effect V11 at the application layer of the first echo cancellation component in the above-mentioned application layer and the voice test effect V12 at the terminal system layer of the second echo cancellation component in the above-mentioned terminal system layer can be compared to determine whether the test result 41a is better than the test result 41b. In this way, if the first comparison result shown in FIG. 7 indicates that the test result 41a is better than the test result 41b, it indicates that the optimized quality corresponding to the first echo cancellation result is better than the optimized quality corresponding to the second echo cancellation result, and the first echo cancellation algorithm in the first pre-processing policy may be the first optimal signal processing policy associated with the sound quality parameter; conversely, if the first comparison result shown in FIG. 7 indicates that the test result 41b is better than the test result 41a, it indicates that the optimized quality corresponding to the second echo cancellation result is better than the optimized quality corresponding to the first echo cancellation result, and the second echo cancellation algorithm in the second pre-processing policy may be the first optimal signal processing policy associated with the sound quality parameter.Optionally, as can be understood, if test result 41a is the same as test result 41b, the first echo cancellation algorithm in the first pre-processing signal policy or the second echo cancellation algorithm in the second pre-processing signal policy can be the first optimal signal processing policy.

[0099] It can be seen that the specific process of the target user terminal determining the second optimal signal processing policy based on the first noise suppression result (for example, the test result 42a shown in the above-mentioned FIG. 7) and the second noise suppression result (for example, the test result 42b shown in the above-mentioned FIG. 7) can be described as follows: the target user terminal can obtain a first noise suppression result corresponding to the first noise suppression algorithm from the first test processing result, and obtain a second noise suppression result corresponding to the second noise suppression algorithm from the second test processing result; and the target user terminal can perform a second comparison between the optimization quality corresponding to the first noise suppression result and the optimization quality corresponding to the second noise suppression result, and obtain a second comparison result. Wherein, it can be seen that, as shown in FIG. 7, the target user terminal can determine the voice test effect of each voice optimization component having the same optimization function based on the test result 42a and the test result 42b. For example, by comparing the voice test effect V21 at the application layer of the first noise suppression component in the above-mentioned application layer with the voice test effect V22 at the terminal system layer of the second noise suppression component in the above-mentioned terminal system layer, it can be determined whether the test result 42a is better than the test result 42b. In this way, if the second comparison result shown in FIG. 7 indicates that test result 42a is better than test result 42b, it indicates that the optimized quality corresponding to the first noise suppression result is better than the optimized quality corresponding to the second noise suppression result, and the first noise suppression algorithm in the first pre-processing policy may be the second optimal signal processing policy associated with the sound quality parameter; conversely, if the second comparison result shown in FIG. 7 indicates that test result 42b is better than test result 42a, it indicates that the optimized quality corresponding to the second noise suppression result is better than the optimized quality corresponding to the first noise suppression result, and in this case, the target user terminal may be the second noise suppression algorithm in the second pre-processing policy as the second optimal signal processing policy associated with the sound quality parameter.Similarly, optionally, if test result 42a is the same as test result 42b, the first noise suppression algorithm in the first pre-processing policy or the second noise suppression algorithm in the second pre-processing policy may be the second optimal signal processing policy.

[0100] As can be seen, the specific process of the target user terminal determining the third optimal signal processing policy based on the first gain control result (for example, the test result 43a shown in the above-mentioned FIG. 7) and the second gain control result (for example, the test result 43b shown in the above-mentioned FIG. 7) can be described as follows: the target user terminal can obtain a first gain control result corresponding to the first gain control algorithm from the first test processing result, and obtain a second gain control result corresponding to the second gain control algorithm from the second test processing result; and the target user terminal can perform a third comparison between the optimization quality corresponding to the first gain control result and the optimization quality corresponding to the second gain control result, and obtain a third comparison result. Wherein, as can be seen, as shown in FIG. 7, the target user terminal can determine the voice test effect of each voice optimization component having the same optimization function based on the test result 43a and the test result 43b. For example, the voice test effect V31 at the application layer of the first gain control component in the above-mentioned application layer and the voice test effect V32 at the terminal system layer of the second gain control component in the above-mentioned terminal system layer can be compared to determine whether the test result 43a is better than the test result 43b. In this way, if the third comparison result shown in Fig. 7 indicates that the test result 43a is better than the test result 43b, it indicates that the optimization quality corresponding to the first gain control result is better than the optimization quality corresponding to the second gain control result, and the first gain control algorithm in the first pre-processing policy may be the third optimal signal processing policy associated with the sound quality parameter, and conversely, if the second comparison result shown in Fig. 7 indicates that the test result 43b is better than the test result 43a, it indicates that the optimization quality corresponding to the second gain control result is better than the optimization quality corresponding to the first gain control result, and the second gain control algorithm in the first pre-processing policy may be the third optimal signal processing policy associated with the sound quality parameter. Similarly, optionally, if the test result 43a is the same as the test result 43b, the first gain control algorithm in the first pre-processing policy or the second gain control algorithm in the second pre-processing policy may be the third optimal signal processing policy.

[0101] Optionally, as can be understood, if the target user terminal does not find a test type matching the current terminal type in the test list (e.g., the above-mentioned test list 301a), the target user terminal determines that the current terminal type belongs to a new model, and when obtaining uplink voice data (e.g., the above-mentioned voice data R3) of the first user through the microphone in the game voice mode, performs voice optimization on the uplink voice data (e.g., the above-mentioned voice data R3) according to a first pre-prefix signal processing policy (i.e., performs real-time voice optimization), obtains a first voice optimization result in real time, and performs voice optimization on the uplink voice data (e.g., the above-mentioned voice data R3) according to a second pre-prefix signal processing policy (i.e., performs real-time voice optimization), obtains a second voice optimization result in real time, and then, based on the first voice optimization result and the second voice optimization result, the target user terminal can determine an optimal signal processing policy associated with the sound quality parameter from the first pre-prefix signal processing policy and the second pre-prefix signal processing policy, and the determined optimal signal processing policy can be the signal processing result associated with the first pre-prefix signal processing policy.

[0102] It can be understood that, when the target user terminal determines that its model is not a new model, it can perform real-time voice optimization on the above-mentioned voice data of the first user obtained in real time by each voice optimization control in the application layer in the above-mentioned game scenario, and obtain a first voice optimization result corresponding to each voice optimization control in the application layer.Similarly, the target user terminal can also perform real-time voice optimization on the above-mentioned voice data of the first user obtained in real time by each voice optimization control in the terminal system layer in the above-mentioned game scenario, and obtain a second voice optimization result corresponding to each voice optimization control in the terminal system layer.Among them, the specific implementation manner of the target user terminal comparing the voice optimization effect of the voice optimization component with the same optimization function can refer to the above description of the voice test effect of the voice optimization component with the same optimization function, and the detailed description is omitted here.

[0103] The first optimization component in the first pre-signal processing policy may include at least one of the following, namely, the first echo cancellation component, the first noise suppression component, and the first gain control component. The second optimization component in the second pre-signal processing policy may include at least one of the following, namely, the second echo cancellation component, the second noise suppression component, and the second gain control component. Among them, the first echo cancellation component and the second echo cancellation component are both used to perform echo cancellation, the first noise suppression component and the second noise suppression component are both used to perform noise suppression, and the first gain control component and the second gain control component are both used to perform gain adjustment.

[0104] In order to avoid the voice optimization components with the same optimization functions being executed redundantly in the application layer and the terminal system layer, the embodiments of the present application propose the following: in the game voice mode, a corresponding switch is provided in the aforementioned application layer to control the on and off of each part (i.e., each voice optimization component) in the pre-signal processing scheme, thereby ensuring that the voice optimization components with the same optimization functions are executed in the application layer or the terminal system layer; in this way, in the real-time voice optimization (i.e., real-time human voice optimization) process in the game scenario, the overall performance consumption of the human voice optimization flow can be reduced, so as to improve the voice interaction experience in the game scenario; and the embodiments of the present application can further avoid the waste of terminal system resources (e.g., the computing resources of the CPU (Central Processing Unit)) in the game voice mode, so as to effectively save the power consumption of the terminal.

[0105] For ease of understanding, refer to FIG. 8, which is a diagram illustrating a scenario of controlling on and off of each voice optimization component in a voice pre-processing scheme provided in an embodiment of the present application. It can be understood that the voice pre-processing scheme here can be the related processing performed by the above-mentioned target user terminal to improve the clarity, loudness, etc. of uplink voice data, for example, the related processing can include echo cancellation, noise suppression, automatic gain, etc. For ease of understanding, a specific process of controlling on and off of each voice optimization component in the voice pre-processing scheme in the application layer is described here by taking as an example that the voice pre-processing scheme includes the above-mentioned first pre-processing policy and the above-mentioned second pre-processing policy.

[0106] The application layer 601a shown in Fig. 8 may be the application layer of the business application described above, and the audio pre-processing scheme corresponding to the application layer 601a may be the first pre-processing policy described above, and thus the first optimization component in the first pre-processing policy at least includes an audio optimization component 61a, an audio optimization component 62a, and an audio optimization component 63a shown in Fig. 8. It can be understood that the audio optimization component 61a shown in Fig. 8 may be a first echo cancellation component for performing the above-mentioned echo cancellation, similarly, the audio optimization component 62a shown in Fig. 8 may be a first noise suppression component for performing the above-mentioned noise suppression, similarly, the audio optimization component 63a shown in Fig. 8 may be a first gain control component for performing the above-mentioned gain adjustment.

[0107] The terminal system layer 602a shown in Fig. 8 may be a system layer below the terminal to which the above-mentioned business application belongs (i.e., the above-mentioned target user terminal), and the voice pre-processing scheme corresponding to the terminal system layer 602a may be the above-mentioned second pre-processing policy, thus the second optimization component in the second pre-processing policy includes at least a voice optimization component 61b, a voice optimization component 62b and a voice optimization component 63b shown in Fig. 8. It can be understood that the voice optimization component 61b shown in Fig. 8 may be a second echo cancellation component for performing the above-mentioned echo cancellation, similarly, the voice optimization component 62b shown in Fig. 8 may be a second noise suppression component for performing the above-mentioned noise suppression, similarly, the voice optimization component 63b shown in Fig. 8 may be a second gain control component for performing the above-mentioned gain adjustment.

[0108] As can be understood, in order to avoid redundant execution of each voice optimization component with the same function, the embodiment of the present application proposes the following: in the application layer 601a shown in FIG. 8, a corresponding switch is provided to help the application layer 601a control the on and off of each voice optimization component in the terminal system layer 602a shown in FIG. 8.

[0109] For example, the switch K11 in the application layer 601a shown in Fig. 8 may be used to control the voice optimization component 61a shown in Fig. 8, and the switch K12 in the application layer may be used to control the voice optimization component 61b in the terminal system layer 602a shown in Fig. 8. As can be understood, since the voice optimization component 61a in the application layer 601a shown in Fig. 8 and the voice optimization component 61b in the terminal system layer 602a shown in Fig. 8 have the same optimization function, the target user terminal can select whether to control on (or off) the voice optimization component 61b in the second front-end signal processing policy in the terminal system layer 602a in the application layer 601a based on the cooperation mechanism (also called the negotiation mechanism) between the application layer 601a and the terminal system layer 602a. For example, as shown in FIG. 8, the target user terminal can control the application layer 601a to turn on the voice optimization component 61a in the first pre-precedence signal processing policy, that is, the target user terminal may generate a first control command for controlling the business switch 64a to turn off switch K11 and turn off switch K12, and at this time, the first control command can be used to instruct the target user terminal to set the second optimization component (e.g., the voice optimization component 61b in FIG. 8) that is turned off in the second pre-precedence signal processing policy as the first coordination component, and to turn on a first optimization component (e.g., the voice optimization component 61a shown in FIG. 8) that has the same optimization function as the first coordination component in the first pre-precedence signal processing policy.

[0110] Similarly, the switch K21 in the application layer 601a may be used to control the voice optimization component 62a shown in Fig. 8, and the switch K22 in the application layer may be used to control the voice optimization component 62b in the terminal system layer 602a shown in Fig. 8. As can be understood, since the voice optimization component 62a in the application layer 601a shown in Fig. 8 and the voice optimization component 62b in the terminal system layer 602a shown in Fig. 8 have the same optimization function, the target user terminal can select whether to control the voice optimization component 62b in the second front-end signal processing policy in the terminal system layer 602a to be turned on (or off) in the application layer 601a based on the cooperation mechanism (also called the negotiation mechanism) between the application layer 601a and the terminal system layer 602a. For example, as shown in FIG. 8, the target user terminal can control the turning on of the voice optimization component 62a of the second pre-precedence signal processing policy at the application layer 601a, that is, the target user terminal can generate a second control command for controlling the business switch 64b to turn off switch K22 and turn off switch K21, and at this time, the second control command can be used to instruct the target user terminal to set the second optimization component (e.g., the voice optimization component 62b in FIG. 8) turned on in the second pre-precedence signal processing policy as the second coordination component, and to turn off the first optimization component (e.g., the voice optimization component 62a shown in FIG. 8) having the same optimization function as the second coordination component in the first pre-precedence signal processing policy.

[0111] Similarly, the switch K31 in the application layer 601a may be used to control the voice optimization component 63a shown in Fig. 8, and the switch K32 in the application layer may be used to control the voice optimization component 63b in the terminal system layer 602a shown in Fig. 8. As can be understood, since the voice optimization component 63a in the application layer 601a shown in Fig. 8 and the voice optimization component 63b in the terminal system layer 602a shown in Fig. 8 have the same optimization function, the target user terminal can select whether to control the voice optimization component 63b in the second pre-processing policy in the terminal system layer 602a in the application layer 601a to turn on (or off) according to the cooperation mechanism (also called the negotiation mechanism) between the application layer 601a and the terminal system layer 602a. Wherein, the target user terminal can generate a third control command for controlling the business switch 64c to turn off the switch K31 and turn off the switch K32. For a specific implementation method, refer to the description of the first control command and the like described above, and a detailed description thereof will be omitted here. At this time, the target user terminal can set the second optimization component (e.g., the voice optimization component 63b in FIG. 8) that is turned off in the second pre-processing policy as a new first linking component, and turn on a first optimization component (e.g., the voice optimization component 63a in FIG. 8) that has the same optimization function as the new first linking component in the first pre-processing policy.

[0112] Step S103: Obtain uplink audio data in game audio mode of a first user corresponding to a business application, and perform audio optimization on the uplink audio data in game audio mode based on a first optimization component turned on in the first pre-processing signal policy and a second optimization component turned on in the second pre-processing signal policy.

[0113] As can be understood, the target user terminal can further perform voice optimization on the uplink voice data of the first user collected in real time in the game scenario based on the first optimization component turned on in the above-mentioned first pre-processing policy and the second optimization component turned on in the above-mentioned second pre-processing policy, thereby ensuring the clarity and loudness of the uplink voice data currently input to the target user terminal. In this way, the target user terminal can transmit the voice of the first user with high clarity and loudness to the communication destination (i.e., the terminal corresponding to the above-mentioned third user) in the game voice mode. In this way, the downlink voice data played by the speaker of the communication destination can be the voice of the first user after the voice optimization process.

[0114] In an embodiment of the present application, a computer device (e.g., a target user terminal as a mobile terminal) can obtain a signal processing result associated with a first pre-processing signal processing policy in an application layer of a business application in a game voice mode, in which, as can be understood, each first optimization component in the first pre-processing signal processing policy has the same optimization function as a corresponding second optimization component in the second pre-processing signal processing policy. Thus, in the subsequent process of processing the human voice of the real-time voice of the game (i.e., voice optimization of the uplink voice data), the phenomenon of voice optimization components having the same function being executed redundantly in the game voice mode can be effectively solved. For example, the embodiment of the present application proposes that, based on the aforementioned signal processing result (i.e., the comparison result of the algorithm corresponding to the voice optimization component having the same function), the application layer controls the on or off of one or more second optimization components in the terminal system layer, so that the voice optimization components having the same function can be executed in the game application layer or the terminal system layer, thus fundamentally reducing the sound quality damage of the uplink voice data. As can be understood, the number and type of second optimization components that are turned on or off in the terminal system layer are not limited here. Furthermore, when the computing device obtains uplink audio data in the game audio mode of the first user, the computing device can quickly perform joint (cooperative) audio optimization on the collected uplink audio data based on the first optimization component that is turned on and the second optimization component that is turned on, thereby improving the audio optimization effect in the game scenario while reducing sound quality damage.

[0115] Please refer to Fig. 9, which is a diagram illustrating a voice data processing method provided in an embodiment of the present application. As shown in Fig. 9, the method may be executed by a user terminal (for example, a target user terminal, which may be the user terminal 3000a shown in Fig. 1 above), and the method may specifically include at least one of the following steps S201 to S213:

[0116] Step S201: When a first user accesses a business application, a system resource packet for loading the business application is obtained, and an analysis process is performed on the system resource packet to obtain system resource data of the business application.

[0117] Step S202: An initialization process is performed on the system resource data, and the business mode of the business application is initially set to the system media mode based on the system resource data after the initialization process.

[0118] For ease of understanding, please refer to Fig. 10, which is a diagram showing a scenario of a resource setting interface provided in an embodiment of the present application. It can be understood that in a game scenario, the game user A shown in Fig. 10 may be the user 1 in the embodiment corresponding to Fig. 4 above.

[0119] As shown in Fig. 10, when the game user A launches the business application shown in Fig. 10 on the target user terminal, a system resource packet for loading the business application can be obtained from the business server shown in Fig. 10, and an encoder in the target user terminal can perform an analysis process on the obtained system resource packet to obtain system resource data of the business application. In addition, the target user terminal can further perform an initialization process on the system resource data, and output the resource setting interface shown in Fig. 10 based on the system resource data after the initialization process, and as shown in Fig. 10, the resource setting interface is used to dynamically output multimedia data in the system resource data after the initialization process, where the multimedia data may include, but is not limited to, the image frames and audio frames shown in Fig. 10. As can be understood, in the embodiment of the present application, the business mode of the business application is initially set to the system media mode based on the system resource data after the initialization process, so that the resource setting interface shown in Fig. 10 can play the media audio data shown in Fig. 10 (i.e., the above-mentioned audio frame data and video frame data) through the speaker. As can be understood, after completing the system setting, the target user terminal can further execute step S103 described below to switch the display interface of the business application from the resource setting interface 800a shown in Fig. 10 to an application display interface including voice control. In this way, when the game user A triggers the voice control in the off state in the application display interface, the business mode of the business application can be switched from the current system media mode to the above-mentioned game voice mode, so that voice interaction can be performed in the game voice mode.

[0120] Step S203: An application display interface for the business application is output based on the system resource data after the initialization process.

[0121] Wherein, the application display interface includes a voice control for instructing the first user to start a voice interaction transaction.

[0122] Step S204: Detect the application type of the business application according to the voice-on operation of the first user for the voice control.

[0123] Step S205: When detecting that the application type of the business application is a game type, generate a first voice call command associated with the game type, and switch the business mode of the business application from a system media mode to a game voice mode based on the first voice call command.

[0124] Optionally, after performing the above-mentioned step S204, when the target user terminal further detects that the application type of the business application is a game type, it can directly switch the business mode of the business application from the system media mode to the game audio mode.

[0125] Step S206: In a game audio mode, obtain a signal processing result associated with a first pre-processing policy in the application layer of the business application.

[0126] Step S207: Based on the signal processing result, the application layer controls the on and off of the second optimization component in the second pre-processing policy in the terminal system layer, or controls the on and off of the first optimization component in the first pre-processing policy.

[0127] Among them, it can be understood that the specific implementation manner of the target user terminal controlling the first optimization component in the first pre-processing policy to be turned on and off based on the comparison result of the algorithm in the application layer can be referred to the description of the specific process of controlling the second optimization component in the second pre-processing policy in the terminal system layer to be turned on and off in the embodiment corresponding to the above-mentioned FIG. 5, so the detailed description thereof will be omitted here.

[0128] Step S208: Obtain uplink audio data in game audio mode of a first user corresponding to a business application, and perform audio optimization on the uplink audio data in game audio mode based on a first optimization component turned on in the first pre-processing signal policy and a second optimization component turned on in the second pre-processing signal policy.

[0129] For a specific implementation method of steps S205 to S208, the description of steps S101 to S104 in the embodiment corresponding to FIG. 5 above can be referred to, and detailed description thereof will be omitted here.

[0130] Step S209: The uplink voice data after voice optimization is set as a target voice optimization result corresponding to the uplink voice data.

[0131] Step S210: The target voice optimization result is sent to a terminal corresponding to a third user associated with the first user, so that the terminal corresponding to the third user plays the uplink voice data after voice optimization through a speaker in a game voice mode.

[0132] Optionally, the first user and the third user are all game users in the same game faction in the game audio mode.

[0133] Optionally, as can be understood, after the computing device executes the above-mentioned step S204, when the target user terminal detects that the application type of the currently executed business application belongs to a non-game type, it can further jump to executing steps S211 to S213 described below, thereby allowing the above-mentioned first user to make a system call with another user (e.g., a second user) in system call mode.

[0134] Step S211: When detecting that the application type of the business application is a non-game type, generate a second voice call command associated with the game type, and switch the business mode of the business application from a system media mode to a system call mode based on the second voice call command.

[0135] Step S212: When determining the call type of the voice interaction business as the system call type according to the system call mode, send a system call request corresponding to the system call type to the second user through the business application.

[0136] Among them, the second user is a user selected by the first user in a business application and requests to make a system call.

[0137] Step S213: When the second user responds to the system call request, establish a system communication channel between the first user and the second user, and make a system call based on the system communication channel.

[0138] For ease of understanding, please refer to FIG. 11, which is a flow chart for providing a double talk service of different types of languages ​​provided in the embodiment of the present application. As shown in FIG. 11, after the first user launches the above-mentioned business application on the target user terminal, the system resource can be initialized by executing step S1 shown in FIG. 11. For example, the target user terminal can perform initialization processing on the system resource data obtained by the above-mentioned analysis, and then execute step S2 shown in FIG. 11 based on the system resource data after the initialization processing to make the target user terminal enter the system media mode by default. Specifically, the target user terminal can initially set the business mode of the business application to the system media mode. Furthermore, when the first user needs to perform voice interaction with another user, the voice call can be started in the application layer of the target user terminal by executing step S3 shown in FIG. 11. At this time, the target user terminal can determine the application type of the business application that starts the voice call by executing step S4 shown in Fig. 11, and if the application type of the business application is a game type, it can enter the game voice mode by executing step S5 shown in Fig. 11, that is, the first user can make a voice call in a game scenario with another user (e.g., the above-mentioned third user) in the system call mode. Otherwise, as shown in Fig. 11, it can enter the system call mode by executing step S11 shown in Fig. 11, that is, the first user can make a system call in a non-game scenario with another user (e.g., the above-mentioned second user) in the system call mode.

[0139] As shown in FIG. 11, after the target user terminal executes step S5, the target user terminal can further execute step S6 to set the voice sampling rate of the terminal (for example, the uplink and downlink sampling rates shown in FIG. 11 are set to ensure the sampling rate) and the number of channels (to ensure the voice quality of the uplink and downlink), where the voice sampling rate and the number of channels can be the above-mentioned voice quality parameters. Furthermore, as shown in FIG. 11, the target user terminal can execute step S7, that is, the target user terminal can turn on the voice pre-processing algorithm of the application layer and turn off the pre-processing algorithm of the terminal system layer based on the comparative effect of the above-mentioned algorithm. Optionally, the target user terminal can also turn on the pre-processing algorithm of the terminal system layer and turn off the voice pre-processing algorithm of the application layer at the same time. In this way, it is possible to ensure that the voice optimization components with the same optimization functions work at the application layer or the terminal system layer in the target user terminal, that is, in the embodiment of the present application, it is possible to ensure that the voice processing algorithms of only one voice optimization component among the first optimization component at the application layer and the second optimization component with the same optimization function at the terminal system layer work simultaneously, thus maximally reducing power consumption and providing optimal sound quality effect.

[0140] Also, as shown in FIG. 11, when a first user executes step S8 shown in FIG. 11 with another user in a game scenario, a multi-way game voice call can be performed in the game voice scenario, that is, in the process of game voice call, the target user terminal performs optimization processing on the uplink voice data of the first user collected in real time by the first optimization component and the second optimization component determined by the above-mentioned negotiation, so as to transmit the voice of the first user after the optimization processing to another user. Also, as can be understood, when the first user does not need to transmit the voice after the optimization processing to other game users in the same camp, the game scenario can execute step S9 shown in FIG. 11, for example, the target user terminal can switch the business mode of the business application from the above-mentioned game voice mode to the system media mode shown in FIG. 11 according to the first user's voice off operation on the above-mentioned voice control. As can be understood, the embodiment of the present application can further play the voice of the other user after the optimization processing transmitted from the terminal corresponding to the other user through the system media mode in the game scenario, for example, the first user corresponding to the target user terminal can listen to the voice of the other user (i.e., the above-mentioned third user) after the optimization processing in the system media mode. At this time, if the first user turns off the voice control, there is no need to continuously perform voice optimization on the uplink voice data of the first user, that is, the first user does not need to transmit the voice of the first user after the optimization process to other users in the game scenario at this time.

[0141] As can be understood, as shown in FIG. 11, after a first user executes a game in the above-mentioned business application, it can exit from the current game system by executing step S10 shown in FIG. 11, and at this time, the target user terminal can release related system resource data.

[0142] As can be seen, when a first user listens to music on the target user terminal, the target user terminal can work in the above-mentioned system media mode; when a first user makes a phone call on the target user terminal, the target user terminal can work in the above-mentioned system call mode; and optionally, when a first user makes a game voice on the target user terminal, the target user terminal can work in the above-mentioned game voice mode. As can be seen, the voice interaction system according to the embodiment of the present application may include two modules as follows: one module is a game voice mode in the target user terminal, which may exist in the target user terminal in parallel with the above-mentioned system call mode and system media mode. As can be seen, in the game voice mode, the uplink, downlink voice sampling rate, and number of channels of the voice set according to the sound quality indicator of the target user terminal do not affect each other. The other module is a front-end signal processing scheme performed in the application layer, for example, the target user terminal can intelligently adjust the front-end signal processing scheme of the application layer according to the voice processing effect of the terminal system layer. In this way, the cooperation of the two modules allows the target user terminal to improve the experience of voice interaction between game users in a game scenario.

[0143] In the embodiment of the present application, the computer device (e.g., the target user terminal) can enter the game voice mode when the application type of the business application is detected as a game type, and in the game voice mode, the application layer can adaptively control one or more second optimization components in the terminal system layer to be turned on or off based on the above-mentioned signal processing result (i.e., the comparison result of the algorithm corresponding to the voice optimization component having the same function) so that the voice optimization component having the same optimization function is executed in the game application layer or the terminal system layer, thus fundamentally reducing the sound quality damage of the uplink voice data. As can be understood, the number and type of the second optimization components turned on or off in the terminal system layer are not limited here. In addition, the computer device (e.g., the target user terminal) can further quickly perform voice optimization on the uplink voice data in the game voice mode based on the first optimization component turned on and the second optimization component turned on when the uplink voice data in the game voice mode of the first user is obtained, thereby reducing the sound quality damage and improving the voice optimization effect in the game scenario. Optionally, in an embodiment of the present application, when the application type of the business application is detected to be a non-game type, a system voice mode can be entered, and the first user can be allowed to make a system call with another user in the system voice mode.

[0144] Referring to FIG. 12, FIG. 12 is a flowchart of another voice data processing method provided in an embodiment of the present application. The method is executed by a computer device, for example, the method may be executed by a user terminal (for example, the above-mentioned target user terminal, which may be the user terminal 10a in the embodiment corresponding to the above-mentioned FIG. 4), or may be executed by a business server (for example, the business server 2000 shown in the above-mentioned FIG. 1), or may be executed by cooperation between the user terminal and the business server. For ease of understanding, the present embodiment will be described taking the method executed by the user terminal as an example. Among them, the voice data processing method may include at least one of the following steps S301 to S302.

[0145] Step S301: In a game audio mode, obtain a signal processing result associated with a first pre-processing policy in an application layer of a business application.

[0146] Step S302: According to the signal processing result, the application layer controls the on-off state of a second optimization component in a second front-end signal processing policy in the terminal system layer.

[0147] Wherein, the first pre-processing policy includes at least one first optimization component, and the second pre-processing policy includes at least one second optimization component.

[0148] For example, based on the signal processing result, the application layer determines a second optimization component that needs to be turned on in the second pre-frontend signal processing policy in the terminal system layer, and / or based on the signal processing result, the application layer determines a second optimization component that needs to be turned off in the second pre-frontend signal processing policy in the terminal system layer. Then, for a second optimization component that needs to be turned on in the second pre-frontend signal processing policy, if the current state of the second optimization component is off, the application layer controls the second optimization component to be turned on, and if the current state of the second optimization component is on, the application layer controls the second optimization component to be turned off, and if the current state of the second optimization component is off, the application layer controls the second optimization component to be turned off.

[0149] Optionally, the application layer controls the on / off state of a first optimization component in the first front-end signal processing policy based on the signal processing result.

[0150] In some embodiments, the above-mentioned step S302 includes: determining a second optimization component to be turned on in the second pre-preprocessing signal processing policy based on the signal processing result; determining a second optimization component to be turned off in the second pre-preprocessing signal processing policy; in the application layer, setting the second optimization component turned off in the second pre-preprocessing signal processing policy as a first coordination component and turning on a first optimization component having the same optimization function as the first coordination component in the first pre-preprocessing signal processing policy; and in the application layer, setting the second optimization component turned on in the second pre-preprocessing signal processing policy as a second coordination component and turning off the first optimization component having the same optimization function as the second coordination component in the first pre-preprocessing signal processing policy.

[0151] In some embodiments, the above-mentioned step S301 includes obtaining a terminal type of the terminal to which the business application belongs, finding a test type that matches the terminal type in a test list associated with the business application, and if a test type that matches the terminal type is found in the test list, obtaining a first test processing result obtained by adopting a first pre-prefix signal processing policy from the test list based on the sound quality parameters, and obtaining a second test processing result obtained by adopting a second pre-prefix signal processing policy from the test list based on the first test processing result and the second test processing result, determining an optimal signal processing policy associated with the sound quality parameters from among the first pre-prefix signal processing policy and the second pre-prefix signal processing policy based on the first test processing result and the second test processing result, and setting the optimal signal processing policy as the signal processing result associated with the first pre-prefix signal processing policy.

[0152] Optionally, if no test type matching the terminal type is found in the test list, when uplink voice data of a first user is obtained by a microphone in a game voice mode, voice optimization is performed on the uplink voice data according to a first pre-prefix signal processing policy to obtain a first voice optimization result, and voice optimization is performed on the uplink voice data according to a second pre-prefix signal processing policy to obtain a second voice optimization result, and based on the first voice optimization result and the second voice optimization result, an optimal signal processing policy associated with the sound quality parameter is determined from the first pre-prefix signal processing policy and the second pre-prefix signal processing policy, and the optimal signal processing policy is the signal processing result associated with the first pre-prefix signal processing policy.

[0153] In some embodiments, the above method further includes obtaining uplink audio data in a gaming audio mode of a first user corresponding to a business application, and performing audio optimization on the uplink audio data in the gaming audio mode based on a first optimization component turned on in the first pre-processing signal policy and a second optimization component turned on in the second pre-processing signal policy.

[0154] In some embodiments, the above-mentioned method further includes: when a first user accesses the business application, obtaining a system resource packet for loading the business application, performing an analysis process on the system resource packet, obtaining system resource data of the business application, performing an initialization process on the system resource data, and initially setting the business mode of the business application to a system media mode based on the system resource data after the initialization process.

[0155] In some embodiments, the above-mentioned method further includes setting the voice-optimized uplink voice data as a target voice optimization result corresponding to the uplink voice data, and sending the target voice optimization result to a terminal corresponding to a third user associated with the first user, so as to cause the terminal corresponding to the third user to play the voice-optimized uplink voice data through a speaker in a game voice mode.

[0156] In an embodiment of the present application, a game audio mode is provided, in which the application layer of a business application has the authority to control the on-off state of the audio optimization component in the terminal system layer, so that the business application can flexibly control the on-off state of the audio optimization component in the terminal system layer based on actual business requirements or needs, and ensure the audio optimization effect in this mode.

[0157] Referring to FIG. 13, FIG. 13 is a flowchart of another voice data processing method provided in an embodiment of the present application. The method is executed by a computer device, for example, the method may be executed by a user terminal (for example, the above-mentioned target user terminal, which may be the user terminal 10a in the embodiment corresponding to the above-mentioned FIG. 4), or may be executed by a business server (for example, the business server 2000 shown in the above-mentioned FIG. 1), or may be executed by cooperation between the user terminal and the business server. For ease of understanding, the present embodiment will be described taking the method executed by the user terminal as an example. Among them, the voice data processing method includes at least one step of the following steps S401 to S402.

[0158] Step S401: In a game audio mode, obtain a signal processing result associated with a first pre-processing policy in an application layer of a business application.

[0159] Step S402: Based on the signal processing result, control the on / off state of a second optimization component in a second pre-processing policy in a terminal system layer, or control the on / off state of a first optimization component in a first pre-processing policy, in which the first optimization component turned on in the first pre-processing policy is different from the second optimization component turned on in the second pre-processing policy.

[0160] Wherein, the first pre-processing policy includes at least one first optimization component, and the second pre-processing policy includes at least one second optimization component.

[0161] Optionally, a first optimization component that is turned on in the first pre-precedence signal processing policy has the same optimization function as a voice optimization component that is turned off in the second pre-precedence signal processing policy, and a second optimization component that is turned on in the second pre-precedence signal processing policy has the same optimization function as a first optimization component that is turned off in the first pre-precedence signal processing policy.

[0162] Optionally, step S402 may be performed by an application layer of a business application, or may be performed by a terminal system layer, or may be performed by cooperation between the application layer and the terminal system layer. For example, the application layer controls the on / off state of a first optimization component in a first pre-processing policy, and the terminal system layer controls the on / off state of a second optimization component in a second pre-processing policy. In such a case, the signal processing results need to be synchronized between the application layer and the terminal system layer, or the first optimization component and / or the second optimization component that need to be turned on and off need to be synchronized.

[0163] In some embodiments, the above-mentioned step S402 includes determining a second optimization component to be turned on in the second pre-prefix signal processing policy based on the signal processing result, determining a second optimization component to be turned off in the second pre-prefix signal processing policy, turning off the second optimization component to be turned off in the second pre-prefix signal processing policy, turning on a first optimization component having the same optimization function as the second optimization component to be turned off in the first pre-prefix signal processing policy, turning on the second optimization component to be turned on in the second pre-prefix signal processing policy, and turning off the first optimization component having the same optimization function as the second optimization component to be turned on in the first pre-prefix signal processing policy. Optionally, for a second optimization component that needs to be turned on in the second pre-processing policy, if the current state of the second optimization component is off, the second optimization component is controlled to be turned on, and if the current state of the second optimization component is on, the second optimization component is held to be turned on; and for a second optimization component that needs to be turned off in the second pre-processing policy, if the current state of the second optimization component is on, the second optimization component is controlled to be turned off, and if the current state of the second optimization component is off, the second optimization component is held to be turned off.

[0164] In some embodiments, the above-mentioned step S401 includes obtaining a terminal type of the terminal to which the business application belongs, finding a test type that matches the terminal type in a test list associated with the business application, and if a test type that matches the terminal type is found in the test list, obtaining a first test processing result obtained by adopting a first pre-prefix signal processing policy from the test list based on the sound quality parameters, and obtaining a second test processing result obtained by adopting a second pre-prefix signal processing policy from the test list based on the first test processing result and the second test processing result, determining an optimal signal processing policy associated with the sound quality parameters from among the first pre-prefix signal processing policy and the second pre-prefix signal processing policy based on the first test processing result and the second test processing result, and setting the optimal signal processing policy as the signal processing result associated with the first pre-prefix signal processing policy.

[0165] Optionally, if no test type matching the terminal type is found in the test list, when uplink voice data of a first user is obtained by a microphone in a game voice mode, voice optimization is performed on the uplink voice data according to a first pre-prefix signal processing policy to obtain a first voice optimization result, and voice optimization is performed on the uplink voice data according to a second pre-prefix signal processing policy to obtain a second voice optimization result, and based on the first voice optimization result and the second voice optimization result, an optimal signal processing policy associated with the sound quality parameter is determined from the first pre-prefix signal processing policy and the second pre-prefix signal processing policy, and the optimal signal processing policy is the signal processing result associated with the first pre-prefix signal processing policy.

[0166] In some embodiments, the above-mentioned method further includes, when a first user accesses the business application, obtaining a system resource packet for loading the business application, performing an analysis process on the system resource packet, obtaining system resource data of the business application, performing an initialization process on the system resource data, and initially setting the business mode of the business application to a system media mode based on the system resource data after the initialization process.

[0167] Optionally, after the above-mentioned step S402, the method further includes obtaining uplink audio data in a game audio mode of a first user corresponding to a business application, and performing audio optimization on the uplink audio data in the game audio mode based on a first optimization component that is turned on in the first pre-processing signal policy and a second optimization component that is turned on in the second pre-processing signal policy.

[0168] In the embodiment of the present application, based on the above-mentioned signal processing result, one or more voice optimization components in the terminal system layer can be controlled to be turned on or off, so that a voice optimization component having the same optimization function can be executed in the application layer or the terminal system layer, thereby fundamentally reducing the sound quality damage of uplink voice data and improving the voice optimization effect in game scenarios.

[0169] In addition, for contents that are not described in detail in the embodiments of Figures 12 and 13, reference may be made to the descriptions of related contents in other embodiments of the present application, and detailed descriptions thereof will be omitted here.

[0170] Refer to Fig. 14, which is a block diagram of a voice data processing device provided in an embodiment of the present application. Wherein, the voice data processing device 1 may include at least one of the following: a processing result acquisition module 12, a component control module 13 and a voice optimization module 14. Optionally, the voice data processing device may further include at least one of the following: a resource packet acquisition module 15, an initialization module 16, an application interface output module 17, a voice-on module 18, a game mode switching module 11, a call mode switching module 19, a call request sending module 20, a communication channel establishment module 21, a target result determination module 22, a target result sending module 23 and a voice-off module 24.

[0171] The processing result acquisition module 12 is used in a game audio mode to obtain a signal processing result associated with a first pre-processing policy in an application layer of a business application, in which the first pre-processing policy includes at least one first optimization component.

[0172] Wherein, the processing result acquisition module 12 includes a sound quality index acquisition unit 121 , a terminal type lookup unit 122 , a test result acquisition unit 123 , an optimal policy determination unit 124 , an optimization result acquisition unit 125 and a processing result determination unit 126 .

[0173] The sound quality index obtaining unit 121 is used in the game sound mode to obtain the sound quality index of the business application, and set the sound quality parameters of the business application according to the sound quality index of the business application.

[0174] The terminal type lookup unit 122 is used to obtain the terminal type of the terminal to which the business application belongs, and find a test type matching the terminal type in the test list associated with the business application.

[0175] When the test result acquisition unit 123 finds a test type in the test list that matches the terminal type, it is used to acquire a first test processing result obtained by adopting a first pre-amplification signal processing policy from the test list based on the sound quality parameters, and to acquire a second test processing result obtained by adopting a second pre-amplification signal processing policy, where the first pre-amplification signal processing policy is a pre-amplification signal processing policy in the application layer of the business application, and the second pre-amplification signal processing policy is a pre-amplification signal processing policy in the system terminal corresponding to the test terminal type.

[0176] The optimal policy determination unit 124 is used to determine an optimal signal processing policy associated with the sound quality parameter from among the first pre-processing signal processing policy and the second pre-processing signal processing policy based on the first test processing result and the second test processing result, and to set the determined optimal signal processing policy as the signal processing result associated with the first pre-processing signal processing policy.

[0177] Wherein, the voice optimization algorithm of the first optimization component in the first pre-processing policy includes at least one of the following: a first echo cancellation algorithm for performing echo cancellation at the application layer, a first noise suppression algorithm for performing noise suppression at the application layer, and a first gain control algorithm for performing gain adjustment at the application layer; and the voice optimization algorithm of the second optimization component in the second pre-processing policy includes at least one of the following: a second echo cancellation algorithm for performing echo cancellation at the terminal system layer, a second noise suppression algorithm for performing noise suppression at the terminal system layer, and a second gain control algorithm for performing gain adjustment at the terminal system layer.

[0178] Wherein, the optimal decision unit 124 includes a first selection subunit 1241 , a second selection subunit 1242 , a third selection subunit 1243 and an optimal policy decision subunit 1244 .

[0179] The first selection subunit 1241 is used to obtain a first echo cancellation result corresponding to a first echo cancellation algorithm from the first test processing results, obtain a second echo cancellation result corresponding to a second echo cancellation algorithm from the second test processing results, select an optimal echo cancellation algorithm from the first echo cancellation algorithm and the second echo cancellation algorithm based on the first echo cancellation result and the second echo cancellation result, and set the optimal echo cancellation algorithm as a first optimal signal processing policy associated with the sound quality parameters.

[0180] Among them, the first selection subunit 1241 is specifically used for obtaining a first echo cancellation result corresponding to a first echo cancellation algorithm from the first test processing result, and obtaining a second echo cancellation result corresponding to a second echo cancellation algorithm from the second test processing result.

[0181] The first selection subunit 1241 is further used for performing a first comparison between the optimization quality corresponding to the first echo cancellation result and the optimization quality corresponding to the second echo cancellation result to obtain a first comparison result.

[0182] The first selection subunit 1241 is further used for setting a first echo cancellation algorithm in the first pre-signal processing policy as a first optimal signal processing policy associated with the sound quality parameter when the first comparison result indicates that the optimization quality corresponding to the first echo cancellation result is better than the optimization quality corresponding to the second echo cancellation result.

[0183] Optionally, the first selection subunit 1241 is further used for setting a second echo cancellation algorithm in the second pre-processing policy as the first optimal signal processing policy associated with the sound quality parameter when the first comparison result indicates that the optimized quality corresponding to the second echo cancellation result is better than the optimized quality corresponding to the first echo cancellation result.

[0184] The second selection subunit 1242 is used to obtain a first noise suppression result corresponding to a first noise suppression algorithm from the first test processing results, obtain a second noise suppression result corresponding to a second noise suppression algorithm from the second test processing results, select an optimal noise suppression algorithm from the first noise suppression algorithm and the second noise suppression algorithm based on the first noise suppression result and the second noise suppression result, and set the optimal noise suppression algorithm as a second optimal signal processing policy associated with the sound quality parameter.

[0185] Among them, the second selection subunit 1242 is specifically used for obtaining a first noise suppression result corresponding to a first noise suppression algorithm from the first test processing result, and obtaining a second noise suppression result corresponding to a second noise suppression algorithm from the second test processing result.

[0186] The second selection subunit 1242 is further used for performing a second comparison between the optimization quality corresponding to the first noise suppression result and the optimization quality corresponding to the second noise suppression result, and obtaining a second comparison result.

[0187] The second selection subunit 1242 is further used to set the first noise suppression algorithm in the first pre-processing signal policy as a second optimal signal processing policy associated with the sound quality parameter when the second comparison result indicates that the optimized quality corresponding to the first noise suppression result is better than the optimized quality corresponding to the second noise suppression result.

[0188] Optionally, the second selection subunit 1242 is further used to set a second noise suppression algorithm in the second pre-processing signal policy as a second optimal signal processing policy associated with the sound quality parameter when the second comparison result indicates that the optimization quality corresponding to the second noise suppression result is better than the optimization quality corresponding to the first noise suppression result.

[0189] The third selection subunit 1243 is used to obtain a first gain control result corresponding to a first gain control algorithm from the first test processing results, obtain a second gain control result corresponding to a second gain control algorithm from the second test processing results, select an optimal gain control algorithm from the first gain control algorithm and the second gain control algorithm based on the first gain control result and the second gain control result, and set the optimal gain control algorithm as a third optimal signal processing policy associated with the sound quality parameter.

[0190] Among them, the third selection subunit 1243 is specifically used for obtaining a first gain control result corresponding to a first gain control algorithm from the first test processing result, and obtaining a second gain control result corresponding to a second gain control algorithm from the second test processing result.

[0191] The third selection subunit 1243 is further used for performing a third comparison between the optimization quality corresponding to the first gain control result and the optimization quality corresponding to the second gain control result, and obtaining a third comparison result.

[0192] The third selection subunit 1243 is further used for setting the first gain control algorithm in the first pre-signal processing policy as a third optimal signal processing policy associated with the sound quality parameter when the third comparison result indicates that the optimization quality corresponding to the first gain control result is better than the optimization quality corresponding to the second gain control result.

[0193] Optionally, the third selection subunit 1243 is further used to set the second gain control algorithm in the first pre-processing policy as a third optimal signal processing policy associated with the sound quality parameter when the third comparison result indicates that the optimization quality corresponding to the second gain control result is better than the optimization quality corresponding to the first gain control result.

[0194] The optimal policy determination subunit 1244 is used to determine a first optimal signal processing policy, a second optimal signal processing policy and a third optimal signal processing policy as signal processing results associated with the first pre-processing signal processing policy.

[0195] For the specific implementation methods of the first selection subunit 1241, the second selection subunit 1242, the third selection subunit 1243 and the optimal policy determination subunit 1244, please refer to the description of the specific implementation method of determining the signal processing result mentioned above, and detailed description thereof will be omitted here.

[0196] Optionally, if the optimization result obtaining unit 125 does not find a test type matching the terminal type in the test list, when uplink voice data of a first user is obtained by the microphone in a game voice mode, it is used to perform voice optimization on the uplink voice data according to a first pre-processing signal policy to obtain a first voice optimization result, and to perform voice optimization on the uplink voice data according to a second pre-processing signal policy to obtain a second voice optimization result.

[0197] The processing result determination unit 126 is used to determine an optimal signal processing policy associated with the sound quality parameter from among the first pre-amplification signal processing policy and the second pre-amplification signal processing policy based on the first audio optimization result and the second audio optimization result, and to set the determined optimal signal processing policy as the signal processing result associated with the first pre-amplification signal processing policy.

[0198] For specific implementation methods of the sound quality index acquisition unit 121, the terminal type lookup unit 122, the test result acquisition unit 123, the optimal policy determination unit 124, the optimization result acquisition unit 125 and the processing result determination unit 126, reference may be made to the description of steps S101 and S102 in the embodiment corresponding to the above-mentioned FIG. 5, and detailed description thereof will be omitted here.

[0199] The component control module 13 is used to control the on-off state of the second optimization component in the second pre-processing policy in the terminal system layer at the application layer, or to control the on-off state of the first optimization component in the first pre-processing policy at the terminal system layer based on the signal processing result.

[0200] Among them, the first optimization component turned on in the first pre-preprocessing signal policy is different from the second optimization component turned on in the second pre-preprocessing signal policy, the first optimization component turned on in the first pre-preprocessing signal policy has the same optimization function as the voice optimization component turned off in the second pre-preprocessing signal policy, and the second optimization component turned on in the second pre-preprocessing signal policy has the same optimization function as the first optimization component turned off in the first pre-preprocessing signal policy.

[0201] Wherein, the component control module 13 includes a coordination mechanism initiating unit 131, a component control unit 132, a first component-on unit 133 and a second component-on unit 134.

[0202] The cooperation mechanism initiation unit 131 is used for initiating an cooperation mechanism between the application layer and the terminal system layer of the terminal to which the business application belongs according to the signal processing result.

[0203] The component control unit 132 is used to control the second optimization component in the second front-end signal processing policy in the terminal system layer to be turned on and off at the application layer based on the coordination mechanism.

[0204] The first component on unit 133 is used in the application layer to turn on a second optimization component that is turned off in the second pre-processing policy as a first interworking component, and to turn on a first optimization component that has the same optimization function as the first interworking component in the first pre-processing policy.

[0205] The second component-on unit 134 is used in the application layer to turn on the second optimization component that is turned on in the second pre-processing signal policy as the second coordination component, and to turn off the first optimization component that has the same optimization function as the second coordination component in the first pre-processing signal policy.

[0206] Regarding the specific implementation methods of the linkage mechanism starting unit 131, the component control unit 132, the first component on unit 133 and the second component on unit 134, reference may be made to the explanation of step S102 in the embodiment corresponding to the above-mentioned FIG. 5, and detailed explanations thereof will be omitted here.

[0207] The audio optimization module 14 is used to obtain uplink audio data in a game audio mode of a first user corresponding to a business application, and perform audio optimization on the uplink audio data in the game audio mode based on a first optimization component turned on in a first pre-processing signal policy and a second optimization component turned on in a second pre-processing signal policy.

[0208] Wherein, the first optimization component in the first pre-signal processing policy includes at least a first echo cancellation component, a first noise suppression component and a first gain control component, and the second optimization component in the second pre-signal processing policy includes at least a second echo cancellation component, a second noise suppression component and a second gain control component, wherein the first echo cancellation component and the second echo cancellation component are all used for performing echo cancellation, the first noise suppression component and the second noise suppression component are all used for performing noise suppression, and the first gain control component and the second gain control component are all used for performing gain adjustment.

[0209] Optionally, the resource packet acquisition module 15 is used to acquire a system resource packet for loading the business application when the first user accesses the business application, perform analysis processing on the system resource packet, and acquire system resource data of the business application.

[0210] The initialization module 16 performs an initialization process on the system resource data, and is used to initially set the business mode of the business application to the system media mode based on the system resource data after the initialization process.

[0211] The application interface output module 17 is used to output an application display interface of a business application based on system resource data after initialization processing, and the application display interface includes a voice control for instructing the first user to start a voice interaction business.

[0212] The voice-on module 18 is used for detecting the application type of the business application according to the voice-on operation of the first user for the voice control.

[0213] As can be understood, the voice on module 18 is used to, when the application type of the business application is detected to be a game type, generate a first voice call command associated with the game type when the application type of the business application is detected to be a game type, and notify the game mode switching module 11 to switch the business mode of the business application from the system media mode to the game voice mode based on the first voice call command.

[0214] Optionally, the voice-on module 18 is further used to, when the application type of the business application is detected to be a non-game type (e.g., a social type), generate a second voice call command associated with the game type when the application type of the business application is detected to be a non-game type, and notify the business mode of the business application to be switched from the system media mode to the system call mode based on the second voice call command.

[0215] The call request sending module 20 is used, when the call type of the voice interaction business is determined to be a system call type based on the system call mode, to send a system call request corresponding to the system call type to a second user through a business application, where the second user is a user selected by the first user in the business application and requesting to make a system call.

[0216] The communication channel establishment module 21 is used to establish a system communication channel between the first user and the second user when the second user responds to the system call request, and to make a system call based on the system communication channel.

[0217] Optionally, the target result determination module 22 is used to determine the uplink voice data after voice optimization as a target voice optimization result corresponding to the uplink voice data.

[0218] The target result transmission module 23 is used to transmit the target voice optimization result to a terminal corresponding to a third user associated with the first user, so as to cause the terminal corresponding to the third user to play the voice-optimized uplink voice data through a speaker in a game voice mode; optionally, the first user and the third user are all game users in the same game camp in the game voice mode.

[0219] Optionally, the sound-off module 24 is used to switch the business mode of the business application from the game sound mode to the system media mode in response to a sound-off operation of the first user on the sound control.

[0220] Among them, the specific implementation methods of the processing result acquisition module 12, the component control module 13, and the voice optimization module 14 can be referred to in the description of steps 101 to S103 in the embodiment corresponding to the above-mentioned FIG. 5, and detailed descriptions thereof will be omitted here. In addition, the specific implementation methods of the resource packet acquisition module 15, the initialization module 16, the application interface output module 17, the voice-on module 18, the game mode switching module 11, the call mode switching module 19, the call request sending module 20, the communication channel establishment module 21, the target result determination module 22, the target result sending module 23, and the voice-off module 24 can be referred to in the description of steps 201 to S213 in the embodiment corresponding to the above-mentioned FIG. 9, and detailed descriptions thereof will be omitted here. In addition, the description of the advantageous effects of adopting the same method will be omitted here.

[0221] In one exemplary embodiment of the present application, there is further provided an audio data processing apparatus, which is used to perform the embodiment of the method shown in FIG. 12, and which may include at least one of the following: a processing result acquisition module and a component control module.

[0222] The processing result acquisition module is used in a game audio mode to obtain a signal processing result associated with a first pre-processing policy in an application layer of a business application, in which the first pre-processing policy includes at least one first optimization component.

[0223] The component control module is used to control, at the application layer, the on-off state of a second optimization component in a second pre-processing policy in the terminal system layer according to the signal processing result, in which the second pre-processing policy includes at least one second optimization component.

[0224] In one exemplary embodiment of the present application, there is further provided an audio data processing apparatus, which is used to perform the embodiment of the method shown in FIG. 13, and which may include at least one of the following: a processing result acquisition module and a component control module.

[0225] The processing result acquisition module is used in a game audio mode to obtain a signal processing result associated with a first pre-processing policy in an application layer of a business application, in which the first pre-processing policy includes at least one first optimization component.

[0226] The component control module is used to control the on / off state of the second optimization component in the second pre-processing policy in the terminal system layer, or to control the on / off state of the first optimization component in the first pre-processing policy based on the signal processing result, wherein the first optimization component turned on in the first pre-processing policy is different from the second optimization component turned on in the second pre-processing policy.

[0227] For details that are not described in detail in the device embodiments, reference can be made to the corresponding method embodiments.

[0228] Further, please refer to FIG. 15, which is a configuration diagram of a computer device provided in an embodiment of the present application. As shown in FIG. 15, the computer device 1000 may be a user terminal, and the user terminal may be the above-mentioned target user terminal. At this time, the computer device 1000 may include a processor 1001, a network interface 1004 and a memory 1005, and the computer device 1000 may further include a user interface 1003 and at least one communication bus 1002. The communication bus 1002 is used to realize the communication connection between these components. The user interface 1003 may include a display and a keyboard, and optionally, the user interface 1003 may further include a wired interface or a wireless interface according to a standard. The network interface 1004 may optionally include a wired interface or a wireless interface according to a standard (e.g., a WI-FI interface). The storage 1005 may be a high speed RAM or may be a non-volatile memory, such as at least one magnetic disk storage device. Optionally, the storage 1005 may further include at least one storage device remote from the processor 1001. As shown in FIG. 15, the storage 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.

[0229] Wherein, the network interface 1004 in the computer device 1000 may further provide a network communication function, and optionally, the user interface 1003 may further include a display and a keyboard. In the computer device 1000 shown in FIG. 15, the network interface 1004 can provide a network communication function, the user interface 1003 mainly provides an interface for a user to input, and the processor 1001 can execute the steps of the voice data processing method in the embodiment corresponding to FIG. 5, FIG. 9, FIG. 12, or FIG. 13 or the embodiment of another method by calling the device control application program stored in the memory 1005, or execute the processing of the voice data processing device 1 in the embodiment corresponding to FIG. 14, and the detailed description thereof is omitted here. Also, the description of the advantageous effects by referring to the same method is omitted here.

[0230] In addition, the embodiments of the present application further provide a computer storage medium, which stores a computer program executed by the above-mentioned voice data processing device 1, and the computer program includes program instructions, and when the processor executes the program instructions, it can execute the steps of the voice data processing method in the embodiments corresponding to the above-mentioned Figure 5, Figure 9, Figure 12, or Figure 13, or other method embodiments, so detailed descriptions thereof are omitted here. Also, descriptions of advantageous effects obtained by referring to the same method are omitted here. Note that, for technical contents not described in detail in the embodiments of the computer storage medium of the present application, reference can be made to the descriptions of the method embodiments of the present application.

[0231] As can be understood, the embodiments of the present application further provide a computer program product or a computer program, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, thereby making the computer device realize the steps of the voice data processing method in the embodiments corresponding to the above-mentioned FIG. 5, FIG. 9, FIG. 12, or FIG. 13, or other method embodiments, and therefore detailed descriptions thereof are omitted here. Also, descriptions of advantageous effects obtained by referring to the same method are omitted here. Note that, for technical contents not described in detail in the embodiments of the computer program product or computer program of the present application, reference can be made to the description of the method embodiments of the present application.

[0232] As can be understood by those skilled in the art, the flow of implementing all or part of the above-mentioned method embodiments can be completed by a computer program instructing relevant hardware, the program can be stored in a computer-readable storage medium, and the program can include the flow of each method embodiment as described above when executed. Among them, the storage medium can be a magnetic disk, an optical disk, a ROM (Read-Only Memory), a RAM (Random Access Memory), etc.

[0233] Although the preferred embodiment of the present application has been described above, the present application is not limited to this embodiment, and any modification to the present application falls within the technical scope of the present application as long as it does not depart from the spirit of the present application.

Claims

1. 1. A computing device implemented method for processing audio data, comprising: obtaining, in a gaming audio mode, a signal processing result associated with a first front-end signal processing policy in an application layer of a business application, the first front-end signal processing policy including at least one first optimization component; A step of controlling, in the application layer, an on-off state of a second optimization component in a second pre-front-end signal processing policy in a terminal system layer, or controlling an on-off state of a first optimization component in the first pre-front-end signal processing policy, based on the signal processing result, where the first optimization component turned on in the first pre-front-end signal processing policy is different from the second optimization component turned on in the second pre-front-end signal processing policy; and The method includes a step of acquiring uplink audio data in the gaming audio mode of a first user corresponding to the business application, and performing audio optimization on the uplink audio data in the gaming audio mode based on a first optimization component turned on in the first pre-processing signal policy and a second optimization component turned on in the second pre-processing signal policy.

2. 2. The method of claim 1, The step of controlling, in the application layer, an on-off state of a second optimization component in a second front-end signal processing policy in a terminal system layer according to the signal processing result, comprises: determining second optimization components to be turned on in the second pre-processing policy based on the signal processing result, and determining second optimization components to be turned off in the second pre-processing policy; In the application layer, a second optimization component that is turned off in the second pre-processing policy is set as a first cooperation component, and a first optimization component having the same optimization function as the first cooperation component is turned on in the first pre-processing policy; and The method includes a step of setting, in the application layer, a second optimization component that is turned on in the second pre-processing policy as a second coordination component, and turning off a first optimization component that has the same optimization function as the second coordination component in the first pre-processing policy.

3. 2. The method of claim 1, a first optimization component in the first pre-signal processing policy comprising at least one of the following: a first echo cancellation component, a first noise suppression component, and a first gain control component; a second optimization component in the second pre-processing policy including at least one of the following: a second echo cancellation component, a second noise suppression component, and a second gain control component; a first echo cancellation component and a second echo cancellation component both used to perform echo cancellation, a first noise suppression component and a second noise suppression component both used to perform noise suppression, and a first gain control component and a second gain control component both used to perform gain adjustment.

4. 2. The method of claim 1, The step of obtaining a signal processing result associated with a first front-end signal processing policy in an application layer of the business application in the game sound mode includes: setting a sound quality parameter of the business application based on a sound quality index of the business application in the game sound mode; obtaining a terminal type of a terminal to which the business application belongs, and finding a test type that matches the terminal type in a test list associated with the business application; When a test type matching the terminal type is found in the test list, obtaining a first test processing result obtained by using the first pre-processing policy from the test list based on the sound quality parameters, and obtaining a second test processing result obtained by using the second pre-processing policy; and The method includes a step of determining an optimal signal processing policy associated with the sound quality parameter from among the first pre-processing signal processing policy and the second pre-processing signal processing policy based on the first test processing result and the second test processing result, and setting the optimal signal processing policy as the signal processing result associated with the first pre-processing signal processing policy.

5. 5. The method of claim 4, The voice optimization algorithm of the first optimization component in the first pre-signal processing policy includes at least one of the following: a first echo cancellation algorithm for performing echo cancellation at the application layer, a first noise suppression algorithm for performing noise suppression at the application layer, and a first gain control algorithm for performing gain adjustment at the application layer; A method in which the voice optimization algorithm of the second optimization component in the second pre-signal processing policy includes at least one of the following: a second echo cancellation algorithm for performing echo cancellation at the terminal system layer, a second noise suppression algorithm for performing noise suppression at the terminal system layer, and a second gain control algorithm for performing gain adjustment at the terminal system layer.

6. 6. The method of claim 5, determining an optimal signal processing policy associated with the sound quality parameter from among the first pre-processing policy and the second pre-processing policy based on the first test processing result and the second test processing result, and setting the optimal signal processing policy as the signal processing result associated with the first pre-processing policy, obtaining a first echo cancellation result corresponding to the first echo cancellation algorithm from the first test processing results, obtaining a second echo cancellation result corresponding to the second echo cancellation algorithm from the second test processing results, selecting an optimal echo cancellation algorithm from the first echo cancellation algorithm and the second echo cancellation algorithm based on the first echo cancellation result and the second echo cancellation result, and setting the optimal echo cancellation algorithm as a first optimal signal processing policy associated with the sound quality parameter; obtaining a first noise suppression result corresponding to the first noise suppression algorithm from the first test processing results, obtaining a second noise suppression result corresponding to the second noise suppression algorithm from the second test processing results, selecting an optimal noise suppression algorithm from the first noise suppression algorithm and the second noise suppression algorithm based on the first noise suppression result and the second noise suppression result, and setting the optimal noise suppression algorithm as a second optimal signal processing policy associated with the sound quality parameter; obtaining a first gain control result corresponding to the first gain control algorithm from among the first test process results, obtaining a second gain control result corresponding to the second gain control algorithm from among the second test process results, selecting an optimal gain control algorithm from among the first gain control algorithm and the second gain control algorithm based on the first gain control result and the second gain control result, and setting the optimal gain control algorithm as a third optimal signal processing policy associated with the sound quality parameter; and determining the first optimal signal processing policy, the second optimal signal processing policy and the third optimal signal processing policy as signal processing results associated with the first pre-processing policy.

7. 7. The method of claim 6, a step of obtaining a first echo cancellation result corresponding to the first echo cancellation algorithm from the first test processing result, obtaining a second echo cancellation result corresponding to the second echo cancellation algorithm from the second test processing result, selecting an optimal echo cancellation algorithm from the first echo cancellation algorithm and the second echo cancellation algorithm based on the first echo cancellation result and the second echo cancellation result, and setting the optimal echo cancellation algorithm as a first optimal signal processing policy associated with the sound quality parameter, obtaining a first echo cancellation result corresponding to the first echo cancellation algorithm from among the first test processing results, and obtaining a second echo cancellation result corresponding to the second echo cancellation algorithm from among the second test processing results; comparing an optimization quality corresponding to the first echo cancellation result with an optimization quality corresponding to the second echo cancellation result to obtain a first comparison result; if the first comparison result indicates that an optimized quality corresponding to the first echo cancellation result is better than an optimized quality corresponding to the second echo cancellation result, then setting the first echo cancellation algorithm in the first pre-processing policy as a first optimal signal processing policy associated with the sound quality parameter; and if the first comparison result indicates that an optimized quality corresponding to the second echo cancellation result is better than an optimized quality corresponding to the first echo cancellation result, then setting the second echo cancellation algorithm in the second pre-processing policy as a first optimal signal processing policy associated with the sound quality parameter.

8. 7. The method of claim 6, a step of obtaining a first noise suppression result corresponding to the first noise suppression algorithm from the first test processing results, obtaining a second noise suppression result corresponding to the second noise suppression algorithm from the second test processing results, selecting an optimal noise suppression algorithm from the first noise suppression algorithm and the second noise suppression algorithm based on the first noise suppression result and the second noise suppression result, and setting the optimal noise suppression algorithm as a second optimal signal processing policy associated with the sound quality parameter, obtaining a first noise suppression result corresponding to the first noise suppression algorithm from the first test process results, and obtaining a second noise suppression result corresponding to the second noise suppression algorithm from the second test process results; comparing an optimization quality corresponding to the first noise suppression result with an optimization quality corresponding to the second noise suppression result to obtain a second comparison result; if the second comparison result indicates that an optimization quality corresponding to the first noise suppression result is better than an optimization quality corresponding to the second noise suppression result, setting the first noise suppression algorithm in the first pre-processing policy as a second optimal signal processing policy associated with the sound quality parameter; and The method includes a step of setting the second noise suppression algorithm in the second pre-processing signal processing policy as a second optimal signal processing policy associated with the sound quality parameter when the second comparison result indicates that the optimization quality corresponding to the second noise suppression result is better than the optimization quality corresponding to the first noise suppression result.

9. 7. The method of claim 6, obtaining a first gain control result corresponding to the first gain control algorithm from the first test processing result, obtaining a second gain control result corresponding to the second gain control algorithm from the second test processing result, selecting an optimal gain control algorithm from the first gain control algorithm and the second gain control algorithm based on the first gain control result and the second gain control result, and setting the optimal gain control algorithm as a third optimal signal processing policy associated with the sound quality parameter, obtaining a first gain control result corresponding to the first gain control algorithm from among the first test process results, and obtaining a second gain control result corresponding to the second gain control algorithm from among the second test process results; comparing an optimization quality corresponding to the first gain control result with an optimization quality corresponding to the second gain control result to obtain a third comparison result; if the third comparison result indicates that an optimized quality corresponding to the first gain control result is better than an optimized quality corresponding to the second gain control result, then setting the first gain control algorithm in the first pre-processing policy as a third optimal signal processing policy associated with the sound quality parameter; and if the third comparison result indicates that an optimization quality corresponding to the second gain control result is better than an optimization quality corresponding to the first gain control result, the second gain control algorithm in the first pre-processing signal processing policy is a third optimal signal processing policy associated with the sound quality parameter.

10. The method of claim 4, further comprising: If no test type matching the terminal type is found in the test list, when uplink voice data of the first user is obtained by a microphone in the game voice mode, performing voice optimization on the uplink voice data according to the first pre-processing policy to obtain a first voice optimization result, and performing voice optimization on the uplink voice data according to the second pre-processing policy to obtain a second voice optimization result; and The method includes a step of determining an optimal signal processing policy associated with the sound quality parameter from among the first pre-amplification signal processing policy and the second pre-amplification signal processing policy based on the first audio optimization result and the second audio optimization result, and setting the optimal signal processing policy as the signal processing result associated with the first pre-amplification signal processing policy.

11. The method of claim 1, further comprising: When the first user accesses the business application, acquiring a system resource packet for loading the business application, performing an analysis process on the system resource packet, and acquiring system resource data of the business application; and The method includes a step of performing an initialization process on the system resource data, and initially setting a business mode of the business application to a system media mode based on the system resource data after the initialization process.

12. The method of claim 11, further comprising: outputting an application display interface of the business application based on the system resource data after the initialization process, the application display interface including a voice control for instructing the first user to start a voice interaction business; Detecting an application type of the business application in response to a voice-on operation of the first user for the voice control; and The method includes switching a business mode of the business application from the system media mode to the game audio mode when detecting that the application type of the business application is a game type.

13. The method of claim 12, further comprising: switching a business mode of the business application from the system media mode to a system call mode when detecting that the application type of the business application is a non-game type; When the call type of the voice interaction business is determined as a system call type based on the system call mode, a system call request corresponding to the system call type is sent to a second user by the business application, where the second user is a user selected by the first user in the business application and requests to make a system call; and when the second user responds to the system call request, establishing a system communication channel between the first user and the second user, and conducting a system call based on the system communication channel.

14. The method of claim 12, further comprising: The method includes switching a business mode of the business application from the game audio mode to the system media mode in response to a sound-off operation of the first user on the sound control.

15. The method according to any one of claims 1 to 14, further comprising: Setting the uplink voice data after voice optimization as a target voice optimization result corresponding to the uplink voice data; and The method includes a step of transmitting the target voice optimization result to a terminal corresponding to a third user associated with the first user, and causing the terminal corresponding to the third user to play the voice-optimized uplink voice data through a speaker in the game voice mode.

16. 1. A computing device implemented method for processing audio data, comprising: obtaining, in a gaming audio mode, a signal processing result associated with a first front-end signal processing policy in an application layer of a business application, the first front-end signal processing policy including at least one first optimization component; and The method includes a step of controlling, at the application layer, an on / off state of a second optimization component in a second front-end signal processing policy in a terminal system layer based on the signal processing result, wherein the second front-end signal processing policy includes at least one second optimization component.

17. 17. The method of claim 16, The step of controlling, in the application layer, an on-off state of a second optimization component in a second front-end signal processing policy in a terminal system layer according to the signal processing result, comprises: determining second optimization components to be turned on in the second pre-processing policy based on the signal processing result, and determining second optimization components to be turned off in the second pre-processing policy; In the application layer, a second optimization component that is turned off in the second pre-processing policy is set as a first cooperation component, and a first optimization component having the same optimization function as the first cooperation component is turned on in the first pre-processing policy; and The method includes a step of setting, in the application layer, a second optimization component that is turned on in the second pre-processing policy as a second coordination component, and turning off a first optimization component that has the same optimization function as the second coordination component in the first pre-processing policy.

18. 17. The method of claim 16, The step of obtaining a signal processing result associated with a first front-end signal processing policy in an application layer of a business application in the game sound mode includes: obtaining a terminal type of a terminal to which the business application belongs, and finding a test type that matches the terminal type in a test list associated with the business application; When a test type matching the terminal type is found in the test list, obtaining a first test processing result obtained by using the first pre-processing policy from the test list based on a sound quality parameter, and obtaining a second test processing result obtained by using the second pre-processing policy; and The method includes a step of determining an optimal signal processing policy associated with the sound quality parameter from among the first pre-processing signal processing policy and the second pre-processing signal processing policy based on the first test processing result and the second test processing result, and setting the optimal signal processing policy as the signal processing result associated with the first pre-processing signal processing policy.

19. 20. The method of claim 18, The voice optimization algorithm of the first optimization component in the first pre-signal processing policy includes at least one of the following: a first echo cancellation algorithm for performing echo cancellation at the application layer, a first noise suppression algorithm for performing noise suppression at the application layer, and a first gain control algorithm for performing gain adjustment at the application layer; A method in which the voice optimization algorithm of the second optimization component in the second pre-signal processing policy includes at least one of the following: a second echo cancellation algorithm for performing echo cancellation at the terminal system layer, a second noise suppression algorithm for performing noise suppression at the terminal system layer, and a second gain control algorithm for performing gain adjustment at the terminal system layer.

20. 20. The method of claim 19, determining an optimal signal processing policy associated with the sound quality parameter from among the first pre-processing policy and the second pre-processing policy based on the first test processing result and the second test processing result, and setting the optimal signal processing policy as the signal processing result associated with the first pre-processing policy, obtaining a first echo cancellation result corresponding to the first echo cancellation algorithm from the first test processing results, obtaining a second echo cancellation result corresponding to the second echo cancellation algorithm from the second test processing results, selecting an optimal echo cancellation algorithm from the first echo cancellation algorithm and the second echo cancellation algorithm based on the first echo cancellation result and the second echo cancellation result, and setting the optimal echo cancellation algorithm as a first optimal signal processing policy associated with the sound quality parameter; obtaining a first noise suppression result corresponding to the first noise suppression algorithm from the first test processing results, obtaining a second noise suppression result corresponding to the second noise suppression algorithm from the second test processing results, selecting an optimal noise suppression algorithm from the first noise suppression algorithm and the second noise suppression algorithm based on the first noise suppression result and the second noise suppression result, and setting the optimal noise suppression algorithm as a second optimal signal processing policy associated with the sound quality parameter; obtaining a first gain control result corresponding to the first gain control algorithm from among the first test process results, obtaining a second gain control result corresponding to the second gain control algorithm from among the second test process results, selecting an optimal gain control algorithm from among the first gain control algorithm and the second gain control algorithm based on the first gain control result and the second gain control result, and setting the optimal gain control algorithm as a third optimal signal processing policy associated with the sound quality parameter; and determining the first optimal signal processing policy, the second optimal signal processing policy and the third optimal signal processing policy as signal processing results associated with the first pre-processing policy.

21. 19. The method of claim 18, further comprising: If no test type matching the terminal type is found in the test list, when uplink voice data of a first user is obtained by a microphone in the game voice mode, performing voice optimization on the uplink voice data according to the first pre-processing policy to obtain a first voice optimization result, and performing voice optimization on the uplink voice data according to the second pre-processing policy to obtain a second voice optimization result; and The method includes a step of determining an optimal signal processing policy associated with the sound quality parameter from among the first pre-amplification signal processing policy and the second pre-amplification signal processing policy based on the first audio optimization result and the second audio optimization result, and setting the optimal signal processing policy as the signal processing result associated with the first pre-amplification signal processing policy.

22. 1. A computing device implemented method for processing audio data, comprising: obtaining a signal processing result associated with a first front-end signal processing policy in an application layer of a business application in a gaming audio mode, the first front-end signal processing policy including at least one first optimization component; and The method includes a step of controlling an on / off state of a second optimization component in a second pre-processing policy in a terminal system layer, or controlling an on / off state of a first optimization component in the first pre-processing policy, based on the signal processing result, wherein the first optimization component turned on in the first pre-processing policy is different from the second optimization component turned on in the second pre-processing policy.

23. 23. The method of claim 22, The step of controlling the on / off state of a second optimization component in a second pre-processing policy in a terminal system layer according to the signal processing result, or controlling the on / off state of a first optimization component in the first pre-processing policy, comprises: determining second optimization components to be turned on in the second pre-processing policy based on the signal processing result, and determining second optimization components to be turned off in the second pre-processing policy; turning off a second optimization component that is turned off in the second pre-processing policy, and turning on a first optimization component that has the same optimization function as the second optimization component that is turned off in the first pre-processing policy; and The method includes turning on a second optimization component that is turned on in the second pre-processing policy, and turning off a first optimization component that has the same optimization function as the second optimization component that is turned on in the first pre-processing policy.

24. 23. The method of claim 22, The step of obtaining a signal processing result associated with a first front-end signal processing policy in an application layer of a business application in the game sound mode includes: obtaining a terminal type of a terminal to which the business application belongs, and finding a test type that matches the terminal type in a test list associated with the business application; When a test type matching the terminal type is found in the test list, obtaining a first test processing result obtained by using the first pre-processing policy from the test list based on a sound quality parameter, and obtaining a second test processing result obtained by using the second pre-processing policy; and The method includes a step of determining an optimal signal processing policy associated with the sound quality parameter from among the first pre-processing signal processing policy and the second pre-processing signal processing policy based on the first test processing result and the second test processing result, and setting the optimal signal processing policy as the signal processing result associated with the first pre-processing signal processing policy.

25. 25. The method of claim 24, The voice optimization algorithm of the first optimization component in the first pre-signal processing policy includes at least one of the following: a first echo cancellation algorithm for performing echo cancellation at the application layer, a first noise suppression algorithm for performing noise suppression at the application layer, and a first gain control algorithm for performing gain adjustment at the application layer; A method in which the voice optimization algorithm of the second optimization component in the second pre-signal processing policy includes at least one of the following: a second echo cancellation algorithm for performing echo cancellation at the terminal system layer, a second noise suppression algorithm for performing noise suppression at the terminal system layer, and a second gain control algorithm for performing gain adjustment at the terminal system layer.

26. 26. The method of claim 25, determining an optimal signal processing policy associated with the sound quality parameter from among the first pre-processing policy and the second pre-processing policy based on the first test processing result and the second test processing result, and setting the optimal signal processing policy as the signal processing result associated with the first pre-processing policy, obtaining a first echo cancellation result corresponding to the first echo cancellation algorithm from the first test processing results, obtaining a second echo cancellation result corresponding to the second echo cancellation algorithm from the second test processing results, selecting an optimal echo cancellation algorithm from the first echo cancellation algorithm and the second echo cancellation algorithm based on the first echo cancellation result and the second echo cancellation result, and setting the optimal echo cancellation algorithm as a first optimal signal processing policy associated with the sound quality parameter; obtaining a first noise suppression result corresponding to the first noise suppression algorithm from the first test processing results, obtaining a second noise suppression result corresponding to the second noise suppression algorithm from the second test processing results, selecting an optimal noise suppression algorithm from the first noise suppression algorithm and the second noise suppression algorithm based on the first noise suppression result and the second noise suppression result, and setting the optimal noise suppression algorithm as a second optimal signal processing policy associated with the sound quality parameter; obtaining a first gain control result corresponding to the first gain control algorithm from among the first test process results, obtaining a second gain control result corresponding to the second gain control algorithm from among the second test process results, selecting an optimal gain control algorithm from among the first gain control algorithm and the second gain control algorithm based on the first gain control result and the second gain control result, and setting the optimal gain control algorithm as a third optimal signal processing policy associated with the sound quality parameter; and determining the first optimal signal processing policy, the second optimal signal processing policy and the third optimal signal processing policy as signal processing results associated with the first pre-processing policy.

27. 25. The method of claim 24, further comprising: If no test type matching the terminal type is found in the test list, when uplink voice data of a first user is obtained by a microphone in the game voice mode, performing voice optimization on the uplink voice data according to the first pre-processing policy to obtain a first voice optimization result, and performing voice optimization on the uplink voice data according to the second pre-processing policy to obtain a second voice optimization result; and The method includes a step of determining an optimal signal processing policy associated with the sound quality parameter from among the first pre-amplification signal processing policy and the second pre-amplification signal processing policy based on the first audio optimization result and the second audio optimization result, and setting the optimal signal processing policy as the signal processing result associated with the first pre-amplification signal processing policy.

28. The method according to any one of claims 22 to 27, further comprising: The method includes a step of acquiring uplink audio data in the gaming audio mode of a first user corresponding to the business application, and performing audio optimization on the uplink audio data in the gaming audio mode based on a first optimization component turned on in the first pre-processing signal policy and a second optimization component turned on in the second pre-processing signal policy.

29. An apparatus for processing audio data, comprising: a processing result acquisition module for acquiring, in a game audio mode, a signal processing result associated with a first front-end signal processing policy in an application layer of a business application, the first front-end signal processing policy including at least one first optimization component; a component control module for controlling, at the application layer, an on-off state of a second optimization component in a second pre-processing policy in a terminal system layer or controlling an on-off state of a first optimization component in the first pre-processing policy according to the signal processing result, where the first optimization component turned on in the first pre-processing policy is different from the second optimization component turned on in the second pre-processing policy; and The device includes a voice optimization module for acquiring uplink voice data in the game voice mode of a first user corresponding to the business application, and performing voice optimization on the uplink voice data in the game voice mode based on a first optimization component turned on in the first pre-processing signal policy and a second optimization component turned on in the second pre-processing signal policy.

30. An apparatus for processing audio data, comprising: a processing result acquisition module for acquiring, in a gaming audio mode, a signal processing result associated with a first front-end signal processing policy in an application layer of a business application, the first front-end signal processing policy including at least one first optimization component; and The device includes a component control module for controlling, at the application layer, an on-off state of a second optimization component in a second front-end signal processing policy in a terminal system layer based on the signal processing result, the second front-end signal processing policy including at least one second optimization component.

31. An apparatus for processing audio data, comprising: a processing result acquisition module for acquiring, in a gaming audio mode, a signal processing result associated with a first front-end signal processing policy in an application layer of a business application, the first front-end signal processing policy including at least one first optimization component; and An apparatus comprising: a component control module for controlling an on / off state of a second optimization component in a second pre-processing signal policy in a terminal system layer or controlling an on / off state of a first optimization component in the first pre-processing signal policy based on the signal processing result, wherein the first optimization component turned on in the first pre-processing signal policy is different from the second optimization component turned on in the second pre-processing signal policy.

32. A computing device including a processor and a memory, the processor is coupled to the memory; The storage device stores a computer program, A computing device, wherein the processor is configured to execute the computer program to implement the method according to any one of claims 1 to 15, or the method according to any one of claims 16 to 21, or the method according to any one of claims 22 to 28.

33. A program for causing a computer to execute the method according to any one of claims 1 to 15, or the method according to any one of claims 16 to 21, or the method according to any one of claims 22 to 28.

Images