Mixed audio processing method and system, medium, equipment and intelligent automobile

Abstract

The invention discloses a mixed audio processing method and system, a medium, equipment and an intelligent automobile. The method comprises the following steps: acquiring a mixed audio signal stream in an audio and video call scene or an audio and video playing scene; wherein each frame of audio signal in the mixed audio signal stream randomly comprises one or more of a voice signal, a music signal and a noise signal; processing the mixed audio signal stream by using an audio type detection algorithm, and determining an audio type to which each frame of audio signal in the mixed audio signal stream belongs; wherein the audio categories divided by the audio category detection algorithm comprise a first audio category with voice as a main part, a second audio category with music as a main part and a third audio category with noise and / or mute; for each frame of audio signal in the mixed audio signal stream, calling a volume adjustment strategy according with the audio category to which each frame of audio signal belongs to carry out volume gain adjustment; wherein different audio categories correspond to different volume adjustment strategies; and outputting each frame of audio signal after volume gain adjustment.

Description

Technical Field

[0001] This specification relates to the field of multimedia technology, and in particular to a mixed audio processing method, system, medium, device and smart car. Background Art

[0002] Volume adjustment is an important part of speech enhancement tasks and is widely used in tasks such as voice calls, interactive front-end processing, and playback volume balancing.

[0003] With the growing and increasingly complex audio processing scenarios, there is a need to balance multiple types of audio signals within the same volume adjustment task. For example, this includes balancing calls and background music in voice calls, and balancing multimedia playback containing multiple types of audio signals.

[0004] However, traditional volume adjustment focuses on uniform volume adjustment. For example, in a voice call, the volume is uniformly adjusted downward when the voice call volume exceeds a threshold, and uniformly adjusted upward when the voice call volume falls below a threshold. This adjustment does not distinguish between different audio signal types. Therefore, existing technologies cannot differentiate and adjust different audio signal types within the same audio task to ensure audio quality. Summary of the Invention

[0005] This specification provides a mixed audio processing method, system, medium, device, and smart car to address or partially address the existing art problem of automatically switching volume adjustment strategies for each frame of audio signal in a mixed audio signal stream by type. This strategy not only adjusts the volume of voice signals, but also enables targeted adjustments for music signals, noise, or silence. Compared with traditional voice volume adjustment, it can perform compatible control of volume gain according to the type of each frame of audio signal, thereby adapting to audio volume adjustment services in more complex scenarios.

[0006] To solve the above technical problems, the first aspect of this specification provides a method for processing mixed audio, the method comprising:

[0007] Obtaining a mixed audio signal stream in an audio and video call scenario or an audio and video playback scenario; wherein each frame of audio signal in the mixed audio signal stream randomly includes one or more of a speech signal, a music signal, and a noise signal;

[0008] Processing the mixed audio signal stream using an audio category detection algorithm to determine the audio category to which each frame of audio signal in the mixed audio signal stream belongs; wherein the audio categories classified by the audio category detection algorithm include: a first audio category primarily based on speech, a second audio category primarily based on music, and a third audio category including noise and / or silence;

[0009] For each frame of audio signal in the mixed audio signal stream, calling a volume adjustment strategy that matches the audio category to which each frame of audio signal belongs to perform volume gain adjustment; wherein different audio categories correspond to different volume adjustment strategies;

[0010] Output the audio signals of each frame after volume gain adjustment.

[0011] In a second aspect of the present specification, a mixed audio processing system is provided, the system further comprising:

[0012] An acquisition unit, configured to acquire a mixed audio signal stream in an audio or video call scenario or an audio or video playback scenario; wherein each frame of audio signal in the mixed audio signal stream randomly includes one or more of a speech signal, a music signal, and a noise signal;

[0013] a determination unit, configured to process the mixed audio signal stream using an audio category detection algorithm to determine the audio category to which each frame of audio signal in the mixed audio signal stream belongs; wherein the audio categories classified by the audio category detection algorithm include: a first audio category primarily based on speech, a second audio category primarily based on music, and a third audio category including noise and / or silence;

[0014] an adjustment unit, configured to adjust the volume gain of each frame of audio signal in the mixed audio signal stream by calling a volume adjustment strategy that matches the audio category to which each frame of audio signal belongs; wherein different audio categories correspond to different volume adjustment strategies;

[0015] The output unit is configured to output the audio signals of each frame after the volume gain is adjusted.

[0016] A third aspect of this specification provides a computer-readable storage medium having a computer program stored thereon, wherein the program implements the steps of the aforementioned method when executed by a processor.

[0017] In a fourth aspect of this specification, a device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the aforementioned method when executing the program.

[0018] In a fifth aspect of this specification, a smart car is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the aforementioned method when executing the program.

[0019] Through one or more embodiments of this specification, this specification has the following beneficial effects or advantages:

[0020] The technical solution of this specification obtains a mixed audio signal stream that randomly contains one or more of voice signals, music signals, and noise signals in an audio or video call or playback scene; uses an audio type detection algorithm to classify and detect each frame of audio signals in the mixed audio signal stream, determines the audio category to which each frame of audio signals belongs, and then calls a volume adjustment strategy that matches its audio category for each frame of audio signal to adjust the volume gain. It can be seen that this technical solution is an overall strategy for adjusting the audio volume by automatically switching the volume adjustment strategy according to the category for each frame of audio signals in the mixed audio signal stream. It can not only achieve volume adjustment of voice signals, but also make targeted adjustments to music signals, noise or silence. Compared with traditional voice volume adjustment, it can perform compatible control of volume gain according to the category to which each frame of audio signals belongs, thereby adapting to audio volume adjustment services in more complex scenarios.

[0021] The above description is only an overview of the technical solution of this specification. In order to more clearly understand the technical means of this specification, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of this specification more obvious and easy to understand, the specific implementation methods of this specification are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present description. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0023] Figure 1 A flowchart of a method for processing mixed audio according to an embodiment of the present specification is shown;

[0024] Figure 2 shows a flowchart of an implementation of a music volume adjustment strategy according to one embodiment of this specification;

[0025] Figure 3 FIG2 shows a logic diagram for implementing gain adjustment of a current frame audio signal and a spliced ​​frame audio signal according to one embodiment of the present specification;

[0026] Figure 4 A schematic diagram showing the overall concept of a hybrid audio processing solution according to an embodiment of the present specification is shown;

[0027] Figure 5 A schematic diagram of a mixed audio processing system according to an embodiment of the present specification is shown;

[0028] Figure 6 A schematic diagram of a device according to an embodiment of the present specification is shown. DETAILED DESCRIPTION

[0029] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0030] Glossary:

[0031] Ultra-wideband audio: refers to audio signals with a sampling rate of 32kHz. It is mainly used in 4G / 6G communications, high-quality voice calls, and general audio playback scenarios, such as online music playback.

[0032] Full-band audio: refers to an audio signal with a sampling rate of 48kHz, which can fully cover the frequency range audible to the human ear. It is mainly used in professional music production and professional music playback scenarios, such as audio playback in movie theaters.

[0033] AGC algorithm is an algorithm that performs gain control on the input signal to adjust the signal volume.

[0034] SpeechAgc algorithm, speech automatic gain control algorithm, is used to perform gain control on the input speech signal to adjust the volume of the speech signal.

[0035] MusicAgc algorithm, music automatic gain control algorithm, is used to perform gain control on the input music signal to adjust the volume of the music signal.

[0036] The first aspect of this specification discloses a method for processing mixed audio, which can be applied to audio and video call scenarios or audio and video playback scenarios. Audio and video call scenarios include phone calls, game interactions, online conferences, and other scenarios requiring audio or video calls. Audio and video playback scenarios include remote education and training, online music listening, movie screenings, and short video browsing, which require audio or video playback.

[0037] It is worth noting that the technical solutions in this specification can be built into electronic devices with audio or video call functions, or multimedia devices with audio or video playback functions, or used in third-party applications, such as in car computers, mobile phones, PCs (Personal Computers), televisions, projectors, music players and other devices, and in various remote training applications, audio and video playback platforms, and game platforms; of course, the technical solutions in this specification can also be integrated into mini-programs or third-party applications, such as as a music adjustment APP for users to download and use.

[0038] In order to explain and illustrate the technical solutions in this specification in detail, please refer to Figure 1 The mixed audio processing method in this manual mainly includes the following steps:

[0039] S101: Obtain a mixed audio signal stream in an audio and video call scenario or an audio and video playback scenario.

[0040] The mixed audio signal stream may be an ultra-wideband audio signal stream or a full-band audio signal stream, and is mainly composed of a plurality of continuous frames of audio signals.

[0041] In this embodiment, a mixed audio signal stream is obtained using a microphone or an application-class interface. For example, the mixed audio signal stream is picked up by a microphone of a call application or a web conferencing application, or an interface is used to intercept a mixed audio signal stream from a distance education application or an audio and video playback application.

[0042] Due to the unpredictability of audio and video call scenarios or audio and video playback scenarios, the signal types contained in each frame of audio signals in the mixed audio signal stream obtained in this application will be random, and each frame of audio signals randomly contains one or more signals among voice signals, music signals, and noise signals.

[0043] For example, if the first frame of audio signal picked up by the microphone contains only a voice signal, and the second frame of audio signal picked up contains music accompaniment, then the second frame of audio signal contains music signal + voice signal. For another example, if the first frame of audio signal picked up by the microphone contains music signal + voice signal, and the second frame of audio signal picked up contains ambient noise, then the second frame of audio signal contains music signal + voice signal + noise signal.

[0044] In order to accurately detect the type of audio signal of each frame in the mixed audio signal stream, an audio type detection algorithm is used to process the mixed audio signal stream.

[0045] S102: Process the mixed audio signal stream using an audio category detection algorithm to determine the audio category to which each frame of audio signal in the mixed audio signal stream belongs.

[0046] Among them, the basic network architecture of the audio type detection algorithm adopts a neural network. The basic network architecture of the audio type detection algorithm can be adaptively adjusted according to actual needs, and this specification does not form a limitation.

[0047] For example, if this solution is implemented on a device with limited computing resources, such as an in-car system, the neural network for the audio category detection algorithm uses a lightweight network architecture: one 1D convolutional layer → one fully connected layer → one or two gated recurrent unit layers → a final fully connected layer. The final fully connected layer outputs the initial probabilities of the three audio categories.

[0048] The three audio categories divided by the audio type detection algorithm include: a first audio category mainly composed of speech, a second audio category mainly composed of music, and a third audio category including noise and / or silence.

[0049] In order to enable the audio type detection algorithm to accurately detect the audio type of each frame of audio signal in the mixed audio signal stream in different scenarios, different category annotations are performed on the same audio frame sample with reference to different application scenarios to train audio type detection algorithms that are suitable for different application scenarios.

[0050] Taking the audio and video playback scenario as an example, considering the needs of audio and video playback, it is necessary to focus on music signals. The audio frame samples are labeled according to the following method:

[0051] If an audio frame sample contains speech but no music, the audio category is marked as 0. 0 represents the first audio category, which is primarily speech. For example, if an audio frame sample contains speech, or speech plus noise, its audio category is marked as 0.

[0052] If a music signal is present in an audio frame sample, the audio category is marked as 1. 1 represents a secondary audio category primarily composed of music. For example, if an audio frame sample contains: music signal, or speech signal + music signal, or speech signal + music signal + noise signal, or music signal + noise signal, its audio category is marked as 1.

[0053] If only noise or silence exists in the audio frame sample, the audio category is marked as 2. Here, 2 represents the third audio category containing noise and / or silence.

[0054] It's worth noting that for audio and video call scenarios, given the call requirements, the focus is on the voice signal, so the audio categories of the same audio frame samples need to be re-labeled. See Table 1 for a reference table of audio labeling for two different scenarios.

[0055] Table 1

[0056]

[0057] In Table 1, the audio categories marked on the same audio frame sample are different in different application scenarios (audio and video call scenario and audio and video playback scenario).

[0058] If the neural network is trained according to the audio frame samples marked in the audio and video call scenario, the trained audio type detection algorithm will classify the audio signal as the first audio category when it detects that the audio signal contains a voice signal + a music signal.

[0059] If the neural network is trained according to the audio frame samples marked in the audio and video playback scenario, the trained audio type detection algorithm will classify the audio signal as the second audio category when it detects that the audio signal contains voice signal + music signal.

[0060] For the same audio signal frame, the audio type detection algorithm trained using different annotation methods outputs different audio types in different application scenarios (audio and video call scenario and audio and video playback scenario). The audio type detection algorithm trained in this embodiment can accurately identify the category of the audio signal in different application scenarios.

[0061] The specific implementation process of the audio type detection algorithm is introduced below.

[0062] For each audio signal frame in the mixed audio signal stream, characteristic parameters of the current audio signal frame are extracted. The current audio signal frame is any one of the audio signal frames. Specifically, characteristic parameters that facilitate distinguishing between music signals, speech signals, and noise signals are extracted from the audio signal to form an n-dimensional feature vector, including but not limited to: energy, correlation, harmonics, audio signal flatness, and speech spectrum width of relevant frequency points. The characteristic parameters of the current audio signal frame are processed using an audio category detection algorithm to obtain an initial probability that the current audio signal frame belongs to each audio category.

[0063] In an optional implementation, VAD (Voice Activity Detection) obtained from the outside assists the audio type detection algorithm in determining the category of each frame of audio signal.

[0064] To prevent audio category detection errors caused by frequent switching between audio signal categories, a smoothing operation is performed after determining the initial probabilities of the current audio signal belonging to each audio category. For example, the initial probabilities can be smoothed based on the historical smoothed probabilities and smoothing coefficients of each audio category to ultimately determine the audio category of the current audio signal. Another example is the historical number of occurrences of the maximum initial probability to determine the audio category of the current audio signal.

[0065] In an optional smoothing operation mode, the initial probabilities of each audio category are smoothed by combining the historical smoothing probabilities and smoothing coefficients of each audio category to obtain the final probabilities that the current frame audio signal belongs to each audio category. Based on the final probabilities that the current frame audio signal belongs to each audio category, the audio category to which the current frame audio signal belongs is determined.

[0066] In a specific implementation process, the initial probabilities that the current frame audio signal belongs to each audio category are: the initial probability S that the audio signal belongs to the first audio category 0初 , the initial probability S that the audio signal belongs to the second audio category 1初 , the initial probability S that the audio signal belongs to the third audio category 2初 The historical smoothing probabilities and smoothing coefficients corresponding to different audio categories are also different. After obtaining the historical smoothing probabilities and smoothing coefficients corresponding to each audio category, smoothing calculations are performed according to the smoothing formula to obtain the final probability of each audio category.

[0067] The smoothing formula is as follows:

[0068] S=α×S 历史 +(1-α)×S 初

[0069] Among them, S represents the final probability, α represents the smoothing coefficient, which is derived from the actual simulation results. sp represents the sampling rate of the audio signal, T represents the smoothing period, which is generally set to 20 frames as a smoothing period, S 历史 represents the historical smoothing probability, S 初 represents the initial probability.

[0070] According to the above formula, the final probability S0 that the audio signal belongs to the first audio category, the final probability S1 that the audio signal belongs to the second audio category, and the final probability S2 that the audio signal belongs to the third audio category are calculated. The maximum probability is determined from S0, S1, and S2, and the audio category corresponding to the maximum probability is used as the audio category to which the audio signal belongs. For example, if the final probability S1 of the second audio category is the maximum among the three final probabilities, the audio category to which the audio signal belongs is determined to be the second audio category.

[0071] In an optional smoothing operation mode, the initial probabilities of the current frame audio signal belonging to each audio category are stored in respective buffer areas according to the category; the number of occurrences of the maximum initial probability values ​​in each buffer area is counted, and the audio category to which the current frame audio signal belongs is determined based on the number of occurrences.

[0072] Specifically, there are three buffers configured for each audio category: the s-buffer for the first audio category, the m-buffer for the second audio category, and the n-buffer for the third audio category. These three buffers use a first-in, first-out mechanism, and their storage lengths are pre-configured.

[0073] See Table 2, which shows the initial probabilities stored in each of the three buffer areas.

[0074] Table 2

[0075] s-buffer m-buffer n-buffer First storage <![CDATA[S 0初 (Maximum)]]> <![CDATA[S 1初 ]]> <![CDATA[S 2初 ]]> Second storage <![CDATA[S 0初 ]]> <![CDATA[S 1初 (Maximum)]]> <![CDATA[S 2初 ]]> 3rd storage <![CDATA[S 0初 ]]> <![CDATA[S 1初 (Maximum)]]> <![CDATA[S 2初 ]]> 4th storage <![CDATA[S 0初 ]]> <![CDATA[S 1初 ]]> <![CDATA[S 2初 (Maximum)]]> 5th storage (current) <![CDATA[S 0初 ]]> <![CDATA[S 1初 (Maximum)]]> <![CDATA[S2 初 ]]>

[0076] In Table 2, the first storage is the initial probability of the first audio signal belonging to each audio category, the second storage is the initial probability of the second audio signal belonging to each audio category, and so on, the initial probability of the current frame audio signal belonging to each audio category is stored in the respective buffer area according to category. Specifically, the initial probability S of the current frame audio signal belonging to the first audio category is stored. 0初 Stored in the s-buffer corresponding to the fifth storage, the initial probability S of the current frame audio signal belonging to the second audio category 1初 Stored in the m-buffer corresponding to the fifth storage, the initial probability S that the current frame audio signal belongs to the third audio category 2初 Store it in the n-buffer corresponding to the 5th storage.

[0077] In the process of determining the audio category to which the current frame audio signal belongs, the audio category corresponding to the buffer area with the largest number of occurrences of the initial probability maximum value is determined as the audio category to which the current frame audio signal belongs. Referring to Table 2, if the initial probability maximum value that appears in the buffer area m-buffer appears the most times, then the audio category to which the current frame audio signal belongs is determined to be the second audio category.

[0078] It is worth noting that if the number of occurrences of the initial probability maximum values ​​in two buffers is the same, the audio category of the current frame audio signal is determined by referring to the audio category corresponding to the buffer where the initial probability maximum value of the previous frame audio signal is located. For example, if the initial probability maximum value of the current frame audio signal belonging to the first audio category is S 2初 The n-buffer corresponding to the fifth storage is stored. If the number of initial probability maxima in the buffer m-buffer and the buffer n-buffer is equal, then the audio category corresponding to the buffer where the initial probability maxima during the fourth storage is located is referenced, that is, the third audio category corresponding to the buffer n-buffer. The third audio category is used as the audio category to which the audio signal of the current frame belongs.

[0079] Through the above-mentioned smoothing operation, it is possible to avoid, to a great extent, the audio category detection errors caused by the frequent switching of the types of the audio signals of each frame, thereby ensuring the accuracy of the audio category to which the audio signals of each frame belong.

[0080] After obtaining the audio category of each frame of audio signal, execute S103.

[0081] S103 : For each frame of audio signal in the mixed audio signal stream, a volume adjustment strategy that matches the audio category to which each frame of audio signal belongs is called to adjust the volume gain.

[0082] Volume adjustment policies include: a speech volume adjustment policy for adjusting speech signals, a music volume adjustment policy for adjusting music signals, and a noise reduction adjustment policy for adjusting noise and / or silence. Different audio categories correspond to different volume adjustment policies. Specifically, the first audio category uses the speech volume adjustment policy, the second audio category uses the music volume adjustment policy, and the third audio category uses the noise reduction adjustment policy.

[0083] Each frame of audio signal in the mixed audio signal stream is processed according to the following rules:

[0084] If the audio category of the current audio frame in the mixed audio signal stream is the first audio category, a speech volume adjustment strategy that matches the first audio category is invoked to adjust the volume gain of the current audio frame. The current audio frame refers to any audio frame in the mixed audio signal stream. The speech volume adjustment strategy uses the SpeechAgc algorithm to calculate the gain of the speech signal in the current audio frame, and uses the calculated gain to adjust the volume gain of the current audio frame.

[0085] If the audio category to which the current frame audio signal belongs is the second audio category, a music volume adjustment strategy that matches the second audio category is called to perform volume gain adjustment on the current frame audio signal.

[0086] In the music volume adjustment strategy, the MusicAgc algorithm is used to adjust the volume gain of the current frame audio signal.

[0087] See below Figure 2 , introduces the implementation process of music volume adjustment strategy.

[0088] In the music volume adjustment strategy, the current frame audio signal is subjected to noise reduction processing to remove the background noise in the current frame audio signal or weaken the noise in the current frame audio signal. Of course, the noise reduction processing operation can be selectively performed according to the characteristics of the audio signal itself.

[0089] Furthermore, the current frame audio signal is sent to the main link and the branch link for parallel processing. Optionally, the current frame audio signal can be copied into two identical audio signals, one audio signal is input to the main link for processing, and the other audio signal is input to the branch link for processing, and the two links are processed in parallel.

[0090] In the branch link, step S201 is executed to calculate the sample-level volume gain value of the current frame audio signal.

[0091] The sample-level volume gain value refers to a volume gain value calculated in units of sampling points, and actually includes the volume gain values ​​of all sampling points in the current frame audio signal.

[0092] Since the current frame audio signal is a continuous signal, the current frame audio signal is sampled according to the sampling rate and the volume gain value is calculated to obtain the sample-level volume gain value in the current frame audio signal.

[0093] During the specific calculation process, several real-time parameter values ​​for gain calculation are obtained from each sampling point of the current frame audio signal. Sample-level gain calculation is performed with reference to these real-time parameter values ​​for each sampling point, resulting in an initial sample-level volume gain value for the current frame audio signal. Specifically, each sampling point has its own real-time parameter values. These real-time parameter values ​​for each sampling point are combined with pre-set parameters, and the initial volume gain value corresponding to each sampling point is calculated on a per-sampling basis, ultimately resulting in the initial sample-level volume gain value for the current frame audio signal. Furthermore, the current real-time loudness of the current frame audio signal is calculated. This current real-time loudness is frame-level loudness, calculated on a per-frame basis according to preset loudness calculation rules.

[0094] Optionally, in order to prevent the current real-time loudness from changing suddenly, a smoothing formula can be used to smooth the current real-time loudness. The specific smoothing formula is:

[0095] L'=α'×L+(1-α')×L d

[0096] Among them, L' represents the smoothed loudness, α' represents the loudness smoothing coefficient, L represents the historical smoothed loudness, L d Indicates the current real-time loudness.

[0097] Furthermore, a frame-level volume gain compensation value for the current frame of the audio signal is determined based on the current real-time loudness, the target loudness, and a loudness gain control parameter. The loudness gain control parameter is used to control the gain progression based on the current real-time loudness. Within the constraints of the target loudness, the frame-level volume gain compensation value is obtained based on the current real-time loudness and the loudness gain control parameter.

[0098] The sample-level volume gain initial value and the frame-level volume gain compensation value are sample-level fused to obtain the sample-level volume gain value. Sample-level fusion refers to gain fusion performed on a sampling basis.

[0099] During the sample-level fusion process, the initial sample-level volume gain value is fused with the frame-level volume gain compensation value for each sample point, obtaining the volume gain value for each sample point. An example fusion method is additive fusion, but this is not a limitation.

[0100] Furthermore, a redundant threshold value is used to limit the sample-level volume gain value. Specifically, the redundant threshold value and the sample-level volume gain value are used for monitoring; if the sample-level volume gain value exceeds the redundant threshold value, the redundant threshold value is used to replace the sample-level volume gain value. With this design, the sample-level volume gain values ​​of the current frame audio signal are all within the redundant threshold value, so that the current frame audio signal has redundancy from clipping, which can improve comfort. For example, general digital audio signal processing uses 0db as the clipping threshold. In this solution, -2db is set as the redundant threshold value. If the obtained sample-level volume gain value makes the sample after gain application greater than -2db, the redundant threshold (-2db) will be referenced to limit the gain applied at this time so that the signal after gain application is near the redundant threshold to prevent signal clipping (greater than or equal to 0db). After the current frame audio signal is processed in this way, it will not produce clipping when transmitted to other modules for processing or transmitted to a player for playback, which can improve comfort.

[0101] S202 is executed in the main link. The tail signal of the current frame audio signal and the previous frame audio signal are intercepted according to the set duration, and the tail signal of the previous frame audio signal is spliced ​​to the head of the current frame audio signal to obtain a spliced ​​frame audio signal. For example, the tail signal of the previous frame audio signal is intercepted for D milliseconds and spliced ​​to the head of the current frame audio signal. The tail of the current frame audio signal is then intercepted for D milliseconds for use in the next frame. Between two adjacent frames of the same audio category, due to the continuity of the audio signal itself, the gains calculated for the previous and next frames are also continuous and smooth. Therefore, the audio signals after the gains are applied to the previous and next frames are also smooth. In addition, the tail signal of the previous frame audio signal is spliced ​​for D milliseconds to the head of the current frame audio signal. The gain is adjusted based on the head of the current frame for D milliseconds, which can smooth out clipping caused by abnormal fluctuations between the previous and next frames. Because the gain is calculated after a delay of D milliseconds, the signal in the first D milliseconds can be better controlled to prevent excessive gain. For example, if the signal suddenly increases within the current D milliseconds, the calculated gain will become smaller or even negative (indicating an attenuated signal). Therefore, it is necessary to start controlling the gain amplitude D milliseconds in advance to prevent clipping caused by untimely gain adjustment.

[0102] After the spliced ​​frame audio signal is obtained, step S203 is executed to adjust the sample-level volume gain of the spliced ​​frame audio signal using the sample-level volume gain value of the current frame audio signal, wherein the volume gain of the spliced ​​frame audio signal is adjusted in units of sampling points.

[0103] During the volume gain adjustment process, a mapping relationship is established between each sampling point in the current audio frame and each sampling point in the spliced ​​audio frame according to a set duration. Each sampling point in the current audio frame has a time delay of the set duration from its corresponding sampling point in the spliced ​​audio frame. Based on this mapping relationship, the volume gain value of each sampling point in the current audio frame is applied to the volume sample value of the corresponding sampling point in the spliced ​​audio frame.

[0104] See below Figure 3 , is an implementation logic diagram of gain adjustment of the current frame audio signal and the spliced ​​frame audio signal.

[0105] The sampling points in the current audio frame are 1, 2, ..., n. The spliced ​​audio frame is composed of the tail signal of the previous audio frame, so its sampling points are 0, 1, 2, ..., n-1. The sampling points between 0 and 1 represent the tail signal of the previous audio frame. Because the spliced ​​audio frame has a delay of D milliseconds relative to the current audio frame, the sampling points of the current audio frame and the corresponding sampling points in the spliced ​​audio frame are separated by D milliseconds.

[0106] When adjusting the volume gain, the volume gain value of sampling point 1 in the current frame audio signal is applied to the volume sampling value corresponding to sampling point 0 in the spliced ​​frame audio signal. The sampling point 1 in the current frame audio signal and its corresponding sampling point in the spliced ​​frame audio signal are sampling point 0, and the interval between the two is set to D milliseconds. The volume gain value of sampling point 2 in the current frame audio signal is applied to the volume sampling value corresponding to sampling point 1 in the spliced ​​frame audio signal, and the sampling point 2 in the current frame audio signal and its corresponding sampling point in the spliced ​​frame audio signal are sampling point 1, and the interval between the two is set to D milliseconds. This process is repeated until the volume gain values ​​of all sampling points in the current frame audio signal are adjusted, thereby achieving volume amplification or volume reduction control of the current frame audio signal.

[0107] When adjusting the volume gain, the volume gain value of the head sampling point in the current frame audio signal is adjusted to the tail signal of the previous frame audio signal spliced ​​in the spliced ​​frame audio signal, and the volume gain values ​​of other sampling points in the current frame audio signal are staggered and adjusted to the corresponding sampling points in the spliced ​​frame audio signal according to the set time interval, so that the adjusted entire spliced ​​frame audio signal can avoid clipping caused by abnormal fluctuations, making the spliced ​​frame audio signal smoother.

[0108] Furthermore, after adjusting the sample-level volume gain of the spliced ​​frame audio signal using the sample-level volume gain value, in order to smooth the gain-adjusted music signal and achieve comprehensive preservation of the music sound effects, a weighted superposition is performed on the spliced ​​frame audio signal and the spliced ​​frame audio signal after the volume gain adjustment to obtain an adjusted audio signal. The weighted superposition is performed on the spliced ​​frame audio signal before and after the volume gain adjustment, using sampling points as units, and the weights used in the weighted superposition are determined based on empirical values.

[0109] Furthermore, the weighted superimposed spliced ​​frame audio signal is subjected to a limiting process to prevent signal clipping. Specifically, a preset threshold is used to limit the volume of each sampling point in the spliced ​​frame audio signal. When the volume exceeds the preset threshold, the preset threshold is used to replace the volume value at the sampling point, so that the sample-level volume value of the audio frame is within the preset threshold.

[0110] The above is the processing logic for the current frame audio signal of the second audio category.

[0111] Traditional volume adjustment solutions generally adopt a unified volume adjustment method, which cannot realize automatic switching and adjustment of different types of audio signals to improve the user's listening experience. For example, in entertainment and social scenarios, it is impossible to achieve targeted adjustment of voice signals and music signals to improve the user's listening experience, especially the music signal. In this technical solution, the MusicAgc algorithm is used to make targeted adjustments to the music signal, so that the volume adjustment solution in this manual can not only adjust audio signals such as voice and noise, but also make special adjustments to music signals such as music accompaniment and playing music. It can perform compatible control of volume gain according to the type of audio signal, thereby adapting to audio volume adjustment services in more complex scenarios.

[0112] If the audio category to which the current frame audio signal belongs is the third audio category, a noise reduction adjustment strategy that matches the third audio category is called to perform volume gain adjustment on the current frame audio signal.

[0113] In the noise adjustment strategy, the sample-level noise gain value of the current frame audio signal is calculated. Specifically, the SpeechAgc algorithm or the MusicAgc algorithm can be used to calculate the sample-level noise gain value of the current frame audio signal. The specific calculation process has been described above and will not be repeated here.

[0114] The sample-level noise gain value is subjected to gain attenuation in different degrees with reference to the current real-time loudness of the current frame audio signal and / or the initial probability that the current frame audio signal belongs to the third audio category.

[0115] Here, the sample level noise gain value is attenuated according to the following rules.

[0116] Rule 1: The current real-time loudness is less than a first threshold, and the initial probability that the current frame audio signal belongs to the third audio category determines that the audio frame belongs to the third audio category of noise or silence, and the sample-level noise gain value is attenuated by a first attenuation value.

[0117] Rule 2: The initial probability that the current frame audio signal belongs to the third audio category determines that the audio frame belongs to a noise signal in the third audio category and the initial probability is higher than a second threshold, and the sample-level noise gain value is attenuated by a first attenuation value.

[0118] Rule 3: The initial probability that the current frame audio signal belongs to the third audio category determines that the category to which the audio frame belongs is a noise signal in the third audio category and the initial probability is lower than a second threshold, and the sample-level noise gain value is attenuated by a second attenuation value; wherein the second attenuation value is less than the first attenuation value.

[0119] Rule 4: The initial probability that the current frame audio signal belongs to the third audio category determines that the audio frame belongs to a silent signal in the third audio category and the current real-time loudness is greater than the first threshold, and the sample-level noise gain value is attenuated by a second attenuation value.

[0120] The volume gain of the current frame audio signal is adjusted using the sample-level noise gain value after gain attenuation.

[0121] The above is the processing logic for the current frame audio signal of the third audio category.

[0122] It is worth noting that the various gain adjustment strategies in this specification will generate some intermediate variables and parameters during the internal calculation process. When the type of each frame audio signal is switched, the intermediate variables and parameters generated by the gain adjustment strategy corresponding to the previous frame audio signal need to be saved in the relevant structure, and the relevant structure of the gain adjustment strategy corresponding to the current frame audio signal needs to be called to realize the calculation update of the gain adjustment strategy.

[0123] It is worth noting that since the signal types contained in each frame of audio signals in the mixed audio signal stream are random, after detection using the audio type detection algorithm, there are cases where the audio categories of the previous and next audio signals are the same, and there are also cases where the audio categories of the previous and next audio signals are different.

[0124] Considering that the mixed audio signal stream is a continuous audio signal, the gain of each frame of the audio signal needs to be kept smooth between frames to further ensure the smoothness of the mixed audio signal stream output after the gain is applied.

[0125] When the audio signals of the preceding and following frames belong to the same audio category, since the interior of a single audio signal is smooth and the preceding and following frame audio signals of the same category are processed using the same volume adjustment strategy, the preceding and following frame audio signals and the intervals between frames can maintain smoothness.

[0126] When the audio categories of the previous and subsequent frames are different, since the previous and subsequent frames use different gain adjustment strategies, such as SpeechAgc, MusicAgc or noise adjustment strategies, the gains of the tail of the previous frame audio signal and the head of the current frame audio signal will have a sudden change. For example, the gain calculation of the previous frame audio signal is for the speech signal, and the gain calculation of the current frame audio signal is for the noise signal or the silence signal; or the gain calculation of the previous frame audio signal is for the speech signal, and the gain calculation of the current frame audio signal is for the music signal, then the gain value of the last sampling point of the previous frame audio signal and the gain value of the first sampling point of the current frame audio signal will have a sudden change.

[0127] Therefore, in order to maintain inter-frame smoothness when the audio categories of the audio signals of the previous and next frames are different, after executing step S103, if the audio category of the current frame audio signal is different from the audio category of the previous frame audio signal, the gain value of the last sample point of the previous frame audio signal is used to perform inter-frame smoothing processing on the gain values ​​of the first n sampling points of the current frame audio signal, where n ≥ 2 and is a positive integer.

[0128] In an optional inter-frame smoothing method, a gain value of the last sample of the previous audio frame and a gain value of the nth sample corresponding to the beginning of the current audio frame are determined. Where n ≥ 2 and is a positive integer, a gain difference between the gain value of the nth sample and the gain value of the last sample is determined. The gain difference is then used to smooth the gain values ​​of the first n-1 sample points corresponding to the beginning of the current audio frame.

[0129] For example, the gain value of the last sample point at the end of the previous frame audio signal is subtracted from the gain value of the third sample point at the head of the current frame audio signal to obtain a gain difference, and the gain difference is used to replace the gain values ​​of the first two sample points at the head of the current frame audio signal, thereby achieving inter-frame gain smoothing and ensuring the smoothness of the continuous mixed audio signal stream output.

[0130] In an optional inter-frame smoothing processing method, the gain value of the tail sample point of the previous frame audio signal and the gain value of the first n sample points corresponding to the head of the current frame audio signal are determined. n ≥ 2 and is a positive integer; the gain difference between the gain value of the nth sample point and the gain value of the tail sample point is determined; and the n gain differences are used to perform a smoothing calculation to obtain a gain smoothing value. For example, the n gain differences are averaged to obtain a gain smoothing value. The gain smoothing value is used to smooth the gain values ​​of the first n sample points corresponding to the head of the current frame audio signal. The gain smoothing value is used to replace the gain values ​​of the first n sample points corresponding to the head of the current frame audio signal, thereby achieving inter-frame gain smoothing and ensuring the smoothness of the output of the continuous mixed audio signal stream.

[0131] S104: Outputting each frame of audio signal after volume gain adjustment.

[0132] Specifically, after the volume gain of each frame of audio signal is adjusted, the audio signal of each frame after the volume gain adjustment is output to the corresponding application through the earpiece or interface for playback.

[0133] In this technical solution, for each frame of audio signal in the mixed audio signal stream, an overall strategy of audio volume adjustment is implemented by automatically switching the volume adjustment strategy by category. This strategy can not only adjust the volume of the voice signal, but also make targeted adjustments to music signals, noise or silence. Compared with traditional voice volume adjustment, it can perform compatible control of the volume gain according to the category of each frame of audio signal, thereby adapting to audio volume adjustment services in more complex scenarios.

[0134] In addition, this technical solution proposes a gain strategy selection and inter-frame smoothing processing solution to ensure switching between different types of audio signals and support smooth switching of user listening experience.

[0135] To illustrate and explain the solutions in this specification, see Figure 4 , introduces the overall concept of the hybrid audio processing solution in this manual.

[0136] S401: Input a mixed audio signal stream in an audio or video call scenario or an audio or video playback scenario into an audio category detection algorithm, and output the category to which each frame of audio signal in the mixed audio signal stream belongs.

[0137] S402 : For each frame of audio signal in the mixed audio signal stream, call a volume adjustment strategy that matches the audio category to which each frame of audio signal belongs to perform volume gain adjustment.

[0138] The category of each frame of audio signal is one of the first audio category, the second audio category, and the third audio category.

[0139] If the audio signal belongs to the first audio category, the voice volume adjustment strategy is called to adjust the volume gain.

[0140] If the audio signal belongs to the second audio category, the music volume adjustment strategy is called to adjust the volume gain.

[0141] If the audio signal belongs to the third audio category, the noise reduction adjustment strategy is called to adjust the volume gain.

[0142] After each frame of audio signal is adjusted, S403 is executed to output it frame by frame.

[0143] S404, determine whether it is the last frame, if so, stop, if not, proceed to the next frame processing.

[0144] Under the overall concept of this technical solution, for each frame of audio signal in the mixed audio signal stream, an overall strategy of audio volume adjustment is implemented by automatically switching the volume adjustment strategy by category, and compatible control of the volume gain is performed according to the category of each frame of audio signal, thereby adapting to audio volume adjustment services in more complex scenarios.

[0145] Based on the same inventive concept as in the above embodiment, the third aspect of this specification provides a mixed audio processing system, see Figure 5 , the system further comprises:

[0146] An acquisition unit 501 is configured to acquire a mixed audio signal stream in an audio or video call scenario or an audio or video playback scenario; wherein each frame of audio signal in the mixed audio signal stream randomly includes one or more of a speech signal, a music signal, and a noise signal;

[0147] a determination unit 502 configured to process the mixed audio signal stream using an audio category detection algorithm to determine the audio category to which each audio signal frame in the mixed audio signal stream belongs; wherein the audio categories classified by the audio category detection algorithm include: a first audio category primarily characterized by speech, a second audio category primarily characterized by music, and a third audio category comprising noise and / or silence;

[0148] an adjusting unit 503 configured to adjust the volume gain of each frame of audio signal in the mixed audio signal stream by calling a volume adjustment strategy that matches the audio category to which each frame of audio signal belongs; wherein different audio categories correspond to different volume adjustment strategies;

[0149] The output unit 504 is configured to output the audio signal frames after the volume gain is adjusted.

[0150] Based on the same inventive concept as in the aforementioned embodiment, the third aspect of this specification provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any method embodiment in this specification when executed by a processor.

[0151] Based on the same inventive concept as in the above embodiment, the fourth aspect of this specification further provides a device, referring to Figure 6 , including a memory 64, a processor 62 and a computer program stored in the memory 64 and executable on the processor, wherein the processor 62 implements the steps of any of the aforementioned methods when executing the program.

[0152] Among them, Figure 6In the embodiment of the present invention, a bus architecture (represented by bus 60) is shown. Bus 60 may include any number of interconnected buses and bridges, and bus 60 connects various circuits including one or more processors represented by processor 62 and memory represented by memory 64. Bus 60 may also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. Bus interface 65 provides an interface between bus 60 and receiver 61 and transmitter 63. Receiver 61 and transmitter 63 may be the same component, namely a transceiver, which provides a unit for communicating with various other devices over a transmission medium. Processor 62 is responsible for managing bus 60 and general processing, while memory 64 may be used to store data used by processor 62 when performing operations.

[0153] In actual applications, the device can be an electronic device (mobile phone, PC, etc.), a multimedia player, a volume adjustment control device (such as a device that requires volume adjustment in social education or entertainment), a car computer, and other devices.

[0154] Based on the same inventive concept as in the aforementioned embodiment, the fifth aspect of this specification also provides an intelligent car, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in any method embodiment when executing the program.

[0155] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems may also be used in conjunction with the teachings herein. Based on the above description, it is apparent that the structure required to construct such systems is suitable. Furthermore, this specification is not directed to any particular programming language. It should be understood that various programming languages ​​may be utilized to implement the contents of this specification as described herein, and the above description of specific languages ​​is intended to disclose preferred embodiments of this specification.

[0156] In the description provided herein, numerous specific details are described. However, it is understood that the embodiments of this description may be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0157] Similarly, it should be understood that in order to streamline the present disclosure and aid understanding of one or more of the various inventive aspects, in the above description of the exemplary embodiments of this specification, various features of this specification are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting an intention that the claimed specification requires more features than are expressly recited in each claim. Rather, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Accordingly, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this specification.

[0158] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition may be divided into multiple submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed herein may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0159] Furthermore, those skilled in the art will appreciate that although some embodiments herein include certain features included in other embodiments but not other features, combinations of features from different embodiments are intended to be within the scope of this specification and to form different embodiments. For example, in the claims below, any of the claimed embodiments may be used in any combination.

[0160] The various component embodiments of this specification can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It will be appreciated by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functions of some or all of the components in the gateway, proxy server, or system according to the embodiments of this specification. This specification can also be implemented as a device or apparatus program (e.g., computer program and computer program product) for executing part or all of the methods described herein. Such a program implementing this specification can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0161] It should be noted that the above embodiments illustrate rather than limit this specification, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between brackets should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This specification may be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

Claims

1. A method for processing mixed audio, the method comprising: Obtaining a mixed audio signal stream in an audio and video call scenario or an audio and video playback scenario; wherein each frame of audio signal in the mixed audio signal stream randomly includes one or more of a speech signal, a music signal, and a noise signal; Processing the mixed audio signal stream using an audio category detection algorithm to determine the audio category to which each frame of audio signal in the mixed audio signal stream belongs; wherein the audio categories classified by the audio category detection algorithm include: a first audio category primarily based on speech, a second audio category primarily based on music, and a third audio category including noise and / or silence; For each frame of audio signal in the mixed audio signal stream, calling a volume adjustment strategy that matches the audio category to which each frame of audio signal belongs to perform volume gain adjustment; wherein different audio categories correspond to different volume adjustment strategies; Output the audio signals of each frame after volume gain adjustment.

2. The method according to claim 1, wherein processing the mixed audio signal stream using an audio category detection algorithm to determine the audio category to which each frame of audio signal in the mixed audio signal stream belongs comprises: For each frame of audio signal in the mixed audio signal stream, extracting characteristic parameters of the current frame of audio signal; Processing characteristic parameters of the current frame audio signal using the audio category detection algorithm to obtain initial probabilities that the current frame audio signal belongs to each audio category; performing a smoothing calculation on the initial probabilities of the same category by combining the historical smoothing probabilities and smoothing coefficients of the respective audio categories to obtain final probabilities that the current frame audio signal belongs to the respective audio categories; and determining the audio category to which the current frame audio signal belongs based on the final probabilities that the current frame audio signal belongs to the respective audio categories; or, The initial probabilities that the current frame audio signal belongs to each audio category are stored in respective buffer areas according to the categories; the number of occurrences of the maximum value of the initial probability in each buffer area is counted, and the audio category to which the current frame audio signal belongs is determined based on the number of occurrences.

3. The method according to claim 1 or 2, further comprising: after adjusting the volume gain of each audio signal frame in the mixed audio signal stream by invoking a volume adjustment strategy that matches the audio category to which each audio signal frame belongs, the method further comprising: If the audio category of the current frame audio signal is different from that of the previous frame audio signal, inter-frame smoothing processing is performed on the gain values ​​of the first n sampling points of the current frame audio signal using the gain value of the last sample point of the previous frame audio signal, where n is ≥ 2 and is a positive integer.

4. The method according to claim 1, wherein the volume adjustment strategy comprises: a voice volume adjustment strategy for adjusting the voice signal, a music volume adjustment strategy for adjusting the music signal, and a noise reduction adjustment strategy for adjusting noise and / or silence; The step of adjusting the volume gain of each frame of audio signal in the mixed audio signal stream by calling a volume adjustment strategy that matches the audio category to which each frame of audio signal belongs specifically includes: If the audio category to which the current frame audio signal in the mixed audio signal stream belongs is the first audio category, calling the voice volume adjustment strategy that matches the first audio category to adjust the volume gain of the current frame audio signal; If the audio category to which the current frame audio signal belongs is the second audio category, calling the music volume adjustment strategy that matches the second audio category to adjust the volume gain of the current frame audio signal; If the audio category to which the current frame audio signal belongs is the third audio category, the noise reduction adjustment strategy consistent with the third audio category is invoked to perform volume gain adjustment on the current frame audio signal.

5. The method according to claim 4, wherein calling the music volume adjustment strategy that matches the second audio category to adjust the volume gain of the current frame audio signal specifically comprises: In the music volume adjustment strategy, a sample-level volume gain value of the current frame audio signal is calculated; intercepting the tail signal of the current frame audio signal and the previous frame audio signal according to a set time length, and splicing the tail signal of the previous frame audio signal to the head of the current frame audio signal to obtain a spliced ​​frame audio signal; The sample-level volume gain of the spliced ​​frame audio signal is adjusted by using the sample-level volume gain value of the current frame audio signal.

6. The method according to claim 5, wherein calculating the sample-level volume gain value of the current frame audio signal specifically comprises: Acquire a plurality of real-time parameter values ​​for calculating gains from each sampling point of the current frame audio signal; Performing sample-level gain calculation with reference to a plurality of real-time parameter values ​​of each sampling point to obtain an initial value of the sample-level volume gain of the current frame audio signal; Calculating the current real-time loudness of the current frame audio signal; Determining a frame-level volume gain compensation value of the current frame audio signal according to the current real-time loudness, the target loudness, and the loudness gain control parameter; The sample-level volume gain initial value and the frame-level volume gain compensation value are sample-level fused to obtain the sample-level volume gain value.

7. The method according to claim 6, further comprising: performing sample-level fusion on the sample-level volume gain initial value and the frame-level volume gain compensation value to obtain the sample-level volume gain value; Monitoring is performed using a redundancy threshold value and the sample-level volume gain value; If the sample-level volume gain value exceeds the redundancy threshold value, the sample-level volume gain value is replaced by the redundancy threshold value.

8. The method according to claim 5, after performing sample-level volume gain adjustment on the spliced ​​frame audio signal using the sample-level volume gain value, the method further comprises: The spliced ​​frame audio signal and the spliced ​​frame audio signal after volume gain adjustment are weightedly superimposed to obtain an adjusted audio signal.

9. The method according to claim 4, wherein calling the noise reduction adjustment strategy that matches the third audio category to adjust the volume gain of the current frame audio signal specifically comprises: In the noise reduction adjustment strategy, a sample-level noise gain value of the current frame audio signal is calculated; performing different degrees of gain attenuation on a sample-level noise gain value of the current frame audio signal, referring to a current real-time loudness of the current frame audio signal and / or an initial probability that the current frame audio signal belongs to the third audio category; The volume gain of the current frame audio signal is adjusted by using the sample-level noise gain value after gain attenuation.

10. A mixed audio processing system, the system further comprising: An acquisition unit, configured to acquire a mixed audio signal stream in an audio or video call scenario or an audio or video playback scenario; wherein each frame of audio signal in the mixed audio signal stream randomly includes one or more of a speech signal, a music signal, and a noise signal; a determination unit, configured to process the mixed audio signal stream using an audio category detection algorithm to determine the audio category to which each frame of audio signal in the mixed audio signal stream belongs; wherein the audio categories classified by the audio category detection algorithm include: a first audio category primarily based on speech, a second audio category primarily based on music, and a third audio category including noise and / or silence; an adjustment unit, configured to adjust the volume gain of each frame of audio signal in the mixed audio signal stream by calling a volume adjustment strategy that matches the audio category to which each frame of audio signal belongs; wherein different audio categories correspond to different volume adjustment strategies; The output unit is configured to output the audio signals of each frame after the volume gain is adjusted.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 9 when executing the program.

13. An intelligent car comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 9 when executing the program.

Images