Volume self-adaptive adjusting method, live sound card, storage medium and electronic device

Abstract

The application discloses a volume self-adaptive adjusting method, a live sound card, a storage medium and an electronic device. The method comprises the following steps: acquiring a first audio signal transmitted by a live terminal and a second audio signal collected by a microphone; performing voice detection on the first audio signal and the second audio signal respectively to obtain effective audio data; collecting the effective audio data within a preset time length and calculating RMS values according to time blocks to obtain a first RMS sequence and a second RMS sequence; sorting the first RMS sequence and the second RMS sequence in ascending order respectively and extracting the first M maximum RMS values as peak value sequences; calculating the ratio of the RMS values at corresponding positions in the two peak value sequences to obtain a decibel difference sequence; performing least square fitting after removing abnormal values to obtain a final decibel difference value; and adjusting the gain of the second audio signal according to the final decibel difference value to align the volumes of the two audio signals. The application is executed in the time domain throughout, the calculation complexity is O(N), can be run in real time on a low-resource sound card, and can obtain stable and accurate alignment effects through peak comparison and fitting without manual intervention.

Description

Technical Field

[0001] This invention relates to the field of audio signal processing technology, specifically to a volume adaptive adjustment method for live streaming sound cards, a live streaming sound card implementing the method, and a computer-readable storage medium and electronic device incorporating the method. It is particularly suitable for automatic volume alignment between audio downloaded from a mobile phone (such as AI robot voice) and human voice audio captured by a microphone in unmanned live streaming scenarios. Background Technology

[0002] With the rapid development of the live streaming industry, new models such as unmanned live streaming and AI-assisted live streaming are gradually becoming more widespread. As a core audio processing device, the live streaming sound card needs to process two audio streams simultaneously in such scenarios: one stream is the AI ​​robot voice or background music downloaded from the live streaming terminal (e.g., a smartphone), and the other stream is the real host's voice captured from the microphone. Although the two audio streams may not appear simultaneously, their loudness (volume) must remain basically consistent to ensure a consistent listener experience.

[0003] However, most small live streaming sound cards on the market currently only offer basic mixing and equalization functions, lacking adaptive volume adjustment capabilities. Streamers often need to manually adjust microphone gain using knobs or commands to match the volumes of the two channels. This method is not only cumbersome but also slow to respond, failing to promptly address dynamic changes during live streaming, such as AI audio file changes or microphone repositioning.

[0004] Existing audio volume alignment techniques mainly fall into two categories. One is based on frequency domain analysis, which extracts the energy of the human voice frequency band by performing a Fast Fourier Transform (FFT) on the audio to compare loudness differences. This type of method has a computational complexity of at least O(N·logN) and requires a large amount of memory to store intermediate results, placing high demands on chip computing power and making it difficult to run in real time on low-power, low-cost small sound card main control chips. The other is a simple time-domain RMS averaging method, which directly calculates the difference between the RMS average values ​​of two audio sources over a period of time to adjust the gain. However, this method lacks targeted processing for human voice audio; low-energy data during speech intervals can severely interfere with the averaging calculation, resulting in differences that do not accurately reflect the loudness differences actually perceived by the human ear, leading to poor volume alignment. In addition, many automatic gain control schemes use fixed preset thresholds as a reference, failing to address the problem of real-time matching based on another dynamic audio source. Summary of the Invention

[0005] To address the aforementioned technical problem of accurately matching the volume differences between two audio streams at different times on resource-constrained embedded sound cards, this invention provides a volume adaptive adjustment method, a live streaming sound card, a computer-readable storage medium, and an electronic device.

[0006] In one aspect, the present invention provides a volume adaptive adjustment method, comprising: acquiring a first audio signal from a live streaming terminal and a second audio signal from a microphone; performing human voice detection on the first audio signal and the second audio signal respectively to obtain first valid audio data and second valid audio data; collecting the first valid audio data and the second valid audio data within a preset duration T, and dividing the valid audio data into multiple time blocks according to a preset time block length Δt, calculating the root mean square (RMS) value of the first valid audio data in each time block to obtain a first RMS sequence, and the root mean square (RMS) value of the second valid audio data to obtain a second RMS sequence; wherein, the first RMS sequence and the second RMS sequence each contain N RMS values, N = T / Δt; sorting the first RMS sequence and the second RMS sequence in ascending order respectively, and extracting the top M RMS values ​​with the largest values ​​as the first peak sequence and the second peak sequence; wherein, M < N; Calculate the decibel difference sequence based on the ratio of the RMS values ​​at corresponding sorting positions in the first peak sequence and the second peak sequence; Remove outliers from the decibel difference sequence to obtain the effective decibel difference sequence; Fit the effective decibel difference sequence to obtain the final decibel difference value; Adjust the gain of the second audio signal according to the final decibel difference value so that the volume of the adjusted second audio signal is aligned with the volume of the first audio signal.

[0007] The above method operates entirely in the time domain, with a computational complexity of O(N). Compared to frequency domain methods, it significantly reduces the computational demands on the chip, enabling real-time execution on low-cost, low-power sound card controllers. By first detecting human voices in both audio streams, interference from silent and background noise segments is effectively removed. The effective audio data is then divided into blocks, and RMS values ​​are calculated and sorted in ascending order to extract the peak values. This avoids the influence of low-energy values ​​during speech intervals on the average value, allowing subsequent comparisons to focus more on the main part of human voice loudness. By comparing the peak RMS values ​​at corresponding positions, a series of decibel differences reflecting the loudness difference between the two audio streams during the main sound periods is obtained. Further outlier removal and fitting further eliminate interference from occasional spikes or transient noise, resulting in a stable and reliable final decibel difference value. Based on this difference, the microphone audio gain is adjusted, enabling real-time, automatic, and accurate alignment of the volumes of the two audio streams, significantly improving the consistency of the listening experience for viewers of live content.

[0008] In some preferred embodiments, to balance sampling sufficiency and response speed, the preset duration T ranges from 5 to 15 seconds; the preset time block length Δt ranges from 10 milliseconds to 30 milliseconds; and the number M of the first M RMS values ​​satisfies the following: M as a percentage of N ranges from 15% to 25%. Too short a T will lead to insufficient samples and inaccurate fitting, while too long a T will reduce response speed. An excessively large M value will introduce low-energy data including speech intervals, while an excessively small M value will lose peak information. By optimizing the above parameter ranges, it is possible to maintain a rapid response to volume changes while ensuring the statistical significance of the data.

[0009] In some preferred embodiments, the voice detection specifically employs a variable single-threshold mechanism. First, using the RMS value as an energy feature, an initial audio segment is acquired upon the first input of each audio stream. The RMS values ​​of all time blocks within this audio segment are calculated, and the average of the lowest 25% of these RMS values ​​is taken as the threshold value. The condition for passing voice detection is that the number of times the energy value exceeds the threshold value within a preset time window reaches a preset threshold. This threshold initialization method can adapt to the background noise levels of different microphones or different live streaming terminals, effectively avoiding misjudgments or missed judgments that may be caused by a fixed threshold, and improving the robustness of voice detection.

[0010] In some preferred embodiments, to enable the algorithm to handle overall changes in audio loudness during live streaming, the method further includes a deviation judgment step. After each successful human voice detection, the upper and lower limits of the variable interval are determined based on the high-value distribution of the human voice audio in the initially acquired RMS sequence. When the number of times the energy value continuously exceeds the upper limit of the variable interval or continuously falls below the lower limit of the variable interval reaches a preset threshold within a preset time window, an overall change in audio loudness is determined; if the energy value partially exceeds the upper limit and partially falls below the lower limit within the same time window, no change is determined. When a change is determined, the acquired RMS sequence is cleared, and the steps of acquisition, calculation, sorting, extraction, ratio calculation, outlier removal, fitting, and adjustment are re-executed. This mechanism solves the long-term mismatch problem caused by the overall replacement of AI audio files or changes in microphone volume during live streaming, enabling the system to maintain volume alignment continuously in dynamic environments.

[0011] In some preferred embodiments, the outlier removal of the decibel difference series employs the interquartile range (IQR) box method, which includes: calculating the first quartile Q1 and the third quartile Q3 of the decibel difference series, and calculating the interquartile range IQR = Q3 - Q1; data points below Q1 - 1.5 × IQR or above Q3 + 1.5 × IQR are identified as outliers and removed. The IQR box method does not rely on the normality assumption of the data distribution and has higher robustness to outliers that may exist in actual audio decibel difference data, thus providing a better data foundation for subsequent fitting.

[0012] In some preferred embodiments, to ensure that the continuity of data acquisition is not interrupted by other tasks, the voice detection and deviation judgment are performed periodically only during idle periods when the first and second RMS sequences are not being acquired. During the RMS sequence acquisition process, the voice detection and deviation judgment are paused. This avoids introducing additional detection calculations while collecting energy data, ensuring the purity of the acquired RMS sequence and thus improving the accuracy of volume alignment.

[0013] On the other hand, the present invention also provides a live streaming sound card for implementing the above-described method. The sound card includes: a first interface for connecting to a live streaming terminal and receiving a first audio signal; a second interface for connecting to a microphone and receiving a second audio signal; and a main control MCU electrically connected to the first and second interfaces. The main control MCU is configured to execute the above-described volume adaptive adjustment method, align the volume of the first and second audio signals, mix the adjusted audio signals, and send the mixed audio data to the live streaming terminal through the first interface. This live streaming sound card directly integrates the above-described low-computational-complexity volume alignment algorithm into the hardware main control, achieving plug-and-play automatic volume balancing without additional user intervention.

[0014] In some preferred embodiments, the first interface of the live streaming sound card is a Type-C OTG interface, and the second interface is a three-pole 3.5mm audio jack. The sound card also includes a third interface, a fourth interface, and a switch. The third interface, a three-pole 3.5mm audio jack, is used to connect monitoring headphones and transmit the mixed audio signal unidirectionally. The fourth interface, a Type-C interface, is used to connect a charging adapter, supports fast charging protocols, and allows the charging adapter to power the sound card through the fourth interface and charge the live streaming terminal through the first interface. The switch is a two-position toggle switch, connected in series in the audio signal path of the second interface, used to control the on / off state of the microphone audio input. The main control MCU establishes USB AudioClass protocol communication with the live streaming terminal through the DP / DN pin of the first interface to transmit audio data. This structure integrates power supply, charging, monitoring, and transmission, improving the convenience of live streaming operations.

[0015] In some preferred embodiments, the fourth interface supports PD 3.1 and / or QC 2.0 fast charging protocols; the sound card also includes a fast charging IC, connecting the CC pin and VBUS pin of the first and fourth interfaces, automatically switching between power supply and charging modes and negotiating the fast charging voltage according to the connection status. For monitoring, the monitoring audio signal output from the third interface has the same volume as the audio signal returned to the live streaming terminal from the first interface and cannot be adjusted by the sound card; when the switch is off, the main control MCU ignores the audio input from the second interface; when the switch is on and there is microphone input, the main control MCU mixes the first and second audio signals and uploads them to the live streaming terminal; when the user only needs to upload microphone audio, the application on the live streaming terminal side stops downloading the first audio signal. This provides users with a flexible and stable means of audio monitoring and control.

[0016] In another aspect, the present invention also provides an electronic device including a memory and a processor, wherein the processor, when executing a computer program in the memory, implements the steps of the above-described volume adaptive adjustment method. Furthermore, a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described volume adaptive adjustment method.

[0017] This invention offers the following advantages: The method extracts RMS values ​​in time-domain blocks, sorts peaks in ascending order, constructs a decibel difference series based on corresponding position ratios, and combines outlier removal and fitting to achieve real-time and accurate volume alignment of two asynchronous audio streams within O(N) time complexity. Human voice detection effectively eliminates interference from silence and background noise, while ascending peak sorting avoids the dilution of loudness represented by low-energy segments. The combination of these two methods makes the calculation of volume differences closer to the subjective perception of the human ear. The variable threshold mechanism improves the device's adaptability to different noise environments, and the deviation judgment and re-trigger mechanism ensure long-term stability after changes in the overall loudness of the sound source. Furthermore, by embedding the algorithm into a live streaming sound card with specific audio interfaces and control logic, an integrated low-computing-power audio processing solution is provided for scenarios such as unmanned live streaming, significantly improving the quality of live streaming audio and user ease of operation. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the system architecture of a live streaming sound card provided in an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of the overall flow of the volume adaptive adjustment method provided in an embodiment of the present invention.

[0020] Figure 3 This is a complete process reduction diagram provided for an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0022] This invention provides a volume adaptive adjustment method, a live streaming sound card, a computer-readable storage medium, and an electronic device to solve the technical problem that, in scenarios such as unmanned live streaming and AI-assisted live streaming, the audio transmitted by the live streaming terminal (such as AI robot voice or background music) and the real human voice captured by the microphone come from different sound sources, resulting in inconsistent loudness between the two audio sources and affecting the listener's experience.

[0023] In one optional implementation, the technical solution of this invention can be applied to a live sound card device with a main control MCU. The main control MCU receives audio signals from the live terminal and the microphone through appropriate audio interfaces and completes all volume alignment operations in the time domain with a computational complexity of O(N). No frequency domain transformation is required, so it can run in real time on embedded platforms with limited computing power.

[0024] Before describing specific embodiments, some of the terms used herein will be explained.

[0025] RMS (Root Mean Square) is a measure of the effective amplitude of an audio signal. It is calculated by averaging the squares of the amplitudes of each sample point over a certain period and then taking the square root. The RMS value has a good correlation with the loudness perceived by the human ear, and therefore it is used in this embodiment to characterize the energy of human voice.

[0026] IQR (Interquartile Range): The difference between the third quartile (Q3) and the first quartile (Q1) in a dataset, i.e., IQR = Q3 − Q1. IQR measures the dispersion of data and is robust to non-normally distributed data.

[0027] OTG (On-The-Go): A USB supplemental standard that allows devices to switch between host and peripheral modes. In this embodiment, the Type-C OTG function of the first interface enables the sound card to transmit audio data to the live streaming terminal as a peripheral, while simultaneously receiving audio downloaded from the live streaming terminal.

[0028] Reference Figure 1This is a schematic diagram of the system architecture of a live streaming sound card provided in an embodiment of the present invention. The live streaming sound card 100 includes: a first interface 110, a second interface 120, a third interface 130, a fourth interface 140, a switch 150, a main control MCU 160, and an optional fast charging IC 170.

[0029] The first interface 110 is a Type-C OTG interface, which connects to the live streaming terminal 200 (e.g., a smartphone) via a USB cable. The DP / DN pin of the first interface 110 is used to establish USB Audio Class protocol communication with the live streaming terminal 200 to transmit bidirectional audio data. The VBUS and CC pins of the first interface 110 are also used to transmit charging current; the specific connection relationship will be detailed later.

[0030] The second interface 120 is a three-level 3.5mm audio jack used to connect the microphone 300 or a wireless microphone receiver. The second interface 120 is a unidirectional input interface, used only to receive the second audio signal collected by the microphone 300. The switch 150 is a two-position toggle switch connected in series in the audio signal path of the second interface 120. In an optional embodiment, the switch 150 is positioned between the second interface 120 and the ADC (analog-to-digital converter) input of the main control MCU 160. When it is in the open state, the audio signal input to the second interface 120 is physically cut off, and the value read by the corresponding sampling channel of the main control MCU 160 is zero or noise floor.

[0031] The third interface 130 is a three-stage 3.5mm audio jack used to connect the monitoring headphones 400. The third interface 130 is a unidirectional output interface. The main control MCU 160 outputs the mixed audio signal to this interface through a DAC (digital-to-analog converter) for the broadcaster to monitor in real time.

[0032] The fourth interface 140 is a Type-C interface used to connect the charging adapter 500. The charging adapter 500 can be an AC adapter or a power bank that supports fast charging protocols. The VBUS and CC pins of the fourth interface 140 are connected to the fast charging IC 170, which is also connected to the CC and VBUS pins of the first interface 110. The fast charging IC 170 is used to detect the connection status at both ends, automatically switch between power supply and charging modes, and negotiate a suitable fast charging voltage according to the protocols supported by the connected charging adapter 500 (such as PD3.1, QC 2.0). In one possible implementation, when the fourth interface 140 is connected to the charging adapter 500, the fast charging IC 170 converts the voltage provided by the charging adapter 500, supplying one path to the various modules of the sound card 100 itself, and the other path to provide charging current to the live streaming terminal 200 through the VBUS pin of the first interface 110.

[0033] The main control MCU 160 is the core processing unit of the sound card 100. In one optional implementation, it can be a microcontroller based on the ARM Cortex-M series or similar architecture, with built-in ADC, DAC, USB controller, and sufficient SRAM for audio buffering and algorithm calculation. The main control MCU 160's memory (which can be on-chip flash memory or external memory) stores a computer program that implements the volume adaptive adjustment method described in this application. By executing this computer program, the main control MCU 160 adaptively adjusts the volume of the first audio signal and the second audio signal obtained from the first interface 110 and the second interface 120, mixes the processed audio data, and returns it to the live broadcast terminal 200 through the DP / DN pin of the first interface 110, while simultaneously transmitting it synchronously to the third interface 130 for monitoring.

[0034] With the above system architecture as a framework, the specific implementation process of the volume adaptive adjustment method will be detailed below. It should be understood that the main control MCU 160 executes each step by reading computer program instructions and operating its internal processing units and peripheral interfaces.

[0035] Reference Figure 2 A volume adaptive adjustment method is provided, applied to the aforementioned live streaming sound card 100, and executed by the main control MCU 160, including the following steps:

[0036] Acquire a first audio signal and a second audio signal, wherein the first audio signal originates from the audio transmitted from the live streaming terminal 200 connected through the first interface 110, and the second audio signal originates from the audio collected by the microphone 300 connected through the second interface 120 (for ease of citation, this is referred to as step S110 below; note that the step numbers in the specification are for illustrative purposes only and do not constitute a limitation on the scope of protection).

[0037] Human voice detection is performed on the first audio signal and the second audio signal respectively, and the effective audio data during the period when someone is speaking is extracted from each audio stream to obtain the first effective audio data and the second effective audio data (step S120).

[0038] Collect the first and second valid audio data within a preset duration T, and divide the collected valid audio data into multiple time blocks according to a preset time block length Δt. Calculate the root mean square (RMS) value of the first valid audio data in each time block to obtain the first RMS sequence, and the root mean square (RMS) value of the second valid audio data to obtain the second RMS sequence. Each of the first and second RMS sequences contains N RMS values, where N = T / Δt (step S130).

[0039] Sort the first RMS sequence and the second RMS sequence in ascending order to obtain the first ascending sequence and the second ascending sequence; extract the top M largest RMS values ​​from the first ascending sequence as the first peak sequence; extract the top M largest RMS values ​​from the second ascending sequence as the second peak sequence; where M is a positive integer less than N (step S140).

[0040] Based on the RMS values ​​corresponding to the sorting positions in the first peak sequence and the second peak sequence, the ratio of each pair of RMS values ​​is calculated sequentially, and each ratio is converted into a corresponding decibel value to obtain a data sequence containing M decibel differences, which is called the decibel difference sequence (step S150).

[0041] Outlier removal is performed on the decibel difference sequence, and the identified outlier data points are removed from the sequence to obtain the effective decibel difference sequence (step S160).

[0042] The effective decibel difference series is linearly fitted using the least squares method to obtain a scalar value, which is used as the final decibel difference value (step S170).

[0043] Adjust the overall gain of the second audio signal based on the final decibel difference so that the volume of the second audio signal after gain adjustment is close to the volume of the first audio signal (step S180).

[0044] The above steps constitute a complete time-domain processing chain from acquiring two signals to volume alignment. All calculations—including RMS calculation, sorting, ratio calculation, outlier detection, and least squares fitting—are completed in the time domain without any frequency domain transformation. The overall time complexity is determined by the most time-consuming sorting step, which is only O(N log M) or better when using an appropriate sorting algorithm. Considering that M is much smaller than N, the actual overhead is close to O(N), which can be fully handled in real time by the main control MCU160. From a technical perspective, the solution employs a two-tiered screening mechanism to ensure the accuracy of volume comparison: First, step S120 uses voice detection to eliminate silent segments and periods containing only background noise, allowing subsequent processing to focus on authentic human voices. Then, in steps S130-S140, further filtering is achieved through time-blocking, RMS value analysis, and peak sorting to remove brief pauses between voice sentences and low-energy segments within voice sentences, retaining only the strongest energy portions of each time block for volume comparison. This statistically captures the "loud" portion of sound perceived by the human ear, rather than the arithmetic mean of the entire audio segment. Finally, outlier removal and least-squares fitting converge multiple sets of difference data into a stable and reliable difference, overcoming the random fluctuations that may arise from single-point comparisons. This "funnel-shaped" information condensation process, from massive sampling points to a single gain adjustment, balances computational lightness and robustness of results.

[0045] The specific implementation methods of each of the above steps will be explained in detail below.

[0046] In step S110, the main control MCU 160 establishes a USB Audio Class (UAC) bidirectional audio stream with the live streaming terminal 200 via its built-in USB controller and the DP / DN differential signal line of the first interface 110. Under the UAC protocol framework, the live streaming terminal 200 acts as a USB host, and the sound card 100 acts as a USB audio device. The main control MCU 160 refers to the audio data transmitted from the live streaming terminal 200 as the first audio signal, which is usually a decoded PCM (Pulse Code Modulation) data stream with a sampling rate of, for example, 48kHz and a bit depth of, for example, 16bit (this embodiment does not limit this). At the same time, the main control MCU 160 receives the second audio signal from the microphone 300 in analog mode via the second interface 120 through its built-in ADC or dedicated audio codec chip, and converts it into digital PCM data. The two PCM data streams are maintained in the audio buffer inside the main control MCU 160 in the form of double buffering or circular buffering, respectively, for use in subsequent steps. It should be noted that the first and second audio signals can reside in independent buffer areas within the main control MCU 160, as subsequent processing only requires their respective raw PCM data. In an optional implementation, to avoid time alignment issues caused by the difference in the two sampling rates, the main control MCU 160 can resample both audio signals to the same target sampling rate, or, when selecting the time block length, use a uniform absolute time (milliseconds) as a reference to calculate the number of sampling points for each signal separately.

[0047] In step S120, human voice detection is performed on the first and second audio signals respectively. The purpose is to extract segments with spoken voices from the continuous audio stream, excluding pure silence segments and pure background noise segments. Human voice detection can be performed frame by frame, with each frame having a length of, for example, 20 milliseconds, consistent with or an integer multiple of the subsequent block length Δt. The energy of a frame is represented by its RMS value: for PCM samples x1, x2, ..., x_k within a frame, RMS = √[(x1² + x2² + ... + x_k²) / k]. When the RMS value of a frame exceeds a certain human voice threshold, the frame is determined to contain human voices. The specific method for determining the human voice threshold will be detailed in a more preferred embodiment later. In the basic scheme, a threshold based on background noise statistics can be used, or a preset empirical value can be used. Once several consecutive frames meet the human voice condition, it is considered that a "speaking period" has begun, and the main control MCU160 begins to mark or copy the corresponding PCM data as valid audio data until several subsequent consecutive frames are below the threshold, at which point the speaking is considered to have ended. The two valid audio data streams are accumulated into their respective buffer queues for use in step S130.

[0048] For step S130, when the cumulative duration of valid audio data reaches a preset duration T, the main control MCU 160 initiates a volume difference calculation. The first valid audio data within duration T is divided into consecutive time blocks in chronological order, with each time block having a length of Δt, and a total number of blocks N = T / Δt. For each time block, the RMS value of all sampling points is calculated. Thus, N RMS values ​​are obtained for the first valid audio data, denoted in chronological order as [R...]. 11 , R 12 , ..., R 1n This is the first RMS sequence; for the second valid audio data, N RMS values ​​are obtained through the same process, denoted as [R]. 21 , R 22 , ..., R 2n This is the second RMS sequence. The RMS value, as a reliable measure of the energy of a sound signal, is squared and proportional to the sound power; therefore, the logarithmic difference of the RMS values ​​directly corresponds to the decibel difference. The preset duration T determines the statistical basis of the volume assessment—the larger T is, the more periods of speech rise and fall are included, and the more stable the statistics; the smaller T is, the more responsive the response, but may produce random errors due to insufficient samples. In a specific example, T is 10 seconds, Δt is 20 milliseconds, then N = 10,000 / 20 = 500, meaning 500 RMS values ​​are obtained for each channel. The main control MCU 160 can internally allocate two arrays of length N to store these two RMS sequences.

[0049] In step S140, after obtaining the first and second RMS sequences, the main control MCU 160 does not directly calculate the average or point-by-point difference between the two sequences. Instead, it employs a "peak-based comparison" strategy. The principle is that during periods of speech, the RMS value fluctuates significantly over time (syllable stress, sentence spacing, etc.). Segments with higher RMS values ​​directly determine the perceived loudness, while segments with lower RMS values ​​during speech intervals, although affecting the arithmetic mean, do not represent the "loudness" of the voice. Therefore, step S140 first sorts the two RMS sequences in ascending order, resulting in two ordered sequences. Since the maximum value is at the end after ascending, the last M values ​​are selected as the M largest RMS values. The choice of M needs to ensure that the selected peak values ​​sufficiently represent the main body of loudness while avoiding the inclusion of too many low-energy intermediate segments. For example, with N=500, if M is set to 100, retaining 20% ​​of the peak RMS values ​​can effectively cover the strong syllables and main vocal segments of the voice.

[0050] For step S150, both the first peak sequence and the second peak sequence output in step S140 are ascending sequences, each containing M RMS values. In one implementation, the main control MCU 160 pairs RMS values ​​at the same rank in the two sequences: that is, the i-th maximum value (the N-M+ith value after sorting) of the first peak sequence is paired with the i-th maximum value of the second peak sequence, i=1,2,...,M. For each pair of RMS values, the ratio r_i = RMS1_i / RMS2_i is calculated, and then converted to decibels: dB_i = 20 · log 10 (r_i). This results in a decibel difference sequence of length M [dB1, dB2, ..., dB_M]. The reason for using paired ratios instead of the global average is that audio signals are non-stationary, with varying volume levels at different times. Pairing ratios more precisely reflect the differences between the two audio signals at multiple energy levels, providing a rich sample for subsequent statistical analysis. In practice, logarithmic calculations can be implemented using a lookup table to reduce CPU overhead and ensure the real-time performance of the embedded MCU.

[0051] In steps S160 and S170, the decibel difference series [dB1, dB2, ..., dB_M] inevitably contains some unreasonable values ​​due to sudden noise, transient interference, or missed human voice detection (e.g., a certain dB value deviates significantly from other values). Therefore, outlier removal is performed before fitting. Step S160 can employ various statistical methods, such as the 3σ principle based on standard deviation or the box plot method (IQR method). After removing outliers, the remaining set of effective decibel differences constitutes the effective decibel difference series, denoted as length K (K ≤ M). Then, step S170 performs least-squares linear fitting on the effective decibel difference series, which is essentially equivalent to calculating the mean of this set of data (when fitting a constant model), or taking the value of a certain confidence point after fitting a straight line (when considering time trends). In the basic scheme, since there is no pre-defined time order dependency among the points in the effective decibel difference sequence, the least squares fitting degenerates into calculating the arithmetic mean. In this case, the final decibel difference dB_final = (Σ dB_valid_j) / K. The advantage of using the least squares method is that even if a time-weighted fitting is subsequently introduced, the method framework does not need to be modified.

[0052] In step S180, the main control MCU 160 adjusts the gain of the second audio signal according to the value of dB_final. If dB_final is positive, it means that the volume of the first audio signal (downloaded from the live streaming terminal) is greater than that of the second audio signal (microphone), and the gain of the second audio signal needs to be increased; conversely, if it is negative, the gain is decreased. The gain adjustment amount ΔG = −dB_final, in dB. The gain is applied in the digital domain, that is, each PCM sampling point of the second audio signal is multiplied by a corresponding linear gain coefficient g = 10^(ΔG / 20). The adjusted second audio signal is mixed with the first audio signal in the digital domain, and then returned to the live streaming terminal 200 through the first interface 110, and output for monitoring through the third interface 130.

[0053] In practice, the strategy of "taking the peak ratio and fitting" has a significant advantage over directly calculating the ratio of the average RMS values ​​of the two audio signals. Taking a typical test scenario as an example, suppose the first audio signal (AI robot voice) is read steadily with a smooth RMS value over time; while the second audio signal (human voice) has distinct intonation and fluctuates significantly, with large RMS values ​​and noticeable low-energy pauses between sentences. If the traditional mean method is used, the average value of the second RMS sequence will be significantly lowered by the low-energy pauses, resulting in an overestimation of the calculated decibel difference. This leads to overcompensation, making the adjusted microphone volume too high and perceived as harsh by the listener. This solution, however, only takes the M highest RMS values ​​for pairing, effectively filtering out interference from pauses. The calculated decibel difference better matches the subjective comparison of the "loudness" of the two audio signals, resulting in a balanced, natural, and comfortable volume after adjustment.

[0054] To further improve the accuracy, robustness, and environmental adaptability of volume alignment, the basic solution described above can be optimized in several dimensions, provided that the computing power and storage resources of the main control MCU 160 are sufficient. These optimizations will be elaborated below.

[0055] To achieve a better balance between sampling sufficiency and response speed adjustment, in one optional implementation, the preset duration T, time block length Δt, and peak count M are not fixed values, but are adaptively selected within a certain range based on the current audio state, or specific values ​​obtained through experimental optimization are used. Specifically, the value of T can range from 5 seconds to 15 seconds. When T is 5 seconds, a complete volume alignment can be completed within 5 seconds, with a very fast response, suitable for scenarios with frequent audio source switching; when T is 15 seconds, more effective audio samples can be accumulated, giving the fitting results higher confidence, suitable for long-term stable live streaming scenarios. The value of Δt can range from 10 milliseconds to 30 milliseconds. A smaller Δt (e.g., 10 milliseconds) can capture the transient characteristics of human voices more precisely, but it will also increase the sequence length N (N is larger under the same T), thus increasing the computational burden of sorting; a larger Δt (e.g., 30 milliseconds) can reduce the computational burden, but may blur individual brief high-energy pulses. The proportion of M to N can range from 15% to 25%. When M is 15% of N, only a small number of RMS values ​​at the true peak of energy are retained, almost completely eliminating interference from the low-energy parts of the human voice. When M is 25% of N, the peak coverage is wider, ensuring sufficient paired samples even if the human voice characteristics are relatively flat. In an exemplary preferred configuration, T=10 seconds, Δt=20 milliseconds (where N=500), and M=100 (i.e., 20%), this combination has been experimentally verified to simultaneously meet the accuracy and real-time requirements in live streaming scenarios.

[0056] In practical implementation, by reasonably limiting the parameter range, this method can flexibly adapt to different types of live streaming content. For unmanned live streams dominated by AI voice with a uniform speaking speed, a slightly longer T (e.g., 12 seconds) and a slightly smaller M ratio (e.g., 18%) can provide a stable and highly filtered alignment effect; for interactive live streams with frequent interruptions from real people and large changes in speaking speed, a shorter T (e.g., 7 seconds) and a slightly larger M ratio (e.g., 22%) can track volume changes faster and ensure responsiveness. This "configurable" rather than "fixed" parameter system gives the sound card an adaptability advantage in different application scenarios.

[0057] To enhance the adaptability of voice detection to different acoustic environments, in one optional implementation, voice detection does not employ a fixed threshold but rather a variable single threshold, whose value is automatically initialized upon the first input of each audio signal. Specifically, when the first audio signal (or the second audio signal) first enters the audio buffer of the main control MCU 160, a "noise floor evaluation" stage is entered before the normal volume alignment loop for effective audio data acquisition. In this stage, the main control MCU 160 acquires audio data of a preset initial duration, for example, 10 seconds. The audio within this 10-second period is divided into blocks of the same time block length Δt, and the RMS value of each block is calculated to obtain an initial RMS set. The RMS values ​​in this set are sorted in ascending order, and the lowest 25% of the RMS values ​​are taken. The arithmetic mean of these low-value RMS values ​​is calculated as the voice threshold value for that audio signal. The technical principle behind this approach is as follows: When the device is first turned on and no one is speaking, the audio signal mainly consists of background noise, and its overall RMS value is low. Even if there is brief ambient noise, its duration is much less than 10 seconds. Therefore, the average of the lower 25% RMS values ​​after ascending order can stably represent the background noise level without being dragged down too much by individual extremely low sampling points (such as silent sampling). After obtaining the threshold value, subsequent human voice detection uses this threshold as the judgment benchmark: within a preset time window (e.g., 200 milliseconds), if the number of times the RMS value exceeds the threshold value reaches a preset threshold (e.g., 60% of the total number of frames in the window), it is determined that someone is speaking. Since the threshold value is calculated in real time based on the actual access device, different microphones (dynamic / condenser) and different background noise transmitted from different live streaming terminals (different mobile phone models have different audio output quality) can be automatically adapted, avoiding the problems of "too high sensitivity (misjudging background noise as human voice)" or "too low sensitivity (missing soft human voice)" that may be caused by a universal fixed threshold.

[0058] To address sudden changes in audio loudness during live streaming—for example, if the broadcaster changes the AI ​​voice pack mid-stream, and the average level of the new voice pack is significantly higher than the old one; or if the microphone gain is manually misadjusted, or the relative position of the microphone and the sound source changes—in one optional implementation, the method further includes a deviation judgment step, which automatically re-triggers the volume alignment process after a deviation is detected. The deviation judgment is performed after each successful human voice detection. Its initialization occurs after the first volume alignment is completed and the RMS sequence is acquired. Specifically, the main control MCU 160 uses the data from the first and second acquired RMS sequences, each determined to be the high-value portion of the human voice (e.g., RMS values ​​above the 95th percentile in ascending order), to calculate its numerical range and set an upper limit UB and a lower limit LB for a variable interval. For example, UB = the third quartile of the high-value RMS + 1.5 * (the third quartile of the high-value RMS - the first quartile), LB = the first quartile of the high-value RMS - 1.5 * (the third quartile of the high-value RMS - the first quartile). This variable interval represents the normal range of human voice loudness distribution. After each round of voice detection, the energy value of the audio stream is immediately analyzed: if, within a preset time window (e.g., 500 milliseconds), the number of times the energy value is consistently higher than UB reaches a preset threshold (i.e., multiple consecutive time blocks with abnormally high energy), or the number of times it is consistently lower than LB reaches a preset threshold (multiple consecutive time blocks with abnormally low energy), then it is determined that the overall loudness has shifted, and the deviation judgment passes. It should be noted that if, within the same time window, some time blocks have energy higher than UB and some time blocks have energy lower than LB, this is usually normal speech fluctuation rather than an overall shift, and therefore is not judged as a deviation. When the deviation judgment passes, the main control MCU 160 clears all currently acquired but unprocessed RMS arrays (including those still being collected and not yet full T), resets the state machine to the acquisition step, and restarts the complete process of steps S130 to S180. This mechanism enables the algorithm to continuously track long-term changes in the audio environment and automatically restore volume alignment without manual intervention.

[0059] To improve the data purity of the decibel difference series before fitting, in one optional implementation, outlier removal in step S160 specifically employs the interquartile range (IQR) box-line method. The IQR method does not rely on the assumption of a normal data distribution and is more robust to the small number of outliers that may appear in actual audio data. The operation process is as follows: First, sort the decibel difference series [dB1, dB2, ..., dB_M] in ascending order, calculate its first quartile Q1 (i.e., the 25th percentile) and third quartile Q3 (i.e., the 75th percentile), and the interquartile range IQR = Q3 − Q1. Then, set the lower bound to Q1 − 1.5 × IQR and the upper bound to Q3 + 1.5 × IQR. Any data points falling outside this interval are considered outliers and removed from the series. For example, if most values ​​in a certain decibel difference series are concentrated between -3 dB and +1 dB, but there are two points at +12 dB and -9 dB, the IQR box plot method can effectively identify and remove these two points, preventing them from biasing the subsequent least squares fitting results. The effective decibel difference series after removing outliers is then used for the least squares fitting in step S170, resulting in a more reliable final decibel difference value dB_final.

[0060] To ensure data consistency during RMS sequence acquisition is not interfered with by other tasks, in one optional implementation, all the aforementioned voice detection and deviation judgment operations are not allowed to be executed at all times, but are limited to periods when the system is in an "idle state." An idle state is defined as: there is currently no ongoing acquisition process for the first or second RMS sequence. That is, when the main control MCU 160 has started the timing accumulation of valid audio data, but the accumulated duration has not yet reached the preset duration T, the system is in a "busy acquisition" state. During this period, the routine execution of voice detection and deviation judgment is temporarily suspended, and the computing power of the main control MCU 160 is concentrated on the continuous caching of audio data and the pre-calculation of RMS by time block. Voice detection and deviation judgment are only resumed when a single acquisition is completed (full T), the system enters the subsequent sorting, peak taking, and fitting process, or when an alignment adjustment has just been completed and a new cycle has not yet started. This mechanism avoids time base deviations caused by the loss or delay of certain time blocks of data due to the insertion of additional calculations for human voice detection or deviation judgment during data acquisition. It ensures that the calculation of each value in the RMS sequence is based on a strictly continuous and equidistant time window, thereby guaranteeing the accuracy of the time correspondence of each decibel difference value in the decibel difference series.

[0061] Within the framework of the basic and optimized solutions, this method can cover a variety of typical scenarios and conditional branches during live streaming.

[0062] When the live streaming sound card powers on and establishes its initial connection with the live streaming terminal and microphone, the main control MCU 160 first enters the noise floor assessment and human voice threshold initialization phase. At this time, the first interface 110 and the second interface 120 have audio signal input, but there may be no human voice in the audio stream (the anchor has not yet started speaking, and the live streaming terminal has not yet pushed AI voice). In this scenario, human voice detection continues to run but fails each time, so the system remains idle. Once any audio channel detects human voice for the first time, the threshold value is initialized based on the noise floor data cached within the aforementioned preset initial duration, and the system enters the ready state. Thereafter, each time human voice detection passes, deviation judgment (if it has been initialized) is initiated in the idle state or the monitoring state continues.

[0063] When the live stream is in a typical unmanned state—an AI robot continuously plays audio while a live anchor intermittently speaks—both audio streams contain valid human voices, and each RMS sequence can accumulate sufficient valid data. In this scenario, volume alignment is continuously updated with a preset duration T. If, within a certain T-period, only one live stream terminal has continuous human voice (AI voice), while the microphone only picks up ambient noise (the live anchor is temporarily absent), the amount of second valid audio data may be extremely small, resulting in an overall low RMS value in the second RMS sequence. In this case, after sorting and peaking in step S140, a relatively large peak can still be extracted (which may still be an occasional high point in the background noise). The calculated decibel difference series in this situation may have large deviations and be biased to one side. However, through outlier removal in step S160 and fitting in step S170, excessively deviated points are filtered out, and the final decibel difference value will tend to maintain the result of the last valid alignment or tend towards a conservative value, without adjusting the microphone gain to an extremely high level to introduce huge noise. When the live anchor resumes speaking, the second set of valid audio data is restored, and the volume can be quickly adjusted back to a reasonable level during the next alignment cycle.

[0064] When the broadcaster changes the AI ​​audio file on the live streaming terminal, and the overall level of the new audio differs significantly from the original audio, listeners at the speaker end will immediately perceive a volume mismatch between the two channels under the gain aligned with the old level in the first round. Since the deviation judgment checks whether the energy exceeds the old variable range after each round of voice detection, the high (or low) level of the new audio will quickly trigger the condition of "continuously above the upper limit" or "continuously below the lower limit," causing the deviation judgment to pass. The system clears any incomplete RMS sequence currently being acquired and immediately restarts steps S130 to S180. In the new acquisition cycle, the first RMS sequence will reflect the level of the new audio, and the final decibel difference will be updated accordingly, with the gain adjusted accordingly. The two audio channels are re-aligned within 1 to 2 T cycles. This adaptive re-triggering process requires no broadcaster intervention throughout.

[0065] When the microphone toggle switch (switch 150) on the sound card is turned off, the audio input path of the second interface 120 is physically cut off. The second audio signal acquired by the corresponding ADC channel of the main control MCU 160 becomes extremely low (containing only circuit background noise). In this scenario, the RMS values ​​of the second RMS sequence are all extremely small, and voice detection fails on the second channel, resulting in the inability to accumulate second valid audio data. Therefore, the volume alignment process will not be triggered due to the lack of second valid audio data, and the main control MCU 160 only maintains the pass-through of the first audio signal without adjusting the gain of the second audio signal. The monitored and uploaded audio only contains the first audio signal downloaded from the live broadcast terminal. When the switch is turned back on and the microphone input is restored, once the voice detection passes, the system resumes normal volume alignment.

[0066] When a user wants to upload microphone audio separately through the live streaming terminal application without mixing it with the audio downloaded from the live streaming terminal, the application stops downloading the first audio signal to the sound card. At this time, the first audio signal disappears, and the first RMS sequence contains only zero values ​​or background noise, thus failing to trigger the alignment process. The system only uploads the second audio signal (after processing the original gain), achieving a pure microphone mode.

[0067] To ensure the safety of the algorithm operation and the stability of the audio output, this method sets boundary controls and exit conditions at multiple stages.

[0068] In the gain adjustment stage, to avoid clipping distortion or signal submersion caused by excessively large or small gain adjustments to the second audio signal due to abnormal calculations, the main control MCU 160 sets a reasonable range for the gain coefficient g. For example, the lower limit of g is the linear coefficient corresponding to −12 dB, and the upper limit is the linear coefficient corresponding to +12 dB. When the final decibel difference calculated as ΔG exceeds this range, the gain is clamped at the boundary value. In specific implementation, after completing the gain adjustment, the main control MCU 160 performs a peak detection on the output PCM data. If clipping is detected (multiple consecutive sampling points reaching the maximum quantization value), the gain can be further fine-tuned or a soft limiting algorithm can be selected to protect subsequent audio links and the listener's listening experience.

[0069] In the deviation detection phase, to ensure that deviation detection is not frequently and falsely triggered by occasional short-term interference, leading to repeated system restarts for data acquisition, strict constraints are imposed on external triggering conditions: a deviation is only recognized when the deviation "continuously" exceeds the upper / lower limit of the interval within a preset time window (e.g., 500 milliseconds) and the number of triggers reaches a preset threshold (e.g., more than 60% of the total number of time blocks within the window). Single or occasional spikes are not counted. Furthermore, when the RMS sequence is cleared and data acquisition is restarted after a deviation detection is passed, the old variable interval is not immediately discarded but is retained for the new round of alignment initialization. It is updated after the first data acquisition of the new round reaches a full T, ensuring reference continuity during the transition period.

[0070] In the parameter configuration phase, the lower limit of the preset duration T is limited to no less than 3 seconds to avoid calculating meaningless decibel differences due to extremely insufficient samples. Similarly, the lower limit of M is set to at least 5 to ensure that the minimum sample size can support the subsequent outlier removal and fitting steps.

[0071] Next, refer to Figure 3 From the perspective of the system state machine, a complete process reduction description of the above method is provided.

[0072] In step S210, the sound card 100 is powered on and initialized, the main control MCU 160 loads the program, and each interface starts working.

[0073] Step S220: Upon detecting the input of the first audio signal and / or the second audio signal, the system enters the noise floor evaluation stage, collects initial audio data, and initializes the thresholds of each vocal channel using the method of statistically calculating the low-value RMS mean. After completion, the system enters the idle state.

[0074] Step S230: Perform voice detection periodically in the idle state. If a voice is detected, proceed to step S240; otherwise, remain in step S230.

[0075] In step S240, if the human voice detection passes, begin collecting valid audio data and construct the current frame of the first RMS sequence and the second RMS sequence. Simultaneously, if the deviation judgment interval has been established and the execution conditions are met, perform a deviation judgment. If the deviation judgment fails, continue collecting data; if the deviation judgment passes, clear the collected sequence and return to step S230 to start again.

[0076] Step S250: Determine whether the cumulative collection time has reached the preset time T. If not, return to step S240 to continue collecting the next time block; if it has, proceed to step S260.

[0077] Step S260: Sort the two RMS sequences in ascending order and extract the top M largest RMS values ​​as the first peak sequence and the second peak sequence.

[0078] Step S270: Calculate the ratio of the RMS values ​​at the corresponding sorting positions in the two peak sequences, and convert them to obtain the decibel difference sequence.

[0079] Step S280: Perform IQR outlier removal on the decibel difference series to obtain the effective decibel difference series.

[0080] Step S290: Perform least squares fitting on the effective decibel difference sequence to obtain the final decibel difference value dB_final.

[0081] Step S300: Calculate the gain adjustment amount based on dB_final, and adjust the digital gain of the second audio signal under clamp protection.

[0082] In step S310, the adjusted second audio signal is mixed with the first audio signal and uploaded to the live streaming terminal through the first interface, while simultaneously being sent to the third interface for monitoring. One round of alignment is complete.

[0083] In step S320, the system returns to the idle state, clears the RMS sequence buffer, waits for the next round of human voice detection to be triggered, and optionally updates the variable range of deviation judgment.

[0084] It should be noted that the numbering and order of the steps in the above process reduction are only for the purpose of helping to understand the logical architecture of the entire method. In the actual implementation of a computer program, some steps can be performed concurrently (e.g., the buffered reception of audio signals and human voice detection can be a combination of a background interrupt service routine and a foreground processing task), and the way the steps are divided can also be flexibly adjusted according to the specific software architecture design. Any reasonable change in the order of steps, the splitting or merging of steps that can be conceived by anyone skilled in the art after reading the above description, without departing from the core concept of this invention, should fall within the protection scope of this invention.

[0085] Corresponding to the above method, the main control MCU 160 in the live sound card 100 provided in this application embodiment can be divided into multiple logic modules to collaboratively complete the above method. Specifically, the audio acquisition module is used to acquire the first audio signal and the second audio signal by interacting with the drivers of the first interface 110 and the second interface 120, thus implementing the signal acquisition step in claim 1. The voice detection module is used to perform voice detection on the two audio signals respectively, obtaining the first valid audio data and the second valid audio data, thus implementing the voice detection step in claim 1 and the fine-grained logic of variable single-threshold initialization and judgment in claim 3. The RMS sequence calculation module is used to calculate the RMS values ​​of the first valid audio data and the second valid audio data according to time block Δt within the acquisition duration T, forming the first RMS sequence and the second RMS sequence respectively, thus implementing the RMS sequence construction in claim 1 and the parameter range limitation in claim 2. The sorting and peak extraction module is used to sort the two RMS sequences in ascending order and extract the first M maximum values ​​as the first peak sequence and the second peak sequence, thus implementing the sorting and peak extraction step in claim 1. The decibel difference calculation module calculates the decibel difference between corresponding RMS ratios in two peak sequences and forms a decibel difference sequence, thus implementing the decibel difference sequence generation step of claim 1. The outlier handling module removes outliers from the decibel difference sequence using IQR box-line method or equivalent methods to obtain an effective decibel difference sequence, thus implementing the specific removal method of claim 5. The fitting module performs least-squares fitting on the effective decibel difference sequence to obtain the final decibel difference value, thus implementing the fitting step of claim 1. The gain adjustment module adjusts the gain of the second audio signal according to the final decibel difference value, thus implementing the gain adjustment step of claim 1. The deviation detection and re-triggering module clears the RMS sequence and re-triggers the acquisition and adjustment process when the deviation judgment passes, thus implementing the additional logic of claims 4 and 6. The system control module controls the input on / off of the second audio signal according to the state of switch 150 and controls the mixing logic according to the signal states of the first interface 110 and the second interface 120, thus implementing the audio transmission control logic described in claim 8.

[0086] Based on the above-mentioned basic solutions and various optimizations, the volume adaptive adjustment method and live streaming sound card provided in this application have achieved multifaceted technical improvements at the overall system level.

[0087] From an algorithm performance perspective, all operations are performed in the time domain, with the core algorithm chain consisting of RMS sequence sorting, peak extraction, ratio calculation, and fitting. Under typical parameters N=500 and M=100, the computational cost of the sorting step is approximately 500·log2100, representing only a tiny fraction of the processing power of the main control MCU 160, far lower than the computational resources required for the thousands of points of the frequency domain FFT. For a 512-point FFT, the number of complex multiplications and additions is in the thousands, requiring additional bit-inversion addressing and window function processing. This solution, however, only requires real-number multiplications and additions and comparison operations. On a comparable MCU platform, the measured runtime can be reduced to less than one-tenth of that of the FFT method. This order-of-magnitude efficiency advantage allows this method to be deployed on cost-sensitive consumer-grade sound card chips, filling the gap in existing technologies where embedded devices cannot run complex audio analysis algorithms.

[0088] From the perspective of perceived accuracy of volume alignment, multiple sets of experiments have verified that, compared with the direct averaging method, the five-fold data refinement chain of this scheme—"voice detection + ascending peak selection + M-peak pairing + outlier removal + fitting"—can reduce the alignment error rate of subjective human ear evaluation (i.e., the proportion of time periods when listeners perceive obvious volume differences) from approximately 12%-18% in the averaging method to 2%-4%. The root of this improvement lies in the fact that the method grasps the core of human voice loudness perception—peak energy represents loudness—and suppresses random fluctuations in individual time periods at the statistical level of multiple pairings, so that the final gain adjustment amount fluctuates within a small range close to the ideal value, rather than fluctuating drastically with short-term content.

[0089] From a system robustness perspective, the deviation judgment and re-trigger mechanism enable the method to have closed-loop adaptive capabilities to long-term changes in the live streaming environment. Even if it runs continuously for several hours, during which events such as AI audio replacement, microphone re-plugging, or significant increases or decreases in ambient noise occur, the system can autonomously complete the complete closed loop of detection-restart-realignment without affecting normal live streaming, without any operation from the broadcaster. The task separation strategy in idle states further ensures the temporal consistency of data collection and eliminates potential race conditions.

[0090] From the perspective of the final product experience, the aforementioned algorithms are integrated into a small sound card hardware with Type-C OTG, multiple 3.5mm audio ports, and intelligent charging management. This allows broadcasters to simply connect the sound card to their phone and microphone, turn on the power, and obtain a fully automated volume balancing experience. This "configuration-free, real-time adaptive" ready-to-use feature addresses the real pain points of many non-professional users in the unmanned live streaming and AI-assisted live streaming sectors, significantly reducing the technical threshold for live audio tuning.

[0091] Corresponding to the above method embodiments, this application also provides a specific implementation structure for a live streaming sound card. See again... Figure 1The live streaming sound card 100 includes all the aforementioned hardware components. In a practical product implementation, the main control MCU 160 can be a microcontroller supporting USB OTG and I²S audio interfaces, such as, but not limited to, Espressif ESP32 series, Rockchip RK series, or dedicated audio DSP chips. The physical layer of the first interface 110 uses a Type-C female connector. DP / DN is connected to the USB PHY of the main control MCU 160 according to standard pin definitions, CC1 / CC2 is connected to the fast charging IC 170, and VBUS and GND are connected to the power network respectively. The second interface 120 and the third interface 130 use standard 3.5mm TRS three-pole female connectors, which are soldered onto the PCB board respectively. Necessary coupling capacitors and ESD protection devices are added between the traces and the ADC and DAC channels of the main control MCU 160. The switch 150 uses a double-pole double-throw or single-pole double-throw miniature toggle switch, soldered between the signal line of the second interface 120 and the ADC input, or the main control MCU 160 can shield the second audio signal in the firmware after detecting the switch status via GPIO. The fast charging IC 170 can be a charging management chip supporting the PD / QC protocol, such as the Injoinic IP series or the Southchip SC series. Its typical circuit design is based on the chip manufacturer's datasheet and will not be described in detail in this application. The entire board adopts a dual-path design of USB bus power supply and battery power supply (battery not shown in the figure) to adapt to mobile live streaming scenarios.

[0092] The mapping relationship between the above hardware structure and the aforementioned method steps has been explained in the previous section and will not be repeated here.

[0093] Although the foregoing embodiments mainly focused on the volume alignment of two audio streams, the core concept of this invention—calculating the decibel difference by pairing ratios after sorting and peaking the time-domain RMS sequence—can also be extended to loudness normalization scenarios for multiple (three or more) audio streams.

[0094] In one variant embodiment, the live streaming sound card 100 may further include an additional audio input channel (e.g., a Bluetooth audio module or an additional line input interface) for receiving a third audio signal (e.g., background music). In this variant, the main control MCU 160 can first use the first audio signal (downloaded from the live streaming terminal) as a reference, and align the second audio signal (microphone) with the first audio signal according to the method of this application; then, using the first audio signal as a reference again, align the third audio signal with the first audio signal in the same process. This aligns all input audio to the same reference level before mixing. The process involves multiple serial or parallel executions of the original method, and the technical essence remains unchanged from the volume difference calculation framework based on peak sequence ratio disclosed in this application.

[0095] In another variant embodiment, the method is not limited to human voice signals. For application scenarios where non-human voice audio (such as instrumental music or ambient sound) is the primary audio source, the human voice detection step can be eliminated, or a signal activity detection (VAD) based on multi-band energy detection can be used instead. The RMS values ​​of the entire frequency band time block can be directly included in the sequence for peak comparison, thus making it suitable for loudness alignment between background music and human voice narration in unmanned live broadcasts. In this case, the core steps S130-S180 remain unchanged, with only the input conditions and judgment logic of step S120 being adjusted.

[0096] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A volume adaptive adjustment method, characterized in that, include: Acquire the first audio signal from the live streaming terminal and the second audio signal from the microphone; Human voice detection is performed on the first audio signal and the second audio signal respectively to obtain first valid audio data and second valid audio data; Collect the first and second valid audio data within a preset duration T, and divide the valid audio data into multiple time blocks according to a preset time block length Δt. Calculate the root mean square (RMS) value of the first valid audio data in each time block to obtain a first RMS sequence, and the root mean square (RMS) value of the second valid audio data to obtain a second RMS sequence. Each of the first and second RMS sequences contains N RMS values, where N = T / Δt. Sort the first RMS sequence and the second RMS sequence in ascending order respectively, and extract the top M RMS values ​​with the largest values ​​as the first peak sequence and the second peak sequence; where M < N. The decibel difference sequence is calculated based on the ratio of the RMS values ​​at corresponding sorting positions in the first peak sequence and the second peak sequence. Outlier removal is performed on the aforementioned decibel difference sequence to obtain the effective decibel difference sequence; The effective decibel difference sequence is fitted to obtain the final decibel difference value; The gain of the second audio signal is adjusted according to the final decibel difference so that the volume of the adjusted second audio signal is aligned with the volume of the first audio signal.

2. The volume adaptive adjustment method according to claim 1, characterized in that, The preset duration T ranges from 5 to 15 seconds; the preset time block length Δt ranges from 10 milliseconds to 30 milliseconds; and the number M of the first M RMS values ​​satisfies the following: the percentage of M to N ranges from 15% to 25%.

3. The volume adaptive adjustment method according to claim 1, characterized in that, The human voice detection includes: Using RMS value as the energy characteristic, a variable single threshold is used for judgment; The initialization method of the variable single threshold is as follows: when each audio input is first received, audio data of a preset initial duration is collected, the RMS value of all time blocks within the preset initial duration is calculated, and the average value of the lowest 25% of the RMS values ​​is taken as the threshold value. The condition for passing the human voice judgment is: within a preset time window, the number of times the energy value exceeds the threshold value reaches a preset threshold.

4. The volume adaptive adjustment method according to claim 1, characterized in that, It also includes deviation judgment steps: After each voice detection passes, the upper and lower limits of the variable range are determined based on the high value distribution of the voice audio in the initially acquired RMS sequence. When the number of times the energy value continuously exceeds the upper limit of the variable range or continuously falls below the lower limit of the variable range within a preset time window reaches a preset threshold, it is determined that the overall loudness has changed; if some energy values ​​exceed the upper limit and some energy values ​​fall below the lower limit within the same time window, it is not determined that the overall loudness has changed. When it is determined that the overall audio loudness has changed, the acquired RMS sequence is cleared, and the steps of acquisition, calculation, sorting, extraction, ratio calculation, outlier removal, fitting, and adjustment are re-executed.

5. The volume adaptive adjustment method according to claim 1, characterized in that, The outlier removal process for the decibel difference series includes: Calculate the first quartile Q1 and the third quartile Q3 of the decibel difference sequence, and calculate the interquartile range IQR = Q3 - Q1; Data points that are below Q1 - 1.5×IQR or above Q3 + 1.5×IQR are identified as outliers and removed.

6. The volume adaptive adjustment method according to claim 4, characterized in that, The voice detection and deviation judgment are performed only in the idle state when the first RMS sequence and the second RMS sequence are not being acquired; during the acquisition of the RMS sequence, the voice detection and deviation judgment are paused.

7. A live streaming sound card, characterized in that, To implement the method according to any one of claims 1 to 6, comprising: The first interface is used to connect to the live streaming terminal and receive the first audio signal; The second interface is used to connect a microphone to receive a second audio signal; The main control MCU is electrically connected to the first interface and the second interface. The main control MCU is configured to execute the method to adaptively adjust the volume of the first audio signal and the second audio signal. The main control MCU is also used to mix the adjusted audio signal and send the mixed audio data to the live broadcast terminal through the first interface.

8. The live streaming sound card according to claim 7, characterized in that, The first interface is a Type-C OTG interface, and the second interface is a three-pole 3.5mm audio interface; the live streaming sound card also includes: The third interface is a three-level 3.5mm audio interface, used to connect monitoring headphones and transmit the mixed audio signal in one direction. The fourth interface is a Type-C interface, used to connect a charging adapter so that the charging adapter supplies power to the sound card through the fourth interface and charges the live streaming terminal through the first interface. The fourth interface supports fast charging protocols. The switch is a two-position toggle switch, connected in series in the audio signal path of the second interface, used to control the on / off state of the microphone audio input; The main control MCU establishes USB Audio Class protocol communication with the live streaming terminal through the DP / DN pin of the first interface to transmit audio data.

9. The live streaming sound card according to claim 8, characterized in that, The fourth interface supports PD 3.1 and / or QC2.0 fast charging protocols; the sound card also includes a fast charging IC, which is connected to the CC pin and VBUS pin of the first interface and the fourth interface, and is used to automatically switch the power supply / charging mode and negotiate the fast charging voltage according to the connection status.

10. The live streaming sound card according to claim 8, characterized in that, The monitoring audio signal output by the third interface has the same volume as the audio signal returned to the live streaming terminal by the first interface, and the volume of the monitoring audio signal cannot be adjusted by the sound card. When the switch is off, the main control MCU ignores the audio input from the second interface; When the switch is turned on and there is audio input at the second interface, the main control MCU mixes the first audio signal and the second audio signal and uploads them to the live streaming terminal.

11. An electronic device comprising a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the method according to any one of claims 1 to 6.

12. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6.

Images