Multi-channel audio acquisition method in multi-application concurrent scenario

Abstract

This invention discloses a multi-channel audio acquisition method for concurrent multi-application scenarios, specifically relating to the field of audio processing technology. It allocates an independent channel, focus queue, and buffer for each audio application; monitors the switching frequency of focus request events for each channel; if the frequency of a channel exceeds a preset threshold, adjusts its focus retry interval and backoff parameters; if the adjusted frequency still exceeds the threshold or a circular dependency exists, freezes the channel and inserts forced silence; identifies the source of silence, distinguishes between fake and real silence, and marks it; collects the switching frequency change rate and silence injection rate, and provides feedback to adjust the preset threshold and step size coefficient. This invention reduces data loss by dynamically adjusting the focus retry interval and priority backoff parameters, blocks circular dependencies and distinguishes fake silence by using circuit breaking operations and silence source identification, and adaptively adjusts the threshold and step size coefficient by feeding back the switching frequency change rate and silence injection rate, thereby improving the stability, continuity, and adaptability of concurrent multi-application audio acquisition.

Description

Technical Field

[0001] This invention relates to the field of audio processing technology, and more specifically, to a multi-channel audio acquisition method in multi-application concurrent scenarios. Background Technology

[0002] In environments such as smart terminals, in-vehicle infotainment systems, smart homes, and mobile communication devices, users often run multiple applications that require audio capture simultaneously. For example, in a car cabin, the driver might use a hands-free phone, the front passenger might simultaneously activate a voice assistant to check routes, a navigation app might continuously broadcast traffic information, and rear passengers might use an entertainment app for voice searches. All these applications need to acquire audio data from the microphone in real time and independently perform voice recognition, call transmission, or recording processing.

[0003] To coordinate access to the microphone, a scarce hardware resource, by multiple applications, existing operating systems typically employ audio focus management mechanisms. The basic idea behind this mechanism is to assign a priority to each audio application. When a high-priority application (such as a call) requests focus, the system automatically mutes or pauses lower-priority applications (such as music playback), thus ensuring that the audio stream for critical business operations is not interfered with. This static management method based on priority arbitration can generally meet the needs in scenarios with a small number of applications and simple interactions.

[0004] However, with the increasing frequency of human-computer interaction and the rise in business concurrency, the aforementioned mechanism has revealed significant limitations in adaptability. When multiple applications frequently request and release focus—for example, when a user repeatedly wakes up the voice assistant during a call, or when navigation and multimedia applications alternately compete for playback resources—the system experiences multiple focus switches within a very short period. Each switch is accompanied by audio device reconfiguration, buffer reset, and stream pause and resumption. This high-frequency focus jitter not only leads to audio data loss and sampling rate errors but also causes some applications to receive zero-value silence data injected due to focus contention. Upper-layer applications (such as speech recognition engines) cannot distinguish between these fake silences and genuine ambient silence, resulting in problems such as premature termination of recognition and misjudgment of no voice input, severely impacting user experience and system reliability. Therefore, this invention proposes a multi-channel audio acquisition method for multi-application concurrent scenarios to address the aforementioned problems. Summary of the Invention

[0005] To achieve the above objectives, the present invention provides the following technical solution: A method for multi-channel audio acquisition in multi-application concurrent scenarios includes the following steps: Each audio application is allocated a separate audio channel, and a separate focus request queue and audio data buffer are established for each audio channel; Real-time monitoring of focus request events in each audio channel, and calculation of the switching frequency of focus request events within a preset time window; When the switching frequency of any audio channel exceeds the preset frequency threshold corresponding to that audio channel, the focus retry interval and priority backoff parameters of that audio channel are dynamically adjusted according to the value of the current switching frequency. After dynamically adjusting the focus retry interval and priority backoff parameters, if the switching frequency of the same audio channel still exceeds the preset frequency threshold corresponding to the audio channel, or if a circular dependency pattern is detected between multiple audio channels, a circuit breaker operation is performed. The circuit breaker operation includes: freezing the focus request permission of at least one audio channel and inserting a forced silence gap in the audio data buffer corresponding to the audio channel. The source of silence data in the audio data stream of each audio channel is identified, and the fake silence generated by the forced silence gap injected by the circuit breaker operation is distinguished from the real environment silence. A silence type mark is added to the audio data stream of that audio channel. The system collects real-time operating parameters for each audio channel after adjustment. These parameters include at least the switching frequency change rate and the silence injection rate for each audio channel. The switching frequency change rate is the ratio of the difference in switching frequencies between two adjacent preset time windows to the duration of the preset time window. The system then feeds back these real-time operating parameters to the step of real-time monitoring of focus request events in each audio channel and calculating the switching frequency. This is used to adjust the preset frequency threshold and priority backoff parameters for each audio channel in the following manner: increasing or decreasing the preset frequency threshold based on the switching frequency change rate, and increasing or decreasing the step size coefficient of the priority backoff parameters based on the silence injection rate. The step size coefficient is the amount of change when updating the priority backoff parameters each time.

[0006] In a preferred embodiment, the initial value of the preset frequency threshold is set according to the type of audio application corresponding to each audio channel.

[0007] In a preferred embodiment, calculating the switching frequency of focus request events within a preset time window specifically includes the following steps: Divide the preset time window into M consecutive sub-time intervals, each with an equal duration, where M is an integer greater than or equal to three; Count the number of focus request events that occur within each sub-time interval; Calculate the absolute value of the difference in the number of occurrences between two adjacent sub-time intervals; The fluctuation factor is obtained by summing the absolute values ​​of all differences and dividing by the difference of M minus one, where the difference of M minus one is the number of adjacent differences, used to avoid division by zero. The switching frequency is obtained by multiplying the total number of focus request events within the preset time window by the fluctuation factor and then dividing by the duration of the preset time window.

[0008] In a preferred embodiment, dynamically adjusting the focus retry interval and priority backoff parameters of the audio channel specifically includes the following steps: Calculate the ratio of the current switching frequency to the preset frequency threshold. When the preset frequency threshold is greater than zero, use the ratio as the load factor. When the preset frequency threshold is equal to zero, set the load factor to a predefined positive value. Calculate the natural logarithm of the sum of the load factor and the value of one, and take the integer part as the retreat index; The focus retry interval is updated as follows: the new focus retry interval is equal to the original focus retry interval multiplied by two raised to the power of the backoff exponent; The priority backoff parameter is updated as follows: the new priority backoff parameter is equal to the original priority backoff parameter minus the product of the backoff exponent and the step size coefficient, and the new priority backoff parameter is restricted to be no less than zero. The retry interval for new focus points is limited to a predefined maximum retry interval, and the backoff parameter for new priorities is limited to a predefined maximum backoff parameter.

[0009] In a preferred embodiment, identifying a cyclic dependency pattern among multiple audio channels specifically includes: A focus request directed graph is established for each audio channel. Nodes in the directed graph represent audio channels, and directed edges represent one audio channel initiating focus preemption for another audio channel. A recursive backtracking algorithm is used to traverse the directed graph. When a duplicate node is found in a certain traversal path, it is determined that a circular dependency pattern exists. Record all audio channel identifiers involved in this circular dependency pattern.

[0010] In a preferred embodiment, the duration for which focus request permission is frozen for at least one of the audio channels is determined as follows: If the preset frequency threshold is greater than zero, calculate the difference between the current switching frequency of the frozen audio channel and the preset frequency threshold, and then divide it by the preset frequency threshold to obtain the frequency excess ratio. When the frequency exceeds the ratio but is less than the first preset ratio threshold, the first freeze duration is applied. When the frequency exceeds the ratio by a factor greater than or equal to the first preset ratio threshold and less than or equal to the second preset ratio threshold, a second freeze duration is adopted, wherein the second freeze duration is greater than the first freeze duration. When the frequency exceeds the ratio by more than the second preset ratio threshold, a third freeze duration is adopted, wherein the third freeze duration is greater than the second freeze duration; The first freeze duration, the second freeze duration, and the third freeze duration are all positive numbers, and each is not less than one complete audio data packet transmission cycle.

[0011] In a preferred embodiment, distinguishing between simulated silence generated by the forced silence gap injected during the circuit breaker operation and real-world silence specifically includes: Record the execution timestamp of each circuit breaker operation, as well as the start and end times of the injected forced silence gap; When multiple consecutive zero-value audio frames appear in the audio data stream, the continuous data segment formed by these multiple zero-value audio frames is taken as the silence segment to be determined, and the start time and end time of the silence segment to be determined are extracted. When there is a forced silence gap injected by a circuit breaker operation, such that the absolute value of the difference between the start time of the forced silence gap and the start time of the silence segment to be determined is less than a preset time deviation threshold, and the absolute value of the difference between the duration of the forced silence gap and the duration of the silence segment to be determined is also less than the preset time deviation threshold, the silence segment to be determined is determined to be a fake silence. Otherwise, it will be judged as a real-world silent environment.

[0012] In a preferred embodiment, adjusting the preset frequency threshold and priority backoff parameters corresponding to each audio channel specifically includes the following steps: Collect the switching frequency change rate of each audio channel within two consecutive preset time windows; When the rate of change is greater than the first rate of change threshold, the preset frequency threshold corresponding to the audio channel is increased by the first step length. When the rate of change is less than the second rate of change threshold and the current priority backoff parameter has been adjusted to the preset lower limit, the preset frequency threshold corresponding to the audio channel is lowered by the second step. The mute injection rate of each audio channel within K consecutive preset time windows is calculated. The mute injection rate is the ratio of the number of forged mute data packets to the total number of data packets. When the silent injection rate is greater than the first injection rate threshold, the step size coefficient is multiplied by the first amplitude scaling factor, where the first amplitude scaling factor is less than one. When the silent injection rate is less than the second injection rate threshold, the step size coefficient is multiplied by the second amplitude scaling factor, where the second amplitude scaling factor is greater than one. Where K is an integer greater than or equal to ten.

[0013] The technical effects and advantages of this invention are as follows: This invention allocates an independent audio channel to each audio application and establishes an independent focus request queue and audio data buffer for each audio channel. Simultaneously, it monitors focus request events in each audio channel in real time and calculates the switching frequency of focus request events within a preset time window. When the switching frequency of any audio channel exceeds a preset frequency threshold for that channel, the focus retry interval and priority backoff parameters for that audio channel are dynamically adjusted based on the current switching frequency value. This invention possesses dynamic response capabilities, enabling precise control of the behavior of each channel based on the actual switching frequency value. This reduces unnecessary focus switching and device reconfiguration, thereby significantly reducing the probability of audio data loss and sampling rate errors, and improving the continuity and smoothness of audio acquisition in multi-application concurrent scenarios.

[0014] This invention, after dynamically adjusting the focus retry interval and priority backoff parameters, if the switching frequency of the same audio channel still exceeds the preset frequency threshold corresponding to that audio channel, or if a cyclic dependency pattern is identified between multiple audio channels, then a circuit breaker operation is performed. This freezes the focus request permission of at least one audio channel and inserts a forced silence gap into the audio data buffer corresponding to that audio channel. This is combined with subsequent source identification of silence data in the audio data stream of each audio channel, distinguishing between fake silence generated by the forced silence gap injected by the circuit breaker operation and real environmental silence, and adding a silence type marker to the audio data stream. This invention actively cuts off abnormal loops through circuit breaker operations and avoids upper-layer applications being misled by fake silence through silence source identification, thereby providing a progressive anomaly handling capability from soft adjustment to hard blocking, enhancing the stability of the system under concurrent conditions.

[0015] This invention collects real-time system operating parameters after adjustment for each audio channel. These parameters include at least the switching frequency change rate and mute injection rate for each audio channel. The switching frequency change rate is the ratio of the difference in switching frequencies between two adjacent preset time windows to the duration of the preset time window. These real-time operating parameters are then fed back to the step of real-time monitoring of focus request events in each audio channel and calculating the switching frequency. This closed-loop feedback mechanism allows the system to adaptively optimize its control parameters. For example, when a rapid increase in the switching frequency is detected, the threshold is increased to reduce sensitivity; when the mute injection rate is too high, the step size coefficient is decreased to slow down the parameter change. After multiple feedback iterations, the system can converge to a stable operating point that matches the current concurrent load, avoiding over-adjustment or under-adjustment caused by fixed thresholds or step sizes. This provides the system with self-learning and self-optimization capabilities, maintaining good performance across different hardware platforms and application scenarios, thus improving the versatility and robustness of the technology. Attached Figure Description

[0016] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings; Figure 1 This is a schematic diagram of the multi-channel audio acquisition method for multiple concurrent application scenarios in this invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0018] Reference Figure 1 The following examples were obtained: Example 1: A multi-channel audio acquisition method for concurrent application scenarios, comprising the following steps: Each audio application is allocated a separate audio channel, and a separate focus request queue and audio data buffer are established for each audio channel; Real-time monitoring of focus request events in each audio channel and calculation of the switching frequency of focus request events within a preset time window; a focus request event refers to a message sent by an audio channel to the audio system requesting to obtain focus control. When a focus request event is rejected or successfully preempted by another channel, the audio channel must wait for a period of time before initiating the next focus request event. This waiting time is the focus retry interval. When the switching frequency of any audio channel exceeds the preset frequency threshold corresponding to that audio channel, the focus retry interval and priority backoff parameter of that audio channel are dynamically adjusted based on the current switching frequency value. The priority backoff parameter is a coefficient used to adjust the focus retry interval. The actual effect of dynamically adjusting the priority backoff parameter is to change the specific value of the focus retry interval. The specific correlation is as follows: the new focus retry interval is equal to the original focus retry interval multiplied by twice the backoff exponent raised to the power of 2. The backoff exponent is calculated from the load factor, and the load factor includes the priority backoff parameter as a subtraction term. In other words, changes in the priority backoff parameter will cause changes in the backoff exponent, thereby changing the multiplier factor of the focus retry interval. After dynamically adjusting the focus retry interval and priority backoff parameters, if the switching frequency of the same audio channel still exceeds the preset frequency threshold corresponding to the audio channel, or if a circular dependency pattern is detected between multiple audio channels, a circuit breaker operation is performed. The circuit breaker operation includes: freezing the focus request permission of at least one audio channel and inserting a forced silence gap in the audio data buffer corresponding to the audio channel. The source of silence data in the audio data stream of each audio channel is identified, and the fake silence generated by the forced silence gap injected by the circuit breaker operation is distinguished from the real environment silence. A silence type mark is added to the audio data stream of that audio channel. The system collects real-time operating parameters for each audio channel after adjustment. These parameters include at least the switching frequency change rate and the silence injection rate for each audio channel. The switching frequency change rate is the ratio of the difference in switching frequencies between two adjacent preset time windows to the duration of the preset time window. The system then feeds back these real-time operating parameters to the step of real-time monitoring of focus request events in each audio channel and calculating the switching frequency. This is used to adjust the preset frequency threshold and priority backoff parameters for each audio channel in the following manner: increasing or decreasing the preset frequency threshold based on the switching frequency change rate, and increasing or decreasing the step size coefficient of the priority backoff parameters based on the silence injection rate. The step size coefficient is the amount of change when updating the priority backoff parameters each time.

[0019] The initial value of the preset frequency threshold is set according to the type of audio application corresponding to each audio channel: for example, in an in-vehicle system, the preset frequency threshold corresponding to the audio channel of a voice call application is higher than the preset frequency threshold corresponding to the audio channel of a voice assistant application, the preset frequency threshold corresponding to the audio channel of a voice assistant application is higher than the preset frequency threshold corresponding to the audio channel of a navigation broadcast application, and the preset frequency threshold corresponding to the audio channel of a navigation broadcast application is higher than the preset frequency threshold corresponding to the audio channel of a media playback application.

[0020] The initial value of the preset frequency threshold is set according to the tolerance of each audio application to focus switching. The setting is based on experimental statistical results: in a typical in-vehicle cabin environment, one hundred test users were rated for their focus switching experience. When the focus switching frequency of voice call applications exceeds 3.0 times per second, more than 80% of users will feel obvious call interruptions. Therefore, the initial value of the preset frequency threshold for voice call applications is set to a value between 3.0 times per second and 3.5 times per second, for example, 3.2 times per second.

[0021] For voice assistant applications, experimental data shows that when the focus switching frequency exceeds 2.5 times per second, the voice wake-up recognition success rate drops by more than 30%. Therefore, the initial value of its preset frequency threshold is set to a value between 2.0 times per second and 2.5 times per second, such as 2.2 times per second.

[0022] For navigation and announcement apps, users have a relatively high tolerance for interruptions. Statistics show that attention is only drawn when the focus switching frequency exceeds 1.8 times per second. Therefore, the initial preset frequency threshold is set between 1.5 and 1.8 times per second, for example, 1.6 times per second. For media playback apps, users have a weaker perception of brief interruptions in background music or video audio. Experiments show that a noticeable auditory impact is only achieved when the focus switching frequency exceeds 1.0 times per second. Therefore, the initial preset frequency threshold is set between 0.8 and 1.0 times per second, for example, 0.9 times per second.

[0023] For example, when a system simultaneously runs a voice call application (such as hands-free calling), a voice assistant application (such as a car assistant), a navigation application (such as map navigation), and a media playback application (such as music playback), the system assigns initial preset frequency thresholds to these four audio channels: 3.2 times per second for the call channel, 2.2 times per second for the assistant channel, 1.6 times per second for the navigation channel, and 0.9 times per second for the media channel. The system only triggers subsequent dynamic adjustments when the focus switching frequency of a channel exceeds its corresponding threshold, thus avoiding the problem of the call channel being overly sensitive or the media channel being overly sluggish by applying a uniform threshold to all channels.

[0024] Calculate the switching frequency of focus request events within a preset time window, specifically including the following steps: The preset time window is divided into M consecutive sub-time intervals, each with an equal duration, where M is an integer greater than or equal to three. The value of M is based on the statistical characteristic analysis of focus request events. Empirical data shows that when M is between five and ten, it can effectively reflect frequency fluctuations without excessively amplifying noise. In a typical example, M is eight. In this case, the duration of each sub-time interval is the preset time window duration divided by eight. For example, if the preset time window duration is eight hundred milliseconds, then the duration of each sub-time interval is one hundred milliseconds.

[0025] The number of focus request events is counted within each sub-time interval. For example, in eight consecutive sub-time intervals, the number of focus request events counted are three, five, two, four, six, three, five, and two, respectively.

[0026] Calculate the absolute value of the difference in the number of occurrences between two adjacent sub-time intervals. The absolute values ​​of the adjacent differences are |3-5|=2, |5-2|=3, |2-4|=2, |4-6|=2, |6-3|=3, |3-5|=2, and |5-2|=3, for a total of seven differences.

[0027] The volatility factor is obtained by summing the absolute values ​​of all differences and dividing by the difference of M minus one. The difference of M minus one is the number of adjacent differences, which is used to avoid division by zero. The sum of the above seven differences is 2+3+2+2+3+2+3=17, and the difference of M minus one is 7. Therefore, the volatility factor is 17 divided by 7, which is approximately equal to 2.4286.

[0028] The switching frequency is obtained by multiplying the total number of focus request events within the preset time window by the fluctuation factor and then dividing by the duration of the preset time window. In the example above, the total number of focus request events within the preset time window is 3+5+2+4+6+3+5+2=30. Multiplying this by the fluctuation factor of 2.4286 gives 72.858. Dividing this by the preset time window duration of 800 milliseconds (i.e., 0.8 seconds) gives a final switching frequency of 72.858 divided by 0.8, which equals 91.0725 times per second. This switching frequency takes into account the degree of clustering and fluctuation of events on the time axis and reflects the intensity of actual focus conflicts better than simply dividing the total number of events by the duration.

[0029] Dynamically adjusting the focus retry interval and priority backoff parameters of the audio channel includes the following steps: The ratio of the current switching frequency to a preset frequency threshold is calculated. When the preset frequency threshold is greater than zero, this ratio is used as the load factor. When the preset frequency threshold is equal to zero, the load factor is set to a predefined positive value. This predefined positive value is set according to the minimum detectable load value of the audio driver in the device manual, typically 0.5. For example, if the current switching frequency is 91 times per second and the preset frequency threshold is 30 times per second, then the ratio is 91.0 divided by 30.0, which is approximately 3.033, and the load factor is 3.033. If the preset frequency threshold becomes equal to zero due to an anomaly, the load factor is directly set to 0.5 to avoid division by zero errors.

[0030] Calculate the natural logarithm of the sum of the load factor and the value 1, and take the integer part as the backoff exponent. The natural logarithm is base e, and taking the integer part means rounding down. Continuing the previous example, the load factor is 3.033, adding one gives 4.033, the natural logarithm is approximately 1.395, the integer part is 1, therefore the backoff exponent is 1. If the load factor is 0.5, adding one gives 1.5, the natural logarithm is approximately 0.405, the integer part is zero, and the backoff exponent is zero. This calculation method borrows the binary exponential backoff concept from network congestion control. Rounding down ensures that the adjustment step size is a discrete integer, avoiding excessively fine changes that could cause system oscillations.

[0031] The focus retry interval is updated as follows: the new focus retry interval equals the original focus retry interval multiplied by two raised to the power of the backoff exponent. The original focus retry interval is set by the device manual's recommended value, typically ten milliseconds for most audio codecs. For example, if the backoff exponent is one, then two raised to the power of one equals two, and the new focus retry interval is ten milliseconds multiplied by two equals twenty milliseconds. If the backoff exponent is zero, then two raised to the power of zero equals one, and the new focus retry interval remains unchanged at ten milliseconds. This update causes the retry interval to increase faster the focus switching is more frequent, thereby reducing preemption density.

[0032] Update the priority backoff parameters as follows: The new priority backoff parameter equals the original priority backoff parameter minus the product of the backoff exponent and the step size coefficient. The new priority backoff parameter is restricted to be no less than zero. The initial value of the step size coefficient is preset according to the type of audio application: 0.2 for voice call applications, 0.1 for voice assistant applications, 0.08 for navigation broadcast applications, and 0.05 for media playback applications. This initial value is set based on the parameter combination that achieves the fastest system convergence through traversal testing in a typical in-vehicle cockpit environment. The step size coefficient is used to calculate the change in priority backoff parameters each time they are updated. The initial value of the original priority backoff parameter is set according to the application type; for example, 5.0 for voice call applications and 3.0 for voice assistant applications. If the original priority backoff parameter is 5.0, the backoff exponent is 1, and the step size coefficient is 0.1, then the product is 0.1, and the new priority backoff parameter is 5.0 minus 0.1, which equals 4.9. The new parameter is restricted to be no less than zero to prevent negative values ​​from causing priority logic confusion.

[0033] The new focus retry interval is limited to a predefined maximum retry interval, and the new priority backoff parameter is also limited to a predefined maximum backoff parameter. The maximum retry interval is set based on an empirical value of the audio data packet transmission cycle, typically 100 milliseconds; exceeding this value will cause noticeable audio stuttering. The maximum backoff parameter is set based on the upper limit of the audio focus arbitration protocol, typically 10.0; exceeding this value will make it almost impossible for the application to obtain focus. For example, if the new focus retry interval is 20 milliseconds, which does not exceed 100 milliseconds, it remains unchanged. If a new priority backoff parameter is calculated to be 11.0, it is then limited to 10.0.

[0034] A circular dependency pattern was identified among multiple audio channels, specifically including: A directed graph of focus requests is created for each audio channel. Nodes in the directed graph represent audio channels, and directed edges represent one audio channel attempting to claim focus from another. For example, if channel A requests focus from channel B, channel B requests focus from channel C, and channel C requests focus from channel A, then the directed graph contains three directed edges: A→B, B→C, and C→A. The number of nodes in the directed graph equals the total number of audio channels allocated in the current system, typically ranging from two to eight nodes, as concurrent audio applications in vehicles or mobile devices usually do not exceed eight.

[0035] A recursive backtracking algorithm is used to traverse a directed graph. When a duplicate node is found in a traversal path, a circular dependency pattern is determined. The specific implementation of the recursive backtracking algorithm is as follows: Starting from each node, a depth-first traversal is performed on all its reachable successor nodes, and the set of nodes on the current path is recorded. For example, starting from node A, following edge A→B to node B, the path [A,B] is recorded; then from B, following edge B→C to node C, the path [A,B,C] is recorded; then, attempting to enter node A from C along edge C→A, if node A is already found in the current path, a circular dependency pattern is determined. The time complexity of this algorithm is O(N+E), where N is the number of nodes and E is the number of edges. O is the Big O notation in computer algorithms. O(N+E) indicates that the time complexity of the algorithm is linear, that is, the execution time of the algorithm is proportional to the sum of the number of nodes N and the number of edges E. In actual mobile devices or in-vehicle systems, the number of concurrent audio channels N is usually no more than 8. Even considering the most complex fully connected preemptive relationship, the number of edges E is limited (at most N×(N-1)). Therefore, N+E is a very small constant, so the computational overhead of the algorithm is very small and can be ignored.

[0036] Record all audio channel identifiers involved in the circular dependency pattern. When a circular dependency pattern is detected, extract all node identifiers from the repeating node to the end of the current path to form a closed-loop list. Continuing the example above, if the path is [A,B,C,A] and the repeating node is A, then extract all nodes from the first A to the last A, i.e., nodes B and C, plus the repeating node A itself, to obtain the set of audio channel identifiers involved in the circular dependency {A, B, C}. This set is used in subsequent circuit breaking operations to determine which audio channels' focus request permissions need to be frozen.

[0037] The duration for freezing focus request permission for at least one of the audio channels is determined as follows: If the preset frequency threshold is greater than zero, calculate the difference between the current switching frequency of the frozen audio channel and the preset frequency threshold, then divide by the preset frequency threshold to obtain the frequency excess ratio. The frequency excess ratio quantifies the severity of the current focus conflict; a higher value indicates that the switching frequency of the audio channel deviates further from its tolerance threshold. For example, if the current switching frequency of the frozen audio channel is 90 times per second, and its preset frequency threshold is 30 times per second, the difference is 60 times per second. Dividing this by 30 times per second gives a frequency excess ratio of 2.0, which is 200%.

[0038] When the frequency exceedance ratio is less than a first preset ratio threshold, a first freeze duration is applied. The first preset ratio threshold is set based on the statistical distribution of minor conflicts in the experimental data, typically with a value of 0.5, indicating that a switching frequency exceeding the threshold by less than 50% is considered a minor anomaly. The first freeze duration is set based on a complete audio data packet transmission cycle, typically one data packet cycle, i.e., 2.5 milliseconds, to briefly interrupt the focus preemption loop. For example, if the frequency exceedance ratio is 0.3, which is less than 0.5, then the first freeze duration is 2.5 milliseconds.

[0039] When the frequency exceeds the first preset threshold but is less than or equal to the second preset threshold, a second freeze duration is applied, where the second freeze duration is longer than the first freeze duration. The second preset threshold is set based on the critical point of moderate conflict, typically with a value of 1.5, indicating an exceedance of 50% to 150%. The second freeze duration is set based on three to five audio data packet transmission cycles, typically with a value of ten milliseconds. For example, if the frequency exceeds the threshold by 0.8%, falling between the first preset threshold of 0.5% and the second preset threshold of 1.5%, then the second freeze duration is ten milliseconds.

[0040] When the frequency exceeds a second preset threshold, a third freeze duration is applied, which is longer than the second freeze duration. The third freeze duration is set based on the need for a longer cooling-off time during severe conflicts, typically set to 20 milliseconds, equivalent to eight audio data packet transmission cycles. For example, if the frequency exceeds a threshold of 2.0, which is greater than 1.5, the third freeze duration is 20 milliseconds. All freeze durations are no less than one complete audio data packet transmission cycle, determined by the audio hardware sampling rate, typically 2.5 milliseconds (48 kHz sampling rate, 120 samples per frame). The first, second, and third freeze durations are all positive numbers and no less than this cycle value.

[0041] The first and second preset ratio thresholds are set based on the following: by collecting one hundred sets of focus conflict data and statistically analyzing the relationship between the frequency exceeding the ratio and the system recovery time, it was found that when the exceeding ratio is less than 0.5, a short-term freeze (first freeze duration) can effectively block the conflict; when the exceeding ratio is between 0.5 and 1.5, a medium-term freeze (second freeze duration) is required; and when the exceeding ratio is greater than 1.5, a long-term freeze (third freeze duration) is required. Therefore, the first preset ratio threshold is set to 0.5, and the second preset ratio threshold is set to 1.5.

[0042] Distinguishing between simulated silence generated by forced silence gaps injected during circuit breaking operations and real-world silence specifically includes: Record the execution timestamp of each circuit breaker operation, as well as the start and end times of the injected forced silence gap. The execution timestamp is based on the system clock's millisecond count. For example, if a circuit breaker operation is executed at 1200 milliseconds of system operation, the execution timestamp is 1200, the start time of the forced silence gap is 1200 milliseconds, the end time is 1210 milliseconds, and the duration is 10 milliseconds. All timestamps are stored in a circular buffer, the size of which is set according to the maximum number of concurrent circuit breakers, typically sixteen entries.

[0043] When multiple consecutive zero-value audio frames appear in the audio data stream, the continuous data segment consisting of these multiple zero-value audio frames is considered as a silence segment to be determined, and the start and end times of this silence segment are extracted. A zero-value audio frame refers to a data frame where all audio sample point values ​​are zero. The duration of each frame is determined by the audio sampling rate; for example, at a sampling rate of 48000 Hz, each frame with 120 sample points corresponds to 2.5 milliseconds. Multiple consecutive zero-value audio frames are defined as at least four consecutive zero-value frames to avoid misjudging brief zero-point noise as a silence segment. For example, if five consecutive zero-value audio frames appear starting at system time 1203 milliseconds and lasting for 12.5 milliseconds, the start time of the silence segment to be determined is 1203 milliseconds, and the end time is 1215.5 milliseconds.

[0044] When a forced silence gap injected by a circuit breaker operation exists, such that the absolute value of the difference between the start time of the forced silence gap and the start time of the silence segment to be judged is less than a preset time deviation threshold, and the absolute value of the difference between the duration of the forced silence gap and the duration of the silence segment to be judged is also less than the preset time deviation threshold, the silence segment to be judged is determined to be a fake silence. The preset time deviation threshold is set based on the statistical value of audio driver interrupt response time and system scheduling jitter. Experimental data shows that 99% of the jitter is less than 0.5 milliseconds, so the typical value of the preset time deviation threshold is 0.5 milliseconds. Using the above example, the forced silence gap injected by the circuit breaker operation starts at 1200 milliseconds and lasts for 10 milliseconds, while the silence segment to be judged starts at 1203 milliseconds and lasts for 12.5 milliseconds. The absolute value of the difference between their start times is 3 milliseconds, which is greater than 0.5 milliseconds. Therefore, the condition is not met, and the silence segment to be judged is not determined to be a fake silence. If the start time of another undetermined silent segment is 1200 milliseconds and the duration is 10.2 milliseconds, then the difference in start time is 0 milliseconds (less than 0.5 milliseconds) and the difference in duration is 0.2 milliseconds (less than 0.5 milliseconds), and it is determined to be a fake silence.

[0045] Otherwise, it is judged as true ambient silence. True ambient silence refers to zero-value audio data generated when the user does not actually make a sound or the environment itself is quiet, such as when the user pauses speaking during a call or when no one is talking in the car. For example, if multiple consecutive zero-value audio frames appear in the audio data stream when the system has not performed any circuit breaker operation, it is judged as true ambient silence, and upper-layer applications such as voice assistants can normally trigger the silence timeout or end the voice detection.

[0046] Adjusting the preset frequency thresholds and priority backoff parameters for each audio channel involves the following steps: The switching frequency change rate of each audio channel is collected within two consecutive preset time windows. The calculation method for the switching frequency change rate is: the difference between the switching frequency of the later preset time window and the switching frequency of the earlier preset time window, divided by the duration of the preset time window. This duration is set between 400 milliseconds and 1000 milliseconds based on the system response speed requirements, with a typical value of 800 milliseconds. For example, if the switching frequency of a certain audio channel is 91 times per second in the first preset time window and 120 times per second in the second window, the difference is 29 times per second. Dividing this by 0.8 seconds gives 36.25 times per second squared, which is the switching frequency change rate.

[0047] When the rate of change exceeds the first rate of change threshold, the preset frequency threshold corresponding to that audio channel is increased by one step. The first rate of change threshold is set based on the critical rate of change at which the system begins to exhibit focus jitter in the experimental data, typically set to 20 times per second squared. The first step size is set so that each increase does not exceed 10% of the initial threshold to avoid overshoot, typically set to 0.2 times per second. For example, if the rate of change is 36.25 times per second squared, which is greater than 20, the preset frequency threshold is increased from 30 times per second to 30.2 times per second.

[0048] When the rate of change is less than the second rate of change threshold and the current priority backoff parameter has been adjusted to the preset lower limit, the preset frequency threshold corresponding to the audio channel is lowered by a second step. The second rate of change threshold is set based on the allowable small fluctuation range after the system stabilizes, typically five times the square of one second. The preset lower limit is defined based on the minimum value of the priority backoff parameter, typically zero. It should be noted that in this embodiment, the lower limit of the priority backoff parameter is set to zero. Simultaneously, when the priority backoff parameter drops to zero, the system starts a recovery timer with a duration of five seconds. After the timer expires, the priority backoff parameter of the audio channel is automatically reset to its initial value. As an alternative implementation, the lower limit can also be set to a decimal greater than zero (e.g., 0.1), in which case a recovery timer is not required.

[0049] The first and second rate of change thresholds are set based on the fact that the unit of the switching frequency change rate is times per second². Experiments show that when the rate of change exceeds 20 times per second², the system is about to enter a state of focus turbulence, so the first rate of change threshold is set to 20 times per second². When the rate of change is below 5 times per second² and the priority backoff parameter has been adjusted to the lower limit, the system is too conservative, so the second rate of change threshold is set to 5 times per second². The first step size and the second step size are both taken as 0.5% of the preset frequency threshold, with a typical value of 0.2 times per second. For example, if the rate of change is three times squared per second, which is less than five, and the current priority backoff parameter is already zero, then the preset frequency threshold is lowered from thirty times per second to twenty-nine point eight times per second.

[0050] The silence injection rate for each audio channel is calculated over K consecutive preset time windows. The silence injection rate is the ratio of the number of forged silence data packets to the total number of data packets. The value of K is chosen based on statistical significance requirements. Experiments show that at least ten windows are needed to obtain a stable injection rate estimate; therefore, K is an integer greater than or equal to ten, typically twenty. For example, if an audio channel receives one thousand audio data packets over twenty consecutive preset time windows, and sixty of them are forged silence data packets, then the silence injection rate is 0.06%, or six percent.

[0051] When the silence injection rate exceeds the first injection rate threshold, the step size coefficient is multiplied by the first amplitude scaling factor, where the first amplitude scaling factor is less than one. The first injection rate threshold is set based on the audio application's tolerance limit for fake silence, typically set to 5%. The first amplitude scaling factor is set based on the principle that decreasing it by 25% each time can smoothly reduce the adjustment intensity, typically set to 0.75. For example, if the original step size coefficient is 0.1, and the silence injection rate is 6%, which is greater than 5%, then the new step size coefficient is 0.1 multiplied by 0.75, which equals 0.075.

[0052] When the silent injection rate is less than the second injection rate threshold, the step size coefficient is multiplied by the second amplitude scaling factor, where the second amplitude scaling factor is greater than one. The second injection rate threshold is set based on the acceptable low interference level of the system, typically taking a value of 1%. The second amplitude scaling factor is set based on the principle that increasing it by 25% each time can accelerate the convergence speed, typically taking a value of 1.25. For example, if the original step size coefficient is 0.1, and the silent injection rate is 0.5%, which is less than 1%, then the new step size coefficient is 0.1 multiplied by 1.25, which equals 0.125.

[0053] The above algorithms or formulas are all dimensionless and numerical calculations, and the results are obtained by software simulation based on a large amount of collected data to obtain the most recent real-world results. The preset parameters are set by those skilled in the art according to the actual situation.

[0054] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0055] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0056] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0057] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A multi-channel audio acquisition method for concurrent application scenarios, characterized in that, Includes the following steps: Each audio application is allocated a separate audio channel, and a separate focus request queue and audio data buffer are established for each audio channel; Real-time monitoring of focus request events in each audio channel, and calculation of the switching frequency of focus request events within a preset time window; When the switching frequency of any audio channel exceeds the preset frequency threshold corresponding to that audio channel, the focus retry interval and priority backoff parameters of that audio channel are dynamically adjusted according to the value of the current switching frequency. After dynamically adjusting the focus retry interval and priority backoff parameters, if the switching frequency of the same audio channel still exceeds the preset frequency threshold corresponding to the audio channel, or if a circular dependency pattern is detected between multiple audio channels, a circuit breaker operation is performed. The circuit breaker operation includes: freezing the focus request permission of at least one audio channel and inserting a forced silence gap in the audio data buffer corresponding to the audio channel. The source of silence data in the audio data stream of each audio channel is identified, and the fake silence generated by the forced silence gap injected by the circuit breaker operation is distinguished from the real environment silence. A silence type mark is added to the audio data stream of that audio channel. The system collects real-time operating parameters for each audio channel after adjustment. These parameters include at least the switching frequency change rate and the silence injection rate for each audio channel. The switching frequency change rate is the ratio of the difference in switching frequencies between two adjacent preset time windows to the duration of the preset time window. The system then feeds back these real-time operating parameters to the step of real-time monitoring of focus request events in each audio channel and calculating the switching frequency. This is used to adjust the preset frequency threshold and priority backoff parameters for each audio channel in the following manner: increasing or decreasing the preset frequency threshold based on the switching frequency change rate, and increasing or decreasing the step size coefficient of the priority backoff parameters based on the silence injection rate. The step size coefficient is the amount of change when updating the priority backoff parameters each time.

2. The multi-channel audio acquisition method for multi-application concurrent scenarios according to claim 1, characterized in that, The initial value of the preset frequency threshold is preset according to the type of audio application corresponding to each audio channel.

3. The multi-channel audio acquisition method for multi-application concurrent scenarios according to claim 1, characterized in that, Calculate the switching frequency of focus request events within a preset time window, specifically including the following steps: Divide the preset time window into M consecutive sub-time intervals, each with an equal duration, where M is an integer greater than or equal to three; Count the number of focus request events that occur within each sub-time interval; Calculate the absolute value of the difference in the number of occurrences between two adjacent sub-time intervals; The fluctuation factor is obtained by summing the absolute values ​​of all differences and dividing by the difference of M minus one, where the difference of M minus one is the number of adjacent differences. The switching frequency is obtained by multiplying the total number of focus request events within the preset time window by the fluctuation factor and then dividing by the duration of the preset time window.

4. The multi-channel audio acquisition method in multi-application concurrent scenarios according to claim 1, characterized in that, Dynamically adjusting the focus retry interval and priority backoff parameters of the audio channel includes the following steps: Calculate the ratio of the current switching frequency to the preset frequency threshold. When the preset frequency threshold is greater than zero, use the ratio as the load factor. When the preset frequency threshold is equal to zero, set the load factor to a predefined positive value. Calculate the natural logarithm of the sum of the load factor and the value of one, and take the integer part as the retreat index; The focus retry interval is updated as follows: the new focus retry interval is equal to the original focus retry interval multiplied by two raised to the power of the backoff exponent; The priority backoff parameter is updated as follows: the new priority backoff parameter is equal to the original priority backoff parameter minus the product of the backoff exponent and the step size coefficient, and the new priority backoff parameter is restricted to be no less than zero. The retry interval for new focus points is limited to a predefined maximum retry interval, and the backoff parameter for new priorities is limited to a predefined maximum backoff parameter.

5. The multi-channel audio acquisition method in a multi-application concurrent scenario according to claim 1, characterized in that, A circular dependency pattern was identified among multiple audio channels, specifically including: A focus request directed graph is established for each audio channel. Nodes in the directed graph represent audio channels, and directed edges represent one audio channel initiating focus preemption for another audio channel. A recursive backtracking algorithm is used to traverse the directed graph. When a duplicate node is found in a certain traversal path, it is determined that a circular dependency pattern exists. The execution time of the recursive backtracking algorithm is linearly related to the sum of the number of audio channels and the number of focus preemption relationships between channels. It records all audio channel identifiers involved in the circular dependency pattern.

6. The multi-channel audio acquisition method in a multi-application concurrent scenario according to claim 1, characterized in that, The duration for freezing focus request permission for at least one of the audio channels is determined as follows: If the preset frequency threshold is greater than zero, calculate the difference between the current switching frequency of the frozen audio channel and the preset frequency threshold, and then divide it by the preset frequency threshold to obtain the frequency excess ratio. When the frequency exceeds the ratio but is less than the first preset ratio threshold, the first freeze duration is applied. When the frequency exceeds the ratio by a factor greater than or equal to the first preset ratio threshold and less than or equal to the second preset ratio threshold, a second freeze duration is adopted, wherein the second freeze duration is greater than the first freeze duration. When the frequency exceeds the ratio by more than the second preset ratio threshold, a third freeze duration is adopted, wherein the third freeze duration is greater than the second freeze duration; The first freeze duration, the second freeze duration, and the third freeze duration are all positive numbers, and each is not less than one complete audio data packet transmission cycle.

7. The multi-channel audio acquisition method for multi-application concurrent scenarios according to claim 1, characterized in that, Distinguishing between simulated silence generated by forced silence gaps injected during circuit breaking operations and real-world silence specifically includes: Record the execution timestamp of each circuit breaker operation, as well as the start and end times of the injected forced silence gap; When multiple consecutive zero-value audio frames appear in the audio data stream, the continuous data segment formed by these multiple zero-value audio frames is taken as the silence segment to be determined, and the start time and end time of the silence segment to be determined are extracted. When there is a forced silence gap injected by a circuit breaker operation, such that the absolute value of the difference between the start time of the forced silence gap and the start time of the silence segment to be determined is less than a preset time deviation threshold, and the absolute value of the difference between the duration of the forced silence gap and the duration of the silence segment to be determined is also less than the preset time deviation threshold, the silence segment to be determined is determined to be a fake silence. Otherwise, it will be judged as a real-world silent environment.

8. The multi-channel audio acquisition method in a multi-application concurrent scenario according to claim 1, characterized in that, Adjusting the preset frequency thresholds and priority backoff parameters for each audio channel involves the following steps: Collect the switching frequency change rate of each audio channel within two consecutive preset time windows; When the rate of change is greater than the first rate of change threshold, the preset frequency threshold corresponding to the audio channel is increased by the first step length. When the rate of change is less than the second rate of change threshold and the current priority backoff parameter has been adjusted to the preset lower limit, the preset frequency threshold corresponding to the audio channel is lowered by the second step. The mute injection rate of each audio channel within K consecutive preset time windows is calculated. The mute injection rate is the ratio of the number of forged mute data packets to the total number of data packets. When the silent injection rate is greater than the first injection rate threshold, the step size coefficient is multiplied by the first amplitude scaling factor, where the first amplitude scaling factor is less than one. When the silent injection rate is less than the second injection rate threshold, the step size coefficient is multiplied by the second amplitude scaling factor, where the second amplitude scaling factor is greater than one. Where K is an integer greater than or equal to ten.

Images