Method and system for main processor decision and resource optimization within audio processor scheduling

Abstract

This invention discloses a resource optimization method and system for main processor decision-making and audio processor internal scheduling. The method includes acquiring processor load information and channel characteristic information of each audio input channel when the audio processor performs audio processing on multiple audio input channels; when the processor load information meets preset load conditions, the main processor determines the adjustment result of the audio processing configuration based on the processor load information and the channel characteristic information, according to preset resource optimization rules; the main processor generates and sends control commands to the audio processor according to the adjustment result; the audio processor updates the mixing routing relationship according to the control commands in its internal scheduling, and switches or adjusts the processing links of each channel according to the processing algorithm configuration. This invention improves the continuity and stability of conference audio processing without changing the hardware structure.

Description

Technical Field

[0001] This invention relates to the field of audio processing technology, and in particular to a resource optimization method, system, computer device, and storage medium for main processor decision-making and audio processor scheduling. Background Technology

[0002] Audio conferencing terminals, conferencing hosts, and audio processing devices with multiple inputs typically need to simultaneously connect to multiple microphones, remote audio feedback, and local playback signal sources. The audio processor performs mixing, gain control, echo cancellation, feedback suppression, and noise reduction on each input before outputting to speakers or the feedback link. To meet real-time requirements, engineering implementations often employ a division of labor where the main processor handles business control and system management, while the audio processor handles real-time audio algorithms and mixing scheduling. In conferencing scenarios, the number of channels, speaker activity, and the availability of algorithm links dynamically change over time, causing sudden fluctuations in the computational load on the audio processor.

[0003] In existing technologies, audio processors typically operate continuously according to preset mixing routes and algorithm links. When the number of participating channels increases, multiple channels are active simultaneously, or many high-load algorithms are enabled, the audio processor is prone to approaching its processing limit or even becoming overloaded. Since audio processing is a highly real-time task, excessive load can lead to frame loss, latency jitter, audio quality degradation, or even link anomalies, thereby affecting the continuity of the meeting and the intelligibility of key speaking channels. At the same time, the conferencing system must prioritize ensuring the processing quality of important channels under load fluctuations and promptly restore necessary processing capacity once conditions recover to avoid long-term degradation and a decline in user experience. Summary of the Invention

[0004] The purpose of this application is to propose a resource optimization method, system, computer device, and storage medium for main processor decision-making and audio processor scheduling, in order to solve the technical problem of limited ability to maintain system processing stability and prioritize conference output quality under limited processing resources in a main processor and audio processor collaborative architecture.

[0005] To address the aforementioned technical problems, this application provides a resource optimization method for main processor decision-making and audio processor intra-processor scheduling, employing the following technical solution:

[0006] The processor load information and channel characteristic information of each audio input channel are obtained when the audio processor performs audio processing on multiple audio input channels. The channel characteristic information is used to characterize at least the activity state or channel level of the channel.

[0007] When the processor load information meets the preset load conditions, the main processor determines the adjustment result of the audio processing configuration based on the processor load information and the channel feature information according to the preset resource optimization rules. The adjustment result indicates at least one of the channel set that needs to be degraded and the channel set that needs to be prioritized.

[0008] The main processor generates and sends control commands to the audio processor based on the adjustment results. The control commands include at least mixing routing configuration and processing algorithm configuration.

[0009] The audio processor updates the mixing routing relationship according to the control instructions in its internal scheduling, and switches or adjusts the processing links of each channel according to the processing algorithm configuration.

[0010] To address the aforementioned technical problems, this application also provides a resource optimization system for main processor decision-making and audio processor intra-processor scheduling, employing the following technical solution:

[0011] The acquisition module is used to acquire processor load information and channel characteristic information of each audio input channel when the audio processor performs audio processing on multiple audio input channels. The channel characteristic information is used to characterize the activity state or channel level of the channel at least.

[0012] The determination module is used to determine the adjustment result of the audio processing configuration by the main processor based on the processor load information and the channel feature information according to the preset resource optimization rules when the processor load information meets the preset load conditions. The adjustment result indicates at least one of the set of channels that need to be degraded and the set of channels that need to be prioritized.

[0013] A sending module is used to generate and send control commands to the audio processor based on the adjustment results by the main processor. The control commands include at least mixing routing configuration and processing algorithm configuration.

[0014] The update module is used by the audio processor to update the mixing routing relationship according to the control instructions in its internal scheduling, and to switch or adjust the processing links of each channel according to the processing algorithm configuration.

[0015] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0016] A computer device includes a memory and a processor, the memory storing computer-readable instructions, wherein the processor, when executing the computer-readable instructions, implements the steps of the resource optimization method for main processor decision-making and audio processor intra-scheduling as described above.

[0017] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0018] A computer-readable storage medium storing computer-readable instructions, which, when executed by a processor, implement the steps of the resource optimization method for main processor decision-making and audio processor scheduling as described above.

[0019] Compared with the prior art, the embodiments of this application have the following main advantages:

[0020] The resource optimization method for main processor decision-making and audio processor internal scheduling disclosed in this application simultaneously acquires the processor load information of the audio processor and the channel characteristic information of each audio input channel. When the load meets preset conditions, the main processor determines the adjustment result of the audio processing configuration according to preset resource optimization rules, and drives the audio processor to synchronously update the mixing routing relationship and channel processing link in the internal scheduling with control instructions. This enables real-time adaptive resource adjustment under the conditions of multi-channel audio concurrency and changes in algorithm complexity, reduces the risk of frame drops and latency jitter caused by overload, and improves the continuity and stability of conference audio processing without changing the hardware structure. Attached Figure Description

[0021] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of an embodiment of the resource optimization method for main processor decision-making and audio processor scheduling according to this application;

[0023] Figure 2 This is a schematic diagram of a structure of an embodiment of the resource optimization system for main processor decision-making and audio processor scheduling according to this application;

[0024] Figure 3 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0026] In a typical implementation of this invention, the system adopts a collaborative architecture between a main processor and an audio processor. The main processor performs business control, policy decisions, and configuration distribution, while the audio processor executes real-time audio algorithms and mixing processes, and performs internal scheduling based on external configurations during operation. Internal scheduling refers to the audio processor dynamically updating the mixing routing relationships from each input channel to each output without interrupting basic audio stream processing, and starting, stopping, or adjusting the processing intensity of the processing link corresponding to each input channel. A processing link can be understood as a combination of several processing modules sequentially connected to the same audio stream, such as the ordered execution path of modules like echo cancellation, feedback suppression, noise reduction, automatic gain control, equalization, or dynamic range control. Adjusting the processing intensity can be achieved through feasible methods such as reducing the algorithm order, reducing filter length, switching to a low-complexity mode, or reducing the update frequency.

[0027] refer to Figure 1 The diagram illustrates a flowchart of an embodiment of a resource optimization method for main processor decision-making and audio processor intra-processor scheduling according to this application. The resource optimization method for main processor decision-making and audio processor intra-processor scheduling includes the following steps:

[0028] Step S101: Obtain processor load information and channel characteristic information of each audio input channel when the audio processor performs audio processing on multiple audio input channels. The channel characteristic information is used to characterize the activity state or channel level of the channel.

[0029] In this embodiment, the resource optimization method for main processor decision-making and audio processor scheduling, on which the electronic device runs, can send or receive data via wired or wireless connection. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra-wideband) connections, and other currently known or future-developed wireless connection methods.

[0030] In this embodiment, processor load information is used to characterize the current computational pressure of the audio processor. This can be periodically collected and reported by the audio processor, for example, by calculating the proportion of computational load used for algorithms within an audio frame period, the real-time margin of the task queue, or whether the processing latency is close to the upper limit of the frame period. In the implementation of the conferencing terminal, the audio processor typically processes audio data with a fixed frame length. The main processor can read the load value from the audio processor at a preset period to obtain load information reflecting real-time pressure. Channel characteristic information is used at least to characterize the activity state or channel level of the channel. A channel refers to each independent input audio signal source, such as multiple microphone channels, remote transmission channels, playback and re-acquisition channels, etc. The activity state can be obtained through voice activity detection, i.e., determining whether there is valid speech or a valid sound source in the channel within several consecutive frames. The channel level can be obtained by calculating the audio amplitude or energy, for example, calculating the root mean square energy of one or more frames and converting it into a level value, to reflect whether the channel is currently in a state of significant speaking or high energy. By simultaneously acquiring load information and channel characteristic information, the main processor can determine whether the system is close to overload, as well as the participation value and activity level of each channel at the current moment, thus providing clear input for subsequent resource optimization decisions.

[0031] Step S102: When the processor load information meets the preset load conditions, the main processor determines the adjustment result of the audio processing configuration based on the processor load information and the channel feature information according to the preset resource optimization rules. The adjustment result indicates at least one of the channel set that needs to be downgraded and the channel set that needs to be prioritized.

[0032] In this embodiment, when the processor load information meets the preset load conditions, the main processor determines the adjustment result of the audio processing configuration based on the processor load information and channel characteristic information, according to the preset resource optimization rules. The preset load conditions can be set by the system at the factory or during deployment to define when the audio processor is considered to be in a resource-scarce state. For example, this condition is met when the load exceeds a certain threshold for several frames, or when the processing latency margin is lower than a certain threshold. The preset resource optimization rules refer to the decision rules for selecting degraded and protected objects when the load is limited. Their input includes at least load information and channel characteristic information, and the output is the adjustment result of the audio processing configuration. The audio processing configuration can be understood as a combination configuration of mixing routing and algorithm links. The adjustment result indicates at least one of the sets of channels that need to be degraded and the sets of channels that need to be prioritized for protection. The channel sets are used to explicitly assign the decision result to specific channel objects. The set of channels requiring degraded processing refers to a group of channels whose processing quality or participation can be reduced when resources are insufficient. Examples include low-value channels with low current levels and prolonged inactivity, or non-critical business channels. The set of channels requiring priority protection refers to a group of channels that should maintain a relatively complete processing chain and stable output even with limited resources. Examples include the chairman's microphone, the main speaker channel, or channels currently in continuous speaking. Taking a conference host as an example, when twelve microphones are simultaneously connected and multiple algorithm modules are enabled, if multiple simultaneous speaking causes an increase in the audio processor's load, the main processor can prioritize channels based on their activity status and level, placing continuously active channels with high levels into the priority protection set and relegating inactive channels with low levels to the degraded set. This releases computing resources without compromising the intelligibility of critical speeches. In another example, if the remote return channel and the local playback recording channel are essential channels, resource optimization rules can directly fix them as priority protection channels, thus avoiding echo path instability or abnormal listening due to degraded channels.

[0033] In step S103, the main processor generates and sends control instructions to the audio processor based on the adjustment results. The control instructions include at least mixing routing configuration and processing algorithm configuration.

[0034] In this embodiment, the main processor generates and sends control commands to the audio processor based on the adjustment results. These control commands include at least a mixing routing configuration and a processing algorithm configuration. The mixing routing configuration describes how each input channel participates in the mixing relationship of at least one output, such as whether certain input channels are mixed into the main output, recording output, or return transmission output, or whether their mixing weight is reduced, thus directly affecting the final mixing result at the output. The processing algorithm configuration describes which processing modules in each channel's processing link are enabled, which are disabled, or in a low-complexity state, and the corresponding processing intensity settings, thus directly affecting the audio processor's computational load and processing effect. When the main processor generates control commands based on the channel set obtained above, it can reduce or cancel the mixing participation of degraded channels in the mixing routing configuration, and simultaneously disable or reduce the intensity of high-load algorithms for degraded channels in the processing algorithm configuration, while maintaining the necessary basic processing links for priority channels to ensure a balance between core channel sound quality and system real-time performance. Control commands can be sent through the control interface between the main processor and the audio processor in the system. The interface type can be a serial bus or other feasible control channels. The key is that the control commands can be parsed by the audio processor and drive its internal configuration updates.

[0035] In step S104, the audio processor updates the mixing routing relationship in its internal scheduling according to the control instructions, and switches or adjusts the processing links of each channel according to the processing algorithm configuration.

[0036] In this embodiment, the audio processor updates the mixing routing relationship according to control instructions in its internal scheduling, and switches or adjusts the processing links of each channel according to the processing algorithm configuration. Updating the mixing routing relationship means that the audio processor applies the mixing routing configuration in the control instructions to the currently running mixing matrix or routing table, so that the participation of degraded channels at the target output is reduced or removed, while priority channels are kept in the mixing participation at the target output. Switching or adjusting the processing links means that the audio processor applies the processing algorithm configuration to the algorithm module graph of each channel. For example, it may turn off high-load modules such as echo cancellation or feedback suppression for degraded channels, or switch them to a low-complexity mode, while maintaining basic processing such as noise reduction and echo cancellation for priority channels, thereby reducing the overall computational load and avoiding processing frame backlog when the load is high. Taking a real-world meeting scenario as an example, when the system detects that the audio processor load has entered a preset load condition, the main processor identifies several microphone channels that have been silent for a long time as degraded channels and issues routing configurations to cancel their participation in the main output mixing. At the same time, it issues algorithm configurations to shut down high-complexity processing modules for these degraded channels. The audio processor updates the mixing matrix and reconstructs the channel processing links without interrupting the continuity of the main output audio, so that the main output still prioritizes the protection of the channels. The overall system load falls back to a range that can be processed in real time, thereby avoiding audio stuttering, frame drops, or significant latency jitter.

[0037] This application simultaneously acquires the processor load information of the audio processor and the channel characteristic information of each audio input channel. When the load meets preset conditions, the main processor determines the adjustment result of the audio processing configuration according to preset resource optimization rules, and drives the audio processor to synchronously update the mixing routing relationship and channel processing link in the internal scheduling with control instructions. This enables real-time adaptive resource adjustment under the conditions of multi-channel audio concurrency and changes in algorithm complexity, reduces the risk of frame drops and latency jitter caused by overload, and improves the continuity and stability of conference audio processing without changing the hardware structure.

[0038] In some optional implementations of this embodiment, the step of determining the adjustment result of the audio processing configuration by the main processor based on the processor load information and the channel feature information according to a preset resource optimization rule further includes:

[0039] Obtain meeting status information and read the resource protection constraints corresponding to the meeting status information from the pre-stored scenario configuration;

[0040] The resource protection constraints include at least channel priority rules and a basic processing set that must be retained when the load is limited, so that the adjustment results are generated under the constraints.

[0041] In this embodiment, when the main processor determines the adjustment result of the audio processing configuration based on processor load information and channel characteristic information, it does not make a degradation decision solely based on the current load level or the level of a certain channel. Instead, it further introduces the constraints of meeting status information and pre-stored scene configuration to limit the decision boundary. The meeting status information here is used to characterize the current interaction mode of the meeting. For example, in the speaker mode, it is often necessary to stably ensure the processing quality of the speaker channel; in the discussion mode, it may be necessary to simultaneously ensure the processing quality of multiple speaking channels; and in the silent or standby state, processing overhead can be more aggressively reduced. The pre-stored scene configuration can be understood as a set of policy parameters pre-fixed by the system for different meeting states. The resource protection constraints include at least channel priority rules and the basic processing set that must be retained when the load is limited. Channel priority rules are used to clarify the priority of different channels when resources are scarce. For example, the chairman microphone, the main speaker channel, or the fixed return channel are set to higher priority so that the mixing participation and key processing capabilities of these channels are preserved first when degrading. The basic processing set refers to a minimum set of processing modules that should be maintained even when the load is limited. For example, maintaining necessary noise reduction or necessary echo path stabilization processing to avoid a significant decrease in output quality or system stability due to excessive trimming.

[0042] This application incorporates meeting status information and pre-stored scenario configurations into resource optimization decisions, making the adjustment results subject to resource protection constraints. When the load is limited, it can prioritize the protection of critical channels according to the meeting business context and retain the necessary basic processing capabilities. This avoids the degradation of audio quality in critical speaking channels or a sudden drop in meeting experience caused by coarse-grained degradation based solely on load or level, thereby improving the controllability, interpretability, and adaptability of resource optimization strategies.

[0043] In some optional implementations of this embodiment, the step of determining the adjustment result of the audio processing configuration by the main processor based on the processor load information and the channel feature information according to a preset resource optimization rule further includes:

[0044] The activity status of each audio input channel is determined based on the channel feature information, and the set of channels that need to be prioritized and the set of channels that need to be downgraded are determined according to the channel priority rules.

[0045] Channels with an inactive status and a priority no higher than a preset level are identified as the set of channels that need to be downgraded, while channels with an active status and a priority no lower than a preset level are identified as the set of channels that need to be prioritized.

[0046] In this embodiment, channel feature information is used to determine the activity status of each audio input channel. The activity status can be obtained by voice activity detection or energy persistence determination, meaning whether there is effective speech or obvious sound source in the channel within several consecutive frames. The main processor, based on channel priority rules, identifies channels with inactive activity and priority no higher than a preset level as the set of channels requiring degradation. These channels contribute little to the conference output at the current moment and have low priority, making them suitable for resource reclamation. Simultaneously, channels with active activity and priority no lower than a preset level are identified as the set of channels requiring priority protection. These channels are usually currently speaking or set as critical, and their mixing participation and basic processing capabilities should be preserved when the load is limited. Taking multi-microphone conferences as an example, when some microphones are silent for extended periods and set as ordinary participant channels, they can be classified into the degradation set, while the chairman's microphone or currently continuously speaking channels will be classified into the protection set even if the load is high. This ensures that the adjustment results have reproducible judgment logic and avoids arbitrariness.

[0047] This application utilizes channel characteristic information to determine the channel activity status and combines it with channel priority rules to form a set of channels that need to be downgraded and a set of channels that need to be prioritized. This provides clear and reproducible criteria for determining which channels to downgrade and which to prioritize, thereby enabling the targeted recovery of processing overhead from low-value channels and the centralized protection of active and high-priority channels when resources are scarce. This improves resource release efficiency and reduces the probability of mistakenly downgrading critical channels.

[0048] In some optional implementations of this embodiment, the determination step of whether the processor load information meets the preset load conditions includes:

[0049] Set entry and exit thresholds, and trigger the adjustment of the audio processing configuration only when the processor load information continuously meets the entry threshold for a preset duration;

[0050] After triggering, restorative adjustments are allowed only when the processor load information continuously meets the exit threshold for a preset duration, in order to suppress repeated switching.

[0051] In this embodiment, the entry threshold is used to define when the audio processor is considered to be overloaded or under resource strain. Audio processing configuration adjustments are only triggered when the processor load information continuously meets the entry threshold for a preset duration. This filters out false triggers caused by instantaneous load spikes. The exit threshold is used to define when the load is considered to have recovered to an acceptable range. After adjustment has been triggered, restorative adjustments are only allowed when the processor load information continuously meets the exit threshold for a preset duration. This avoids repeated degradation and recovery caused by load fluctuations around the threshold, thereby reducing audio output latency jitter, frequent processing link reconstruction, and the resulting auditory instability. Taking frame processing as an example, if the system runs with a fixed frame length, the entry threshold can be configured for a load exceeding a certain percentage for several frames, and the exit threshold can be configured for a load falling below another percentage for several frames. This creates a hysteresis interval between triggering and recovery, thereby improving overall operational stability.

[0052] This application achieves hysteresis and anti-jitter by using an entry threshold and an exit threshold combined with a duration determination mechanism. This effectively suppresses frequent degradation and back-switching caused by load fluctuations near the threshold, reduces auditory abrupt changes and system jitter caused by repeated reconstruction of mixing routing and algorithm links, thereby improving the stability of audio output and the reliability of real-time processing under long-term operation.

[0053] In some optional implementations of this embodiment, the above-mentioned mixing routing configuration includes mixing matrix parameters for updating mixing routing relationships; the processing algorithm configuration includes channel identification information and algorithm link parameters associated with the channel identification information, wherein the channel identification information is used to identify the set of channels that need to be degraded, and the algorithm link parameters are used to indicate whether the processing links of the set of channels that need to be degraded are turned off, enabled, or the processing intensity is adjusted.

[0054] The control commands are sent through a serial peripheral interface and include a serial number and an acknowledgment feedback mechanism to ensure that the mixing routing configuration and the processing algorithm configuration are consistent and effective on the audio processor side.

[0055] In this embodiment, the mixing routing configuration is carried by mixing matrix parameters. These parameters characterize the mixing participation relationships and weights from each input channel to each output, enabling the audio processor to update the mixing routing relationships accordingly. The processing algorithm configuration is carried by channel identification information and algorithm link parameters. Channel identification information identifies the set of channels requiring downgrade processing. Algorithm link parameters, associated with the channel identification information, instruct the processing links of the downgraded channel set to be disabled, enabled, or have their processing intensity adjusted. For example, high-complexity algorithm modules may be disabled or switched to low-complexity mode for specific channels, while maintaining the established links for other channels. Control commands are sent via a serial peripheral interface and include a sequence number and an acknowledgment feedback mechanism. The sequence number distinguishes different batches of configurations, and the acknowledgment feedback mechanism informs the main processor whether the audio processor has received and applied the configuration corresponding to the sequence number. This ensures that the mixing routing configuration and processing algorithm configuration are consistently effective on the audio processor side, avoiding abnormal states where only the mixing matrix is ​​updated but the algorithm links are not updated synchronously, or vice versa. Taking actual deployment as an example, after the main processor issues a control command for a certain serial number, the audio processor returns confirmation information after completing the mixing matrix update and link switching. Based on this, the main processor decides whether to continue issuing the next configuration or maintain the current configuration, thus forming a verifiable configuration closed loop.

[0056] This application structures the mixing routing configuration and processing algorithm configuration into parsable mixing matrix parameters, channel identification information, and algorithm link parameters, enabling control commands to be precisely applied to the mixing and algorithm adjustments at the channel level. At the same time, the application uses a sequence number and confirmation feedback mechanism to ensure that the two types of configurations are consistently effective on the audio processor side, reducing the risk of abnormal sound quality, routing mismatch, or inconsistent processing link status caused by configuration asynchrony, thereby improving the determinism and verifiability of scheduling execution.

[0057] In some optional implementations of this embodiment, the step of updating the mixing routing relationship by the audio processor in its internal scheduling according to the control instructions includes:

[0058] For the set of channels that need to be downgraded, cancel their mixing participation relationship to at least one target output, and for the set of channels that need to be prioritized, maintain their mixing participation relationship to the at least one target output.

[0059] After canceling the mixing participation relationship, input gain suppression is performed on the channel whose mixing was canceled to make its input gain meet the preset suppression condition, so as to reduce the coupling interference to the at least one target output terminal.

[0060] In this embodiment, the specific actions of the audio processor in updating the mixing routing relationship according to control instructions are further defined. It emphasizes the use of differentiated mixing participation strategies for the channel sets requiring degradation and those requiring priority protection, and performs input gain suppression after canceling the mixing to reduce coupling interference. Canceling the mixing participation of the channel sets requiring degradation to at least one target output means that these channels no longer participate in generating the mixing result of that target output, or their mixing weight is reduced to an equivalent level of non-participation. Maintaining the mixing participation of the channel sets requiring priority protection to the target output ensures that critical channels continue to output to the main output or return transmission. After canceling the mixing participation, performing input gain suppression on the canceled mixing channels means reducing the input gain of these channels to a range that meets preset suppression conditions, such as reducing their preamp gain or limiting their automatic gain control upper limit, to reduce the risk of crosstalk, noise coupling, or feedback to the system, thereby avoiding the introduction of unstable factors due to high gain even though they are not participating in the mixing. Taking multi-microphone conferencing as an example, even if some microphones that have been silent for a long time and have been degraded are no longer mixed into the main output, if they still maintain a high preamp gain, it may lead to an increase in internal coupling interference when the ambient noise changes. Input gain suppression can further stabilize the output quality and system stability after degradation.

[0061] This application cancels the mixing participation of the channels that need to be degraded to the target output while maintaining the mixing participation of the priority channels. This concentrates the mixing energy at the output on the key channels, thereby prioritizing the intelligibility and subjective listening experience of the target output when the load is limited. After canceling the mixing, input gain suppression is performed on the canceled mixing channels to reduce coupling interference, further reducing noise crosstalk and potential feedback risks, and improving the stability and anti-interference capability of the target output.

[0062] In some optional implementations of this embodiment, the step of switching or adjusting the processing link of each channel according to the processing algorithm configuration includes:

[0063] For the set of channels that need to be downgraded, at least one type of high-load processing algorithm shall be turned off or reduced, and a preset basic processing set shall be maintained for the set of channels that need to be prioritized.

[0064] Set an exit threshold for recovery determination, and when the processor load information is continuously not higher than the exit threshold for a preset duration, and there is a target channel in the set of channels that need to be downgraded that continuously meets the preset activity conditions for a preset duration, restore the mixing participation relationship from the target channel to the at least one target output, and restore the high load processing algorithm corresponding to the target channel as needed.

[0065] In this embodiment, shutting down or reducing at least one type of high-load processing algorithm for the channel set requiring degradation processing means shutting down or downgrading processing modules with high computational complexity that can be degraded on non-critical channels. Examples include reducing the length of echo cancellation filters, reducing the update frequency of feedback suppression, shutting down certain enhancement processing modules, or switching to a low-complexity mode. Maintaining a preset basic processing set for the channel set requiring priority protection means maintaining a minimum but necessary set of processing modules for critical channels to ensure the intelligibility of key speech and the basic auditory stability of the system. Recovery determination is achieved through an exit threshold and a preset duration. The exit threshold characterizes the load falling back to a recoverable level. When the processor load information continuously does not exceed the exit threshold for a preset duration, and a target channel in the channel set requiring degradation processing continuously meets preset activity conditions for a preset duration, it indicates that the system load has recovered and the target channel has become important again. Therefore, the mixing participation relationship of the target channel to at least one target output is restored, and the high-load processing algorithm corresponding to the target channel is restored as needed, so that it obtains a more complete processing link after the switchback. Taking a meeting discussion scenario as an example, a regular microphone is degraded and canceled from participating in the mixing during an overload period. When the load drops and the microphone detects continuous speaking, the system uses the above-mentioned recovery judgment to make it participate in the main output mixing and restore the necessary algorithms. This maintains the overall real-time stability during the overload period and improves the processing quality and meeting experience of newly added speaking channels in a timely manner when the conditions are met.

[0066] This application addresses the issue of limiting computing resources to the necessary processing of critical channels by shutting down or reducing high-load processing algorithms for the set of channels requiring degradation during the degradation phase, while maintaining a preset basic processing set for priority channels. This balances resource release with ensuring the core audio quality of the conference. Simultaneously, by jointly determining the recovery timing through exit thresholds and channel activity conditions, the application promptly restores the mixing participation and corresponding high-load algorithms of the target channel when the load decreases and the target channel becomes active again. This achieves closed-loop scheduling from degradation to switchback, thereby avoiding the loss of experience caused by long-term degradation and maintaining the continuity and adaptability of the conference process.

[0067] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0068] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0069] Further reference Figure 2 As a response to the above Figure 1 To implement the method shown, this application provides an embodiment of a resource optimization system for main processor decision-making and audio processor intra-processor scheduling. This system embodiment is similar to... Figure 1 Corresponding to the method embodiments shown, the system can be specifically applied to various electronic devices.

[0070] like Figure 2 As shown, the resource optimization system 200 for main processor decision-making and audio processor scheduling in this embodiment includes: an acquisition module 201, a determination module 202, a sending module 203, and an update module 204. Wherein:

[0071] The acquisition module 201 is used to acquire processor load information and channel characteristic information of each audio input channel when the audio processor performs audio processing on multiple audio input channels. The channel characteristic information is used to characterize the activity state or channel level of the channel at least.

[0072] The determining module 202 is used to determine the adjustment result of the audio processing configuration based on the processor load information and the channel feature information according to the preset resource optimization rules when the processor load information meets the preset load conditions. The adjustment result indicates at least one of the channel set that needs to be degraded and the channel set that needs to be prioritized.

[0073] The sending module 203 is used to generate and send control instructions to the audio processor based on the adjustment results by the main processor. The control instructions include at least mixing routing configuration and processing algorithm configuration.

[0074] The update module 204 is used by the audio processor to update the mixing routing relationship according to the control instructions in its internal scheduling, and to switch or adjust the processing links of each channel according to the processing algorithm configuration.

[0075] The resource optimization system for main processor decision-making and audio processor intra-scheduling provided in this embodiment of the invention can realize all the processes of the resource optimization method for main processor decision-making and audio processor intra-scheduling in the above embodiments. The functions and technical effects of each module in the device are the same as those of the resource optimization method for main processor decision-making and audio processor intra-scheduling in the above embodiments, and will not be repeated here.

[0076] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 3 , Figure 3 This is a basic structural block diagram of the computer device in this embodiment.

[0077] The computer device 3 includes a memory 31, a processor 32, and a network interface 33 that are interconnected via a system bus. It should be noted that only the computer device 3 with components 31-33 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0078] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0079] The memory 31 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 31 may be an internal storage unit of the computer device 3, such as the hard disk or memory of the computer device 3. In other embodiments, the memory 31 may also be an external storage device of the computer device 3, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 3. Of course, the memory 31 may also include both the internal storage unit and its external storage device of the computer device 3. In this embodiment, the memory 31 is typically used to store the operating system and various application software installed on the computer device 3, such as computer-readable instructions for resource optimization methods for main processor decision-making and audio processor scheduling. In addition, the memory 31 can also be used to temporarily store various types of data that have been output or will be output.

[0080] In some embodiments, the processor 32 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 32 is typically used to control the overall operation of the computer device 3. In this embodiment, the processor 32 is used to execute computer-readable instructions stored in the memory 31 or to process data, for example, to execute computer-readable instructions for resource optimization methods involving main processor decision-making and audio processor scheduling.

[0081] The network interface 33 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 3 and other electronic devices.

[0082] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the resource optimization method of main processor decision and audio processor scheduling as described above.

[0083] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0084] The above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for host processor decision and resource optimization within audio processor scheduling, comprising: Includes the following steps: ​ The processor load information and channel characteristic information of each audio input channel are obtained when the audio processor performs audio processing on multiple audio input channels. The channel characteristic information is used to characterize the activity state or channel level of the channel at least. When the processor load information meets the preset load conditions, the main processor determines the adjustment result of the audio processing configuration based on the processor load information and the channel feature information according to the preset resource optimization rules. The adjustment result indicates at least one of the channel set that needs to be degraded and the channel set that needs to be prioritized. The main processor generates and sends control commands to the audio processor based on the adjustment results. The control commands include at least mixing routing configuration and processing algorithm configuration. The mixing routing configuration includes mixing matrix parameters for updating mixing routing relationships; the processing algorithm configuration includes channel identification information and algorithm link parameters associated with the channel identification information, wherein the channel identification information is used to identify the set of channels that need to be degraded, and the algorithm link parameters are used to indicate whether to disable, enable, or adjust the processing intensity of the processing links for the set of channels that need to be degraded. The control commands are sent through a serial peripheral interface and include a serial number and an acknowledgment feedback mechanism to ensure that the mixing routing configuration and the processing algorithm configuration are consistent and effective on the audio processor side. The audio processor updates the mixing routing relationship according to the control instructions in its internal scheduling, and switches or adjusts the processing links of each channel according to the processing algorithm configuration. The step of updating the mixing routing relationship by the audio processor in its internal scheduling according to the control instructions includes: For the set of channels that need to be downgraded, cancel their mixing participation relationship to at least one target output, and for the set of channels that need to be prioritized, maintain their mixing participation relationship to the at least one target output. After canceling the mixing participation relationship, input gain suppression is performed on the channel whose mixing was canceled to make its input gain meet the preset suppression condition, so as to reduce the coupling interference to the at least one target output terminal; The step of switching or adjusting the processing link of each channel according to the processing algorithm configuration includes: For the set of channels that need to be downgraded, at least one type of high-load processing algorithm shall be turned off or reduced, and a preset basic processing set shall be maintained for the set of channels that need to be prioritized. Set an exit threshold for recovery determination, and when the processor load information is continuously not higher than the exit threshold for a preset duration, and there is a target channel in the set of channels that need to be downgraded that continuously meets the preset activity conditions for a preset duration, restore the mixing participation relationship from the target channel to the at least one target output, and restore the high load processing algorithm corresponding to the target channel as needed.

2. The method of claim 1, wherein, The step of the main processor determining the adjustment result of the audio processing configuration based on the processor load information and the channel characteristic information according to the preset resource optimization rules further includes: Obtain meeting status information and read the resource protection constraints corresponding to the meeting status information from the pre-stored scenario configuration; The resource protection constraints include at least channel priority rules and a basic processing set that must be retained when the load is limited, so that the adjustment results are generated under the constraints.

3. The method of claim 2, wherein, The step of the main processor determining the adjustment result of the audio processing configuration based on the processor load information and the channel characteristic information according to the preset resource optimization rules further includes: The activity status of each audio input channel is determined based on the channel feature information, and the set of channels that need to be prioritized and the set of channels that need to be downgraded are determined according to the channel priority rules. Channels with an inactive status and a priority no higher than a preset level are identified as the set of channels that need to be downgraded, while channels with an active status and a priority no lower than a preset level are identified as the set of channels that need to be prioritized.

4. The method of claim 1, wherein, The steps for determining whether the processor load information meets the preset load conditions include: Set entry and exit thresholds, and trigger the adjustment of the audio processing configuration only when the processor load information continuously meets the entry threshold for a preset duration; After triggering, restorative adjustments are allowed only when the processor load information continuously meets the exit threshold for a preset duration, in order to suppress repeated switching.

5. A system for host processor decision and resource optimization within audio processor scheduling, comprising: include: The acquisition module is used to acquire processor load information and channel characteristic information of each audio input channel when the audio processor performs audio processing on multiple audio input channels. The channel characteristic information is used to characterize the activity state or channel level of the channel at least. The determination module is used to determine the adjustment result of the audio processing configuration by the main processor based on the processor load information and the channel feature information according to the preset resource optimization rules when the processor load information meets the preset load conditions. The adjustment result indicates at least one of the set of channels that need to be degraded and the set of channels that need to be prioritized. A sending module is used to generate and send control commands to the audio processor based on the adjustment results by the main processor. The control commands include at least mixing routing configuration and processing algorithm configuration. The mixing routing configuration includes mixing matrix parameters for updating mixing routing relationships; the processing algorithm configuration includes channel identification information and algorithm link parameters associated with the channel identification information, wherein the channel identification information is used to identify the set of channels that need to be degraded, and the algorithm link parameters are used to indicate whether to disable, enable, or adjust the processing intensity of the processing links for the set of channels that need to be degraded. The control commands are sent through a serial peripheral interface and include a serial number and an acknowledgment feedback mechanism to ensure that the mixing routing configuration and the processing algorithm configuration are consistent and effective on the audio processor side. The update module is used by the audio processor to update the mixing routing relationship according to the control instructions in its internal scheduling, and to switch or adjust the processing links of each channel according to the processing algorithm configuration. The system is also used for: For the set of channels that need to be downgraded, cancel their mixing participation relationship to at least one target output, and for the set of channels that need to be prioritized, maintain their mixing participation relationship to the at least one target output. After canceling the mixing participation relationship, input gain suppression is performed on the channel whose mixing was canceled to make its input gain meet the preset suppression condition, so as to reduce the coupling interference to the at least one target output terminal; For the set of channels that need to be downgraded, at least one type of high-load processing algorithm shall be turned off or reduced, and a preset basic processing set shall be maintained for the set of channels that need to be prioritized. Set an exit threshold for recovery determination, and when the processor load information is continuously not higher than the exit threshold for a preset duration, and there is a target channel in the set of channels that need to be downgraded that continuously meets the preset activity conditions for a preset duration, restore the mixing participation relationship from the target channel to the at least one target output, and restore the high load processing algorithm corresponding to the target channel as needed.

6. A computer device, comprising: The system includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor, when executing the computer-readable instructions, implements the steps of the resource optimization method for main processor decision-making and audio processor scheduling as described in any one of claims 1 to 4.

7. A computer readable storage medium characterized by, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the resource optimization method for main processor decision-making and audio processor scheduling as described in any one of claims 1 to 4.

Images