A multi-device management method and system for an audio conference system based on a BS architecture

Abstract

The application provides a multi-device management method and system for an audio conference system based on a B / S architecture, and the application builds a distributed cluster B / S service architecture including a user terminal, an external bottom-layer audio device and a plurality of control hosts as cluster nodes, collects corresponding data in real time through the cluster nodes, feeds back the data to the user terminal, controls the corresponding external bottom-layer audio device through a control instruction issued by the user terminal, triggers a hardware interrupt when a new external bottom-layer audio device is connected to a POE++ interface of the control host, powers on and registers the new external bottom-layer audio device, performs audio channel distribution and fragmented multicast stream subscription, releases a speech request signal or puts the speech request signal into a queuing queue, realizes isolated transmission of mixed audio data stream data fragments through a UDP protocol, and performs echo cancellation and noise reduction on the audio. The application can realize management and control of multiple devices, ensure stable operation of multiple devices, suppress echo howling, and improve audio quality.

Description

Technical Field

[0001] This invention relates to the field of audio conferencing communication technology, and in particular to a multi-device management method and system for audio conferencing systems based on a B / S architecture. Background Technology

[0002] With the rapid popularization of remote collaboration and online conferencing scenarios, audio conferencing systems, as core tools for cross-regional communication, have undergone multiple generations of evolution in their technical architecture and application models, with continuously increasing requirements for device access capabilities, compatibility, and stability. Early audio conferencing systems mainly relied on dedicated hardware line equipment, such as dedicated conferencing terminals based on ISDN lines. These systems depended on dedicated hardware and fixed line deployment, resulting in high costs, poor scalability, and the ability to support only a small number of fixed devices. They also could not achieve flexible cross-regional meetings, limiting their application scenarios to fixed settings such as enterprise meeting rooms, and failing to meet the needs of mobile office and multi-terminal collaboration.

[0003] With the popularization of internet technology, software-based audio conferencing systems based on the C / S architecture have gradually become mainstream. These systems require users to install client software, supporting more devices and cross-regional meetings. However, client installation, upgrades, and maintenance costs are high, and they are limited by operating system compatibility, making them difficult to adapt to complex scenarios such as mobile devices and public terminals, and unable to achieve seamless switching between multiple devices. In recent years, to solve the compatibility issues of the C / S architecture, audio conferencing systems based on the B / S architecture have developed rapidly. Users only need a browser to join the meeting, without installing a client, significantly lowering the barrier to entry and leading to their widespread application in online education, remote work, and government meetings. However, with the increasing complexity of meeting scenarios and the surge in demand for simultaneous access from the same user or multiple devices in the same meeting, the shortcomings of existing B / S architecture systems in multi-device management capabilities are becoming increasingly apparent.

[0004] 1. Disorganized device status management: Most existing solutions only support single device access or simple multiple devices online at the same time. They lack a unified management mechanism for the access status, audio stream, and permissions of multiple devices, which can easily lead to problems such as echo, howling, and volume conflicts caused by multiple devices accessing at the same time, seriously affecting the audio quality of the meeting.

[0005] 2. Cumbersome configuration and lack of plug-and-play capability: Most existing solutions cannot achieve plug-and-play functionality for MIC (audio source) and speaker (loudspeaker). Users need to manually complete the audio routing configuration, which is complicated and has a high barrier to entry, seriously affecting the user experience.

[0006] 3. Insufficient system capacity, making it difficult to support the needs of large-scale conferences: Existing systems typically only support a few to a dozen microphones, and are mostly single solutions that are purely wired or purely wireless, which cannot meet the needs of modern large-scale conference systems for massive device access and complex sound reinforcement. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a multi-device management method and system for audio conferencing systems based on a B / S architecture, in order to solve the aforementioned technical problems.

[0008] In a first aspect, the present invention provides a multi-device management method for an audio conferencing system based on a B / S architecture, comprising:

[0009] A distributed cluster B / S service architecture is established, including user terminals, external underlying audio devices, and multiple control hosts as cluster nodes. Each control host has a built-in WEB server, which generates a standardized device management webpage. The user terminal accesses the WEB server through a browser.

[0010] The cluster nodes collect real-time hardware computing load data of the control host, data of external underlying audio devices, and status data of underlying audio devices, and push them to the user terminal through the WEB server;

[0011] The control host receives control commands sent by the user through the browser of the user terminal, parses the control commands and verifies their legality, operation permissions and partition ownership. After verification, the control commands are routed and distributed to the corresponding external underlying audio devices according to the device type.

[0012] The new external audio device is connected to the control host via the PoE++ interface, triggering a hardware interrupt. After the reserved margin verification is passed, the new external audio device is powered on and registered, and audio channel allocation and segmented multicast stream subscription are performed.

[0013] It collects all microphone speaking request signals in real time, and allows or puts the current speaking request into the chairman mode queue according to the speaking threshold, and monitors in real time whether there are dual-end audio loops under the same account;

[0014] Based on the UDP protocol, the global mixing data stream is logically segmented and isolated for transmission according to the venue partition. When bandwidth is congested, the audio sampling rate of non-critical peripherals is automatically reduced.

[0015] The predicted echo is calculated based on the speaker playback audio as a reference, and the clean near-end audio after echo cancellation is calculated based on the original near-end audio of the microphone and the predicted echo. Then, the clean near-end audio after echo cancellation is denoised based on the noise suppression mask coefficient Mask to obtain the denoised output spectrum.

[0016] Secondly, the present invention provides a multi-device management system for an audio conferencing system based on a B / S architecture, comprising:

[0017] The B / S service architecture module is used to build a distributed cluster B / S service architecture that includes user terminals, external underlying audio devices, and multiple control hosts as cluster nodes. Each control host has a built-in WEB server, which generates a standardized device management webpage. The user terminal accesses the WEB server through a browser.

[0018] The data acquisition module is used to collect hardware computing load data of the control host, data of external underlying audio devices, and status data of underlying audio devices in real time through the cluster nodes, and push them to the user terminal through the WEB server.

[0019] The instruction parsing module is used to receive control instructions issued by the user through the browser of the user terminal, and to parse the control instructions through the control host and verify the legality, operation permissions and partition ownership of the control instructions. After verification, the control instructions are routed and distributed to the corresponding external underlying audio devices according to the device type.

[0020] The new device management module is used to trigger a hardware interrupt when a new external underlying audio device is connected to the control host via the PoE++ interface. After passing the reserved margin verification, the new external underlying audio device is powered on and registered, and audio channel allocation and segmented multicast stream subscription are performed.

[0021] The concurrent conflict suppression module is used to collect all microphone speaking request signals in real time, and to allow the current speaking request or put it into the chairman mode queue according to the speaking threshold, and to monitor in real time whether there are dual-end audio loops under the same account.

[0022] The audio segmentation and scheduling module is used to logically segment and isolate the global mixing data stream according to the venue partition based on the UDP protocol. When bandwidth is congested, it automatically reduces the audio sampling rate of non-critical peripherals.

[0023] The audio processing module is used to calculate the predicted echo based on the speaker playback audio as a reference, and to calculate the clean near-end audio after echo cancellation based on the original near-end audio of the microphone and the predicted echo; and to perform noise reduction processing on the clean near-end audio after echo cancellation based on the noise suppression mask coefficient Mask to obtain the noise-reduced output spectrum.

[0024] Thirdly, the present invention provides an electronic device, comprising:

[0025] At least one processor;

[0026] and a memory communicatively connected to the at least one processor;

[0027] The memory stores a computer program that can be executed by at least one processor. When the computer program is executed by at least one processor, it implements the multi-device management method of the audio conferencing system based on the BS architecture.

[0028] The beneficial effects of this invention are as follows:

[0029] 1. This invention constructs a distributed cluster B / S service architecture consisting of user terminals, external underlying audio devices, and multiple control hosts as cluster nodes. It also sets up a three-layer isolated network and a unified floating IP service gateway. Users can access the WEB server of the control host through a browser to control and manage the external underlying audio devices. This invention can stably connect 199 wired microphones, 199 wireless microphones, and 199 speakers simultaneously, and a single cluster can support a hybrid network of nearly 600 peripheral devices.

[0030] 2. Before the device is inserted, the total power consumption of the whole machine is automatically checked. When the power margin is insufficient, the power reserve of idle peripheral devices is dynamically reduced to avoid the machine crashing when plugging and unplugging. The instantaneous howling suppression rate of the device during plugging and unplugging is increased to more than 98%. After the new device is connected, it automatically completes identity registration, cluster partition binding, audio channel allocation, and partition multicast stream subscription. There is no need to manually configure the audio input and output routing, which further improves the connection efficiency of the new external device.

[0031] 3. This invention controls external devices through a user terminal, and all front-end control commands undergo a triple serial verification of legality, operation permissions, and partition affiliation. It can also issue independent power supply, speaking permissions, and audio parameter policies for different meeting rooms.

[0032] 4. This invention performs hardware muting on microphones exceeding the threshold and constructs a FIFO queuing queue, and dynamically rotates speaking channels according to priority; and identifies in real time scenarios where microphone input and browser speaker output coexist under the same account, automatically muting the mobile device's audio output;

[0033] 5. This invention allocates independent UDP multicast addresses according to the venue to achieve logical isolation of audio between sections. Audio data packets are fixedly fragmented and embedded with microsecond-level timestamps. Multi-threaded UDP whole packet parallel parsing and processing is adopted to solve the problem of device congestion due to concurrency.

[0034] 6. This invention can accurately suppress nonlinear echoes from high-power loudspeakers, and the noise reduction mask distinguishes human voices from noise on a frequency-by-frequency basis; the noise reduction and echo cancellation intensity are adaptively adjusted according to the concurrent microphone and loudspeaker power. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating an embodiment of the present invention;

[0036] Figure 2This is a schematic diagram of the distributed cluster B / S service architecture according to an embodiment of the present invention;

[0037] Figure 3 This is a hardware framework diagram of the distributed cluster B / S service architecture according to an embodiment of the present invention;

[0038] Figure 4 This is a schematic diagram illustrating the process of connecting a new external low-level audio device to the control host according to an embodiment of the present invention;

[0039] Figure 5 This is a schematic diagram of the system structure according to an embodiment of the present invention.

[0040] In the diagram, 10 is the B / S service architecture module; 20 is the data acquisition module; 30 is the instruction parsing module; 40 is the new device management module; 50 is the concurrency conflict suppression module; 60 is the audio segmentation and scheduling module; and 70 is the audio processing module. Detailed Implementation

[0041] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings:

[0042] like Figure 1 As shown, this embodiment provides a multi-device management method for an audio conferencing system based on a B / S architecture, including:

[0043] S1: Build a distributed cluster B / S service architecture including user terminals, external underlying audio devices, and multiple control hosts as cluster nodes. Each control host has a built-in WEB server, which generates a standardized device management webpage. The user terminal accesses the WEB server through a browser.

[0044] S2: The cluster nodes collect real-time hardware computing load data of the control host, data of external underlying audio devices, and status data of underlying audio devices, and push them to the user terminal through the WEB server;

[0045] S3: Receive control commands sent by the user through the browser of the user terminal. The control host parses the control commands and verifies their legality, operation permissions, and partition ownership. After verification, the control commands are routed and distributed to the corresponding external underlying audio devices according to the device type.

[0046] S4: When a new external low-level audio device is connected to the control host POE++ interface, a hardware interrupt is triggered. After the reserved margin verification is passed, the new external low-level audio device is powered on and registered, and audio channel allocation and segmented multicast stream subscription are performed.

[0047] S5: Real-time acquisition of all microphone speaking request signals, and based on the speaking threshold, allows the current speaking request to be allowed or put into the chairman mode queue, and monitors in real time whether there are dual-end audio loops under the same account;

[0048] S6: Based on the UDP protocol, the global mixing data stream is logically segmented and isolated for transmission according to the venue partition. When bandwidth is congested, the audio sampling rate of non-critical peripherals is automatically reduced.

[0049] S7: Calculate the predicted echo based on the speaker playback audio as a reference, and calculate the clean near-end audio after echo cancellation based on the original near-end audio of the microphone and the predicted echo; and perform noise reduction processing on the clean near-end audio after echo cancellation based on the noise suppression mask coefficient Mask to obtain the noise-reduced output spectrum.

[0050] In this embodiment, step S1 involves establishing a distributed cluster B / S service architecture including a user terminal, external underlying audio devices, and multiple control hosts as cluster nodes. Each control host has a built-in web server that generates a standardized device management webpage. The user terminal accesses the web server through a browser, such as... Figure 2 , 3 As shown, it includes:

[0051] S11: Deploy multiple control hosts as cluster nodes, and the distributed cluster B / S service architecture is set up with a three-layer isolation network: an external underlying audio device access network, a dedicated internal network for cluster nodes, and an external network for user access.

[0052] S12: The cluster nodes negotiate the cluster topology through a TCP long connection handshake and automatically elect a cluster master node. The cluster master node maintains the global registry of the underlying audio devices.

[0053] S13: Each of the aforementioned control hosts is configured with an independent web server, which provides static pages to users based on Nginx lightweight HTTP or HTTPS service, and distributes static resource requests to all cluster nodes in a round-robin manner through a unified gateway to distribute the static resource access pressure; and allocates real-time status push requests of external underlying audio devices to a single cluster node based on the user terminal session ID hash.

[0054] S14: The cluster master node allocates underlying audio device slices based on the real-time computing power of each cluster slave node, the remaining power supply capacity of the PoE port, the network bandwidth usage, and the number of external devices already supported.

[0055] When a cluster slave node is detected as a faulty node, the cluster master node will migrate the external underlying audio devices of the faulty node to the normal cluster slave node with the lowest load within the cluster.

[0056] When a new external audio device is connected to any cluster node, the cluster master node obtains the device registration information of the new external audio device, retrieves the normal cluster slave node with the lowest current load, issues a sharding assignment instruction, and assigns the new external audio device to the corresponding cluster slave node for management.

[0057] In this embodiment, in step S2, the cluster node collects real-time data on the hardware computing power load of the control host, data from external underlying audio devices, and status data of the underlying audio devices, and pushes this data to the user terminal via a web server, including:

[0058] S21: The control host collects hardware computing power load data, external low-level audio device data, and low-level audio device status data through three independent threads;

[0059] The data collection of the control host hardware computing load is as follows:

[0060] Read the real-time CPU utilization, remaining memory capacity, and cluster node heartbeat status of the control host to obtain the computing power load index of the control host.

[0061] Poll the POE++ hardware registers of the control host to read the output voltage, output current and instantaneous output power of each port in real time, and generate a POE port power load array for the whole machine;

[0062] Obtain the hardware status of the analog audio interface.

[0063] The acquisition of data from the external underlying audio device includes:

[0064] Obtain all wired and wireless microphones under the segmented management of the control host, read the real-time sampling amplitude of the audio input buffer, and calculate the microphone input volume;

[0065] Iterate through all wired speakers, read the audio output buffer data and the power amplifier output register value, and calculate the real-time output power of the speakers;

[0066] The current ambient noise intensity value and echo path residual suppression value are obtained in real time and used as real-time noise and echo intensity audio indicators.

[0067] Acquire the percentage of effective human voice signal from concurrent microphones and auxiliary audio parameters such as the distortion of multi-channel voice superposition.

[0068] The underlying audio device status data includes the device's offline or online status, wired or wireless connection type, speaking or muting status, and PoE power supply status.

[0069] S22: The control host hardware computing power load data, external underlying audio device data, and underlying audio device status data are encapsulated and pushed to the user terminal through the WEB server.

[0070] In this embodiment, step S3 involves receiving control commands issued by the user through the browser on the user terminal, parsing the control commands through the control host, and verifying their legality, operation permissions, and partition affiliation. Once verified, the control commands are routed and distributed to the corresponding external underlying audio devices according to device type. This includes:

[0071] S31: When a user performs single-device control or partition batch operation in the browser on the user terminal, a standardized JSON instruction package carrying the account token, cluster partition ID, target device set, operation type, and parameters is generated and uploaded to the unified gateway via WebSocket or HTTP interface;

[0072] S32: The unified gateway verifies the validity of the session token, filters out illegal parameters that exceed the hardware threshold, and forwards the real-time lightweight control command to the slave node of the sharded cluster to which the device belongs;

[0073] S33: The cluster slave nodes perform legality, operation permission, and partition ownership verification on real-time lightweight control commands; after verification, the command parsing service distributes the commands to the corresponding hardware control units according to the managed objects, including:

[0074] Distribute power to multiple analog interfaces to achieve gain and phantom power control;

[0075] Distribute power to the PoE++ interface to achieve single-channel or zoned batch power supply switch control;

[0076] Distribute data to the audio control unit to enable / disable AI-ANC and AI-AEC algorithms and adjust parameters;

[0077] The audio dispatch and peripheral detection unit is distributed to wired conference microphones, wireless conference microphones, wireless conference hosts, and wired speakers to perform connection removal, speaking permissions, mute, forced offline, and factory reset.

[0078] S34: The hardware control unit calls the hardware unified abstraction API adaptation layer to convert the standardized upper-layer instructions into the underlying hardware protocols of POE port I2C, wired MIC RS485, wireless host private UDP, and audio codec register, and sends them down to the corresponding external underlying audio devices to complete the hardware actions.

[0079] S35: After the external low-level audio device completes the hardware action, it feeds back the execution status and sends it back to the instruction parsing service level by level. The execution result is pushed to the user terminal's browser via WebSocket through the unified gateway.

[0080] In this embodiment, in step S4, the new external underlying audio device triggers a hardware interrupt upon accessing the control host's PoE++ interface. After the reserved margin verification passes, the new external underlying audio device is powered on and registered, and audio channel allocation and segmented multicast stream subscription are performed. Figure 4 As shown, it includes:

[0081] S41: When a new external audio device is connected to the PoE++ interface of the control host, a hardware interrupt is triggered. The real-time total power consumption of the control host is calculated. If the addition of the device causes the real-time total power consumption of the control host to exceed the rated power limit of the current PoE host, the power reserve of the currently idle peripherals is reduced. If there is no reserve in all PoE++ interfaces of the current control host, a power overload alarm is pushed to the user terminal. After verification, the new external audio device is powered on in a time-sharing and step-by-step manner.

[0082] S42: After the new external low-level audio device is powered on, set a certain hardware stabilization delay and monitor the power supply voltage of the currently connected POE++ port. If an abnormal power supply voltage is detected, power off and report a hardware fault.

[0083] S43: After the hardware stabilization delay ends, the peripheral detection unit shakes hands with the peripheral to read the unique device ID, hardware model, and device type of the newly connected external underlying audio device, automatically matches and binds the partition ID and cluster shard identifier, writes it to the local shard registry and synchronizes it to the global registry of the underlying audio device of the cluster master node to complete the device registration.

[0084] S44: After completing device registration, allocate an independent audio channel to the new external low-level audio device and automatically subscribe to the microphone voice stream. Aggregate all microphone audio mixes in the partition to generate a partition-specific segmented multicast stream. Send instructions to control all wired speakers in the partition to automatically subscribe to the segmented multicast stream.

[0085] In this embodiment, if the user chooses to speak via a wired microphone, the control host automatically subscribes to the audio stream of the wired microphone. If the user chooses to speak via a wireless microphone, the wireless host allocates a channel for the corresponding wireless microphone and pushes the received wireless audio data to the control host via the audio stream.

[0086] S45: The user terminal issues a speaking command, verifies speaking permissions, enables microphone mixing weights, and establishes a complete sound reinforcement path from the microphone to the zone speaker.

[0087] S46: When a new external low-level audio device is connected and disconnected, the microphone audio subscription is cut off, audio data is removed from the mixing channel, and instantaneous noise from the speaker is blocked; after waiting 3 seconds for the audio buffer to be emptied, the power supply to the corresponding PoE++ port is turned off, the device offline status is updated, and the cluster registry is synchronized.

[0088] In this embodiment, step S5 involves real-time acquisition of all microphone speaking request signals, and allowing or queuing current speaking requests according to speaking thresholds, while also real-time monitoring of whether dual-ended audio loops exist under the same account, including:

[0089] S51: Set up the chairman microphone. The chairman has continuous speaking privileges. Collect all microphone speaking request signals in real time, count the number of microphones currently speaking, and if the current online speaking requests are less than the speaking threshold, allow all microphone audio streams to enter the partitioned mixing multicast and automatically balance the voice gain of multiple microphones.

[0090] If the current online speaking requests are greater than or equal to the speaking threshold, the excess microphones will enter the first-in-first-out queuing queue and be temporarily muted by hardware. The queuing microphones will be dynamically scheduled to speak in turn based on the speaking duration and volume of the voice.

[0091] When the original microphone releases its speaking privileges, the queued microphone is automatically retrieved from the head of the queue, unmuted, and added to the concurrent speaking channel.

[0092] S52: Real-time monitoring of whether a microphone audio input device and a terminal audio output device exist simultaneously under the same account; if a dual-end audio loop is determined, a mute command is sent to the unified gateway to temporarily mute the audio output of all browsers for the corresponding account.

[0093] In this embodiment, step S6 involves logically segmenting and isolating the global mixing data stream according to the venue partition based on the UDP protocol, and automatically reducing the audio sampling rate of non-critical peripherals when bandwidth is congested, including:

[0094] S61: Based on the cluster partition ID, assign an independent UDP multicast address and UDP multicast port to each venue, and encapsulate the global mixed audio data stream after mixing multiple microphones in a single cluster partition with fixed-length UDP fragments. The UDP fragment header embeds the cluster partition ID, UDP fragment sequence number, device priority, and microsecond timestamp to distinguish the priorities of three types of devices: primary speaking microphones, secondary queuing microphones, and tertiary idle peripherals.

[0095] S62: Collect all wired and wireless microphone audio streams in this cluster partition according to the cluster partition to complete the mixing, cut them into independent UDP fragment data packets according to the preset fragment length, and send the UDP fragment data packets to the UDP multicast address of this cluster partition. All speaker hardware in this cluster partition automatically subscribes to the UDP multicast port corresponding to this cluster partition.

[0096] S63: Real-time statistics on UDP fragmented data packet sending rate, data packet queuing buffer length, and fragmented packet loss rate; when bandwidth usage exceeds the warning threshold, automatically reduce the audio sampling rate of non-critical peripherals.

[0097] In this embodiment, each of the wired and wireless microphones is allocated 199 independent device address ranges. Based on the 8-channel PoE++ interface of the control host and the expansion of the external PoE expansion switch, 199 wired and 199 wireless microphones can be online simultaneously. Multi-threaded UDP whole packet parallel parsing and processing of peripheral audio data is adopted, and 24 high-priority independent audio buffers are reserved to carry concurrent microphones, supporting 24 microphones to speak synchronously without blocking.

[0098] In this embodiment, in step S7, the predicted echo is calculated based on the speaker playback audio as a reference, and the clean near-end audio after echo cancellation is calculated based on the original near-end audio from the microphone and the predicted echo; and the clean near-end audio after echo cancellation is denoised based on the noise suppression mask coefficient Mask to obtain the denoised output spectrum, including:

[0099] S71: Parallel reception of two data streams: raw audio from the microphone near the microphone and playback audio from the speaker, with frame buffering;

[0100] S72: Calculates and predicts echoes using the speaker's playback audio as a reference. And based on the raw audio from the near end of the microphone and the predicted echo Calculate the clean near-end audio after echo cancellation Based on the pure near-end audio after echo cancellation Update the path weight coefficients, that is:

[0101] ;

[0102] ;

[0103] ;

[0104] In the formula, This represents the total order of the filter. For the first Real-time echo path weighting coefficient; To play audio back through the speakers; This is the raw audio from the near end of the microphone; The index of the discrete audio time-domain sampling point; The basic iteration step size; These are nonlinear correction coefficients; Based on speaker output power The nonlinear fitting function;

[0105] S73: Pure near-end audio after echo cancellation The frequency domain spectrum is obtained by performing a short-time Fourier transform. And based on frequency domain spectrum Calculate the short-time energy of a single frame Based on single-frame short-time energy Voice baseband Human voice features are obtained by harmonic correlation Corr weighting. Based on the average energy of noisy frames Calculate noise characteristics Based on human voice characteristics and noise characteristics Calculate the noise suppression mask coefficient, i.e.:

[0106] ;

[0107] ;

[0108] ;

[0109] ;

[0110] In the formula, It is the discrete frequency domain angular frequency; , Preset weighting coefficients for human voice features; , These are the weighting coefficients for the noise branch; The noise distortion coefficient;

[0111] S74: Clean near-end audio after echo cancellation based on noise suppression mask coefficients (Mask). Noise reduction processing is performed to obtain the noise-reduced output spectrum. ;

[0112] ;

[0113] S75: Clean near-end audio after echo cancellation and the output spectrum of noise reduction Perform temporal smoothing filtering and optimize the voice audio backhaul for use in zone mixing and multicast.

[0114] Benru Figure 5 As shown, another embodiment of this application provides a multi-device management system for an audio conferencing system based on a B / S architecture, including:

[0115] B / S service architecture module 10 is used to build a distributed cluster B / S service architecture including user terminals, external underlying audio devices and multiple control hosts as cluster nodes. Each control host has a built-in WEB server, which generates a standardized device management webpage. The user terminal accesses the WEB server through a browser.

[0116] The data acquisition module 20 is used to collect hardware computing load data of the control host, data of external underlying audio devices, and status data of underlying audio devices in real time through the cluster nodes, and push them to the user terminal through the WEB server.

[0117] The instruction parsing module 30 is used to receive control instructions issued by the user through the browser of the user terminal, and to parse the control instructions through the control host and verify the legality, operation permissions and partition ownership of the control instructions. After the verification is passed, the control instructions are routed and distributed to the corresponding external underlying audio devices according to the device type.

[0118] The new device management module 40 is used to trigger a hardware interrupt on the POE++ interface of the control host when a new external underlying audio device is connected. After the reserved margin verification is passed, the new external underlying audio device is powered on and registered, and audio channel allocation and segmented multicast stream subscription are performed.

[0119] The concurrent conflict suppression module 50 is used to collect all microphone speaking request signals in real time, and to allow the current speaking request or put it into the chairman mode queue according to the speaking threshold, and to monitor in real time whether there are dual-end audio loops under the same account.

[0120] The audio segmentation scheduling module 60 is used to logically segment and isolate the global mixing data stream according to the venue partition based on the UDP protocol, and automatically reduce the audio sampling rate of non-critical peripherals when bandwidth is congested.

[0121] The audio processing module 70 is used to calculate the predicted echo based on the speaker playback audio as a reference, and to calculate the clean near-end audio after echo cancellation based on the original near-end audio of the microphone and the predicted echo; and to perform noise reduction processing on the clean near-end audio after echo cancellation based on the noise suppression mask coefficient Mask to obtain the noise-reduced output spectrum.

[0122] In this embodiment, the B / S service architecture module 10 establishes a distributed cluster B / S service architecture including a user terminal, external underlying audio devices, and multiple control hosts as cluster nodes. Each control host has a built-in web server, which generates a standardized device management webpage. The user terminal accesses the web server through a browser, including:

[0123] Multiple control hosts are deployed as cluster nodes, and the distributed cluster B / S service architecture is configured with a three-layer isolation network: an external underlying audio device access network, a dedicated internal network for cluster nodes, and an external network for user access.

[0124] The cluster nodes negotiate the cluster topology through a TCP long-connection handshake and automatically elect a cluster master node. The cluster master node maintains the global registry of the underlying audio devices.

[0125] Each of the aforementioned control hosts is configured with an independent web server, providing static pages to users based on Nginx lightweight HTTP or HTTPS services. Static resource requests are distributed to all cluster nodes in a round-robin fashion through a unified gateway to distribute the static resource access pressure. Furthermore, based on the user terminal session ID hash, real-time status push requests from external underlying audio devices are allocated to a single cluster node.

[0126] The cluster master node allocates underlying audio device fragments based on the real-time computing power of each cluster slave node, the remaining power supply capacity of the PoE port, the network bandwidth usage, and the number of external devices already connected.

[0127] When a cluster slave node is detected as a faulty node, the cluster master node will migrate the external underlying audio devices of the faulty node to the normal cluster slave node with the lowest load within the cluster.

[0128] When a new external audio device is connected to any cluster node, the cluster master node obtains the device registration information of the new external audio device, retrieves the normal cluster slave node with the lowest current load, issues a sharding assignment instruction, and assigns the new external audio device to the corresponding cluster slave node for management.

[0129] In this embodiment, the data acquisition module 20 collects real-time data on the control host hardware computing power load, external underlying audio device data, and underlying audio device status data through cluster nodes, and pushes this data to the user terminal via a web server, including:

[0130] The control host collects hardware computing power load data, external low-level audio device data, and low-level audio device status data through three independent threads;

[0131] The data collection of the control host hardware computing load is as follows:

[0132] Read the real-time CPU utilization rate, remaining memory capacity, and cluster node heartbeat status of the control host to obtain the computing power load index of the control host.

[0133] Poll the POE++ hardware registers of the control host to read the output voltage, output current and instantaneous output power of each port in real time, and generate a POE port power load array for the whole machine;

[0134] Obtain the hardware status of the analog audio interface.

[0135] The acquisition of data from the external underlying audio device includes:

[0136] Obtain all wired and wireless microphones under the segmented management of the control host, read the real-time sampling amplitude of the audio input buffer, and calculate the microphone input volume;

[0137] Iterate through all wired speakers, read the audio output buffer data and the power amplifier output register value, and calculate the real-time output power of the speakers;

[0138] The current ambient noise intensity value and echo path residual suppression value are obtained in real time and used as real-time noise and echo intensity audio indicators.

[0139] Acquire the percentage of effective human voice signal from concurrent microphones and auxiliary audio parameters such as the distortion of multi-channel voice superposition.

[0140] The underlying audio device status data includes the device's offline or online status, wired or wireless connection type, speaking or muting status, and PoE power supply status.

[0141] The control host hardware computing load data, external underlying audio device data, and underlying audio device status data are encapsulated and pushed to the user terminal through a web server.

[0142] In this embodiment, the instruction parsing module 30 receives control instructions issued by the user through the browser of the user terminal, parses the control instructions through the control host, and verifies the legality, operation permissions, and partition affiliation of the control instructions. After verification, the control instructions are routed and distributed to the corresponding external underlying audio devices according to device type, including:

[0143] When a user performs single-device control or batch operations on a partition in the browser on the user's terminal, a standardized JSON instruction package is generated, carrying the account token, cluster partition ID, target device set, operation type, and parameters, and uploaded to the unified gateway via WebSocket or HTTP interface;

[0144] The unified gateway verifies the validity of the session token, filters out illegal parameters that exceed the hardware threshold, and forwards real-time lightweight control commands to the slave nodes of the sharded cluster to which the device belongs.

[0145] The cluster slave nodes perform legality, operation permission, and partition ownership verification on real-time lightweight control commands; after successful verification, the command parsing service distributes the commands to the corresponding hardware control units according to the managed objects, including:

[0146] Distribute power to multiple analog interfaces to achieve gain and phantom power control;

[0147] Distribute power to the PoE++ interface to achieve single-channel or zoned batch power supply switch control;

[0148] Distribute data to the audio control unit to enable / disable AI-ANC and AI-AEC algorithms and adjust parameters;

[0149] The audio dispatch and peripheral detection unit is distributed to wired conference microphones, wireless conference microphones, wireless conference hosts, and wired speakers to perform connection removal, speaking permissions, mute, forced offline, and factory reset.

[0150] The hardware control unit calls the hardware unified abstraction API adaptation layer to convert standardized upper-layer instructions into the underlying hardware protocols of POE port I2C, wired MIC RS485, wireless host private UDP, and audio codec register, and sends them down to the corresponding external underlying audio devices to complete the hardware actions.

[0151] After the external low-level audio device completes the hardware action, it reports the execution status and sends it back to the instruction parsing service level by level. The execution result is pushed to the user terminal's browser via WebSocket through a unified gateway.

[0152] In this embodiment, the new device management module 40 triggers a hardware interrupt via the PoE++ interface of the control host when a new external underlying audio device is connected. After the reserved margin verification passes, the module powers on and registers the new external underlying audio device, and performs audio channel allocation and segmented multicast stream subscription, including:

[0153] When a new external audio device is connected to the PoE++ interface of the control host, a hardware interrupt is triggered. The real-time total power consumption of the control host is counted. If the addition of the device causes the real-time total power consumption of the control host to exceed the rated power limit of the current PoE host, the power reserve of the currently idle peripherals is reduced. If there is no reserve in all PoE++ interfaces of the current control host, a power overload alarm is pushed to the user terminal. After verification, the new external audio device is powered on in a time-sharing and step-by-step manner.

[0154] Once the new external audio device is powered on, a certain hardware stabilization delay is set, and the power supply voltage of the currently connected PoE++ port is monitored. If an abnormal power supply voltage is detected, the power is cut off and a hardware fault is reported.

[0155] After the hardware stabilization delay ends, the peripheral detection unit shakes hands with the peripheral to read the unique device ID, hardware model, and device type of the newly connected external underlying audio device, automatically matches and binds the partition ID and cluster shard identifier, writes it to the local shard registry and synchronizes it to the global registry of the underlying audio device on the cluster master node to complete the device registration.

[0156] After completing device registration, an independent audio channel is allocated to the newly connected external audio device and the microphone voice stream is automatically subscribed to. All microphone audio in the partition is aggregated and mixed to generate a partition-specific segmented multicast stream. Commands are issued to control all wired speakers in the partition to automatically subscribe to the segmented multicast stream.

[0157] The user terminal issues a speaking command, verifies speaking permissions, enables microphone mixing weight, and establishes a complete sound amplification path from microphone to zone speaker.

[0158] When a new external audio device is connected and disconnected, the microphone audio subscription is cut off, audio data is removed from the mixing channel, and instantaneous speaker noise is blocked. After waiting 3 seconds for the audio buffer to empty, the power supply to the corresponding PoE++ port is turned off, the device offline status is updated, and the cluster registry is synchronized.

[0159] In this embodiment, the concurrent conflict suppression module 50 collects all microphone speaking request signals in real time, and allows or puts the current speaking request into the chairman mode queue according to the speaking threshold, and monitors in real time whether there are dual-end audio loops under the same account, including:

[0160] The chairman microphone is set up, and the chairman has continuous speaking privileges. It collects all microphone speaking request signals in real time, counts the number of microphones currently speaking, and allows all microphone audio streams to enter the partitioned mixing multicast if the current online speaking requests are less than the speaking threshold. It also automatically balances the voice gain of multiple microphones.

[0161] If the current online speaking requests are greater than or equal to the speaking threshold, the excess microphones will enter the first-in-first-out queuing queue and be temporarily muted by hardware. The queuing microphones will be dynamically scheduled to speak in turn based on the speaking duration and volume of the voice.

[0162] When the original microphone releases its speaking privileges, the queued microphone is automatically retrieved from the head of the queue, unmuted, and added to the concurrent speaking channel.

[0163] Real-time monitoring to see if a microphone audio input device and a terminal audio output device exist simultaneously under the same account; if a dual-end audio loop is detected, a mute command is sent to the unified gateway to temporarily mute the audio output of all browsers for the corresponding account.

[0164] In this embodiment, the audio segmentation scheduling module 60 performs logical segmentation and isolation transmission of the global mixing data stream according to the venue partition based on the UDP protocol. When bandwidth is congested, it automatically reduces the audio sampling rate of non-critical peripherals, including:

[0165] Based on the cluster partition ID, each venue is assigned an independent UDP multicast address and UDP multicast port. The global mixed audio data stream after mixing multiple microphones in a single cluster partition is encapsulated in fixed-length UDP fragments. The UDP fragment header embeds the cluster partition ID, UDP fragment sequence number, device priority, and microsecond timestamp to distinguish the priority of three types of devices: primary speaking microphones, secondary queuing microphones, and tertiary idle peripherals.

[0166] The audio streams from all wired and wireless microphones in the cluster partition are collected and mixed according to the cluster partition. The data is then cut into independent UDP fragments according to the preset fragment length and sent to the UDP multicast address of the cluster partition. All speaker hardware in the cluster partition automatically subscribes to the corresponding UDP multicast port of the cluster partition.

[0167] Real-time statistics on UDP fragmented data packet sending rate, data packet queuing buffer length, and fragmented packet loss rate; when bandwidth usage exceeds the warning threshold, automatically reduce the audio sampling rate of non-critical peripherals.

[0168] In this embodiment, each of the wired and wireless microphones is allocated 199 independent device address ranges. Based on the 8-channel PoE++ interface of the control host and the expansion of the external PoE expansion switch, 199 wired and 199 wireless microphones can be online simultaneously. Multi-threaded UDP whole packet parallel parsing and processing of peripheral audio data is adopted, and 24 high-priority independent audio buffers are reserved to carry concurrent microphones, supporting 24 microphones to speak synchronously without blocking.

[0169] In this embodiment, the audio processing module 70 calculates the predicted echo based on the speaker playback audio as a reference, and calculates the clean near-end audio after echo cancellation based on the original near-end audio from the microphone and the predicted echo; and performs noise reduction processing on the clean near-end audio after echo cancellation based on the noise suppression mask coefficient Mask to obtain the noise-reduced output spectrum, including:

[0170] The system receives two data streams in parallel: the raw audio from the microphone and the playback audio from the speaker, and buffers them in frames.

[0171] Echo prediction is calculated using the audio played back by the loudspeaker as a reference. And based on the raw audio from the near end of the microphone and the predicted echo Calculate the clean near-end audio after echo cancellation Based on the pure near-end audio after echo cancellation Update the path weight coefficients, that is:

[0172] ;

[0173] ;

[0174] ;

[0175] In the formula, This represents the total order of the filter. For the first Real-time echo path weighting coefficient; To play audio back through the speakers; This is the raw audio from the near end of the microphone; The index of the discrete audio time-domain sampling point; The basic iteration step size; These are nonlinear correction coefficients; Based on speaker output power The nonlinear fitting function;

[0176] Pure near-end audio after echo cancellation The frequency domain spectrum is obtained by performing a short-time Fourier transform. And based on frequency domain spectrum Calculate the short-time energy of a single frame Based on single-frame short-time energy Voice baseband Human voice features are obtained by harmonic correlation Corr weighting. Based on the average energy of noisy frames Calculate noise characteristics Based on human voice characteristics and noise characteristics Calculate the noise suppression mask coefficient, i.e.:

[0177] ;

[0178] ;

[0179] ;

[0180] ;

[0181] In the formula, It is the discrete frequency domain angular frequency; , Preset weighting coefficients for human voice features; , These are the weighting coefficients for the noise branch; The noise distortion coefficient;

[0182] Based on the noise suppression mask coefficient Mask, the clean near-end audio after echo cancellation is processed. Noise reduction processing is performed to obtain the noise-reduced output spectrum. ;

[0183] ;

[0184] Pure near-end audio after echo cancellation and the output spectrum of noise reduction Perform temporal smoothing filtering and optimize the voice audio backhaul for use in zone mixing and multicast.

[0185] A third embodiment of this application provides an electronic device, comprising:

[0186] At least one processor;

[0187] and a memory communicatively connected to the at least one processor;

[0188] The memory stores a computer program that can be executed by at least one processor. When the computer program is executed by at least one processor, it implements the multi-device management method of the audio conferencing system based on the BS architecture.

[0189] The embodiments and descriptions above are merely illustrative of the principles and preferred embodiments of the present invention. Various changes and modifications may be made to the present invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed.

Claims

1. A multi-device management method for an audio conferencing system based on a B / S architecture, characterized in that, include: A distributed cluster B / S service architecture is established, including user terminals, external underlying audio devices, and multiple control hosts as cluster nodes. Each control host has a built-in WEB server, which generates a standardized device management webpage. The user terminal accesses the WEB server through a browser. The cluster nodes collect real-time hardware computing load data of the control host, data of external underlying audio devices, and status data of underlying audio devices, and push them to the user terminal through the WEB server; The control host receives control commands sent by the user through the browser of the user terminal, parses the control commands and verifies their legality, operation permissions and partition ownership. After verification, the control commands are routed and distributed to the corresponding external underlying audio devices according to the device type. The new external audio device is connected to the control host via the PoE++ interface, triggering a hardware interrupt. After the reserved margin verification is passed, the new external audio device is powered on and registered, and audio channel allocation and segmented multicast stream subscription are performed. It collects all microphone speaking request signals in real time, and allows or puts the current speaking request into the chairman mode queue according to the speaking threshold, and monitors in real time whether there are dual-end audio loops under the same account; Based on the UDP protocol, the global mixing data stream is logically segmented and isolated for transmission according to the venue partition. When bandwidth is congested, the audio sampling rate of non-critical peripherals is automatically reduced. The predicted echo is calculated based on the speaker playback audio as a reference, and the clean near-end audio after echo cancellation is calculated based on the original near-end audio from the microphone and the predicted echo. The noise reduction process is performed on the clean near-end audio after echo cancellation based on the noise suppression mask coefficient Mask to obtain the noise-reduced output spectrum.

2. The multi-device management method for an audio conferencing system according to claim 1, characterized in that: A distributed cluster B / S service architecture is established, comprising user terminals, external underlying audio devices, and multiple control hosts as cluster nodes. Each control host has a built-in web server, which generates standardized device management web pages. User terminals access the web server through a browser. Multiple control hosts are deployed as cluster nodes, and the distributed cluster B / S service architecture is configured with a three-layer isolation network: an external underlying audio device access network, a dedicated internal network for cluster nodes, and an external network for user access. The cluster nodes negotiate the cluster topology through a TCP long-connection handshake and automatically elect a cluster master node. The cluster master node maintains the global registry of the underlying audio devices. Each of the aforementioned control hosts is configured with an independent web server, providing static pages to users based on Nginx lightweight HTTP or HTTPS services, and distributing static resource requests to all cluster nodes in a round-robin fashion through a unified gateway; and allocating real-time status push requests from external underlying audio devices to a single cluster node based on the user terminal session ID hash. The cluster master node allocates underlying audio device fragments based on the real-time computing power of each cluster slave node, the remaining power supply capacity of the PoE port, the network bandwidth usage, and the number of external devices already connected. When a cluster slave node is detected as a faulty node, the cluster master node will migrate the external underlying audio devices of the faulty node to the normal cluster slave node with the lowest load within the cluster. When a new external audio device is connected to any cluster node, the cluster master node obtains the device registration information of the new external audio device, retrieves the normal cluster slave node with the lowest current load, issues a sharding assignment instruction, and assigns the new external audio device to the corresponding cluster slave node for management.

3. The multi-device management method for an audio conferencing system according to claim 2, characterized in that: The system receives control commands issued by the user through the browser on the user terminal, parses the control commands through the control host, and verifies the legality, operation permissions, and partition affiliation of the control commands. Once verified, the control commands are routed and distributed to the corresponding external underlying audio devices according to device type, including: When a user performs single-device control or batch operations on a partition in the browser on the user's terminal, a standardized JSON instruction package is generated, carrying the account token, cluster partition ID, target device set, operation type, and parameters, and uploaded to the unified gateway via WebSocket or HTTP interface; The unified gateway verifies the validity of the session token, filters out illegal parameters that exceed the hardware threshold, and forwards real-time lightweight control commands to the slave nodes of the sharded cluster to which the device belongs. The cluster performs legality, operation permissions, and partition ownership verification on real-time lightweight control commands from the nodes; after the verification is passed, the command parsing service routes and distributes the commands to the corresponding hardware control units according to the managed objects. The hardware control unit calls the hardware unified abstraction API adaptation layer to convert standardized upper-layer instructions into the underlying hardware protocols of POE port I2C, wired MIC RS485, wireless host private UDP, and audio codec register, and sends them down to the corresponding external underlying audio devices to complete the hardware actions. After the external low-level audio device completes the hardware action, it reports the execution status and sends it back to the instruction parsing service level by level. The execution result is pushed to the user terminal's browser via WebSocket through a unified gateway.

4. The multi-device management method for an audio conferencing system according to claim 3, characterized in that: A new external audio device is connected to the control host via the PoE++ interface, triggering a hardware interrupt. After the reserved margin verification passes, the new external audio device is powered on and registered, and audio channel allocation and multicast stream subscription are performed, including: When a new external audio device is connected to the PoE++ interface of the control host, a hardware interrupt is triggered. The real-time total power consumption of the control host is calculated. If the addition of the device causes the real-time total power consumption of the control host to exceed the rated power limit of the current PoE host, the power reserve of the currently idle peripherals is reduced. If there is no reserve in all PoE++ interfaces of the current control host, a power overload alarm is pushed to the user terminal. After verification, the new external audio device is powered on in a time-sharing and step-by-step manner. Once the new external audio device is powered on, a certain hardware stabilization delay is set, and the power supply voltage of the currently connected PoE++ port is monitored. If an abnormal power supply voltage is detected, the power is cut off and a hardware fault is reported. After the hardware stabilization delay ends, the peripheral detection unit shakes hands with the peripheral to read the unique device ID, hardware model, and device type of the newly connected external underlying audio device, automatically matches and binds the partition ID and cluster shard identifier, writes it to the local shard registry and synchronizes it to the global registry of the underlying audio device on the cluster master node to complete the device registration. After completing device registration, an independent audio channel is allocated to the newly connected external audio device and the microphone voice stream is automatically subscribed to. All microphone audio in the partition is aggregated and mixed to generate a partition-specific segmented multicast stream. Commands are issued to control all wired speakers in the partition to automatically subscribe to the segmented multicast stream. The user terminal issues a speaking command, verifies speaking permissions, enables microphone mixing weight, and establishes a complete sound amplification path from microphone to zone speaker. When a new external audio device is connected and disconnected, the microphone audio subscription is cut off, audio data is removed from the mixing channel, and instantaneous speaker noise is blocked. After waiting 3 seconds for the audio buffer to empty, the power supply to the corresponding PoE++ port is turned off, the device offline status is updated, and the cluster registry is synchronized.

5. The multi-device management method for an audio conferencing system according to claim 4, characterized in that: Real-time acquisition of all microphone speaking request signals, and based on speaking thresholds, granting permission for the current speaking request or placing it into the chairman mode queue, including: The chairman microphone is set up, and the chairman has the right to speak continuously. It collects all microphone speaking request signals in real time, counts the number of microphones currently speaking, and allows all microphone audio streams to enter the partitioned mixing multicast if the current online speaking requests are less than the speaking threshold. It also automatically balances the voice gain of multiple microphones. If the current online speaking requests are greater than or equal to the speaking threshold, the excess microphones will enter the first-in-first-out queuing queue and be temporarily muted by hardware. The queuing microphones will be dynamically scheduled to speak in turn based on the speaking duration and volume of the voice. When the original microphone releases its speaking privileges, the queued microphone is automatically retrieved from the head of the queue, unmuted, and added to the concurrent speaking channel.

6. The multi-device management method for an audio conferencing system according to claim 5, characterized in that: Real-time monitoring to see if a microphone audio input device and a terminal audio output device exist simultaneously under the same account; if a dual-end audio loop is detected, a mute command is sent to the unified gateway to temporarily mute the audio output of all browsers for the corresponding account.

7. The multi-device management method for an audio conferencing system according to claim 6, characterized in that: The global mixing data stream is logically segmented and isolated for transmission based on the UDP protocol and venue partitions. When bandwidth is congested, the audio sampling rate of non-critical peripherals is automatically reduced, including: Based on the cluster partition ID, each venue is assigned an independent UDP multicast address and UDP multicast port. The global mixed audio data stream after mixing multiple microphones in a single cluster partition is encapsulated in fixed-length UDP fragments. The UDP fragment header embeds the cluster partition ID, UDP fragment sequence number, device priority, and microsecond timestamp to distinguish the priority of three types of devices: primary speaking microphones, secondary queuing microphones, and tertiary idle peripherals. The audio streams from all wired and wireless microphones in the cluster partition are collected and mixed according to the cluster partition. The data is then cut into independent UDP fragments according to the preset fragment length and sent to the UDP multicast address of the cluster partition. All speaker hardware in the cluster partition automatically subscribes to the corresponding UDP multicast port of the cluster partition. Real-time statistics on UDP fragmented data packet sending rate, data packet queuing buffer length, and fragmented packet loss rate; when bandwidth usage exceeds the warning threshold, automatically reduce the audio sampling rate of non-critical peripherals.

8. The multi-device management method for an audio conferencing system according to claim 7, characterized in that: The predicted echo is calculated based on the speaker playback audio as a reference, and the clean near-end audio after echo cancellation is calculated based on the original near-end audio from the microphone and the predicted echo. The clean near-end audio after echo cancellation is then denoised based on the noise suppression mask coefficients, resulting in a denoised output spectrum, including: The system receives two data streams in parallel: the raw audio from the microphone and the playback audio from the speaker, and buffers them in frames. Echo prediction is calculated using the audio played back by the loudspeaker as a reference. And based on the raw audio from the near end of the microphone and the predicted echo Calculate the clean near-end audio after echo cancellation Based on the pure near-end audio after echo cancellation Update the path weight coefficients, that is: ； ； ； In the formula, This represents the total order of the filter. For the first Real-time echo path weighting coefficient; To play audio back through the speakers; This is the raw audio from the near end of the microphone; The index of the discrete audio time-domain sampling point; The basic iteration step size; These are nonlinear correction coefficients; Based on speaker output power The nonlinear fitting function; Pure near-end audio after echo cancellation The frequency domain spectrum is obtained by performing a short-time Fourier transform. And based on frequency domain spectrum Calculate the short-time energy of a single frame Based on single-frame short-time energy Voice baseband Human voice features are obtained by harmonic correlation Corr weighting. Based on the average energy of noisy frames Calculate noise characteristics Based on human voice characteristics and noise characteristics Calculate the noise suppression mask coefficient Mask, i.e.: ； ； ； ； In the formula, It is the discrete frequency domain angular frequency; , Preset weighting coefficients for human voice features; , These are the weighting coefficients for the noise branch; The noise distortion coefficient; Based on the noise suppression mask coefficient Mask, the clean near-end audio after echo cancellation is processed. Noise reduction processing is performed to obtain the noise-reduced output spectrum. ; ； Pure near-end audio after echo cancellation Output spectrum of noise reduction Perform temporal smoothing filtering and optimize the voice audio backhaul for use in zone mixing and multicast.

9. A multi-device management system for audio conferencing systems based on a B / S architecture, characterized in that, include: The B / S service architecture module is used to build a distributed cluster B / S service architecture that includes user terminals, external underlying audio devices, and multiple control hosts as cluster nodes. Each control host has a built-in WEB server, which generates a standardized device management webpage. The user terminal accesses the WEB server through a browser. The data acquisition module is used to collect hardware computing load data of the control host, data of external underlying audio devices, and status data of underlying audio devices in real time through the cluster nodes, and push them to the user terminal through the WEB server. The instruction parsing module is used to receive control instructions issued by the user through the browser of the user terminal, and to parse the control instructions through the control host and verify the legality, operation permissions and partition ownership of the control instructions. After verification, the control instructions are routed and distributed to the corresponding external underlying audio devices according to the device type. The new device management module is used to trigger a hardware interrupt when a new external underlying audio device is connected to the control host via the PoE++ interface. After passing the reserved margin verification, the new external underlying audio device is powered on and registered, and audio channel allocation and segmented multicast stream subscription are performed. The concurrent conflict suppression module is used to collect all microphone speaking request signals in real time, and to allow the current speaking request or put it into the chairman mode queue according to the speaking threshold, and to monitor in real time whether there are dual-end audio loops under the same account. The audio segmentation and scheduling module is used to logically segment and isolate the global mixing data stream according to the venue partition based on the UDP protocol. When bandwidth is congested, it automatically reduces the audio sampling rate of non-critical peripherals. The audio processing module is used to calculate the predicted echo based on the speaker playback audio as a reference, and to calculate the clean near-end audio after echo cancellation based on the original near-end audio from the microphone and the predicted echo. The noise reduction process is performed on the clean near-end audio after echo cancellation based on the noise suppression mask coefficient Mask to obtain the noise-reduced output spectrum.

10. An electronic device, comprising: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by at least one processor, and when the computer program is executed by at least one processor, it implements the multi-device management method for an audio conferencing system based on a B / S architecture as described in any one of claims 1-8.

Images