Multi-protocol compatible equipment access and data management system

Abstract

The invention relates to the technical field of compatible equipment access management, and discloses a multi-protocol compatible equipment access and data management system, which comprises the following steps: acquiring the equipment type and spatial position of a conference room, allocating a bit rate for each piece of equipment according to the priority of the equipment, and calculating the bandwidth allocation interval of equipment access; monitoring the field end accessed to the online conference area, setting an online conference participation permission according to the conference participation distance, executing a regulation and control instruction on the field end with the online conference participation permission, and optimizing the definition and volume of the field end accessed to the online conference area; the method comprises the following steps: identifying whether a behavior of starting a microphone by a field end accessed to an online conference area is active speaking, regulating and controlling field ends participating in an offline conference area according to an identification result, calculating utility scores of two behaviors of a mobile main control end and a newly accessed field end on the premise of the mobile main control end, and selecting a speaking mode based on the utility scores. And the intelligent level of conference management, the resource utilization efficiency and the multi-protocol compatibility are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of compatible device access management, in particular to a multi-protocol compatible device access and data management system. BACKGROUND

[0002] With the popularity of remote office and hybrid meeting mode, online and offline combined meetings have become the norm of enterprise, education and scientific research activities. Online meetings can break through geographical restrictions and allow remote participants to access information synchronously; offline meetings can realize face-to-face communication and ensure discussion efficiency and interactive experience. The combination of the two improves communication efficiency, takes into account the flexibility and on-site feeling of participants, and is an important trend of modern meeting management.

[0003] In the prior art, the conference system usually adopts fixed allocation bandwidth or first-come-first-served method in the multi-device access scene, resulting in insufficient bandwidth for participants who really need to receive audio and video online, low transmission quality, and phenomena such as blurred picture and audio delay. At the same time, the prior art lacks dynamic permission management based on spatial location and actual meeting needs in the meeting. Even if some on-site ends are close to the main control end and can clearly hear and watch the meeting content, they are still allowed to access the online meeting area to receive audio and video, resulting in unnecessary bandwidth occupation. The existing system lacks refinement and real-time in volume and clarity control, and cannot dynamically adjust according to device location, distance and environmental noise. In addition, offline speech recognition relies on manual prompting or fixed microphone control rules, making it difficult to accurately judge the mis-opening microphone behavior of the on-site end, and also cannot intelligently select the appropriate speaking end according to the speaker's position and device state, resulting in insufficient utilization of conference resources. The prior art lacks comprehensive control means based on space, bandwidth and device state, and cannot meet the efficient, clear and orderly meeting needs in online and offline hybrid meetings.

[0004] The present application provides a multi-protocol compatible device access and data management system to solve the problems mentioned in the background art. SUMMARY

[0005] The present application provides a multi-protocol compatible device access and data management system to facilitate solving the problems mentioned in the background art.

[0006] The present application provides the following technical solution: a multi-protocol compatible device access and data management system, comprising:

[0007] A space-driven bandwidth control module is configured to collect the types and spatial positions of devices connected to the local area network in the conference room, allocate bit rates to each device according to device priority, and calculate the bandwidth allocation interval of the device access.

[0008] The on-site participant device optimization module is configured to monitor on-site terminals accessing the online conference area, set online participation permissions according to participant distances, execute control instructions on the on-site terminals with online participation permissions, and optimize the clarity and volume of the on-site terminals accessing the online conference area.

[0009] The offline speech recognition and control module is configured to identify whether the on-site terminals accessing the online conference area open the microphone for active speech when no offline speech prompt is received, and control the on-site terminals participating in the offline conference area according to the identification result.

[0010] The speaker positioning and decision-making module is configured to locate the speaker position when the offline speech prompt is received, calculate utility scores of the two types of behaviors of the mobile master terminal and the newly accessed on-site terminal on the premise that the mobile master terminal is movable, and select a speech mode based on the utility scores.

[0011] Optionally, the spatial driving bandwidth control module is configured to collect the types and spatial positions of devices accessing the local area network in the conference room, allocate bit rates to each device according to the device priority, and calculate a bandwidth allocation interval of the device access, including:

[0012] The conference room includes an offline conference area and an online conference area, the online conference is initiated by the master terminal, and the remote terminal accesses the online conference to receive the audio and video of the offline conference;

[0013] The speaker in the offline conference area hosts the conference through the master terminal and projects the conference content to the display terminal;

[0014] The offline conference area is composed of multiple on-site participants carrying on-site terminals;

[0015] For any on-site terminal, the distances from the on-site terminal to the master terminal and the display terminal are obtained respectively and ;

[0016] The participant distance is calculated ;

[0017] The participant distances of all on-site terminals are obtained , the mean value is calculated as the standard distance;

[0018] The priority of the on-site terminal is calculated , wherein is an exponential constant, used to control the growth rate of the priority, and the priority indicates that the farther the participant distance of the on-site terminal deviates from the standard distance, the higher the probability of the on-site terminal accessing the online conference;

[0019] The working bandwidth of the master terminal is obtained , and the total bandwidth of the local area network is obtained ;

[0020] Allocate bit rate to field end ;

[0021] limited ;

[0022] Obtain the working bandwidth of the field terminal Calculate the lower limit bandwidth , This represents the bandwidth ratio.

[0023] Calculate the upper limit bandwidth Obtain the bandwidth allocation range .

[0024] Optionally, the step of executing control commands on on-site terminals with online meeting access to optimize the clarity and volume of the on-site terminals accessing the online meeting area includes:

[0025] To set up online participation permissions for on-site participants, the specific steps are as follows:

[0026] like Then, on-site devices are prohibited from accessing the online meeting area to receive audio and video;

[0027] like The system allows on-site devices to access the online meeting area to receive audio and video, confirms that the on-site devices have online meeting participation permissions, and performs resolution adjustment.

[0028] Set distance ratio coefficient ;

[0029] like Calculate the resolution adjustment ratio , ;

[0030] , The minimum resolution adjustment ratio, To adjust the intensity factor, the closer the meeting distance is to the standard distance, the lower the sharpness; the closer it is to the standard distance, the lower the sharpness. The higher the resolution, the better;

[0031] like Clarity ;

[0032] When a live device with online meeting access is in the online meeting area, if it is detected that the live device is using a speaker, volume control will be performed, specifically as follows:

[0033] Calculate the volume based on the distance of the participants at the venue. This is used to prevent the main control unit from receiving audio from the field unit's external speaker, thus avoiding mixing. ;

[0034] As the reference volume, This is the distance attenuation coefficient. This indicates that the volume decreases with distance.

[0035] Optionally, the volume control further includes:

[0036] Multiple measurement points were set up in the conference room space, and the noise pressure at these points was collected in real time. A weighted average of the noise pressure at all measurement points was then calculated, and the result was recorded as follows: ;

[0037] The closer the measurement point is to the field, the higher its weight.

[0038] The maximum volume at the live location is limited to , This is a safety compensation factor used to ensure that the volume at the site is higher than the sound pressure level of the surrounding noise.

[0039] When there are multiple externally mounted field terminals in the conference room, calculate the straight-line distance between any two field terminals;

[0040] If the straight-line distance is lower than the set safe distance, then turn off the external audio at any field end until the straight-line distance at any field end is lower than the safe distance.

[0041] Optionally, the offline speaking recognition and control module, when not receiving an offline speaking prompt, is used to identify whether the microphone activation behavior of the on-site terminal accessing the online meeting area constitutes active speaking, including:

[0042] The video feed, speakers, and microphone in the online meeting area are independently controlled;

[0043] Participants at the venue can turn off their video and speakers when attending the online meeting, and only turn on their microphones to record and transmit audio;

[0044] When a microphone is detected being activated in an offline meeting room:

[0045] Set sound pressure suppression factor This is used to suppress the initial sound pressure signal of the microphone;

[0046] Acquire the initial sound pressure signal from the microphone. ,calculate , as a transmission signal;

[0047] Monitor the behavior of on-site participants at set time intervals, and calculate the increase in sound pressure signal in real time when the microphone volume is increased at the on-site end. Set the sound pressure gain threshold ;

[0048] like Therefore, turning on the microphone is preliminarily determined to be an act of speaking voluntarily;

[0049] If the microphone volume is not increased at the site within the time interval, it is considered an accidental microphone activation and should be immediately disabled.

[0050] Optionally, the step of identifying whether the microphone activation behavior of the on-site terminal accessing the online conference area constitutes active speaking, and adjusting the on-site terminal participating in the offline conference area based on the identification result, includes:

[0051] When the act of turning on the microphone is initially determined to be an active speaking action, a verification strategy is executed, specifically:

[0052] Multiple audio frames are obtained by acquiring sound pressure signals of a fixed duration at a set sampling rate;

[0053] For the Extract amplitude from each audio frame segment Calculate the energy of audio frames ;

[0054] Energy smoothing is used to determine whether the sound pressure signal is persistent. , The smoothing factor is the closest to 1, indicating greater stability. For the first Energy after smoothing an audio frame For the first Energy after smoothing an audio frame segment;

[0055] For the Calculate the probability of language activity for each audio frame. , The closer the energy of an audio frame is to the noise energy, the less likely there is a speech input microphone; otherwise... The highest probability indicates that the microphone is used for voice input;

[0056] For the set noise energy, This is the energy enhancement factor, used as the factor by which language exceeds noise;

[0057] Set a threshold for the probability of language activities ;

[0058] Get The number of audio frames and the total duration are calculated. ;

[0059] Total duration of acquiring sound pressure signal Calculate the proportion of languages ;

[0060] If the proportion of spoken words exceeds the set threshold, then turning on the microphone is considered an active speaking action.

[0061] When it is identified as an active speaker, the on-site terminal at the active speaker's location is closed. The speaker plays the sound from within the distance.

[0062] Optionally, the method for locating the speaker's position, assuming a movable master control terminal, calculates utility scores for two types of behaviors: moving the master control terminal and newly accessing the field terminal, including:

[0063] The main control unit connects to the conference room camera and displays the conference room monitoring screen in real time, allowing the host to locate the speaker's position in the monitoring screen;

[0064] Get distance from speaker's location All field terminals within the scope, for any field terminal:

[0065] Obtain the real-time bit rate at the field end Calculate available bandwidth ;

[0066] Set bandwidth threshold ;

[0067] filter On-site calculation of utility score Specifically:

[0068] Normalize the noise sound pressure level and update it with the normalized value;

[0069] ,in, For signal-to-noise ratio, This is the interference adjustment coefficient, used to quantify the degree to which noise sound pressure suppresses the signal-to-noise ratio. The normalized activation cost, This represents the total available bandwidth, used to penalize overload. The weighting coefficients are set.

[0070] Calculate the utility score of the mobile master terminal. Specifically:

[0071] ,in, This is the estimated change in signal-to-noise ratio before and after the mobile master control unit. Delay caused by movement For mobile costs, This is the cost coefficient. This indicates that the higher the signal gain, the lower the latency and cost, the higher the utility score. Normalize all;

[0072] Calculate confidence discount Modified utility rating , , These are calibration constants used for normalization. The trace of the covariance matrix measures overall uncertainty. The cosine matrix is ​​estimated based on the speaker's location. The confidence discount decreases as the uncertainty of the speaker's location increases.

[0073] Calculate the corrected utility score .

[0074] Optionally, the selection of speaking methods based on utility rating includes:

[0075] Filter the most efficient rating Recorded as ,Compare and Size relationship;

[0076] like ,Will The corresponding on-site device connects to the online meeting and enables the microphone for the speaker's use;

[0077] like The main control unit is then passed to the spokesperson for their use.

[0078] The present invention has the following beneficial effects:

[0079] 1. This multi-protocol compatible device access and data management system dynamically manages bandwidth allocation, speaking permissions, volume, and clarity in multi-device, multi-protocol environments, solving problems such as chaotic device access, uneven audio and video quality, accidental microphone activation interference, and lack of intelligent control over speaking terminal selection in existing technologies. It performs quantitative analysis based on device type, spatial location, and participant distance, intelligently allocating bitrate and bandwidth ranges. Clarity and volume are adjusted for on-site terminals with online participation permissions, ensuring high-quality audio and video for remote terminals and improving the meeting experience. It accurately judges proactive speaking behavior, avoiding the impact of misoperation on meeting order and audio transmission. The speaker positioning and decision-making module quantifies the value of each speaking device through utility scoring, achieving optimal speaking terminal selection, improving system response speed and signal quality, and ensuring synchronous and efficient online and offline meetings. Overall, it significantly improves the intelligence level of meeting management, resource utilization efficiency, and multi-protocol compatibility, enhancing applicability and reliability.

[0080] 2. This multi-protocol compatible device access and data management system calculates the deviation of each on-site device's distance from the standard distance and converts this deviation into priority. This allows for scientific bitrate allocation, prioritizing devices further from the conference center (due to unclear viewing or noise) to ensure high-quality audio and video transmission. By setting upper and lower bandwidth limits and considering the on-site device's operating bandwidth, the system prevents audio and video quality degradation due to insufficient bandwidth while avoiding excessive bandwidth consumption on the main control unit and other devices, thus improving overall network resource utilization efficiency. It enables stable and reliable bandwidth control in a multi-protocol compatible environment, supporting subsequent clarity and volume optimization, speech recognition, and utility scoring decisions. This provides quantifiable and controllable basic data for the system, making device access in mixed online and offline meetings more fair and reasonable, improving the remote user experience, and ensuring smooth and efficient meetings.

[0081] 3. This multi-protocol compatible device access and data management system effectively improves the quality and efficiency of online and offline collaborative meetings by finely configuring online participation permissions and dynamically adjusting clarity and volume. It dynamically calculates the clarity adjustment ratio based on the difference between the participant distance and the standard distance, enabling key devices to obtain higher image quality while avoiding excessive bandwidth consumption by nearby devices, ensuring the accuracy of information reception at the remote end. Volume control combines distance attenuation and conference room noise measurement, setting a safety compensation coefficient and limiting the maximum volume to avoid mixing, echo, and interference with the remote end's experience at the main control end. For multiple external speakers, the system automatically shuts down speakers of devices that are too close based on a safe distance determination, ensuring the stability of conference audio transmission. Real-time dynamic adjustment is achieved in scenarios with multiple devices and multiple participants, ensuring clear online audio and video reception and reasonable volume at the on-site end, improving the remote end's experience and the order of the on-site meeting. Through intelligent clarity and volume management, network resource utilization is significantly optimized, enhancing adaptability and controllability in complex environments, providing an important guarantee for high-quality hybrid meetings.

[0082] 4. This multi-protocol compatible device access and data management system, by monitoring the microphone activation and volume changes at the field end in real time, achieves active speech recognition and control of accidentally activated microphones, solving the problems of existing technologies that rely on manual prompts and have high misjudgment rates. Through sound pressure suppression, initial signal acquisition, incremental change monitoring, and threshold determination, combined with audio frame energy smoothing and speech activity probability calculation, it achieves quantitative analysis of sound pressure signal persistence and speech input probability. Field ends with accidentally activated microphones can be immediately shut down to prevent invalid audio from occupying bandwidth or interfering with the main control end. Simultaneously, it shuts down the external speakers within the standard distance of the active speaker to ensure clear speech. It achieves automated recognition and control of on-site speaking behavior, enhances adaptability to noise and environmental interference, and ensures the accuracy of remote audio reception and meeting order. It achieves highly intelligent management in multi-device, multi-participant environments, improves online and offline collaboration efficiency, reduces human intervention, and achieves a stable, clear, and orderly hybrid meeting experience.

[0083] 5. This multi-protocol compatible device access and data management system calculates utility scores based on speaker location and device status, enabling intelligent speaker selection and solving the problem that existing technologies cannot dynamically decide based on factors such as location and bandwidth. By acquiring indicators such as speaker location, real-time bit rate, and signal-to-noise ratio at the site, and calculating the utility scores of the mobile master terminal and newly accessed site terminals, the value of each speaking device is quantified. Confidence discounting is used to correct for positioning uncertainty, making the scoring results robust and reliable. By comparing the highest utility score with the mobile master terminal's utility score, the optimal speaking terminal is intelligently selected, ensuring high signal quality and reasonable resource utilization. This significantly improves the interactive efficiency and audio-visual quality of online and offline meetings, ensuring smooth meetings, and enhancing the system's adaptability, intelligence, and reliability in multi-protocol and multi-device access scenarios, providing key technical support for efficient hybrid meetings. Attached Figure Description

[0084] Figure 1 This is a schematic diagram of the system modules of the present invention. Detailed Implementation

[0085] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0086] Example 1, refer to Figure 1 A multi-protocol compatible device access and data management system, comprising:

[0087] Control Unit: Responsible for managing, scheduling, and controlling equipment inside and outside the conference room, such as conference computers or conference tablets with control software. Display Unit: Only responsible for presenting the meeting content, such as projectors, large-screen TVs, or interactive whiteboards. On-site Unit: The work terminals of offline participants, which can both view online content and speak, such as laptops, tablets, or portable conference terminals provided in the conference room. Remote Unit: The terminals of online participants, used to receive and speak, such as laptops, tablets, or mobile phones used by remote workers.

[0088] The space-driven bandwidth control module is used to collect the device types and spatial locations connected to the local area network of the conference room, allocate bit rates to each device according to device priority, and calculate the bandwidth allocation range for device access, including:

[0089] The meeting room includes an offline meeting area and an online meeting area. The online meeting is initiated by the main control terminal, and the remote terminal joins the online meeting to receive the audio and video from the offline meeting.

[0090] Speakers in the offline meeting area chair the meeting through the main control terminal and project the meeting content onto the display terminal;

[0091] Multi-protocol compatibility means that the system can handle data from different devices, different audio and video protocols, and different transmission methods, and can be accessed and managed normally on this network structure.

[0092] By deploying this solution in a real conference room environment and conducting experiments compared with traditional meeting management methods, the experimental data shows that by using the space-driven bandwidth control module to allocate bandwidth according to the distance and priority of on-site participants, the clarity of audio and video received by participants far from the main control terminal is improved by an average of about 28%, the volume distortion rate is reduced by about 32%, and the number of audio mixing events received by the remote terminal is reduced by about 45%.

[0093] Furthermore, the offline speech recognition and control module identifies proactive speaking behavior by monitoring microphone volume changes in real time. Experimental results show that accidental microphone activation events were reduced by approximately 60%, effectively reducing invalid audio interference. The application of the speaker location and utility scoring decision module selects the optimal speaking method based on network bandwidth and mobility costs. In scenarios with multiple people speaking continuously, the overall meeting response time was shortened by an average of approximately 22%, while ensuring stable network load and avoiding online audio and video stuttering caused by newly connected devices.

[0094] In summary, the experimental data fully demonstrates that this solution can comprehensively optimize the audio and video quality, network resource utilization, and meeting efficiency of online and offline collaborative meetings, providing stable and reliable technical support for actual meeting scenarios.

[0095] The main control equipment establishes a remote communication channel according to the protocol signaling.

[0096] For any field terminal, obtain the distance from the field terminal to the main control terminal and the display terminal respectively. and ;

[0097] Calculate the distance to the meeting ;

[0098] Get the distance of all participants at the venue Calculate the mean , as the standard distance;

[0099] Calculate the priority of the field end. ,in, It is an exponential constant. This is used to control the growth rate of priority. Priority means that the further the distance of the on-site participant deviates from the standard distance, the higher the probability that the on-site participant will be able to join the online meeting.

[0100] Participants who are far from the conference center (such as the main control unit or display unit) often have difficulty receiving the audio and video information of the conference clearly due to increased physical distance, sound field attenuation, or surrounding noise interference. Therefore, they are more likely to watch the online conference via mobile phone or computer during the offline conference, so they are given high priority.

[0101] Obtain the working bandwidth of the master control unit Get the total bandwidth of the local area network ;

[0102] Allocate bit rate to field end ;

[0103] A higher priority indicates that more bandwidth should be allocated to meet the needs of watching the meeting online.

[0104] limited ;

[0105] Obtain the working bandwidth of the field terminal Calculate the lower limit bandwidth , This represents the bandwidth ratio.

[0106] Calculate the upper limit bandwidth Obtain the bandwidth allocation range .

[0107] The on-site meeting equipment optimization module monitors on-site devices connected to the online meeting area and sets online meeting permissions based on the distance to the device. It executes control commands on on-site devices with online meeting permissions to optimize the clarity and volume of the on-site devices connected to the online meeting area. Specifically, it sets online meeting permissions for on-site devices as follows:

[0108] Online meeting access is limited to the reception of audio and video, not to whether the device is connected to the online meeting.

[0109] like Then, on-site devices are prohibited from accessing the online meeting area to receive audio and video;

[0110] like The system allows on-site devices to access the online meeting area to receive audio and video, confirms that the on-site devices have online meeting participation permissions, and performs resolution adjustment.

[0111] Set distance ratio coefficient ;

[0112] like Calculate the resolution adjustment ratio , ;

[0113] , The minimum resolution adjustment ratio, To adjust the intensity factor, the closer the meeting distance is to the standard distance, the lower the sharpness; the closer it is to the standard distance, the lower the sharpness. The higher the resolution, the better;

[0114] like Clarity As the distance from the display screen and the main control unit increases, the clarity gradually increases, optimizing the conference experience and making it more user-friendly for those attending the conference in person but from a distance.

[0115] When a live device with online meeting access is in the online meeting area, if it is detected that the live device is using a speaker, volume control will be performed, specifically as follows:

[0116] Considering that the meeting room is large or some areas are noisy, and that participants who are far from the main control or display end may collectively have difficulty hearing, if these participants each use their mobile phones or computers to receive audio and video individually, it will consume network resources. Turning on the speakers can extend the audio to a smaller area, so that the surrounding participants can receive the meeting information at the same time.

[0117] Calculate the volume based on the distance of the participants at the venue. This is used to prevent the main control unit from receiving audio from the field unit's external speaker, thus avoiding mixing. ;

[0118] As the reference volume, This is the distance attenuation coefficient. This indicates that the volume decreases with distance.

[0119] Multiple measurement points were set up in the conference room space, and the noise pressure at these points was collected in real time. A weighted average of the noise pressure at all measurement points was then calculated, and the result was recorded as follows: ;

[0120] The maximum volume at the live location is limited to , This is a safety compensation factor used to ensure that the volume at the site is higher than the sound pressure level of the surrounding noise.

[0121] When there are multiple externally mounted field terminals in the conference room, calculate the straight-line distance between any two field terminals;

[0122] If the straight-line distance is lower than the set safe distance, then turn off the external audio at any field end until the straight-line distance at any field end is lower than the safe distance.

[0123] The offline speaking recognition and control module, when no offline speaking prompt is received, is used to identify whether the microphone activation behavior of the on-site terminal accessing the online meeting area constitutes active speaking, including:

[0124] The video feed, speakers, and microphone in the online meeting area are independently controlled for refined bandwidth allocation. This is because: for participants located nearby: they do not need to enable online video or speakerphone; only microphone access is available, avoiding bandwidth consumption and interference with others. For participants at a moderate distance: they can enable online video, but with reduced resolution and limited speakerphone volume to balance viewing experience with bandwidth and environmental interference. For participants located far away: normal viewing and speaking permissions for the online meeting are unrestricted.

[0125] This means that speaking privileges are completely independent of viewing / speaking, which meets the actual needs of meetings. People who are close to the speaker can speak without consuming additional resources, while people who are far away can watch and listen in full.

[0126] When a microphone is detected being activated in an offline meeting room:

[0127] Set sound pressure suppression factor This is used to suppress the initial sound pressure signal of the microphone;

[0128] Acquire the initial sound pressure signal from the microphone. ,calculate , as a transmission signal;

[0129] Monitor the behavior of on-site participants at set time intervals, and calculate the increase in sound pressure signal in real time when the microphone volume is increased at the on-site end. Set the sound pressure gain threshold ;

[0130] like Therefore, turning on the microphone is preliminarily determined to be an act of speaking voluntarily;

[0131] In real-world meeting scenarios, accidental microphone activation is a frequent occurrence. This can happen when participants accidentally touch the microphone switch, forget to mute it, or the microphone is enabled by default on the device. Such misoperations result in invalid audio signals consuming network bandwidth and may generate noise or echoes, interfering with the audio transmission of other speakers, reducing the clarity of audio received remotely, and decreasing the overall efficiency of the meeting. If multiple participants simultaneously activate their microphones, it can even lead to audio mixing or echo accumulation, severely impacting meeting order and the remote experience, preventing participants from accurately obtaining meeting information.

[0132] This solution first reduces the initial microphone audio signal (similar to turning down the volume), and then observes whether participants actively increase their microphone volume within a short period. If they do, it indicates they genuinely want to speak, and their voice is allowed to pass; if the volume doesn't change, it's considered an accidental microphone activation, and the microphone is automatically muted. The advantages are: accidental microphone activation won't disrupt the meeting, the voice of the person who truly wants to speak can be clearly heard, network and audio resources are used efficiently, and remote participants can hear clearly.

[0133] This is used to identify whether the microphone activation behavior of on-site participants accessing the online meeting area constitutes active speaking, and to adjust the settings of on-site participants in the offline meeting area based on the identification results, including:

[0134] Simply adjusting the microphone volume isn't enough to handle real-world situations. For example, someone might accidentally turn on their microphone and speak at a normal volume to a classmate. In this case, the user won't adjust the volume, but the sound of their voice will still enter the microphone because they are close to it. Someone might be intentionally speaking, but their voice might be too low initially. A user might click the microphone button to start speaking, but due to nervousness, habit, or not being close to the microphone initially, they might not manually adjust the volume. Not adjusting the volume doesn't mean they aren't speaking. This also includes accidentally clicking to adjust the microphone volume without actually speaking; relying solely on whether the microphone volume is turned up shouldn't be the sole basis for judgment.

[0135] When the act of turning on the microphone is initially determined to be an active speaking action, a verification strategy is executed, specifically:

[0136] Multiple audio frames are obtained by acquiring sound pressure signals of a fixed duration at a set sampling rate;

[0137] For the Extract amplitude from each audio frame segment Calculate the energy of audio frames ;

[0138] Energy smoothing is used to determine whether the sound pressure signal is persistent. , The smoothing factor is the closest to 1, indicating greater stability. For the first Energy after smoothing an audio frame For the first Energy after smoothing an audio frame segment;

[0139] For the Calculate the probability of language activity for each audio frame. , The closer the energy of an audio frame is to the noise energy, the less likely there is a speech input microphone; otherwise... The highest probability indicates that the microphone is used for voice input;

[0140] For the set noise energy, This is the energy enhancement factor, used as the factor by which language exceeds noise;

[0141] Set a threshold for the probability of language activities ;

[0142] Get The number of audio frames and the total duration are calculated. ;

[0143] Total duration of acquiring sound pressure signal Calculate the proportion of languages ;

[0144] If the proportion of speech exceeds the set threshold, the act of turning on the microphone is considered as active speaking, and the speech energy collected by the microphone continuously exceeds the threshold.

[0145] When it is identified as an active speaker, the on-site terminal at the active speaker's location is closed. The speaker plays the sound from within the distance.

[0146] The speaker location and decision-making module is used to locate the speaker's position when an offline speaking prompt is received. Under the premise of a mobile master terminal, it calculates the utility scores of two types of behaviors: mobile master terminal and newly connected on-site terminal, and selects the speaking method based on the utility scores.

[0147] In real-world meeting scenarios, when offline speakers need to transmit their audio to the online meeting area, directly starting a new on-site terminal for online access, while allowing the speaker's audio to quickly enter the online meeting, may consume excessive network bandwidth. This can negatively impact the audio and video clarity and smoothness of existing online participants, causing noise or dropped frames, thus reducing the overall meeting quality. On the other hand, if the control terminal is moved to the speaker's location to ensure stable network load, the move may cause time delays, increase operational complexity and human intervention costs, especially in situations with multiple speakers or a fast-paced meeting, potentially slowing down the meeting's progress.

[0148] The main control unit connects to the conference room camera and displays the conference room monitoring screen in real time, allowing the host to locate the speaker's position in the monitoring screen;

[0149] Get distance from speaker's location All field terminals within the scope, for any field terminal:

[0150] Obtain the real-time bit rate at the field end Calculate available bandwidth ;

[0151] Set bandwidth threshold ;

[0152] filter On-site calculation of utility score Specifically:

[0153] Normalize the noise sound pressure level and update it with the normalized value;

[0154] ,in, For signal-to-noise ratio, This is the interference adjustment coefficient, used to quantify the degree to which noise sound pressure suppresses the signal-to-noise ratio. The normalized activation cost, This represents the total available bandwidth, used to penalize overload. The weighting coefficients are set.

[0155] Calculate the utility score of the mobile master terminal. Specifically:

[0156] ,in, This is the estimated change in signal-to-noise ratio before and after the mobile master control unit. Delay caused by movement For mobile costs, This is the cost coefficient. This indicates that the higher the signal gain, the lower the latency and cost, the higher the utility score. Normalize all;

[0157] Calculate confidence discount Modified utility rating , , These are calibration constants used for normalization. The trace of the covariance matrix measures overall uncertainty. The cosine matrix is ​​estimated based on the speaker's location. The confidence discount decreases as the uncertainty of the speaker's location increases.

[0158] Calculate the corrected utility score .

[0159] The speaking method is selected based on utility rating, including:

[0160] Filter the most efficient rating Recorded as ,Compare and Size relationship;

[0161] like ,Will The corresponding on-site device connects to the online meeting and enables the microphone for the speaker's use;

[0162] like The main control unit is then passed to the spokesperson for their use.

[0163] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0164] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A multi-protocol compatible device access and data management system, characterized by, The application relates to a conference room space driving bandwidth control method and system. The space driving bandwidth control module is used for collecting the device type and space position of the local area network access of a conference room, allocating a bit rate for each device according to the device priority, and calculating the bandwidth allocation interval of the device access. The on-site participant device optimization module is used for monitoring the on-site end access to the online conference area and setting the online participant permission according to the participant distance, executing a control instruction for the on-site end with the online participant permission, and optimizing the definition and volume of the on-site end access to the online conference area. The offline speech recognition control module is used for identifying whether the on-site end with the microphone opened in the online conference area is active speaking when no offline speech prompt is received, and controlling the on-site end in the offline conference area according to the recognition result. The speaker positioning and decision module is used for positioning the speaker position when the offline speech prompt is received, calculating the utility score of the two types of behaviors of the mobile master end and the newly-accessed on-site end on the premise that the mobile master end is movable, and selecting the speaking mode based on the utility score.

2. The multi-protocol compatible device access and data management system of claim 1, wherein, The space driving bandwidth control module is used for collecting the device type and space position of the local area network access of a conference room, allocating a bit rate for each device according to the device priority, and calculating the bandwidth allocation interval of the device access. The conference room comprises an offline conference area and an online conference area, the online conference is initiated by a master end, and a remote end accesses the online conference to receive the audio and video of the offline conference. The speaker in the offline conference area hosts the conference through the master end and projects the conference content to a display end. The offline conference area is composed of multiple on-site participants carrying on-site ends. For any field end, the distance from the field end to the main control end and the display end is obtained respectively and ; Compute attendance distance ; Acquire all the on-site end of the participant distance , calculate the mean , as the standard distance; The priority of the site terminal is calculated wherein, is an exponential constant, for controlling the growth rate of the priority, the priority indicates that the closer the participant distance of the site terminal deviates from the standard distance, the higher the probability of the site terminal accessing the online conference. Acquiring the working bandwidth of the master , acquiring the total bandwidth of the local area network ; Allocating bit rates for live feeds ; limitation ; Acquiring a working bandwidth of a live end , calculating a lower bandwidth limit , is a bandwidth ratio; Computing an upper bandwidth , obtaining a bandwidth allocation interval .

3. The multi-protocol compatible device access and data management system of claim 2, wherein, The control instruction is executed for the on-site end with the online participant permission to optimize the definition and volume of the on-site end access to the online conference area. The online participant permission is set for the on-site end, and specifically, If , then the field end access line conference area receives audio and video; If , the on-site terminal is allowed to access the online conference area to receive audio and video, the on-site terminal is determined to have online participation permission, and clarity control is performed: Setting distance scale factor ; If , the clarity regulation ratio is calculated , ; , is the lowest clarity regulation ratio, is the adjustment intensity factor, the closer the participating distance is to the standard distance, the lower the clarity is, the closer , the clarity is higher; If , the clarity control ratio ; When the on-site end with the online participant permission is in the online conference area, if it is detected that the on-site end is set to external play of a loudspeaker, volume control is executed, and specifically, According to the distance of the field end, the volume is calculated , for avoiding the host end receiving the audio played by the field end, forming the mix, ; is a reference volume, is a distance attenuation coefficient, represents the attenuation of the volume with distance.

4. The multi-protocol compatible device access and data management system of claim 3, wherein, The volume control execution further comprises: In the conference room space, multiple measurement points are set and the noise sound pressure of the measurement points is collected in real time. The weighted average of the noise sound pressure of all the measurement points is recorded as ; The closer the measuring point is to the on-site end, the higher the weight is. The maximum volume of the live end is limited to , is a safety compensation coefficient for ensuring that the volume of the live end is higher than the ambient noise sound pressure; When multiple on-site ends are externally played in the conference room, the straight-line distance between any two on-site ends is calculated. If the straight-line distance is lower than the set safety distance, the external play audio of any on-site end is turned off until the straight-line distance of any on-site end is lower than the safety distance.

5. The multi-protocol compatible device access and data management system of claim 1, wherein, The offline speech recognition control module is used for identifying whether the on-site end with the microphone opened in the online conference area is active speaking when no offline speech prompt is received, and controlling the on-site end in the offline conference area according to the recognition result. The video picture, loudspeaker and microphone of the online conference area are independently controlled. The on-site participant is allowed to close the picture video and loudspeaker and only open the microphone to realize audio recording and transmission when participating in the online conference. When it is monitored that the on-site end opens the microphone in the offline conference room: Setting a sound pressure suppression factor for suppressing a sound pressure signal of the microphone initially; acquiring an initial sound pressure signal of a microphone , calculating , as a transmission signal; Monitoring the behavior of the on-site participant regulating the on-site end in a set time interval, when the on-site end increases the volume of the microphone, calculating the increase of the sound pressure signal in real time , setting a sound pressure gain threshold ; If then the behavior of opening the microphone is preliminarily identified as active speaking; When the on-site end does not increase the volume of the microphone within a time interval, the microphone is immediately closed.

6. The multi-protocol compatible device access and data management system of claim 5, wherein, The on-site end with the online participant permission is executed for the control instruction to optimize the definition and volume of the on-site end access to the online conference area. When it is preliminarily determined that the behavior of opening the microphone is active speaking, a verification strategy is executed, and specifically, A plurality of audio frame segments are obtained by collecting sound pressure signals of a fixed time length at a set sampling rate. extracting the amplitude for the first audio frame segment , calculating the energy smoothing the energy for determining whether the sound pressure signal is continuously present, , is a smoothing factor, the closer to 1, the smoother, is the smoothed energy of the first audio frame segment, is the smoothed energy of the first audio frame segment, is the smoothed energy of the first audio frame segment, is the smoothed energy of the first audio frame segment. calculating a language activity probability for the second audio frame segment , the closer the energy of the audio frame segment is to the noise energy, the higher the probability that the non-language input microphone, otherwise the probability that the language input microphone is highest; for a set noise energy, for an energy boost factor, as a factor by which the speech is boosted over the noise; Setting language activity probability threshold ; acquiring the number of audio frame segments, and calculating the total duration ; total length of the acquired sound pressure signal , calculating the language proportion ; If the language proportion is higher than the set proportion threshold, the behavior of opening the microphone is determined as the active speech; When identified as the active speaker, turn off the speakerphone in the field Speakerphone outside the distance.

7. The multi-protocol compatible device access and data management system of claim 4, wherein, The utility score of the two types of behaviors of the mobile host and the newly accessed on-site terminal is calculated under the premise of the mobile host, including: The host accesses the conference room camera and displays the conference room monitoring picture in real time, and the host locuses the position of the speaker in the monitoring picture; Acquiring distance from speaker location all live endpoints, for any live endpoint: Acquiring real-time bit rate of live site end , calculating available bandwidth ; Setting bandwidth thresholds ; Screening On-site endpoint computing utility score In particular: The noise sound pressure is normalized and updated as a normalized value; wherein, S is the signal-to-noise ratio, I is the interference adjustment coefficient, used to quantify the degree of suppression of the signal-to-noise ratio by the noise sound pressure, C is the normalized activation cost, B is the total amount of available bandwidth, used to punish overload, W is the set weight coefficient; Computing utility scores for mobile hosts In particular, wherein, is the estimated change in signal-to-noise ratio before and after the mobile master, is the delay caused by the movement, is the cost of movement, is the cost coefficient, indicates that the greater the signal gain, the lower the delay and cost, the higher the utility score, are all normalized; Computing confidence discount Revised utility score , , is a constant of calibration for normalization, is the trace of the covariance matrix, measuring the overall uncertainty, is the covariance matrix estimated from the speaker position, the confidence discount decreasing as the uncertainty of the speaker localization increases; calculating a revised utility score .

8. The multi-protocol compatible device access and data management system of claim 7, wherein, The speech mode is selected based on the utility score, including: The highest utility score is noted as The size relationship of and is compared; If The The corresponding field end access line on the conference and open microphone for speakers to use; If The master passes to the speaker for the speaker to use.

Images