Method for improving audio quality during voice conference of multiple terminals at same place

Abstract

The invention provides a method for improving the audio quality of a voice conference of multiple terminals at the same place. The method comprises the steps of time calibration, detection of equipment in the same room, audio quality calculation, delay detection, data collection and issuing, adjustment and playing of a dynamic buffer area, speaking button control and speaking sound processing. Time is unified through a time service server or a chip, equipment in the same room is detected, audio quality is calculated, delay is detected, equipment information is collected and issued, synchronous playing of a buffer area is adjusted, mutual exclusion of speaking buttons is controlled, speaking sound is processed to reduce howling and acoustic feedback, and use of the audio equipment is optimized. According to the invention, the audio synchronization and quality of the multi-terminal voice conference can be improved, howling and acoustic feedback are reduced, and the user experience is optimized.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and more specifically, to a method for improving audio quality during a voice conference of multiple terminals at the same location. Background Art

[0002] With the popularity of remote work and online education, the demand for multi-terminal voice conferencing in the same location is growing. Existing voice conferencing systems have shortcomings in audio synchronization, quality control, and user experience. Traditional audio processing technologies often rely on fixed buffer settings and simple audio quality assessments, which are difficult to adapt to different network conditions and dynamic changes in device performance. In addition, the lack of effective detection of devices in the same room and audio quality calculation mechanisms leads to frequent audio synchronization problems and howling, affecting the effectiveness of conferences.

[0003] In the process of implementing the embodiments of the present invention, the inventors found that the prior art has at least the following problems or defects: the prior art system is difficult to achieve accurate audio synchronization and quality control when processing multi-terminal voice conferences, and lacks an effective speech management mechanism, resulting in poor user experience. In addition, the prior art fails to fully utilize prediction algorithms and adaptive adjustment strategies to optimize the use of audio devices and improve overall audio quality. Summary of the invention

[0004] The present invention provides a method for improving the audio quality of a voice conference of multiple terminals at the same location, comprising:

[0005] First, time calibration is performed. The client and server times are unified through a timing server or timing chip, and a timestamp is carried in the data frame to provide an accurate timing basis for subsequent operations.

[0006] Then, the device detection in the same room is carried out. The client sends a request to the server, and the server sends a modulated audio. After the client plays it, the server analyzes the devices in the same room based on the feedback.

[0007] Then, the audio quality is calculated and the average signal-to-noise ratio of the speaker playback and microphone recording is obtained to evaluate the device audio performance.

[0008] Then, delay detection is performed to calculate the data transmission delay between the server and the client based on the interaction of data frames with timestamps.

[0009] The server then collects and sorts the device-related information and sends it to the client for reference.

[0010] When receiving voice data frames, some clients with high audio playback quality adjust the buffer to play synchronously according to the delay difference. The playback client is specified by the server.

[0011] The client has a speaking button. When one person in the same room speaks, the buttons for others are disabled, and they are automatically released after the speech ends.

[0012] When a user speaks, the server does not forward his or her voice to other clients in the same room, and the speaking user uses the client to amplify the sound, while technology is used to reduce howling and acoustic feedback.

[0013] Finally, the server selectively forwards the voice according to network delay and device quality, adjusts the buffer size, and switches the sound acquisition method according to conditions when the speaker is in the same room to optimize the use of audio equipment.

[0014] Furthermore, in the time calibration step, Beidou timing service or other suitable timing chip is used for calibration, and a timestamp is embedded in the frame of the transmitted data to ensure time uniformity, provide a unified time standard for subsequent precise calculation of delays, audio data processing, etc., and avoid problems such as audio synchronization caused by time differences.

[0015] Furthermore, in the step of detecting devices in the same room, the voice conferencing software client sends a detection request containing its own identification and other information to the server. The server modulates audio with a specific frequency and duration and sends it to the requesting client. After the client plays it, the server receives audio data fed back by each client, records information such as the reception time, and determines which clients are in the same room by analyzing these data, thereby providing device information in the same room for subsequent audio processing.

[0016] Furthermore, in the audio quality calculation step, for the speaker playback quality, the server first pre-processes the audio signal-to-noise ratio data returned by other clients in the same room, removes outliers and then calculates the average;

[0017] Regarding the microphone recording quality, the client collects audio multiple times, calculates the signal-to-noise ratio, and reports it after filtering and optimization. The server calculates the average based on this to comprehensively evaluate the device audio quality and provide a basis for audio device selection and optimization.

[0018] Furthermore, in the delay detection step, the voice conferencing software client and server interact using data frames with timestamps such as RTP, record the sending and receiving timestamps, calculate the time difference to obtain the data transmission delay, perform statistical analysis on multiple measured delay data, exclude abnormal values ​​and take the average value to ensure accurate delay detection and provide a basis for dynamic buffer adjustment.

[0019] Furthermore, in the data collection and sending steps, the voice conferencing software server summarizes the data delay information and audio quality scores, uses an efficient sorting algorithm to sort the audio quality scores of devices in the same room, and sends the device identification, audio quality score, delay information, etc. through data frames, so that the client can reasonably adjust the playback strategy based on this information.

[0020] Furthermore, in the dynamic buffer adjustment playback step, the server comprehensively considers factors such as device audio playback quality, network delay stability, and load to determine the number of clients playing synchronously in the same room, calculates the difference in audio playback delays of different clients, and proportionally increases the amount of buffer data of clients with smaller delays to achieve playback synchronization, ensure smooth audio playback, and enhance user audio experience.

[0021] Furthermore, in the speaking button control step, the speaking buttons of the voice conferencing software client are mutually exclusive. After one person in the same room presses the speaking button, his information (device identification, room identification, speaking request time, etc.) is sent to the server. The server disables the speaking buttons of others accordingly, and automatically lifts the disablement after the speech is finished, ensuring that the speech is carried out in an orderly manner.

[0022] like Figure 4 Furthermore, in the speech sound processing step, when a user speaks, the server does not forward his voice to other clients in the same room. The speaking user client amplifies the speaker and automatically adjusts the volume according to the ambient noise. It uses advanced echo cancellation algorithms such as adaptive filtering to adjust the filter parameters according to the real-time audio signal, reduce the probability of howling and suppress acoustic feedback, and improve the speech sound quality.

[0023] Furthermore, in the audio device selection and optimization step, the server selects the client to forward the voice according to the audio device's sound pickup and playback quality, uses a prediction algorithm to predict future trends based on historical delay data, adjusts the buffer size in advance to maintain sound synchronization, and switches the sound acquisition method according to the audio signal strength when in the same room with the speaker, reduces the number of audio playback devices in the room, optimizes the use of audio devices, and improves the overall audio quality.

[0024] The above-mentioned embodiments of the present invention have at least the following beneficial effects: The present invention can ensure the synchronization and stability of audio when multiple terminals conduct voice conferences at the same location through precise time calibration and detection of devices in the same room. By calculating audio quality and delay detection, the audio performance of the device can be effectively evaluated and optimized, and howling and acoustic feedback can be reduced, thereby improving the audio quality of the conference. In addition, the dynamic buffer adjustment playback and speech sound processing mechanism can further enhance the fluency and clarity of the audio, ensure that the speaker's voice is prioritized and optimized, and improve the audibility of the speech and the overall efficiency of the meeting.

[0025] The present invention can also select and optimize audio devices through the server, dynamically adjust the forwarding strategy according to network delay and device quality, so as to achieve the optimal allocation of audio resources. This flexible audio processing and optimization strategy can not only improve the audio quality of the meeting, but also enhance the user experience, making the remote voice conference more efficient and convenient. In addition, through the control of the speech button, the speaking order can be managed in an orderly manner to avoid speech confusion, further improving the organization and professionalism of the meeting. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, in which:

[0027] Figure 1 A schematic diagram of a device detection request in the same room provided by an embodiment of the present invention;

[0028] Figure 2 A schematic diagram of other client return recording provided by an embodiment of the present invention;

[0029] Figure 3 A schematic diagram of speech control and designated device playback provided by an embodiment of the present invention;

[0030] Figure 4 A schematic diagram of a client speaking in the same room provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.

[0032] Those skilled in the art know that the embodiments of the present invention can be implemented as a system, device, apparatus, method or computer program product. Therefore, the present invention can be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0033] It should be noted that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction and does not have any limiting meaning.

[0034] Reference below Figure 1 , Figure 1 The flowchart of the method for improving the audio quality of a voice conference of multiple terminals at the same location provided by an embodiment of the present invention is as follows. Figure 1 As shown, a method 100 for improving the audio quality of a voice conference in which multiple terminals are in the same location includes:

[0035] First, time calibration is performed. The client and server times are unified through a timing server or timing chip, and a timestamp is carried in the data frame to provide an accurate timing basis for subsequent operations.

[0036] Then, the device detection in the same room is carried out. The client sends a request to the server, and the server sends a modulated audio. After the client plays it, the server analyzes the devices in the same room based on the feedback.

[0037] Then, the audio quality is calculated and the average signal-to-noise ratio of the speaker playback and microphone recording is obtained to evaluate the device audio performance.

[0038] Then, delay detection is performed to calculate the data transmission delay between the server and the client based on the interaction of data frames with timestamps.

[0039] The server then collects and sorts the device-related information and sends it to the client for reference.

[0040] When receiving voice data frames, some clients with high audio playback quality adjust the buffer to play synchronously according to the delay difference. The playback client is specified by the server.

[0041] The client has a speaking button. When one person in the same room speaks, the buttons for others are disabled, and they are automatically released after the speech ends.

[0042] When a user speaks, the server does not forward his or her voice to other clients in the same room, and the speaking user uses the client to amplify the sound, while technology is used to reduce howling and acoustic feedback.

[0043] Finally, the server selectively forwards the voice according to network delay and device quality, adjusts the buffer size, and switches the sound acquisition method according to conditions when the speaker is in the same room to optimize the use of audio equipment.

[0044] It should be noted that time calibration is a key step to ensure audio synchronization. Timing servers or timing chips are devices used to provide accurate time signals. They can ensure time synchronization between the client and the server, which is essential for subsequent audio processing and synchronization. The data frame carries a timestamp, which means that each data packet contains a time mark, so that the sending and receiving time of the data packet can be accurately tracked, thereby calculating the transmission delay.

[0045] Specifically, the timing server can be a global positioning system (GPS) or other satellite timing systems, such as the Beidou system. The timing chip can be a hardware module integrated in the client device for receiving and processing time signals. The timestamp is usually a timestamp that records the exact time when the data frame is sent or received. In practical applications, the accuracy of the timestamp usually needs to reach the millisecond level to ensure the accuracy of audio synchronization.

[0046] Preferably, multiple timing services can be used as backup. For example, if a problem occurs in the main timing server, it can be immediately switched to the backup timing service to ensure the continuity and reliability of time calibration.

[0047] Furthermore, the generation and processing of timestamps can be implemented by using the existing Network Time Protocol (NTP) or other time synchronization protocols, which can provide high-precision time synchronization services. On the client device, a dedicated time synchronization module can be set up to receive the time signal and generate the timestamp to ensure that the time stamp of the data frame is accurate.

[0048] In some embodiments, in the time calibration step, Beidou timing service or other suitable timing chip is used for calibration, and a timestamp is embedded in the frame of the transmitted data to ensure time uniformity, provide a unified time standard for subsequent precise calculation of delays, audio data processing, etc., and avoid problems such as audio synchronization caused by time differences.

[0049] It should be noted that the use of Beidou timing service or other timing chips is to ensure time synchronization between the client and the server. Beidou timing service is a precise time service provided by the Beidou satellite navigation system, which can provide high-precision time signals to devices worldwide. The timing chip is a hardware integrated in the device for receiving and processing time signals. Embedding timestamp means adding a time mark to the data frame, which records the time when the data frame is sent or received, so that it can be used to calculate the delay of data transmission.

[0050] Specifically, Beidou timing service can provide millisecond-level time accuracy, which is critical for audio synchronization. Timing chips are usually integrated into client devices, such as smartphones, tablets, or conference systems. The generation of timestamps usually involves the collaborative work of hardware clocks and software algorithms to ensure the accuracy of time stamps. When setting parameters, network latency and device processing capabilities need to be taken into account to ensure that the generation and processing of timestamps does not introduce additional delays.

[0051] Preferably, multiple timing services can be used as redundancy, such as combining GPS and Beidou timing services to improve the reliability and accuracy of time synchronization. In addition, for the generation of timestamps, hardware acceleration methods can be used, such as using a dedicated hardware timer, to reduce the time overhead of software processing.

[0052] Furthermore, in the design of the data frame, the timestamp can be placed at the head or tail of the data frame to facilitate rapid identification and processing during network transmission. For the format of the timestamp, an international standard time format, such as ISO 8601, can be used to ensure compatibility and readability between different systems.

[0053] like Figure 1 , Figure 2In some embodiments, in the same-room device detection step, the voice conferencing software client sends a detection request containing its own identification and other information to the server, and the server modulates audio with a specific frequency and duration and sends it to the requesting client. After the client plays it, the server receives audio data fed back by each client, records information such as the receiving time, and determines which clients are in the same room by analyzing these data, providing same-room device information for subsequent audio processing.

[0054] It should be noted that same-room device detection is a technical means for identifying client devices participating in a voice conference in the same physical space. The client device sends a detection request containing its own identification information so that the server can identify and respond. The audio modulated by the server has a specific frequency and duration to ensure that the audio signal can be accurately identified and processed by the client device. After the client plays the audio, the server can determine which devices are in the same room by analyzing the audio data fed back by the client, which is crucial for subsequent audio processing and optimization.

[0055] Specifically, the client identification information may include the IP address, MAC address, or unique device ID of the device. The audio modulated by the server may use specific signal processing techniques, such as sine wave modulation or pulse coding, to ensure the uniqueness and identifiability of the audio signal. The selection of specific frequencies and durations needs to take into account the acoustic characteristics of the room, as well as possible interference factors, to ensure effective transmission and reception of audio signals. When receiving audio data fed back by the client, the server needs to record information such as the reception time and signal strength, and analyze this data through algorithms to determine the relative position and distance between the clients.

[0056] Preferably, the server can use a machine learning algorithm to analyze the audio data to improve the accuracy and efficiency of the detection of devices in the same room. In addition, in order to enhance the robustness of the detection, a variety of audio signal modulation techniques can be used, such as multi-frequency signal modulation or frequency hopping technology, to reduce the interference and error that may be caused by a single signal modulation.

[0057] Furthermore, on the client device, advanced audio processing techniques such as adaptive filtering or noise suppression can be used to improve the clarity and intelligibility of the audio signal. In addition, to further optimize the audio processing, other sensor data such as Wi-Fi signal strength or Bluetooth signal can be used in combination to assist in determining the relative position of the client device.

[0058] In some embodiments, in the audio quality calculation step, for the speaker playback quality, the server first pre-processes the audio signal-to-noise ratio data returned by other clients in the same room, removes outliers and then calculates the average;

[0059] Regarding the microphone recording quality, the client collects audio multiple times, calculates the signal-to-noise ratio, and reports it after filtering and optimization. The server calculates the average based on this to comprehensively evaluate the device audio quality and provide a basis for audio device selection and optimization.

[0060] It should be noted that audio quality calculation is the process of evaluating the performance of speaker playback and microphone recording. Signal-to-Noise Ratio (SNR) is an important indicator for measuring the quality of audio signals. It represents the ratio of signal strength to background noise strength. The server preprocesses the audio signal-to-noise ratio data returned by the client, mainly to eliminate possible outliers, which may be caused by equipment failure, environmental interference or other atypical situations. By removing these outliers and calculating the mean, a more accurate and reliable audio quality evaluation result can be obtained.

[0061] Specifically, the calculation of the signal-to-noise ratio can be achieved by analyzing the power spectral density of the audio signal, where the ratio of the signal power to the noise power is the signal-to-noise ratio. In actual operation, a threshold can be set to identify and remove outliers. For example, if the signal-to-noise ratio of a client is much lower than that of other clients, then this value may be considered abnormal. For the evaluation of microphone recording quality, the client can use the built-in audio processing algorithm to collect audio and calculate the signal-to-noise ratio, and then report this data to the server. After receiving this data, the server can apply a filtering algorithm to optimize the data, such as using a low-pass filter to remove high-frequency noise, and then calculate the mean.

[0062] Preferably, the server can use more advanced data processing techniques, such as machine learning algorithms, to automatically identify and remove outliers, thereby improving the accuracy of audio quality assessment. In addition, for the assessment of microphone recording quality, the client can use adaptive filtering technology to adjust the filtering parameters in real time to adapt to different environmental noise conditions.

[0063] Furthermore, on the server side, statistical analysis methods can be used to evaluate the distribution of signal-to-noise ratio data to better understand the overall trend of audio quality. In addition, in order to further improve the reliability of audio quality assessment, a user feedback mechanism can be introduced to allow users to make subjective evaluations of audio quality, and then these evaluations can be combined with objective measurement results to obtain a more comprehensive audio quality assessment.

[0064] In some embodiments, in the delay detection step, the voice conferencing software client and server interact using data frames with timestamps such as RTP, record the sending and receiving timestamps, calculate the time difference to obtain the data transmission delay, perform statistical analysis on multiple measured delay data, exclude abnormal values ​​and take the average value, to ensure accurate delay detection and provide a basis for dynamic buffer adjustment.

[0065] It should be noted that the RTP (Real-time Transport Protocol) mentioned here is a network protocol specially designed for transmitting real-time data such as audio and video on the Internet. In voice conferencing, the RTP protocol helps calculate the delay of data transmission by embedding timestamps in data frames. The timestamp records the sending and receiving time of the data frame. By comparing these timestamps, the transmission delay of the data in the network can be calculated. This delay detection is crucial for adjusting the buffer size and optimizing audio synchronization.

[0066] Specifically, the RTP protocol contains a timestamp field in each data packet, which records the time when the data packet was sent. The client and server will record the timestamp when sending and receiving data packets. By calculating the difference between the sending and receiving timestamps, the transmission delay of the data packet can be obtained. In order to improve the accuracy of delay measurement, statistical analysis is usually performed on the results of multiple measurements, outliers are excluded, and the average delay is calculated. This average value will be used as the basis for dynamic buffer adjustment to ensure smooth playback and synchronization of audio data.

[0067] Preferably, more advanced statistical methods can be used to process the delay data, such as using a moving average or a weighted average to reduce the fluctuation effect of a single measurement. In addition, in order to further improve the accuracy of delay detection, additional network monitoring tools can be introduced to monitor the network status in real time, so as to adjust the buffer size more accurately.

[0068] Furthermore, in some cases, if the network conditions are unstable, you can consider using QoS (Quality of Service) strategies to prioritize audio packets to reduce latency and jitter. In addition, you can also use adaptive buffer management strategies to dynamically adjust the buffer size according to real-time network conditions to adapt to different network environments and user needs.

[0069] In some embodiments, in the data collection and sending steps, the voice conferencing software server summarizes the data delay information and audio quality scores, uses an efficient sorting algorithm to sort the audio quality scores of devices in the same room, and sends the device identification, audio quality score, delay information, etc. through data frames, so that the client can reasonably adjust the playback strategy based on this information.

[0070] It should be noted that the server plays a core role in voice conferencing, responsible for collecting and processing data from various clients. Data delay information refers to the time difference between sending and receiving data, while the audio quality score is a quantitative evaluation of the client's audio performance. The server aggregates this information and uses an efficient sorting algorithm to sort the devices to ensure that devices with higher audio quality are processed first. The server then sends this key information to the client through data frames, and the client can adjust its playback strategy based on this information to optimize the overall audio experience.

[0071] Specifically, data delay information can be obtained by measuring the sending and receiving timestamps of data packets, and audio quality scores can be determined by analyzing parameters such as signal-to-noise ratio. Efficient sorting algorithms can be quick sort, merge sort, or heap sort, which can sort a large amount of data in a shorter time.

[0072] More specifically, in addition to the device identification, audio quality score, and delay information, the data frame can also include other relevant information, such as the device's network status, current load, etc., to help the client make more comprehensive playback strategy adjustments. The client can adjust the buffer size, playback priority, etc. based on this information to ensure smooth audio playback and synchronization.

[0073] Preferably, the server can use machine learning algorithms to dynamically adjust the sorting strategy and optimize the calculation of the audio quality score according to the real-time network conditions and device performance. In addition, the server can provide a real-time feedback mechanism, allowing the client to report to the server based on the current playback effect, and the server can further adjust the content of the data frame sent accordingly.

[0074] Furthermore, on the client side, an adaptive buffer management strategy can be used to dynamically adjust the buffer size based on the information sent by the server to adapt to different network conditions and user needs. In addition, the client can also use advanced audio processing technologies such as adaptive filtering or noise suppression to further improve audio quality.

[0075] In some embodiments, in the dynamic buffer adjustment playback step, the server comprehensively considers factors such as device audio playback quality, network delay stability, and load to determine the number of clients playing synchronously in the same room, calculates the difference in audio playback delays of different clients, and proportionally increases the amount of buffer data for clients with smaller delays to achieve playback synchronization, ensure smooth audio playback, and enhance the user audio experience.

[0076] It should be noted that dynamic buffer adjustment playback means that the server dynamically adjusts the client's buffer size based on factors such as audio playback quality, network latency, and system load monitored in real time to ensure the synchronization and smoothness of audio playback. The buffer here refers to the area where data is temporarily stored during data transmission, which is used to smooth data flow and reduce playback interruptions caused by network fluctuations. By calculating the difference in audio playback delay between different clients, the server can determine which clients need to increase the amount of buffer data to reduce delay differences and achieve synchronous playback.

[0077] Specifically, the server can evaluate the audio playback quality by monitoring the audio playback status and network status of each client in real time. Network delay stability can be evaluated by measuring the round-trip time (RTT) of data packets, while system load can be determined by monitoring CPU and memory usage.

[0078] More specifically, the server can set a threshold, and when the client's delay exceeds this threshold, the amount of data in its buffer is increased. The increase ratio of the buffer data volume can be determined according to the size of the delay difference. For example, the larger the delay difference, the larger the increase ratio. In addition, the server can also adjust the buffer size according to the client's playback priority, giving priority to ensuring the audio playback quality of important clients.

[0079] Preferably, the server can use a more sophisticated algorithm to dynamically adjust the buffer size, for example, using an adaptive buffer management strategy to dynamically adjust the buffer size based on network conditions and user behavior. In addition, the server can also introduce a prediction model to predict future network conditions and load changes based on historical data, so as to adjust the buffer size in advance.

[0080] Furthermore, on the client side, advanced audio processing techniques such as adaptive filtering or noise suppression can be used to further improve audio quality. In addition, the client can also provide a user interface to allow users to adjust the buffer size according to their needs to obtain the best audio experience.

[0081] like Figure 3 In some embodiments, in the speech button control step, the speech buttons of the voice conference software client are mutually exclusive. When one person in the same room presses the speech button, his information (device identification, room identification, speech request time, etc.) is sent to the server, and the server disables the speech buttons of other people accordingly. After the speech is finished, the disablement is automatically lifted to ensure that the speech is carried out in an orderly manner.

[0082] It should be noted that mutual exclusivity means that only one client can control the right to speak at the same time. In a multi-terminal voice conference, this mechanism can prevent audio conflicts and confusion caused by multiple users speaking at the same time. When a user presses the speak button, the client sends relevant information to the server, including device identification, room identification, and speaking request time. After receiving this information, the server will disable the speak buttons of other users based on this data to ensure the order of speaking. This mechanism can effectively manage the speaking order and improve the efficiency and quality of the meeting.

[0083] Specifically, the device ID can be a unique identifier of the client device, such as a MAC address or device serial number; the room ID refers to the virtual space ID where the voice conference is located, which is used to distinguish different conferences; the speech request time is the specific time when the user requests to speak. After receiving the speech request, the server needs to verify and process this information to ensure the correct allocation of the speech right.

[0084] More specifically, the server can manage the speaking order by setting up a speaking queue. When a user finishes speaking, the server will automatically remove the speaking ban status of other users and allow them to request to speak. In addition, the server can also adjust the speaking order according to the priority and importance of the speech to meet different meeting needs.

[0085] Preferably, the server can adopt a more advanced speech management strategy, such as speech control based on user roles and permissions. For example, the conference host or senior users can have a higher speech priority, or have the final decision-making power on speech in certain situations. In addition, the server can also provide a visual speech management interface so that users can clearly see the speech queue and their own speech status.

[0086] Furthermore, on the client side, a more intuitive user interface design can be adopted, such as using colors or icons to indicate the speaking status, to improve the user experience. In addition, the client can also provide a preview function for speech requests, allowing users to preview their speech content before formally speaking to ensure the accuracy and effectiveness of the speech.

[0087] In some embodiments, in the speech sound processing step, when a user speaks, the server does not forward his voice to other clients in the same room. The speaking user client amplifies the speaker and automatically adjusts the volume according to the ambient noise. Advanced echo cancellation algorithms such as adaptive filtering are used to adjust the filter parameters according to the real-time audio signal to reduce the probability of howling and suppress acoustic feedback, thereby improving the quality of the speech sound.

[0088] It should be noted that when a user speaks in a voice conference, in order to improve the clarity of the speech and avoid sound interference, the server will not forward the voice of the speaking user to other clients in the same room. The speaking user's client will perform amplification, which means that the speaking user's audio will be amplified and played through the speaker so that it can be clearly heard in the room. At the same time, the client will automatically adjust the volume according to the ambient noise to ensure that the speaking voice is neither drowned out by the noise nor uncomfortable due to excessive volume. In addition, the use of adaptive filtering algorithms can adjust the filter parameters in real time to eliminate echoes and acoustic feedback, thereby reducing the probability of howling and improving the sound quality of the speech.

[0089] Specifically, the sound amplification operation involves the amplification of the audio signal, which is usually achieved through the audio processing unit of the client device. The detection of environmental noise can be done through the built-in microphone, and the automatic adjustment of the volume relies on the real-time analysis of the noise level by the algorithm. The adaptive filtering algorithm is a technology that dynamically adjusts the filter parameters. It can optimize the filtering effect based on the characteristics of the real-time audio signal, thereby effectively suppressing echoes and acoustic feedback. This algorithm usually includes an adaptive filter (such as a minimum mean square error filter) and a corresponding control algorithm for adjusting the filter coefficients to minimize the error signal.

[0090] Preferably, more advanced audio processing techniques, such as beamforming or sound source localization techniques, can be used to further improve the clarity of spoken voices and isolate background noise. In addition, machine learning algorithms can be introduced to optimize the performance of the adaptive filter so that it can more accurately identify and eliminate unwanted sounds.

[0091] Furthermore, on the client device, a user interface can be provided to allow users to manually adjust the volume and audio settings to suit different usage scenarios and personal preferences. In addition, a feedback mechanism can be designed to allow users to report speech quality issues, and the server can adjust the audio processing strategy accordingly to improve the overall audio experience.

[0092] In some embodiments, in the audio device selection and optimization step, the server selects the client to forward the voice according to the audio device's sound pickup and playback quality, uses a prediction algorithm to predict future trends based on historical delay data, adjusts the buffer size in advance to maintain sound synchronization, and switches the sound acquisition method based on the audio signal strength when in the same room with the speaker, reduces the number of audio playback devices in the room, optimizes the use of audio devices, and improves the overall audio quality.

[0093] It should be noted that when the server processes the audio data of the voice conference, it will select which client audio should be forwarded based on the pickup and playback quality of the audio device. The pickup quality here refers to the ability of the microphone to capture sound, while the playback quality refers to the effect of the speaker playing the sound. Through the prediction algorithm, the server can predict the future network delay trend based on historical data and adjust the buffer size accordingly to maintain the synchronous playback of the audio. When the speaker is in the same room as the server, the server will decide whether to use the pickup device or the playback device to obtain the sound based on the strength of the audio signal. This can reduce unnecessary audio playback devices in the room, thereby optimizing the use of audio devices and improving the overall audio quality.

[0094] Specifically, the pickup and playback quality of audio devices can be evaluated by a series of parameters, such as signal-to-noise ratio, frequency response, dynamic range, etc. The prediction algorithm can be time series analysis, machine learning models, or other statistical methods that can analyze historical delay data and predict future delay changes. The buffer size needs to be adjusted dynamically based on the predicted delay changes to ensure the continuity and synchronization of audio data. The switching mechanism of audio signal strength needs to be able to respond quickly to environmental changes, such as the movement of the speaker's position or changes in the noise level in the room. This may involve real-time monitoring of the strength of the audio signal and switching the pickup and playback devices based on preset thresholds.

[0095] Preferably, the server can adopt more advanced audio processing technology, such as adaptive encoding and decoding, to further optimize the transmission and processing of audio data. In addition, the server can introduce a user feedback mechanism to allow users to report audio quality problems, and the server adjusts the audio processing strategy accordingly.

[0096] Furthermore, on the client side, a user interface can be provided to allow users to manually adjust audio settings, such as volume, equalizer, etc., to suit different usage scenarios and personal preferences. In addition, an intelligent audio device management strategy can be designed to automatically select the most appropriate audio device for pickup and playback based on the user's usage habits and the type of meeting. In extreme cases, such as when the network conditions are extremely poor, a degradation strategy can be adopted, such as reducing audio quality or switching to an alternate audio transmission channel, to ensure the convening of the meeting.

[0097] Furthermore, the storage medium of the embodiment of the present application stores program instructions that can implement all the above methods, wherein the program instructions can be stored in the above storage medium in the form of a software product, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.

[0098] The above descriptions are only some preferred embodiments of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present invention is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present invention to form a technical solution.

Claims

1. A method for improving the audio quality of a voice conference of multiple terminals at the same location, characterized in that: The main steps include: Perform time calibration, unify the client and server time through the timing server or timing chip and carry timestamps in the data frame to provide an accurate timing basis for subsequent operations; Carry out detection of devices in the same room. The client sends a request to the server, and the server sends a modulated audio. After the client plays it, the server analyzes the devices in the same room based on the feedback. Calculate audio quality and obtain the average signal-to-noise ratio of speaker playback and microphone recording to evaluate device audio performance; Then, delay detection is performed to calculate the data transmission delay between the server and the client based on the interaction of data frames with timestamps; The server collects and sorts device-related information and sends it to the client for reference; When receiving voice data frames, some clients with high audio playback quality adjust the buffer to play synchronously according to the delay difference. The playback client is specified by the server. The client has a speaking button. When one person in the same room speaks, the buttons of others are disabled, and they are automatically released after the speech. When a user speaks, the server does not forward his voice to other clients in the same room, and the speaking user uses the client to amplify the sound, while technology is used to reduce howling and acoustic feedback; The server selectively forwards voice according to network latency and device quality, adjusts the buffer size, and conditionally switches the sound acquisition method when the speaker is in the same room to optimize the use of audio equipment.

2. The method for improving the audio quality of a voice conference of multiple terminals at the same location according to claim 1, characterized in that: When the time calibration is performed, the Beidou timing service chip is used for calibration, and a timestamp is embedded in the frame of the transmitted data to ensure time uniformity.

3. The method for improving the audio quality of a voice conference of multiple terminals at the same location according to claim 1, characterized in that: When detecting devices in the same room, the voice conferencing software client sends a detection request containing its own identification and other information to the server. The server modulates audio with a specific frequency and duration and sends it to the requesting client. After the client plays it, the server receives audio data fed back by each client, records information such as the reception time, and determines which clients are in the same room by analyzing the audio data.

4. The method for improving the audio quality of a voice conference of multiple terminals at the same location according to claim 1, characterized in that: When calculating the audio quality, for the speaker playback quality, the server pre-processes the audio signal-to-noise ratio data returned by other clients in the same room, removes outliers and calculates the average; Regarding the microphone recording quality, the client collects audio multiple times, calculates the signal-to-noise ratio, and reports it after filtering and optimization. The server calculates the average based on this to comprehensively evaluate the device audio quality.

5. The method for improving the audio quality of a voice conference of multiple terminals at the same location according to claim 1, characterized in that: During the delay detection, the voice conference software client and server interact using data frames with timestamps such as RTP, record the sending and receiving timestamps, calculate the time difference to obtain the data transmission delay, perform statistical analysis on multiple measured delay data, exclude abnormal values ​​and take the average value.

6. The method for improving the audio quality of a voice conference of multiple terminals at the same location according to claim 1, characterized in that: The method also includes that the voice conference software server aggregates data delay information and audio quality scores, uses an efficient sorting algorithm to sort the audio quality scores of devices in the same room, and sends the device identification, audio quality score, delay information, etc. through data frames, so that the client adjusts the playback strategy.

7. The method for improving the audio quality of a voice conference of multiple terminals at the same location according to claim 1, characterized in that: When adjusting the buffer size, the server comprehensively considers the audio playback quality of the device, the stability of the network delay and the load factors to determine the number of clients playing synchronously in the same room, calculates the difference in audio playback delays of different clients, and proportionally increases the amount of buffer data of the client with smaller delay to achieve playback synchronization.

8. The method for improving the audio quality of a voice conference of multiple terminals at the same location according to claim 1, characterized in that: The speaking buttons of the voice conferencing software client are mutually exclusive. When one person in the same room presses the speaking button, the corresponding information is sent to the server, and the server disables the speaking buttons of other people accordingly. The disablement is automatically lifted after the speech is finished to ensure that the speech proceeds in an orderly manner. The information includes: device identification, room identification, and speaking request time.

9. The method for improving the audio quality of a voice conference of multiple terminals at the same location according to claim 1, characterized in that: When a user speaks, the server does not forward his or her voice to other clients in the same room. The speaking user's client amplifies the speaker and automatically adjusts the volume according to the ambient noise. An adaptive filtering algorithm is used to adjust the filter parameters according to the real-time audio signal to reduce the probability of howling and suppress acoustic feedback.

10. The method for improving the audio quality of a voice conference of multiple terminals at the same location according to claim 1, characterized in that: The server selects the client to forward the voice according to the audio device's sound pickup and playback quality. It uses a prediction algorithm to predict future trends based on historical delay data and adjusts the buffer size in advance to maintain sound synchronization. When the speaker is in the same room, the sound acquisition method is switched according to the audio signal strength to reduce the number of audio playback devices in the room.

Images