Conference pickup method, terminal, system and computer storage medium

Abstract

The embodiment of the invention provides a conference pickup method, a terminal, a system and a computer storage medium, and the method comprises the steps: a main conference terminal carries out the collection of audio data in a conference process, and obtains main audio data; receiving slave audio data collected for the conference and sent by a slave conference terminal corresponding to the master conference terminal; obtaining time delay information between the master conference terminal and the slave conference terminal, and performing time alignment processing on the master audio data and the slave audio data based on the time delay information to obtain aligned master audio data and aligned slave audio data; and selecting target audio data from the aligned main audio data and the aligned slave audio data to obtain a conference pickup result based on the target audio data. According to the embodiment of the invention, the method can effectively improve the definition of conference pickup, provides high-quality audio experience, and enables participants to be more focused on the conference content.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a conference audio pickup method, terminal, system, and computer storage medium. Background Technology

[0002] Intelligent conferencing technology is a conferencing solution that integrates modern information technology to improve meeting efficiency, participation, and interactivity. It supports features such as remote video conferencing, allowing participants to join meetings via internet connection.

[0003] In meeting rooms that support smart conferencing technologies, regardless of room size or the distance of speakers from the conferencing equipment, ensuring clear and natural audio pickup is paramount. Traditional methods typically involve designing microphone arrays on hardware or using additional desktop microphones. However, these methods have inherent limitations. For example, microphone arrays are complex and expensive to design, and desktop microphone placement is susceptible to interference and limitations, especially in large meeting rooms where ensuring clear capture of all participants' speech is a significant challenge. Due to these issues, regardless of the audio processing algorithms employed, the final sound pickup quality often falls short of expectations.

[0004] Therefore, finding a solution that can provide high-quality audio pickup under various conditions has become an urgent need for the industry. Summary of the Invention

[0005] In view of this, embodiments of this application provide a conference audio pickup solution to at least partially solve the above-mentioned problems.

[0006] According to a first aspect of the embodiments of this application, a conference audio pickup method is provided, applied to a main conference terminal, including:

[0007] During the meeting, audio data is collected to obtain the main audio data; the secondary audio data collected for the meeting is received from the secondary meeting terminal corresponding to the main meeting terminal;

[0008] The time delay information between the main conference terminal and the slave conference terminal is obtained, and the main audio data and the slave audio data are time aligned based on the time delay information to obtain aligned main audio data and aligned slave audio data.

[0009] After alignment, target audio data is selected from the master audio data and the audio data after alignment to obtain the conference sound pickup result based on the target audio data.

[0010] According to a second aspect of the embodiments of this application, a conference audio pickup method is provided, applied from a conference terminal, the method comprising:

[0011] Audio data is collected during the meeting to obtain audio data;

[0012] In response to detecting a main conference terminal in the same conference as the secondary conference terminal, the secondary audio data is sent to the main conference terminal, so that the main conference terminal performs time alignment processing on its own collected main audio data and the secondary audio data based on time delay information, and selects target audio data from the aligned main audio data and aligned secondary audio data to obtain the conference sound pickup result based on the target audio data.

[0013] According to a third aspect of the embodiments of this application, a master conference terminal is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus; the memory is used to store at least one executable instruction, wherein the executable instruction causes the processor to perform an operation corresponding to the method described in the first aspect.

[0014] According to a fourth aspect of the embodiments of this application, a conferencing terminal is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, the executable instruction causing the processor to perform an operation corresponding to the method described in the second aspect.

[0015] According to a fifth aspect of the embodiments of this application, a conference system is provided, including: a main conference terminal as described in the third aspect above, and a slave conference terminal as described in the fourth aspect above.

[0016] According to a sixth aspect of the embodiments of this application, a computer storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method as described in the first or second aspect.

[0017] According to the conference audio pickup scheme provided in the embodiments of this application, in addition to collecting audio data through the main terminal device, audio data is also collected through slave terminal devices located at different locations during the conference. After obtaining the main audio data collected by the main terminal device and the slave audio data collected by the slave terminal devices, the main audio data and the slave audio data are time aligned. Then, the aligned audio data are compared, and the target audio data with higher audio quality is selected. The conference audio pickup result is obtained based on the target audio data.

[0018] On the one hand, the embodiments of this application use a main conference terminal and secondary conference terminals located in different positions to jointly pick up audio, and unify the audio data collected by multiple conference terminals to the main conference terminal for time alignment processing and audio selection processing, thereby effectively improving the audio quality of the final audio data, ensuring clear and natural audio pickup, and enhancing the conference experience of participants.

[0019] On the other hand, as mentioned above, this embodiment uses multiple conference terminals for joint audio pickup. The transmission of audio data between terminals depends on the network status between them. Network jitter, large delays, and data out-of-order issues can severely impact the final audio pickup effect. Furthermore, different conference terminals typically have significant differences in their clock oscillators, which can cause data offset between the audio data acquired by different terminals, also negatively affecting the audio pickup effect. Considering the above problems, in this embodiment, after acquiring each audio data stream, the main conference terminal first performs time alignment processing on each audio data stream to ensure that each audio data stream can be processed subsequently under the same time base, thus eliminating the aforementioned data offset. Therefore, the audio quality of the final audio data can be further improved.

[0020] In summary, the embodiments of this application can effectively improve the clarity of meeting audio pickup, provide a high-quality audio experience, and enable participants to focus more on the meeting content. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0022] Figure 1 A schematic diagram of an exemplary conference system to which the conference audio pickup method of the present application is applicable;

[0023] Figure 2 A schematic diagram of another exemplary conference system to which the conference audio pickup method of the present application is applicable;

[0024] Figure 3 This is a flowchart illustrating the steps of a conference sound pickup method according to Embodiment 1 of this application;

[0025] Figure 4 This is a flowchart illustrating the time alignment process in an embodiment of this application.

[0026] Figure 5 This is a flowchart illustrating the steps of a conference sound pickup method according to Embodiment 2 of this application;

[0027] Figure 6 This is a schematic diagram of the scenario corresponding to the terminal detection process.

[0028] Figure 7 A schematic diagram of the interaction process between the master conference terminal and the slave conference terminals;

[0029] Figure 8 This is a structural block diagram of a conference microphone according to Embodiment 3 of this application;

[0030] Figure 9 This is a structural block diagram of a conference microphone according to Embodiment 4 of this application;

[0031] Figure 10 This is a schematic diagram of the structure of a master conference terminal according to Embodiment 5 of this application;

[0032] Figure 11 This is a schematic diagram of the structure of a conference terminal according to Embodiment Six of this application. Detailed Implementation

[0033] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this application.

[0034] The specific implementation of the embodiments of this application will be further described below with reference to the accompanying drawings.

[0035] See Figure 1 , Figure 1 This is a schematic diagram of an exemplary conference system to which the conference audio pickup method of this application is applicable. For ease of understanding, it is first combined with... Figure 1 The application scenarios of the conference audio pickup method provided in the embodiments of this application will be explained.

[0036] like Figure 1 As shown, the system 100 may include: a main conference terminal 102, and one or more slave conference terminals 104. The main conference terminal 102 can be a conference terminal set up in a conference room, while the slave conference terminals can be any device with this conference application installed, such as mobile terminals used by participants, such as mobile phones, laptops, etc. This system can be used in local conference audio pickup scenarios, specifically:

[0037] During the audio pickup process, the main conference terminal 102 and the slave conference terminals 104 are typically located in the same acoustic space (such as the same conference room). The main conference terminal 102 and the slave conference terminals 104 are connected via a network; for example, they can be located on the same wireless local area network. Both the main conference terminal 102 and each slave conference terminal 104 can use their configured microphones or other audio pickup devices to collect sound from the conference room and convert it into corresponding audio data. For ease of distinction, the audio data collected by the main conference terminal 102 can be referred to as the main audio data, and the audio data collected by the slave conference terminals 104 can be referred to as the slave audio data. The slave conference terminals 104 can send the collected slave audio data to the main conference terminal 104, which selects the audio data with better sound quality from the various audio data streams as the target audio data. The main conference terminal 104 then performs audio processing on this target audio data, including echo cancellation, noise suppression, and automatic gain adjustment, to obtain the final conference audio pickup result. Afterward, subsequent operations can be performed based on the conference audio pickup result, such as speech-to-text transcription, translation, and automatic speech recognition operations like subtitle generation.

[0038] See Figure 2 , Figure 2 This is a schematic diagram of another exemplary conference system to which the conference audio pickup method of this application is applicable. Figure 1 compared to, Figure 2 The conference system in the middle can be applied to remote conferencing scenarios. Figure 2 The conference system shown may include multiple terminals, wherein at least one terminal consists of a master conference terminal and slave conference terminals. For example, Figure 2 The diagram shows two terminals: Terminal 1 and Terminal 2. Terminal 1 includes a master conference terminal and a slave conference terminal, while Terminal 2 can consist of a master conference terminal and a slave conference terminal, or it can be a single conference terminal. Each terminal in the conference system is connected to server 106 via a network, thereby providing remote conferencing services to users through data transmission from server 106.

[0039] As far as the terminal is concerned, its specific sound pickup process is the same as above. Figure 1 The audio pickup process of the conference systems shown is the same, the difference being that... Figure 2 After receiving the audio pickup results, the terminal shown can transmit the audio data to the server 106 via the network. The server 106 then forwards the audio data to the remote terminal. The remote conference terminal processes the received audio data and plays it through the amplification device.

[0040] Example 1

[0041] Reference Figure 3 , Figure 3This is a flowchart illustrating the steps of a conference audio pickup method according to Embodiment 1 of this application. The conference audio pickup method provided in this embodiment can be... Figure 1 or Figure 2 The main conference terminal 102 in the system shown is executing.

[0042] Specifically, the conference audio pickup method provided in this application embodiment may include the following steps:

[0043] Step 302: During the meeting, audio data is collected to obtain the main audio data; the secondary audio data collected for the meeting is received from the secondary meeting terminal corresponding to the main meeting terminal.

[0044] Specifically, during the meeting, the main conference terminal can collect conference audio data through its built-in microphone. Simultaneously, the secondary conference terminal can also collect audio data from the same meeting through its own built-in microphone. For ease of distinction, in this embodiment, the conference audio data collected by the main conference terminal is referred to as main audio data, while the conference audio data collected by the secondary conference terminal is referred to as secondary audio data.

[0045] Step 304: Obtain the time delay information between the main conference terminal and the slave conference terminal, and perform time alignment processing on the main audio data and slave audio data based on the time delay information to obtain aligned main audio data and aligned slave audio data.

[0046] Specifically, in this embodiment, the master conference terminal and the slave conference terminals are connected via a network. Therefore, the transmission of audio data between the terminals depends on the network status between them. Problems such as sudden network jitter, large delays, and out-of-order data affect the timeliness of the master conference terminal receiving the slave audio data. In addition, the master conference terminal and the slave conference terminal are two different types of conference terminals, and their clock crystal oscillators usually have significant differences. This difference will also cause data offset between the audio data collected by different terminals, that is, the timestamp of the same signal is different in the audio data collected by different terminals. For example, if a user outputs the voice "Today's meeting topic is XXX", the start timestamp of the above voice is T1 in the master audio data collected by the master conference terminal, while the start timestamp of the above voice is T2 in the slave audio data collected by the slave conference terminal. T1 and T2 are not equal.

[0047] Considering the above problems, in this embodiment, after acquiring each audio data stream, the main conference terminal can perform time alignment processing on each audio data stream to map them to the same time base, thereby eliminating the aforementioned data offset. Using the above example again, the process of performing time alignment processing on each audio data stream to eliminate the aforementioned data offset is as follows: The time delay information between the main conference terminal and the slave conference terminal is obtained, which is equal to the difference between T1 and T2. After obtaining this difference, the time base of either the main audio data or the slave audio data can be modified according to this difference, so that the starting timestamp of the user's voice "Today's meeting topic is XXX" in the main audio data is the same as the starting timestamp of the user's voice "Today's meeting topic is XXX" in the slave audio data. For example, the starting timestamps can all be changed to T1, or they can all be changed to T2, or they can all be changed to any value other than T1 and T2.

[0048] By performing time alignment on the audio data in this step, the audio data collected by each conference terminal can be synchronized efficiently and accurately, providing a reliable data foundation for subsequent audio data processing and improving the accuracy and efficiency of audio data processing.

[0049] Furthermore, after acquiring the audio data from each channel, this embodiment performs time alignment processing in this step to eliminate data offset. Therefore, regardless of whether there is data offset between the originally acquired audio data channels, and regardless of the severity of the data offset, this embodiment can efficiently and accurately eliminate the aforementioned offset, obtaining audio data under the same time reference. Therefore, this embodiment has lower requirements for the network transmission link between the master and slave conferencing terminals, and also lower requirements for subsequent audio processing algorithms. In other words, this embodiment can ensure that the conferencing system has high robustness and stability in various complex conferencing scenarios.

[0050] In this embodiment, the time delay information between the master conference terminal and the slave conference terminal can characterize the time difference between them. Before performing time alignment processing, the aforementioned time delay information can be obtained first, and then time alignment processing can be performed based on this information.

[0051] Step 306: Select target audio data from the aligned main audio data and the aligned sub-audio data to obtain the conference sound pickup result based on the target audio data.

[0052] Specifically, after time alignment processing of each audio data stream, the audio data stream with better audio quality can be selected as the target audio data. Then, the target audio data is processed by echo cancellation (AEC), automatic gain control (AGC), and noise suppression (NS), i.e., the 3A algorithm, to obtain the conference audio pickup result.

[0053] Specifically, the 3A algorithm in audio data processing is a set of key technologies used to improve the quality and communication effects of audio data. These 3A typically refer to:

[0054] Acoustic echo cancellation (AEC) functions to eliminate or reduce echoes in communication systems, improving speech clarity. The principle is to address the echo problem caused by the microphone picking up sound from the speaker again. AEC uses algorithms to analyze the input signal, identify and eliminate echo components, ensuring that the transmitted speech signal is clear and echo-free.

[0055] Automatic Gain Control (AGC) dynamically adjusts the amplitude of audio data to maintain the output signal within a suitable level range, while preventing sound distortion or excessive amplification of background noise. The principle is as follows: it monitors the amplitude of the input signal in real time, calculates the required gain or attenuation based on the signal's average energy or RMS (root mean square) value, and smooths gain changes using a time constant to avoid sudden volume fluctuations.

[0056] Noise suppression (NS), or background noise suppression (ANS), functions to reduce or eliminate background noise in audio signals, improving speech clarity and intelligibility. The principle is to use algorithms to separate noise and human voice into different channels for processing, suppressing the signal in the noise channel while enhancing the signal in the human voice channel. This method can effectively reduce the interference of environmental noise on voice communication.

[0057] In summary, the 3A algorithm in sound processing effectively improves the quality of audio signals and communication effects, and enhances the user's listening experience through three key technologies: acoustic echo cancellation, automatic gain control, and noise suppression.

[0058] In this embodiment, the selection principles and methods for choosing target audio data from various audio data streams are not limited and can be customized according to actual conditions. For example, multiple quality evaluation dimensions can be set based on experience, and each audio data stream can be evaluated from these dimensions to obtain a quality evaluation value for each audio data stream. Then, based on the magnitude of the quality evaluation value, one audio data stream can be selected as the target audio data stream.

[0059] Optionally, in some embodiments, the process of aligning the master audio data and selecting the target audio data from the aligned audio data may include:

[0060] Obtain the quality assessment values ​​of the aligned main audio data under multiple preset quality assessment dimensions, as well as the weight values ​​corresponding to each preset quality assessment dimension;

[0061] Based on the weight values, the quality evaluation values ​​of the aligned main audio data are weighted and summed to obtain the comprehensive evaluation value of the aligned main audio data.

[0062] The quality assessment values ​​of the aligned audio data under multiple preset quality assessment dimensions are obtained, and the quality assessment values ​​of the aligned audio data are weighted and summed based on the weight values ​​to obtain the comprehensive assessment value of the aligned audio data.

[0063] Based on the overall evaluation value of the aligned master audio data and the overall evaluation value of the aligned slave audio data, target audio data is selected from the aligned master audio data and the aligned slave audio data.

[0064] Optionally, in some embodiments, the aforementioned multiple preset quality evaluation dimensions may include at least two of the following: signal-to-noise ratio dimension, signal energy spectrum width dimension, signal continuity dimension, and the conference terminal performance dimension corresponding to the audio data.

[0065] Signal-to-noise ratio (SNR) is a metric that measures the ratio between signal strength and background noise intensity. A high SNR means the signal is easier to detect and decode accurately, resulting in better communication quality. Specifically in audio processing, a high SNR means clearer sound and less background noise. Therefore, audio quality can be evaluated based on SNR to select audio data with the highest possible SNR as the final target audio data.

[0066] Signal energy spectral width is a parameter describing the spectral distribution of a signal; it measures the range of signal energy distribution in the frequency domain. Signal energy spectral width can be used to measure the concentration of signal energy in the frequency domain; a narrower spectral width indicates more concentrated signal energy, while a wider width indicates more dispersed energy distribution. Therefore, audio quality can be evaluated based on the signal energy spectral width dimension, allowing audio data with a higher signal energy spectral width to be selected as the final target audio data.

[0067] Signal continuity refers to the characteristics of a signal in time or space, describing whether the change in signal value at any point is smooth or abrupt. In the field of audio data processing, the better the signal continuity and the lower the noise level, the clearer the sound will be. Therefore, audio quality can also be evaluated from the perspective of signal continuity, so as to select audio data with better signal continuity as the final target audio data.

[0068] In addition, in this embodiment, audio data quality can also be evaluated from the perspective of the performance of the conferencing terminal corresponding to the audio data. Specifically, the performance of the conferencing terminal corresponding to the audio data can be the performance of the conferencing terminal that collects the audio data, such as CPU utilization. When the conferencing terminal is in a state of high CPU utilization due to processing other tasks, the quality of the collected audio data may be poor. For example, the continuity of the collected audio data may be poor, resulting in a discontinuous auditory effect in the time domain. Therefore, in this embodiment, when evaluating audio data quality, in addition to considering the attributes inherent in the audio data itself, the performance status of the conferencing terminal that collects the audio data can also be analyzed, and the quality of the audio data can be inferred based on its performance status information.

[0069] In summary, the embodiments of this application, from the perspective of multiple different quality assessment dimensions, obtain a comprehensive assessment value for each audio data stream through a weighted fusion of quality assessment values. Then, based on this comprehensive assessment value, the target audio data with better quality is selected from the multiple audio data streams. The above method does not perform quality assessment from a single dimension when selecting target audio data, but comprehensively considers multiple different assessment dimensions. Therefore, the target audio data selection scheme provided by the embodiments of this application can ensure that the target audio data selected from multiple audio data streams has better quality, thereby resulting in a clearer and higher-quality final sound pickup result.

[0070] According to the conference audio pickup scheme provided in the embodiments of this application, in addition to collecting audio data through the main terminal device, audio data is also collected through slave terminal devices located at different locations during the conference. After obtaining the main audio data collected by the main terminal device and the slave audio data collected by the slave terminal devices, the main audio data and the slave audio data are time aligned. Then, the aligned audio data are compared, and the target audio data with higher audio quality is selected. The conference audio pickup result is obtained based on the target audio data.

[0071] On the one hand, the embodiments of this application use a main conference terminal and secondary conference terminals located in different positions to jointly pick up audio, and unify the audio data collected by multiple conference terminals to the main conference terminal for time alignment processing and audio selection processing, thereby effectively improving the audio quality of the final audio data, ensuring clear and natural audio pickup, and enhancing the conference experience of participants.

[0072] On the other hand, as mentioned above, this embodiment uses multiple conference terminals for joint audio pickup. The transmission of audio data between terminals depends on the network status between them. Sudden network jitter, large delays, and data out-of-order issues can severely impact the final audio pickup effect. Furthermore, different conference terminals typically have significant differences in their clock crystal oscillators, which can cause data offset between the audio data acquired by different terminals, also negatively affecting the audio pickup effect. Considering the above problems, in this embodiment, after acquiring each audio data stream, the main conference terminal first performs time alignment processing on each audio data stream to ensure that each audio data stream can be processed subsequently under the same time base, thus eliminating the aforementioned data offset. Therefore, the audio quality of the final audio data can be further improved.

[0073] In summary, the embodiments of this application can effectively improve the clarity of meeting audio pickup, provide a high-quality audio experience, and enable participants to focus more on the meeting content.

[0074] Optionally, in some embodiments, the process of obtaining time delay information between the master conference terminal and the slave conference terminal may include:

[0075] The main conference terminal sends an alignment signal to the slave conference terminals and simultaneously plays the alignment signal through the speaker in the main conference terminal;

[0076] The alignment signal is acquired and played through the microphone in the main conference terminal;

[0077] Receive data packets returned from the conference terminal, the data packets including: alignment signals received from the conference terminal, and environmental signals containing alignment signals collected from the conference terminal;

[0078] Data alignment processing is performed on the alignment signal and environmental signal collected through the microphone to obtain the time delay information between the master conference terminal and the slave conference terminal.

[0079] Specifically, the alignment signal in the above embodiments of this application is a preset sample signal used for latency testing between terminals. In these embodiments, the specific content of the alignment signal is not limited and can be customized according to actual conditions. Furthermore, for ease of detection, a signal with a certain degree of recognizability can be selected as the alignment signal compared to the audio signal that may occur in the conference environment.

[0080] The main conference terminal can play an alignment signal and send the alignment signal to each slave conference terminal connected to the communication connection. At the same time, the main conference terminal can collect the alignment signal it plays through its own configured microphone pickup device and record the acquisition time of the alignment signal.

[0081] Accordingly, from the perspective of the conference terminal, while receiving the aforementioned alignment signal via the network, it can also collect environmental signals containing the aforementioned alignment signal through its own configured microphone pickup device and record the collection time. Then, it sends the received alignment signal and the collected environmental signal to the main conference terminal. After receiving the environmental signal and the environmental signal collection time, the main conference terminal compares its own collected alignment signal with the received environmental signal based on the environmental signal collection time and the collection time when its own microphone pickup device collected the alignment signal, thereby obtaining the time delay information between the main conference terminal and the slave conference terminal.

[0082] When there are multiple conferencing terminals, the above operations can obtain multiple time delay information, with one time delay information corresponding to one conferencing terminal.

[0083] In the above embodiments of this application, the specific content and form of the alignment signal are not limited. For example, it can be a specific sound wave signal, an ultrasonic signal, or other forms of data packets, etc.

[0084] In the aforementioned process of obtaining time delay information, the main conference terminal sends an alignment signal and receives an environmental signal containing the alignment signal. By comparing the alignment signal and the environmental signal, the time delay information between the main conference terminal and the slave conference terminal can be obtained. The alignment signal in this process is sent by the main conference terminal, and the slave conference terminal simply performs the receiving operation. Therefore, the configuration requirements for the slave conference terminal are relatively low. For example, when the alignment signal is a special signal such as ultrasound, in this embodiment, the main conference terminal only needs to be equipped with a corresponding signal transmission module, without requiring the slave conference terminal to also be equipped with the aforementioned module. Therefore, this embodiment has lower configuration requirements for the slave conference terminal and is applicable to a wider range of scenarios.

[0085] See Figure 4 , Figure 4 This is a schematic diagram of the time alignment process in an embodiment of this application. The following is in conjunction with... Figure 4 The time alignment process in this application is explained below:

[0086] Regarding the main conference equipment, it can send alignment signals to each slave conference terminal and simultaneously play the alignment signals through speakers; it can also capture the played alignment signals through microphones. Regarding the slave conference terminals, while receiving the alignment signals, they simultaneously capture ambient signals through microphones and return both the received alignment signals and the captured ambient signals to the main conference terminal.

[0087] After receiving the alignment signal and environmental signal sent by the slave conference terminal, the main conference terminal compares the alignment signal collected by its own microphone with the received environmental signal to obtain the time delay information between the main conference terminal and the slave conference terminal.

[0088] After obtaining the time delay information, upon receiving secondary audio data sent from the conference terminal, the system can perform time alignment processing on its own acquired primary audio data and the received secondary audio data based on this time delay information. This results in aligned primary and secondary audio data, from which data selection is performed to obtain the target audio data. Subsequently, echo cancellation, automatic gain control, and noise suppression are applied to the target audio data (corresponding to...). Figure 3 The 3A processing in the conference audio recording process yields the audio pickup results. Subsequent processing can then be performed based on these results, for example: for... Figure 2 In the scenario shown, the audio pickup results of the conference can be encoded and then sent to the server. The server then forwards the encoded results to other conference terminals located remotely. At this point, the audio pickup process of the main conference terminal is completed.

[0089] Optionally, in some embodiments, after receiving audio data collected for the conference from the conference terminal, the method further includes:

[0090] Obtain the location information of each conferencing terminal;

[0091] Based on location information, a terminal cluster is determined from each slave conference terminal. The terminal cluster consists of multiple slave conference terminals with a spacing of less than a preset spacing threshold.

[0092] Identify the target from the conference terminal within the terminal cluster;

[0093] Preserve the communication connection between the target terminal and the main terminal, and disconnect the communication connections between the remaining terminals in the terminal cluster and the main terminal.

[0094] Specifically, in a conferencing system with multiple slave terminals, there is a possibility that these terminals may be clustered in a relatively small area. In this case, the audio data captured by these clustered slave terminals may have similar content and clarity. Therefore, one slave terminal can be selected as the target slave terminal. This target slave terminal can then capture ambient signals from the surrounding area and transmit them to the main conferencing terminal, while disconnecting the communication between the remaining slave terminals and the main conferencing terminal. This method conserves network bandwidth resources between the main and slave conferencing terminals while maintaining good audio pickup quality.

[0095] Optionally, in some embodiments, determining the target from the conference terminal within the terminal cluster includes:

[0096] Obtain audio quality information of the audio data collected by each slave terminal in the terminal cluster;

[0097] Based on the audio quality information corresponding to each slave conference terminal, the target slave conference terminal is determined in the terminal cluster.

[0098] Specifically, audio quality information can be information characterizing the data quality of audio data. In this embodiment, the specific content of the audio quality information is not limited and can be customized according to actual conditions. For example, the data quality of each audio data point can be determined from dimensions such as signal-to-noise ratio, signal energy spectral width, signal continuity, and the performance of the corresponding conferencing terminal.

[0099] When selecting a target conferencing terminal from multiple conferencing terminals located in a relatively clustered manner, the selection can be based on the quality of the audio data collected by each terminal. The conferencing terminal with the highest quality audio data can be identified as the target conferencing terminal. This method effectively improves the audio pickup effect while reducing network bandwidth consumption between conferencing systems.

[0100] Example 2

[0101] Reference Figure 5 , Figure 5 This is a flowchart illustrating the steps of a conference audio pickup method according to Embodiment 2 of this application. The conference audio pickup method provided in this embodiment can be... Figure 1 or Figure 2 The system shown executes from conference terminal 104.

[0102] Specifically, the conference audio pickup method provided in this application embodiment may include the following steps:

[0103] Step 502: Collect audio data during the meeting to obtain audio data.

[0104] Specifically, for a conference terminal that can provide conference services, during the conference, conference audio data can be collected through a built-in sound pickup device (such as a microphone). For ease of distinction, in this embodiment, the conference audio data collected from the conference terminal is referred to as audio data.

[0105] Step 504: In response to detecting that the main conference terminal is in the same conference as the secondary conference terminal, the secondary audio data is sent to the main conference terminal so that the main conference terminal performs time alignment processing on the main audio data and secondary audio data it has collected based on the time delay information, and selects the target audio data from the aligned main audio data and aligned secondary audio data to obtain the conference sound pickup result based on the target audio data.

[0106] Optionally, in some embodiments, before the step of sending audio data to the main conference terminal in response to detecting that the main conference terminal is located in the same conference as the secondary conference terminal, the method may further include:

[0107] Receive broadcast signals sent by the main conference terminal;

[0108] The broadcast signal is parsed to obtain the main conference identifier where the main conference terminal is located. The main conference identifier is compared with the conference identifier where the slave conference terminal is located to detect whether the main conference terminal and the slave conference terminal are in the same conference.

[0109] Specifically, the main conference terminal can send broadcast signals (or sound wave signals, etc.) to a certain area around it. The broadcast signal may contain the conference identifier of the conference where the main conference terminal is located. Correspondingly, when the secondary conference terminal detects the broadcast signal sent by the main conference terminal, it can parse the broadcast signal to obtain the conference identifier of the main conference terminal and compare the identifier with the conference it is in. When the two match, it indicates that the main conference terminal and the secondary conference terminal are located in the same acoustic space and are in the same conference at the current moment. That is, the secondary conference terminal detects the main conference terminal located in the same conference.

[0110] Through the above method, the sound pickup scheme provided in this application embodiment can be automatically executed based on the detection results without user intervention. Therefore, a distributed sound pickup scheme can be implemented without the user's awareness, thereby improving the sound pickup effect.

[0111] Furthermore, the main conference terminal may or may not support the sound pickup scheme (i.e., the distributed sound pickup scheme) provided in the embodiments of this application. Therefore, the broadcast signal sent by the main conference terminal may also include function identification information, which is used to indicate whether the main conference terminal supports the distributed sound pickup scheme. If so, the secondary conference terminal can send audio data to the main conference terminal so that the main conference terminal can perform subsequent distributed sound pickup operations.

[0112] Furthermore, for Figure 2 In the conference system shown, after the main conference terminal enters the conference, it can first report distributed audio pickup information to the server, indicating that the current conference supports the distributed audio pickup scheme provided in this application embodiment. The server then sends the distributed audio pickup information to other terminal devices participating in the conference, so that other participating conference terminals know the audio pickup status of the main conference terminal. Afterwards, when other conference terminals (from the conference terminal) receive the broadcast signal sent by the main conference terminal, they can compare the function identification information contained in the broadcast signal with the distributed audio pickup information received from the server, so as to more accurately determine the conference audio pickup configuration in the current acoustic space (such as a conference room) (i.e., whether distributed audio pickup is supported).

[0113] To facilitate understanding, the above content will be explained through examples below:

[0114] Assume that a main conference device a is deployed in conference room A, and that the main conference device a supports the distributed conference audio pickup scheme provided in the embodiments of this application. When the main conference device a enters conference T, it can report distributed audio pickup information to the server (this information indicates that the main conference device a supports distributed audio testing). Simultaneously, it can broadcast a signal (this signal also contains functional identifier information indicating that the main conference device a supports distributed audio testing). The server can then forward the distributed audio pickup information reported by the main conference device a to other conference terminals participating in the same conference T. Assuming that user U participates in conference T using their laptop, the other conference terminals include user U's laptop, which will receive the distributed audio pickup information sent by the server. When the user enters conference room a with their laptop, the laptop will also receive the broadcast signal sent by the main conference device a. The laptop will then compare the received distributed audio pickup information with the functional identifier information in the broadcast signal. If they match, the laptop can confirm that the main conference device a in conference room A is in the same conference as itself and that the main conference device a supports distributed audio testing. The laptop can then execute the steps in Embodiment 2 of this application to perform joint audio pickup with the main conference device a.

[0115] See Figure 6 , Figure 6 This is a schematic diagram of the scenario corresponding to the terminal detection process. The following is combined with... Figure 6 The detection process involved in step 504 of the embodiments of this application will be explained as follows:

[0116] In the first phase, the main conference terminal joins the meeting: After successfully joining the meeting, the main conference terminal will first report the distributed audio pickup information used in the current meeting to the server. This process ensures that the server can accurately record the audio pickup status of the main conference terminal.

[0117] In the second phase, the server broadcasts information: After receiving the reported information, the server sends relevant distributed audio pickup information to other conference terminals. Each conference terminal (including desktop and mobile devices) will receive this information simultaneously, thus knowing the audio pickup status of the main conference terminal.

[0118] The third stage is the transmission of acoustic signals: After the main conference terminal successfully joins the conference and reports information to the server, it sends a specific acoustic signal to notify surrounding conference terminals of the online status of the distributed audio pickup function. This acoustic signal is used to further ensure that conference devices within the local network can quickly and accurately detect the current audio pickup configuration.

[0119] The fourth stage involves detection and comparison from the conference terminal: After receiving the sound wave signal, the conference terminal detects the content of the sound wave and compares it with the distributed audio pickup information received from the server to determine the current audio pickup configuration of the main conference terminal. This step ensures that the secondary conference terminal can accurately understand the current audio pickup configuration. If the detection and comparison process is completed successfully and the secondary conference terminal successfully establishes a connection with the main conference terminal, it indicates that both the main and secondary conference terminals are ready to enable the distributed audio pickup function provided in this embodiment.

[0120] The fifth stage involves activating the distributed audio pickup function: After completing the above steps and confirming a successful connection, the distributed audio pickup function will be officially activated on both the primary and secondary conference terminals. Simultaneously, a message will pop up on the secondary conference terminal to notify the user that the distributed audio pickup function has been successfully activated. This notification ensures that the user is aware of the current audio environment status and can make corresponding custom adjustments as needed.

[0121] The conference audio pickup solution provided in this application embodiment, on the one hand, jointly pickups audio from a main conference terminal and slave conference terminals located in different positions, and unifies the audio data collected by multiple conference terminals to the main conference terminal for time alignment and audio selection processing. This effectively improves the audio quality of the final audio data, ensuring clear and natural audio pickup and enhancing the conference experience for participants. On the other hand, after acquiring each audio data stream, the main conference terminal first performs time alignment processing on each audio data stream, ensuring that each audio data stream can be processed under the same time reference, thus eliminating the aforementioned data offset. Therefore, it can further improve the audio quality of the final audio data.

[0122] In summary, the embodiments of this application can effectively improve the clarity of meeting audio pickup, provide a high-quality audio experience, and enable participants to focus more on the meeting content.

[0123] Optionally, in some embodiments, the method further includes:

[0124] If it is detected that the main conference terminal and the secondary conference terminal are not in the same conference, the conference audio pickup results are obtained based on the secondary audio data.

[0125] Specifically, for a slave conference terminal that has entered a conference, if the master conference terminal is detected, but the master conference terminal is not in the same conference as the slave conference terminal, that is, the two conference terminals are in different conferences, the slave conference terminal can maintain its traditional audio pickup function, that is, perform audio processing such as conference cancellation based on the audio data it has collected, and thus get the conference audio pickup results of the conference where the slave conference terminal is located.

[0126] Optionally, in some embodiments, after detecting a master conference terminal located in the same conference as the slave conference terminal, the method further includes:

[0127] Output distributed sound pickup alert information.

[0128] Specifically, when a main conference terminal in the same conference is detected from another conference terminal, in addition to performing the operations described in the above embodiments of this application by enabling the distributed audio pickup function, a distributed audio pickup reminder message can also be output to remind the user. This reminder operation ensures that the user is aware of the current audio environment status and can make corresponding adjustments or operations as needed. Therefore, the user's conference experience can be further improved.

[0129] Optionally, in some embodiments, after detecting a master conference terminal located in the same conference as the slave conference terminal, the method further includes:

[0130] Control the conference terminal to mute the conference.

[0131] Specifically, when a secondary conference terminal detects a primary conference terminal in the same conference, in addition to enabling the distributed audio pickup function and performing the operations described in the above embodiments of this application, the secondary conference terminal can be muted, meaning it will not play conference audio data. This muting operation avoids sound interference caused by both the primary and secondary conference terminals playing the same audio data, thus further enhancing the user's conference experience.

[0132] In this embodiment, the specific method used to control the conference terminal to be in a muted state is not limited, and can be customized according to the actual situation. For example, the conference terminal can be muted by turning off the speaker, or the streaming link between the conference terminal and the server can be disconnected, etc.

[0133] See Figure 7 , Figure 7 This diagram illustrates the interaction flow between the master and slave conferencing terminals. The following section combines... Figure 7 The conference audio pickup scheme provided in the embodiments of this application will be explained and described as follows:

[0134] The first step, the meeting entry process from the conference terminal: When a conference terminal joins a meeting, its built-in microphone begins to collect the current audio data. Simultaneously, the conference terminal determines whether the current meeting belongs to the same meeting as the main conference terminal in the meeting room, and checks whether the main conference terminal supports distributed audio pickup. Specifically:

[0135] If the current meeting and the meeting room meeting belong to the same meeting and the main meeting terminal supports distributed audio pickup, then execute the steps in Embodiment 2 above to activate the distributed audio pickup function and provide a better audio experience for the meeting.

[0136] If the current meeting and the meeting in the conference room belong to the same meeting but the main conference terminal does not support distributed audio pickup, the user will be prompted to mute and turn off the speaker to avoid interference.

[0137] If the current meeting and the meeting in the conference room are not part of the same meeting, distributed audio pickup processing will not be performed. Instead, the acquired audio data will be processed (e.g., echo cancellation, automatic gain control, noise suppression), and then the encoded data will be sent to the server.

[0138] The second step is the main conference terminal joining process: When the main conference terminal joins the conference, it will check whether it supports distributed audio pickup function.

[0139] If the main conference terminal supports distributed audio pickup, it will execute the steps in Embodiment 1 above to activate the distributed audio pickup function and send distributed audio pickup sound wave signals. Simultaneously, the main conference terminal will collect local main audio data and receive secondary audio data transmitted from each secondary conference terminal.

[0140] The audio data will then undergo time alignment to ensure time synchronization and data consistency. The aligned data will be transmitted to the algorithm processing module, which will select the highest quality target audio data and perform more complex audio processing on it, such as echo cancellation, automatic gain control, and noise suppression. Figure 7 (such as the 3A algorithm in the text).

[0141] Finally, the processed target audio data will be encoded and sent to the server.

[0142] Example 3

[0143] Figure 8 This is a structural block diagram of a conference microphone according to Embodiment 3 of this application. The conference microphone provided in this embodiment is located in... Figure 1 or Figure 2 In the main conference terminal 102 of the system shown, the conference audio pickup device includes:

[0144] The audio data collection module 802 is used to collect audio data during the meeting to obtain the main audio data; and to receive the secondary audio data collected for the meeting sent by the secondary meeting terminal corresponding to the main meeting terminal.

[0145] The alignment processing module 804 is used to obtain the time delay information between the master conference terminal and the slave conference terminal, and to perform time alignment processing on the master audio data and slave audio data based on the time delay information to obtain the aligned master audio data and aligned slave audio data.

[0146] The sound pickup result obtaining module 806 is used to select target audio data from the aligned main audio data and the aligned audio data to obtain the conference sound pickup result based on the target audio data.

[0147] Optionally, in some embodiments, the alignment processing module 804, when performing the step of obtaining time delay information between the master conference terminal and the slave conference terminal, is specifically used for:

[0148] The main conference terminal sends an alignment signal to the slave conference terminals and simultaneously plays the alignment signal through the speaker in the main conference terminal;

[0149] The alignment signal is acquired and played through the microphone in the main conference terminal;

[0150] Receive data packets returned from the conference terminal, the data packets including: alignment signals received from the conference terminal, and environmental signals containing alignment signals collected from the conference terminal;

[0151] Data alignment processing is performed on the alignment signal and environmental signal collected through the microphone to obtain the time delay information between the master conference terminal and the slave conference terminal.

[0152] Optionally, in some embodiments, the pickup result obtaining module 806, when performing the steps of selecting target audio data from the aligned main audio data and the aligned audio data, is specifically used for:

[0153] Obtain the quality assessment values ​​of the aligned main audio data under multiple preset quality assessment dimensions, as well as the weight values ​​corresponding to each preset quality assessment dimension;

[0154] Based on the weight values, the quality evaluation values ​​of the aligned main audio data are weighted and summed to obtain the comprehensive evaluation value of the aligned main audio data.

[0155] The quality assessment values ​​of the aligned audio data under multiple preset quality assessment dimensions are obtained, and the quality assessment values ​​of the aligned audio data are weighted and summed based on the weight values ​​to obtain the comprehensive assessment value of the aligned audio data.

[0156] Based on the overall evaluation value of the aligned master audio data and the overall evaluation value of the aligned slave audio data, target audio data is selected from the aligned master audio data and the aligned slave audio data.

[0157] Optionally, in some embodiments, the multiple preset quality evaluation dimensions include at least two of the following: signal-to-noise ratio dimension, signal energy spectrum width dimension, signal continuity dimension, and the conference terminal performance dimension corresponding to the audio data.

[0158] Optionally, in some embodiments, the conference microphone further includes:

[0159] The disconnect module is used to obtain the location information of each slave conference terminal after receiving the audio data collected for the conference from the slave conference terminal; based on the location information, determine the terminal cluster among the slave conference terminals, the terminal cluster consists of multiple slave conference terminals with a spacing of less than a preset spacing threshold; determine the target slave conference terminal in the terminal cluster; maintain the communication connection between the target slave conference terminal and the main conference terminal, and disconnect the communication connection between the remaining slave conference terminals in the terminal cluster and the main conference terminal.

[0160] Optionally, in some embodiments, the disconnection module, when performing the step of determining the target from the conference terminal within the terminal cluster, is specifically configured to:

[0161] Obtain audio quality information of the audio data collected by each slave terminal in the terminal cluster;

[0162] Based on the audio quality information corresponding to each slave conference terminal, the target slave conference terminal is determined in the terminal cluster.

[0163] The conference audio pickup device of this embodiment is used to implement the corresponding conference audio pickup method in the aforementioned embodiment one, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here. In addition, the functional implementation of each module in the conference audio pickup device of this embodiment can refer to the description of the corresponding part in the aforementioned method embodiment one, which will also not be repeated here.

[0164] Example 4

[0165] Figure 9 This is a structural block diagram of a conference microphone according to Embodiment 4 of this application. The conference microphone provided in this embodiment is located in... Figure 1 or Figure 2 The conference audio pickup device in the conference terminal 104 of the system shown includes:

[0166] The audio data acquisition module 902 is used to acquire audio data during the meeting and obtain audio data.

[0167] The audio data sending module 904 is used to send audio data to the main conference terminal in response to detecting that the main conference terminal is in the same conference as the secondary conference terminal. This allows the main conference terminal to perform time alignment processing on the main audio data and secondary audio data it has collected based on time delay information. The main conference terminal then selects target audio data from the aligned main audio data and aligned secondary audio data to obtain the conference sound pickup result based on the target audio data.

[0168] Optionally, in some embodiments, the conference microphone further includes:

[0169] The detection module is used to receive the broadcast signal sent by the main conference terminal before sending audio data to the main conference terminal in response to the detection of the main conference terminal located in the same conference as the slave conference terminal; parse the broadcast signal to obtain the main conference identifier of the main conference terminal; compare the main conference identifier with the conference identifier of the slave conference terminal to detect whether the main conference terminal and the slave conference terminal are located in the same conference.

[0170] Optionally, in some embodiments, the detection module is further configured to:

[0171] If it is detected that the main conference terminal and the secondary conference terminal are not in the same conference, the conference audio pickup results are obtained based on the secondary audio data.

[0172] Optionally, in some embodiments, the conference microphone further includes:

[0173] The reminder module is used to output distributed audio reminder information after detecting a main conference terminal that is in the same conference as the secondary conference terminal.

[0174] Optionally, in some embodiments, the conference microphone further includes:

[0175] The status control module is used to control the slave conference terminal to enter a conference mute state after detecting the master conference terminal which is in the same conference as the slave conference terminal.

[0176] The conference audio pickup device of this embodiment is used to implement the corresponding conference audio pickup method in the aforementioned embodiment two, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here. In addition, the functional implementation of each module in the conference audio pickup device of this embodiment can refer to the description of the corresponding part in the aforementioned method embodiment one, which will also not be repeated here.

[0177] Example 5

[0178] Reference Figure 10 The diagram shows a structural schematic of a main conference terminal according to Embodiment 5 of this application. The specific embodiments of this application do not limit the specific implementation of the main conference terminal.

[0179] like Figure 10 As shown, the electronic device may include: a processor 1002, a communication interface 1004, a memory 1006, and a communication bus 1008.

[0180] in:

[0181] The processor 1002, communication interface 1004, and memory 1006 communicate with each other via communication bus 1008.

[0182] Communication interface 1004 is used to communicate with other electronic devices or servers.

[0183] The processor 1002 is used to execute program 1010, specifically the relevant steps in the above method embodiments.

[0184] Specifically, program 1010 may include program code that includes computer operation instructions.

[0185] The processor 1002 may be a CPU, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.

[0186] Memory 1006 is used to store program 1010. Memory 706 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0187] Program 1010 may include multiple computer instructions. Specifically, program 1010 may use multiple computer instructions to cause processor 1002 to perform the operation corresponding to any of the methods described in the foregoing multiple method embodiments.

[0188] The specific implementation of each step in program 1010 can be found in the corresponding descriptions of the steps and units in the above method embodiments, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0189] Example 6

[0190] Reference Figure 11 The diagram shows a schematic of a conferencing terminal according to Embodiment Six of this application. The specific implementation of the conferencing terminal is not limited by the specific implementation of the conferencing terminal in this application.

[0191] like Figure 11 As shown, the electronic device may include: a processor 1102, a communication interface 1104, a memory 1106, and a communication bus 1108.

[0192] in:

[0193] The processor 1102, communication interface 1104, and memory 1106 communicate with each other via communication bus 1108.

[0194] Communication interface 1104 is used to communicate with other electronic devices or servers.

[0195] The processor 1102 is used to execute program 1110, specifically the relevant steps in the above method embodiments.

[0196] Specifically, program 1110 may include program code that includes computer operation instructions.

[0197] The processor 1102 may be a CPU, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.

[0198] Memory 1106 is used to store program 1110. Memory 1106 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0199] Program 1110 may include multiple computer instructions. Specifically, program 1110 may use multiple computer instructions to cause processor 1102 to perform the operation corresponding to any of the methods described in the foregoing multiple method embodiments.

[0200] The specific implementation of each step in program 1110 can be found in the corresponding descriptions of the steps and units in the above method embodiments, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0201] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in any of the foregoing method embodiments. The computer storage medium includes, but is not limited to, compact disc read-only memory (CD-ROM), random access memory (RAM), floppy disk, hard disk, or magneto-optical disk.

[0202] This application also provides a computer program product, including computer instructions that instruct a computing device to perform an operation corresponding to any of the methods in the above-described multiple method embodiments.

[0203] Furthermore, it should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to sample data used for training the model, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0204] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0205] The methods described above according to the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code originally stored in a remote recording medium or a non-transitory machine-readable medium and to be stored in a local recording medium, downloaded via a network. Thus, the methods described herein can be stored as software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA)). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., random access memory (RAM), read-only memory (ROM), flash memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses the code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0206] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0207] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A conference audio pickup method, applied to a main conference terminal, the method comprising: Audio data is collected during the meeting to obtain the main audio data; Receive audio data collected for the conference from a slave conference terminal corresponding to the main conference terminal; The time delay information between the main conference terminal and the slave conference terminal is obtained, and the main audio data and the slave audio data are time aligned based on the time delay information to obtain aligned main audio data and aligned slave audio data. After alignment, target audio data is selected from the master audio data and the audio data after alignment to obtain the conference sound pickup result based on the target audio data.

2. The method according to claim 1, wherein, The step of obtaining the time delay information between the master conference terminal and the slave conference terminal includes: The main conference terminal sends an alignment signal to the slave conference terminal and simultaneously plays the alignment signal through the speaker in the main conference terminal; The alignment signal is acquired and played through the microphone in the main conference terminal; Receive the data packet returned from the conference terminal, the data packet including: the alignment signal received from the conference terminal, and the environmental signal collected from the conference terminal containing the alignment signal; The alignment signal acquired through the microphone and the environmental signal are processed to obtain the time delay information between the main conference terminal and the slave conference terminal.

3. The method according to claim 1 or 2, wherein, The step of selecting target audio data from the aligned main audio data and the aligned sub-audio data includes: Obtain the quality assessment values ​​of the aligned main audio data under multiple preset quality assessment dimensions, and the weight values ​​corresponding to each preset quality assessment dimension; Based on the weight values, the quality evaluation values ​​of the aligned main audio data are weighted and summed to obtain the comprehensive evaluation value of the aligned main audio data. The quality assessment values ​​of the aligned audio data under the multiple preset quality assessment dimensions are obtained, and based on the weight values, the quality assessment values ​​of the aligned audio data are weighted and summed to obtain the comprehensive assessment value of the aligned audio data. Based on the comprehensive evaluation value of the aligned master audio data and the comprehensive evaluation value of the aligned slave audio data, target audio data is selected from the aligned master audio data and the aligned slave audio data.

4. The method according to claim 3, wherein, The multiple preset quality assessment dimensions include at least two of the following: signal-to-noise ratio dimension, signal energy spectrum width dimension, signal continuity dimension, and the conference terminal performance dimension corresponding to the audio data.

5. The method according to claim 1 or 2, wherein, After receiving the audio data collected for the conference from the conference terminal, the method further includes: Obtain the location information of each conferencing terminal; Based on the location information, a terminal cluster is determined from each slave conference terminal, and the terminal cluster consists of multiple slave conference terminals with a spacing less than a preset spacing threshold; The target is identified from the conference terminal within the terminal cluster; The communication connection between the target slave terminal and the main conference terminal is maintained, while the communication connection between the remaining slave terminals in the terminal cluster and the main conference terminal is disconnected.

6. The method according to claim 5, wherein, The step of determining the target from the conference terminal in the terminal cluster includes: Obtain audio quality information of the audio data collected by each conferencing terminal in the terminal cluster; Based on the audio quality information corresponding to each slave conference terminal, the target slave conference terminal is determined in the terminal cluster.

7. A conference audio pickup method, applied from a conference terminal, the method comprising: Audio data is collected during the meeting to obtain audio data; In response to detecting a main conference terminal in the same conference as the secondary conference terminal, the secondary audio data is sent to the main conference terminal, so that the main conference terminal performs time alignment processing on its own collected main audio data and the secondary audio data based on time delay information, and selects target audio data from the aligned main audio data and aligned secondary audio data to obtain the conference sound pickup result based on the target audio data.

8. The method according to claim 7, wherein, Before sending the secondary audio data to the primary conference terminal in response to detecting a primary conference terminal located in the same conference as the secondary conference terminal, the method further includes: Receive broadcast signals sent by the main conference terminal; The broadcast signal is parsed to obtain the main conference identifier where the main conference terminal is located. The main conference identifier is compared with the conference identifier where the slave conference terminal is located to detect whether the main conference terminal and the slave conference terminal are located in the same conference.

9. The method according to claim 8, wherein, The method further includes: If it is detected that the main conference terminal and the slave conference terminal are not in the same conference, the conference audio pickup result is obtained based on the slave audio data.

10. The method according to any one of claims 7-9, wherein, After detecting a master conference terminal located in the same conference as the slave conference terminal, the method further includes: Output distributed sound pickup alert information.

11. The method according to any one of claims 7-9, wherein, After detecting a master conference terminal located in the same conference as the slave conference terminal, the method further includes: Control the conference terminal to be in a mute state.

12. A main conference terminal, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the operation corresponding to the method as described in any one of claims 1-6.

13. A device from a conference terminal, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the operation corresponding to the method as described in any one of claims 7-11.

14. A conference system, comprising: The master conference terminal according to claim 12, and the slave conference terminal according to claim 13.

15. A computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 1-11.

16. A computer program product comprising computer instructions that instruct a computing device to perform an operation corresponding to any one of the methods described in claims 1-11.

Images