Microphone voice acquisition method in multi-person conference scene

Abstract

The invention relates to the technical field of voice recognition, in particular to a microphone voice acquisition method in a multi-person conference scene, which comprises the following steps: acquiring a sound source audio captured by any microphone in the multi-person conference scene and preprocessing to obtain to-be-analyzed sound source data; separating into a plurality of original component signals, analyzing the possibility that the original component signals are main signals to obtain a main signal group, calculating a possibility index of a main speaking component based on the target analysis signal, and obtaining the possibility that the target analysis signal is main component information corresponding to a microphone of a main spokesman; the final possibility that the target analysis signal is the main component information corresponding to the microphone of the main spokesman is obtained; determining an adjustment factor based on the ultimate possibility of the main component information corresponding to the microphone of the main spokesman, and weighting and reconstructing each corresponding signal through the adjustment factor; and calculating the signal intensity of any microphone in combination with the reconstructed signal, and obtaining the reconstructed audio information of the corresponding microphone through the signal intensity.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to a method for collecting microphone speech in a multi-person conference scenario. Background Art

[0002] In contemporary multi-person conference scenarios, the processing and analysis of audio signals is a key link to ensure clear transmission and accurate recording of conference content. However, the current means of processing audio environments limit their practical application effects and popularity. They may be insufficient in making full use of audio phase information. For example, when generating high-quality speech signals by weighted reconstruction of separated audio components, it is impossible to accurately evaluate the importance of each component and assign appropriate weights. If the weights are improperly distributed, it may directly lead to a significant reduction in the quality of the reconstructed signal; or multi-channel signal processing technology is used to capture signals using multiple microphones, and spatial filtering and beamforming technology are used to enhance the clarity of the target speech while suppressing the influence of background noise. However, this method is based on the complexity of the algorithm, has high latency, and is greatly affected by the environment, which affects the real-time processing effect and the overall performance is unstable. Summary of the invention

[0003] In order to solve the technical problems of uneven weight distribution in the existing method, affecting the quality of the reconstructed signal and the low timeliness of the existing processing means, the purpose of the present invention is to provide a microphone voice collection method in a multi-person conference scenario, and the technical solution adopted is as follows: Acquire the sound source audio captured by any microphone in a multi-person conference scene and perform preprocessing to obtain the sound source data to be analyzed, wherein the microphone includes the microphone of the main speaker; Separate the sound source data to be analyzed into a plurality of original component signals in sequence, analyze the possibility that the original component signal is the main signal to obtain a main signal group, define any stage signal in the main signal group as a target analysis signal, calculate the possibility index of the main speech component based on the target analysis signal, and obtain the possibility that the target analysis signal is the main component information corresponding to the microphone of the main speaker according to the possibility of the main signal and the possibility index of the main speech component; calculate the difference index between the target analysis signal and any stage signal of the non-target analysis signal in the main signal group to obtain the final possibility that the target analysis signal is the main component information corresponding to the microphone of the main speaker; determine the adjustment factor based on the final possibility of the main component information corresponding to the microphone of the main speaker, and reconstruct each corresponding signal by weighting the adjustment factor; The signal strength of any microphone is calculated based on the reconstructed signal, and the reconstructed audio information of the corresponding microphone is obtained through the signal strength.

[0004] Preferably, the original component signal is composed of any single sound source and / or multiple sound sources of a noise source, a main speaker sound source and a secondary speaker sound source.

[0005] Preferably, the possibility of analyzing the original component signal as the main signal is calculated as follows: in, and Represent the original component signals and The possibility of being the main signal; Represents the original component signal and Dynamic time warping distance of Represents the original component signal and The mean voltage of Represents the original component signal and The absolute value of the difference in the voltage mean; Represents the normalization function.

[0006] Preferably, the main signal group is obtained by: judging whether the possibility of the main signal exceeds a preset threshold, if so, determining that the current original component signal is the main signal group; if not, determining that the current original component signal is the non-main signal group.

[0007] Preferably, calculating the likelihood index of the main speech component based on the target analysis signal includes: Convert the signals in the main signal group from time domain signals to frequency domain signals; The probability index of the main speech component is calculated based on the target analysis signal, and the corresponding calculation formula is: in, represents the target analysis signal; Indicates the probability index of the frequency domain signal corresponding to the target analysis signal being the main speech component; Indicates the stability of energy distribution of the target analysis signal corresponding to the frequency domain signal; Indicates the frequency number of the frequency domain signal corresponding to the target analysis signal; Indicates the target analysis signal corresponds to the frequency domain signal The amplitude at the frequency; Indicates the target analysis signal corresponds to the frequency domain signal The amplitude at a frequency.

[0008] Preferably, the possibility of obtaining the target analysis signal as the main component information corresponding to the microphone of the main speaker includes: Get the highest amplitude of all frequency domain signals in the main signal group; The corresponding calculation formula is: in, Indicates the possibility that the target analysis signal is the main component information corresponding to the microphone of the main speaker; Indicates the probability that the target analysis signal is the main signal; Indicates the highest amplitude of all frequency domain signals in the main signal group; Indicates the highest amplitude of the frequency domain signal corresponding to the target analysis signal.

[0009] Preferably, the difference index between the target analysis signal and any stage signal of the non-target analysis signal in the main signal group is calculated, and the corresponding calculation formula is: in, Any stage signal representing non-target analysis signal in the main signal group; Indicates the frequency domain signal corresponding to the target analysis signal and The difference index of the corresponding frequency domain signal; Indicates that the target analysis signal is The phase at the frequency; express In the The phase at the frequency; Indicates the target analysis signal corresponds to the frequency domain signal The amplitude at the frequency; express The corresponding frequency domain signal The amplitude at the frequency; Indicates the frequency number of the frequency domain signal corresponding to the target analysis signal; is a natural constant.

[0010] Preferably, the target analysis signal is obtained as the final probability of the main component information corresponding to the microphone of the main speaker, and the corresponding calculation formula is: in, Indicates the final probability that the target analysis signal is the main component information corresponding to the microphone of the main speaker; Indicates the possibility that the target analysis signal is the main component information corresponding to the microphone of the main speaker; represents the number of remaining non-target analysis signals in the main signal group; Any stage signal representing non-target analysis signal in the main signal group; Indicates the frequency domain signal corresponding to the target analysis signal and The difference index of the corresponding frequency domain signal.

[0011] Preferably, the adjustment factor is determined based on the final likelihood of the main component information corresponding to the microphone of the main speaker, and each corresponding signal is reconstructed by weighting the adjustment factor, including: Obtain the ultimate possibility of all corresponding main component information of the main speaker's microphone, and sort them from large to small, and define the largest value as the adjustment factor; Counting the number of all signals in the non-main signal group and calculating their proportion in the sound source data to be analyzed, and obtaining the adjustment factor of the non-main component information in the main signal group; All adjustment factors of the sound source data to be analyzed are obtained, the main component information and non-main component information corresponding to each microphone are classified, corresponding adjustment factors are used for corresponding adjustment, and the adjusted signal of each microphone is obtained through weighted reconstruction.

[0012] Preferably, the signal strength of any microphone is calculated in combination with the reconstructed signal, and the reconstructed audio information of the corresponding microphone is obtained through the signal strength, including: Convert the reconstructed signal from a time domain signal to a frequency domain signal; To calculate the signal strength of any microphone, the corresponding calculation formula is: in, Indicates The signal strength of the reconstructed signal from each microphone; Indicates the number of horizontal coordinates of the frequency domain signal corresponding to the signal reconstructed by the kth microphone; Indicates The reconstructed signal of the microphone corresponds to the frequency domain signal The horizontal axis corresponds to the amplitude; The signal strengths of all the microphone reconstructed signals are obtained, sorted from large to small, and the corresponding frequency domain with the largest signal strength is converted back to the time domain signal to obtain the audio information reconstructed by the corresponding microphone.

[0013] The present invention has the following beneficial effects: By improving the existing method so that it can more effectively utilize phase information and more accurately evaluate the importance of each component, the quality of the reconstructed signal can be significantly improved, and the overall effect of audio processing can be improved, so that remote participants or conference recording equipment can more clearly capture and understand the content of the meeting, and enhance the user experience; optimizing the weight distribution strategy can also reduce the impact of noise and other interference, and further improve the clarity and intelligibility of the audio signal; significantly improve the usability and reliability of audio processing methods in multi-person conference environments, and provide users with a better audio experience, that is, in a complex acoustic environment, it can still accurately reconstruct the signal, and can maintain high-quality signal processing effects under changing acoustic conditions, ensuring the complete transmission of information. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 A flowchart of a method for collecting microphone voice in a multi-person conference scenario provided by an embodiment of the present invention; Figure 2 A signal time-frequency diagram of a microphone voice collection method in a multi-person conference scenario provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the microphone voice collection method in a multi-person conference scenario proposed by the present invention, its specific implementation method, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0017] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0018] The following specifically describes a method for collecting microphone voices in a multi-person conference scenario provided by the present invention in conjunction with the accompanying drawings.

[0019] See also Figure 1 , which shows a flowchart of the steps of a microphone voice collection method in a multi-person conference scenario provided by an embodiment of the present invention, the method comprising: Step S1: obtaining the sound source audio captured by any microphone in a multi-person conference scene and preprocessing it to obtain the sound source data to be analyzed, the microphone including the microphone of the main speaker; Step S2: Separate the sound source data to be analyzed into a plurality of original component signals in sequence, analyze the possibility that the original component signal is a main signal to obtain a main signal group, define any stage signal in the main signal group as a target analysis signal, calculate the possibility index of the main speech component based on the target analysis signal, and obtain the possibility that the target analysis signal is the main component information corresponding to the microphone of the main speaker according to the possibility of the main signal and the possibility index of the main speech component; calculate the difference index between the target analysis signal and any stage signal of the non-target analysis signal in the main signal group, and obtain the final possibility that the target analysis signal is the main component information corresponding to the microphone of the main speaker; determine the adjustment factor based on the final possibility of the main component information corresponding to the microphone of the main speaker, and reconstruct each corresponding signal by weighting the adjustment factor; Step S3: Calculate the signal strength of any microphone in combination with the reconstructed signal, and obtain the reconstructed audio information of the corresponding microphone through the signal strength.

[0020] To better illustrate, a multi-person conference scene is a typical complex audio environment, including speeches by several people, and background noise may also exist at the same time, such as the sound of air conditioning, keyboard tapping, or the operating sound of other equipment. The voice collection method proposed in this embodiment is to identify the main sound source for multi-person conference scenes and amplify it, thereby improving the quality of conference voice playback.

[0021] To explain, in step S1, preprocessing includes filtering and purifying the sound source audio, that is, targeted elimination of unnecessary frequency components, reduction of environmental interference and noise, ensuring the purity and audibility of the audio signal, generating the sound source data to be analyzed, and laying a solid foundation for subsequent analysis; wherein, the microphone also includes the microphone of the secondary speaker. It can be explained that the main speaker and the secondary speaker are relative concepts, so as to distinguish the importance of different speakers in the discussion or meeting, so as to facilitate the understanding and analysis of each speech content in the sound source data to be analyzed.

[0022] Further, the original component signal is composed of any single sound source and / or multiple sound sources of a noise source, a main speaker sound source, and a secondary speaker sound source.

[0023] It is explained that in a multi-source environment, it is difficult to obtain accurate results by directly analyzing the mixed sound source data to be analyzed. Different sound source data may interfere with each other, resulting in the inability to identify the characteristics of a single sound source. Therefore, the mixed sound source data to be analyzed is firstly subjected to sound source separation to improve the accuracy of speech recognition and processing, wherein the sound source data after preprocessing by each microphone is decomposed into several independent original component signals based on the blind source separation technology. Each original component signal may be composed of a single sound source, such as only containing a noise source or only containing a main speaker sound source, or may be composed of a combination of multiple sound sources, such as containing both the main speaker sound source and the secondary speaker sound source.

[0024] It can be understood that in an offline multi-person meeting, each speaker corresponds to a microphone, and the environmental noise in the original audio corresponding to each microphone is basically the same, and the corresponding noise component performance should also be basically consistent, that is, each microphone is located in a relatively fixed physical position, and the environmental noise components contained in the captured original audio signal are basically similar. At this time, the environmental noise components of each microphone are also roughly the same in nature and intensity; by analyzing the original component signals converted by each microphone, the connection between the signals and possible common sources can be obtained, and then the importance of each component signal can be more accurately judged, and grouped to identify potential major signals and noise interference.

[0025] Furthermore, in step S2, the possibility of the original component signal being the main signal is analyzed, and the corresponding calculation formula is: in, and Represent the original component signals and The possibility of being the main signal; Represents the original component signal and Dynamic time warping distance of Represents the original component signal and The mean voltage of Represents the original component signal and The absolute value of the difference in the voltage mean; Represents the normalization function.

[0026] See also Figure 2 , which shows a schematic diagram of part of the time domain signal and the spectrum signal in an audio signal, wherein the left side of the schematic diagram represents the time domain signal, the horizontal axis is time, and the vertical axis is amplitude; the right side of the schematic diagram represents the frequency domain signal, the horizontal axis is frequency, and the vertical axis is amplitude.

[0027] Specifically, a corresponding time domain signal is obtained based on several original component signals, and the signal is converted into a digital form by using an analog-to-digital converter (ADC) or other appropriate conversion equipment. The converted digital signal is usually represented by a series of discrete sampling points, each sampling point corresponds to the value of the original analog signal at a specific time point, and they are arranged in a certain order to form a time series of a digital signal; the time series of all digital signals corresponding to all original component signals of all microphones are arbitrarily combined in pairs, and the correlation between the two is obtained based on the above calculation formula.

[0028] It can be explained that calculating the correlation is used to analyze the possibility between the original component signals, so as to facilitate subsequent judgment; specifically, the original component signals and It is a time series of equal length from different microphones and the same time period; and Represent the original component signals and The possibility of being the main signal is used for preliminary judgment and the same possibility is given, which will be further analyzed later; Represents the original component signal and Dynamic Time Warping (DTW) is an algorithm used to measure the similarity between two time series. It elastically stretches or compresses the time series to align the two series on the time axis to find the best matching path. The smaller the value, the better the original component signal. and The higher the similarity, the negative correlation between them; Represents the original component signal and The larger the voltage value, the stronger the original component signal and The more likely it is to be the main signal; Represents the original component signal and The larger the absolute value of the difference between the voltage mean and the original component signal, the and The smaller the possibility of being the same signal source, the lower the similarity between the two.

[0029] Furthermore, in step S2, the main signal group is obtained as follows: Based on the possibility of the main signal, it is determined whether it exceeds a preset threshold. If so, the current original component signal is determined to be a main signal group; if not, the current original component signal is determined to be a non-main signal group.

[0030] Preferably, in this embodiment, the preset threshold is 0.6. When the possibility of the main signal is greater than 0.6, that is, the similarity between the two original component signals is greater than 0.6, it indicates that the original component signal currently analyzed is the main signal group, otherwise it is the non-main signal group. It can be explained that the non-main signal group may contain noise components and secondary components. The grouping of the main signal group and the non-main signal group is determined based on the voltage change. The voltage in the non-main signal group is low and will be shielded during the processing. Therefore, in this embodiment, voice collection and recognition are mainly performed on the main signal group.

[0031] It is understandable that after the above grouping is completed, there may be misjudged original component signals in all time series of the main signal group. For example, a microphone close to the main speaker's microphone may capture the main speaker's information. Although this information has a louder volume, when superimposed with the signal of the main microphone, it may cause increased noise, thereby reducing the quality of the played voice. Therefore, under certain specific conditions, these secondary components may be mistakenly identified as main signals, and further analysis is required to avoid such misjudgment; the main speaker's voice component has a large amplitude, and its environmental noise basically has the same impact in the original audio corresponding to each microphone. At this time, if there is a cough or other sudden sounds such as mobile phone sounds, tables and chairs moving in the surrounding environment, the amplitude of the volume at this moment will also be higher, and it will not be the main speech sound. The main speech component and the sudden noise component are distinguished by the frequency domain energy distribution.

[0032] Furthermore, in step S2, the possibility index of the main speech component is calculated based on the target analysis signal, including: Convert the signals in the main signal group from time domain signals to frequency domain signals; The probability index of the main speech component is calculated based on the target analysis signal, and the corresponding calculation formula is: in, represents the target analysis signal; Indicates the probability index of the frequency domain signal corresponding to the target analysis signal being the main speech component; Indicates the stability of energy distribution of the target analysis signal corresponding to the frequency domain signal; Indicates the frequency number of the frequency domain signal corresponding to the target analysis signal; Indicates the target analysis signal corresponds to the frequency domain signal The amplitude at the frequency; Indicates the target analysis signal corresponds to the frequency domain signal The amplitude at a frequency.

[0033] It can be explained that defining any stage signal in the main signal group as the target analysis signal means extracting a section of signal from all time domain signals in the main signal group as the target analysis signal.

[0034] Specifically, the time domain signal corresponding to the main signal group is converted into a frequency domain signal through Fourier transform. Spectral analysis can obtain the energy distribution between different frequency components, which is conducive to identifying and separating the signal of the main speaker and other sound signals. Among them, Fourier transform means that any periodic function can be expressed as a superposition of sine waves of different frequencies, amplitudes and phases. It can be explained that the amplitude of the main speaker's voice changes relatively smoothly, reflecting the normal speech rhythm and emphasis, while the amplitude of sudden noise may suddenly increase, that is, in a meeting, the main speaker's voice usually maintains a certain amplitude change, which is conducive to the audience's understanding and following the speaker's ideas; sudden noise will break the original sound balance; therefore The smaller the difference between the two, the more likely the target analysis signal is the main speech component; It indicates the energy distribution stability of the target analysis signal corresponding to the frequency domain signal. By calculating the standard deviation of the energy at each frequency of the target analysis signal corresponding to the frequency domain signal, the larger the value, the more stable it is, and the more likely it is the main speech component. That is, the energy distribution of the main speech component is relatively stable throughout the entire time period, while the energy distribution of sudden noise changes rapidly in a short period of time and fluctuates greatly.

[0035] Furthermore, in step S2, obtaining the possibility that the target analysis signal is the main component information corresponding to the microphone of the main speaker includes: Get the highest amplitude of all frequency domain signals in the main signal group; The corresponding calculation formula is: in, Indicates the possibility that the target analysis signal is the main component information corresponding to the microphone of the main speaker; Indicates the probability that the target analysis signal is the main signal; Indicates the highest amplitude of all frequency domain signals in the main signal group; Indicates the highest amplitude of the frequency domain signal corresponding to the target analysis signal.

[0036] To explain, It represents the ratio of the highest amplitude of the target analysis signal to the highest amplitude of all frequency domain signals in the main signal group. The larger the value is, the greater the possibility that the main speech component corresponds to the microphone of the main speaker.

[0037] It can be understood that the phase angle of each frequency component is obtained for the frequency domain signal corresponding to each microphone, and its phase value is arrive For continuous frequency components, phase unwrapping is performed to ensure phase continuity and avoid discontinuous jumps; and the phase difference between each time series is calculated to determine the main speech component of the sound source path of the two time series through the amplitudes at two frequencies, which can more accurately identify the voice of the main speaker and ensure the accuracy and reliability of audio processing.

[0038] Furthermore, in step S2, the difference index between the target analysis signal and any stage signal of the non-target analysis signal in the main signal group is calculated, and the corresponding calculation formula is: in, Any stage signal representing non-target analysis signal in the main signal group; Indicates the frequency domain signal corresponding to the target analysis signal and The difference index of the corresponding frequency domain signal; Indicates that the target analysis signal is The phase at the frequency; express In the The phase at the frequency; Indicates the target analysis signal corresponds to the frequency domain signal The amplitude at the frequency; express The corresponding frequency domain signal The amplitude at the frequency; Indicates the frequency number of the frequency domain signal corresponding to the target analysis signal; is a natural constant.

[0039] To explain, Any stage signal of non-target analysis signal in the main signal group, that is, any stage signal in the main signal group except the target analysis signal; Indicates when the phase difference When it is far from 0, the two signals currently being analyzed may come from different sound sources or paths; Indicates the frequency domain signal corresponding to the target analysis signal and The corresponding amplitude difference of the frequency domain signal, the smaller the value is, the greater the possibility that the two stage signals currently analyzed are the main speakers.

[0040] Further, in step S2, the final probability that the target analysis signal is the main component information corresponding to the microphone of the main speaker is obtained, and the corresponding calculation formula is: in, Indicates the final probability that the target analysis signal is the main component information corresponding to the microphone of the main speaker; Indicates the possibility that the target analysis signal is the main component information corresponding to the microphone of the main speaker; represents the number of remaining non-target analysis signals in the main signal group; Any stage signal representing non-target analysis signal in the main signal group; Indicates the frequency domain signal corresponding to the target analysis signal and The difference index of the corresponding frequency domain signal.

[0041] It can be explained that here Indicates the number of remaining non-target analysis signals in the main signal group, which is the same as the above. Indicates the number of remaining non-target analysis signals in the main signal group. All are averaged over the current analysis signals to ensure that the calculation results are more accurate and representative.

[0042] To explain, The larger the value of If there is a significant difference between them, the probability that the target analysis signal is the main speech component corresponding to the main speaker microphone is greater.

[0043] Furthermore, in step S2, an adjustment factor is determined based on the final probability of the main component information corresponding to the microphone of the main speaker, and each corresponding signal is reconstructed by weighting the adjustment factor, including: Step S21: obtaining the final probability of all the main component information corresponding to the microphone of the main speaker, and sorting them from large to small, and defining the largest value as the adjustment factor; Step S22: Counting the number of all signals in the non-main signal group and calculating their proportion in the sound source data to be analyzed, and obtaining the adjustment factor of the non-main component information in the main signal group; Step S23: Obtain all adjustment factors of the sound source data to be analyzed, complete the classification of the main component information and non-main component information corresponding to each microphone, use the corresponding adjustment factors to make corresponding adjustments, and obtain the adjusted signal of each microphone through weighted reconstruction.

[0044] It can be explained that weight-based reconstruction of audio signals means that after the signal is assigned weights, the reconstruction process will use these weight values ​​to weighted sum the signals to generate a new audio signal. While maintaining the main features of the original signal, this new signal will be optimized in some aspects, such as improving the signal-to-noise ratio, enhancing specific frequency components or reducing the amount of data.

[0045] It is explained that the determination of the adjustment factor is used for the subsequent weight determination, thereby ensuring that the main speech component of the main speaker can be given priority and highlighted; and the determination of the adjustment factor of the secondary component can distinguish between the main signal and the non-main signal, all for the purpose of highlighting the corresponding audio information of the main speaker; further, each signal can obtain an adaptive weight for signal reconstruction to achieve the best audio effect, ensuring that the importance of the information can be more intelligible in remote meetings or meeting records.

[0046] Furthermore, step S3 includes: Step S31: converting the reconstructed signal from a time domain signal to a frequency domain signal; Step S32: Calculate the signal strength of any microphone, the corresponding calculation formula is: in, represents the signal strength of the reconstructed signal of the kth microphone; Indicates the number of horizontal coordinates of the frequency domain signal corresponding to the signal reconstructed by the kth microphone; Indicates The reconstructed signal of the microphone corresponds to the frequency domain signal The horizontal axis corresponds to the amplitude; Step S33: Obtain the signal strengths of the signals reconstructed by all microphones, sort them from large to small, convert the corresponding frequency domain with the largest signal strength back into the time domain signal, and obtain the audio information reconstructed by the corresponding microphone.

[0047] To explain, Indicates The number of horizontal axes of the frequency domain signal corresponding to the signal reconstructed by each microphone refers to the corresponding number of amplitudes in the horizontal axis of the spectrum diagram.

[0048] It can be explained that selecting the signal with the highest signal strength as the maximum output of the reconstructed signal can ensure that the quality and reliability of the reconstructed signal are in the best state to minimize signal distortion and noise interference, thereby improving the overall performance of the signal; while the remaining reconstructed signals are set to a disabled state, that is, they are not used for subsequent processing or playback, and the information of the currently selected maximum output signal is displayed to the user, such as the corresponding microphone number, speaker identity, etc., and the operation and results of each selection of the maximum output signal are recorded for subsequent analysis and optimization of system performance.

[0049] It can be understood that by improving the existing method so that it can more effectively utilize phase information and more accurately evaluate the importance of each component, the quality of the reconstructed signal can be significantly improved, and the overall effect of audio processing can be improved, so that remote participants or conference recording equipment can more clearly capture and understand the content of the meeting and enhance the user experience; optimizing the weight distribution strategy can also reduce the impact of noise and other interference, and further improve the clarity and intelligibility of the audio signal; significantly improve the usability and reliability of audio processing methods in multi-person conference environments, and provide users with a better audio experience, that is, in a complex acoustic environment, it can still accurately reconstruct the signal, and can maintain high-quality signal processing effects under changing acoustic conditions, ensuring the complete transmission of information.

[0050] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0051] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A microphone voice collection method in a multi-person conference scenario, characterized in that: The method comprises: Acquire the sound source audio captured by any microphone in a multi-person conference scene and perform preprocessing to obtain the sound source data to be analyzed, wherein the microphone includes the microphone of the main speaker; Separate the sound source data to be analyzed into a plurality of original component signals in sequence, analyze the possibility that the original component signal is the main signal to obtain a main signal group, define any stage signal in the main signal group as a target analysis signal, calculate the possibility index of the main speech component based on the target analysis signal, and obtain the possibility that the target analysis signal is the main component information corresponding to the microphone of the main speaker according to the possibility of the main signal and the possibility index of the main speech component; calculate the difference index between the target analysis signal and any stage signal of the non-target analysis signal in the main signal group to obtain the final possibility that the target analysis signal is the main component information corresponding to the microphone of the main speaker; determine the adjustment factor based on the final possibility of the main component information corresponding to the microphone of the main speaker, and reconstruct each corresponding signal by weighting the adjustment factor; The signal strength of any microphone is calculated based on the reconstructed signal, and the reconstructed audio information of the corresponding microphone is obtained through the signal strength.

2. The method for collecting microphone voice in a multi-person conference scenario as claimed in claim 1, characterized in that: The original component signal is composed of any single sound source and / or multiple sound sources including a noise source, a main speaker sound source and a secondary speaker sound source.

3. The method for collecting microphone voice in a multi-person conference scenario as claimed in claim 1, characterized in that: Analyze the possibility that the original component signal is the main signal, and the corresponding calculation formula is: in, and Represent the original component signals and The possibility of being the main signal; Represents the original component signal and Dynamic time warping distance of Represents the original component signal and The mean voltage of Represents the original component signal and The absolute value of the difference in the voltage mean; Represents the normalization function.

4. The method for collecting microphone voice in a multi-person conference scenario as claimed in claim 3, characterized in that: The main signal group is obtained as follows: based on the possibility of the main signal, it is determined whether it exceeds a preset threshold. If so, the current original component signal is determined to be the main signal group; if not, the current original component signal is determined to be the non-main signal group.

5. The method for collecting microphone voice in a multi-person conference scenario as claimed in claim 4, characterized in that: The probability index of the main speech component is calculated based on the target analysis signal, including: Convert the signals in the main signal group from time domain signals to frequency domain signals; The probability index of the main speech component is calculated based on the target analysis signal, and the corresponding calculation formula is: in, represents the target analysis signal; Indicates the probability index of the frequency domain signal corresponding to the target analysis signal being the main speech component; Indicates the stability of energy distribution of the target analysis signal corresponding to the frequency domain signal; Indicates the frequency number of the frequency domain signal corresponding to the target analysis signal; Indicates the target analysis signal corresponds to the frequency domain signal The amplitude at the frequency; Indicates the target analysis signal corresponds to the frequency domain signal The amplitude at a frequency.

6. The method for collecting microphone voice in a multi-person conference scenario as claimed in claim 5, characterized in that: The possibility of obtaining the main component information corresponding to the microphone of the main speaker when the target analysis signal is the main component information includes: Get the highest amplitude of all frequency domain signals in the main signal group; The corresponding calculation formula is: in, Indicates the possibility that the target analysis signal is the main component information corresponding to the microphone of the main speaker; Indicates the probability that the target analysis signal is the main signal; Indicates the highest amplitude of all frequency domain signals in the main signal group; Indicates the highest amplitude of the frequency domain signal corresponding to the target analysis signal.

7. The method for collecting microphone voice in a multi-person conference scenario as claimed in claim 6, characterized in that: Calculate the difference index between the target analysis signal and any stage signal of the non-target analysis signal in the main signal group. The corresponding calculation formula is: in, Any stage signal representing non-target analysis signal in the main signal group; Indicates the frequency domain signal corresponding to the target analysis signal and The difference index of the corresponding frequency domain signal; Indicates that the target analysis signal is The phase at the frequency; express In the The phase at the frequency; Indicates the target analysis signal corresponds to the frequency domain signal The amplitude at the frequency; express The corresponding frequency domain signal The amplitude at the frequency; Indicates the frequency number of the frequency domain signal corresponding to the target analysis signal; is a natural constant.

8. The method for collecting microphone voice in a multi-person conference scenario as claimed in claim 7, characterized in that: The final probability of obtaining the target analysis signal as the main component information corresponding to the microphone of the main speaker is calculated as follows: in, Indicates the final probability that the target analysis signal is the main component information corresponding to the microphone of the main speaker; Indicates the possibility that the target analysis signal is the main component information corresponding to the microphone of the main speaker; represents the number of remaining non-target analysis signals in the main signal group; Any stage signal representing non-target analysis signal in the main signal group; Indicates the frequency domain signal corresponding to the target analysis signal and The difference index of the corresponding frequency domain signal.

9. The method for collecting microphone voice in a multi-person conference scenario as claimed in claim 8, characterized in that: An adjustment factor is determined based on the final likelihood of the main component information corresponding to the microphone of the main speaker, and each corresponding signal is reconstructed by weighting the adjustment factor, including: Obtain the ultimate possibility of all corresponding main component information of the main speaker's microphone, and sort them from large to small, and define the largest value as the adjustment factor; Counting the number of all signals in the non-main signal group and calculating their proportion in the sound source data to be analyzed, and obtaining the adjustment factor of the non-main component information in the main signal group; All adjustment factors of the sound source data to be analyzed are obtained, the main component information and non-main component information corresponding to each microphone are classified, corresponding adjustment factors are used for corresponding adjustment, and the adjusted signal of each microphone is obtained through weighted reconstruction.

10. The method for collecting microphone voice in a multi-person conference scenario as claimed in claim 9, characterized in that: The signal strength of any microphone is calculated based on the reconstructed signal, and the reconstructed audio information of the corresponding microphone is obtained through the signal strength, including: Convert the reconstructed signal from a time domain signal to a frequency domain signal; To calculate the signal strength of any microphone, the corresponding calculation formula is: in, Indicates The signal strength of the reconstructed signal from each microphone; Indicates the number of horizontal coordinates of the frequency domain signal corresponding to the signal reconstructed by the kth microphone; Indicates The reconstructed signal of the microphone corresponds to the frequency domain signal The horizontal axis corresponds to the amplitude; The signal strengths of all the microphone reconstructed signals are obtained, sorted from large to small, and the corresponding frequency domain with the largest signal strength is converted back to the time domain signal to obtain the audio information reconstructed by the corresponding microphone.

Images