Pickup control method and device based on sound position recognition

Abstract

The invention relates to a pickup control method and device based on sound position recognition, and the method comprises the steps: obtaining input data of a plurality of microphones, determining sound source feature data based on the input data of the plurality of microphones, and modulating the output of a target sound source and / or a non-target sound source based on the sound source feature data. According to the embodiment of the invention, the target sound and the non-target sound can be automatically identified and separated according to the sound source feature data, the output of the target sound is enhanced, the communication quality is improved, the non-target sound is effectively inhibited, the interference of background noise is reduced, and the responsiveness and efficiency of a sound processing system are improved.

Description

[Technical field] The present application relates to the technical field of conference sound pickup, and in particular to a sound pickup control method and device based on voice position recognition. [Background technology] In modern communication and automation systems, sound pickup technology plays a vital role. In the fields of multimedia communication, conference systems, smart homes and security monitoring, audio data is mainly collected by a single or a limited number of microphones. These systems usually improve audio quality through basic sound processing algorithms such as noise suppression and echo cancellation.

[0003] However, in simple or controlled environments, the above settings may be sufficient. As application requirements become more complex, single or limited microphone configurations often cannot meet a wider range of application scenarios. When dealing with complex sound environments, especially when there are multiple sound sources, they show obvious limitations and often find it difficult to accurately locate the direction of the sound source, resulting in an inability to effectively suppress background noise and affecting the clarity of the target sound. [Summary of the invention] In order to solve the existing problems, on the one hand, the present invention provides a sound pickup control method based on voice position recognition, comprising: Get input data from multiple microphones; Determining sound source characteristic data based on input data from the plurality of microphones; Based on the sound source characteristic data, the output of the target sound source and / or the non-target sound source is modulated.

[0005] After the step of determining the sound source characteristic data based on the input data of the plurality of microphones, the method further includes: A target sound source and / or a non-target sound source is determined based on at least the input data of one of the microphones.

[0006] The input data includes an intensity difference parameter, a time difference parameter, a phase difference parameter, and a timbre difference parameter, and the sound source characteristic data includes a distance parameter and a direction parameter. The sound source characteristic data is determined based on the input data of the plurality of microphones, including: Obtain the sound intensity from the sound source to each microphone and the phase parameters of the sound source at each microphone; Based on the sound intensity from the sound source to each microphone, obtaining an intensity difference parameter; Based on the phase parameters from the sound source to each microphone, a phase difference parameter is obtained; Determining a distance parameter based on the intensity difference parameter and the phase difference parameter; Based on the time before and after the sound arrives at each microphone, a time difference parameter is obtained; Perform spectrum analysis on the sound source reaching each microphone to obtain the main frequency parameters; Determining timbre difference parameters based on the main frequency parameters of each microphone; The direction parameter is determined based on the time difference parameter and the timbre difference parameter.

[0007] Determining a target sound source and / or a non-target sound source based on at least one of the microphone input data comprises: Acquire preset sound pickup parameters, wherein the sound pickup parameters include a sound pickup range and a sound pickup direction; Based on the distance parameter, determining whether the sound source falls within a preset sound pickup range; Based on the direction parameter, determining whether the sound source is in the sound pickup direction; If both the distance parameter and the direction parameter satisfy the sound pickup parameter, the sound source that meets the condition is classified as the target sound output; If the distance parameter and / or the direction parameter do not satisfy the sound pickup parameter, the sound source that meets the condition is classified as a non-target sound output.

[0008] Modulate the output of a target sound source and / or a non-target sound source, including: The non-target sound output is suppressed and the target sound output is enhanced.

[0009] The suppressing the non-target sound output and enhancing the target sound output comprises: Obtaining environmental noise parameters, frequency characteristics of non-target sound output, and average volume of target sound output; Obtaining background noise characteristics based on the environmental noise parameters and the frequency characteristics of the non-target sound output; adjusting the audio output of the non-target sound output according to the background noise characteristics; Determining a gain parameter based on the ambient noise parameter and an average volume of the target sound output; The audio output of the target sound output is adjusted according to the gain parameter and the sound source characteristic data.

[0010] The environmental noise parameters include sound pressure parameters and spectrum distribution parameters. The background noise characteristics are obtained based on the environmental noise parameters and the frequency characteristics of the non-target sound output. The audio output of the non-target sound output is adjusted according to the background noise characteristics, including: Based on the sound pressure parameter, a noise threshold is obtained, and the sound exceeding the noise threshold is transmitted, and the sound below the noise threshold is blocked or greatly weakened; Based on the spectrum distribution parameter, dynamic range compression is performed on non-target sound output exceeding the noise threshold.

[0011] Determining a gain parameter based on the environmental noise parameter and an average volume of the target sound output, and adjusting the audio output of the target sound output according to the gain parameter and the sound source characteristic data, including: Obtaining a signal-to-noise ratio according to the environmental noise parameter and an average volume of the target sound output; Based on the signal-to-noise ratio, determining a gain parameter; Analyze the frequency response, volume, and time domain characteristics of the target sound output to determine the stability and change trend of the target sound output; Dynamically adjust gain parameters if the target sound output moves or background noise changes; According to the gain parameter, gain adjustment is performed on the target sound output.

[0012] In a second aspect, an embodiment of the present application provides a sound pickup control device based on voice position recognition, comprising: An acquisition module, used for acquiring input data of multiple microphones; A determination module, used to determine sound source feature data according to input data of the plurality of microphones; A processing module is used to modulate the non-target sound output and / or the target sound output according to the sound source characteristic data.

[0013] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps in the sound pickup control method based on voice position recognition as described in the first aspect above are implemented.

[0014] In a fourth aspect, an embodiment of the present application provides an electronic device, characterized in that it includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps in the sound pickup control method based on voice position recognition as described in the first aspect above are implemented.

[0015] The embodiment of the present invention can automatically identify and separate target sounds and non-target sounds based on sound source feature data, which not only enhances the output of target sounds and improves communication quality, but also effectively suppresses non-target sounds, reduces interference from background noise, and improves the responsiveness and efficiency of the sound processing system.

Brief Description of the Drawings

[0017] Figure 1 is a flow chart of a sound pickup control method based on voice position recognition according to a first embodiment of the present invention; Figure 2 yes Figure 1 Detailed flow chart of step S12; Figure 3 yes Figure 1 Detailed flow chart of step S13; Figure 4 yes Figure 1 Detailed flow chart of step S14; Figure 5 yes Figure 4 Detailed flow chart of step S143; Figure 6 yes Figure 4 Detailed flow chart of step S145; Figure 7 is a structural block diagram of a sound pickup control device based on voice position recognition according to a second embodiment of the present invention; Figure 8 is a structural block diagram of a computer device according to another embodiment of the present invention; Fig. 9 It is a structural block diagram of an electronic device according to another embodiment of the present invention. [Specific implementation method] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments, and the well-known modules, units and their connections, links, communications or operations are not shown or described in detail. In addition, the described features, architectures or functions can be combined in any way in one or more embodiments. It should be understood by those skilled in the art that the various embodiments described below are only for illustration and not for limiting the scope of protection of the present invention. It can also be easily understood that the modules or units or processing methods in the various embodiments described herein and shown in the drawings can be combined and designed according to various different configurations. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0019] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0020] The limitations of various nouns or methods referred to in the following embodiments, except for those that are logically untenable, are generally based on the broad concepts that can be implemented under the premise of the disclosure in the embodiments. Under such understanding, the various specific subordinate specific limitations of the nouns or methods should be regarded as the inventive content of the present invention, and should not be understood narrowly or interpreted in a biased manner on the grounds that the specification does not disclose the specific limitations. Similarly, under the premise that it can be implemented logically, the order of the steps in the method is flexible and changeable, and the specific subordinate specific limitations in the broad concepts of various nouns or methods all belong to the protection scope of the present invention.

[0021] First embodiment: Please refer to Figures 1 to 6 As shown, this embodiment proposes a sound pickup control method based on voice position recognition, including S11-S14, wherein: S11, obtaining input data from multiple microphones.

[0022] Specifically, the input data includes intensity difference parameters, time difference parameters, phase difference parameters, and timbre difference parameters.

[0023] Specifically, the intensity difference parameter represents the difference in sound intensity of the same sound source captured by different microphones, and is used to determine the relative distance between the sound source and each microphone. A microphone closer to the sound source generally receives a greater sound intensity. By calculating the intensity difference parameter, the location information of the sound source can be estimated.

[0024] Specifically, the time difference parameter is the time difference between the sound signal reaching different microphones. Using these time differences, the direction of the sound source can be accurately determined through sound source localization technology (such as beamforming or triangulation).

[0025] Specifically, the phase difference parameter is the phase difference of the sound signal when it is received by different microphones, which can provide a finer spatial resolution than the time difference and is used to enhance the accuracy of sound source positioning.

[0026] Specifically, the timbre difference parameter is based on the frequency response differences of sounds received by different microphones. Different sound sources (such as human voices, vehicles, etc.) have different performances on the frequency spectrum, which can reveal the type and characteristics of the sound source. By analyzing the timbre difference, specific types of sound sources can be distinguished and prioritized, such as giving priority to clear transmission of the speaker's voice in a conference system.

[0027] S12, determining sound source feature data based on input data from the plurality of microphones.

[0028] The sound source characteristic data includes distance parameters and direction parameters; Specifically, multiple microphones are deployed in the environment. The sound signals captured by each microphone include information such as timestamp, intensity, frequency, and phase, and the data collected from each microphone is synchronized in time. All the above parameters are fused through an algorithm to obtain specific distance parameters and direction parameters.

[0029] As a preferred solution but not a limitation, please refer to Figure 2 As shown, this step S12 also includes S121-S128, wherein: S121, obtaining the sound intensity from the sound source to each microphone and the phase parameter of the sound source at each microphone; S122, obtaining an intensity difference parameter based on the sound intensity from the sound source to each microphone; S123, obtaining a phase difference parameter based on the phase parameter from the sound source to each microphone; S124, determining a distance parameter based on the intensity difference parameter and the phase difference parameter; S125, obtaining a time difference parameter based on the time before and after the sound reaches each microphone; S126, performing spectrum analysis on the sound source reaching each microphone to obtain main frequency parameters; S127, determining a timbre difference parameter based on a main frequency parameter of each microphone; S128, determining the direction parameter based on the time difference parameter and the timbre difference parameter.

[0030] In steps S121-S122, the intensity of the sound waves emitted by the sound source is first recorded (completed by the analog-to-digital converter of the microphone, which converts the pressure changes of the sound waves into voltage changes, and then converts them into digital signals representing the amplitude of the sound), and then the raw intensity data collected by each microphone is normalized to eliminate deviations caused by different microphone sensitivities or distances.

[0031] Select a reference microphone (usually the microphone at the center of the location or the one that receives the sound signal first) and calculate the difference between the sound intensity received by the other microphones and the reference microphone.

[0032] The calculated intensity difference parameter reflects the relative distance difference between the sound source and each microphone (how to calculate the intensity difference is a prior art). The closer the sound source is to a microphone, the greater the sound intensity recorded by the microphone, and the smaller the corresponding intensity difference.

[0033] In step S123, each microphone captures the phase value of the sound wave when it arrives, selects a reference microphone, and calculates the difference between the phase of the sound wave received by each microphone and the phase of the reference microphone (how to calculate the phase value is a prior art and will not be repeated in the following text).

[0034] In step S124, during the propagation of sound waves in the air, the sound intensity and phase will change with the increase of distance. The attenuation of intensity with distance follows the inverse square law, while the relationship between the change of phase and distance is relatively direct. The intensity difference parameter is used to preliminarily estimate the distance of the sound source (the relationship between sound intensity and distance is relatively clear), and the phase difference parameter provides an accurate measure of the propagation distance of the sound wave. Especially in short-distance or high-frequency sound processing, the intensity difference parameter and the phase difference parameter can be combined to accurately estimate the distance of the sound source by solving a set of simultaneous equations or using an optimization algorithm. For example, the most likely sound source position can be determined by minimizing the prediction errors of the intensity difference and the phase difference.

[0035] In step S125, by comparing the time when different sounds arrive at the microphone, the time difference of the sound arrival can be obtained, and then multiple time difference parameters are combined and used to further accurately determine the three-dimensional position of the sound source through multilateral measurement or other algorithms (the sound source position is updated in real time).

[0036] In step S126, first, the sound signals captured from each microphone are preprocessed, such as filtering and gain adjustment, to optimize the quality of spectrum analysis, and then the time domain sound signals are converted into frequency domain signals using FFT. The converted frequency spectrum is analyzed to identify the frequency with the highest energy or the main frequency component, and the nature or category of the sound source (human voice, mechanical sound, etc.) is inferred by comparing the main frequencies received by different microphones.

[0037] In step S127 , the spectrum data of each microphone (the intensity and distribution of the main frequency components) are first compared, and then the spectrum difference index (such as the difference or ratio of the frequency components, etc.) is calculated.

[0038] In step S128, the sound wave arrival time information provided by the time difference parameter and the spectrum information provided by the timbre difference parameter are used to accurately estimate the direction of the sound source (such as multidimensional scaling analysis or machine learning algorithm), and then the parameter calculation model (existing technology) is adjusted according to environmental feedback to optimize the accuracy of the direction estimation.

[0039] In general, steps S121 to S128 constitute a systematic sound source feature data identification and processing flow, which greatly improves the accuracy and efficiency of sound signal processing. At the same time, steps S121 to S129 also provide a comprehensive technical framework for the precise processing of sound signals. From sound source detection to sound optimization, each step enhances the overall performance and flexibility.

[0040] S13, determining a target sound source and / or a non-target sound source based on input data of at least one of the microphones.

[0041] In step S13, the input data of at least one microphone is used to determine the target sound source and / or non-target sound source. This is because in some simple or resource-constrained applications, there may be only one available microphone. In this case, the sound source can be classified relying solely on the data of this one microphone, ensuring that the system can operate normally even in the case of hardware limitations. Secondly, in complex or noisy environments, some microphones may capture unreliable data due to poor positioning or temporary failures. Data from other microphones can be selected for sound source determination, thereby enhancing the system's fault tolerance and environmental adaptability.

[0042] Specifically, when the location of the sound source is relatively clear or the environment is relatively simple, the data from a single microphone may be sufficient to effectively classify the sound source, which can reduce the consumption of computing resources and improve the processing speed of the system. In scenarios that require a quick response, quick judgment based on a single microphone can significantly reduce latency and provide a more timely user experience.

[0043] In cost-sensitive product designs, it is preferred to use fewer microphones. Designs that rely on only one microphone can significantly reduce overall hardware costs and make products easier to deploy and maintain in situations where physical space is limited or the device needs to be portable.

[0044] Step S13 identifies and classifies the target sound source and the non-target sound source according to the sound source feature data determined from the multiple microphone input data collected and analyzed in step S12. This step is a key step in realizing sound signal processing and directly affects the quality of subsequent sound processing and output. The details are as follows: As a preferred solution but not a limitation, please refer to Figure 3 As shown, this step S13 also includes S131-S135, wherein: S131, obtaining preset sound pickup parameters, where the sound pickup parameters include a sound pickup range and a sound pickup direction; S132, based on the distance parameter, determining whether the sound source falls within a preset sound pickup range; S133, judging whether the sound source is in the sound pickup direction based on the direction parameter; S134, if both the distance parameter and the direction parameter satisfy the sound pickup parameter, the sound source that meets the condition is classified as a target sound output; S135: If the distance parameter and / or the direction parameter do not satisfy the sound pickup parameter, the sound source that meets the condition is classified as a non-target sound output.

[0045] In step S131, the pickup range refers to the physical or geometric range of the sound signal that should be paid attention to and processed. The pickup range can be based on a specific distance metric, such as the farthest and closest receiving distances, or the shape of a specific area, such as a cone, a sphere, etc., and the pickup direction defines the direction in which the sound is expected to be captured. In a multi-microphone system, one or more directions will be specified to prioritize the sound from these directions. The pickup parameters are usually preset by the user according to the specific application scenario. For example, in a conference room, the pickup range covers the entire conference table, and the pickup direction is for the speaker sitting on one side of the table. In more advanced systems, the pickup parameters can be dynamically adjusted by automatically detecting the environment and sound sources, such as using a sound source localization algorithm to automatically determine the most active sound source direction.

[0046] Specifically, by analyzing the voice intensity and direction of each participant in real time, the current main speaker is automatically identified, and using sound source localization technology (existing technologies such as beamforming or sound source localization algorithms), the system can dynamically adjust the directionality of the microphone array to ensure that the microphone array always points to the current speaker. When the system detects a new sound source or the current speaker moves, the microphone array will automatically adjust its pickup pattern.

[0047] In steps S132-S135, the distance parameter obtained from step S124 is used to determine whether the sound source is within the preset sound pickup range. For example, if the distance parameter of the sound source indicates that it exceeds the preset maximum receiving distance, the sound source is regarded as a non-target sound source.

[0048] Specifically, the distance parameter of each sound source is compared with the preset pickup range. If the distance of the sound source is less than or equal to the maximum distance of the preset range, the sound source is judged to be within the pickup range; if it exceeds this distance, the sound source is regarded as a non-target sound source. For example, in a conference room, the preset pickup range is a circular area 5 meters outward from the center of the microphone. The actual distance between the speaker and the center of the microphone is calculated. If this distance is 6 meters, that is, it exceeds the preset range of 5 meters, then the speaker's voice will be automatically classified as a non-target sound source and will not be actively enhanced or recorded.

[0049] S14, modulating the output of the target sound source and / or the non-target sound source based on the sound source feature data.

[0050] Specifically, step S14 is to suppress the non-target sound output and enhance the target sound output.

[0051] Step S14 involves modulating and processing the audio signals that have been classified into target sound sources and non-target sound sources to optimize the output quality and clarity of the sound. The following is a detailed explanation of this step: As a preferred solution but not a limitation, please refer to Figure 4 As shown, this step S14 also includes S141-S145, wherein: S141, obtaining environmental noise parameters, frequency characteristics of non-target sound output, and average volume of target sound output; S142, obtaining background noise characteristics based on the environmental noise parameters and the frequency characteristics of the non-target sound output; S143, adjusting the audio output of the non-target sound output according to the background noise characteristics; S144, determining a gain parameter based on the environmental noise parameter and an average volume of the target sound output; S145: Adjust the audio output of the target sound output according to the gain parameter and the sound source characteristic data.

[0052] In step S141 , the noise and volume levels are quantitatively measured using a sound level meter or spectrum analysis software.

[0053] Specifically, the environmental noise parameters include the intensity, frequency distribution and duration of the noise, which can be obtained from the environmental audio data captured by the microphone through spectrum analysis.

[0054] Specifically, the frequency characteristics of non-target sound output: analyze the main frequency composition of non-target sound sources and identify which frequencies are common interfering audio.

[0055] Specifically, the average volume of the target sound output: Measure the average volume of the target sound source for comparison with the ambient noise and appropriate volume adjustment.

[0056] In step S142, the sound in the environment is first continuously monitored by the microphone array, including background noise and various non-target sound sources (such as traffic noise, air conditioning sound, etc.), and the frequency characteristics of the non-target sound are recorded. The frequency characteristics of the non-target sound include the intensity, frequency distribution, duration and time pattern of the sound. Before performing spectral analysis on the sound signal, some preprocessing steps are required to remove the DC offset, perform window function processing to reduce spectrum leakage, and apply basic noise suppression algorithms to clearly separate the sound signal. Then, the fast Fourier transform (FFT) or other spectral analysis methods are used to extract frequency components from the environmental noise and non-target sound, determine which frequencies are the main components of the environmental noise, combine the environmental noise parameters with the frequency characteristics of the non-target sound, analyze the overlap and interaction between the two, and based on the results of the frequency analysis, identify and mark those frequency bands that are significant in both the environmental noise and the non-target sound. Then, based on the synthesized data, extract the characteristics of the background noise.

[0057] In step S143, it is determined which frequency bands need to be suppressed or adjusted according to the characteristics of the background noise, with reference to steps S1431-S1432 for details.

[0058] In step S144, based on the volume comparison between the target sound and the background noise, the required gain is calculated, and based on the average volume of the ambient noise and the target sound, it is determined whether the target sound is prominent enough. If the target sound is not obvious relative to the background noise, then the gain needs to be increased, and then the gain is adjusted according to the degree of prominence required.

[0059] Specifically, the average volume of road traffic noise outside the conference room and the air conditioning noise inside the conference room is about 60 dB. A speaker is speaking, but due to the distance from the microphone, the average volume of his voice is only 55 dB. The system first measures the ambient noise (60 dB) and the speaker's voice (55 dB). Since the volume of the target sound (55 dB) is lower than the ambient noise (60 dB), the system recognizes that the volume of the speaker's voice needs to be increased to ensure that it stands out in the noise. Assuming that the system design requires the speaker's voice to be at least 10 dB higher than the background noise to ensure clear hearing, the target sound needs to reach at least 70 dB, and the speaker's voice needs to be increased from 55 dB to 70 dB, so a gain of at least 15 dB needs to be set.

[0060] Specifically, the process of increasing from 55 dB to 70 dB can use adaptive gain control AGC (existing technology). The AGC system monitors the volume of the sound in real time and automatically adjusts the gain to maintain the prominence of the target sound relative to the background noise to achieve a gain of at least 15 dB.

[0061] In step S145, the audio output of the target sound output is adjusted according to the gain parameter and the sound source characteristic data, with specific reference to steps S1451-S1455.

[0062] As a preferred solution but not a limitation, please refer to Figure 5 As shown, this step S143 also includes S1431-S1432, wherein: The environmental noise parameters include sound pressure parameters and spectrum distribution parameters; The sound pressure parameter is a physical parameter that measures the intensity of sound and is an important indicator for assessing the environmental noise level.

[0063] The spectrum distribution parameters describe the distribution and intensity of different frequency components in the sound signal. The spectrum distribution parameters can show which frequency areas are most prominent and the distribution of sound energy on the spectrum.

[0064] S1431, based on the sound pressure parameter, obtaining a noise threshold, wherein sounds exceeding the noise threshold are transmitted, and sounds below the noise threshold are blocked or greatly weakened; S1432: Based on the spectrum distribution parameters, dynamic range compression is performed on non-target sound output exceeding the noise threshold.

[0065] A noise threshold is set based on the measured sound pressure of ambient noise. By comparing the sound pressure of each sound event with the noise threshold, a sound event is retained if its sound pressure is higher than the noise threshold. If it is lower than the noise threshold, it is regarded as background noise and is blocked or its volume is greatly reduced accordingly. The noise threshold is set to distinguish important sounds from background noise, so that the system can focus only on sounds that are loud enough and may contain important information.

[0066] The noise threshold is a set boundary used to distinguish which sounds are loud enough to be transmitted and which sounds are too low to be blocked or attenuated. This threshold is set based on the normal noise level in the environment and aims to retain useful sound signals while filtering out unnecessary background noise.

[0067] Dynamic range compression is an audio processing technique used to reduce the gap between the maximum and minimum volumes in a sound signal. By applying this technique, the sound can be made more uniform and prominent peaks can be reduced, thus preventing non-target sounds from interfering with the clarity of the target sound.

[0068] In steps S1431-S1432, the ambient sound data is first collected using a microphone array, and then the sound pressure level of the sound is measured using a sound level meter or audio analysis software to obtain the average sound pressure of the ambient noise, thereby providing a basis for setting the noise threshold. Based on the measured ambient noise sound pressure, a noise threshold is set, and a spectrum analysis is performed on all sounds that exceed the noise threshold (the purpose of the analysis is to identify which frequency components are too prominent). The dynamic range compression technology is used to process the identified overly prominent frequency components (dynamic range compression aims to reduce the difference between the maximum and minimum volume in the sound signal). By adjusting the compression ratio and threshold, the system reduces the dynamic range of these frequency components, thereby reducing their energy or prominence, and dynamically adjusts according to real-time audio input to most effectively process non-target sounds and reduce interference with target sounds.

[0069] As a preferred solution but not a limitation, please refer to Figure 6 As shown, this step S145 also includes steps S1451-S1455, wherein: S1451, obtaining a signal-to-noise ratio according to the environmental noise parameter and an average volume of the target sound output; S1452, determining a gain parameter based on the signal-to-noise ratio; S1453, analyzing the frequency response, volume, and time domain characteristics of the target sound output to determine the stability and change trend of the target sound output; S1454, if the target sound output moves or the background noise changes, dynamically adjust the gain parameter; S1455: Perform gain adjustment on the target sound output according to the gain parameter.

[0070] in: The signal-to-noise ratio is a key indicator for measuring the clarity of the target sound, which represents the ratio of the intensity of the target sound to the intensity of the background noise. A high signal-to-noise ratio means that the target sound is clearer and more discernible than the background noise.

[0071] The gain parameter is a setting value for adjusting the target sound output to ensure that it stands out auditorily. The gain parameter is determined based on the signal-to-noise ratio and is intended to enhance the target sound to sufficiently exceed the background noise to ensure the quality of communication or recording.

[0072] Frequency response describes the response strength of the target sound at different frequencies.

[0073] The time domain characteristics involve the changing characteristics of the sound signal over time, such as the fluctuations and duration of the sound.

[0074] Stability and change trend are determined by analyzing these data to determine the stability and possible change trend of the target sound.

[0075] In steps S1451-S1455, a microphone array is used to continuously monitor and record the noise level in the environment, and then the average volume of the ambient noise is calculated using a sound level meter or dedicated audio analysis software. At the same time, the microphone array is used to capture the target sound signal and calculate the average volume of the target sound to determine the signal-to-noise ratio (the signal-to-noise ratio is the difference between the target sound volume and the ambient noise volume). If the signal-to-noise ratio is lower than the expected value, the system needs to calculate the amount of gain that needs to be increased to increase the volume of the target sound to the ideal level. The specific value of the gain will be determined based on the difference in the signal-to-noise ratio and the performance standard of the sound processing system.

[0076] Specific: 1. Use the conference deep learning model to identify and classify background noise and target sounds in real time. Model training includes sound samples in a variety of typical environments to improve its recognition accuracy.

[0077] 2. Model environmental noise using advanced sound analysis tools such as spectral subtraction and sound activity detection, and update the noise model in real time to adapt to environmental changes.

[0078] 3. Integrated machine learning algorithms not only calculate the current signal-to-noise ratio, but also predict the change in signal-to-noise ratio in the short term in the future to adjust the gain in advance.

[0079] 4. Dynamically adjust the gain based on the predicted signal-to-noise ratio, automatically determine the timing and amplitude of gain adjustment, and optimize the listening quality of the target sound.

[0080] 5. Analyze the frequency response of the target sound, identify key voice bands, and ensure that these bands are enhanced first.

[0081] 6. Apply time series analysis techniques to evaluate the time domain characteristics of the target sound, such as energy variation and stability, to more finely control the audio output.

[0082] 7. Implement sound source localization technology and ambient noise tracking to adjust gain instantly when the speaker moves or the background noise level changes.

[0083] Specifically, the training method of the conference deep learning model is as follows: S1. Data collection and preprocessing Multi-environment recordings: Collect sound data from a variety of typical and atypical environments, such as traffic noise, inside conference rooms, public places, etc. The data should include different types of background noise and multiple target sounds (such as different speakers, speech of different pitches and intensities).

[0084] High-resolution recordings: Using high-quality microphones and recording equipment to capture a wide frequency range at a high sampling rate (e.g., 96 kHz) ensures that the data contains rich acoustic information.

[0085] Sound Enhancement and Denoising: Sound enhancement and denoising algorithms, such as spectral subtraction and wavelet transform, are applied after recording to remove non-target noise and enhance speech signals.

[0086] S2. Feature Engineering Deep feature extraction: Use deep learning techniques such as convolutional neural networks (CNN) or recurrent neural networks (RNN) to automatically learn and extract features from audio data. In particular, Mel frequency cepstral coefficients (MFCC) or spectrograms can be used as input features.

[0087] Spatiotemporal feature fusion: Long short-term memory network (LSTM) or Transformer model is used to process the temporal dependencies of audio signals, while capturing dynamic changes in time through cross-frame feature fusion.

[0088] S3. Model structure and training strategy Multi-task learning: Design a multi-task learning architecture to simultaneously predict the noise category, signal-to-noise ratio, and intelligibility of the target speech. This can be achieved by using multiple output layers at the end of the network, each dedicated to one prediction task.

[0089] Adversarial training: Generative adversarial networks (GANs) are used to enhance the model’s robustness to noise. In this way, the generator tries to generate noise samples that are difficult to distinguish, while the discriminator strives to improve the ability to distinguish between noise and speech.

[0090] Data augmentation: Use audio data augmentation techniques, such as speed change, pitch change, and adding synthetic noise, to increase the generalization ability of the model. In particular, these parameters can be dynamically adjusted during training to make the model adapt to extreme and unseen noise conditions.

[0091] S4. Optimization and evaluation Hyperparameter optimization: Use automated machine learning (AutoML) tools to optimize network architecture and learning parameters such as learning rate, batch size, and number of layers.

[0092] Cross-validation: A stratified cross-validation strategy is used to evaluate the performance of the model to ensure that the model maintains stable performance in various environments.

[0093] Real-time performance monitoring: Use real-time performance monitoring in actual deployments to adjust model parameters through continuous feedback to cope with environmental changes or new types of noise.

[0094] The first embodiment of the present invention can automatically identify and separate target sounds and non-target sounds based on sound source feature data, which not only enhances the output of target sounds and improves communication quality, but also effectively suppresses non-target sounds, reduces interference from background noise, and improves the responsiveness and efficiency of the sound processing system.

[0095] Second embodiment: Please refer to Figure 7 As shown, this embodiment proposes a sound pickup control device 100 based on voice position recognition, including: An acquisition module 110, used to acquire input data from multiple microphones; A determination module 120, configured to determine sound source feature data based on input data from the plurality of microphones; The processing module 130 is used to modulate the non-target sound output and / or the target sound output according to the sound source characteristic data.

[0096] The second embodiment of the sound pickup control device 100 based on voice position recognition efficiently optimizes sound capture and processing through its three core modules - acquisition module 110, determination module 120 and processing module 130: the acquisition module 110 uses multiple microphones to collect sound data from multiple spatial positions to ensure accurate capture of sound sources from various environments; the determination module 120 further analyzes these data to identify key features such as the position and movement direction of the sound source in real time; finally, the processing module 130 effectively enhances the target sound and suppresses background noise based on these features, which not only significantly improves the clarity and intelligibility of the sound, but also greatly optimizes the user's auditory experience.

[0097] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the above-described system, device and unit can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0098] An embodiment of the present invention further provides a computer storage medium on which a computer program is stored. When the program is executed by a processor, a sound pickup control method based on voice position recognition as described in the above embodiments is implemented.

[0099] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of each embodiment of the pickup control method based on voice position recognition as described above. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0100] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to execute all or part of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, RAM, ROM, magnetic disks or optical disks.

[0101] Corresponding to the above-mentioned computer storage medium, in one embodiment, a computer device is also provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a sound pickup control method based on voice position recognition as described in the above-mentioned embodiments is implemented.

[0102] The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a sound pickup control method based on voice position recognition is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0103] The electronic device in the embodiment of the present application can be various types of electronic devices, or it can be a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or it can be other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0104] Alternatively, if Fig. 9 As shown, the embodiment of the present application also provides an electronic device 200, which can be a sound pickup speaker for conferences, including a processor 201, a memory 202, and a program or instruction stored in the memory 202 and executable on the processor 201. When the program or instruction is executed by the processor 201, each process of the above-mentioned embodiment of a sound pickup control method based on voice position recognition is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0105] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0106] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A sound pickup control method based on voice position recognition, characterized in that: include: Get input data from multiple microphones; Determining sound source characteristic data based on input data from the plurality of microphones; Based on the sound source characteristic data, the output of the target sound source and / or the non-target sound source is modulated.

2. The control method according to claim 1, characterized in that: After the step of determining the sound source feature data based on the input data of the plurality of microphones, the method further includes: A target sound source and / or a non-target sound source is determined based on at least the input data of one of the microphones.

3. The control method according to claim 1, characterized in that: The input data includes an intensity difference parameter, a time difference parameter, a phase difference parameter, and a timbre difference parameter, and the sound source characteristic data includes a distance parameter and a direction parameter. The step of determining the sound source characteristic data based on the input data of the plurality of microphones includes: Obtain the sound intensity from the sound source to each microphone and the phase parameters of the sound source at each microphone; Based on the sound intensity from the sound source to each microphone, obtaining an intensity difference parameter; Based on the phase parameters from the sound source to each microphone, a phase difference parameter is obtained; Determining a distance parameter based on the intensity difference parameter and the phase difference parameter; Based on the time before and after the sound arrives at each microphone, a time difference parameter is obtained; Perform spectrum analysis on the sound source reaching each microphone to obtain the main frequency parameters; Determining timbre difference parameters based on the main frequency parameters of each microphone; The direction parameter is determined based on the time difference parameter and the timbre difference parameter.

4. The control method according to claim 3, characterized in that: The determining of the target sound source and / or the non-target sound source based on at least one of the microphone input data comprises: Acquire preset sound pickup parameters, wherein the sound pickup parameters include a sound pickup range and a sound pickup direction; Based on the distance parameter, determining whether the sound source falls within a preset sound pickup range; Based on the direction parameter, determining whether the sound source is in the sound pickup direction; If both the distance parameter and the direction parameter satisfy the sound pickup parameter, the sound source that meets the condition is classified as the target sound output; If the distance parameter and / or the direction parameter do not satisfy the sound pickup parameter, the sound source that meets the condition is classified as a non-target sound output.

5. The control method according to claim 1, characterized in that: The step of modulating the output of the target sound source and / or the non-target sound source comprises: The non-target sound output is suppressed and the target sound output is enhanced.

6. The control method according to claim 5, characterized in that: The suppressing the non-target sound output and enhancing the target sound output comprises: Obtaining environmental noise parameters, frequency characteristics of non-target sound output, and average volume of target sound output; Obtaining background noise characteristics based on the environmental noise parameters and the frequency characteristics of the non-target sound output; adjusting the audio output of the non-target sound output according to the background noise characteristics; Determining a gain parameter based on the ambient noise parameter and an average volume of the target sound output; The audio output of the target sound output is adjusted according to the gain parameter and the sound source characteristic data.

7. The control method according to claim 6, characterized in that: The environmental noise parameters include sound pressure parameters and spectrum distribution parameters, and the background noise characteristics are obtained based on the environmental noise parameters and the frequency characteristics of the non-target sound output, and the audio output of the non-target sound output is adjusted according to the background noise characteristics, including: Based on the sound pressure parameter, a noise threshold is obtained, and the sound exceeding the noise threshold is transmitted, and the sound below the noise threshold is blocked or greatly weakened; Based on the spectrum distribution parameter, dynamic range compression is performed on non-target sound output exceeding the noise threshold.

8. The control method according to claim 7, characterized in that: The step of determining a gain parameter based on the ambient noise parameter and the average volume of the target sound output, and adjusting the audio output of the target sound output according to the gain parameter and the sound source characteristic data, comprises: Obtaining a signal-to-noise ratio according to the environmental noise parameter and an average volume of the target sound output; Based on the signal-to-noise ratio, determining a gain parameter; Analyze the frequency response, volume, and time domain characteristics of the target sound output to determine the stability and change trend of the target sound output; Dynamically adjust gain parameters if the target sound output moves or background noise changes; According to the gain parameter, gain adjustment is performed on the target sound output.

9. A sound pickup control device based on voice position recognition, characterized in that: include: An acquisition module, used for acquiring input data of multiple microphones; A determination module, used to determine sound source feature data according to input data of the plurality of microphones; A processing module is used to modulate the non-target sound output and / or the target sound output according to the sound source characteristic data.

10. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the sound pickup control method based on voice position recognition as claimed in any one of claims 1 to 8 when executing the computer program.

Images