Microphone array adaptive pickup system, method and computer readable storage medium

Abstract

The embodiment of the present disclosure discloses a microphone array adaptive sound pickup system, method and computer readable storage medium, the microphone array adaptive sound pickup system comprises a signal acquisition module, a structure adjusting module, a position sensing module, a modeling module and a speech processing module, the signal acquisition module comprises at least two microphones; the structure adjusting module is used for adjusting the position of at least part of the microphone; the position sensing module is used for obtaining the position information of a plurality of microphones; the modeling module is used for generating control parameters based on a plurality of position information; the speech processing module is in communication connection with the signal acquisition module, and the speech processing module can perform speech enhancement processing on the sound information collected by a plurality of microphones based on the control parameters.

Description

Technical Field

[0001] This invention relates to the field of microphone pickup technology, and more specifically, to a microphone array adaptive pickup system, method, and computer-readable storage medium. Background Technology

[0002] With the rapid iteration and popularization of AI voice interaction technology, functions such as voice wake-up, long-distance speech recognition, and intelligent human-computer interaction have been widely applied in various electronic devices such as smart terminals, in-vehicle equipment, conference terminals, and service robots. Among them, microphone arrays, with their advantages of spatial filtering, directional sound pickup, and multi-channel speech enhancement, are gradually replacing traditional single-microphone pickup solutions and have become the core hardware foundation for far-field sound pickup, environmental noise suppression, and directional speech enhancement. They can effectively utilize spatial phase differences to distinguish target sound sources from interference noise and improve the quality of speech acquisition in complex sound field environments.

[0003] Currently, most microphone arrays in existing technologies adopt fixed geometric configurations, including fixed linear arrays, fixed ring arrays, and fixed planar arrays. The number of array elements, the spacing between array elements, the spatial distribution, and the array aperture are all fixed in the product design and assembly stage, and cannot be dynamically adjusted according to the actual working conditions during the use of the equipment.

[0004] However, the aforementioned fixed array structures have unavoidable performance defects and technical bottlenecks when facing complex and ever-changing real-world application scenarios, and cannot meet the high-precision sound pickup requirements of all scenarios and all distance ranges. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide a novel microphone array adaptive pickup system, method and computer-readable storage medium.

[0006] According to one aspect of the present invention, a microphone array adaptive pickup system is provided, comprising: A signal acquisition module, wherein the signal acquisition module includes at least two microphones; A structural adjustment module, the structural adjustment module being used to adjust the position of at least a portion of the microphone; A location sensing module, which is used to acquire the location information of the multiple microphones; A modeling module is used to generate control parameters based on multiple locations; The voice processing module is communicatively connected to the signal acquisition module, and the voice processing module is capable of performing voice enhancement processing on the sound information acquired by the multiple microphones based on the control parameters.

[0007] Optionally, the position sensing module can match the position information of multiple microphones based on the adjustment parameters of the structure adjustment module, wherein the control parameters are linear beamforming parameters.

[0008] Optionally, the microphone array has a near-field mode, a mid-field mode, and a far-field mode. When the microphone array is in the near-field mode, the speech processing module performs speech enhancement processing on the sound information collected by the multiple microphones based on the first linear beamforming parameters. When the microphone array is in mid-field mode, the voice processing module performs voice enhancement processing on the sound information collected by the multiple microphones based on the second linear beamforming parameters; When the microphone array is in far-field mode, the voice processing module performs voice enhancement processing on the sound information collected by the multiple microphones based on the third linear beamforming parameters.

[0009] Optionally, the system further includes a control module, which is communicatively connected to both the location sensing module and the modeling module. The control module is used to control the modeling module to generate control parameters based on multiple location information.

[0010] Optionally, the structure adjustment module includes a rotating arm rotatably connected to the housing, and the rotating arm is provided with a plurality of microphones.

[0011] Optionally, the structure adjustment module includes two oppositely arranged rotating arms, which are capable of rotating around the housing to switch between a folded state and an unfolded state.

[0012] Optionally, the plurality of microphones on each of the rotating arms are arranged at intervals along the length of their respective rotating arms.

[0013] Optionally, the spacing between two adjacent microphones on the same rotating arm is not equal; And / or, the positions of the multiple microphones on the two rotating arms correspond one-to-one.

[0014] Optionally, the position sensing module is an angle detection element, which is used to detect the rotation angle of the rotating arm relative to the housing, and match the position information of multiple microphones based on the rotation angle.

[0015] According to another aspect of the present invention, a microphone array adaptive sound pickup method is provided, comprising: Adjust the position of at least part of the microphone; Obtain the position information of multiple microphones; Control parameters are generated based on the multiple location information sources; Based on the control parameters, the sound information collected by the multiple microphones is subjected to speech enhancement processing.

[0016] Optionally, before adjusting the position of at least part of the microphone, the method further includes: A mapping relationship between the adjustment parameters and the position information of the multiple microphones is established in advance; And / or, the control parameters are linear beamforming parameters. The speech enhancement processing based on the control parameters of the sound information collected by the multiple microphones can be achieved through adaptive beamforming.

[0017] According to another aspect of the invention, a computer-readable storage medium is provided storing a computer program that is executed by a processor to implement any of the methods described above.

[0018] One technical advantage of the embodiments disclosed herein is that: The system in this embodiment can realize closed-loop adaptive control logic of structural adjustment, real-time position perception, dynamic parameter modeling, and adaptive voice enhancement. Regardless of the position or configuration changes of the microphone array due to working conditions, assembly, or adjustment actions, the system can match the optimal voice processing parameters in real time, thereby ensuring stable sound pickup and noise reduction effects in different scenarios and array states, which helps to improve the device's scene adaptability and environmental anti-interference ability.

[0019] Other features and advantages of the invention will become clear from the following detailed description of exemplary embodiments of the invention with reference to the accompanying drawings. Attached Figure Description

[0020] The accompanying drawings, which form part of this specification, illustrate embodiments of the invention and, together with the specification, serve to explain the principles of the invention.

[0021] Figure 1 This is a schematic diagram of a microphone array adaptive pickup system according to an embodiment of the present disclosure; Figure 2 This is a schematic diagram of a near-field mode of a microphone array according to an embodiment of the present disclosure; Figure 3 This is a schematic diagram of a mid-field mode of a microphone array according to an embodiment of the present disclosure; Figure 4 This is a schematic diagram of a far-field mode of a microphone array according to an embodiment of the present disclosure; Figure 5 This is a cross-sectional view of a microphone array according to an embodiment of the present disclosure.

[0022] Explanation of reference numerals in the attached figures: 1. Signal acquisition module; 11. Microphone; 2. Structural adjustment module; 21. Housing; 22. Rotating arm; 23. Hinge structure; 231. Rotating shaft; 232. Damping limit component; 233. Angle detection component; 234. Flexible line transmission component; 3. Position sensing module; 4. Modeling module; 5. Voice processing module. Detailed Implementation

[0023] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0024] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0025] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0026] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0027] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0028] This invention provides an adaptive microphone array pickup system, which can be applied to electronic devices with voice acquisition needs, such as smart terminals, in-vehicle voice devices, conference pickup terminals, and service robots. It can adaptively adjust the voice processing algorithm parameters according to the real-time position of the microphone array.

[0029] like Figure 1 As shown, the microphone array adaptive pickup system provided in this embodiment of the invention includes: Signal acquisition module 1, the signal acquisition module 1 including at least two microphones 11; Structural adjustment module 2, the structural adjustment module 2 being used to adjust the position of at least a portion of the microphone 11; Position sensing module 3, the position sensing module 3 is used to acquire the position information of the multiple microphones 11; Modeling module 4, which is used to generate control parameters based on multiple location information; The voice processing module 5 is communicatively connected to the signal acquisition module 1. The voice processing module 5 is capable of performing voice enhancement processing on the sound information acquired by the multiple microphones 11 based on the control parameters.

[0030] Specifically, the signal acquisition module 1 may include at least two microphones 11, for example, 2-6 microphones 11 may be used to form a linear array or a planar array. The specific configuration of the microphone array can be adapted according to the device size and the sound pickup scenario, and there is no limitation thereto.

[0031] Each microphone 11 is evenly or differently arranged in the sound pickup area inside the device housing. Each microphone 11 is matched with an independent sound pickup hole, sound guide channel and passive noise reduction structure, so that each microphone 11 can independently collect environmental acoustic signals, including multiple raw acoustic data such as target speech signals, environmental noise, airflow noise, and directional interference noise.

[0032] Each microphone 11 is equipped with an independent signal acquisition channel and synchronously completes analog-to-digital conversion of acoustic signals, converting analog acoustic signals into digital audio signals and transmitting them in real time to the voice processing module 5 connected to the communication link. Compared with the single-microphone 11 pickup structure, this system uses a multi-channel array for acquisition, which can provide multi-dimensional data support for subsequent spatial domain noise suppression and beamforming processing, and helps to achieve adaptive directional sound pickup.

[0033] The structural adjustment module 2 is used to adjust the position of some or all of the microphones 11 to achieve dynamic adjustment of the microphone array configuration, element spacing, and array angle, thereby adapting to complex sound pickup requirements such as different pickup distances, different sound source angles, and different noise distribution scenarios.

[0034] In one embodiment, the structural adjustment module 2 may include a micro drive component and a microphone array support bracket. Multiple microphones 11 are fixedly mounted on the microphone array support bracket. The micro drive component includes, but is not limited to, a stepper motor, a micro electric push rod, and a micro rotary servo motor. It has high-precision micro-adjustment capability and can drive the microphone array support bracket to perform linear displacement adjustment or angular deflection adjustment according to actual needs, thereby adjusting the position of the microphones 11 on it.

[0035] In one embodiment, the structure adjustment module 2 can independently adjust the position of a single or multiple microphones 11. For example, for a linear array microphone 11, the linear spacing of the end microphones 11 can be adjusted to achieve dynamic scaling of the array aperture; for a planar array microphone 11, the pitch angle and horizontal offset position of the local microphones 11 can be adjusted to change the spatial layout of the microphone array.

[0036] In one embodiment, the structure adjustment module 2 can drive the microphone array to rotate and swing as a whole to adjust the overall orientation of the microphone array, thereby adapting to the sound pickup scenario of a moving sound source.

[0037] Therefore, by setting up the structural adjustment module 2, the spatial position, spacing and deflection angle of at least some of the microphones 11 can be dynamically adjusted according to the actual sound pickup needs, and the overall array configuration and physical aperture can be flexibly changed to adapt to complex and differentiated scenarios such as near-field sound pickup, far-field sound pickup, mobile sound source sound pickup, and multiple noise interference. This can broaden the applicable boundaries of the microphone array at the hardware structure level and solve the technical problems of poor versatility and insufficient adaptability of traditional fixed array scenarios.

[0038] It is worth noting that the adjustment action of the structure adjustment module 2 can be automatically triggered according to the preset working conditions, or it can be adjusted in a closed loop according to the sound pickup quality parameters fed back by the back-end algorithm. Through the dynamic adaptation of the hardware structure, the basic sound pickup conditions of the microphone array can be optimized from the physical level, making up for the poor adaptability of the fixed array microphone 11.

[0039] The position perception module 3 is set up in correspondence with the structure adjustment module 2 and the signal acquisition module 1. It is used to acquire the precise position information of multiple microphones 11 in real time, including multi-dimensional position parameters such as the spatial coordinates, spatial spacing, deflection angle, and relative position deviation of each microphone 11, so as to provide a precise position reference for subsequent algorithm modeling.

[0040] Specifically, the position sensing module 3 can use a high-precision position detection device. For the linear displacement adjustment scenario of the microphone 11, a linear encoder can be configured to collect displacement data; for the angle deflection adjustment scenario of the microphone 11, a rotary encoder or a magnetoresistive angle detection chip can be configured to collect angle position data; for the overall spatial position change of the microphone array, an inertial detection unit can be used to assist in calibrating the position information.

[0041] During operation, the position sensing module 3 can collect the real-time position parameters of each microphone 11 after adjustment, and synchronously summarize the relative positions of all microphones 11 and the overall array configuration parameters to form a complete array position dataset. This position information is then transmitted to the modeling module 4 in real time to ensure that the modeling parameters are completely matched with the current physical structure of the microphone array, thus avoiding the model failure problem caused by structural displacement in traditional fixed parameter algorithms.

[0042] The modeling module 4 is communicatively connected to the position awareness module 3 and the speech processing module 5. Based on the real-time position information of multiple microphones 11 obtained by the position awareness module 3, the modeling module 4 can dynamically generate speech processing control parameters that are adapted to the current microphone array structure, so that the speech processing module 5 can achieve adaptive sound pickup.

[0043] Specifically, the modeling module 4 can pre-store an array acoustic modeling algorithm library, covering common modeling parameters for linear beamforming, including basic modeling factors such as the number of array elements, real-time element spacing, array configuration, sound source steering vector, phase compensation parameters, and beam constraint parameters. After receiving real-time array position information, the modeling module 4 first completes data calibration and analysis, eliminating abnormal data caused by detection errors and assembly deviations. Then, based on the actual spatial structure of the current array, it updates core control parameters such as the array steering vector, phase compensation of each element, and weighting coefficient constraints in real time.

[0044] Among them, the modeling module 4 can adapt to the parameter modeling of two types of linear beamforming algorithms: the first is for low power consumption scenarios, which dynamically generates the delay compensation parameters and fixed weighting parameters for Delay-and-Sum (delay summation beamforming); the second is for high noise reduction requirements scenarios, which dynamically generates adaptive control parameters such as the covariance matrix, distortion-free constraint coefficients, and diagonal loading regularization parameters for MVDR (minimum variance distortion-free response beamforming).

[0045] After the modeling module 4 updates the control parameters according to the real-time array structure, the latest set of speech processing control parameters is sent to the speech processing module 5 in real time, replacing the traditional fixed algorithm parameters. This ensures that the algorithm model always adapts to the physical state of the current microphone array, thereby solving problems such as beam shift, null failure, and decreased noise reduction accuracy caused by structural changes.

[0046] The voice processing module 5 is communicatively connected to the signal acquisition module 1 and interacts with the modeling module 4. Based on the real-time control parameters issued by the modeling module 4, the voice processing module 5 can perform adaptive voice enhancement processing on the multi-channel sound information collected by multiple microphones 11, and achieve spatial noise suppression and target voice fidelity optimization.

[0047] Specifically, the signal acquisition module 1 acquires real-time acoustic signals from multiple microphones 11 and transmits them to the speech processing module 5. The speech processing module 5 retrieves control parameters generated by the modeling module 4 that are adapted to the current array structure, and completes the weighted synthesis of multiple acoustic signals based on a dynamically updated beamforming algorithm. This directionally enhances the speech signal in the direction of the target sound source, while simultaneously forming a suppression null in the direction of interference noise and achieving spatial domain noise suppression.

[0048] Thus, the system in this embodiment can realize closed-loop adaptive control logic of structural adjustment, real-time position perception, dynamic parameter modeling, and adaptive voice enhancement. Regardless of any changes in the position or configuration of the microphone array due to operating conditions, assembly, or adjustment actions, the system can match the optimal voice processing parameters in real time, thereby ensuring stable sound pickup and noise reduction effects in different scenarios and array states, which helps to improve the device's scene adaptability and environmental anti-interference capabilities.

[0049] Furthermore, the modeling module 4 can dynamically iterate and generate beamforming control parameters adapted to the current array configuration based on the array position information collected in real time by the position awareness module 3. It is compatible with the parameter adaptive configuration of the Delay-and-Sum low-power beamforming algorithm and the MVDR high-precision adaptive beamforming algorithm, and updates core parameters such as steering vector, phase compensation amount, weighted constraint coefficient, and covariance matrix in real time. This enables real-time dynamic matching between the hardware structure and the algorithm model, thereby solving the technical problems of poor model adaptability and parameter solidification lag under variable array structures from the algorithm level.

[0050] Furthermore, the speech processing module 5, based on the real-time precise control parameters output by the modeling module 4, completes adaptive beamforming of multiple acoustic signals. This allows for precise beamforming of the effective speech signal in the direction of the target sound source, while simultaneously constructing suppression nulls in the direction of directional interference noise, achieving precise noise reduction in the spatial dimension. Compared to traditional single fixed noise reduction algorithms, the system of this invention can adaptively adjust the noise reduction strategy according to the array state, noise distribution, and sound source location, balancing low-power operation requirements with high-precision noise reduction needs. This effectively suppresses various complex noise interferences, maximizes the preservation of target speech details, and helps improve the clarity and fidelity of the output speech.

[0051] Optionally, the position sensing module 3 can match the position information of multiple microphones 11 based on the adjustment parameters of the structure adjustment module 2, wherein the control parameters are linear beamforming parameters.

[0052] Specifically, multiple positional information of the microphone array can be pre-calibrated, and a mapping relationship can be established between the adjustment parameters of the structure adjustment module 2 and the multiple positional information of the microphone array, such as a mapping function or a mapping table. The adjustment parameters can be rotation angles, displacements, etc., depending on the specific design of the structure adjustment module 2.

[0053] Thus, the position sensing module 3 can quickly analyze position information by matching structural adjustment parameters, thereby improving the system response speed and position matching accuracy. Unlike traditional position acquisition methods that rely on external detection, this embodiment combines the adjustment command parameters from the drive end of the structural adjustment module 2 with the detection data from the sensing end for joint matching and calibration. On the one hand, it can quickly predict the displacement and angle change trends of the microphone 11, shortening the delay in position information acquisition and analysis; on the other hand, it can also correct for detection anomalies and vibration interference errors, avoiding position jumps and data distortion problems caused by a single detection method. This results in smoother and more accurate output microphone 11 position information, providing a high-real-time, high-reliability raw data foundation for the modeling module 4, and helping to improve the system's adaptive adjustment response efficiency.

[0054] Among them, linear beamforming parameters can be used as core control parameters, so that all adaptive control parameters output by modeling module 4 are uniformly classified into the linear beamforming system. Specifically, it can cover the delay summation (DAS) fixed beamforming parameters and the MVDR minimum variance distortion-free response adaptive beamforming parameters.

[0055] Linear beamforming algorithms are characterized by their simple structure, strong robustness, good hardware portability, and absence of nonlinear distortion. Compared to nonlinear noise reduction algorithms, they are less prone to issues such as speech waveform distortion and frequency distortion. By uniformly defining the control parameters as linear beamforming parameters, the iterative complexity of the modeling algorithm and the chip's computing power consumption can be reduced, making it suitable for the operational needs of small embedded terminals and low-power voice devices. At the same time, it ensures the consistency and stability of the algorithm output under different array configurations and different adjustment conditions.

[0056] Optionally, the speech enhancement processing of the speech processing module 5 includes multi-level noise reduction optimization.

[0057] For example, spatial domain directional noise reduction can be achieved by relying on adaptive beamforming to filter out lateral and back-facing directional interference noise; then, frequency domain noise suppression algorithms can be combined to eliminate ambient diffuse white noise and airflow turbulence noise, ultimately completing the repair and enhancement of the speech signal, which can improve the signal-to-noise ratio and intelligibility of the output speech.

[0058] Optionally, the microphone array has a near-field mode, a mid-field mode, and a far-field mode. When the microphone array is in the near-field mode, the voice processing module 5 performs voice enhancement processing on the sound information collected by the multiple microphones 11 based on the first linear beamforming parameters. When the microphone array is in mid-field mode, the voice processing module 5 performs voice enhancement processing on the sound information collected by the multiple microphones 11 based on the second linear beamforming parameters; When the microphone array is in far-field mode, the voice processing module 5 performs voice enhancement processing on the sound information collected by the multiple microphones 11 based on the third linear beamforming parameters.

[0059] In this way, by adapting beam parameters according to the three-level pickup modes of near-field mode, mid-field mode and far-field mode, adaptive and accurate noise reduction can be achieved across the entire distance range, solving the problem that a single algorithm parameter cannot be adapted to multi-distance scenarios.

[0060] In near-field scenarios, sound sources are close-range, direct sound energy is strong, and interference is mostly from close-range human voice interactions. The voice processing module 5, adapted to the first linear beamforming parameters, can optimize near-field beam focusing characteristics, suppress near-field noise and airflow noise, and ensure the fullness of human voices. Figure 2 As shown.

[0061] The mid-field sound field distribution is balanced, with a high degree of mixing between noise and human voice. The speech processing module 5 adapts to the second linear beamforming parameters, balancing the main lobe width and side lobe suppression capability, thus achieving a good balance between pickup range and noise reduction accuracy. Figure 3 As shown.

[0062] In far-field scenarios, human voice attenuation is severe, environmental noise is high, and there is significant long-distance interference. The voice processing module 5, adapted to the third linear beamforming parameters, can optimize array gain and null suppression depth, improving the ability to extract weak human voice signals. Figure 4 As shown.

[0063] In this way, by matching the hierarchical mode with the linear beamforming parameters, the system can automatically switch to the optimal beam strategy according to the pickup distance, achieving high-quality voice enhancement across all scenarios and distances, which helps to improve the precision of device scenario adaptation.

[0064] Optionally, it also includes a control module, which is communicatively connected to the position sensing module 3 and the modeling module 4, respectively. The control module is used to control the modeling module 4 to generate control parameters based on multiple position information.

[0065] Specifically, the control module is configured to communicate with both the position sensing module 3 and the modeling module 4. This allows the control module to uniformly schedule and control the modeling module 4 to generate corresponding control parameters based on the real-time position information detected by the multiple microphones 11 from the position sensing module 3. This effectively filters invalid position data and avoids the problem of frequent parameter iteration and oscillation caused by minor structural vibrations, making the updating of modeling parameters more stable and orderly. At the same time, it realizes the modular decomposition of sensing, control, and modeling functions, with a clear logical hierarchy, which facilitates program iteration, function expansion, and fault diagnosis, and helps to improve the overall stability, controllability, and intelligence level of the system.

[0066] Furthermore, during operation, the control module can continuously monitor changes in the microphone array. When a significant change in the microphone array is detected, the modeling module 4 is triggered to update the control parameters, enabling a smooth transition of the control parameters.

[0067] Optionally, the structure adjustment module 2 includes a rotating arm 22 rotatably connected to the housing 21, and the rotating arm 22 is provided with a plurality of microphones 11.

[0068] like Figures 2 to 4 As shown, a rotating arm 22 rotatably connected to the housing 21 can be used to support multiple microphones 11, i.e., a microphone array, so as to realize the variable adjustment of the spatial configuration of the microphone array. It can also realize the overall synchronous adjustment of multiple microphones 11, with stronger structural linkage and higher mechanical consistency, which can avoid problems such as array skew, position offset, and cumulative assembly errors caused by independent adjustment of multiple microphones 11.

[0069] In addition, the rotating arm 22 and the housing 21 are in a rotating fit structure, which makes the structure compact and occupies little internal space. It is suitable for the assembly requirements of miniaturized smart terminals and can ensure the overall structural regularity and reliability of the equipment while achieving adjustable array posture.

[0070] Optionally, the structure adjustment module 2 includes two oppositely arranged rotating arms 22, which can rotate around the housing 21 to switch between a folded state and an unfolded state.

[0071] like Figures 2 to 4 As shown, by setting two oppositely arranged rotating arms 22, the two rotating arms 22 can rotate around the housing 21 to switch between a folded state and an unfolded state. In standby, storage, and transportation scenarios, the two rotating arms 22 are in the folded state, which can reduce the overall space occupied by the microphone array, reduce the size of the device, and improve product portability. In voice acquisition scenarios, the two rotating arms 22 switch to the unfolded state, which can expand the overall physical aperture of the microphone array and increase the array baseline span. The increased array aperture can compress the main lobe width of the beam, reduce the sidelobe level, and improve the far-field directional sound pickup capability and spatial noise suppression capability. Thus, the device can adaptively switch between "small storage mode" and "high noise reduction sound pickup mode" according to the working conditions, so as to balance the product's structural portability and superior acoustic performance.

[0072] Optionally, the plurality of microphones 11 on each of the rotating arms 22 are arranged at intervals along the length of their respective rotating arms 22.

[0073] Specifically, multiple microphones 11 on each rotating arm 22 are spaced apart along the length of the arm, so that the array sampling points are distributed in an orderly manner along the extension direction. This can avoid the sound field sampling imbalance caused by the microphones 11 clustering together, dense local sampling or gaps, and ensure that the acoustic signals collected by the multiple microphones 11 have regular and stable time delay difference and phase difference. This makes the subsequent steering vector solution, phase compensation and beam weighting calculation more accurate, which helps to improve the accuracy of adaptive modeling and the directional accuracy of beamforming.

[0074] Optionally, the spacing between two adjacent microphones 11 on the same rotating arm 22 is not equal.

[0075] Specifically, two adjacent microphones 11 on the same rotating arm 22 can be arranged at unequal intervals, which can break the periodicity of array spatial sampling, eliminate or suppress grating lobe pseudo-peaks, broaden the effective working frequency band of the array, improve the suppression effect of large-angle lateral interference and broadband noise, make the main lobe of the beam more focused and the spatial filtering accuracy higher, and help to adapt to the sound pickup needs of complex broadband sound field environments.

[0076] For example, the spacing between two adjacent microphones 11 on the outer side of the same rotating arm 22 can be set to be greater than the spacing between two adjacent microphones 11 on the inner side, which can reduce the spatial overlap caused by a fixed spacing.

[0077] Optionally, the positions of the multiple microphones 11 on the two rotating arms 22 correspond one-to-one.

[0078] Specifically, multiple microphones 11 on the dual rotating arms 22 can be configured in a one-to-one correspondence to form a symmetrical microphone array configuration. This ensures balanced sound field sampling and symmetrical phase distribution on both sides, avoiding problems such as beam shift, main lobe skew, and inconsistent noise reduction on both sides caused by asymmetrical arrays. The symmetrical array structure makes the linear beamforming parameters solved by the modeling module 4 more accurate and regular, and the weighting coefficients of MVDR and delay summation beamforming more stable, which helps to improve the consistency of speech enhancement effects and sound field balance.

[0079] Optionally, the rotating arm 22 has an arc-shaped structure.

[0080] Specifically, the rotating arm 22 can be configured with an arc-shaped structure. Compared to a straight arm structure, the arc-shaped arm can form a smooth acoustic flow profile, which can weaken the sound wave reflection, diffraction, and local resonance phenomena caused by structural corners, and reduce acoustic clutter interference. At the same time, the arc-shaped outer wall can smoothly guide the external airflow, avoid the airflow turbulence and howling noise generated by right-angle structures, reduce the original background noise at the front end, and help improve the noise reduction and fidelity of the overall adaptive pickup system.

[0081] Optionally, the position sensing module 3 is an angle detection element, which is used to detect the rotation angle of the rotating arm 22 relative to the housing 21, and match the position information of multiple microphones 11 based on the rotation angle.

[0082] Specifically, an angle detection element can be used to detect the rotation angle of the rotating arm 22 in real time to achieve rapid calculation of the microphone 11 position information. This method features a simple structure, rapid response, high matching accuracy, and adaptability to the adaptive modeling requirements of rotating arrays. The angle detection element includes, but is not limited to, angle sensors and position encoders.

[0083] In this way, by detecting the real-time rotation angle of the rotating arm 22 relative to the housing 21 through the angle detection element, the real-time spatial coordinates, array opening spacing, array configuration parameters and other position information of all microphones 11 can be matched. There is no need for multi-point displacement detection equipment, which can simplify the hardware detection architecture and reduce hardware costs and device power consumption.

[0084] The calculation method based on rotation angle matching position information also has strong real-time performance. It can update position data synchronously with the dynamic rotation process of the rotating arm 22, ensuring that the modeling module 4 can iterate the linear beamforming control parameters in real time throughout the folding, unfolding, and dynamic adjustment process. This achieves millisecond-level synchronization between mechanical structure attitude changes and algorithm model updates, which helps to improve the system's adaptive response speed and sound pickup stability under dynamic working conditions.

[0085] like Figures 2 to 4 As shown, the structural adjustment module 2 also includes two hinge structures 23, each rotating arm 22 being rotatably connected to one side of the housing 21 via a hinge structure 23, so that the microphone array can switch between a folded state and an unfolded state.

[0086] like Figure 5 As shown, the hinge structure 23 may include a pivot 231, a damping limiter 232, an angle detection element 233, and a flexible line transmission element 234. The pivot 231 is used to enable the rotating arm 22 to rotate around the housing 21, the damping limiter 232 is used to control the stability of the rotating arm 22 at different unfolding angles, the angle detection element 233 is used to detect the current rotation angle of the rotating arm 22, and the flexible line transmission element 234 is used to transmit the microphone 11 signal to the control module.

[0087] like Figures 2 to 4 As shown, three microphones 11 are spaced apart on the left rotating arm 22, and three microphones 11 are spaced apart on the right rotating arm 22. In the initial folded state, the left and right rotating arms 22 are folded together on both sides of the housing 21, forming a compact array structure among the microphones 11. In the unfolded state, the left and right rotating arms 22 rotate around the hinge structure 23, thereby increasing the spatial spacing between the multiple microphones 11 and forming a large-aperture array structure.

[0088] Specifically, when the rotation angle is At that time, the coordinates of microphone 11 on the left rotating arm 22 satisfy:

[0089] The coordinates of microphone 11 on the right rotating arm 22 satisfy:

[0090] in, This indicates the distance of each microphone 11 from the center of the hinge.

[0091] As the rotation angle changes, the spatial coordinates between each microphone 11 also change, thereby forming a mapping relationship between the rotation angle and the position information of the multiple microphones 11.

[0092] Among them, it can be set when At <20°, the microphone array is in near-field mode; When 20°≤ When the angle is less than 60°, the microphone array is in mid-range mode; when When the angle is ≥60°, the microphone array is in far-field mode.

[0093] when When the change is greater than 3°, the control module can trigger the modeling module 4 to update the control parameters so that the control parameters can be smoothly transitioned.

[0094] The present invention also provides a microphone array adaptive sound pickup method, comprising: Adjust the position of at least part of microphone 11; Obtain the position information of the multiple microphones 11; Control parameters are generated based on the multiple location information sources; Based on the control parameters, the sound information collected by the multiple microphones 11 is subjected to speech enhancement processing.

[0095] Thus, the system in this embodiment can realize closed-loop adaptive control logic of structural adjustment, real-time position perception, dynamic parameter modeling, and adaptive voice enhancement. Regardless of any changes in the position or configuration of the microphone array due to operating conditions, assembly, or adjustment actions, the system can match the optimal voice processing parameters in real time, thereby ensuring stable sound pickup and noise reduction effects in different scenarios and array states, which helps to improve the device's scene adaptability and environmental anti-interference capabilities.

[0096] Optionally, before adjusting the position of at least a portion of the microphone 11, the method further includes: A mapping relationship between adjustment parameters and the position information of the multiple microphones 11 is established in advance.

[0097] The mapping relationships include, but are not limited to, mapping functions and mapping tables. Based on the specific design of the structural adjustment module 2, the adjustment parameters can be rotation angles, displacements, etc. This allows for rapid analysis of position information by matching the structural adjustment parameters, thereby improving system response speed and position matching accuracy.

[0098] Optionally, the control parameters are linear beamforming parameters. The speech enhancement processing of the sound information collected by the multiple microphones 11 based on the control parameters can be achieved through adaptive beamforming.

[0099] Specifically, linear beamforming parameters can be used as core control parameters, so that all adaptive control parameters output by modeling module 4 are uniformly classified under the linear beamforming system. Specifically, it can cover the Delay Summation (DAS) fixed beamforming parameters and the MVDR minimum variance distortion-free response adaptive beamforming parameters.

[0100] Linear beamforming algorithms are characterized by their simple structure, strong robustness, good hardware portability, and absence of nonlinear distortion. Compared to nonlinear noise reduction algorithms, they are less prone to issues such as speech waveform distortion and frequency distortion. By uniformly defining the control parameters as linear beamforming parameters, the iterative complexity of the modeling algorithm and the chip's computing power consumption can be reduced, making it suitable for the operational needs of small embedded terminals and low-power voice devices. At the same time, it ensures the consistency and stability of the algorithm output under different array configurations and different adjustment conditions.

[0101] The present invention also provides a computer-readable storage medium storing a computer program that is executed by a processor to implement the method.

[0102] Optionally, the computer-readable storage medium may be a non-transitory storage medium, but is not limited thereto; it may also be a temporary storage medium.

[0103] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0104] This disclosure may be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having a computer-readable program loaded thereon for causing a processor to implement any of the methods in the foregoing embodiments of this disclosure.

[0105] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media may include, for example, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), compact disc-read-only memory (CD-ROM), digital versatile disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any combination thereof. The computer-readable storage medium used herein is not to be interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0106] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include one or more of copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to computer-readable storage media in the respective computing / processing device.

[0107] The computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object programs written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and conventional procedural programming languages ​​(such as the "C" language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network (e.g., a local area network or a wide area network), or it may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays, or programmable logic arrays, can execute computer-readable program instructions to implement various aspects of the embodiments of this disclosure by utilizing state information from the computer-readable program instructions.

[0108] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0109] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0110] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0111] The above embodiments mainly describe the differences between the various embodiments. As long as the different optimization features between the various embodiments are not contradictory, they can be combined to form a better embodiment. For the sake of brevity, they will not be elaborated here.

[0112] While specific embodiments of the invention have been described in detail by way of examples, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of the invention. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.

Claims

1. A microphone array adaptive pickup system, characterized in that, include: A signal acquisition module, wherein the signal acquisition module includes at least two microphones; A structural adjustment module, the structural adjustment module being used to adjust the position of at least a portion of the microphone; A location sensing module, which is used to acquire the location information of the multiple microphones; The modeling module is used to update the array steering vector, the phase compensation amount of each array element, and the weighting coefficient constraint conditions in real time based on multiple location information, and use them as control parameters. The voice processing module is communicatively connected to the signal acquisition module. Based on the control parameters, the voice processing module can perform weighted synthesis of multi-channel sound information acquired by multiple microphones through a beamforming algorithm, directionally enhance the voice signal in the direction of the target sound source, and form a suppression null in the direction of interference noise to achieve voice enhancement processing.

2. The microphone array adaptive pickup system according to claim 1, characterized in that, The position sensing module can match the position information of multiple microphones based on the adjustment parameters of the structure adjustment module, and the control parameters are linear beamforming parameters.

3. The microphone array adaptive pickup system according to claim 2, characterized in that, The microphone array has a near-field mode, a mid-field mode, and a far-field mode. When the microphone array is in the near-field mode, the speech processing module performs speech enhancement processing on the sound information collected by the multiple microphones based on the first linear beamforming parameters. When the microphone array is in mid-field mode, the voice processing module performs voice enhancement processing on the sound information collected by the multiple microphones based on the second linear beamforming parameters; When the microphone array is in far-field mode, the voice processing module performs voice enhancement processing on the sound information collected by the multiple microphones based on the third linear beamforming parameters.

4. The microphone array adaptive pickup system according to claim 1, characterized in that, It also includes a control module, which is communicatively connected to the location sensing module and the modeling module, respectively. The control module is used to control the modeling module to generate control parameters based on multiple location information.

5. The microphone array adaptive pickup system according to any one of claims 1 to 4, characterized in that, The structural adjustment module includes a rotating arm that is rotatably connected to the housing, and the rotating arm is equipped with a plurality of microphones.

6. The microphone array adaptive pickup system according to claim 5, characterized in that, The structural adjustment module includes two oppositely arranged rotating arms, which can rotate around the housing to switch between a folded state and an unfolded state.

7. The microphone array adaptive pickup system according to claim 6, characterized in that, The microphones on each of the rotating arms are arranged at intervals along the length of their respective rotating arms.

8. The microphone array adaptive pickup system according to claim 7, characterized in that, The spacing between two adjacent microphones on the same rotating arm is not equal; And / or, the positions of the multiple microphones on the two rotating arms correspond one-to-one.

9. The microphone array adaptive pickup system according to claim 5, characterized in that, The position sensing module is an angle detection element, which is used to detect the rotation angle of the rotating arm relative to the housing, and match the position information of multiple microphones based on the rotation angle.

10. A sound pickup method applied to the microphone array adaptive sound pickup system according to any one of claims 1 to 9, characterized in that, include: Adjust the position of at least part of the microphone; Obtain the position information of multiple microphones; Control parameters are generated based on the multiple location information sources; Based on the control parameters, the sound information collected by the multiple microphones is subjected to speech enhancement processing.

11. The sound pickup method according to claim 10, characterized in that, Prior to adjusting the position of at least part of the microphone, the method further includes: A mapping relationship between the adjustment parameters and the position information of the multiple microphones is established in advance; And / or, the control parameters are linear beamforming parameters. The speech enhancement processing based on the control parameters of the sound information collected by the multiple microphones can be achieved through adaptive beamforming.

12. A computer-readable storage medium, characterized in that, The device contains a computer program that is executed by a processor to implement the sound pickup method as described in claim 10 or 11.

Images