Intelligent audio scene adaptation method and device based on MEMS loudspeaker

Abstract

The invention discloses an intelligent audio scene adaptation method based on an MEMS loudspeaker, and relates to the technical field of audio scene adaptation. Audio is collected through a microphone array, and features such as frequency domain entropy are fused to construct vectors; a scene is identified by using BiLSTM and a self-attention model, and rapid adaptation is carried out in combination with transfer learning; optimizing loudspeaker parameters based on a coupling model and a genetic algorithm, and compensating temperature influence; the equal-sound curve is dynamically adjusted according to the scene, the audio is enhanced, and cross-scene adaptation, parameter pre-adjustment and multi-loudspeaker collaborative optimization are realized through the technologies of adversarial training, model prediction control and the like. According to the invention, through multi-modal feature fusion and deep learning, the scene recognition accuracy is improved, the loudspeaker parameters are optimized by using the coupling model, and distortion and power consumption are reduced; the audio enhancement parameters are dynamically adjusted, multi-scene adaptation is achieved, the tone quality and the user experience are improved, and the application bottleneck of a traditional MEMS loudspeaker is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of audio scene adaptation, and in particular to an intelligent audio scene adaptation method and device based on a MEMS speaker. Background Art

[0002] With the prevalence of smart devices, users' demands for a better audio experience continue to rise. MEMS speakers, due to their small size and low power consumption, are widely used in devices such as headphones and smart speakers. However, traditional MEMS speakers often suffer from poor audio quality in different usage scenarios. For example, in noisy outdoor environments, the sound output by the speaker is easily drowned out by ambient noise, resulting in reduced speech clarity. In quiet indoor settings, excessive volume or uneven frequency response can cause auditory discomfort. Existing technologies often use fixed-parameter audio processing solutions that cannot dynamically adjust to environmental changes, making it difficult to meet the needs of diverse scenarios.

[0003] Existing audio scene recognition methods primarily rely on single audio feature analysis, such as identifying scenes solely by calculating Mel-Frequency Cepstral Coefficients (MFCCs), which can easily lead to misjudgments in complex environments. Furthermore, traditional methods often use simple classification models with limited ability to extract and express scene features, making them unable to effectively distinguish between similar scenes (such as a cafe and a restaurant), resulting in inaccurate subsequent audio parameter adjustments. Furthermore, while some systems have scene recognition capabilities, they only adjust single parameters such as volume after recognition, failing to comprehensively consider the acoustic characteristics of the speakers and the human auditory perception, making it difficult to achieve high-quality audio optimization.

[0004] When it comes to driving and controlling MEMS speakers, existing technologies lack dynamic compensation mechanisms for the physical characteristics of the speakers. For example, after prolonged operation, the frequency response curve of MEMS speakers can shift due to rising temperatures, affecting sound quality. The output sound pressure level and distortion can also vary under different supply voltages. Furthermore, when multiple speakers operate in concert, the phase difference and volume balance between them are difficult to precisely control, making it impossible to create an ideal sound field. Existing solutions fail to fully account for these factors, resulting in poor performance of MEMS speakers in complex application scenarios, limiting their application scope and improving user experience. Summary of the Invention

[0005] The present invention proposes an intelligent audio scene adaptation method and device based on MEMS speakers to solve the problems mentioned in the above-mentioned prior art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: an intelligent audio scene adaptation method based on MEMS speakers, comprising:

[0007] Scene feature fusion extraction steps: collect audio signals through a four-element ring microphone array, use short-time Fourier transform to obtain spectral features, combine the improved dynamic time warping DTW algorithm to calculate the similarity of adjacent frames, and construct a frequency domain entropy Zero-crossing rate coefficient of variation 16-dimensional feature vector, where p(f i ) is the frequency f i The probability density of ZCR 、μ ZCR is the standard deviation and mean of the zero-crossing rate;

[0008] Scene dynamic recognition step: input the feature vector into the model of bidirectional long short-term memory network BiLSTM and self-attention mechanism, and calculate the Dynamically adjust the weights of different time series features and combine transfer learning to achieve cross-scenario adaptation;

[0009] MEMS speaker parameter collaborative optimization steps: establish a speaker mechanical vibration and acoustic response coupling model, optimize the driving voltage waveform through genetic algorithm, and the objective function F = min (THD × Power / SPL 2 ), reduce total harmonic distortion THD and power consumption while ensuring the sound pressure level SPL;

[0010] Audio enhancement processing steps: dynamically adjust equal loudness curve compensation parameters based on scene type, design adaptive loudness control algorithm, low-frequency compensation coefficient The amount of high-frequency boost is dynamically adjusted based on the ambient noise spectrum.

[0011] Furthermore, the scene dynamic recognition step also includes constructing a scene feature transfer network, using adversarial training to minimize the difference between domains, and implementing it through the gradient reversal layer GRL. where p s 、p t They are the source domain and target domain distribution respectively; a dynamic model update mechanism is designed to automatically trigger incremental learning when the confidence level of scene recognition for 5 consecutive frames is lower than 0.8, and the model parameters are fine-tuned using new data; a scene context prediction module is introduced to predict the probability distribution of future scenes through Transformer based on historical scene sequences, and optimize audio parameters in advance.

[0012] Furthermore, the MEMS speaker parameter collaborative optimization step also includes establishing a speaker temperature-frequency response compensation model, and obtaining the frequency shift compensation formula Δf=a0+a1T+a2T by fitting. 2 , where T is the speaker temperature, a0, a1, a2 are fitting coefficients; the model predictive control MPC algorithm is used to adjust the driving parameters in advance based on the scene prediction results of the next 300ms. The control time domain formula is: where yref is the reference output, y is the predicted output; design the speaker collaborative sound field optimization strategy, and adjust the phase difference formula Realize virtual sound source localization of speaker array.

[0013] Furthermore, the audio enhancement processing step also includes implementing real-time noise reduction based on deep learning, using the generative adversarial network GAN to build a noise reduction model, and the generator loss function Suppress ambient noise while maintaining speech clarity; design a dynamic spatial audio rendering algorithm to adjust reverberation time according to the scene space size. Where V is the volume of space, S is the surface area, is the average sound absorption coefficient; the frequency domain masking effect is used to optimize audio coding, and the quantization parameters are adjusted according to the human hearing threshold to reduce high-frequency redundant data transmission.

[0014] Furthermore, the scene feature transfer network adopts a meta-learning mechanism to adapt to new scenes through the MAML algorithm, and the model parameter update formula is: Implement adaptation in this scenario.

[0015] Furthermore, in the speaker collaborative sound field optimization strategy, a virtual sound source localization algorithm is used to compensate for the time delay. and amplitude adjustment, where d is the distance from the sound source and c is the speed of sound.

[0016] Furthermore, the dynamic spatial audio rendering algorithm introduces scene semantic information and automatically adjusts reverberation and delay parameters based on conference room and theater scene labels in combination with a pre-trained semantic-acoustic parameter mapping model.

[0017] A device applying the MEMS speaker-based intelligent audio scene adaptation method comprises:

[0018] Perception module: Integrates a four-element ring MEMS microphone array, infrared distance sensor, and temperature and humidity sensor to achieve synchronous collection of audio and environmental information;

[0019] Processing module: Equipped with a dual-core heterogeneous processor, including the ARM Cortex-A75 application core and accelerator, supporting real-time computing of BiLSTM and Transformer models;

[0020] Drive optimization module: built-in programmable power management chip, supports 0-3.3V dynamic voltage regulation, integrated MEMS speaker dedicated drive circuit, frequency response range of 20Hz-20kHz;

[0021] Audio processing module: uses 24-bit audio codec, supports 32kHz-192kHz sampling rate, and integrates dynamic equalization, noise reduction and spatial audio processing units.

[0022] Furthermore, the perception module also includes an ultrasonic array sensor for detecting space dimensions and obstacle distribution, a light sensor for judging scene brightness changes in combination with audio features, and an air pressure sensor for assisting in correcting acoustic parameters.

[0023] Furthermore, the drive optimization module also includes a temperature sensor to monitor the speaker temperature in real time and trigger thermal protection and frequency response compensation; an adaptive pre-distortion circuit to achieve nonlinear distortion compensation based on a lookup table; and a power amplifier that supports mono, stereo, and 2.1-channel output, with a power range of 0.5W-10W.

[0024] Compared with the existing technology, the beneficial effects of the present invention are:

[0025] In terms of scene recognition, the use of multimodal perception and improved deep learning models, combined with innovative features such as frequency domain entropy and zero-crossing rate variation coefficient, has greatly improved the accuracy and speed of scene recognition. Even in similar scenes such as cafes and restaurants, the recognition accuracy rate can reach more than 95%, effectively solving the problem of misjudgment caused by traditional methods.

[0026] To optimize MEMS speaker parameters, this patent establishes a coupled model of mechanical vibration and acoustic response. Combining genetic algorithms with model predictive control algorithms, this method not only compensates for frequency response shifts caused by temperature fluctuations in real time but also adjusts driver parameters in advance based on scenario predictions. Actual testing shows that after two hours of continuous operation, the speaker's frequency response fluctuations are kept to a minimal range, while simultaneously reducing power consumption by 30% and total harmonic distortion by 25%, achieving both improved sound quality and energy efficiency.

[0027] In terms of audio enhancement processing, the adaptive loudness control and dynamic spatial audio rendering algorithms designed in this patent can automatically adjust the equal loudness curves and reverberation time according to the scene. In noisy outdoor environments, the loudness of the voice band can be enhanced to improve voice clarity; in indoor viewing scenes, the reverberation parameters can be adjusted to create an immersive feeling. In addition, the multi-speaker collaborative sound field optimization strategy achieves high-precision virtual sound source positioning, with a positioning error of less than 5° in a 2×2 speaker array, bringing users a better audio experience. Overall, this patent effectively solves the performance bottleneck of traditional MEMS speakers in diverse scene applications, and improves the audio quality and user satisfaction of smart devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a schematic block diagram of the intelligent audio scene adaptation device based on MEMS speakers proposed in the present invention;

[0029] Figure 2This is a schematic block diagram of the intelligent audio scene adaptation method based on MEMS speakers proposed in the present invention;

[0030] Figure 3 This is a schematic diagram comparing the recognition accuracy of the MEMS speaker-based intelligent audio scene adaptation method proposed in the present invention in different scenarios;

[0031] Figure 4 This is a schematic block diagram of the comparison of frequency response fluctuations of the MEMS speaker in continuous operation according to the intelligent audio scene adaptation method proposed in the present invention. DETAILED DESCRIPTION

[0032] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0033] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0034] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the said features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined. In addition, the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be a connection between the two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. The present invention will be further described in detail below with reference to the accompanying drawings.

[0035] Reference Figure 1 and Figure 4:A specific implementation method and device for intelligent audio scene adaptation based on MEMS speakers

[0036] 1. Overall system architecture The system hardware consists of a multimodal perception module, an intelligent processing module, a drive optimization module and an audio processing module. Each module is connected to the I / O bus via a high-speed SPI bus. 2 C bus for data exchange. At the software level, a control program is built based on an embedded real-time operating system (RTOS) to achieve the coordinated operation of scene recognition, parameter optimization, and audio processing.

[0037] 2. Specific implementation steps of the method

[0038] (1) Scene feature fusion extraction steps: Use 4 MEMS microphones with a sensitivity of -42dBFS and a signal-to-noise ratio of 65dB to form a ring array and collect audio signals at a sampling rate of 48kHz. Perform short-time Fourier transform on the signal to obtain spectral features. Calculate frequency domain entropy The audio spectrum is divided into N = 128 frequency bands, and the probability density p(f i ) ; Calculate the coefficient of variation of the zero-crossing rate First, calculate the zero-crossing rate with 512 sampling points as one frame, and calculate the standard deviation σ by counting 100 frames of data. ZCR and mean μ ZCR Combined with Mel-frequency cepstral coefficients (MFCCs), a 16-dimensional feature vector is finally constructed for subsequent scene recognition.

[0039] (II) Scene dynamic recognition step: Input the feature vector into the model that integrates the bidirectional long short-term memory network (BiLSTM) and the self-attention mechanism. The model contains two BiLSTM layers, each with 128 neurons, and the self-attention mechanism uses 8-head attention calculation. Where d_k = 64. Through transfer learning, a model pre-trained on a large-scale audio scene dataset is fine-tuned for a new scene using a small amount of data. When the scene recognition confidence level for five consecutive frames falls below 0.8, incremental learning is triggered and the model parameters are updated. Simultaneously, based on the historical scene sequence, the Transformer is used to predict the probability distribution of the next three scenes, preparing for parameter optimization in advance.

[0040] (III) MEMS speaker parameter collaborative optimization steps: Establish a coupled model of mechanical vibration and acoustic response of the MEMS speaker, considering the interaction between the diaphragm vibration equation and the sound radiation equation. Use genetic algorithm to optimize the driving voltage waveform, set the population size to 50, the number of iterations to 100, and take the objective function F = min (THD × Power / SPL 2) is the optimization direction. A speaker temperature-frequency response compensation model is established. The frequency response data at different temperatures are measured experimentally, and the frequency shift compensation formula Δf=a0+a1T+a2T is obtained through polynomial fitting. 2 , where a0 = -0.1, a1 = 0.05, a2 = 0.001. The model predictive control (MPC) algorithm is used to predict the scene changes within the next 300ms. The control time domain formula is: The prediction time domain Np = 10, and the driving parameters are adjusted in advance. For multi-speaker collaboration, according to the formula By adjusting the time delay Δt of each speaker driving signal, phase difference control is achieved to complete virtual sound source positioning.

[0041] (IV) Audio enhancement processing steps: Based on the scene type, use the adaptive loudness control algorithm to adjust the equal loudness curve compensation parameters. Low frequency compensation coefficient Where α=0.1, T LF =60dB. A noise reduction model is constructed using a generative adversarial network (GAN). Both the generator and the discriminator are 3-layer fully connected neural networks. 200 epochs are trained to suppress ambient noise while maintaining speech clarity. According to the scene space size, the formula Calculate the reverberation time, where the space volume V and surface area S are measured by the ultrasonic array sensor, and the average sound absorption coefficient Dynamic spatial audio rendering is achieved based on scene type presets. At the same time, based on the human hearing threshold, the quantization parameters of the audio encoding are adjusted in the frequency domain to reduce high-frequency redundant data.

[0042] 3. Specific implementation methods of the device

[0043] (1) Multimodal perception module: In addition to four MEMS microphones, it integrates an infrared distance sensor with a detection range of 0.1-5m, a light sensor with a measurement range of 0.1-100klx, a pressure sensor with a measurement range of 300-1100hPa and an accuracy of ±0.1hPa, a temperature and humidity sensor with a detection range of 0-100%RH and an accuracy of ±3%RH, and an 8-channel ultrasonic array sensor (measurement accuracy of ±1cm), enabling comprehensive collection of audio and environmental information.

[0044] (2) Intelligent processing module: Equipped with a dual-core heterogeneous processor, the ARM Cortex-A75 core has a main frequency of 1.8GHz and is responsible for system management and user interaction; the dedicated AI accelerator has a computing power of 5TOPS and a built-in neural network computing unit, which supports efficient computing of models such as BiLSTM and Transformer to meet real-time processing needs.

[0045] (3) Driver Optimization Module: Utilizes a programmable power management chip, supporting dynamic voltage regulation from 0-3.3V in 0.01V steps. It integrates a dedicated MEMS speaker driver circuit and a built-in temperature sensor (accuracy ±0.5°C) for real-time speaker temperature monitoring and triggering frequency response compensation. An adaptive pre-distortion circuit uses a lookup table to compensate for nonlinear distortion, reducing total harmonic distortion (THD) to below 0.5%. A multi-channel power amplifier supports 0.5W-10W power output, adapting to diverse scenarios.

[0046] (IV) Audio Processing Module: Utilizes a 24-bit audio codec, supporting sampling rates from 32kHz to 192kHz. An integrated dynamic equalizer allows for ±12dB adjustment across 10 frequency bands. A noise reduction unit uses a deep learning model to achieve real-time noise reduction. A spatial audio processing unit dynamically adjusts parameters such as reverberation and delay based on the scene, achieving stereo and 2.1-channel output.

[0047] 4. Data Representation and Analysis

[0048]

[0049] Traditional audio processing solutions have many shortcomings in scene recognition, speaker performance optimization and audio rendering. This application uses multimodal feature fusion and advanced deep learning models to significantly improve the scene recognition accuracy from 82% to 96%, effectively reducing misjudgments. In terms of speaker performance optimization, with the help of coupling models and multiple optimization algorithms, the frequency response fluctuation is reduced by 83.3%, THD is reduced by 66.7%, and power consumption is reduced by 30%, significantly improving sound quality and energy efficiency. The virtual sound source positioning error is reduced from 12° to 4°, combined with dynamic spatial audio rendering, to bring users a more accurate and immersive audio experience. These data show that the technology of this application effectively solves the pain points of traditional solutions, has significant advantages in practical applications, and can be widely used in smart speakers, headphones and other devices to enhance product competitiveness and user satisfaction.

[0050] The above are only preferred specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solutions and inventive concepts of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. An intelligent audio scene adaptation method based on MEMS speakers, characterized in that: include: Scene feature fusion extraction steps: collect audio signals through a four-element ring microphone array, use short-time Fourier transform to obtain spectral features, combine the improved dynamic time warping DTW algorithm to calculate the similarity of adjacent frames, and construct a frequency domain entropy Zero-crossing rate coefficient of variation 16-dimensional feature vector, where p(f i ) is the frequency f i The probability density of ZCR 、μ ZCR is the standard deviation and mean of the zero-crossing rate; Scene dynamic recognition step: input the feature vector into the model of bidirectional long short-term memory network BiLSTM and self-attention mechanism, and calculate the Dynamically adjust the weights of different time series features and combine transfer learning to achieve cross-scenario adaptation; MEMS speaker parameter collaborative optimization steps: establish a speaker mechanical vibration and acoustic response coupling model, optimize the driving voltage waveform through genetic algorithm, and the objective function F = min (THD × Power / SPL 2 ), reduce total harmonic distortion THD and power consumption while ensuring the sound pressure level SPL; Audio enhancement processing steps: dynamically adjust equal loudness curve compensation parameters based on scene type, design adaptive loudness control algorithm, low-frequency compensation coefficient The amount of high-frequency boost is dynamically adjusted based on the ambient noise spectrum.

2. The intelligent audio scene adaptation method based on MEMS speaker according to claim 1 is characterized in that: The scene dynamic recognition step also includes constructing a scene feature migration network, using adversarial training to minimize inter-domain differences, and implementing it through the gradient reversal layer GRL. where p s 、p t They are the source domain and target domain distribution respectively; a dynamic model update mechanism is designed to automatically trigger incremental learning when the confidence level of scene recognition for 5 consecutive frames is lower than 0.8, and the model parameters are fine-tuned using new data; a scene context prediction module is introduced to predict the probability distribution of future scenes through Transformer based on historical scene sequences, and optimize audio parameters in advance.

3. The intelligent audio scene adaptation method based on MEMS speaker according to claim 1 is characterized in that: The MEMS speaker parameter collaborative optimization step also includes establishing a speaker temperature-frequency response compensation model, and obtaining the frequency shift compensation formula Δf=a0+a1T+a2T by fitting. 2 , where T is the speaker temperature, a0, a1, a2 are fitting coefficients; the model predictive control MPC algorithm is used to adjust the driving parameters in advance based on the scene prediction results of the next 300ms. The control time domain formula is: where y ref is the reference output, y is the predicted output; design the speaker collaborative sound field optimization strategy, and adjust the phase difference formula Realize virtual sound source localization of speaker array.

4. The intelligent audio scene adaptation method based on MEMS speaker according to claim 1, characterized in that: The audio enhancement processing step also includes implementing real-time noise reduction based on deep learning, using the generative adversarial network GAN to build a noise reduction model, and the generator loss function Suppress ambient noise while maintaining speech clarity; design a dynamic spatial audio rendering algorithm to adjust reverberation time according to the scene space size. Where V is the volume of space, S is the surface area, is the average sound absorption coefficient; the frequency domain masking effect is used to optimize audio coding, and the quantization parameters are adjusted according to the human hearing threshold to reduce high-frequency redundant data transmission.

5. The intelligent audio scene adaptation method based on MEMS speaker according to claim 2, characterized in that: The scene feature transfer network adopts a meta-learning mechanism and adapts to new scenes through the MAML algorithm. The model parameter update formula is: Implement adaptation in this scenario.

6. The intelligent audio scene adaptation method based on MEMS speaker according to claim 3 is characterized in that: In the speaker collaborative sound field optimization strategy, a virtual sound source localization algorithm is used to compensate for the delay. and amplitude adjustment, where d is the distance from the sound source and c is the speed of sound.

7. The intelligent audio scene adaptation method based on MEMS speaker according to claim 4, characterized in that: The dynamic spatial audio rendering algorithm introduces scene semantic information and automatically adjusts reverberation and delay parameters based on conference room and theater scene labels in combination with a pre-trained semantic-acoustic parameter mapping model.

8. A device using the intelligent audio scene adaptation method based on a MEMS speaker according to any one of claims 1 to 7, characterized in that: include: Perception module: Integrates a four-element ring MEMS microphone array, infrared distance sensor, and temperature and humidity sensor to achieve synchronous collection of audio and environmental information; Processing module: Equipped with a dual-core heterogeneous processor, including the ARM Cortex-A75 application core and accelerator, supporting real-time computing of BiLSTM and Transformer models; Drive optimization module: built-in programmable power management chip, supports 0-3.3V dynamic voltage regulation, integrated MEMS speaker dedicated drive circuit, frequency response range of 20Hz-20kHz; Audio processing module: uses 24-bit audio codec, supports 32kHz-192kHz sampling rate, and integrates dynamic equalization, noise reduction and spatial audio processing units.

9. The intelligent audio scene adaptation device based on MEMS speaker according to claim 8, characterized in that: The perception module also includes an ultrasonic array sensor for detecting space dimensions and obstacle distribution, a light sensor for judging scene brightness changes in combination with audio features, and an air pressure sensor for assisting in correcting acoustic parameters.

10. The intelligent audio scene adaptation device based on MEMS speaker according to claim 8, characterized in that: The drive optimization module also includes a temperature sensor to monitor the speaker temperature in real time and trigger thermal protection and frequency response compensation; The adaptive pre-distortion circuit realizes nonlinear distortion compensation based on a lookup table. The power amplifier supports mono, stereo and 2.1-channel output, and the power range is adjustable from 0.5W to 10W.

Images