Audio processing system
Through the audio processing system of 2n microphones with bilinear arrangement, the spatial layout limitations and poor performance of traditional ring and linear microphone arrays in sound source positioning are solved, and more accurate and efficient sound source positioning is achieved.
Patent Information
- Application Number
- CN202323581460.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Utility models(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-26
- Publication Date
- 2025-02-28
- Estimated Expiration
- 2033-12-26
AI Technical Summary
Traditional multi-microphone annular arrays and linear arrays have spatial layout limitations and poor performance in sound source positioning.
The audio processing system of 2n microphones is adopted in a bilinear arrangement, and is connected to the microphone array module through a central processor and a digital audio processor to achieve sound source positioning, enhancement, echo cancellation and noise suppression.
With the same number of microphones, the system takes up less space, has low layout requirements, is easier to implement, and can more accurately locate the sound source.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of audio processing, and in particular to an audio processing system. Background Art
[0002] In traditional consumer products, in order to simultaneously achieve functions such as sound source localization, echo cancellation, and noise reduction, most of them use multi-microphone ring arrays or 4 / 8 microphone linear arrays.
[0003] For multi-microphone ring arrays, spatial phase differences can be used to better localize the sound source, but the device layout requirements make this impossible for most products. As for 4 / 8 microphone linear arrays, due to the linear arrangement of the devices, it is impossible to achieve better sound source localization. Utility Model Content
[0004] The main purpose of the present application is to propose an audio processing system, which aims to solve the problem of spatial arrangement limitation of the annular microphone array and the problem of poor sound source positioning of the linear microphone.
[0005] To achieve the above object, the present application provides an audio processing system, the audio processing system comprising: a central processing unit, a digital audio processor and a microphone array module;
[0006] The microphone array module includes 2n microphones arranged bilinearly;
[0007] The central processing unit is connected to the digital audio processor, and the digital audio processor is connected to the 2n microphones respectively.
[0008] Optionally, the digital audio processor includes: a microphone sound source localization module, a microphone sound source enhancement module, a microphone echo cancellation module and a microphone noise suppression module;
[0009] The microphone sound source localization module, the microphone sound source enhancement module, the microphone echo cancellation module and the microphone noise suppression module are respectively connected to the central processor and the 2n microphones.
[0010] Optionally, the audio processing system further includes: a power amplifier and a speaker;
[0011] The power amplifier is connected to the speaker and the digital audio processor respectively.
[0012] Optionally, the digital audio processor includes: a first analog-to-digital conversion interface and a first digital-to-analog conversion interface;
[0013] The first digital-to-analog conversion interface is connected to the first analog-to-digital conversion interface and the power amplifier respectively.
[0014] Optionally, the digital audio processor further includes: an audio interface;
[0015] The audio interface is connected to 2n microphones.
[0016] Optionally, the audio processing system further includes: headphones, the headphones including a headphone microphone and a headphone speaker; the digital audio processor further includes: a second analog-to-digital conversion interface and a second digital-to-analog conversion interface;
[0017] The second analog-to-digital conversion interface is connected to the earphone microphone, and the second digital-to-analog conversion interface is connected to the earphone speaker.
[0018] Optionally, the audio processing system further includes: an earphone and a signal processing module, the earphone including an earphone microphone and an earphone speaker;
[0019] The signal processing module is connected to the earphone microphone, the earphone speaker and the central processing unit respectively.
[0020] Optionally, the microphone array modules are arranged symmetrically based on the center line of the device.
[0021] Optionally, the lateral spacing range of the microphones is 30 mm to 40 mm, and the longitudinal spacing range of the microphones is 20 mm to 25 mm.
[0022] Optionally, the number of microphones arranged horizontally is not less than the number of microphones arranged vertically.
[0023] The present application provides an audio processing system, which includes: a central processing unit, a digital audio processor and a microphone array module; the microphone array module includes 2n microphones arranged bilinearly; the central processing unit is connected to the digital audio processor, and the digital audio processor is respectively connected to the 2n microphones. The present application uses 2n microphones arranged bilinearly to occupy less space than microphones arranged in a ring, and there is no need to reserve a ring space. Only the linear space of the first and second arrangements needs to be reserved. The arrangement requirements are lower than those of microphones arranged in a ring, and it is easier to implement, which solves the problem of spatial arrangement limitations of the annular microphone array; at the same time, based on the positional relationship between the first and second arrangements, audio information can be collected in a three-dimensional space to obtain audio data with a spatial phase difference, and the sound source can be located more accurately. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 This is a schematic diagram of the framework structure of an embodiment of the audio processing system of the present application;
[0025] Figure 2A schematic diagram of microphone arrangement and application of an embodiment of the audio processing system of the present application;
[0026] Figure 3 A schematic diagram of the framework of a digital audio processor of an embodiment of the audio processing system of the present application;
[0027] Figure 4 A schematic diagram of the principle of a microphone sound source localization module of an embodiment of the audio processing system of the present application;
[0028] Figure 5 This is a schematic diagram of an application of a microphone noise suppression module according to an embodiment of the audio processing system of the present application;
[0029] Figure 6 A schematic diagram of the framework structure of another embodiment of the audio processing system of the present application;
[0030] Figure 7 This is a connection diagram of a digital audio processor of an audio processing system according to an embodiment of the present application.
[0031] Description of Figure Numbers:
[0032] Label name Label name 10 Central Processing Unit 20 Digital Audio Processor 30 Microphone array module 21 Microphone sound source localization module 22 Microphone sound source enhancement module 23 Microphone echo cancellation module 24 Microphone Noise Suppression Module 20a First analog-to-digital conversion interface 20b First digital-to-analog conversion interface 20c Second digital-to-analog conversion interface 20d Second analog-to-digital conversion interface 20e Audio Interface 40 Power Amplifier 50 speaker 60 earphone 61 Headset Microphone 62 Headphone Speaker 63 switch 70 Signal processing module
[0033] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0034] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0035] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0036] It should be noted that all directional indications in the embodiments of the present application (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.
[0037] In this application, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.
[0038] In addition, in this application, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.
[0039] The present application provides an audio processing system.
[0040] In one embodiment of the present application, refer to Figure 1 , Figure 1 This is a schematic diagram of the framework structure of an embodiment of the audio processing system of the present application, wherein the audio processing system comprises: a central processing unit 10, a digital audio processor 20 and a microphone array module 30;
[0041] The microphone array module 30 includes 2n microphones M arranged bilinearly;
[0042] The central processor 10 is connected to the digital audio processor 20 , and the digital audio processor 20 is connected to the 2n microphones M respectively.
[0043] It is understandable that a microphone array is formed by a certain number of acoustic sensors (usually microphones) arranged in a specific pattern to sample and process the spatial characteristics of the sound field. The principle of microphone array technology is to extract the characteristic information of the signal by analyzing the time domain and frequency domain of the microphone collected signal, and to weight and fuse the characteristic information to obtain a more accurate and reliable audio signal. In practical applications, microphone array technology can optimize the processing of audio signals through algorithms such as adaptive filtering, beamforming, and noise suppression.
[0044] It should be noted that the central processing unit 10 is a CPU (Central Processing Unit), and the digital audio processor 20 is a DSP (Digital Signal Processing, a power amplifier that optimizes and manages audio parameters using digital signal processing algorithms).
[0045] The central processing unit 10 and the digital audio processor 20 are connected via an I2S (Inter-IC Sound) bus or a UART (Universal Asynchronous Receiver / Transmitter) communication. The digital audio processor 20 sends the audio data to the central processing unit 10 for processing, and the central processing unit 10 sends the audio data to the digital audio processor 20.
[0046] In this embodiment, the digital audio processor 20 is specifically a SOC (System on Chip) circuit module.
[0047] In this embodiment, the microphone array module 30 is a bilinear arrangement, that is, it includes a first arrangement and a second arrangement, specifically, n microphones M are linearly arranged as a first arrangement, and the remaining n microphones M are linearly arranged as a second arrangement, wherein n is a positive integer, and the digital audio processor 20 receives the audio signals of 2n microphones M, and performs sound source localization and noise reduction on the audio signals. In the case of the same number of microphones, the multiple microphones arranged in a bilinear manner occupy less space than the microphones arranged in a ring, and there is no need to reserve a ring space, only the linear space of the first arrangement and the second arrangement needs to be reserved, which has lower arrangement requirements than the microphones arranged in a ring and is easier to implement; at the same time, based on the positional relationship between the first arrangement and the second arrangement, audio information can be collected in a three-dimensional space to obtain audio data with a spatial phase difference, and the sound source can be located more accurately. Specifically, based on the phase difference, incident angle and delay difference between the first arrangement and the second arrangement, the accuracy of sound source localization is improved.
[0048] For example, reference Figure 2 , Figure 2 This is a schematic diagram of microphone arrangement and application of an embodiment of the audio processing system of the present application. Figure 2 FIG. 1 is a schematic diagram of a tablet computer including an audio processing system. It should be noted that the device including the audio processing system may also be a mobile phone, a stereo, etc. Figure 2As an application diagram of only one device, the microphone array module 30 includes 6 microphones M, the first row is the upper row of 3 microphones M, namely microphone M1, microphone M2 and microphone M3, the second row is the lower row of 3 microphones M, namely microphone M4, microphone M5 and microphone M6, microphone M2 and microphone M5 are set on the midline of the tablet computer, and two microphones are set on the left and right of the midline respectively, so that the microphone array module 30 is arranged symmetrically based on the midline, the left and right microphones are 30mm to 40mm away from the midline respectively, and the microphone spacing between the first row and the second row can be set to 20mm to 25mm. In addition, the microphone array module 30 can also be 8 microphones, with 4 microphones in the upper and lower rows respectively.
[0049] It can be understood that there is no limit to the number of microphones in the microphone array module 30, which is determined based on the specific application scenario, but the arrangement pattern and spacing between the microphones must meet the requirements of this application, that is, the 2n microphones must be arranged bilinearly and symmetrically based on the midline of the device. The lateral spacing range of the microphones is 30 mm to 40 mm, and the longitudinal spacing range of the microphones is 20 mm to 25 mm.
[0050] Among them, the number of microphones arranged vertically is 2 to meet the bilinear arrangement requirements, and the number of microphones arranged horizontally is not less than the number of microphones arranged vertically, that is, the number of microphones arranged horizontally is at least 2, so that the audio data collected by the microphone array has a spatial phase difference.
[0051] Optionally, in some feasible embodiments, reference Figure 3 and Figure 4 , Figure 3 This is a schematic diagram of the framework of a digital audio processor of an embodiment of the audio processing system of the present application. Figure 4 It is a schematic diagram of the principle of a microphone sound source localization module of an embodiment of the audio processing system of the present application. The digital audio processor 20 includes: a microphone sound source localization module 21;
[0052] The microphone sound source localization module 21 is connected to the central processor 10 and the 2n microphones M respectively.
[0053] In this embodiment, the digital audio processor 20 receives audio signals sent by 2n microphones M, and the microphone sound source localization module 21 performs sound source localization based on the phase difference of each of the 2n microphones M and the main peak amplitude in the audio signal. Figure 4 , Figure 4Only one row of microphones is shown. When a sound source signal approaches the microphone, the incident angle θ of each microphone is different, and the time delay of the sound source signal reaching the microphone is different. The position of the sound source signal is obtained through weighted calculation and processing, and the position of the sound source signal is sent to the central processor 10. Specifically, d is the lateral spacing of the microphones, k is the signal point, and S(k) is the representative function of the signal point. When the sound source signal S(k) propagates, a pickup beam from the sound source signal S(k) to each microphone is formed. The distance from the sound source signal to the microphone M1 is r1, the distance from the sound source signal to the microphone M2 is r2, and the distance from the sound source signal to the microphone M3 is r3. The incident angle of microphone M1 is θ1, the incident angle of microphone M2 is θ2, and the incident angle of microphone M3 is θ3. It should be noted that the function expressions X(k) and Y(k) of each microphone M are different. Therefore, the position of the sound source signal S(k) is calculated based on the above parameters. Based on the positional relationship between the first arrangement and the second arrangement, audio information can be collected in a three-dimensional space to obtain audio data with a spatial phase difference, and the sound source can be located more accurately.
[0054] It should be noted that the sound source localization technology of the microphone sound source localization module 21 belongs to the existing technology. The present application uses 2n microphones arranged bilinearly to enrich the phase difference of the microphones received by the microphone sound source localization module 21 and the main peak amplitude in the audio signal, so as to improve the accuracy of sound source localization.
[0055] Optionally, in some feasible embodiments, reference Figure 3 , the digital audio processor 20 includes: a microphone sound source enhancement module 22;
[0056] The microphone sound source enhancement module 22 is connected to the central processor 10 and the 2n microphones M respectively.
[0057] In this embodiment, the original sound source data of 2n microphones M are sent to the digital audio processor 20 at the same time, and the microphone sound source enhancement module 22 of the digital audio processor 20 amplifies and filters the original sound source data, and outputs the enhanced signal to the central processor 10.
[0058] Optionally, in some feasible embodiments, reference Figure 3 , the digital audio processor 20 includes: a microphone echo cancellation module 23;
[0059] The microphone echo cancellation module 23 is connected to the central processor 10 and the 2n microphones M respectively.
[0060] Generally speaking, common consumer products are usually equipped with both speakers and microphones, and the speakers are usually close to the microphones. During the communication process, the speakers and microphones may work at the same time. In order to meet the long-distance communication scenarios of the products, the volume of the speakers is usually as loud as possible. Therefore, during the communication process, the sound of the speakers will inevitably be transmitted to the microphone, allowing the remote user to hear the sound of the speakers, thus causing an echo.
[0061] In this embodiment, the microphone echo cancellation module 23 of the digital audio processor 20 receives the sound source data sent by each of the 2n microphones M. The microphone echo cancellation module 23 removes the echo signal from the sound source data through a preset subtraction model, that is, removes the signal of the speaker and only retains the sound source signal collected for the first time output by the microphone M to achieve the effect of echo cancellation.
[0062] Optionally, in some feasible embodiments, reference Figure 3 and Figure 5 , Figure 5 This is a schematic diagram of the application of a microphone noise suppression module of an embodiment of the audio processing system of the present application. The digital audio processor 20 includes: a microphone noise suppression module 24;
[0063] The microphone noise suppression module 24 is connected to the central processor 10 and the 2n microphones M respectively.
[0064] In this embodiment, the microphone noise suppression module 24 builds a model in the early stage to learn the spectrum of human voice, and removes the signal other than the human voice signal from the original sound source data by comparing the human voice spectrum in the audio signal to achieve noise reduction. Figure 5 In the figure, the downstream B is the original sound source data, i.e., the original microphone signal, and the upstream A is the sound source data after being processed by the microphone noise suppression module 24. By comparing A and B, the human voice signal is amplified and the noise signal is reduced after being processed by the microphone noise suppression module 24 to achieve the effect of noise suppression.
[0065] Optionally, in some feasible embodiments, reference Figure 6 , Figure 6 This is a schematic diagram of the framework structure of another embodiment of the audio processing system of the present application, wherein the audio processing system further comprises: a power amplifier 40 and a speaker 50;
[0066] The power amplifier 40 is connected to the speaker 50 and the digital audio processor 20 respectively. In this embodiment, the digital audio processor 20 processes the original sound source data, and the processed sound source data includes sound source localization, sound source enhancement, echo cancellation and microphone noise suppression. The digital audio processor 20 sends the processed sound source data to the power amplifier 40 for amplification, and the power amplifier 40 sends the amplified sound source data to the speaker 50 for playback. At the same time, the sound played by the speaker 50 is input into the digital audio processor 20 to assist the digital audio processor 20 in performing echo cancellation processing.
[0067] It should be noted that the loudspeaker 50 may be multiple, such as Figure 6 As shown, the power amplifier 40 is connected to two speakers 50 respectively; the power amplifier 40 may be multiple, such as Figure 7 As shown, two power amplifiers 40 are respectively connected to two speakers 50. The power amplifiers are specifically analog power amplifiers.
[0068] Optionally, in some feasible embodiments, reference Figure 7 , Figure 7 This is a connection diagram of a digital audio processor of an embodiment of the audio processing system of the present application, wherein the digital audio processor 20 comprises: a first analog-to-digital conversion interface 20a and a first digital-to-analog conversion interface 20b;
[0069] The first digital-to-analog conversion interface 20 b is connected to the first analog-to-digital conversion interface 20 a and the power amplifier 40 respectively.
[0070] In this embodiment, the digital audio processor 20 receives the sound source signal sent by the microphone array module 30, performs sound source positioning, sound source enhancement, noise suppression and other processing, and then sends it to the power amplifier 40 through the first digital-to-analog conversion interface 20b. The power amplifier 40 amplifies the processed sound source signal and plays it through the speaker 50. The first digital-to-analog conversion interface 20b sends the processed sound source signal as a collection signal to the first analog-to-digital conversion interface 20a. The first analog-to-digital conversion interface 20a sends the collection signal to the microphone echo cancellation module 23. After the microphone echo cancellation module 23 receives the sound source signal of the next frequency, it subtracts the collection signal from the sound source signal of the next frequency, and repeats this process. Thus, the microphone echo cancellation module 23 removes the collection signal from the sound source data through a preset subtraction model, that is, removes the signal of the speaker, and only outputs the sound source signal collected by the microphone M for the first time. The sound source signal collected for the first time is, for example, the near-end human voice.
[0071] Optionally, in some feasible embodiments, reference Figure 7 , the digital audio processor 20 further includes: an audio interface 20e;
[0072] The audio interface 20e is connected to the 2n microphones M.
[0073] In this embodiment, the audio interface 20e is a PDM (Pulse Density Modulation, a modulation method for representing analog signals with digital signals) interface or an I2S (Inter-IC Sound bus, an integrated circuit built-in audio bus) interface. Figure 7 The 2n microphones M of the microphone array module 30 send the collected sound source information to the digital audio processor 20 through the DATA0 serial port, DATA1 serial port and DATA2 serial port and the CLK0 serial port, CLK1 serial port and CLK2 serial port of the PDM interface or the I2S interface.
[0074] Exemplarily, the I2S interfaces of the six microphones M are connected to the I2S interface of the digital audio processor 20. After the sound source signal is converted from an analog signal to a digital signal by the microphone M, it is sent to the digital audio processor 20 through the I2S link and sampled and processed in the digital audio processor 20.
[0075] Optionally, in some feasible embodiments, reference Figure 7 , the audio processing system further includes: earphone 60, the earphone 60 includes: earphone microphone 61 and earphone speaker 62, the digital audio processor 20 further includes: a second analog-to-digital conversion interface 20d and a second digital-to-analog conversion interface 20c;
[0076] The second analog-to-digital conversion interface 20 d is connected to the earphone microphone 61 , and the second digital-to-analog conversion interface 20 c is connected to the earphone speaker 62 .
[0077] In this embodiment, the headphone microphone 61 of the headphone 60 collects the headphone sound source data and sends it to the second analog-to-digital conversion module 20d. The digital audio processor 20 processes the headphone sound source data, specifically performs sound source positioning, sound source enhancement, echo elimination, noise suppression, etc. on the headphone sound source data, and sends it to the headphone speaker 62 after processing by the central processing unit 10.
[0078] For example, reference Figure 7 A switch 63 may also be connected between the earphone speaker 62 and the second digital-to-analog conversion interface 20 c , and the switch 63 is used to control the sound of the earphone speaker 62 .
[0079] Optionally, in some feasible embodiments, reference Figure 6 , the audio processing system further includes: earphone 60 and signal processing module 70, the earphone 60 includes: earphone microphone 61 and earphone speaker 62;
[0080] The signal processing module 70 is connected to the earphone microphone 61 , the earphone speaker 62 and the central processor 10 respectively.
[0081] It should be noted that the earphone 60 is directly connected to the central processor 70 via the signal processing module 70, that is, the signal processing module 70 is a separate processing module of the earphone 60, and the signal processing module 70 is connected to the central processor. The signal processing module 70 can be specifically a PMU (Power Management Unit).
[0082] For example, reference Figure 6 The earphone speaker 62 may specifically be a right ear speaker and a left ear speaker. The right ear speaker 62 , the left ear speaker 62 and the earphone microphone 61 are processed by the signal processing module 70 respectively.
[0083] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the application concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.
Claims
1. An audio processing system, characterized in that: The audio processing system comprises: a central processing unit, a digital audio processor and a microphone array module; The microphone array module includes 2n microphones arranged bilinearly; The central processing unit is connected to the digital audio processor, and the digital audio processor is connected to the 2n microphones respectively.
2. The audio processing system according to claim 1, characterized in that The digital audio processor includes: a microphone sound source localization module, a microphone sound source enhancement module, a microphone echo cancellation module and a microphone noise suppression module; The microphone sound source localization module, the microphone sound source enhancement module, the microphone echo cancellation module and the microphone noise suppression module are respectively connected to the central processor and the 2n microphones.
3. The audio processing system according to claim 2, characterized in that The audio processing system also includes: a power amplifier and a speaker; The power amplifier is connected to the speaker and the digital audio processor respectively.
4. The audio processing system according to claim 3, characterized in that The digital audio processor comprises: a first analog-to-digital conversion interface and a first digital-to-analog conversion interface; The first digital-to-analog conversion interface is connected to the first analog-to-digital conversion interface and the power amplifier respectively.
5. The audio processing system according to claim 4, characterized in that The digital audio processor also includes: an audio interface; The audio interface is connected to 2n microphones.
6. The audio processing system according to claim 5, characterized in that The audio processing system further includes: a headset, the headset including a headset microphone and a headset speaker; the digital audio processor further includes: a second analog-to-digital conversion interface and a second digital-to-analog conversion interface; The second analog-to-digital conversion interface is connected to the earphone microphone, and the second digital-to-analog conversion interface is connected to the earphone speaker.
7. The audio processing system according to claim 5, characterized in that The audio processing system further comprises: an earphone and a signal processing module, wherein the earphone comprises an earphone microphone and an earphone speaker; The signal processing module is connected to the earphone microphone, the earphone speaker and the central processing unit respectively.
8. The audio processing system according to any one of claims 1 to 7, characterized in that: The microphone array modules are arranged symmetrically based on the center line of the device.
9. The audio processing system according to claim 8, characterized in that The lateral spacing of the microphones ranges from 30 mm to 40 mm, and the longitudinal spacing of the microphones ranges from 20 mm to 25 mm.
10. The audio processing system according to claim 9, characterized in that The number of microphones arranged horizontally is not less than the number of microphones arranged vertically.