A sound collection method and device based on small-size PCBA
Through small-size PCBA and quadruple cross microphone array, the distance between the microphone and the sound generator is confirmed, the delay estimate is calculated using generalized cross-correlation function, and the microphone array is optimized to solve the problem of inaccurate sound signal acquisition, which has achieved more High acquisition accuracy and positioning accuracy.
Patent Information
- Application Number
- CN202311530337.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-15
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2043-11-15
AI Technical Summary
The existing sound acquisition method has the problem of inaccurate sound signal acquisition, especially when the distance between the microphone and the sound generator does not meet the preset conditions, resulting in inaccurate delay estimates.
The sound acquisition method is realized by using small-size PCBA. Through the quadruple cross-type microphone array and sound generator, it is confirmed that the distance between the microphone and the sound generator meets the preset conditions, and the delay estimate is obtained using the generalized cross-correlation function to construct an analytical equation. The group calculates the position parameters of the sound generator and optimizes the microphone array to improve the accuracy of sound signal acquisition.
It improves the accuracy of sound signal acquisition and the accuracy of sound generator positioning, reduces frequent optimization of microphone arrays caused by errors, and enhances the optimization effect of sound signals.
Smart Images

Figure CN117768816B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of sound collection, and in particular to a method and device for realizing sound collection based on a small-size PCBA. Background Art
[0002] With the development of society and the application of scenarios such as video conferencing and multimedia classrooms, it is particularly important to be able to accurately collect sound.
[0003] Currently, the method for sound collection is: using a microphone to collect sounds in a sound collection environment, and performing noise reduction on the collected sounds to achieve sound collection.
[0004] Although the above method can realize the collection of sound, there is a problem of inaccurate collection of sound signals. Summary of the invention
[0005] The present invention provides a sound collection method, device and computer-readable storage medium based on a small-size PCBA, the main purpose of which is to solve the problem of inaccurate sound signal collection.
[0006] To achieve the above purpose, the present invention provides a method and device for implementing sound collection based on a small-size PCBA, comprising:
[0007] receiving a sound collection instruction, and confirming a sound collection environment based on the sound collection instruction, wherein the sound collection environment includes: a four-element cross-shaped microphone array and a sound generator for collecting sound, and four microphones in the four-element cross-shaped microphone array are located on the same circle;
[0008] After confirming that the distance between the microphone and the sound generator meets a preset distance condition, the sound generator is used to generate a sound signal to obtain a first sound signal;
[0009] Acquire a reference coordinate origin based on the four-element cross-shaped microphone array, construct a reference coordinate system using the reference coordinate origin, perform an identification operation on four microphones based on the reference coordinate system to obtain a coded microphone set, acquire the coordinates of each microphone in the coded microphone set, and obtain a coded microphone coordinate set, wherein the coded microphone coordinates in the coded microphone coordinate set correspond one-to-one to the coded microphones in the coded microphone set;
[0010] A combination of the first microphone and other microphones is obtained based on the encoded microphone set to obtain multiple microphone combinations, microphones are extracted from the four microphones in sequence, and the following operations are performed on the extracted microphones:
[0011] Using the extracted microphone to receive a first sound signal in a sound collection environment to obtain an initial sound signal, preprocessing the initial sound signal to obtain a target sound signal, and after confirming that the number of frames of the target sound signal is a preset frame length, obtaining a frame of sound signal;
[0012] Extract microphone combinations from the multiple microphone combinations in sequence, and perform the following operations on each of the extracted microphone combinations:
[0013] Obtaining a delay estimation value based on two different one-frame sound signals in the microphone combination;
[0014] The step of obtaining a delay estimation value based on two different one-frame sound signals in the microphone combination includes:
[0015] A generalized cross-correlation function is constructed based on two different one-frame sound signals in the microphone combination, and a time delay estimation value is obtained using the generalized cross-correlation function;
[0016] The generalized cross-correlation function is as follows:
[0017]
[0018] Among them, x 1 is a frame of sound signal from the first microphone, x a is a frame of sound signal from the ath microphone in the microphone combination, and the value of a is 2, 3, 4, represents the generalized cross-correlation function between a frame of sound signals of the first microphone and the a-th microphone at the same time, represents the cross power spectrum between a frame of sound signals of the first microphone and the ath microphone at the same time, are the cross power spectra of a frame of sound signal of the first microphone and the ath microphone with themselves, respectively, 1 (ω) represents a frame of sound signal from the first microphone, x' a (ω) represents the complex conjugate of a frame of sound signal from the ath microphone, e is the natural base, ω represents the frequency, j represents the imaginary unit, and t represents the time delay of a frame of sound signal relative to another frame of sound signal;
[0019] The delay estimation value is obtained based on the generalized cross-correlation function. The calculation formula is as follows:
[0020]
[0021] Among them, t a1 is the estimated value of the time delay of a frame of sound signal respectively acquired by the first microphone and the a-th microphone in the sound collection environment;
[0022] Summarizing the delay estimation values to obtain a delay estimation value set, constructing an analytical equation group based on the delay estimation value set and the encoding microphone coordinate set, and calculating a plurality of parameters for determining the position of the sound generator according to the analytical equation group, the parameters including a first pitch angle and a first azimuth angle;
[0023] The initial pitch angle and initial azimuth of the four-element cross microphone array are obtained, and the optimized pitch angle and optimized azimuth are obtained based on the initial pitch angle, the initial azimuth, the first pitch angle and the first azimuth. After adjusting the four-element cross microphone array based on the optimized pitch angle and the optimized azimuth, an optimized four-element cross microphone array is obtained, and the step of extracting microphones from the four microphones in sequence is returned, and the one frame sound signal is optimized based on the pre-built neural network model to obtain an optimized sound signal, thereby realizing the collection of sound.
[0024] Optionally, confirming that the distance between the microphone and the sound generator meets a preset distance condition includes:
[0025] Obtain the reference frequency range of the sound generator, and calculate the judgment distance according to the reference frequency range, wherein the calculation relationship is as follows:
[0026]
[0027] Among them, r is the judgment distance, d is the distance between two adjacent microphones in the four-element cross microphone array, v is the propagation speed of the sound emitted by the sound generator in the sound collection environment, and f is the distance between two adjacent microphones in the four-element cross microphone array. max Indicates the maximum value of the reference frequency in the reference frequency range;
[0028] Extract microphones from the microphone array in sequence, and perform the following operations on each of the extracted microphones:
[0029] Obtain the extracted distance between the microphone and the sound generator to obtain the target distance;
[0030] Compare the target distance with the judged distance;
[0031] If the target distance is greater than or equal to the judgment distance, it is confirmed that the extracted distance between the microphone and the sound generator meets the preset distance condition. Otherwise, it is prompted to adjust the extracted distance between the microphone and the sound generator until the target distance is greater than or equal to the judgment distance.
[0032] Optionally, obtaining a reference coordinate origin based on the four-element cross-shaped microphone array, constructing a reference coordinate system using the reference coordinate origin, and performing an identification operation on four microphones based on the reference coordinate system to obtain a coded microphone set includes:
[0033] A reference circle is obtained based on the four microphones, the center of the reference circle is taken as the reference coordinate origin, a reference coordinate system is constructed based on the reference coordinate origin, and the numbers of the four microphones are obtained according to the reference coordinate system, wherein the microphones located in the positive direction of the x-axis and in the negative direction of the x-axis are the first microphone and the third microphone respectively, and the microphones located in the positive direction of the y-axis and in the negative direction of the y-axis are the second microphone and the fourth microphone respectively;
[0034] The first microphone, the second microphone, the third microphone and the fourth microphone constitute the encoding microphone set.
[0035] Optionally, the preprocessing the initial sound signal to obtain a target sound signal includes:
[0036] Filtering the initial sound signal using a pre-built bandpass filter and the reference frequency range to obtain a noise-reduced sound signal;
[0037] The noise reduction sound signal is framed and windowed to obtain the target sound signal, wherein the frame length during framing is 30 ms, the frame shift is 10 ms, and the function during windowing is a Hamming window.
[0038] Optionally, the analytical equations are as follows:
[0039]
[0040] Among them, (x 0 ,y 0 ,z 0 ) represents the coordinates of the sound generator in the reference coordinate system, (x 1 ,y 1 ,z 1 ) represents the coordinate of the first microphone in the reference coordinate system, (x 2 ,y 2 ,z 2 ) represents the coordinates of the second microphone in the reference coordinate system, (x 3 ,y 3 ,z 3 ) represents the coordinates of the third microphone in the reference coordinate system, (x 4 ,y 4 ,z 4 ) represents the coordinates of the fourth microphone in the reference coordinate system, h 0 、h 1 、h 2 、h 3 、h 4 represent the distances between the sound generator and the reference coordinate origin, the first microphone, the second microphone, the third microphone, and the fourth microphone, respectively, 21 ,t 31 and t 41They respectively represent the estimated values of the time delays when the second microphone, the third microphone and the fourth microphone receive signals with the first microphone, and s represents the distance between the microphone and the coordinate origin.
[0041] Optionally, acquiring the optimized pitch angle and the optimized azimuth angle based on the initial pitch angle, the initial azimuth angle, the first pitch angle, and the first azimuth angle includes:
[0042] Acquire a reference pitch angle and a reference azimuth angle according to the initial pitch angle, the initial azimuth angle, the first pitch angle and the first azimuth angle, wherein the reference pitch angle is an absolute difference between the initial pitch angle and the first pitch angle, and the reference azimuth angle is an absolute difference between the initial azimuth angle and the first azimuth angle;
[0043] Respectively comparing the reference pitch angle with a preset pitch angle threshold and the reference azimuth angle with a preset azimuth angle threshold;
[0044] If the reference pitch angle is greater than the pitch angle threshold, the reference pitch angle is taken as the optimized pitch angle, otherwise the optimized pitch angle is taken as 0;
[0045] If the reference azimuth angle is greater than the azimuth angle threshold, the reference azimuth angle is taken as the optimized azimuth angle, otherwise the optimized azimuth angle is taken as 0.
[0046] Optionally, the neural network model is a convolutional recurrent neural network.
[0047] In order to solve the above problems, the present invention also provides a sound collection device based on a small-size PCBA, the device comprising:
[0048] A sound collection environment confirmation module, used to receive a sound collection instruction and confirm a sound collection environment based on the sound collection instruction, wherein the sound collection environment includes: a four-element cross-shaped microphone array and a sound generator for collecting sound, and the four microphones in the four-element cross-shaped microphone array are located on the same circle;
[0049] A distance condition confirmation module, used to confirm that the distance between the microphone and the sound generator meets a preset distance condition, and then generate a sound signal using the sound generator to obtain a first sound signal;
[0050] a preprocessing module, configured to obtain a reference coordinate origin based on the four-element cross-shaped microphone array, construct a reference coordinate system using the reference coordinate origin, perform an identification operation on four microphones based on the reference coordinate system to obtain a coded microphone set, obtain the coordinates of each microphone in the coded microphone set, and obtain a coded microphone coordinate set, wherein the coded microphone coordinates in the coded microphone coordinate set correspond one-to-one to the coded microphones in the coded microphone set;
[0051] A combination of the first microphone and other microphones is obtained based on the encoded microphone set to obtain multiple microphone combinations, microphones are extracted from the four microphones in sequence, and the following operations are performed on the extracted microphones:
[0052] Using the extracted microphone to receive a first sound signal in a sound collection environment to obtain an initial sound signal, preprocessing the initial sound signal to obtain a target sound signal, and after confirming that the number of frames of the target sound signal is a preset frame length, obtaining a frame of sound signal;
[0053] The position parameter calculation module is used to sequentially extract microphone combinations from the multiple microphone combinations, and perform the following operations on each of the extracted microphone combinations:
[0054] Obtaining a delay estimation value based on two different one-frame sound signals in the microphone combination;
[0055] The step of obtaining a delay estimation value based on two different one-frame sound signals in the microphone combination includes:
[0056] A generalized cross-correlation function is constructed based on two different one-frame sound signals in the microphone combination, and a time delay estimation value is obtained using the generalized cross-correlation function;
[0057] The generalized cross-correlation function is as follows:
[0058]
[0059] Among them, x 1 is a frame of sound signal from the first microphone, x a is a frame of sound signal from the ath microphone in the microphone combination, and the value of a is 2, 3, 4, represents the generalized cross-correlation function between a frame of sound signals of the first microphone and the a-th microphone at the same time, represents the cross power spectrum between a frame of sound signals of the first microphone and the ath microphone at the same time, are the cross power spectra of a frame of sound signal of the first microphone and the ath microphone with themselves, respectively, 1 (ω) represents a frame of sound signal from the first microphone, x' a (ω) represents the complex conjugate of a frame of sound signal from the ath microphone, e is the natural base, ω represents the frequency, j represents the imaginary unit, and t represents the time delay of a frame of sound signal relative to another frame of sound signal;
[0060] The delay estimation value is obtained based on the generalized cross-correlation function. The calculation formula is as follows:
[0061]
[0062] Among them, t a1 is the estimated value of the time delay of a frame of sound signal respectively acquired by the first microphone and the a-th microphone in the sound collection environment;
[0063] Summarizing the delay estimation values to obtain a delay estimation value set, constructing an analytical equation group based on the delay estimation value set and the encoding microphone coordinate set, and calculating a plurality of parameters for determining the position of the sound generator according to the analytical equation group, the parameters including a first pitch angle and a first azimuth angle;
[0064] A sound collection and processing module is used to obtain an initial pitch angle and an initial azimuth angle of the four-element cross-type microphone array, obtain an optimized pitch angle and an optimized azimuth angle based on the initial pitch angle, the initial azimuth angle, the first pitch angle and the first azimuth angle, adjust the four-element cross-type microphone array based on the optimized pitch angle and the optimized azimuth angle to obtain an optimized four-element cross-type microphone array, return to the step of extracting microphones from the four microphones in sequence, and optimize the one frame of sound signal based on a pre-built neural network model to obtain an optimized sound signal, thereby realizing sound collection.
[0065] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:
[0066] at least one processor; and,
[0067] a memory communicatively connected to the at least one processor; wherein,
[0068] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to implement the above-mentioned sound collection method and device based on a small-size PCBA.
[0069] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one instruction is stored. The at least one instruction is executed by a processor in an electronic device to implement the above-mentioned sound collection method and device based on a small-size PCBA.
[0070] Compared with the problem described in the background technology, the embodiment of the present invention confirms that the distance between the microphone and the sound generator meets the preset distance condition, and then uses the sound generator to generate a sound signal to obtain a first sound signal. Furthermore, the distance condition is met, so that the first sound signal is a plane wave when it propagates to the microphone, thereby eliminating the problem of inaccurate delay estimation due to the propagation form of the sound wave, and laying a foundation for increasing the accuracy of sound signal acquisition. The extracted microphone is used to receive a first sound signal in a sound collection environment to obtain an initial sound signal, the initial sound signal is preprocessed to obtain a target sound signal, and after confirming that the number of frames of the target sound signal is a preset frame length, a frame of sound signal is obtained. It can be seen that the embodiment of the present invention further improves the accuracy of positioning the sound generator by preprocessing the initial sound signal and framing the target sound signal, and obtaining a time delay estimation value through a regular and stable frame of sound signal, thereby improving the accuracy of sound signal collection. Furthermore, the time delay estimation values are summarized to obtain a time delay estimation value set, and an analytical equation group is constructed based on the time delay estimation value set and the encoded microphone coordinate set. According to the analytical equation group, multiple parameters for determining the position of the sound generator are calculated to obtain a first pitch angle and a first azimuth angle, and the initial pitch angle and initial azimuth angle of the four-element cross microphone array are obtained. Based on the initial pitch angle, the initial azimuth angle, the first pitch angle, the initial azimuth angle, the initial ... The optimized pitch angle and the optimized azimuth angle are obtained by adjusting the four-element cross-type microphone array based on the optimized pitch angle and the optimized azimuth angle, and the optimized four-element cross-type microphone array is obtained, and the step of extracting microphones from the four microphones in sequence is returned, and the one frame of sound signal is optimized based on the pre-built neural network model to obtain the optimized sound signal to realize the collection of sound. It can be seen that the embodiment of the present invention calculates the pitch angle and the azimuth angle in each frame of sound signal, and determines whether to optimize the four-element cross-type microphone array in an iterative manner, and the optimized pitch angle and the optimized azimuth angle are also calculated before the optimization. Further, the problem of frequently optimizing the four-element cross-type microphone array due to errors is solved by optimizing the pitch angle and the optimized azimuth angle, thereby ensuring the feasibility of the embodiment of the present invention, and when optimizing the sound signal, the form of a neural network is adopted, thereby improving the accuracy of the sound signal collection. Therefore, the sound collection method and device based on a small-size PCBA proposed in the present invention can solve the problem of inaccurate sound signal collection. BRIEF DESCRIPTION OF THE DRAWINGS
[0071] Figure 1 A schematic diagram of a flow chart of a method and device for implementing sound collection based on a small-size PCBA according to an embodiment of the present invention;
[0072] Figure 2 for Figure 1 A detailed implementation flow chart of one of the steps in the process;
[0073] Figure 3 for Figure 1 A detailed implementation flow diagram of another step in FIG.
[0074] Figure 4 A functional module diagram of a sound collection device based on a small-size PCBA provided in one embodiment of the present invention;
[0075] Figure 5 A schematic diagram of the structure of an electronic device for implementing the sound collection method and device based on a small-size PCBA provided in one embodiment of the present invention.
[0076] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION
[0077] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0078] The embodiment of the present application provides a method and device for implementing sound collection based on a small-size PCBA. The execution subject of the method and device for implementing sound collection based on a small-size PCBA includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided in the embodiment of the present application. In other words, the method and device for implementing sound collection based on a small-size PCBA can be executed by software or hardware installed in a terminal device or a server device. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc.
[0079] Embodiment 1:
[0080] Reference Figure 1 FIG. 1 is a flow chart of a method and device for implementing sound collection based on a small-size PCBA according to an embodiment of the present invention. In this embodiment, the method and device for implementing sound collection based on a small-size PCBA include:
[0081] S1. Receive a sound collection instruction, and confirm a sound collection environment based on the sound collection instruction, wherein the sound collection environment includes: a four-element cross-shaped microphone array and a sound generator for collecting sound, and four microphones in the four-element cross-shaped microphone array are located on the same circle.
[0082] It should be understood that PCBA is the abbreviation of Printed Circuit Board + Assembly, which refers to printed circuit board assembly or printed circuit board assembly.
[0083] It is understandable that the sound collection instruction is an instruction issued by the sound receiving personnel in order to receive the sound, the sound collection environment is a necessary environment for collecting sound, the sound generator refers to the object whose sound needs to be collected in the sound collection environment, for example: when a meeting is in progress, the host who presides over the meeting, and the microphone array refers to a microphone group composed of a certain number of microphones arranged in preset positions. Compared with the prior art, the embodiments of the present invention are to solve the problem of inaccurate sound collection. For example: the microphone array is suitable for the situation where the speaker does not move. If the speaker does not move while speaking, the position of the microphone array is manually adjusted, and the adjusted microphone array is used to collect sound. There are errors in the manual adjustment, which will lead to the problem of inaccurate sound collection.
[0084] Furthermore, when using a four-element cross-shaped microphone array to collect sound signals, noise in the sound collection environment will also be collected. Generally, the four-element cross-shaped microphone array does not necessarily need to satisfy that the positions of the four microphones are located in the same circle. The four-element cross-shaped microphone array proposed in the present invention refers to four microphones arranged in a cross shape. The layout of the microphone array will be based on a center point, and the four microphones are located in four directions starting from the center point. The figure formed by the four directions is a cross shape, and the four microphones in the four-element cross-shaped microphone array are located on the same circle. The microphones selected in the embodiment of the present invention are directional microphones. The four microphones are located on the same circle to reduce the time and resources required for solving the delay estimation value based on the four microphones. The direction of the sound generator can be determined in a shorter time, and by adjusting the position of the four-element cross-shaped microphone array and utilizing the characteristics of the directional microphone, the environmental noise in the specified direction can be filtered out while collecting the sound signal, thereby improving the accuracy of the sound signal collection. Optionally, a four-element cross-shaped microphone array is constructed based on a cardioid directional microphone. It should be explained that confirming the sound collection environment based on the sound collection instruction includes: parsing the sound collection instruction to obtain multiple instructions for confirming the sound collection environment, the instructions including a sound generator start instruction and a microphone array start instruction, and using the sound generator start instruction and the microphone array start instruction to start the sound generator and the four-element cross microphone array respectively.
[0085] For example, Xiao Zhang is a sound receiver of a speech. In order to receive the speaker's speech on the stage, Xiao Zhang issues a sound collection instruction.
[0086] S2. After confirming that the distance between the microphone and the sound generator meets a preset distance condition, the sound generator is used to generate a sound signal to obtain a first sound signal.
[0087] It should be understood that the first sound signal is the sound signal generated by the sound generator after the preset sound signal is input to the sound generator. In addition, the preset sound signal should be consistent with the frequency range of the sound signal to be actually used. For example, if the sound collection environment is a meeting to be held, and the speaker of the meeting is a male, then the frequency range of the male sound signal needs to be obtained. According to the frequency range of the male sound signal, the sound signal used to test whether the sound collector is working normally is selected, and the frequency of the sound signal is consistent with the frequency range of the male sound signal.
[0088] Understandably, see Figure 2 As shown, the step of confirming that the distance between the microphone and the sound generator meets a preset distance condition includes:
[0089] S21, obtaining a reference frequency range of the sound generator, and calculating the judgment distance according to the reference frequency range, wherein the calculation relationship is as follows:
[0090]
[0091] Among them, r is the judgment distance, d is the distance between two adjacent microphones in the four-element cross microphone array, v is the propagation speed of the sound emitted by the sound generator in the sound collection environment, and f is the distance between two adjacent microphones in the four-element cross microphone array. max Indicates the maximum value of the reference frequency in the reference frequency range;
[0092] S22, extracting microphones from the microphone array in sequence, and performing the following operations on each of the extracted microphones:
[0093] Obtain the extracted distance between the microphone and the sound generator to obtain the target distance;
[0094] S23, comparing the target distance with the judgment distance;
[0095] If the target distance is greater than or equal to the judgment distance, it is confirmed that the extracted distance between the microphone and the sound generator meets the preset distance condition. Otherwise, it is prompted to adjust the extracted distance between the microphone and the sound generator until the target distance is greater than or equal to the judgment distance.
[0096] For example, the four microphones in the four-element cross-shaped microphone array are located on the same circle, and the distances between the four microphones and the coordinate origin are all 1. Then the distance between two adjacent microphones in the four-element cross-shaped microphone array is
[0097] It should be understood that the purpose of confirming that the distance between the microphone and the sound generator meets the preset distance condition is to ensure that when the first sound signal is propagated to the microphones in the four-element cross-type microphone array, the waveform is a plane wave, and there is only a phase delay in the received signal between the microphones in the four-element cross-type microphone array. Otherwise, the distance difference between the sound generator and each microphone in the four-element cross-type microphone array cannot be ignored, resulting in that when the sound generator is not directly opposite the center of the four-element cross-type microphone array, the microphones in the four-element cross-type microphone array will be delayed in receiving the sound signal because the propagation mode of the sound signal is a spherical wave, and the position of the four-element cross-type microphone array cannot be accurately adjusted.
[0098] S3. Obtain a reference coordinate origin based on the four-element cross-shaped microphone array, construct a reference coordinate system using the reference coordinate origin, perform identification operations on the four microphones based on the reference coordinate system to obtain a coded microphone set, obtain the coordinates of each microphone in the coded microphone set to obtain a coded microphone coordinate set, obtain a combination of the first microphone and other microphones based on the coded microphone set to obtain multiple microphone combinations.
[0099] It is understandable that the encoded microphone coordinates in the encoded microphone coordinate set correspond one-to-one to the encoded microphones in the encoded microphone coordinate set, and multiple microphone combinations are: a combination of the first microphone and the second microphone, a combination of the first microphone and the third microphone, and a combination of the first microphone and the fourth microphone.
[0100] It should be understood that the obtaining of the reference coordinate origin based on the four-element cross-shaped microphone array, the construction of a reference coordinate system using the reference coordinate origin, and the performing of identification operations on the four microphones based on the reference coordinate system to obtain the coded microphone set include:
[0101] A reference circle is obtained based on the four microphones, the center of the reference circle is taken as the reference coordinate origin, a reference coordinate system is constructed based on the reference coordinate origin, and the numbers of the four microphones are obtained according to the reference coordinate system, wherein the microphones located in the positive direction of the x-axis and in the negative direction of the x-axis are the first microphone and the third microphone respectively, and the microphones located in the positive direction of the y-axis and in the negative direction of the y-axis are the second microphone and the fourth microphone respectively;
[0102] The first microphone, the second microphone, the third microphone and the fourth microphone constitute the encoding microphone set.
[0103] Furthermore, the encoded microphone coordinate set is a set of coordinates corresponding to microphones after the microphones in the four-element cross microphone array are marked.
[0104] It is understandable that the reference circle refers to a circle where four microphones are simultaneously located on the same circle, and the reference coordinate system is a spatial coordinate system used to determine the coordinate position of the microphone and the sound generator relative to the four-element cross microphone array. Optionally, the Cartesian coordinate system is selected as the reference coordinate system. In addition, the purpose of performing the identification operation on the microphone is to facilitate the selection of the first microphone as a reference from the four microphones, and to solve the parameters used to determine the position of the sound generator at the current moment based on the first microphone.
[0105] Exemplarily, a Cartesian coordinate system is selected as the reference coordinate system, and a coordinate system is constructed with the center of the reference circle as the origin, so that the four microphones are respectively located on the x-axis and y-axis of the reference coordinate system, and the four microphones are identified so that the microphones located in the positive direction of the x-axis and the negative direction of the x-axis are the first microphone and the third microphone, respectively, and the microphones located in the positive direction of the y-axis and the negative direction of the y-axis are the second microphone and the fourth microphone, respectively.
[0106] S4. Extract microphones from the four microphones in sequence, and perform the following operations on the extracted microphones: use the extracted microphones to receive a first sound signal in a sound collection environment to obtain an initial sound signal, pre-process the initial sound signal to obtain a target sound signal, and after confirming that the number of frames of the target sound signal is a preset frame length, obtain a frame of sound signal.
[0107] It should be understood that there is environmental noise in the sound collection environment, and environmental noise will also be collected when the first sound signal is collected using a four-element cross-shaped microphone. Therefore, it is necessary to perform noise reduction on the initial sound signal to obtain a noise-reduced sound signal. The noise-reduced sound signal is a signal obtained by noise reduction of the initial sound signal through a bandpass filter, and the noise-reduced sound signal is a non-stationary signal, but the noise-reduced sound signal shows stability and regularity in a short time. Therefore, it is necessary to frame and window the noise-reduced sound signal so that each frame of the sound signal in the target sound signal has stability and regularity, thereby obtaining a delay estimate by analyzing each frame of the sound signal, and obtaining multiple parameters for determining the position of the sound generator based on the delay estimate, and adjusting the four-element cross-shaped microphone array through the parameters to make the sound collected by the four-element cross-shaped microphone array more accurate. If framing and windowing are not performed, the noise-reduced sound signal is not stable, and the delay estimate cannot be obtained from the noise-reduced signal.
[0108] Further, the preprocessing of the initial sound signal to obtain the target sound signal includes:
[0109] Filtering the initial sound signal using a pre-built bandpass filter and the reference frequency range to obtain a noise-reduced sound signal;
[0110] The noise reduction sound signal is framed and windowed to obtain the target sound signal, wherein the frame length during framing is 30 ms, the frame shift is 10 ms, and the function during windowing is a Hamming window.
[0111] It should be understood that the range of the bandpass filter is set as the reference frequency range, and the initial sound signal is filtered based on the bandpass filter, wherein the technology of filtering the initial sound signal through the bandpass filter is a prior art and will not be described in detail here.
[0112] S5. Extract microphone combinations from the multiple microphone combinations in sequence, and perform the following operations on the extracted microphone combinations: obtain delay estimation values based on two different one-frame sound signals in the microphone combination; summarize the delay estimation values to obtain a delay estimation value set, construct an analytical equation group based on the delay estimation value set and the encoded microphone coordinate set, and calculate multiple parameters for determining the position of the sound generator according to the analytical equation group, the parameters including a first pitch angle and a first azimuth angle.
[0113] It should be understood that the step of obtaining the delay estimation value based on two different one-frame sound signals in the microphone combination includes:
[0114] A generalized cross-correlation function is constructed based on two different one-frame sound signals in the microphone combination, and a time delay estimation value is obtained using the generalized cross-correlation function.
[0115] It should be explained that the generalized cross-correlation function is as follows:
[0116]
[0117] Among them, x 1 is a frame of sound signal from the first microphone, x a is a frame of sound signal from the ath microphone in the microphone combination, and the value of a is 2, 3, 4, represents the generalized cross-correlation function between a frame of sound signals of the first microphone and the a-th microphone at the same time, represents the cross power spectrum between a frame of sound signals of the first microphone and the ath microphone at the same time, are the cross power spectra of a frame of sound signal of the first microphone and the ath microphone with themselves, respectively, 1 (ω) represents a frame of sound signal from the first microphone, x' a (ω) represents the complex conjugate of a frame of sound signal from the ath microphone, e is the natural base, ω represents the frequency, j represents the imaginary unit, and t represents the time delay of a frame of sound signal relative to another frame of sound signal;
[0118] It is understandable that the difference between one frame of sound signal and another frame of sound signal lies in the first sound signals received by different microphones. Due to the different positions of the microphones, the first sound signals generated by the sound generator in the sound collection environment received by them are also different.
[0119] The delay estimation value is obtained based on the generalized cross-correlation function. The acquisition method is: Among them, t a1 is the estimated value of the time delay of a frame of sound signal respectively acquired by the first microphone and the ath microphone in the sound collection environment.
[0120] It should be explained that the analytical equations are as follows:
[0121]
[0122] Among them, (x 0 ,y 0 ,z 0 ) represents the coordinates of the sound generator in the reference coordinate system, (x 1 ,y 1 ,z 1 ) represents the coordinate of the first microphone in the reference coordinate system, (x 2 ,y 2 ,z 2 ) represents the coordinates of the second microphone in the reference coordinate system, (x 3 ,y 3 ,z 3 ) represents the coordinates of the third microphone in the reference coordinate system, (x 4 ,y 4 ,z 4 ) represents the coordinates of the fourth microphone in the reference coordinate system, h 0 、h 1 、h 2 、h 3 、h 4 represent the distances between the sound generator and the reference coordinate origin, the first microphone, the second microphone, the third microphone, and the fourth microphone, respectively, 21 ,t 31 and t 41 They respectively represent the estimated values of the time delays when the second microphone, the third microphone and the fourth microphone receive signals with the first microphone, and s represents the distance between the microphone and the coordinate origin.
[0123] Furthermore, the pitch angle and azimuth angle are:
[0124]
[0125] Wherein, α represents the pitch angle between the sound generator and the coordinate origin, is the azimuth between the sound generator and the coordinate origin.
[0126] It should be explained that the pitch angle refers to the angle between the straight line where the coordinate origin and the sound generator are located and the xoy plane in the constructed reference coordinate system when the coordinate origin points to the sound generator, and the azimuth angle refers to the angle between the straight line formed by the coordinates after the sound generator is projected to the xoy plane and the coordinate origin and the x-axis.
[0127] Exemplarily, the spatial coordinates of the sound generator in the reference coordinate system are (1, 1, 1), then the pitch angle is 45 degrees, and the azimuth angle is 45 degrees.
[0128] S6. Obtain an initial pitch angle and an initial azimuth angle of the four-element cross microphone array, obtain an optimized pitch angle and an optimized azimuth angle based on the initial pitch angle, the initial azimuth angle, the first pitch angle and the first azimuth angle, adjust the four-element cross microphone array based on the optimized pitch angle and the optimized azimuth angle to obtain an optimized four-element cross microphone array, return to the step of extracting microphones from the four microphones in sequence, and optimize the one frame of sound signal based on the pre-built neural network model to obtain an optimized sound signal, thereby realizing sound collection.
[0129] It should be explained that before the angle of the four-element cross-type microphone array is optimized, it is necessary to ensure that the four-element cross-type microphone array is facing the sound generator, and the initial pitch angle and the initial azimuth angle are both 0, and after returning to the step of extracting microphones from the four microphones in sequence, the first pitch angle solved in the previous step is taken as the initial pitch angle, and the first azimuth angle solved in the previous step is taken as the initial azimuth. In addition, the process of adjusting the four-element cross-type microphone array based on the optimized pitch angle and the optimized azimuth angle is: using the optimized pitch angle and the optimized azimuth angle to adjust the four-element cross-type microphone array, so that the sound collection area of the four-element cross-type microphone array is facing the sound generator, and the distance between each microphone in the four-element cross-type microphone array and the sound generator is equal.
[0130] It is understandable that in actual use, the four-element cross-shaped microphone array may not be facing the moving sound generator. If the position facing the four-element cross-shaped microphone array is the position of the sound generator, the first sound signal received based on the four-element cross-shaped microphone array is more accurate. For example: Xiao Wang is a speaker giving a speech on the stage. During the speech, Xiao Wang cannot ensure that the position of his mouth remains fixed. Therefore, it is necessary to adjust the position of the four-element cross-shaped microphone array to achieve accurate collection of sound signals.
[0131] It should be explained that see Figure 3As shown, the step of obtaining the optimized pitch angle and the optimized azimuth angle based on the initial pitch angle, the initial azimuth angle, the first pitch angle, and the first azimuth angle includes:
[0132] S61, acquiring a reference pitch angle and a reference azimuth angle according to the initial pitch angle, the initial azimuth angle, the first pitch angle and the first azimuth angle, wherein the reference pitch angle is an absolute difference between the initial pitch angle and the first pitch angle, and the reference azimuth angle is an absolute difference between the initial azimuth angle and the first azimuth angle;
[0133] S62, respectively comparing the reference pitch angle with a preset pitch angle threshold and the reference azimuth angle with a preset azimuth angle threshold;
[0134] S63, if the reference pitch angle is greater than the pitch angle threshold, taking the reference pitch angle as the optimized pitch angle, otherwise taking the optimized pitch angle as 0;
[0135] If the reference azimuth angle is greater than the azimuth angle threshold, the reference azimuth angle is taken as the optimized azimuth angle, otherwise the optimized azimuth angle is taken as 0.
[0136] It should be understood that the neural network model is a convolutional recurrent neural network.
[0137] It is understandable that the technology for enhancing sound signals based on convolutional recurrent neural networks is a prior art and will not be described in detail here.
[0138] Compared with the problem described in the background technology, the embodiment of the present invention confirms that the distance between the microphone and the sound generator meets the preset distance condition, and then uses the sound generator to generate a sound signal to obtain a first sound signal. Furthermore, the distance condition is met, so that the first sound signal is a plane wave when it propagates to the microphone, thereby eliminating the problem of inaccurate delay estimation due to the propagation form of the sound wave, and laying a foundation for increasing the accuracy of sound signal acquisition. The extracted microphone is used to receive a first sound signal in a sound collection environment to obtain an initial sound signal, the initial sound signal is preprocessed to obtain a target sound signal, and after confirming that the number of frames of the target sound signal is a preset frame length, a frame of sound signal is obtained. It can be seen that the embodiment of the present invention further improves the accuracy of positioning the sound generator by preprocessing the initial sound signal and framing the target sound signal, and obtaining a time delay estimation value through a regular and stable frame of sound signal, thereby improving the accuracy of sound signal collection. Furthermore, the time delay estimation values are summarized to obtain a time delay estimation value set, and an analytical equation group is constructed based on the time delay estimation value set and the encoded microphone coordinate set. According to the analytical equation group, multiple parameters for determining the position of the sound generator are calculated to obtain a first pitch angle and a first azimuth angle, and the initial pitch angle and initial azimuth angle of the four-element cross microphone array are obtained. Based on the initial pitch angle, the initial azimuth angle, the first pitch angle, the initial azimuth angle, the initial ... The optimized pitch angle and the optimized azimuth angle are obtained by adjusting the four-element cross-type microphone array based on the optimized pitch angle and the optimized azimuth angle, and the optimized four-element cross-type microphone array is obtained, and the step of extracting microphones from the four microphones in sequence is returned, and the one frame of sound signal is optimized based on the pre-built neural network model to obtain the optimized sound signal to realize the collection of sound. It can be seen that the embodiment of the present invention calculates the pitch angle and the azimuth angle in each frame of sound signal, and determines whether to optimize the four-element cross-type microphone array in an iterative manner, and the optimized pitch angle and the optimized azimuth angle are also calculated before the optimization. Further, the problem of frequently optimizing the four-element cross-type microphone array due to errors is solved by optimizing the pitch angle and the optimized azimuth angle, thereby ensuring the feasibility of the embodiment of the present invention, and when optimizing the sound signal, the form of a neural network is adopted, thereby improving the accuracy of the sound signal collection. Therefore, the sound collection method and device based on a small-size PCBA proposed in the present invention can solve the problem of inaccurate sound signal collection.
[0139] Embodiment 2:
[0140] like Figure 4 , which is a functional module diagram of a sound collection device based on a small-size PCBA provided by an embodiment of the present invention.
[0141] The sound collection device 100 based on a small-size PCBA of the present invention can be installed in an electronic device. According to the functions implemented, the sound collection device 100 based on a small-size PCBA can include a sound collection environment confirmation module 101, a distance condition confirmation module 102, a preprocessing module 103, a position parameter calculation module 104 and a sound collection and processing module 105. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.
[0142] The sound collection environment confirmation module 101 is used to receive a sound collection instruction and confirm the sound collection environment based on the sound collection instruction, wherein the sound collection environment includes: a four-element cross-shaped microphone array and a sound generator for collecting sound, and the four microphones in the four-element cross-shaped microphone array are located on the same circle;
[0143] The distance condition confirmation module 102 is used to confirm that the distance between the microphone and the sound generator meets the preset distance condition, and then use the sound generator to generate a sound signal to obtain a first sound signal;
[0144] The preprocessing module 103 is used to obtain a reference coordinate origin based on the four-element cross-shaped microphone array, construct a reference coordinate system using the reference coordinate origin, perform an identification operation on four microphones based on the reference coordinate system to obtain a coded microphone set, obtain the coordinates of each microphone in the coded microphone set, and obtain a coded microphone coordinate set, wherein the coded microphone coordinates in the coded microphone coordinate set correspond one-to-one to the coded microphones in the coded microphone set;
[0145] Based on the encoded microphone set, a combination of the first microphone and other microphones is obtained to obtain multiple microphone combinations, microphones are extracted from the four microphones in sequence, and the following operations are performed on the extracted microphones:
[0146] Using the extracted microphone to receive a first sound signal in a sound collection environment to obtain an initial sound signal, preprocessing the initial sound signal to obtain a target sound signal, and after confirming that the number of frames of the target sound signal is a preset frame length, obtaining a frame of sound signal;
[0147] The position parameter calculation module 104 is used to sequentially extract microphone combinations from the multiple microphone combinations, and perform the following operations on each of the extracted microphone combinations:
[0148] Obtaining a delay estimation value based on two different one-frame sound signals in the microphone combination;
[0149] The step of obtaining a delay estimation value based on two different one-frame sound signals in the microphone combination includes:
[0150] A generalized cross-correlation function is constructed based on two different one-frame sound signals in the microphone combination, and a time delay estimation value is obtained using the generalized cross-correlation function;
[0151] The generalized cross-correlation function is as follows:
[0152]
[0153] Among them, x 1 is a frame of sound signal from the first microphone, x a is a frame of sound signal from the ath microphone in the microphone combination, and the value of a is 2, 3, 4, represents the generalized cross-correlation function between a frame of sound signals of the first microphone and the a-th microphone at the same time, represents the cross power spectrum between a frame of sound signals of the first microphone and the ath microphone at the same time, are the cross power spectra of a frame of sound signal of the first microphone and the ath microphone with themselves, respectively, 1 (ω) represents a frame of sound signal from the first microphone, x' a (ω) represents the complex conjugate of a frame of sound signal from the ath microphone, e is the natural base, ω represents the frequency, j represents the imaginary unit, and t represents the time delay of a frame of sound signal relative to another frame of sound signal;
[0154] The delay estimation value is obtained based on the generalized cross-correlation function. The calculation formula is as follows:
[0155]
[0156] Among them, t a1 is the estimated value of the time delay of a frame of sound signal respectively acquired by the first microphone and the a-th microphone in the sound collection environment;
[0157] Summarizing the delay estimation values to obtain a delay estimation value set, constructing an analytical equation group based on the delay estimation value set and the encoding microphone coordinate set, and calculating a plurality of parameters for determining the position of the sound generator according to the analytical equation group, the parameters including a first pitch angle and a first azimuth angle;
[0158] The sound collection and processing module 105 is used to obtain the initial pitch angle and initial azimuth of the four-element cross microphone array, obtain the optimized pitch angle and optimized azimuth based on the initial pitch angle, the initial azimuth, the first pitch angle and the first azimuth, adjust the four-element cross microphone array based on the optimized pitch angle and the optimized azimuth to obtain the optimized four-element cross microphone array, return to the step of extracting microphones from the four microphones in sequence, and optimize the one frame of sound signal based on the pre-built neural network model to obtain the optimized sound signal, thereby realizing the collection of sound.
[0159] In detail, the modules in the sound collection device 100 based on the small-size PCBA in the embodiment of the present invention are used in the same manner as above. Figure 1 The same technical means as the method and device for realizing sound collection based on small-size PCBA described in the specification and the like can produce the same technical effects, which will not be described in detail here.
[0160] Embodiment 3:
[0161] like Figure 5 , is a schematic diagram of the structure of an electronic device for implementing a sound collection method and device based on a small-size PCBA provided by an embodiment of the present invention.
[0162] The electronic device 1 may include a processor 10, a memory 11, a bus 12 and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a sound collection program implemented based on a small-size PCBA.
[0163] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. The memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, such as a mobile hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Further, the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 may not only be used to store application software and various types of data installed in the electronic device 1, such as the code for implementing a sound acquisition program based on a small-size PCBA, but may also be used to temporarily store data that has been output or is to be output.
[0164] The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or a plurality of integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules stored in the memory 11 (for example, a sound collection program based on a small-size PCBA, etc.), and calls data stored in the memory 11 to execute various functions of the electronic device 1 and process data.
[0165] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.
[0166] Figure 5 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 5 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.
[0167] For example, although not shown, the electronic device 1 may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The electronic device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.
[0168] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices.
[0169] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.
[0170] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.
[0171] The sound collection program based on the small-size PCBA stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve:
[0172] receiving a sound collection instruction, and confirming a sound collection environment based on the sound collection instruction, wherein the sound collection environment includes: a four-element cross-shaped microphone array and a sound generator for collecting sound, and four microphones in the four-element cross-shaped microphone array are located on the same circle;
[0173] After confirming that the distance between the microphone and the sound generator meets a preset distance condition, the sound generator is used to generate a sound signal to obtain a first sound signal;
[0174] Acquire a reference coordinate origin based on the four-element cross-shaped microphone array, construct a reference coordinate system using the reference coordinate origin, perform an identification operation on four microphones based on the reference coordinate system to obtain a coded microphone set, acquire the coordinates of each microphone in the coded microphone set, and obtain a coded microphone coordinate set, wherein the coded microphone coordinates in the coded microphone coordinate set correspond one-to-one to the coded microphones in the coded microphone set;
[0175] A combination of the first microphone and other microphones is obtained based on the encoded microphone set to obtain multiple microphone combinations, microphones are extracted from the four microphones in sequence, and the following operations are performed on the extracted microphones:
[0176] Using the extracted microphone to receive a first sound signal in a sound collection environment to obtain an initial sound signal, preprocessing the initial sound signal to obtain a target sound signal, and after confirming that the number of frames of the target sound signal is a preset frame length, obtaining a frame of sound signal;
[0177] Extract microphone combinations from the multiple microphone combinations in sequence, and perform the following operations on the extracted microphone combinations:
[0178] Obtaining a delay estimation value based on two different one-frame sound signals in the microphone combination;
[0179] The step of obtaining a delay estimation value based on two different one-frame sound signals in the microphone combination includes:
[0180] A generalized cross-correlation function is constructed based on two different one-frame sound signals in the microphone combination, and a time delay estimation value is obtained using the generalized cross-correlation function;
[0181] The generalized cross-correlation function is as follows:
[0182]
[0183] Among them, x 1 is a frame of sound signal from the first microphone, x a is a frame of sound signal from the ath microphone in the microphone combination, and the value of a is 2, 3, 4, represents the generalized cross-correlation function between a frame of sound signals of the first microphone and the a-th microphone at the same time, represents the cross power spectrum between a frame of sound signals of the first microphone and the ath microphone at the same time, are the cross power spectra of a frame of sound signal of the first microphone and the ath microphone with themselves, respectively, 1 (ω) represents a frame of sound signal from the first microphone, x' a (ω) represents the complex conjugate of a frame of sound signal from the ath microphone, e is the natural base, ω represents the frequency, j represents the imaginary unit, and t represents the time delay of a frame of sound signal relative to another frame of sound signal;
[0184] The delay estimation value is obtained based on the generalized cross-correlation function. The calculation formula is as follows:
[0185]
[0186] Among them, t a1 is the estimated value of the time delay of a frame of sound signal respectively acquired by the first microphone and the a-th microphone in the sound collection environment;
[0187] Summarizing the delay estimation values to obtain a delay estimation value set, constructing an analytical equation group based on the delay estimation value set and the encoding microphone coordinate set, and calculating a plurality of parameters for determining the position of the sound generator according to the analytical equation group, the parameters including a first pitch angle and a first azimuth angle;
[0188] The initial pitch angle and initial azimuth of the four-element cross microphone array are obtained, and the optimized pitch angle and optimized azimuth are obtained based on the initial pitch angle, the initial azimuth, the first pitch angle and the first azimuth. After adjusting the four-element cross microphone array based on the optimized pitch angle and the optimized azimuth, an optimized four-element cross microphone array is obtained, and the step of extracting microphones from the four microphones in sequence is returned, and the one frame sound signal is optimized based on the pre-built neural network model to obtain an optimized sound signal, thereby realizing the collection of sound.
[0189] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to Figures 1 to 4 The description of the relevant steps in the corresponding embodiments will not be repeated here.
[0190] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).
[0191] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor of an electronic device, the computer program can implement:
[0192] receiving a sound collection instruction, and confirming a sound collection environment based on the sound collection instruction, wherein the sound collection environment includes: a four-element cross-shaped microphone array and a sound generator for collecting sound, and four microphones in the four-element cross-shaped microphone array are located on the same circle;
[0193] After confirming that the distance between the microphone and the sound generator meets a preset distance condition, the sound generator is used to generate a sound signal to obtain a first sound signal;
[0194] Acquire a reference coordinate origin based on the four-element cross-shaped microphone array, construct a reference coordinate system using the reference coordinate origin, perform an identification operation on four microphones based on the reference coordinate system to obtain a coded microphone set, acquire the coordinates of each microphone in the coded microphone set, and obtain a coded microphone coordinate set, wherein the coded microphone coordinates in the coded microphone coordinate set correspond one-to-one to the coded microphones in the coded microphone set;
[0195] Based on the encoded microphone set, a combination of the first microphone and other microphones is obtained to obtain multiple microphone combinations, microphones are extracted from the four microphones in sequence, and the following operations are performed on the extracted microphones:
[0196] Using the extracted microphone to receive a first sound signal in a sound collection environment to obtain an initial sound signal, preprocessing the initial sound signal to obtain a target sound signal, and after confirming that the number of frames of the target sound signal is a preset frame length, obtaining a frame of sound signal;
[0197] Extract microphone combinations from the multiple microphone combinations in sequence, and perform the following operations on the extracted microphone combinations:
[0198] Obtaining a delay estimation value based on two different one-frame sound signals in the microphone combination;
[0199] The step of obtaining a delay estimation value based on two different one-frame sound signals in the microphone combination includes:
[0200] A generalized cross-correlation function is constructed based on two different one-frame sound signals in the microphone combination, and a time delay estimation value is obtained using the generalized cross-correlation function;
[0201] The generalized cross-correlation function is as follows:
[0202]
[0203] Among them, x 1 is a frame of sound signal from the first microphone, x a is a frame of sound signal from the ath microphone in the microphone combination, and the value of a is 2, 3, 4, represents the generalized cross-correlation function between a frame of sound signals of the first microphone and the a-th microphone at the same time, represents the cross power spectrum between a frame of sound signals of the first microphone and the ath microphone at the same time, are the cross power spectra of a frame of sound signal of the first microphone and the ath microphone with themselves, respectively, 1 (ω) represents a frame of sound signal from the first microphone, x' a (ω) represents the complex conjugate of a frame of sound signal from the ath microphone, e is the natural base, ω represents the frequency, j represents the imaginary unit, and t represents the time delay of a frame of sound signal relative to another frame of sound signal;
[0204] The delay estimation value is obtained based on the generalized cross-correlation function. The calculation formula is as follows:
[0205]
[0206] Among them, ta1 is the estimated value of the time delay of a frame of sound signal respectively acquired by the first microphone and the a-th microphone in the sound collection environment;
[0207] Summarizing the delay estimation values to obtain a delay estimation value set, constructing an analytical equation group based on the delay estimation value set and the encoding microphone coordinate set, and calculating a plurality of parameters for determining the position of the sound generator according to the analytical equation group, the parameters including a first pitch angle and a first azimuth angle;
[0208] The initial pitch angle and initial azimuth of the four-element cross microphone array are obtained, and the optimized pitch angle and optimized azimuth are obtained based on the initial pitch angle, the initial azimuth, the first pitch angle and the first azimuth. After adjusting the four-element cross microphone array based on the optimized pitch angle and the optimized azimuth, an optimized four-element cross microphone array is obtained, and the step of extracting microphones from the four microphones in sequence is returned, and the one frame sound signal is optimized based on the pre-built neural network model to obtain an optimized sound signal, thereby realizing the collection of sound.
[0209] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.
[0210] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0211] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.
[0212] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.
[0213] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.
Claims
1. A method and device for sound collection based on a small-size PCBA, characterized in that: The method comprises: receiving a sound collection instruction, and confirming a sound collection environment based on the sound collection instruction, wherein the sound collection environment includes: a four-element cross-shaped microphone array and a sound generator for collecting sound, and four microphones in the four-element cross-shaped microphone array are located on the same circle; After confirming that the distance between the microphone and the sound generator meets a preset distance condition, the sound generator is used to generate a sound signal to obtain a first sound signal; Acquire a reference coordinate origin based on the four-element cross-shaped microphone array, construct a reference coordinate system using the reference coordinate origin, perform an identification operation on four microphones based on the reference coordinate system to obtain a coded microphone set, acquire the coordinates of each microphone in the coded microphone set, and obtain a coded microphone coordinate set, wherein the coded microphone coordinates in the coded microphone coordinate set correspond one-to-one to the coded microphones in the coded microphone set; Based on the encoded microphone set, a combination of the first microphone and other microphones is obtained to obtain a plurality of microphone combinations, microphones are extracted from the four microphones in sequence, and the following operations are performed on the extracted microphones: a first sound signal in a sound collection environment is received by the extracted microphone to obtain an initial sound signal, the initial sound signal is preprocessed to obtain a target sound signal, and after confirming that the number of frames of the target sound signal is a preset frame length, a frame of sound signal is obtained; Extracting microphone combinations from the multiple microphone combinations in sequence, and performing the following operations on the extracted microphone combinations: acquiring a delay estimation value based on two different one-frame sound signals in the microphone combination; The step of obtaining a delay estimation value based on two different one-frame sound signals in the microphone combination includes: A generalized cross-correlation function is constructed based on two different one-frame sound signals in the microphone combination, and a time delay estimation value is obtained using the generalized cross-correlation function; The generalized cross-correlation function is as follows: Among them, x1 is a frame of sound signal from the first microphone, x a is a frame of sound signal from the ath microphone in the microphone combination, and the value of a is 2, 3, 4, represents the generalized cross-correlation function between a frame of sound signals of the first microphone and the a-th microphone at the same time, represents the cross power spectrum between a frame of sound signals of the first microphone and the ath microphone at the same time, are the cross power spectra of a frame of sound signal of the first microphone and the ath microphone with themselves, x1(ω) represents a frame of sound signal of the first microphone, x , a (ω) represents the complex conjugate of a frame of sound signal from the ath microphone, e is the natural base, ω represents the frequency, j represents the imaginary unit, and t represents the time delay of a frame of sound signal relative to another frame of sound signal; The delay estimation value is obtained based on the generalized cross-correlation function. The calculation formula is as follows: Among them, t a1 is the estimated value of the time delay of a frame of sound signal respectively acquired by the first microphone and the a-th microphone in the sound collection environment; Summarizing the delay estimation values to obtain a delay estimation value set, constructing an analytical equation group based on the delay estimation value set and the encoding microphone coordinate set, and calculating a plurality of parameters for determining the position of the sound generator according to the analytical equation group, the parameters including a first pitch angle and a first azimuth angle; The initial pitch angle and initial azimuth of the four-element cross microphone array are obtained, and the optimized pitch angle and optimized azimuth are obtained based on the initial pitch angle, the initial azimuth, the first pitch angle and the first azimuth. After adjusting the four-element cross microphone array based on the optimized pitch angle and the optimized azimuth, an optimized four-element cross microphone array is obtained, and the step of extracting microphones from the four microphones in sequence is returned, and the one frame sound signal is optimized based on the pre-built neural network model to obtain an optimized sound signal, thereby realizing the collection of sound.
2. The method and device for realizing sound collection based on a small-size PCBA as claimed in claim 1, characterized in that: The step of confirming that the distance between the microphone and the sound generator satisfies a preset distance condition includes: Obtain the reference frequency range of the sound generator, and calculate the judgment distance according to the reference frequency range, wherein the calculation relationship is as follows: Among them, r is the judgment distance, d is the distance between two adjacent microphones in the four-element cross microphone array, v is the propagation speed of the sound emitted by the sound generator in the sound collection environment, and f is the distance between two adjacent microphones in the four-element cross microphone array. max Indicates the maximum value of the reference frequency in the reference frequency range; Extract microphones from the microphone array in sequence, and perform the following operations on each of the extracted microphones: Obtain the extracted distance between the microphone and the sound generator to obtain the target distance; Compare the target distance with the judged distance; If the target distance is greater than or equal to the judgment distance, it is confirmed that the extracted distance between the microphone and the sound generator meets the preset distance condition. Otherwise, it is prompted to adjust the extracted distance between the microphone and the sound generator until the target distance is greater than or equal to the judgment distance.
3. The method and device for realizing sound collection based on a small-size PCBA as claimed in claim 1, characterized in that: The method of obtaining a reference coordinate origin based on the four-element cross-shaped microphone array, constructing a reference coordinate system using the reference coordinate origin, and performing an identification operation on four microphones based on the reference coordinate system to obtain a coded microphone set includes: A reference circle is obtained based on the four microphones, the center of the reference circle is taken as the reference coordinate origin, a reference coordinate system is constructed based on the reference coordinate origin, and the numbers of the four microphones are obtained according to the reference coordinate system, wherein the microphones located in the positive direction of the x-axis and in the negative direction of the x-axis are the first microphone and the third microphone respectively, and the microphones located in the positive direction of the y-axis and in the negative direction of the y-axis are the second microphone and the fourth microphone respectively; The first microphone, the second microphone, the third microphone and the fourth microphone constitute the encoding microphone set.
4. The method and device for realizing sound collection based on a small-size PCBA as claimed in claim 2, characterized in that: The preprocessing of the initial sound signal to obtain the target sound signal includes: Filtering the initial sound signal using a pre-built bandpass filter and the reference frequency range to obtain a noise-reduced sound signal; The noise reduction sound signal is framed and windowed to obtain the target sound signal, wherein the frame length during framing is 30 ms, the frame shift is 10 ms, and the function during windowing is a Hamming window.
5. The method and device for realizing sound collection based on small-size PCBA as claimed in claim 2, characterized in that: The analytical equations are as follows: Wherein, (x0, y0, z0) represents the coordinates of the sound generator in the reference coordinate system, (x1, y1, z1) represents the coordinates of the first microphone in the reference coordinate system, (x2, y2, z2) represents the coordinates of the second microphone in the reference coordinate system, (x3, y3, z3) represents the coordinates of the third microphone in the reference coordinate system, (x4, y4, z4) represents the coordinates of the fourth microphone in the reference coordinate system, h0, h1, h2, h3, h4 represent the distances between the sound generator and the reference coordinate origin, the first microphone, the second microphone, the third microphone and the fourth microphone respectively, t 21 ,t 31 and t 41 They respectively represent the estimated values of the time delays when the second microphone, the third microphone and the fourth microphone receive signals with the first microphone, and s represents the distance between the microphone and the coordinate origin.
6. The method and device for realizing sound collection based on a small-size PCBA as claimed in claim 1, characterized in that: The step of obtaining the optimized pitch angle and the optimized azimuth angle based on the initial pitch angle, the initial azimuth angle, the first pitch angle, and the first azimuth angle includes: Acquire a reference pitch angle and a reference azimuth angle according to the initial pitch angle, the initial azimuth angle, the first pitch angle and the first azimuth angle, wherein the reference pitch angle is an absolute difference between the initial pitch angle and the first pitch angle, and the reference azimuth angle is an absolute difference between the initial azimuth angle and the first azimuth angle; Respectively comparing the reference pitch angle with a preset pitch angle threshold and the reference azimuth angle with a preset azimuth angle threshold; If the reference pitch angle is greater than the pitch angle threshold, the reference pitch angle is taken as the optimized pitch angle, otherwise the optimized pitch angle is taken as 0; If the reference azimuth angle is greater than the azimuth angle threshold, the reference azimuth angle is taken as the optimized azimuth angle, otherwise the optimized azimuth angle is taken as 0.
7. The method and device for realizing sound collection based on small-size PCBA as claimed in claim 1, characterized in that: The neural network model is a convolutional recurrent neural network.
8. A sound collection device based on a small-size PCBA, characterized in that: The device comprises: A sound collection environment confirmation module, used to receive a sound collection instruction and confirm a sound collection environment based on the sound collection instruction, wherein the sound collection environment includes: a four-element cross-shaped microphone array and a sound generator for collecting sound, and the four microphones in the four-element cross-shaped microphone array are located on the same circle; A distance condition confirmation module, used to confirm that the distance between the microphone and the sound generator meets a preset distance condition, and then generate a sound signal using the sound generator to obtain a first sound signal; a preprocessing module, configured to obtain a reference coordinate origin based on the four-element cross-shaped microphone array, construct a reference coordinate system using the reference coordinate origin, perform an identification operation on four microphones based on the reference coordinate system to obtain a coded microphone set, obtain the coordinates of each microphone in the coded microphone set, and obtain a coded microphone coordinate set, wherein the coded microphone coordinates in the coded microphone coordinate set correspond one-to-one to the coded microphones in the coded microphone set; Based on the encoded microphone set, a combination of the first microphone and other microphones is obtained to obtain multiple microphone combinations, microphones are extracted from the four microphones in sequence, and the following operations are performed on the extracted microphones: Using the extracted microphone to receive a first sound signal in a sound collection environment to obtain an initial sound signal, preprocessing the initial sound signal to obtain a target sound signal, and after confirming that the number of frames of the target sound signal is a preset frame length, obtaining a frame of sound signal; The position parameter calculation module is used to sequentially extract microphone combinations from the multiple microphone combinations, and perform the following operations on each of the extracted microphone combinations: Obtaining a delay estimation value based on two different one-frame sound signals in the microphone combination; The step of obtaining a delay estimation value based on two different one-frame sound signals in the microphone combination includes: A generalized cross-correlation function is constructed based on two different one-frame sound signals in the microphone combination, and a time delay estimation value is obtained using the generalized cross-correlation function; The generalized cross-correlation function is as follows: Among them, x1 is a frame of sound signal from the first microphone, x a is a frame of sound signal from the ath microphone in the microphone combination, and the value of a is 2, 3, 4, represents the generalized cross-correlation function between a frame of sound signals of the first microphone and the a-th microphone at the same time, represents the cross power spectrum between a frame of sound signals of the first microphone and the ath microphone at the same time, are the cross power spectra of a frame of sound signal of the first microphone and the ath microphone with themselves, x1(ω) represents a frame of sound signal of the first microphone, x , a (ω) represents the complex conjugate of a frame of sound signal from the ath microphone, e is the natural base, ω represents the frequency, j represents the imaginary unit, and t represents the time delay of a frame of sound signal relative to another frame of sound signal; The delay estimation value is obtained based on the generalized cross-correlation function. The calculation formula is as follows: Among them, t a1 is the estimated value of the time delay of a frame of sound signal respectively acquired by the first microphone and the a-th microphone in the sound collection environment; Summarizing the delay estimation values to obtain a delay estimation value set, constructing an analytical equation group based on the delay estimation value set and the encoding microphone coordinate set, and calculating a plurality of parameters for determining the position of the sound generator according to the analytical equation group, the parameters including a first pitch angle and a first azimuth angle; A sound collection and processing module is used to obtain an initial pitch angle and an initial azimuth angle of the four-element cross-type microphone array, obtain an optimized pitch angle and an optimized azimuth angle based on the initial pitch angle, the initial azimuth angle, the first pitch angle and the first azimuth angle, adjust the four-element cross-type microphone array based on the optimized pitch angle and the optimized azimuth angle to obtain an optimized four-element cross-type microphone array, return to the step of extracting microphones from the four microphones in sequence, and optimize the one frame of sound signal based on a pre-built neural network model to obtain an optimized sound signal, thereby realizing sound collection.
Citation Information
Patent Citations
Earphone, earphone system and earphone charging system
CN110099330A
Linear microphone array performance test method and clamp
CN110691314A
