Sound receiving processing method and sound signal processing device

Abstract

The invention provides a radio processing method and a sound signal processing device. In a method, a plurality of input sound signals are received by a microphone array. Those input sound signals are combined into a primary sound signal based on beamforming. A plurality of primary extraction signals belonging to the primary sound signal are extracted from those input sound signals, respectively. A plurality of estimated sound source signals corresponding to those sound sources are separated from those primarily extracted signals by blind signal separation. A plurality of re-extracted signals belonging to the estimated sound source signals are extracted from the primarily extracted signals, respectively. And determining the distance of the first source relative to the microphone array according to the power ratio. An estimated sound source signal corresponding to the first source is determined according to the distance limit. Therefore, the sound signal of the sound source to be obtained can be obtained.

Description

Technical Field

[0001] The present invention relates to a sound signal processing technology, and in particular, also relates to a sound collection processing method and a sound signal processing device. Background Art

[0002] In order to provide mobile devices with better recording and call quality, many signal processing applications for noise reduction are provided, among which beamforming algorithm is the most popular signal processing technology. The beamforming technology on the market can eliminate the noise around the environment and only retain the sound signal directly in front of the mobile device.

[0003] However, when there is an interference signal and the user is right in front of the mobile device, the current beamforming algorithm cannot distinguish which sound needs to be retained, and the interference signal affects the user's voice signal. Summary of the invention

[0004] The present invention is directed to a sound processing method and a sound signal processing device, which can distinguish the sound source and retain the sound signal of the target source.

[0005] According to an embodiment of the present invention, a sound processing method includes (but is not limited to) the following steps: receiving a plurality of input sound signals through a microphone array, wherein the microphone array includes a plurality of microphones for respectively receiving those input sound signals; combining those input sound signals into a main sound signal based on beamforming, wherein beamforming is used to form a radiation pattern corresponding to a first source among a plurality of sound sources in the microphone array, and the first microphone in the microphone array is closer to the first source than other microphones; extracting a plurality of primary extraction signals belonging to the main sound signal from those input sound signals respectively, wherein a plurality of first sound feature associations between those input sound signals and the main sound signals are trained in advance, and the first sound feature associations are used to extract the part belonging to the main sound signal; and extracting a plurality of primary extraction signals belonging to the main sound signal by blind signal separation. Separation (BSS) separates a plurality of estimated sound source signals corresponding to those sound sources from those primary extraction signals; extracts a plurality of re-extracted signals belonging to those estimated sound source signals respectively from those primary extraction signals, wherein a plurality of second sound feature associations between those primary extraction signals and those estimated sound source signals are trained in advance, and those second sound feature associations are respectively used to extract parts belonging to those estimated sound source signals; determines the distance of a first source relative to a microphone array according to a power ratio, wherein the power ratio is a ratio of a first value to a second value, the first value is the sum of the powers of those re-extracted signals corresponding to the first source and corresponding to other microphones in the microphone array that are not the first microphone, and the second value is the power of a re-extracted signal corresponding to the first source and corresponding to the first microphone, the smaller the power ratio is, the closer the distance of the first source relative to the microphone array is, and the larger the power ratio is, the farther the distance of the first source relative to the microphone array is; and determines the estimated sound source signal corresponding to the first source according to a distance limit, wherein the distance of the first source relative to the microphone array is within the distance limit.

[0006] According to an embodiment of the present invention, a sound signal processing device includes a microphone array, a memory and a processor. The microphone array includes a plurality of microphones that respectively receive a plurality of input sound signals. The memory is used to store program codes. The processor couples the microphone array and the memory. The processor is used to load program codes to execute: combining the input sound signals into a main sound signal based on beamforming, wherein beamforming is used to form a radiation field pattern corresponding to a first source among a plurality of sound sources in the microphone array, and the first microphone in the microphone array is closer to the first source than other microphones; extracting a plurality of primary extraction signals belonging to the main sound signal from the input sound signals respectively, wherein a plurality of first sound feature associations between the input sound signals and the main sound signals are trained in advance, and the first sound feature associations are used to extract the part belonging to the main sound signal; separating a plurality of estimated sound source signals corresponding to the sound sources from the primary extraction signals by blind signal separation; extracting a plurality of re-extraction signals belonging to the estimated sound source signals from the primary extraction signals respectively, wherein the primary extraction signals are trained in advance. The extraction signal is respectively associated with a plurality of second sound features of those estimated sound source signals, and those second sound feature associations are respectively used to extract the parts belonging to those estimated sound source signals; the distance of the first source relative to the microphone array is determined according to a power ratio, wherein the power ratio is a ratio of a first value to a second value, the first value is the sum of the powers of those re-extracted signals corresponding to the first source and corresponding to other microphones in the microphone array that are not the first microphone, and the second value is the power of a re-extracted signal corresponding to the first source and corresponding to the first microphone, the smaller the power ratio is, the closer the distance of the first source relative to the microphone array is, and the larger the power ratio is, the farther the distance of the first source relative to the microphone array is; and the estimated sound source signal corresponding to the first source is determined according to a distance limit, wherein the distance of the first source relative to the microphone array is within the distance limit.

[0007] Based on the above, the sound processing method and the sound signal processing device of the embodiment of the present invention can distinguish the sound source signal through beamforming, sound extraction and blind signal separation, determine the relative distance of the sound source signal to the source, and determine the sound source signal of a specific source accordingly. In this way, the sound signals of other users within the radiation field can be removed, thereby improving the recording and call quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The accompanying drawings are included to provide a further understanding of the present invention and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the present invention and together with the description serve to explain the principles of the present invention.

[0009] Figure 1 is a block diagram of components of a sound signal processing device according to an embodiment of the present invention;

[0010] Figure 2 is a schematic diagram of microphone installation positions and radiation patterns according to an embodiment of the present invention;

[0011] Figure 3 is a flow chart of a sound processing method according to an embodiment of the present invention;

[0012] Figure 4 is a schematic diagram illustrating signal processing according to an embodiment of the present invention;

[0013] Figure 5 is a schematic diagram illustrating a network architecture according to an embodiment of the present invention;

[0014] Figure 6 is a schematic diagram illustrating blind signal separation according to an embodiment of the present invention;

[0015] Figure 7 is a schematic diagram illustrating a positional relationship according to an embodiment of the present invention.

[0016] Description of Figure Numbers

[0017] 100: sound signal processing device;

[0018] 110: microphone array;

[0019] 11-1~11-M: Microphone;

[0020] 120: memory;

[0021] 130: processor;

[0022] S1~S6: sound source;

[0023] C B :B item;

[0024] C C :C item;

[0025] P B1 ~P B2 , P C1 ~P C4 :Location;

[0026] RP: radiation pattern;

[0027] S310~S370: steps;

[0028] x1, x2, x3: input sound signal;

[0029] X B : Main sound signal;

[0030] x 1 ~x R : Sample signal of known input signal;

[0031] y 1 ~y R : The part of the known input signal that is the main sound signal;

[0032] a 11 ~a 1i 、a 21 ~a 2j : Neuron;

[0033] Weight coefficient;

[0034] predicting an output sampling signal;

[0035] Initial extraction signal;

[0036] SS1, SS2, SS3: sound source signals;

[0037] M1, M2, M3: mixed signal;

[0038] y1, y2, y3: estimated sound source signal;

[0039] Extract the signal again. DETAILED DESCRIPTION

[0040] Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.

[0041] Figure 1 is a block diagram of components of a sound signal processing device 100 according to an embodiment of the present invention. Figure 1 The sound signal processing device 100 includes (but is not limited to) a microphone array 110, a memory 120, and a processor 130. The sound signal processing device 100 may be a smart phone, a voice recorder, a tablet computer, an intelligent assistant device, a wearable device, a vehicle system, a laptop computer, or other electronic devices.

[0042] The microphone array 110 includes a plurality of microphones 11-1, 11-2, ..., 11-M (M is a positive integer greater than one or greater than two). The microphones 11-1, 11-2, ..., 11-M may be dynamic, condenser, or electret condenser microphones. The microphones 11-1, 11-2, ..., 11-M may also be other electronic components that can receive sound waves (e.g., human voice, ambient sound, machine operation sound, etc.) (i.e., receive or record sound) and convert them into sound signals, an analog-to-digital converter, a filter, and a combination of an audio processor.

[0043] Figure 2 is a schematic diagram of microphone placement and radiation pattern according to an embodiment of the present invention. Figure 2 The sound signal processing device 100 takes a laptop computer 200 as an example. Assume that the laptop computer 200 includes microphones 11-1 to 11-3 (for example, M is 3). The cover C of the laptop computer 200 C Position P C1 ~P C4 The microphone 11-1 can be installed, and its B part (or B surface / cover) C B Position P B1 ~P B2 Microphones 11-2 and 11-3 may be provided. Generally speaking, the main user (eg, sound source S1) is located in front of the notebook computer 200 (eg, B, C B The direction facing). Set at C C The microphone 11-1 is compared with the microphone 11-1 located at the B part C. B The other microphones 11 - 2 and 11 - 3 are also close to the sound source S1 (hereinafter referred to as the first source).

[0044] It should be noted that the number and location of the microphones 11 - 1 , 11 - 2 , . . . , 11 -M may have other variations.

[0045] The memory 120 may be any type of fixed or removable random access memory (RAM), read only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD) or similar components. In one embodiment, the memory 120 is used to store program code, software modules, configurations, data (e.g., sound signals, algorithm parameters, etc.) or files, and its embodiments will be described in detail later.

[0046] The processor 130 is coupled to the microphone array 110 and the memory 120. The processor 130 may be a central processing unit (CPU), a graphics processing unit (GPU), or other programmable general-purpose or special-purpose microprocessor, a digital signal processor (DSP), a programmable controller, a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a neural network accelerator, or other similar components or a combination of the above components. In one embodiment, the processor 130 is used to execute all or part of the operations of the sound signal processing device 100, and can load and execute various program codes, software modules, files, and data stored in the memory 120. In some embodiments, the functions of the processor 130 can be implemented by software or chips.

[0047] Hereinafter, the method described in the embodiment of the present invention will be described with reference to the components and modules in the sound signal processing device 100 and / or the notebook computer 200. Each process of the method can be adjusted according to the implementation situation, and is not limited thereto.

[0048] Figure 3 is a flow chart of a method for processing sound according to an embodiment of the present invention. Figure 3 , the processor 130 receives multiple input sound signals through the microphone array 110 (step S310). Specifically, the multiple microphones 11-1, 11-2, ..., 11-M receive multiple sound signals respectively. For example, each microphone 11-1, 11-2, ..., 11-M receives sound waves (e.g., human voice, environmental sound, machine operation sound, etc.) respectively (i.e., picks up or records sound) and converts the sound waves into input sound signals.

[0049] The processor 130 combines multiple input sound signals into a main sound signal based on beamforming (step S320). Specifically, beamforming is used to form the microphone array 11 into a radiation pattern corresponding to the first source of multiple sound sources. Beamforming can adjust the parameters of the basic unit of the phase array (for example, phase and / or amplitude) so that signals at certain angles obtain constructive interference, while signals at other angles obtain destructive interference. Therefore, different parameters will form different radiation / beam patterns, and the directional angles of their main beams may be different. The processor 130 can predefine or generate the directional angle of the radiation pattern based on user input operations. For example, 90°, 85° or 105°, but not limited to this.

[0050] by Figure 2 For example, generally speaking, the main user is located in front of the device. Assume that the microphone 11-1 is located at position P C1 , and microphones 11-2 and 11-3 are respectively arranged at positions P B1 , P B2 The microphone 11 - 1 is closer to a sound source S1 (ie, the first source) than the other microphones 11 - 2 and 11 - 3 . Figure 4 is a schematic diagram illustrating signal processing according to an embodiment of the present invention. Figure 2 , assuming that the three microphones 11-1 to 11-3 receive input sound signals x1, x2, and x3 respectively. The processor 130 adjusts the amplitude and / or phase of the input sound signals x1, x2, and x3 of the microphones 11-1 to 11-3 respectively, and sums the adjusted input sound signals x1, x2, and x3 to generate a main sound signal X B . This main sound signal X B A radiation pattern RP (whose pointing angle is toward the bottom of the figure) can be formed. B The direction facing. Figure 2 As shown, the sound signal of the sound source S1 within the coverage of the radiation pattern RP has a higher gain, but the sound signals of other sound sources S2 and S3 outside the coverage of the radiation pattern RP have a lower gain (or even are removed).

[0051] Please refer to Figure 3, the processor 130 extracts a plurality of primary extraction signals belonging to the main sound signal from the plurality of input sound signals respectively (step S330). Specifically, the processor 130 or other devices / processors may pre-train a plurality of first sound feature associations between the plurality of input sound signals and the main sound signal respectively, and the first sound feature associations are used to extract the part belonging to the main sound signal. There are many kinds of sound features. For example, fundamental frequency, multiple frequency, speaking speed or sentence segmentation, but not limited thereto. The main sound signal may be assumed to be a sound source signal belonging to a certain sound source. Each first sound feature association is known to belong to the sound feature of the main sound signal, and is trained to obtain only the part belonging to the main sound signal from the input sound signal according to the sound feature of the main sound signal. The first sound feature association may be a comparison table, a mathematical relationship or a conversion model based on machine learning. That is, each first sound feature association understands the conversion relationship between the corresponding input sound signal and the main sound signal, and can be used to extract or convert the part belonging to the main sound signal from the input sound signal.

[0052] In one embodiment, the processor 130 can train multiple first sound feature associations between the main sound signal and multiple known input signals corresponding to the multiple microphones 11-1 to 11-M of the microphone array 110 through a machine learning algorithm. The machine learning algorithm is a multilayer perceptron (Multiple Layer Perception, MLP) of backpropagation, a recurrent neural network (Recurrent Neural Network, RNN), or a temporal convolutional network (Temporal Convolutional Network, TCN) (for example, Conv-TasNet), but is not limited thereto. Each first sound feature association includes a weight coefficient of an extraction model corresponding to one microphone 11-1, ..., 11-M. In other words, this extraction model is a model based on the machine learning algorithm. In the training process of the extraction model, the part of the known input signal received by each microphone 11-1, ..., 11-M that belongs to the main sound signal is known, so each known input signal (i.e., a known model output sample) and the part belonging to the main sound signal (i.e., a known model output sample) can be used as a training sample of the extraction model, and the extraction model is trained / optimized accordingly.

[0053] Figure 5 is a schematic diagram illustrating a network architecture according to an embodiment of the present invention. Figure 5 , taking the back-propagation multilayer perceptron as an example, x 1 ~x R is a sampled signal of a known input signal received by a certain microphone 11-1, ..., 11-M (R is the number of samples). 1 ~yR is the part of the known input signal that is the main sound signal. The input vector of the training sample is: {(x 1 ,y 1 ),…,(x r ,y r ),…,(x R ,y R )}, r is an integer from 1 to R. a 11 ~a 1i 、a 21 ~a 2j is the neuron of the hidden layer, and i and j are positive integers (without limiting their values). 11 The value of x 1 and The weight coefficient is: l is a positive integer.

[0054] In one embodiment, the processor 130 may define a loss function. The loss function is based on the difference between the portion of a known input signal of each microphone 11-1, ..., 11-M that is estimated to belong to the main sound signal based on the first sound feature association and the (real) main sound signal. For example, the loss function is: f() is the extraction model, f(x r ; W) is the sampling signal x of the known input signal after inputting the extraction model with the configuration weight coefficient W r The output signal (eg, the predicted output sample signal) That is, the part of the known input signal estimated to belong to the main sound signal based on the correlation with the first sound feature). It should be noted that the loss function may also be based on mean square error, root mean square error, mean absolute value error or other objective functions.

[0055] The processor 130 may minimize the loss function to determine the weight coefficients of the extraction model. For example, W * is the weight coefficient that can minimize the loss function.

[0056] Next, the processor 130 may use the extraction model with the weight coefficients to extract a plurality of primary extraction signals belonging to the main sound signal from the input sound signal. Figure 4 For example, the processor 130 can extract the main sound signal x from the input sound signals x1, x2, and x3 of the microphones 11-1 to 11-3. B The primary extraction signal

[0057] Please refer to Figure 3, the processor 130 separates multiple estimated sound source signals corresponding to the sound source from the primary extraction signal through blind signal separation (step S340). Specifically, in actual application scenarios, multiple sound sources often appear at the same time. "Blind" refers to a mixed signal formed by receiving sound signals (or sound source signals) from multiple sound sources, and one of the goals of the blind signal separation algorithm includes separating the sound signal of the main sound source when there is only a mixed signal.

[0058] For example, Figure 6 is a schematic diagram illustrating blind signal separation according to an embodiment of the present invention. Figure 6 , assuming that there are three users in the environment. The three users generate sound source signals SS1, SS2, and SS3 respectively. The input sound signals received by the three microphones 11-1 to 11-3 are mixed signals M1, M2, and M3 respectively. The mixed signals M1, M2, and M3 may all include sound source signals SS1, SS2, and SS3. Blind signal separation is used to separate the mixed signals M1, M2, and M3 into sound source signals SS1, SS2, and SS3.

[0059] Blind signal separation algorithms include, for example, independent component analysis (ICA), independent vector analysis (IVA), sparse component analysis (SCA), and independent low-rank matrix analysis (ILRMA) in tensorizing BSS. Taking independent low-rank matrix as an example, its analysis integrates independent vector analysis and non-negative matrix factorization, and replaces the vector source model in independent vector analysis with the source model of low-rank time-frequency structure.

[0060] by Figure 4 For example, the initial extraction signal The estimated sound source signals y1, y2, and y3 corresponding to the three sound sources can be separated through blind signal separation.

[0061] In some application scenarios, the number of microphones 11 - 1 ˜ 11 -M is greater than or equal to the number of estimated sound source signals to be separated.

[0062] Please refer to Figure 3, the processor 130 extracts a plurality of re-extracted signals belonging to the estimated sound source signal from the plurality of primary extraction signals respectively (step S350). Specifically, the processor 130 or other devices / processors may pre-train a plurality of second sound feature associations between the plurality of primary extraction signals and the estimated sound source signals respectively, and these second sound feature associations are used to extract the parts belonging to those estimated sound source signals respectively. Each estimated sound source signal may be assumed to be a sound source signal belonging to a certain sound source. Each second sound feature association is known to belong to a sound feature of a certain estimated sound source signal, and is trained to obtain only the part belonging to the estimated sound source signal from the primary extraction signal according to the sound feature of the estimated sound source signal. This second sound feature association may be a comparison table, a mathematical relationship or a conversion model based on machine learning. That is to say, each second sound feature association understands the conversion relationship between the corresponding primary extraction signal and a certain estimated sound source signal, and can be used to extract or convert the part belonging to the estimated sound source signal from the primary extraction signal.

[0063] In one embodiment, the processor 130 can train multiple second sound feature associations between the estimated sound source signal and the multiple known input signals corresponding to the multiple microphones 11-1 to 11-M of the microphone array 110 through a machine learning algorithm. The training of the second sound feature association can refer to the aforementioned description of the first sound feature association, which will not be repeated here. Each second sound feature association includes a weight coefficient of the second extraction model corresponding to a microphone 11-1, ..., 11-M. In the training process of the second extraction model, the part of the known input signal received by each microphone 11-1, ..., 11-M that belongs to a certain estimated sound source signal is known, so each known input signal (i.e., a known model output sample) and the part belonging to this estimated sound source signal (i.e., a known model output sample) can be used as training samples of the second extraction model, and the extraction model can be trained / optimized accordingly.

[0064] In one embodiment, the processor 130 may define a second loss function. The second loss function is based on the difference between a portion of a known input signal of each microphone 11-1, ..., 11-M estimated to belong to a certain estimated sound source signal based on the second sound feature association and the (real) estimated sound source signal. The processor 130 may minimize the second loss function to determine the weight coefficient of the second extraction model. The details of the definition of the second loss function and the minimization of the second loss function may refer to the aforementioned description of the loss function and the minimization of the loss function, respectively, and will not be repeated here.

[0065] by Figure 4 For example, the processor 130 may extract the signal from the initial The re-extracted signals of the estimated sound source signals y1, y2, and y3 belonging to the three sound sources are extracted respectively From the initial extraction signal The re-extracted signals of the estimated sound source signals y1, y2, and y3 belonging to the three sound sources are extracted respectively And from the initial extraction signal The re-extracted signals of the estimated sound source signals y1, y2, and y3 belonging to the three sound sources are extracted respectively

[0066] Please refer to Figure 3 , the processor 130 determines the distance of the first source relative to the microphone array according to the power ratio (step S360). Specifically, the power ratio is the ratio of the first value to the second value, and the first value corresponds to the first source (for example, Figure 2 The second value is the sum of the power of the re-extracted signal corresponding to the first source and the re-extracted signal corresponding to the (first) microphone 11-1. The power is, for example, the square of the amplitude of the signal. Taking the three microphones 11-1 to 11-3 as an example, is the power ratio, is the first value ( is the power of the re-extracted signal corresponding to the sound source S1 and corresponding to the microphone 11-2, and is the power of the re-extracted signal corresponding to the sound source S1 and corresponding to the microphone 11-3), and is the second value.

[0067] It is worth noting that the power of the signal from the sound source S1 reaching different microphones 11-1 to 11-M will linearly attenuate as the distance (relative to the microphones 11-1 to 11-M or the microphone array 110) increases. Microphone 11-1 is closer to the first source than other microphones 11-2 to 11-M. Therefore, the smaller the power ratio (i.e., the smaller the difference between the first value and the second value), the closer the first source is to the microphone array 110. In addition, the larger the power ratio (i.e., the larger the difference between the first value and the second value), the farther the first source is to the microphone array 110. Figure 2 For example, when When , the distance between the user (eg, the sound source S1) and the screen of the notebook computer 200 may be 50 centimeters (cm); when =1.5, the distance between the user (eg, the sound source S1) and the screen of the notebook computer 200 may be 100 cm.

[0068] In one embodiment, the processor 130 may determine the distance of the first source relative to the microphone array 110 according to the distance relationship ratio. The distance relationship ratio is the ratio of the third value and the fourth value, the third value is the distance coefficient, the fourth value is the difference between one and the fifth value, and the fifth value is the product of the adjustment coefficient and the power ratio. γ is the distance coefficient, is the fourth value, is the fifth value, α is the adjustment coefficient, and is the power ratio. The distance coefficient is the distance between the microphone closest to the first source and the first source. The distance coefficient is, for example, 25, 30 or 40 centimeters, but is not limited thereto.

[0069] Similarly, the processor 130 can determine the distances of other sound sources relative to the microphone array 110. For example, the first value and the second value are respectively changed to values ​​corresponding to other sound sources. Taking the first value as an example, the first value can be changed to correspond to the sum of the power of the re-extracted signal of another sound source and other microphones in the microphone array other than the first microphone; taking the second value as an example, the second value can be changed to correspond to the power of the re-extracted signal of another sound source and the first microphone. The detailed description will not be repeated.

[0070] It should be noted that the representative position of the microphone array 110 may be the position of any microphone 11 - 1 ˜ 11M or its statistical value (eg, centroid or geometric center), and is used to determine the distance of the sound source relative to the microphone array 110 .

[0071] Please refer to Figure 3 , the processor 130 determines the estimated sound source signal corresponding to the first source according to the distance limit (step S370). Specifically, the distance of the first source relative to the microphone array 110 is within the distance limit. The distance limit is, for example, 45, 50 or 60 centimeters. It should be noted that the value of the distance limit can be adjusted according to actual needs and is not limited to the above examples. The definition of the distance limit can be related to the distance of the user relative to the sound signal processing device 100, and may be based on data statistics or actual data.

[0072] Assuming that the first source is within the distance limit, and the sound source signal of the sound source within the distance limit is the sound signal to be retained (for subsequent sound signal processing or applications such as recording or calling). If the first source is within the distance limit (i.e., the distance of the first source relative to the microphone array 110 is less than or equal to the distance limit), the processor 130 retains the estimated sound source signal of the first source. Figure 4 For example, the processor 130 retains the estimated sound source signal y1.

[0073] In one embodiment, in response to the distance of another source other than the first source among the multiple sound sources relative to the microphone array 110 being within the distance limit, the processor 130 may retain the estimated sound source signal corresponding to the other source (for subsequent sound signal processing or applications such as recording or calling). In other words, more than one sound source is within the distance limit. In this case, the estimated sound source signals of these sound sources within the distance limit may all be retained.

[0074] In response to the fact that the distance of the other source relative to the microphone array 110 is not within the distance limit, the processor 130 may remove the estimated sound source signal corresponding to the other source. In other words, the other source is outside the distance limit. Therefore, the processor 130 may filter out, ignore or delete the estimated sound source signal corresponding to the other source (regarded as an interference signal).

[0075] In other embodiments, the processor 130 may retain only the estimated sound source signal of the closest sound source within the distance limit. Assume that the primary user is closest to the sound signal processing device 100. That is, even if more than one sound source is within the distance limit, only the estimated sound source signal of the closest sound source is retained, but the estimated sound source signals of other sound sources are removed.

[0076] For example, Figure 7 is a schematic diagram illustrating the positional relationship according to an embodiment of the present invention. Figure 7 , assuming the distance limit is 50 cm. Sound sources S1, S3, S4, S5, and S6 are within the distance limit, but the radiation pattern RP only covers the sound source S1. Therefore, only the estimated sound source signal of the sound source S1 is retained. Even if the radiation pattern RP further covers the sound source S2, but the sound source S2 is outside the distance limit, the estimated sound source signal of the sound source S2 can still be removed.

[0077] In summary, in the sound processing method and the sound signal processing device of the embodiment of the present invention, the main sound signal corresponding to the radiation field pattern is combined through beamforming, the primary extraction signal corresponding to the main sound signal is extracted from the input sound signals of different microphones, the estimated sound source signals of different sound sources are separated from the primary extraction signal, and the re-extracted signals corresponding to different sound sources are extracted from the primary extraction signal. These re-extracted signals can be used to estimate the distance of the sound source relative to the microphone array, and this distance can be used to screen the estimated sound source signals to be retained. In this way, only the sound signal of the main user can be received, and the recording or call quality can be improved accordingly.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A sound processing method, characterized in that: include: receiving a plurality of input sound signals through a microphone array, wherein the microphone array comprises a plurality of microphones for respectively receiving the input sound signals; Combining the input sound signals into a main sound signal based on beamforming, wherein the beamforming is used to form the microphone array into a radiation pattern corresponding to a first source among a plurality of sound sources, and the first microphone in the microphone array is closer to the first source than other microphones; Extracting a plurality of primary extraction signals belonging to the main sound signal from the input sound signal respectively, wherein a plurality of first sound feature associations between the input sound signal and the main sound signal are trained in advance, and the first sound feature associations are used to extract a portion belonging to the main sound signal; Separating a plurality of estimated sound source signals corresponding to the sound source from the primary extracted signal by blind signal separation; Extracting a plurality of re-extracted signals belonging to the estimated sound source signal from the initial extraction signal respectively, wherein a plurality of second sound feature associations between the initial extraction signal and the estimated sound source signal are trained in advance, and the second sound feature associations are respectively used to extract parts belonging to the estimated sound source signal; determining a distance of the first source relative to the microphone array according to a power ratio, wherein the power ratio is a ratio of a first value to a second value, the first value being the sum of powers of the re-extracted signals corresponding to the first source and corresponding to other microphones in the microphone array that are not the first microphone, and the second value being the powers of the re-extracted signals corresponding to the first source and corresponding to the first microphone, the smaller the power ratio is, the closer the distance of the first source is relative to the microphone array, and the larger the power ratio is, the farther the distance of the first source is relative to the microphone array; and The estimated sound source signal corresponding to the first source is determined according to a distance limit, wherein the distance of the first source relative to the microphone array is within the distance limit.

2. The sound collection processing method according to claim 1, wherein the step of determining the distance of the first source relative to the microphone array according to the power ratio comprises: The distance of the first source relative to the microphone array is determined according to a distance relationship ratio, wherein the distance relationship ratio is the ratio of a third value to a fourth value, the third value is a distance coefficient, the fourth value is a difference between one and a fifth value, and the fifth value is a product of an adjustment coefficient and the power ratio.

3. The sound processing method according to claim 1, further comprising: In response to the distance of another source among the sound sources that is not the first source relative to the microphone array being within the distance limit, retaining the estimated sound source signal corresponding to the another source; as well as In response to the distance of the other source relative to the microphone array not being within the distance limit, the estimated sound source signal corresponding to the other source is removed.

4. The sound processing method according to claim 1, further comprising: Training the first sound feature associations between the main sound signal and a plurality of known input signals corresponding to the microphones of the microphone array respectively by a machine learning algorithm, wherein each of the first sound feature associations comprises a weight coefficient of an extraction model corresponding to the microphone; as well as The parts of the input sound signal belonging to the main sound signal are determined respectively by the extraction models corresponding to the microphones.

5. The sound processing method according to claim 4, wherein the machine learning algorithm is a back-propagation multi-layer perceptron, and the step of training the first sound feature association comprises: defining a loss function, wherein the loss function is based on a difference between a portion of the known input signal of each microphone estimated to belong to the main sound signal based on the first sound feature association and the main sound signal; and The loss function is minimized to determine the weight coefficients of the extraction model.

6. A sound signal processing device, characterized in that: include: A microphone array, comprising a plurality of microphones for respectively receiving a plurality of input sound signals; A memory for storing program codes; as well as A processor is coupled to the microphone array and the memory, and is used to load the program code to execute: Combining the input sound signals into a main sound signal based on beamforming, wherein the beamforming is used to form the microphone array into a radiation pattern corresponding to a first source among a plurality of sound sources, and the first microphone in the microphone array is closer to the first source than other microphones; Extracting a plurality of primary extraction signals belonging to the main sound signal from the input sound signal respectively, wherein a plurality of first sound feature associations between the input sound signal and the main sound signal are trained in advance, and the first sound feature associations are used to extract a portion belonging to the main sound signal; Separating a plurality of estimated sound source signals corresponding to the sound source from the primary extracted signal by blind signal separation; Extracting a plurality of re-extracted signals belonging to the estimated sound source signal from the initial extraction signal respectively, wherein a plurality of second sound feature associations between the initial extraction signal and the estimated sound source signal are trained in advance, and the second sound feature associations are respectively used to extract parts belonging to the estimated sound source signal; determining a distance of the first source relative to the microphone array according to a power ratio, wherein the power ratio is a ratio of a first value to a second value, the first value being the sum of powers of the re-extracted signals corresponding to the first source and corresponding to other microphones in the microphone array that are not the first microphone, and the second value being the powers of the re-extracted signals corresponding to the first source and corresponding to the first microphone, the smaller the power ratio is, the closer the distance of the first source is relative to the microphone array, and the larger the power ratio is, the farther the distance of the first source is relative to the microphone array; and The estimated sound source signal corresponding to the first source is determined according to a distance limit, wherein the distance of the first source relative to the microphone array is within the distance limit.

7. The sound signal processing device according to claim 6, wherein the processor is further configured to: The distance of the first source relative to the microphone array is determined according to a distance relationship ratio, wherein the distance relationship ratio is the ratio of a third value to a fourth value, the third value is a distance coefficient, the fourth value is a difference between one and a fifth value, and the fifth value is a product of an adjustment coefficient and the power ratio.

8. The sound signal processing device according to claim 6, wherein the processor is further configured to: In response to a distance of another source among the sound sources that is not the first source relative to the microphone array being within the distance limit, retaining the estimated sound source signal corresponding to the another source; and In response to the distance of the other source relative to the microphone array not being within the distance limit, the estimated sound source signal corresponding to the other source is removed.

9. The sound signal processing device according to claim 6, wherein the processor is further configured to: Training the first sound feature associations between the main sound signal and a plurality of known input signals corresponding to the microphones of the microphone array respectively by a machine learning algorithm, wherein each of the first sound feature associations comprises a weight coefficient of an extraction model corresponding to the microphone; and The parts of the input sound signal belonging to the main sound signal are determined respectively by the extraction models corresponding to the microphones.

10. The sound signal processing device according to claim 9, wherein the machine learning algorithm is a back-propagation multi-layer perceptron, and the processor is further configured to: defining a loss function, wherein the loss function is based on a difference between a portion of the known input signal of each microphone estimated to belong to the main sound signal based on the first sound feature association and the main sound signal; and The loss function is minimized to determine the weight coefficients of the extraction model.

Images