Sound signal processing method, device, electronic device and storage medium

Abstract

The present invention discloses a sound signal processing method, device, electronic device and storage medium. The present invention adds a signal quality evaluation mechanism in signal separation, performs sound source separation processing on sound source data to be processed, obtains candidate sound sources corresponding to the sound source data to be processed and sound signals belonging to each candidate sound source in the sound source data to be processed; performs quality evaluation on the sound signal of each candidate sound source, determines the evaluation value of the sound signal of each candidate sound source; determines a target sound source from multiple candidate target sound sources according to the evaluation value of the sound signal of each candidate sound source; processes the sound signal of the target sound source, thus, by performing quality evaluation on the sound signal of each candidate sound source, obtains the evaluation value of each candidate sound source, determines the final target sound source, improves the accuracy of the separated sound source signal, and improves the problem of low stability of signal separation.

Description

Technical Field

[0001] The present invention relates to the technical field of signal processing, and in particular to a sound signal processing method, device, electronic equipment and storage medium. Background Art

[0002] When electronic devices collect sound signals through multiple microphone channels, these collected signals often contain interference signals that are not sound sources, such as environmental noise, interference-related source signals, and reflected signals in the environment. Due to the various complex characteristics of the signal during transmission, the interference signal is mixed with the sound source signal, making it difficult to extract the sound source signal. Blind source separation is an effective way to solve this problem. Its purpose is to extract the sound source signal from the complex mixed signal.

[0003] Although the existing blind source separation methods can separate the sound source signals from the complex mixed sound signals, they cannot distinguish whether the separated multiple sound source signals are valid and whether the quality meets the requirements, which makes the accuracy of the separated sound source signals low and the stability of blind source separation low. Summary of the invention

[0004] The embodiments of the present invention provide a sound signal processing method, device, electronic device and storage medium, which can improve the stability of signal separation.

[0005] An embodiment of the present invention provides a sound signal processing method, comprising:

[0006] Performing sound source separation processing on the sound source data to be processed to obtain candidate sound sources corresponding to the sound source data to be processed and sound signals belonging to each of the candidate sound sources in the sound source data to be processed;

[0007] Performing a quality assessment on the sound signal of each of the candidate sound sources to determine an assessment value of the sound signal of each of the candidate sound sources;

[0008] Determine a target sound source from a plurality of candidate target sound sources according to an evaluation value of a sound signal of each candidate sound source;

[0009] The sound signal of the target sound source is processed.

[0010] Accordingly, an embodiment of the present invention further provides a sound signal processing device, comprising:

[0011] A separation module, configured to perform sound source separation processing on the sound source data to be processed, to obtain candidate sound sources corresponding to the sound source data to be processed and sound signals belonging to each of the candidate sound sources in the sound source data to be processed;

[0012] An evaluation module, used to perform quality evaluation on the sound signal of each candidate sound source and determine an evaluation value of the sound signal of each candidate sound source;

[0013] A selection module, configured to determine a target sound source from a plurality of candidate target sound sources according to an evaluation value corresponding to a sound signal of each candidate sound source;

[0014] A processing module is used to process the sound signal of the target sound source.

[0015] Accordingly, an embodiment of the present invention further provides an electronic device, including:

[0016] A memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to perform the operations in the sound signal processing method.

[0017] In addition, an embodiment of the present invention further provides a storage medium, wherein the storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor to execute the steps in the sound signal processing method.

[0018] The embodiment of the present invention adds a signal quality evaluation mechanism in signal separation, performs sound source separation processing on the sound source data to be processed, obtains candidate sound sources corresponding to the sound source data to be processed and sound signals belonging to each candidate sound source in the sound source data to be processed; performs quality evaluation on the sound signal of each candidate sound source, determines the evaluation value of the sound signal of each candidate sound source; determines a target sound source from multiple candidate sound sources according to the evaluation value of the sound signal of each candidate sound source; and processes the sound signal of the target sound source. In this way, by adding a signal quality evaluation mechanism in signal separation, the sound signal quality of each candidate sound source is evaluated according to the evaluation value of each candidate sound source, so as to select an effective target sound source, improve the accuracy of the separated sound source signal, and improve the problem of low stability of signal separation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 is a flow chart of a sound signal processing method provided by an embodiment of the present invention;

[0021] Figure 2It is a flow chart of a sound source separation process in the sound signal processing method provided by an embodiment of the present invention;

[0022] Figure 3 is a flow chart of a method for estimating a candidate sound source in a sound signal processing method provided by an embodiment of the present invention;

[0023] Figure 4 It is a flowchart of another sound source separation process in the sound signal processing method provided by an embodiment of the present invention;

[0024] Figure 5 is a structural schematic diagram of a sound signal processing device provided by an embodiment of the present invention;

[0025] Figure 6 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0027] As described in the background technology, when the prior art performs blind source separation through AuxICA (full name: Auxiliary Function Based Independent Component Analysis, Chinese: independent component analysis method based on auxiliary function) and AuxIVA (full name: Auxiliary Function Based Independent Vector Analysis, independent vector analysis method based on auxiliary function), the sound quality of the separated target sound source is not evaluated and screened, and the accuracy of the target sound source cannot be guaranteed, thereby reducing the quality of the separated sound signal, making the stability of blind source separation low.

[0028] Based on this, in order to improve the stability of the blind source separation output and achieve the effect of noise reduction, an embodiment of the present invention provides a sound signal processing method, which determines the final target sound source through the evaluation values ​​of each candidate sound source, improves the accuracy of the separated sound source signal, improves the problem of low stability of signal separation, and improves the audio-visual effect.

[0029] Please refer to Figure 1 , Figure 1It is a flow chart of the sound signal processing method provided in an embodiment of the present invention. The sound signal processing method shown can be applied to electronic devices. In some embodiments of the present invention, the electronic device can be a mobile terminal, such as a mobile phone, a tablet computer, a computer, and a television; in some embodiments of the present invention, the electronic device can also be a voice device, such as a Bluetooth speaker, a smart speaker, a microphone, a smart home, etc.

[0030] The sound signal processing method shown includes steps 101 to 104:

[0031] 101 , performing sound source separation processing on the sound source data to be processed to obtain candidate sound sources in the sound source data to be processed and sound signals belonging to each candidate sound source in the sound source data to be processed.

[0032] The sound source data to be processed refers to the speech signal in the current environment collected by the electronic device, which includes the speech signal of the sound source and the noise in the environment. The candidate sound source refers to the sound source that may exist in the current environment estimated based on the sound source data to be processed, which includes the target sound source.

[0033] In some embodiments of the present invention, the sound source data to be processed may be a voice signal in the current environment collected in real time; or may be a voice signal in the current environment collected within a preset time period.

[0034] A microphone array is provided in an electronic device, and the electronic device collects voice signals in the current environment where the electronic device is located through the microphone array. Since the distance between the sound source and each microphone channel in the microphone array is different, each microphone channel in the microphone array may receive the sound signal of the sound source, and the reverberation of the room, the interference of other sound sources, the noise in the environment, and the noise inside the device inevitably reduce the quality of the voice signal and the clarity of the language. However, the current speech recognition technology cannot completely have high sensitivity and robustness like human hearing, and can distinguish various sound sources and eliminate interference. Therefore, these interferences cause noise in the sound source data to be processed. If the sound source data to be processed is directly used for output, the audio-visual effect is affected and the performance of the electronic device that uses voice as an interactive mode is reduced. Therefore, it is necessary to perform sound source separation processing on the sound source data to be processed to determine the sound source in the environment. Among them, the microphone array can be a ring microphone array, a linear microphone array, or a distributed microphone array, wherein the microphone array includes at least one microphone channel.

[0035] In some embodiments of the present invention, there are multiple ways to perform sound source separation processing on the sound source data to be processed, and exemplary methods include:

[0036] (1) The features of different sound sources can be separated from the sound source data to be processed by a sound source separation method based on a deep neural network, and candidate sound sources in the sound source data to be processed and the sound signal of each candidate sound source can be obtained according to the features of the separated different sound sources. The sound source separation method based on a deep neural network includes but is not limited to a sound source separation method based on deep clustering, a sound source separation method based on permutation invariant training, and an end-to-end sound source separation method.

[0037] (2) The sound source data to be processed can be blindly separated by a separation method based on independent subspace analysis to obtain candidate sound sources in the sound source data to be processed and the sound signal of each candidate sound source.

[0038] (3) The sound source data to be processed can be blindly separated by a separation method based on non-negative matrix decomposition to obtain candidate sound sources in the sound source data to be processed and the sound signal of each candidate sound source.

[0039] (4) The sound source data to be processed can be blindly separated by a clustering-based separation method to obtain candidate sound sources in the sound source data to be processed and the sound signal of each candidate sound source. For example, the sound source data to be processed can be clustered by a Gaussian mixture model to obtain candidate sound sources in the sound source data to be processed and the sound signal of each candidate sound source.

[0040] (5) Principal component analysis can be used to perform blind source separation on the sound source data to be processed, thereby obtaining candidate sound sources in the sound source data to be processed and the sound signal of each candidate sound source.

[0041] (6) A separation method based on independent component analysis can be used to perform blind source separation on the sound source data to be processed by analyzing the independent statistical characteristics of the signals in the sound source data to be processed, thereby obtaining candidate sound sources in the sound source data to be processed and the sound signal of each candidate sound source.

[0042] (7) The sound source data to be processed can be blindly separated based on the separation method of independent vector analysis to obtain candidate sound sources in the sound source data to be processed and the sound signal of each candidate sound source.

[0043] (8) The sound source data to be processed can be blindly separated by using an independent vector analysis separation method based on auxiliary function optimization to obtain candidate sound sources in the sound source data to be processed and the sound signal of each candidate sound source.

[0044] It should be noted that the above-mentioned sound source separation processing method is only an exemplary description and does not constitute a limitation of the sound signal processing method provided in the embodiment of the present invention. For example, the sound source data to be processed can also be subjected to sound source separation processing by using a separation method based on overdetermined independent vector analysis optimized by auxiliary functions to obtain candidate sound sources in the sound source data to be processed and the sound signal of each candidate sound source.

[0045] 102 , performing quality evaluation on the sound signal of each candidate sound source to determine an evaluation value of the sound signal of each candidate sound source.

[0046] The evaluation value represents the sound quality of the sound signal of each candidate sound source and is used to quantify the probability that each candidate sound source is the target sound source.

[0047] In some embodiments of the present invention, there are multiple ways to evaluate the quality of the sound signal of each candidate sound source, exemplary of which include:

[0048] (1) The quality of the sound signal of each candidate sound source can be evaluated by calculating the signal-to-interference ratio of the sound signal of each candidate sound source, thereby determining an evaluation value of the sound signal of each candidate sound source.

[0049] (2) The quality of the sound signal of each candidate sound source can be evaluated by calculating the signal-distortion ratio of the sound signal of each candidate sound source, thereby determining an evaluation value of the sound signal of each candidate sound source.

[0050] (3) The quality of the sound signal of each candidate sound source can be evaluated by calculating the maximum likelihood ratio of the sound signal of each candidate sound source to determine the evaluation value of the sound signal of each candidate sound source.

[0051] (4) The quality of the sound signal of each candidate sound source can be evaluated by calculating the cepstrum clustering of the sound signal of each candidate sound source, and the evaluation value of the sound signal of each candidate sound source can be determined.

[0052] (5) The quality of the sound signal of each candidate sound source can be evaluated by calculating the frequency-weighted segmented signal-to-noise ratio of the sound signal of each candidate sound source, and an evaluation value of the sound signal of each candidate sound source can be determined.

[0053] (6) The quality of the sound signal of each candidate sound source can be evaluated by calculating the speech quality perception evaluation score of the sound signal of each candidate sound source, and the evaluation value of the sound signal of each candidate sound source can be determined.

[0054] (7) The quality of the sound signal of each candidate sound source can be evaluated by calculating the kurtosis value of the sound signal of each candidate sound source to determine the evaluation value of the sound signal of each candidate sound source.

[0055] (8) The quality of the sound signal of each candidate sound source can be evaluated by calculating the probability score corresponding to the speech feature vector of the sound signal of each candidate sound source, and determining the evaluation value of the sound signal of each candidate sound source. The probability score is used to represent the probability that the sound signal of each candidate sound source is the speech signal of the target sound source.

[0056] It should be noted that the above method for evaluating the quality of the sound signal of each candidate sound source is only an exemplary description and does not constitute a limitation on the sound signal processing method provided in the embodiment of the present invention. In practical applications, the corresponding evaluation value determination method can be selected according to the computing efficiency of the electronic device in the actual application scenario.

[0057] 103 , determining a target sound source from a plurality of candidate target sound sources according to the evaluation value of the sound signal of each candidate sound source.

[0058] In some embodiments of the present invention, step 103 includes: selecting a candidate sound source corresponding to a maximum evaluation value according to the evaluation value of the sound signal of each candidate sound source, and setting the selected candidate sound source corresponding to the maximum evaluation value as the target sound source.

[0059] In some embodiments of the present invention, step 103 includes: obtaining statistical features of the evaluation values ​​according to the evaluation values ​​of the sound signals of each candidate sound source, and determining a target sound source from a plurality of candidate target sound sources according to the statistical features of the evaluation values, wherein the statistical features include the median or mode of the evaluation values. Specifically, obtaining the median or mode of the evaluation values ​​according to the evaluation values ​​of the sound signals of each candidate sound source, selecting an evaluation value greater than the median or an evaluation value greater than the mode from a plurality of candidate target sound sources according to the median or mode of the evaluation values, setting the selected candidate sound source corresponding to the evaluation value greater than the median as the target sound source, or setting the evaluation value greater than the mode as the target sound source.

[0060] 104, processing the sound signal of the target sound source.

[0061] Among them, processing the sound signal of the target sound source includes but is not limited to voice output, voice recognition, voice transmission, voice storage, etc.

[0062] In some embodiments of the present invention, when the electronic device is an electronic device that uses voice as an interactive mode, step 104 includes: obtaining a sound signal corresponding to the target sound source, performing semantic analysis on the sound signal corresponding to the target sound source, obtaining voice information in the sound signal corresponding to the target sound source, and the electronic device responding to an instruction corresponding to the voice information and executing an operation corresponding to the instruction, such as conducting a conversation interaction, executing a query operation, executing a music playback operation, etc.

[0063] In some embodiments of the present invention, when the electronic device is a voice collection device, step 104 includes: obtaining a sound signal corresponding to the target sound source, storing the voice signal, such as a sound receiving device, or transmitting the voice signal to a server that is communicatively connected to the electronic device.

[0064] The embodiment of the present invention provides a sound signal processing method, which determines the final target sound source through the evaluation value of each candidate sound source, improves the accuracy of the separated sound source signal, and improves the problem of low stability of signal separation.

[0065] Considering that the existing blind source separation method cannot utilize the location information of the sound source, when the location of the sound source changes, the existing blind source separation method cannot accurately detect the change of the location of the sound source, so the target sound source separated by the existing blind source separation method has a certain uncertainty, which leads to the instability of the sound signal of the separated target sound source. Therefore, in order to further improve the certainty of the output result obtained by blind source separation, reduce the noise in the output voice signal, and achieve the effect of noise reduction, in some embodiments of the present invention, in the sound source separation process of step 101, the sound source is estimated for the sound source data to be processed to obtain the sound source prior information in the sound source data to be processed, and the sound source separation process is performed based on the sound source prior information to improve the accuracy of the candidate sound source obtained by blind source separation, thereby ensuring the accuracy of the final target sound source. Wherein, the sound source prior information refers to the location information of the candidate sound source that may exist in the environment where the electronic device that collects the sound source data to be processed is located. Wherein the location information of the candidate sound source can be the spatial coordinate value of the candidate sound source, or it can be the pitch angle and azimuth angle of the candidate sound source in the space of the environment where the electronic device that collects the sound source data to be processed is located. It should be noted that the embodiment of the present invention does not limit the way of establishing the spatial coordinate system. For example, the spatial coordinate system can be established with the geometric center of the electronic device as the origin.

[0066] like Figure 2 As shown, Figure 2 2 is a flow chart of a sound source separation process in a sound signal processing method provided by an embodiment of the present invention. The sound source separation process method shown includes steps 201 to 203:

[0067] 201 , estimate the sound source position of the sound source data to be processed, and determine the candidate sound sources corresponding to the sound source data to be processed and the position information of each candidate sound source.

[0068] In some embodiments of the present invention, the sound source data to be processed can be estimated by the SPR (full name: Steered Response Power, Chinese: Controllable Response Power) method to obtain the candidate sound sources corresponding to the sound source data to be processed and the position information of each candidate sound source. Specifically, it includes: estimating the power spectrum distribution of the sound source data to be processed in space by the SPR method, and determining the candidate sound sources corresponding to the sound source data to be processed and the position information of each candidate sound source according to the power spectrum distribution.

[0069] In some embodiments of the present invention, the position of the maximum power can be determined according to the power spectrum distribution, and the selected position of the maximum power is set as the position information of the candidate sound source. In some embodiments of the present invention, multiple positions whose power is greater than or equal to the preset power value can also be determined according to the power spectrum distribution, and the selected multiple positions whose power is greater than the preset power value can be set as the position information of the candidate sound source. Among them, the preset power value can be the average power value in the power spectrum distribution, or it can be sorting the power of each position in space in order from large to small according to the power spectrum distribution, and setting the power ranked Sth in the sorted power to the preset power value. Among them, S is an integer greater than 0, and the value of S can be set according to the actual application scenario. For example, the value of S can be 2, 3, 4, 5, etc.

[0070] In some embodiments of the present invention, considering that when estimating the power spectrum of the sound source data to be processed by the SPR method, the high-frequency part is prone to aliasing. Since the aliasing phenomenon is prone to occur, the accuracy of the estimated position information of the candidate sound source is affected. Based on this, when estimating the power spectrum of the sound source data to be processed by the SPR method, the embodiment of the present invention performs sound source estimation on the low-frequency part of the sound source data to be processed by SPR to obtain the estimated area where the candidate sound source is located, and performs sound source estimation on the high-frequency part of the sound source data to be processed by SPR, and selects the candidate sound source corresponding to the sound source data to be processed and the position information of each candidate sound source from the estimated area where the candidate sound source is located. Specifically, the sound source position estimation includes steps a1 to a4:

[0071] Step a1, performing frequency domain conversion on the sound source data to be processed to obtain a frequency domain signal of the sound source data to be processed.

[0072] Step a2, filtering the frequency domain signal of the sound source data to be processed by a filter to obtain a low-frequency signal and a high-frequency signal of the frequency signal. The filter can be a low-pass filter or a high-pass filter.

[0073] Step a3, according to the low-frequency signal of the frequency signal, the time delay of the low-frequency signal of each microphone channel in the microphone array set in the electronic device is estimated by the SPR method, and the controllable response power function value of the microphone array in each preset area is obtained, and multiple preset areas whose controllable response power function values ​​are greater than or equal to the preset function value threshold are selected, and the selected preset areas are set as the estimation areas where the candidate sound sources are located.

[0074] Step a4, according to the high-frequency signal of the frequency signal, the time delay of the high-frequency signal of each microphone channel in the microphone array set in the electronic device is estimated by the SPR method, and the controllable response power function value of the microphone array in each estimation area is obtained, and the estimation area whose controllable response power function value is greater than or equal to the preset function value threshold is selected, and each selected estimation area whose controllable response power function value is greater than or equal to the preset function value threshold is set as the position of each candidate sound source, and the position information of each selected estimation area whose controllable response power function value is greater than or equal to the preset function value threshold is set as the position information of each candidate sound source.

[0075] In some embodiments of the present invention, steps a3 to a4 include: dividing the spatial coordinate system into a plurality of first grid areas, wherein each first grid area corresponds to a position information consisting of a pitch angle and an azimuth angle; performing a time delay estimation on a low-frequency signal of each microphone channel in a microphone array provided in an electronic device by an SPR method, and obtaining a first controllable response power function value of the microphone array in each first grid area; selecting a first grid area having a maximum first controllable response power function value, and setting the selected first grid area as an estimation area where a candidate sound source is located; dividing the estimation area where the candidate sound source is located into a plurality of second grid areas, wherein each second grid area corresponds to a position information consisting of a pitch angle and an azimuth angle, and the angle between every two adjacent second grid areas is The angle difference is less than the angle difference between every two adjacent first grid areas; the high-frequency signal of each microphone channel in the microphone array provided in the electronic device is estimated by the SPR method to obtain the second controllable response power function value of the microphone array in each second grid area, and according to the second controllable response power function value of the microphone array in each second grid area, the second grid area whose second controllable response power function value is greater than or equal to the preset function value threshold is selected, and each selected second grid area whose second controllable response power function value is greater than or equal to the preset function value threshold is set as the position of each candidate sound source, and the position information of each selected second grid area whose second controllable response power function value is greater than or equal to the preset function value threshold is set as the position information of each candidate sound source.

[0076] 202 , according to the positions of each sound channel of the sound source data to be processed and the position information of each candidate sound source, determine and obtain the position guidance information of each candidate sound source.

[0077] The sound channel position for collecting the sound source data to be processed refers to the position information of each microphone channel in the microphone array provided in the electronic device for collecting the sound source data to be processed in a preset spatial coordinate system.

[0078] The position guidance information is used to determine the position vector between the position information of each candidate sound source and the position of each sound channel for collecting the sound source data to be processed.

[0079] In some embodiments of the present invention, since the existing blind source separation method does not consider the sound source position, which results in noise in the separated sound signal, in order to improve the stability of blind source separation, in blind source separation, the position information of each candidate sound source is obtained according to the sound source estimation, and the position vector between the position information of each candidate sound source and the position of each sound channel of the sound source data to be processed is determined. In blind source separation, the sound source data to be processed is separated by the position vector between the position information of each candidate sound source and the position of each sound channel of the sound source data to be processed, and the sound signal at each sound channel position of the sound source data to be processed is collected to obtain the sound signal component of each candidate sound source in the sound signal at each sound channel position in the sound source data to be processed, and the sound signal of each candidate sound source is obtained by collecting the sound signal component at each candidate sound source position in the sound signal at each sound channel position in the sound source data to be processed.

[0080] In some embodiments of the present invention, the distance between the position information of each candidate sound source and each sound channel position of the sound source data to be processed can be obtained based on the sound channel position of the sound source data to be processed and the position information of each candidate sound source. The angle information between the position information of each candidate sound source and each sound channel position of the sound source data to be processed can be obtained based on the distance between the position information of each candidate sound source and each sound channel position of the sound source data to be processed, and the angle information between the position information of each candidate sound source and each sound channel position of the sound source data to be processed can be obtained. The position guidance information of each candidate sound source can be obtained based on the angle information between the position information of each candidate sound source and each sound channel position of the sound source data to be processed. For example, the angle information between the position information of each candidate sound source and each sound channel position of the sound source data to be processed can be obtained by The positional orientation information of each candidate sound source is obtained. Wherein, θ represents the angle information between the position information of each candidate sound source and the position of each sound channel for collecting the sound source data to be processed, and M represents the number of sound channels for collecting the sound source data to be processed.

[0081] In some embodiments of the present invention, the distance between the position information of each candidate sound source and each sound channel position of the sound source data to be processed can be obtained based on the sound channel position of the sound source data to be processed and the position information of each candidate sound source. The time information required for the sound signal to reach each sound channel position of the sound source data to be processed from the position information of each candidate sound source can be obtained based on the distance between the position information of each candidate sound source and each sound channel position of the sound source data to be processed. The position guidance information of each candidate sound source can be obtained based on the time information required for the sound signal to reach each sound channel position of the sound source data to be processed from the position information of each candidate sound source. For example, the time information required for the sound signal to reach each sound channel position of the sound source data to be processed from the position information of each candidate sound source can be obtained based on the time information required for the sound signal to reach each sound channel position of the sound source data to be processed. The position guidance information of each candidate sound source is obtained, where τ represents the time information required for the sound signal to reach the position of each sound channel for collecting the sound source data to be processed from the position information of each candidate sound source, and j represents a complex number.

[0082] 203 , performing sound source separation on the sound source data to be processed according to the position guidance information of each candidate sound source to obtain a sound signal of each candidate sound source.

[0083] In some embodiments of the present invention, the sound source data to be processed can be separated according to the positional orientation information of each candidate sound source according to the separation method in step 101 to obtain the sound signal of each candidate sound source. For example, the separation method based on overdetermined independent vector analysis optimized by auxiliary functions is used as an example for description.

[0084] Among them, in the separation method based on overdetermined independent vector analysis optimized by auxiliary function, in the sound source separation of the processed sound source data, it is assumed that the received sound source data is composed of N transmitting end source signals S in the environment 1 ,S 2 ,...,S N Through the transfer function h mn The mixed signal x received by M sound channels after mixing 1 ,x 2 ,...,x M , that is, the sound source data is expressed as Among them, the N transmitting end source signals are N sound source signals in the environment, and the sound source data is converted into the frequency domain by performing short-time Fourier transform on the sound source data to obtain the frequency domain signal of the sound source data X(l,k)=H(k)S(l,k), l=1,...,L, where L is the number of frames of short-time Fourier transform, S(l,k)=[S 1 (l,k),...,S N (l,k)] Trepresents the transmitting source signal at frequency point k, X(l,k)=[X 1 (l,k),...,X M (l,k)] T is the frequency domain signal of the received mixed signal, H(k) is the mixing matrix, and according to the frequency domain signal of the sound source data X(l,k)=H(k)S(l,k), l=1,...,L, the separated signal at each frequency point can be expressed as Y(l,k)=W(k)X(l,k), where Y(l,k)=[Y 1 (l,k),...,Y N (l,k)] T is the separation signal at frequency point k, which is approximately N transmitting end source signals, W(k) is the separation parameter at frequency point k, and the separation parameters of each frame of frequency domain signal in the frequency domain signal of the sound source data at each frequency point are solved by optimizing based on the auxiliary function, and the frequency domain signals of each frame of frequency domain signal in the frequency domain signal of the sound source data at each frequency point are separated from the frequency signals of each frame of frequency domain signal in the frequency domain signal of the sound source data to be processed at each frequency point through the separation parameters, and the sound signal of the candidate sound source is obtained through inverse short-time Fourier transform.

[0085] In some embodiments of the present invention, in order to improve the reliability of the sound signal of the candidate sound source obtained by separation and reduce the noise in the separated candidate sound source signal, in the sound source separation of the sound source data by the separation method of overdetermined independent vector analysis based on auxiliary function optimization, the separation parameters of each frame of the frequency domain signal of the sound source data at each frequency point are solved by the guidance information combined with the auxiliary function optimization, and the sound signal of the candidate sound source is separated from the frequency signal of each frame of the frequency domain signal of the sound source data at each frequency point by the separation parameters and the frequency signal of each frame of the frequency domain signal of the sound source data at each frequency point. Specifically, the separation method includes steps b1 to b2:

[0086] Step b1, determining separation parameters for the sound source data to be processed according to the position guidance information of each candidate sound source.

[0087] The sound source data to be processed may be a current frame frequency domain signal in the frequency domain signal of the sound source data.

[0088] In some embodiments of the present invention, step b1 includes: after obtaining the auxiliary parameters corresponding to the sound source data to be processed, the auxiliary parameters corresponding to the sound source data to be processed are corrected according to the position guidance information of each candidate sound source to obtain the corrected auxiliary parameters corresponding to the sound source data to be processed, and the separation parameters of the sound source data to be processed are optimized and solved based on the corrected auxiliary parameters. The auxiliary parameters include auxiliary parameters in the frequency signal of each frame of the frequency domain signal of the sound source data to be processed at each frequency point. Specifically, the method for determining the separation parameters of the sound source data to be processed according to the position guidance information includes:

[0089] (1) Obtain historical separation parameters of historical sound source data and auxiliary parameters corresponding to the sound source data to be processed.

[0090] (2) According to the position guidance information of each candidate sound source, the auxiliary parameters are corrected to obtain corrected auxiliary parameters.

[0091] (3) According to the corrected auxiliary parameters and the historical separation parameters, the separation parameters of the sound source data to be processed are obtained.

[0092] Among them, the historical separation parameters of the historical sound source data refer to the separation parameters of the previous frame of sound source data of the sound source data to be processed. Since the sound signal of the sound source is correlated in time series, the separation method based on the overdetermined independent vector analysis optimized by the auxiliary function performs sound source separation on the sound source data to be processed by alternately updating the separation parameters and the auxiliary parameters, wherein the update of the auxiliary parameters is achieved by updating the separation parameters of the previous frame of sound source data through the frequency domain signal of the sound source data to be processed; the update of the separation parameters is achieved by updating the separation parameters of the previous frame of sound source data through the auxiliary parameters of the frequency domain signal of the sound source data to be processed.

[0093] In some embodiments of the present invention, the step of obtaining auxiliary parameters corresponding to the sound source data to be processed includes: obtaining historical separation parameters of historical sound source data and historical auxiliary parameters of discrete sound source data, obtaining the energy of each candidate sound source outputted by the previous frame of sound source data according to the historical separation parameters and the sound signal of the sound source data to be processed, and obtaining the auxiliary parameters corresponding to the sound source data to be processed according to the historical auxiliary parameters, the sound signal of the sound source data to be processed and the energy of each candidate sound source outputted by the previous frame of sound source data. The historical auxiliary parameters refer to the auxiliary parameters of the previous frame of sound source data. For example, when the number S of separated candidate sound sources is less than or equal to the number M of sound channels of the sound source data to be processed, it can be obtained by Get auxiliary parameters of the sound source data to be processed

[0094] V(l,k)=[V 1 (l,k),V 2 (l,k),...,Vs (l,k),...,V S (l,k)], where α is the forgetting factor, and α∈[0,1], l is the number of frames of the sound source data to be processed, V s (l-1,k) is the auxiliary parameter of the kth frequency point in the previous frame of sound source data, that is, the historical auxiliary parameter. is the energy of each candidate sound source output, W s (l-1,k) is the separation parameter of the kth frequency point in the previous frame of sound source data, that is, the historical separation parameter, (·) H represents the conjugate transpose, where the first frame processes the auxiliary parameters V of the sound source data s (1, k) is a matrix with the values ​​of the diagonal elements preset to 1 and the values ​​of the elements at other positions to zero.

[0095] In some embodiments of the present invention, modifying the auxiliary parameter matrix according to the position guidance information of each candidate sound source includes: calculating the auxiliary parameter V of each candidate sound source in the auxiliary parameter according to the position guidance information of each candidate sound source s (l,k) corresponding correction parameters, V s (l, k) and the correction parameter for each candidate sound source auxiliary parameter V s (l,k) is modified to obtain the modified auxiliary parameters. Among them, the position guidance information of each candidate sound source can be pass Get the auxiliary parameter V of each candidate sound source s (l,k) corresponds to the correction parameter β, where λ s is a preset constant. After determining the auxiliary parameter V of each candidate sound source s After the correction parameter β corresponding to (l,k) is obtained, V s (l,k)+β to obtain the auxiliary parameters D of each corrected candidate sound source s (l,k), summarize the auxiliary parameters D of each corrected candidate sound source s (l,k) to obtain the corrected auxiliary parameter D(l,k)=[D 1 (l,k),D 2 (l,k),...,D s (l,k),...,D S (l,k)].

[0096] In some embodiments of the present invention, the separation parameter W(l,k) of the sound source data to be processed can be obtained according to the corrected auxiliary parameter and the historical separation parameter through D(l,k)W(l-1,k).

[0097] In some embodiments of the present invention, the modified auxiliary parameter and the historical separation parameter can be used to obtain the value of (W(l-1, k)D s (l,k)) -1 Get the sth intermediate parameter P s (l,k), through Get the separation parameter W(l,k).

[0098] In some embodiments of the present invention, in order to increase the accuracy of the separation result and solve the ambiguity problem of blind source separation, in the separation parameter determination, the corrected auxiliary parameters and the historical separation parameters can be used to obtain the separation result by (W(l-1, k)V s (l,k)) -1 Get the sth first intermediate parameter P s (l,k), through Get the sth corrected first intermediate parameter Q s (l,k), through P s H (l,k)D s (l,k)P s (l,k) gets the sth second intermediate parameter Ψ s (l,k), through P s H (l,k)D s (l,k)Q s (l,k) gets the sth third intermediate parameter Φ s (l, k), the separation parameter W of the sth element is obtained according to the sth first intermediate parameter, the sth modified first intermediate parameter, the sth second intermediate parameter and the sth third intermediate parameter s (l,k), the separation parameters of each element are summarized to obtain the separation parameter W(l,k). Specifically, for the sth element W in the separation parameter W(l,k), s (l,k), the third intermediate parameter Φ of this element s The value of (l,k) is compared with the preset value; if the third intermediate parameter Φ of the element s If the value of (l,k) is consistent with the preset value, then according to the second intermediate parameter Ψ of the element s (l, k), the first intermediate parameter P s (l, k) and the modified first intermediate parameter Q s (l,k), through Get the sth element W in the separation parameter W(l,k) s (l,k); if the third intermediate parameter Φ of the element s If the value of (l,k) is inconsistent with the preset value, then the second intermediate parameter Ψ of the element is used. s(l,k), the third intermediate parameter Φ of the element s (l, k), the first intermediate parameter P s (l, k) and the modified first intermediate parameter Q s (l,k), through Get the sth element W in the separation parameter W(l,k) s (l,k). The preset value may be 0.

[0099] It should be noted that the above-mentioned method for determining the separation parameters is only an exemplary description of determining the separation parameters in the separation method based on overdetermined independent vector analysis with auxiliary function optimization. In practical applications, the method for determining the separation parameters can be adjusted according to the separation method adopted.

[0100] Step b2: performing sound source separation on the sound signal in the sound source data to be processed according to the separation parameters, and determining to obtain the sound signal of each candidate sound source.

[0101] In some embodiments of the present invention, step b2 includes: after obtaining the separation parameter, separating the separation signal from the sound source data to be processed by calculating the product of the separation parameter and the frequency domain signal of the sound signal in the sound source data to be processed, wherein each element in the separation signal represents the frequency domain signal of each candidate sound source, and performing an inverse short-time Fourier transform on the frequency domain signal of each candidate sound source separated to obtain the sound signal of each candidate sound source.

[0102] In some embodiments of the present invention, step b2 includes: after obtaining the separation parameters, obtaining the noise separation parameters according to the separation parameters, obtaining the total separation parameters of the sound source data to be processed through the noise separation parameters and the separation parameters, separating the separation signal from the sound source data to be processed by calculating the product of the total separation parameters and the frequency domain signal of the sound signal in the sound source data to be processed, wherein each element in the separation signal represents the frequency domain signal of each candidate sound source, and performing an inverse short-time Fourier transform on the frequency domain signal of each candidate sound source separated to obtain the sound signal of each candidate sound source.

[0103] In some embodiments of the present invention, the method for determining the noise separation parameter includes: according to the separation parameter, by (A 2 C(l,k)W H (l,k))(A 1 C(l,k)W H (l,k)) -1 The noise subspace J(l,k) is calculated and [J(l,k),-I M-S ] to obtain the noise separation parameter U(l,k), where A 1 and A 2 are all constant matrices, and A 1 =[IS ,O S×M-S ], A 1 =[O (M-S)×S ,I M-S ], I is the identity matrix, Q *×* is a zero matrix, C(l,k) is an M*M noise parameter matrix, and in some embodiments of the present invention, the preceding noise parameter matrix C(l-1,k) in the preceding noise separation parameter of the previous frame of sound source data and the sound signal of the sound source data to be processed can be obtained by αC(l-1,k)+(1-α)X(l,k)X H (l,k) obtains C(l,k), where α is the forgetting factor in the auxiliary parameter calculation, and its numerical setting is the same as the forgetting factor setting in the auxiliary parameter, and can be set to 0.95; in some embodiments of the present invention, the noise parameter matrix C(1,k) of the first frame processing sound source data of C(l,k) is a zero matrix.

[0104] In some embodiments of the present invention, after obtaining the noise separation parameter U(l,k), by Get the total separation parameters By calculating the total separation parameter The product of the frequency domain signal of the sound signal X(l,k) in the sound source data to be processed Separate the separated signal Y(l,k) from the sound source data to be processed, where the separated signal Y(l,k) is a column vector with S elements, where each element Y s (l,k),s=1,2,...,S represents the frequency domain signal of each candidate sound source. The frequency domain signal of each candidate sound source is separated by inverse short-time Fourier transform to obtain the sound signal y of each candidate sound source. s (l).

[0105] In some embodiments of the present invention, in order to increase the stability of the separated signal and the accuracy of the separation result to solve the ambiguity problem of blind source separation, in step b2, after obtaining the separation parameters, the noise separation parameters are obtained according to the separation parameters. Get the total separation parameters of the sound source data to be processed pass Get the first transformation matrix of the total separation parameters Extract the first transformation matrix The elements from the first row to the Sth row in get the second transformation matrix W bp (l,k), through Separate the separation signal Y(l,k) from the sound source data to be processed. Among them, A(l,k) is an M*M diagonal matrix, and the diagonal elements in A(l,k) are the total separation parameters The diagonal elements after inversion; (·)H represents the conjugate transpose.

[0106] It should be noted that the above-mentioned method of separating the sound signals of candidate sound sources from the sound source data to be processed by separation parameters is only an exemplary description of the separation method based on overdetermined independent vector analysis optimized by auxiliary functions. In practical applications, the method of separating the sound signals of candidate sound sources from the sound source data to be processed by separation parameters can be adjusted according to the separation method adopted.

[0107] In some embodiments of the present invention, in step 201, an initial sound source area can be selected from the sound source space where the electronic device collecting the sound source data to be processed is located, and the initial sound source area is evenly divided according to the preset azimuth angle to obtain multiple direction vectors, each direction vector is set as an initial sound source position, and the power value of the sound source data to be processed at each initial sound source position is calculated by SRP (full name: Steered Response Power, full name: controlled response power), and a candidate sound source position is selected from the multiple initial sound source positions according to the power value of each initial sound source position, and a candidate sound source and the position information of the candidate sound source are obtained according to the selected candidate sound source position. Specifically, Figure 3 As shown, Figure 3 30 is a flow chart of a method for estimating a candidate sound source in a sound signal processing method provided by an embodiment of the present invention. The method for estimating a candidate sound source shown includes steps 301 to 304:

[0108] 301. Determine and obtain multiple initial sound source positions according to preset azimuth angles.

[0109] In some embodiments of the present invention, the geometric center of the microphone array in the electronic device can be used as the origin, and a spatial coordinate system is established with the origin. An initial sound source area with a preset angle range is selected in a clockwise or counterclockwise direction with the origin as the center of the circle and a preset distance as the radius. There is at least one initial sound source in the initial sound source area. With the origin as the center of the circle and the preset distance as the radius, the circle is rotated in a clockwise direction or a counterclockwise direction. In the initial sound source area, a position is selected for each rotation of a preset azimuth angle. Multiple positions are selected, each selected position is set as the initial sound source position, and the azimuth angle of each selected position and the pitch angle formed by each selected position and the origin are used to obtain the direction vector of each selected position, and the direction vector of each selected position is set as the position information of the initial sound source position.

[0110] 302. Determine, according to each initial sound source position, the distance between each initial sound source position and each sound channel position.

[0111] In some embodiments of the present invention, step 302 includes: determining the position coordinates of each microphone channel of the microphone array in the electronic device in the spatial coordinate system according to the spatial coordinate system, setting the position coordinates of each microphone channel of the microphone array in the electronic device in the spatial coordinate system as each sound channel position for collecting the sound source data to be processed, determining the position coordinates of each initial sound source position according to the position information of each initial sound source position, and for each initial sound source position, obtaining the distance between the initial sound source position and each sound channel position according to the position coordinates of the initial sound source position and each sound channel position. In some embodiments of the present invention, the distance between the initial sound source position and each sound channel position can be obtained by calculating the 2-norm between the position coordinates of the initial sound source position and each sound channel position; and the distance between the initial sound source position and each sound channel position can also be obtained by calculating the Euclidean distance or Mahalanobis distance between the position coordinates of the initial sound source position and each sound channel position.

[0112] 303 , determining the power of the sound signal at each initial sound source position according to the distance between each initial sound source position and each sound channel position.

[0113] In some embodiments of the present invention, step 303 includes: for each initial sound source position, according to the distance between the initial sound source position and each sound channel position, obtaining the distance difference between the initial sound source position and each two adjacent sound channel positions, according to the distance difference, obtaining the time difference between the two adjacent sound channel positions receiving the signal of the initial sound source position, according to the sound source data to be processed, determining the sound signal at each sound channel position of the two adjacent sound channel positions, according to the time difference and the sound signal at each sound channel position of the two adjacent sound channel positions, obtaining the power of the signal at the initial sound source position at the previous sound channel position of the two adjacent sound channel positions, summing up the power of the signal at each sound channel position of the candidate sound source position, and obtaining the power of the sound signal at the initial sound source position. The previous sound channel position of the two adjacent sound channel positions may be a sound channel position whose position coordinates of the sound channel are smaller than the position coordinates of another sound channel in the two adjacent sound channel positions.

[0114] In some embodiments of the present invention, step 303 may also calculate the power of the sound signal at each initial sound source position according to steps a1 to a3.

[0115] 304 , determining candidate sound sources and position information of the candidate sound sources according to the power of the sound signals at the positions of the initial sound sources.

[0116] In some embodiments of the present invention, step 304 includes: sorting the initial sound source positions in descending order of power according to the power of the sound signal at each initial sound source position, selecting a preset number of target initial sound source positions from the sorted initial sound source positions, setting the selected target initial sound source positions as candidate sound sources, and setting the position information of each target initial sound source position as the position information of the candidate sound source. It should be noted that the embodiment of the present invention does not limit the specific value of the preset number, that is, the number of candidate sound sources is not limited. For example, in order to reduce the amount of calculation in the sound source separation process, the number of candidate sound sources can be set to a number that is less than or equal to the number of sound channels for collecting the sound source data to be processed.

[0117] In some embodiments of the present invention, step 304 includes: comparing the power of the sound signal at each initial sound source position with the power threshold in turn, selecting the initial sound source position with a power greater than or equal to the preset power, setting the selected initial sound source position with a power greater than or equal to the preset power as a candidate sound source, and setting the position information of each selected initial sound source position with a power greater than or equal to the preset power as the position information of the candidate sound source. In some embodiments of the present invention, the power threshold may be pre-set, and may be determined based on the average, mode or median of the power of the sound signal at each initial sound source position. It may also be based on the power of the sound signal at each initial sound source position, sorting the powers in order from large to small, and setting the power value at the preset number of the sorted powers as the power threshold.

[0118] In some embodiments of the present invention, step 304 includes: determining the maximum power among the powers of the sound signals at each initial sound source position based on the powers of the sound signals at each initial sound source position, calculating the power difference between the power of the sound signals at each initial sound source position and the maximum power, setting the initial sound source position corresponding to the power with a power difference less than or equal to a preset power difference threshold as a candidate sound source, and setting the position information of the initial sound source position corresponding to the power with a power difference less than or equal to the preset power difference threshold as the position information of the candidate sound source.

[0119] In some embodiments of the present invention, in step 303, for each initial sound source position, time information of when the signal at the initial sound source position reaches each sound channel position can be obtained through the initial sound source position and each sound channel position, and the power of the sound signal at the initial sound source position is determined according to the time information of when the signal at the initial sound source position reaches each sound channel position. Specifically, the power calculation method of the initial sound source position includes steps c1 to c3:

[0120] Step c1, for each initial sound source position, according to the distance between the initial sound source position and the positions of each sound channel, determine the time information of the signal at the initial sound source position reaching the positions of each sound channel.

[0121] In some embodiments of the present invention, for each initial sound source position, the distance between the initial sound source position and each sound channel position for collecting the sound source data to be processed can be obtained through the initial sound source position and the sound channel positions. According to the sound propagation speed and the distance between the initial sound source position and each sound channel position, the time information of the signal from the initial sound source position arriving at each sound channel position can be obtained.

[0122] Step c2, determining the power of the sound signal at each sound channel position according to the time information of the signal at the initial sound source position arriving at each sound channel position.

[0123] In some embodiments of the present invention, the time delay can be estimated based on the time information of the signal from the initial sound source position reaching each sound channel position, and the controllable response power function value of each sound channel can be obtained, and the controllable response power function value of each sound channel can be set to the power of the sound signal at each sound channel position. Among them, the controllable response power function value can be obtained by estimating the time delay based on the time information of the signal from the initial sound source position reaching each sound channel position through a generalized cross-correlation function weighted by phase transformation. Specifically, the method of determining the power of the sound signal at each sound channel position according to the generalized cross-correlation function weighted by phase transformation includes:

[0124] (1) For each sound channel position, determine the first time information when the signal at the initial sound source position arrives at the next sound source channel position adjacent to the sound channel position, and the second time information when the signal at the initial sound source position arrives at the next sound source channel position adjacent to the sound channel position.

[0125] (2) Determine the time difference between the first time information and the second time information.

[0126] (3) Determine the power at the sound channel position based on the time difference, the sound signal at the sound channel position, and the sound signal at the next sound source channel position adjacent to the sound channel position.

[0127] In some embodiments of the present invention, the time difference τ between the signal at the initial sound source position reaching the sound channel position and the signal at the next sound source channel position adjacent to the sound channel position can be obtained by subtracting the first time information of the signal at the initial sound source position reaching the sound channel position from the second time information of the signal at the initial sound source position reaching the next sound source channel position adjacent to the sound channel position. ij (d n ), where dn represents the initial sound source position, i represents the i-th sound source channel, j represents the j-th sound source channel, and j=i+1.

[0128] In some embodiments of the present invention, after obtaining the time difference τ ij (d n ) then, according to the frequency domain signal X of each frequency point k of the sound signal at the sound channel position i (k) and the frequency domain signal X of the sound signal at each frequency point k of the next sound source channel position adjacent to the sound channel position j (k) by Get the controllable response power function value R of the sound channel position ij [τ ij (d n )], the controllable response power function value of the sound channel position is set to the power of the sound signal at the sound channel position, where (·) * represents conjugation, F s is the sampling frequency of the sound signal in the sound source data to be processed, and K is the number of frequency points of the short-time Fourier transform.

[0129] Step c3, determining the power of the sound signal at the initial sound source position according to the power at each sound channel position.

[0130] In some embodiments of the present invention, after obtaining the power of the sound signal at the sound channel position, The power F(d n ).

[0131] In some embodiments of the present invention, considering that for the initial sound source position, the signal quality received by different sound channel positions in the electronic device at the initial sound source position is different, in the method for determining the power of the sound signal at each sound channel position based on the generalized cross-correlation function weighted by phase transformation, if for each sound channel position, the signal quality received at the sound channel position is not considered, only by The power F(d n ), which may reduce the accuracy of subsequent sound source estimation, and, in practical applications, the quality of the received signal of the pair of sound channel positions can be characterized by the maximum value of the controllable response power function of the sound channel position pair.

[0132] Based on this, in step c2, the initial power of each sound channel position and the initial power of the next sound channel position adjacent to each sound channel position are obtained through the time difference between the signal at the initial sound source position reaching each sound channel position and reaching the next sound channel position adjacent to each sound channel position, and the power weight of each sound channel position is obtained according to the maximum value of the initial power of each sound channel position and the initial power of the next sound channel position adjacent to each sound channel position. The power of the sound signal at each sound channel position is obtained through the initial power of each sound channel position and the power weight of the sound channel position. Specifically, the power determination method of the sound signal based on the weight includes:

[0133] (1) According to the time information of the signal at the initial sound source position arriving at each sound channel position, the initial power of the sound signal at each sound channel position is determined.

[0134] (2) According to the initial power corresponding to each sound channel position, a target power among the initial powers corresponding to every two adjacent sound channel positions is determined, where the target power represents a larger value among the initial powers corresponding to every two adjacent sound channel positions.

[0135] (3) For each sound channel position, the power weight of the sound channel position is determined according to the initial power corresponding to the sound channel position, the initial power corresponding to the next sound channel position adjacent to the sound channel position, and each target power.

[0136] (4) Determine the power at the sound channel position according to the initial power corresponding to the sound channel position and the power weight of the sound channel position.

[0137] Among them, for the initial sound source position, the initial power of the sound signal at each sound channel position can be obtained according to the method of determining the power of the sound signal at each sound channel position based on the above-mentioned generalized cross-correlation function weighted by phase transformation.

[0138] In some embodiments of the present invention, for each sound channel position where the sound source data to be processed is collected, the maximum initial power R is determined between the initial power of the sound signal at the sound channel position and the initial power of the sound signal at the next sound channel position adjacent to the sound channel position. max ij [τ ij (d n )], the maximum initial power R max ij [τ ij (d n)] Set the target power. After obtaining each target power, by accumulating each target power, the target power total value of the sound channel position where the sound source data to be processed is collected at the initial sound source position is obtained. For each sound channel position, determine the maximum initial power of the initial power of the sound signal at the sound channel position and the initial power of the sound signal at the next sound channel position adjacent to the sound channel position, and calculate the maximum initial power by comparing the maximum initial power with the target power total value. The ratio of the initial power of the sound signal at the sound channel position and the initial power of the sound signal at the next sound channel position adjacent to the sound channel position are normalized to obtain the power weight ω of the sound signal at the sound channel position. i,j .

[0139] In some embodiments of the present invention, when obtaining the power weight ω of the sound signal at the sound channel position, i,j , the initial power R of the sound signal at the sound channel position ij [τ ij (d n )], according to ω i,j R ij [τ ij (d n )] to obtain the power of the sound channel position.

[0140] In some embodiments of the present invention, by ω i,j R ij [τ ij (d n )] After obtaining the power of the sound channel position, The power F(d n ).

[0141] In some embodiments of the present invention, in step 202, the time information of the signal of the position information of each candidate sound source reaching the position of each sound channel of the sound source data to be processed can be obtained based on the position information of the candidate sound source and the position of each sound channel of the sound source data to be processed, and the position guidance information of each candidate sound source can be obtained based on the time information of the signal of the position information of each candidate sound source reaching the position of each sound channel of the sound source data to be processed. Specifically, the method for determining the position guidance information includes steps d1 to d2:

[0142] Step d1: for each candidate sound source, according to the position information of the candidate sound source, determine the time information when the signal of the candidate sound source arrives at the position of each sound source channel.

[0143] Step d2, obtaining the position guidance information of the candidate sound source according to the time information when the signal of the candidate sound source arrives at the position of each sound source channel.

[0144] In some embodiments of the present invention, step d1 includes: according to the established spatial coordinate system, according to the position information of each sound source channel for collecting the sound source data to be processed in the spatial coordinate system and the position information of the coordinate origin in the spatial coordinate system, obtaining the position vector of each sound source channel for collecting the sound source data to be processed; for each candidate sound source, according to the inner product between the direction vector of the position information of the candidate sound source and the position vector of each sound source channel for collecting the sound source data to be processed, obtaining the propagation distance of the signal at the position of the candidate sound source to the position of each sound source channel for collecting the sound source data to be processed; according to the propagation distance of the signal of the candidate sound source to each sound source channel for collecting the sound source data to be processed and the sound propagation speed, obtaining the time information of the signal of the candidate sound source arriving at the position of each sound source channel for collecting the sound source data to be processed.

[0145] In some embodiments of the present invention, step d2 includes: inputting the time information of the signal of the candidate sound source arriving at each sound source channel position where the sound source data to be processed is collected into a preset vector model Get the position information of the candidate sound source in, Indicates the location information of the candidate sound source, ω is the preselected simulated angular frequency, τm ,m=1,2,...,M represents the time information of the signal of the candidate sound source arriving at each sound source channel position where the sound source data to be processed is collected.

[0146] In some embodiments of the present invention, after obtaining the position guidance information of each candidate sound source, in step 203, the sound source data to be processed is separated according to the position guidance information of each candidate sound source to obtain the sound signal of each candidate sound source.

[0147] In the sound source separation process of the embodiment of the present invention, the sound source of the sound source data to be processed is first located to obtain the number of candidate sound sources and the position information of the candidate sound sources, and then the position information of each candidate sound source is used to obtain the position guidance information of each candidate sound source, and the sound source separation is performed on the sound source data to be processed by using the separation method of overdetermined independent vector analysis combined with the position guidance information, and the sound signal of each candidate sound source is separated from the sound source data to be processed. The embodiment of the present invention uses the position guidance information to guide blind source separation, enhances the stability of the sound signal output by separation, and avoids the situation that the existing separation method that only uses overdetermined independent vector analysis may output pure noise when the sound source position changes.

[0148] In some embodiments of the present invention, considering that the sound source separation processing method shown in 201 to 203 needs to estimate the number of candidate sound sources, which increases the calculation amount of the sound signal processing method, in order to reduce the calculation amount of the sound signal processing method, the embodiment of the present invention provides a sound source separation processing method without sound source estimation, specifically, as Figure 4 As shown, Figure 4 4 is a flow chart of another sound source separation process in the sound signal processing method provided by an embodiment of the present invention. The sound source separation process method shown includes steps 401 to 403:

[0149] 401 , performing sound source separation processing on the sound source data to be processed to obtain a plurality of predicted sound sources and a sound signal of each initial predicted sound source.

[0150] Considering that the existing separation method based on independent vector analysis needs to perform sound source estimation to determine the number of candidate sound sources or know in advance the number of candidate sound sources to be separated when performing sound source separation, which increases the calculation amount of the sound signal processing method. In order to solve the problem that the number of candidate sound sources must be estimated when the separation method based on independent vector analysis performs blind source separation, the embodiment of the present invention establishes an initial separation parameter W according to the number m of sound channels of the sound source data to be processed when the separation method based on independent vector analysis performs sound source separation. M×M (l), by iteratively updating the initial separation parameter W M×M (l) Separate sound signals of M predicted sound sources from the sound source data to be processed, perform redundant signal detection and elimination on the separated sound signals of the M predicted sound sources, and extract sound signals of candidate sound sources from the separated sound signals of the M predicted sound sources.

[0151] In some embodiments of the present invention, in step 401, the initial separation parameter W can be established according to the number m of sound channels of the sound source data to be processed. M×M (l), through an iterative model based on an equivariant adaptive decomposition algorithm For the initial separation parameter W M×M (l) Iterate, and in each iteration, separate the separated signal of the iteration from the sound source data to be processed, and when the number of iterations reaches the preset number of iterations, obtain the final separated signal, and separate the sound signals of M predicted sound sources from the sound source data to be processed. Wherein, I is an m*m dimensional unit matrix, l represents the number of iterations, α(l) represents the iteration step, E represents the expectation, y represents a nonlinear function, which is related to the probability density function of the sound signal of the sound source data to be processed, y represents the separated signal obtained by the lth extraction, and T represents the transpose.

[0152] In some embodiments of the present invention, in step 401, the initial separation parameter W is also established according to the number m of sound channels of the sound source data to be processed. M×M (l), through an iterative model based on the natural gradient method For the initial separation parameter W M×M (l) Iteration is performed, and in each iteration, the separated signal of the iteration is separated from the sound source data to be processed. When the number of iterations reaches a preset number of iterations, the final separated signal is obtained, and the sound signals of M predicted sound sources are separated from the sound source data to be processed. The parameter meanings of the iterative model based on the natural gradient method are the same as those of the iterative model based on the equivariant adaptive decomposition algorithm, and are not repeated here.

[0153] In some embodiments of the present invention, in step 401, the time domain signals x of the M sound channels of the sound source data to be processed may be x={x 1 ;x 2 ;...;x M}Short-time Fourier transform is performed to obtain the frequency domain signals X(k) of the M sound channels of the sound source data to be processed, k = 1, 2, ..., K, where K is the number of points of the short-time Fourier transform, X(k) = {X 1 (k); ...; X M (k)}; The frequency domain signal X(k) of each frequency point is separated from the sound source according to the separation method of independent vector analysis based on auxiliary function optimization, and the sound signals of M predicted sound sources are separated from the sound source data to be processed.

[0154] 402, calculating the mutual correlation coefficients between the sound signals of each predicted sound source to obtain a correlation coefficient matrix.

[0155] In some embodiments of the present invention, when the number of candidate sound sources is unknown, there are S independent components in the sound signals of the separated M predicted sound sources, and the remaining MS components are copies of one or more independent components or zero signals, wherein the S independent components are the sound signals of the S candidate sound sources. Since the correlation between the S independent components in the sound signals of the M predicted sound sources is relatively low, while there is a correlation between the MS components composed of copies of one or more independent components or zero signals, the sound signals of the S candidate sound sources can be extracted from the sound signals of the M predicted sound sources through the mutual correlation coefficients between the sound signals corresponding to each predicted sound source.

[0156] Specifically, step 402 includes: for each predicted sound source, calculating the autocorrelation coefficient between the sound signal corresponding to the predicted sound source and the sound signal corresponding to the predicted sound source, and the mutual correlation coefficient between the sound signal corresponding to the predicted sound source and the sound signal of each predicted sound source except the sound signal corresponding to the predicted sound source, to obtain the correlation coefficient of the sound signal corresponding to the predicted sound source; establishing a correlation coefficient matrix corresponding to the sound signal of the predicted sound source according to the correlation coefficient of the sound signal corresponding to each initial sound source

[0157] 403 , determining candidate sound sources and sound signals of the candidate sound sources from the predicted sound sources according to the correlation coefficient matrix.

[0158] In some embodiments of the present invention, in the correlation coefficient matrix, the diagonal elements of the matrix represent the autocorrelation coefficients of the sound signals of the predicted sound sources, which must all be 1, and the other elements in the matrix represent the mutual correlation coefficients between the sound signals of any two predicted sound sources; the values ​​of the non-diagonal elements in each column or row of the correlation coefficient matrix are compared with the preset coefficients; if there is a target element in the values ​​of the non-diagonal elements in each column or row of the correlation coefficient matrix, and the absolute difference between the value of the element and the preset coefficient is less than or equal to the preset threshold, it means that there is a redundant signal in the predicted sound source that is the same as or similar to the sound signal of the predicted sound source corresponding to the diagonal element, and the predicted sound source corresponding to the target element is eliminated; the redundant signals are eliminated by the correlation coefficient matrix, and data cleaning is performed on multiple predicted sound sources to obtain candidate sound sources and sound signals of candidate sound sources.

[0159] Considering that the sound signal of the separated candidate sound source includes the sound signal of the target sound source and the sound signal of the non-target sound source, and the sound signal of the target sound source has very few other aliased signals or noise components, and the voice quality is good, while the sound signal of the non-target sound source has a voice quality worse than the quality of the sound signal of the target sound source due to aliased noise or other signals, the sound signal of the non-target sound source can be selected from the sound signals of the separated candidate sound sources by evaluating the voice quality of the sound signal of each candidate sound source. Therefore, after obtaining the candidate sound sources in the sound source data to be processed and the sound signal of each candidate sound source, in order to further remove the noise in the candidate sound source, the embodiment of the present invention evaluates the voice quality of the sound signal of each candidate sound source, obtains the evaluation value of the sound signal of each candidate sound source, and screens the candidate sound sources according to the evaluation value, and selects the target sound source from multiple candidate sound sources.

[0160] In some embodiments of the present invention, the evaluation value corresponding to the sound signal of each candidate sound source can be obtained by calculating the kurtosis value of the sound signal of each candidate sound source, wherein the kurtosis value is used to describe the speech characteristics of the sound signal, and the larger the kurtosis value of the sound signal, the higher the speech quality of the sound signal. Specifically, the sound signal evaluation method based on the kurtosis value includes:

[0161] (1) Performing time-frequency domain conversion on the sound signal of each candidate sound source to obtain a time domain signal of the sound signal of each candidate sound source.

[0162] (2) Determine the kurtosis value corresponding to the time domain signal of the sound signal of each candidate sound source, and set the kurtosis value as the evaluation value corresponding to the sound signal of the candidate sound source.

[0163] In some embodiments of the present invention, after separating the sound signals of multiple candidate sound sources, the frequency domain signal Y(l,k)=[Y 1 (l,k),Y 2 (l,k),...,Y S (l,k)], and through inverse short-time Fourier transform, the time domain signals of the sound signals of multiple candidate sound sources are obtained y(l)=[y 1 (l),y 2 (l),...,y S (l)], for each candidate sound source, according to the time domain signal y of the sound signal of the candidate sound source s (l), s = 1, ..., S, through Get the kurtosis value K(y) of the sound signal of the candidate sound source s (l)), the kurtosis value K(y s (l)) is set to the evaluation value corresponding to the sound signal of the candidate sound source.

[0164] In some embodiments of the present invention, when the electronic device is an electronic device that uses voice as an interactive mode, the wake-up word score in the sound signal of each candidate sound source can also be determined based on the voice features of the sound signal of each candidate sound source, and the wake-up word score in the sound signal of each candidate sound source is set to the evaluation value corresponding to the sound signal of each candidate sound source, that is, the target sound source is selected from multiple candidate sound sources by the wake-up word score in the sound signal of each candidate sound source. Among them, the wake-up word score is used to quantify the voice quality in the sound signal of each candidate sound source. In some embodiments of the present invention, the wake-up word score in the sound signal of each candidate sound source can be determined by determining the probability that the sound feature corresponding to the sound signal of each candidate sound source is the sound feature of the wake-up word. Specifically, the sound signal evaluation method based on the wake-up word score includes:

[0165] (1) Obtain the speech feature vector of the sound signal of each candidate sound source.

[0166] (2) Determine the probability score corresponding to the speech feature vector of the sound signal of each candidate sound source.

[0167] (3) Determine the evaluation value corresponding to the sound signal of each candidate sound source according to the probability score corresponding to the speech feature vector of the sound signal of each candidate sound source.

[0168] The probability score represents the probability that the speech feature vector is the speech feature vector corresponding to the wake-up word.

[0169] In some embodiments of the present invention, the frequency domain signal Y(l,k)=[Y 1 (l,k),Y 2 (l,k),...,Y S (l,k)], and through inverse short-time Fourier transform, the time domain signals of the sound signals of multiple candidate sound sources are obtained y(l)=[y 1 (l),y 2 (l),...,y S (l)], for each candidate sound source, according to the time domain signal y of the sound signal of the candidate sound source s (l), s = 1, ..., S, for each candidate sound source time domain signal y s (l), by deriving the Mel-frequency cepstral coefficients from the spectrum s (l) extracts key feature parameters reflecting the characteristics of the speech signal to form a feature vector sequence, and sets the feature vector sequence as the speech feature vector of the sound signal of the candidate sound source.

[0170] In some embodiments of the present invention, the semantic features of the sound signal of each candidate sound source can be obtained according to the speech feature vector of the sound signal of each candidate sound source, the semantic features of the sound signal of each candidate sound source are compared with the preset semantic features, the similarity between the semantic features of the sound signal of each candidate sound source and the preset semantic features is obtained, and the similarity between the semantic features of the sound signal of each candidate sound source and the preset semantic features is set as the probability score corresponding to the speech feature vector of the sound signal of each candidate sound source. The semantic features of the sound signal of each candidate sound source can be obtained according to the semantic feature extraction method in step 102.

[0171] In some embodiments of the present invention, for each candidate sound source, the probability score corresponding to the speech feature vector of the sound signal of the candidate sound source may be set as the evaluation value corresponding to the sound signal of the candidate sound source.

[0172] In some embodiments of the present invention, for each candidate sound source, the probability score corresponding to the speech feature vector of the sound signal of the candidate sound source can be compared with a preset probability threshold. If the probability score corresponding to the speech feature vector of the sound signal of the candidate sound source is greater than the preset probability threshold, the evaluation value corresponding to the sound signal of the candidate sound source is set to a first preset value; if the probability score corresponding to the speech feature vector of the sound signal of the candidate sound source is less than or equal to the preset probability threshold, the evaluation value corresponding to the sound signal of the candidate sound source is set to a second preset value. The first preset value can be 1, and the second preset value can be 0; the first preset value can also be 100, and the second preset value can also be 0.

[0173] In some embodiments of the present invention, for each candidate sound source, the probability score corresponding to the speech feature vector of the sound signal of the candidate sound source can be queried for pre-stored evaluation data to determine the probability interval in which the probability score corresponding to the speech feature vector of the sound signal of the candidate sound source is located, and the evaluation score corresponding to the probability interval, and the evaluation score corresponding to the probability interval is set as the evaluation value corresponding to the sound signal of the candidate sound source. The pre-stored evaluation data includes multiple probability intervals and the evaluation score corresponding to each probability interval.

[0174] In some embodiments of the present invention, after obtaining the evaluation value corresponding to the sound signal of each candidate sound source, the candidate sound source corresponding to the maximum evaluation value can be determined according to the evaluation value corresponding to the sound signal of each candidate sound source, the candidate sound source corresponding to the maximum evaluation value can be set as the target sound source, and the sound signal of the candidate sound source corresponding to the maximum evaluation value can be set as the sound signal of the target sound source.

[0175] The sound signal processing method provided by the embodiment of the present invention determines the final target sound source by evaluating the sound signal of each candidate sound source obtained by blind source separation, improves the problem of low stability of blind source separation, and reduces noise by improving the accuracy of the target sound source to improve the audio-visual effect.

[0176] In order to better implement the sound signal processing method provided by the embodiment of the present invention, based on the embodiment of the sound signal processing method, the embodiment of the present invention also provides a sound signal processing device, such as Figure 5 As shown, Figure 5 : is a schematic diagram of the structure of a sound signal processing device provided by an embodiment of the present invention, wherein the sound signal processing device shown includes:

[0177] The separation module 501 is used to perform sound source separation processing on the sound source data to be processed, and obtain candidate sound sources corresponding to the sound source data to be processed and sound signals belonging to each candidate sound source in the sound source data to be processed;

[0178] An evaluation module 502 is used to perform a quality evaluation on the sound signal of each candidate sound source and determine an evaluation value of the sound signal of each candidate sound source;

[0179] A selection module 503 is used to determine a target sound source from a plurality of candidate target sound sources according to the evaluation value corresponding to the sound signal of each candidate sound source;

[0180] The processing module 504 is used to process the sound signal of the target sound source.

[0181] In some embodiments of the present invention, the separation module 501 includes:

[0182] A sound source estimation unit, used to estimate the sound source position of the sound source data to be processed, and determine the candidate sound sources corresponding to the sound source data to be processed and the position information of each candidate sound source;

[0183] A vector determination unit, for determining the positional orientation information of each candidate sound source according to the positions of each sound channel of the sound source data to be processed and the positional information of each candidate sound source;

[0184] The separation unit is used to perform sound source separation on the sound source data to be processed according to the position guidance information of each candidate sound source to obtain the sound signal of each candidate sound source.

[0185] In some embodiments of the present invention, the separation unit:

[0186] A separation parameter subunit, used to determine a separation parameter according to the position orientation information of each candidate sound source;

[0187] The separation subunit is used to perform sound source separation on the sound signal in the sound source data to be processed according to the separation parameters, and determine to obtain the sound signal of each candidate sound source.

[0188] In some embodiments of the present invention, the separation parameter subunit is used to:

[0189] Obtain historical separation parameters of historical sound source data and auxiliary parameters corresponding to the sound source data to be processed;

[0190] According to the position orientation information of each candidate sound source, the auxiliary parameter matrix is ​​modified to obtain a modified auxiliary parameter;

[0191] According to the corrected auxiliary parameters and the historical separation parameters, the separation parameters of the sound source data to be processed are obtained.

[0192] In some embodiments of the present invention, the sound source estimation unit is used to:

[0193] According to the preset azimuth angles, multiple initial sound source positions are determined;

[0194] According to each initial sound source position, determining the distance between each initial sound source position and each sound channel position for collecting the sound source data to be processed;

[0195] Determining the power of the sound signal at each initial sound source position according to the distance between each initial sound source position and each sound channel position;

[0196] According to the power of the sound signal at each initial sound source position, the candidate sound source and the position information of the candidate sound source are determined.

[0197] In some embodiments of the present invention, the sound source estimation unit is used to:

[0198] For each initial sound source position, determine the time information of the signal at the initial sound source position arriving at each sound channel position according to the distance between the initial sound source position and each sound channel position;

[0199] Determine the power of the sound signal at each sound channel position according to the time information of the signal at the initial sound source position arriving at each sound channel position;

[0200] The power of the sound signal at the initial sound source position is determined according to the power of the sound signal at each sound channel position.

[0201] In some embodiments of the present invention, the sound source estimation unit is used to:

[0202] For each sound channel position, determine first time information when the signal at the initial sound source position reaches the sound channel position, and second time information when the signal at the initial sound source position reaches the next sound source channel position adjacent to the sound channel position;

[0203] determining a time difference between the first time information and the second time information;

[0204] The power of the sound signal at the sound channel position is determined based on the time difference, the sound signal at the sound channel position, and the sound signal at the next sound source channel position adjacent to the sound channel position.

[0205] In some embodiments of the present invention, the sound source estimation unit is used to:

[0206] Determine the initial power of the sound signal at each sound channel position according to the time information of the signal at the initial sound source position arriving at each sound channel position;

[0207] According to the initial power corresponding to each sound channel position, a target power among the initial powers corresponding to every two adjacent sound channel positions is determined, where the target power represents a larger value among the initial powers corresponding to every two adjacent sound channel positions;

[0208] For each sound channel position, determine the power weight of the sound channel position according to the initial power corresponding to the sound channel position, the initial power corresponding to the next sound channel position adjacent to the sound channel position, and each target power;

[0209] The power at the sound channel position is determined according to the initial power corresponding to the sound channel position and the power weight of the sound channel position.

[0210] In some embodiments of the present invention, the vector determination unit is used to:

[0211] For each candidate sound source, according to the position information of the candidate sound source, determine the time information when the signal of the candidate sound source arrives at the position of each sound source channel;

[0212] According to the time information of the signal of the candidate sound source arriving at the position of each sound source channel, the position guidance information of the candidate sound source is obtained.

[0213] In some embodiments of the present invention, the separation module 501 includes:

[0214] An initial separation unit is used to perform sound source separation on the sound source data to be processed, and obtain the predicted sound source corresponding to the sound source data to be processed and the sound signals belonging to each predicted sound source in the sound source data to be processed;

[0215] A correlation calculation unit, used to calculate the mutual correlation coefficients between the sound signals of each predicted sound source to obtain a correlation coefficient matrix;

[0216] The screening unit is used to determine the candidate sound source and the sound signal of the candidate sound source from each predicted sound source according to the correlation coefficient matrix.

[0217] In some embodiments of the present invention, the evaluation module 502 is used to:

[0218] Performing a time-frequency conversion on the sound signal of each candidate sound source to obtain a time-domain signal of the sound signal of each candidate sound source;

[0219] The kurtosis value corresponding to the time domain signal of the sound signal of each candidate sound source is determined, and the kurtosis value is set as the evaluation value corresponding to the sound signal of the candidate sound source.

[0220] In some embodiments of the present invention, the evaluation module 502 is used to:

[0221] Obtaining a speech feature vector of a sound signal of each candidate sound source;

[0222] Determine a probability score corresponding to the speech feature vector of the sound signal of each candidate sound source; the probability score represents the probability that the speech feature vector is the speech feature vector corresponding to the wake-up word;

[0223] The evaluation value corresponding to the sound signal of each candidate sound source is determined according to the probability score corresponding to the speech feature vector of the sound signal of each candidate sound source.

[0224] In some embodiments of the present invention, the selection module 503 is used to:

[0225] Determine the candidate sound source corresponding to the maximum evaluation value according to the evaluation value corresponding to the sound signal of each candidate sound source;

[0226] The candidate sound source corresponding to the maximum evaluation value is set as the target sound source.

[0227] The sound signal processing device provided by the embodiment of the present invention determines the final target sound source by evaluating the sound signal of each candidate sound source obtained by blind source separation, improves the problem of low stability of blind source separation, and reduces noise by improving the accuracy of the target sound source to improve the audio-visual effect.

[0228] Accordingly, an embodiment of the present invention further provides an electronic device, such as Figure 6 As shown, the electronic device may include a radio frequency (RF) circuit 601, a memory 602 including one or more computer-readable storage media, an input unit 603, a display unit 604, a sensor 605, an audio circuit 606, a wireless fidelity (WiFi) module 607, a processor 608 including one or more processing cores, and a power supply 609. Those skilled in the art will appreciate that Figure 6 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0229] The RF circuit 601 can be used for receiving and sending signals during information transmission or calls. In particular, after receiving the downlink information of the base station, it is handed over to one or more processors 608 for processing; in addition, the data related to the uplink is sent to the base station. Generally, the RF circuit 601 includes but is not limited to an antenna, at least one amplifier, a tuner, one or more oscillators, a subscriber identity module (SIM) card, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit 601 can also communicate with the network and other devices through wireless communication. Wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0230] The memory 602 can be used to store software programs and modules, and the processor 608 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device (such as audio data, a phone book, etc.), etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 602 may also include a memory controller to provide the processor 608 and the input unit 603 with access to the memory 602.

[0231] The input unit 603 can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control. Specifically, in a specific embodiment, the input unit 603 may include a touch-sensitive surface and other input devices. The touch-sensitive surface, also known as a touch display screen or a touch pad, can collect the user's touch operations on or near it (such as the user's operation on or near the touch-sensitive surface using any suitable object or accessory such as a finger, a stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch-sensitive surface may include a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 608, and can receive and execute the command sent by the processor 608. In addition, the touch-sensitive surface can be implemented using multiple types such as resistive, capacitive, infrared and surface acoustic waves. In addition to the touch-sensitive surface, the input unit 603 may also include other input devices. Specifically, other input devices may include, but are not limited to, one or more of a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, and the like.

[0232] The display unit 604 can be used to display information input by the user or information provided to the user and various graphical user interfaces of the electronic device, which can be composed of graphics, text, icons, videos and any combination thereof. The display unit 604 may include a display panel. Optionally, the display panel may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch-sensitive surface may cover the display panel. When the touch-sensitive surface detects a touch operation on or near it, it is transmitted to the processor 608 to determine the type of touch event. Subsequently, the processor 608 provides a corresponding visual output on the display panel according to the type of touch event. Although in Figure 6 In the embodiment, the touch-sensitive surface and the display panel are used as two independent components to realize input and output functions, but in some embodiments, the touch-sensitive surface and the display panel can be integrated to realize input and output functions.

[0233] The electronic device may also include at least one sensor 605, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel according to the brightness of the ambient light, and the proximity sensor may turn off the display panel and / or backlight when the electronic device is moved to the ear. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the electronic device, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0234] The audio circuit 606, the speaker, and the microphone can provide an audio interface between the user and the electronic device. The audio circuit 606 can transmit the electrical signal converted from the received audio data to the speaker, which is converted into a sound signal for output; on the other hand, the microphone converts the collected sound signal into an electrical signal, which is received by the audio circuit 606 and converted into audio data, and then the audio data is output to the processor 608 for processing, and then sent to another electronic device through the RF circuit 601, or the audio data is output to the memory 602 for further processing. The audio circuit 606 may also include an earplug jack to provide communication between an external headset and the electronic device.

[0235] WiFi is a short-range wireless transmission technology. Electronic devices can help users send and receive emails, browse web pages, and access streaming media through WiFi module 607. It provides users with wireless broadband Internet access. Figure 6 A WiFi module 607 is shown, but it is understandable that it is not an essential component of the electronic device and can be omitted as needed without changing the essence of the invention.

[0236] The processor 608 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire mobile phone. By running or executing software programs and / or modules stored in the memory 602, and calling data stored in the memory 602, it executes various functions of the electronic device and processes data, thereby monitoring the mobile phone as a whole. Optionally, the processor 608 may include one or more processing cores; preferably, the processor 608 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and computer programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 608.

[0237] The electronic device also includes a power supply 609 (such as a battery) for supplying power to each component. Preferably, the power supply can be logically connected to the processor 608 through a power management system, so that the power management system can manage charging, discharging, and power consumption management. The power supply 609 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0238] Although not shown, the electronic device may also include a camera, a Bluetooth module, etc., which will not be described in detail here. Specifically in this embodiment, the processor 608 in the electronic device will load the executable files corresponding to the processes of one or more computer programs into the memory 602 according to the following instructions, and the processor 608 will run the computer programs stored in the memory 602 to achieve various functions:

[0239] Performing sound source separation processing on the sound source data to be processed to obtain candidate sound sources corresponding to the sound source data to be processed and sound signals belonging to each candidate sound source in the sound source data to be processed;

[0240] Performing a quality assessment on the sound signal of each candidate sound source to determine an assessment value of the sound signal of each candidate sound source;

[0241] Determine a target sound source from a plurality of candidate target sound sources according to an evaluation value of a sound signal of each candidate sound source;

[0242] Process the sound signal of the target sound source.

[0243] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0244] To this end, an embodiment of the present invention provides a storage medium in which a plurality of instructions are stored, and the plurality of instructions can be loaded by a processor to execute the steps in any one of the sound signal processing methods provided in the embodiment of the present invention. For example, the computer program can execute the following steps:

[0245] Performing sound source separation processing on the sound source data to be processed to obtain candidate sound sources corresponding to the sound source data to be processed and sound signals belonging to each candidate sound source in the sound source data to be processed;

[0246] Performing a quality assessment on the sound signal of each candidate sound source to determine an assessment value of the sound signal of each candidate sound source;

[0247] Determine a target sound source from a plurality of candidate target sound sources according to an evaluation value of a sound signal of each candidate sound source;

[0248] Process the sound signal of the target sound source.

[0249] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0250] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0251] Since the computer program stored in the storage medium can execute the steps in any sound signal processing method provided in the embodiments of the present invention, the beneficial effects that can be achieved by any sound signal processing method provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0252] The above is a detailed introduction to a sound signal processing method, device, electronic device and storage medium provided in an embodiment of the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A sound signal processing method, characterized in that: The method comprises: Performing sound source position estimation on the sound source data to be processed, determining candidate sound sources corresponding to the sound source data to be processed and position information of each candidate sound source, determining position guidance information of each candidate sound source based on the positions of each sound channel of the sound source data to be processed and the position information of each candidate sound source, performing sound source separation on the sound source data to be processed based on the position guidance information of each candidate sound source, and obtaining a sound signal of each candidate sound source; Performing a quality assessment on the sound signal of each of the candidate sound sources to determine an assessment value of the sound signal of each of the candidate sound sources; Determine a target sound source from a plurality of candidate target sound sources according to an evaluation value of a sound signal of each candidate sound source; The sound signal of the target sound source is processed.

2. The sound signal processing method according to claim 1, characterized in that: The step of performing sound source separation on the sound source data to be processed according to the position guidance information of each candidate sound source to obtain a sound signal of each candidate sound source includes: Determine a separation parameter according to the position guidance information of each candidate sound source; According to the separation parameters, sound source separation is performed on the sound signal in the sound source data to be processed to determine and obtain the sound signal of each candidate sound source.

3. The sound signal processing method according to claim 2, characterized in that: Determining the separation parameter according to the position guidance information of each candidate sound source includes: Acquire historical separation parameters of historical sound source data and auxiliary parameters corresponding to the sound source data to be processed; According to the position guidance information of each candidate sound source, the auxiliary parameter is modified to obtain a modified auxiliary parameter; The separation parameters of the sound source data to be processed are obtained according to the corrected auxiliary parameters and the historical separation parameters.

4. The sound signal processing method according to claim 1, characterized in that: The process of estimating the sound source position of the sound source data to be processed and determining the candidate sound source corresponding to the sound source data to be processed and the position information of each candidate sound source comprises: According to the preset azimuth angles, multiple initial sound source positions are determined; According to each of the initial sound source positions, determining the distance between each of the initial sound source positions and the positions of each sound channel for collecting the sound source data to be processed; Determining the power of the sound signal at each of the initial sound source positions according to the distances between each of the initial sound source positions and each of the sound channel positions; According to the power of the sound signal at each of the initial sound source positions, a candidate sound source and position information of the candidate sound source are determined.

5. The sound signal processing method according to claim 4, characterized in that: Determining the power of the sound signal at each of the initial sound source positions according to the distance between each of the initial sound source positions and each of the sound channel positions includes: For each of the initial sound source positions, determine the time information of the signal at the initial sound source position reaching each of the sound channel positions according to the distance between the initial sound source position and each of the sound channel positions; Determine the power of the sound signal at each of the sound channel positions according to the time information of the signal at the initial sound source position arriving at each of the sound channel positions; The power of the sound signal at the initial sound source position is determined according to the power of the sound signal at each of the sound channel positions.

6. The sound signal processing method according to claim 5, characterized in that: The step of determining the power of the sound signal at each sound channel position according to the time information of the signal at the initial sound source position arriving at each sound channel position comprises: For each of the sound channel positions, determining first time information of the signal at the initial sound source position arriving at the sound channel position, and second time information of the signal at the initial sound source position arriving at the next sound source channel position adjacent to the sound channel position; determining a time difference between the first time information and the second time information; The power of the sound signal at the sound channel position is determined based on the time difference, the sound signal at the sound channel position, and the sound signal at the next sound source channel position adjacent to the sound channel position.

7. The sound signal processing method according to claim 5, characterized in that: The step of determining the power of the sound signal at each sound channel position according to the time information of the signal at the initial sound source position arriving at each sound channel position comprises: Determine the initial power of the sound signal at each sound channel position according to the time information when the signal at the initial sound source position arrives at each sound channel position; According to the initial powers corresponding to the positions of the sound channels, a target power among the initial powers corresponding to the positions of every two adjacent sound channels is determined, wherein the target power represents a larger value among the initial powers corresponding to the positions of every two adjacent sound channels; For each of the sound channel positions, determine a power weight of the sound channel position according to an initial power corresponding to the sound channel position, an initial power corresponding to a next sound channel position adjacent to the sound channel position, and each of the target powers; The power at the sound channel position is determined according to the initial power corresponding to the sound channel position and the power weight of the sound channel position.

8. The sound signal processing method according to claim 1, characterized in that: The step of determining the position guidance information of each candidate sound source according to the positions of each sound channel of the sound source data to be processed and the position information of each candidate sound source includes: For each of the candidate sound sources, according to the position information of the candidate sound source, determine the time information of the signal of the candidate sound source arriving at the position of each sound source channel; The position guidance information of the candidate sound source is obtained according to the time information when the signal of the candidate sound source arrives at the position of each sound source channel.

9. The sound signal processing method according to claim 1, characterized in that: The performing sound source separation on the sound source data to be processed to obtain candidate sound sources corresponding to the sound source data to be processed and sound signals belonging to each of the candidate sound sources in the sound source data to be processed comprises: Performing sound source separation on the sound source data to be processed to obtain predicted sound sources corresponding to the sound source data to be processed and sound signals belonging to each of the predicted sound sources in the sound source data to be processed; Calculating the mutual correlation coefficients between the sound signals of the predicted sound sources to obtain a correlation coefficient matrix; According to the correlation coefficient matrix, candidate sound sources and sound signals of the candidate sound sources are determined from the predicted sound sources.

10. The sound signal processing method according to claim 1, characterized in that: The performing quality assessment on the sound signal of each candidate sound source and determining the assessment value corresponding to the sound signal of each candidate sound source comprises: Performing time-frequency domain conversion on the sound signal of each candidate sound source to obtain a time domain signal of the sound signal of each candidate sound source; The kurtosis value corresponding to the time domain signal of the sound signal of each candidate sound source is determined, and the kurtosis value is set as the evaluation value corresponding to the sound signal of the candidate sound source.

11. The sound signal processing method according to claim 1, characterized in that: The performing quality assessment on the sound signal of each candidate sound source and determining the assessment value corresponding to the sound signal of each candidate sound source comprises: Acquire a speech feature vector of a sound signal of each of the candidate sound sources; Determine a probability score corresponding to the speech feature vector of the sound signal of each candidate sound source; the probability score represents the probability that the speech feature vector is the speech feature vector corresponding to the wake-up word; An evaluation value corresponding to the sound signal of each candidate sound source is determined according to the probability score corresponding to the speech feature vector of the sound signal of each candidate sound source.

12. The sound signal processing method according to any one of claims 1 to 11, characterized in that: The step of determining a target sound source from a plurality of candidate sound sources according to an evaluation value corresponding to a sound signal of each candidate sound source comprises: Determine, according to the evaluation values ​​corresponding to the sound signals of each of the candidate sound sources, a candidate sound source corresponding to the maximum evaluation value; The candidate sound source corresponding to the maximum evaluation value is set as the target sound source.

13. A sound signal processing device, characterized in that: The device comprises: A separation module is used to estimate the sound source position of the sound source data to be processed, determine the candidate sound source corresponding to the sound source data to be processed and the position information of each candidate sound source, determine the position guidance information of each candidate sound source according to the positions of each sound channel of the sound source data to be processed and the position information of each candidate sound source, perform sound source separation on the sound source data to be processed according to the position guidance information of each candidate sound source, and obtain the sound signal of each candidate sound source; An evaluation module, used to perform quality evaluation on the sound signal of each candidate sound source and determine an evaluation value of the sound signal of each candidate sound source; A selection module, configured to determine a target sound source from a plurality of candidate target sound sources according to an evaluation value corresponding to a sound signal of each candidate sound source; A processing module is used to process the sound signal of the target sound source.

14. An electronic device, characterized in that: It comprises a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to perform the operations in the sound signal processing method according to any one of claims 1 to 12.

15. A storage medium, characterized in that: The storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps in the sound signal processing method according to any one of claims 1 to 12.

Images