Sound source separation method and apparatus thereof, vehicle, and electronic device

Abstract

The present disclosure provides a sound source separation method and device, and relates to the technical field of intelligent vehicles, which comprises the following steps: performing sound source separation on a plurality of sound collection signals in a previous observation window according to a separation matrix corresponding to a previous round of sound source separation to obtain a plurality of separation estimation signals corresponding to the previous round of sound source separation, and obtaining a speech existence probability corresponding to the separation estimation signals; updating the separation matrix corresponding to the previous round of sound source separation according to the speech existence probability; and performing sound source separation on a plurality of sound collection signals in a current observation window according to the updated separation matrix to obtain a plurality of separation estimation signals corresponding to the current round of sound source separation. The speech existence probability of the separation estimation signals obtained based on the previous round of sound source separation is used to update the separation matrix, so that the speech existence information of the sound collection signals in the previous observation window is added when the separation matrix is updated, and the separation coefficients of various sound sources are adjusted accordingly, thereby enhancing the separation effect of the sound sources.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent vehicle technology, and in particular to a sound source separation method and apparatus, vehicle and electronic equipment. Background Technology

[0002] Currently, with the continuous development of intelligent vehicle technology, creating multifunctional intelligent cockpits and providing a better user experience has become an unstoppable trend in the automotive field. Voice, as a new generation of human-computer interaction, is increasingly being used in embedded devices, such as in-vehicle systems and home appliances. Generally, a sound acquisition signal contains multiple sound sources in different frequency ranges. It is necessary to separate the target sound source from the sound acquisition signal and perform interaction based on the target sound source. Therefore, the effectiveness of sound source separation can significantly affect the effectiveness of human-computer interaction and the user experience.

[0003] Therefore, how to enhance the effect of sound source separation has become an urgent problem to be solved. Summary of the Invention

[0004] This disclosure aims to at least partially address one of the technical problems in the related art.

[0005] Therefore, one objective of this disclosure is to propose a method for sound source separation.

[0006] The second objective of this disclosure is to provide a sound source separation device.

[0007] The third objective of this disclosure is to propose a vehicle.

[0008] The fourth objective of this disclosure is to provide an electronic device.

[0009] The fifth objective of this disclosure is to provide a computer-readable storage medium.

[0010] To achieve the above objectives, the first aspect of this disclosure proposes a sound source separation method, comprising: performing sound source separation on multiple sound acquisition signals within a previous observation window based on a separation matrix corresponding to the previous round of sound source separation, to obtain multiple separation estimation signals corresponding to the previous round of sound source separation; obtaining the speech presence probability corresponding to each separation estimation signal; updating the separation matrix corresponding to the previous round of sound source separation based on the multiple speech presence probabilities; and performing sound source separation on multiple sound acquisition signals within the current observation window based on the updated separation matrix, to obtain multiple separation estimation signals corresponding to the current round of sound source separation.

[0011] According to one embodiment of this disclosure, the sound acquisition signal is obtained by the sound acquisition device from sound sources in multiple sound zones, and the total number of sound acquisition signals is the same as the total number of sound sources.

[0012] According to one embodiment of this disclosure, updating the separation matrix corresponding to the previous round of sound source separation based on the plurality of speech presence probabilities includes: obtaining a plurality of first auxiliary functions corresponding to the previous round of sound source separation, wherein the first auxiliary functions are covariance matrices corresponding to a plurality of sound acquisition signals within the previous observation window; updating the first auxiliary functions according to the speech presence probabilities to obtain a plurality of second auxiliary functions corresponding to the current round of sound source separation, wherein the second auxiliary functions are covariance matrices corresponding to a plurality of sound acquisition signals within the current observation window; and generating the updated separation matrix based on the plurality of second auxiliary functions.

[0013] According to one embodiment of this disclosure, the step of updating the first auxiliary function corresponding to the speech presence probability to obtain multiple second auxiliary functions corresponding to the current round of sound source separation includes: obtaining the corresponding second auxiliary function based on Formula 1, according to a preset observation window length, a forgetting factor, and the first auxiliary function corresponding to the speech presence probability; Formula 1 is:

[0014]

[0015] Where L is the observation window length, α is the forgetting factor, and y k (ω, t) is the separation estimation signal corresponding to the previous round of sound source separation, V k (ω, τ) is the second auxiliary function, V k (ω, τ-L) is the first auxiliary function, k = 1, 2, ..., M, and X(ω, τ) is the sound acquisition signal x. i A matrix of M*1 (ω, τ) is formed, i = 1, 2, ..., M, where M is the total number of sound acquisition signals and the total number of sound sources, p k The separation estimation signal y corresponding to the previous round of sound source separation. k The probability of the existence of a speech sound at (ω, τ), where ω represents frequency and τ represents time.

[0016] According to one embodiment of this disclosure, generating the updated separation matrix based on a plurality of second auxiliary functions includes: updating the corresponding separation sub-matrix based on formulas two and three according to a plurality of second auxiliary functions, wherein the separation sub-matrix is ​​a 1*M matrix;

[0017] Formula 2 is as follows:

[0018] w k ′(ω,τ)=(W(ω,τ-L)V k (ω,τ)) -1 e k ;

[0019] Formula 3 is as follows:

[0020]

[0021] Where W(ω, τ-L) is the separation matrix corresponding to the previous round of sound source separation, w k ′(ω, τ) is the intermediate matrix, w k (ω, τ) is the separating submatrix, e k For an M*1 matrix, the e k The k-th element is 1, and the rest are 0. k ′H (ω, τ) is w k The conjugate transpose of ′(ω, τ); based on multiple updated separating submatrices, the updated separating matrix is ​​obtained, and the updated separating matrix is ​​an M*M matrix.

[0022] According to one embodiment of this disclosure, obtaining the speech presence probability corresponding to the separation estimation signal includes: performing speech endpoint detection on the separation estimation signal to obtain valid speech segments containing speech in the separation estimation signal; and obtaining the speech presence probability corresponding to the separation estimation signal based on the valid speech segments.

[0023] According to one embodiment of this disclosure, a separation estimation signal corresponding to a target sound source is selected from multiple separation estimation signals obtained after multiple rounds of sound source separation; and a target estimation signal corresponding to the target sound source is generated based on the separation estimation signal corresponding to the target sound source.

[0024] To achieve the above objectives, a second aspect of this disclosure provides a sound source separation device, comprising: a first acquisition module, configured to perform sound source separation on multiple sound acquisition signals within a previous observation window based on a separation matrix corresponding to the previous round of sound source separation, to obtain multiple separation estimation signals corresponding to the previous round of sound source separation; a second acquisition module, configured to acquire the speech presence probability corresponding to each of the separation estimation signals; an update module, configured to update the separation matrix corresponding to the previous round of sound source separation based on the multiple speech presence probabilities; and a separation module, configured to perform sound source separation on multiple sound acquisition signals within the current observation window based on the updated separation matrix, to obtain multiple separation estimation signals corresponding to the current round of sound source separation.

[0025] According to one embodiment of this disclosure, the sound acquisition signal is obtained by the sound acquisition device from sound sources in multiple sound zones, and the total number of sound acquisition signals is the same as the total number of sound sources.

[0026] According to one embodiment of this disclosure, the updating module is further configured to: obtain a plurality of first auxiliary functions corresponding to the previous round of sound source separation, wherein the first auxiliary function is a covariance matrix corresponding to a plurality of sound acquisition signals in the previous observation window; update the first auxiliary function according to the speech existence probability to obtain a plurality of second auxiliary functions corresponding to the current round of sound source separation, wherein the second auxiliary function is a covariance matrix corresponding to a plurality of sound acquisition signals in the current observation window; and obtain the updated separation matrix according to the plurality of second auxiliary functions.

[0027] According to one embodiment of this disclosure, the updating module is further configured to: obtain a corresponding second auxiliary function based on Formula 1, according to the first auxiliary function corresponding to the preset observation window length, forgetting factor, and the probability of speech presence; Formula 1 is:

[0028]

[0029] Where L is the observation window length, α is the forgetting factor, and y k (ω, t) is the separation estimation signal corresponding to the previous round of sound source separation, V k (ω, τ) is the second auxiliary function, V k (ω, τ-L) is the first auxiliary function, k = 1, 2, ..., M, and X(ω, τ) is the sound acquisition signal x. i A matrix of M*1 (ω, τ) is formed, i = 1, 2, ..., M, where M is the total number of sound acquisition signals and the total number of sound sources, p k The separation estimation signal y corresponding to the previous round of sound source separation. k The probability of the existence of a speech sound at (ω, τ), where ω represents frequency and τ represents time.

[0030] According to one embodiment of this disclosure, the updating module is further configured to: update the corresponding separating sub-matrix based on Formula 2 and Formula 3 according to a plurality of second auxiliary functions, wherein the separating sub-matrix is ​​a 1*M matrix;

[0031] Formula 2 is as follows:

[0032] w k ′(ω,τ)=(W(ω,τ-L)V k (ω,τ)) -1 e k ;

[0033] Formula 3 is as follows:

[0034]

[0035] Where W(ω, τ-L) is the separation matrix corresponding to the previous round of sound source separation, w k ′(ω, τ) is the intermediate matrix, w k (ω, τ) is the separating submatrix, e k For an M*1 matrix, the e k The k-th element is 1, and the rest are 0. k ′H (ω, τ) is w k The conjugate transpose of ′(ω, τ); based on multiple updated separating submatrices, the updated separating matrix is ​​obtained, and the updated separating matrix is ​​an M*M matrix.

[0036] According to one embodiment of this disclosure, the second acquisition module is further configured to: perform speech endpoint detection on the separation estimation signal to acquire valid speech segments containing speech in the separation estimation signal; and obtain the speech presence probability corresponding to the separation estimation signal based on the valid speech segments.

[0037] According to one embodiment of this disclosure, the separation module is further configured to: select a separation estimation signal corresponding to the target sound source from a plurality of separation estimation signals obtained after multiple rounds of sound source separation; and generate a target estimation signal corresponding to the target sound source based on the separation estimation signal corresponding to the target sound source.

[0038] To achieve the above objectives, a third aspect of this disclosure provides a vehicle comprising: a sound source separation device as described in the second aspect of this disclosure.

[0039] To achieve the above objectives, a fourth aspect of this disclosure provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the sound source separation method as described in the first aspect of this disclosure.

[0040] To achieve the above objectives, a fifth aspect of this disclosure provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the sound source separation method as described in the first aspect of this disclosure.

[0041] In summary, the sound source separation method of this disclosure performs sound source separation on multiple sound acquisition signals within the previous observation window based on the separation matrix corresponding to the previous round of sound source separation, obtaining multiple separation estimation signals corresponding to the previous round of sound source separation. For each separation estimation signal, the speech presence probability corresponding to the separation estimation signal is obtained. The separation matrix corresponding to the previous round of sound source separation is updated based on the multiple speech presence probabilities. Sound source separation is then performed on multiple sound acquisition signals within the current observation window based on the updated separation matrix, obtaining multiple separation estimation signals corresponding to the current round of sound source separation. This sound source separation method performs multiple rounds of sound source separation on the sound acquisition signals based on the observation window. In each round of sound source separation, the separation coefficients of each sound source are adjusted specifically based on the speech presence probability of the separation estimation signal obtained in the previous round of sound source separation, obtaining the separation matrix corresponding to the current round of sound source separation, thus achieving the current round of sound source separation. This process is repeated multiple times to enhance the sound source separation effect. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating a sound source separation method according to an exemplary embodiment of the present disclosure;

[0043] Figure 2 This is a flowchart illustrating another sound source separation method according to an exemplary embodiment of the present disclosure;

[0044] Figure 3 This is a flowchart illustrating another sound source separation method according to an exemplary embodiment of the present disclosure;

[0045] Figure 4 This is a block diagram illustrating a sound source separation device according to an exemplary embodiment of the present disclosure;

[0046] Figure 5 This is a schematic diagram of the structure of a vehicle according to an exemplary embodiment of the present disclosure;

[0047] Figure 6 This is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0048] Embodiments of this disclosure are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure.

[0049] Figure 1 This is a flowchart illustrating a sound source separation method according to an exemplary embodiment of the present disclosure, such as... Figure 1 As shown, the sound source separation method includes the following steps:

[0050] S101, based on the separation matrix corresponding to the previous round of sound source separation, perform sound source separation on multiple sound acquisition signals in the previous observation window to obtain multiple separation estimation signals corresponding to the previous round of sound source separation.

[0051] The sound source separation method of this disclosure can be executed by the sound source separation device provided in this disclosure, which can be installed in an electronic device, such as a vehicle, a mobile phone, or an electronic device in the cloud.

[0052] The sound source separation method of this disclosure can be applied to in-vehicle voice interaction scenarios.

[0053] This embodiment of the disclosure performs multiple rounds of sound source separation on the sound acquisition signal to be separated based on the observation window. Before performing the current round of sound source separation, multiple separation estimation signals obtained from the previous round of sound source separation are acquired.

[0054] In some embodiments, an observation window is set up, and sound source separation is performed by constructing an auxiliary function to implement independent vector analysis (AuxIVA) within the observation window, thereby achieving multi-round sound source separation through sliding window operation.

[0055] In some embodiments, each round of sound source separation corresponds to a separation matrix. In the process of the previous round of sound source separation, the multiple acquired signals in the previous observation window are separated according to the separation matrix corresponding to the previous round of sound source separation to obtain the multiple separation estimation signals corresponding to the previous round of sound source separation.

[0056] It is understandable that in a voice interaction scenario involving multiple sound zones, the observation signal (i.e., the sound acquisition signal) acquired by the sound acquisition device consists of sound sources and noise from multiple sound zones. Sound source separation aims to separate the sound sources from multiple sound zones in the sound acquisition signal, i.e., blind source separation (BSS). The output signal (i.e., the separation estimate signal) is correlated with the aforementioned sound sources, and the total number of separation estimate signals is equal to the total number of sound sources.

[0057] S102, for each separation estimation signal, obtain the probability of speech presence corresponding to the separation estimation signal.

[0058] In this embodiment of the disclosure, for each separation estimation signal obtained in the previous round of sound source separation, the probability of speech presence corresponding to the separation estimation signal is obtained. The separation estimation signal can be obtained by separating the sound acquisition signal within an observation window. Therefore, the probability of speech presence can be understood as the proportion of the duration in which speech exists in the separation estimation signal within the length of an observation window. The length of the observation window can be set as needed, for example, it can be set to the duration of 3 frames.

[0059] For example, if the length of an observation window is 3 frames, and there is speech in only one frame of the separated estimated signal, then the probability of speech in the separated estimated signal is 1 / 3.

[0060] S103, update the separation matrix corresponding to the previous round of sound source separation based on the probability of the existence of multiple speech sources.

[0061] In this embodiment, the separation matrix corresponding to the previous round of sound source separation is updated based on the speech presence probabilities of the multiple separation estimation signals corresponding to the previous round of sound source separation, thus obtaining the separation matrix corresponding to the current round of sound source separation. This disclosure determines whether speech exists in the corresponding sound acquisition signal based on the speech presence probabilities of the separation estimation signals. The speech presence information of the sound acquisition signal within the previous observation window is used as reference information when updating the separation matrix, and the separation coefficients corresponding to different sound sources are updated specifically to enhance the sound source separation effect of the separation matrix.

[0062] S104, based on the updated separation matrix, perform sound source separation on multiple sound acquisition signals within the current observation window to obtain multiple separation estimation signals corresponding to the sound source separation in this round.

[0063] In this embodiment of the disclosure, multiple sound acquisition signals within the current observation window are separated into sound sources based on the updated separation matrix (i.e., the separation matrix corresponding to the current round of sound source separation), thereby obtaining multiple separation estimation signals corresponding to the current round of sound source separation. Thus, the multiple separation estimation signals corresponding to the current round of sound source separation can be used to update the separation matrix for the next round of sound source separation. Based on the update of the separation matrix, multiple rounds of sound source separation are iteratively completed.

[0064] It should be noted that the acquisition, storage, and application of user personal information involved in the embodiments of this disclosure comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0065] In summary, the sound source separation method of this disclosure performs sound source separation on multiple sound acquisition signals within the previous observation window based on the separation matrix corresponding to the previous round of sound source separation, obtaining multiple separation estimation signals corresponding to the previous round of sound source separation. For each separation estimation signal, the probability of speech presence corresponding to the separation estimation signal is obtained; the separation matrix is ​​updated based on the multiple speech presence probabilities; and sound source separation is performed on multiple sound acquisition signals within the current observation window based on the updated separation matrix, obtaining multiple separation estimation signals corresponding to the current round of sound source separation. This sound source separation method performs multiple rounds of sound source separation on the sound acquisition signals based on the observation window. In each round of sound source separation, the separation matrix corresponding to the previous round of sound source separation is updated based on the probability of speech presence in the separation estimation signals obtained from the previous round of sound source separation. This adds speech presence information from the sound acquisition signals within the previous observation window when updating the separation matrix, thereby specifically adjusting the separation coefficients of each sound source and enhancing the sound source separation effect of the separation matrix.

[0066] In some embodiments, the total number of sound acquisition signals is the same as the total number of sound sources to improve the speed of sound source separation. For example, in a vehicle voice interaction scenario, the vehicle cabin space is divided into M sound zones, each sound zone corresponding to a sound source. A microphone array or distributed microphones are used in the vehicle cabin to collect speech from all directions inside the vehicle, i.e., the sound sources of each sound zone. If the number of microphones is M, then the number of microphone signals (i.e., sound acquisition signals) is M. When performing sound source separation on the M microphone signals according to the embodiments of this disclosure, M separation estimation signals can be obtained after each round of sound source separation.

[0067] Based on the above embodiments, the sound source separation method of this disclosure further includes: after multiple rounds of sound source separation, selecting the separation estimation signal corresponding to the target sound source from the multiple separation estimation signals obtained, and generating the target estimation signal corresponding to the target sound source.

[0068] In this embodiment of the disclosure, after multiple rounds of sound source separation to complete the sound source separation of the sound acquisition signal, a target estimation signal corresponding to the sound source is generated based on the separation estimation signal related to the same sound source output in each round of sound source separation. For example, the separation estimation signal corresponding to the target sound source is selected from the multiple separation estimation signals obtained in each round of sound source separation, and the multiple separation estimation signals corresponding to the target sound source are combined into a target estimation signal based on time sequence.

[0069] For example, in a vehicle voice interaction scenario, a microphone array (M microphones) in the vehicle cabin collects passenger voices in M ​​frequency bands. Each microphone signal includes passenger voices in M ​​frequency bands. Real-time sound source separation is performed according to the sound source separation method of this embodiment. Based on a preset observation window length, a first round of sound source separation is performed on the collected microphone signals, resulting in M ​​separation estimation signals output from the first round. The speech presence probability corresponding to each of these M separation estimation signals is calculated. The separation matrix is ​​updated based on the speech presence probability. The observation window is slid, and the next round of sound source separation continues based on the updated separation matrix, resulting in the M separation estimation signals output from this round of sound source separation, until the passenger voice ends. One separation estimation signal related to the same passenger voice (i.e., the same sound source) from the multiple separation estimation signals output from each round of sound source separation is selected. This separation estimation signal is used to generate a target estimation signal corresponding to the passenger voice, thereby enabling voice content recognition and voice interaction. Furthermore, during the sound source separation process, there may be situations where passengers in one or more audio regions do not speak during a certain period of time. In such cases, this disclosure can adaptively update the separation matrix by calculating the speech presence probability of the separation estimation signal obtained during that period of time, thereby enhancing the sound source separation effect.

[0070] Therefore, a complete estimated signal (i.e., target estimated signal) corresponding to any sound source can be obtained. This estimated signal can be used to replace the sound source input to subsequent speech recognition modules to obtain the user's complete speech content.

[0071] Based on the above embodiments, such as Figure 2 As shown, step S103, "updating the separation matrix corresponding to the previous round of sound source separation based on the existence probabilities of multiple speech sounds," may include the following steps:

[0072] S201, obtain multiple first auxiliary functions corresponding to the previous round of sound source separation. The first auxiliary function is the covariance matrix corresponding to multiple sound acquisition signals in the previous observation window.

[0073] This embodiment of the disclosure implements sound source separation based on AuxIVA, and the update of the separation matrix is ​​achieved based on auxiliary functions. Multiple first auxiliary functions corresponding to the previous round of sound source separation are obtained. Each first auxiliary function is a covariance matrix corresponding to multiple sound acquisition signals within the previous observation window, used to assist in solving the separation matrix. The separation matrix is ​​updated by updating the auxiliary functions, wherein the total number of first auxiliary functions is the same as the total number of sound sources and also the same as the total number of separation estimation signals.

[0074] S202, update the first auxiliary function according to the probability of speech existence to obtain multiple second auxiliary functions corresponding to the sound source separation in this round. The second auxiliary function is the covariance matrix corresponding to multiple sound acquisition signals in the current observation window.

[0075] In this embodiment of the disclosure, each first auxiliary function corresponds to a separation estimation signal output from the previous round of sound source separation, and therefore each auxiliary function corresponds to a speech presence probability (i.e., the speech presence probability of the aforementioned separation estimation signal).

[0076] Based on Formula 1, and according to the first auxiliary function corresponding to the preset observation window length, forgetting factor, and speech presence probability, the corresponding second auxiliary function is obtained, where Formula 1 is:

[0077]

[0078] Where L is the observation window length, α is the forgetting factor, and y k (ω, t) is the separation estimation signal corresponding to the previous round of sound source separation, V k (ω, τ) is the second auxiliary function, V k (ω, τ-L) is the first auxiliary function, k = 1, 2, ..., M, and X(ω, τ) is the sound acquisition signal x. i A matrix of size M*1 (ω, τ) is formed, i = 1, 2, ..., M, where M is the total number of sound acquisition signals and the total number of sound sources, p k The separation estimation signal y corresponds to the previous round of sound source separation. k The probability of the existence of a speech sound at (ω, τ), where ω represents frequency and τ represents time.

[0079] Based on a preset time window, a forgetting factor, and the proportion of speech in the separation estimation signal output from the previous round of sound source separation (i.e., the probability of speech presence), the auxiliary function is updated in real time to meet the needs of online sound source separation.

[0080] S203 generates an updated separation matrix based on multiple second auxiliary functions.

[0081] In this embodiment of the disclosure, a separation matrix (i.e., an updated separation matrix) for this round of sound source separation is obtained based on a plurality of updated second auxiliary functions.

[0082] Therefore, the first auxiliary function is updated based on the probability of multiple speech occurrences to obtain the second auxiliary function under different speech proportions. This allows the second auxiliary function to adaptively adjust to the changes in the probability of speech occurrence, dynamically adjusting the separation coefficients of different sound sources in the separation matrix and enhancing the sound source separation effect.

[0083] Based on the above embodiments, such as Figure 3 As shown, step S203, "generating the updated separation matrix based on multiple second auxiliary functions," may include the following steps:

[0084] S301, generate the corresponding separation sub-matrix based on multiple second auxiliary functions. The separation sub-matrix is ​​a 1*M matrix.

[0085] In this embodiment of the disclosure, a separation sub-matrix is ​​generated based on Formula 2 and Formula 3: the separation sub-matrix corresponding to the sound source separation in this round is generated based on the separation matrix corresponding to the previous round of sound source separation and the second auxiliary function.

[0086] Formula 2 is:

[0087] w k ′(ω,τ)=(W(ω,τ-L)V k (ω,τ)) -1 e k ;

[0088] Formula 3 is:

[0089]

[0090] Where W(ω, τ-L) is the separation matrix corresponding to the previous round of sound source separation, w k ′(ω, τ) is the intermediate matrix, w k (ω, τ) is the separating submatrix, e k Let e ​​be an M*1 matrix. k The k-th element is 1, and the rest are 0. k ′H (ω, τ) is w k The conjugate transpose of ′(ω,τ).

[0091] S302, based on multiple updated separation sub-matrices, obtain the updated separation matrix, which is an M*M matrix.

[0092] In this embodiment of the disclosure, the separation matrix is ​​composed of separation sub-matrices, that is, the updated separation matrix is ​​composed of M updated separation sub-matrices, W(ω, τ)=[w1; w2; ......; w M ] T .

[0093] Therefore, based on the updated separation matrix, this round of sound source separation is performed, and the M sound acquisition signals X(ω, τ)=[x1; x2; ......; w are analyzed within the current observation window. M ] T Sound source separation yields multiple separation estimation signals Y(ω, τ) = [y1; y2; ...; y...]. M ] T , where Y(ω,τ)=W(ω,τ)X(ω,τ).

[0094] Based on the above embodiments, the "obtaining the probability of speech presence corresponding to the separation estimation signal" in step S102 can be achieved through the following process: performing speech endpoint detection on the separation estimation signal, detecting the start and end positions of speech in the separation estimation signal, obtaining the effective speech segments in the separation estimation signal, calculating the proportion of the effective speech segments in the entire audio segment corresponding to the separation estimation signal, and obtaining the probability of speech presence corresponding to the separation estimation signal.

[0095] This allows for the accurate acquisition of the speech presence probability in each separated estimated signal, enabling adaptive updates to the first auxiliary function to generate the separation matrix for this round of sound source separation.

[0096] Figure 4 This is a block diagram illustrating a sound source separation device according to an exemplary embodiment of the present disclosure, such as... Figure 4 As shown, the sound source separation device 400 includes: a first acquisition module 401, a second acquisition module 402, an update module 403, and a separation module 404.

[0097] The first acquisition module 401 is used to perform sound source separation on multiple sound acquisition signals in the previous observation window according to the separation matrix corresponding to the previous round of sound source separation, so as to obtain multiple separation estimation signals corresponding to the previous round of sound source separation.

[0098] The second acquisition module 402 is used to acquire the probability of speech presence corresponding to each separation estimation signal.

[0099] The update module 403 is used to update the separation matrix corresponding to the previous round of sound source separation based on the probability of the existence of multiple speech sources.

[0100] The separation module 404 is used to perform sound source separation on multiple sound acquisition signals in the current observation window according to the updated separation matrix, and obtain multiple separation estimation signals corresponding to the sound source separation in this round.

[0101] In this embodiment of the disclosure, the sound acquisition signal is obtained by the sound acquisition device from sound sources in multiple sound ranges, and the total number of sound acquisition signals is the same as the total number of sound sources.

[0102] In this embodiment of the disclosure, the update module 403 is further configured to: obtain multiple first auxiliary functions corresponding to the previous round of sound source separation, wherein the first auxiliary function is the covariance matrix corresponding to multiple sound acquisition signals in the previous observation window; update the corresponding first auxiliary function according to the speech existence probability to obtain multiple second auxiliary functions corresponding to the current round of sound source separation, wherein the second auxiliary function is the covariance matrix corresponding to multiple sound acquisition signals in the current observation window; and generate an updated separation matrix according to the multiple second auxiliary functions.

[0103] In this embodiment of the disclosure, the update module 403 is further configured to: based on Formula 1, obtain the corresponding second auxiliary function according to the first auxiliary function corresponding to the preset observation window length, forgetting factor and speech existence probability;

[0104] Formula 1 is:

[0105]

[0106] Where L is the observation window length, α is the forgetting factor, and y k (ω, t) is the separation estimation signal corresponding to the previous round of sound source separation, V k (ω, τ) is the second auxiliary function, V k (ω, τ-L) is the first auxiliary function, k = 1, 2, ..., M, and X(ω, τ) is the sound acquisition signal x. i A matrix of M*1 (ω, τ) is formed, i = 1, 2, ..., M, where M is the total number of sound acquisition signals and the total number of sound sources, p k The separation estimation signal y corresponds to the previous round of sound source separation. k The probability of the existence of a speech sound at (ω, τ), where ω represents frequency and τ represents time.

[0107] In this embodiment of the disclosure, the update module 403 is further configured to: generate a corresponding separation sub-matrix based on Formula 2 and Formula 3 according to multiple second auxiliary functions, wherein the separation sub-matrix is ​​a 1*M matrix;

[0108] Formula 2 is:

[0109] w k ′(ω,τ)=(W(ω,τ-L)V k (ω,τ)) -1 e k ;

[0110] Formula 3 is:

[0111]

[0112] Where W(ω, τ-L) is the separation matrix corresponding to the previous round of sound source separation, w k ′(ω, τ) is the intermediate matrix, w k (ω, τ) is the separating submatrix, e k Let e ​​be an M*1 matrix. k The k-th element is 1, and the rest are 0. k ′H (ω, τ) is w k The conjugate transpose of ′(ω, τ); based on multiple updated separating submatrices, the updated separating matrix is ​​obtained, which is an M*M matrix.

[0113] In this embodiment of the disclosure, the second acquisition module 402 is further configured to: perform speech endpoint detection on the separation estimation signal to acquire valid speech segments containing speech in the separation estimation signal; and obtain the speech existence probability corresponding to the separation estimation signal based on the valid speech segments.

[0114] In this embodiment of the disclosure, the separation module 404 is further configured to: select the separation estimation signal corresponding to the target sound source from the multiple separation estimation signals obtained after multiple rounds of sound source separation; and generate the target estimation signal corresponding to the target sound source based on the separation estimation signal corresponding to the target sound source.

[0115] It should be noted that the above explanation of the sound source separation method embodiments also applies to the sound source separation device of the present disclosure embodiments, and the specific process will not be repeated here.

[0116] In summary, the sound source separation device of this disclosure performs sound source separation on multiple sound acquisition signals within the previous observation window based on the separation matrix corresponding to the previous round of sound source separation, obtaining multiple separation estimation signals corresponding to the previous round of sound source separation. For each separation estimation signal, the probability of speech presence corresponding to the separation estimation signal is obtained; the separation matrix corresponding to the previous round of sound source separation is updated based on the multiple speech presence probabilities; and sound source separation is performed on multiple sound acquisition signals within the current observation window based on the updated separation matrix, obtaining multiple separation estimation signals corresponding to the current round of sound source separation. The sound source separation method of this disclosure performs multiple rounds of sound source separation on the sound acquisition signals based on the observation window. In each round of sound source separation, the separation matrix corresponding to the previous round of sound source separation is updated based on the probability of speech presence in the separation estimation signals obtained from the previous round of sound source separation. This adds speech presence information of the sound acquisition signals within the previous observation window when updating the separation matrix, thereby specifically adjusting the separation coefficients of each sound source to obtain the separation matrix corresponding to the current round of sound source separation, enhancing the separation effect of the separation matrix on sound sources.

[0117] To achieve the above embodiments, this disclosure also proposes a vehicle 500, such as... Figure 5 As shown, the vehicle 500 may specifically include: a sound source separation device 400 as shown in the above embodiment.

[0118] To implement the above embodiments, this disclosure also proposes an electronic device 600, such as... Figure 6 As shown, the electronic device 600 may specifically include: a memory 601, a processor 602, and a computer program stored on the memory 601 and executable on the processor 602. When the processor 602 executes the program, it implements the sound source separation method as shown in the above embodiment.

[0119] To implement the above embodiments, this disclosure also proposes a computer-readable storage medium storing a computer program that is executed by a processor to implement the sound source separation method as shown in the above embodiments.

[0120] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise expressly specified.

[0121] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0122] Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A method for separating sound sources, characterized in that, include: Based on the separation matrix corresponding to the previous round of sound source separation, multiple sound acquisition signals within the previous observation window are separated to obtain multiple separation estimation signals corresponding to the previous round of sound source separation; For each of the separation estimation signals, obtain the speech presence probability corresponding to the separation estimation signal; Update the separation matrix corresponding to the previous round of sound source separation based on the probability of the existence of multiple speech sounds; Based on the updated separation matrix, multiple sound acquisition signals within the current observation window are separated into sound sources to obtain multiple separation estimation signals corresponding to this round of sound source separation.

2. The sound source separation method according to claim 1, characterized in that, The sound acquisition signals are obtained by the sound acquisition device from sound sources in multiple sound ranges, and the total number of sound acquisition signals is the same as the total number of sound sources.

3. The sound source separation method according to claim 1, characterized in that, The step of updating the separation matrix corresponding to the previous round of sound source separation based on the probability of the presence of multiple speech sources includes: Obtain multiple first auxiliary functions corresponding to the previous round of sound source separation. The first auxiliary function is the covariance matrix corresponding to the multiple sound acquisition signals in the previous observation window. The first auxiliary function corresponding to the probability of the speech is updated to obtain multiple second auxiliary functions corresponding to the sound source separation in this round. The second auxiliary function is the covariance matrix corresponding to multiple sound acquisition signals in the current observation window. The updated separation matrix is ​​generated based on a plurality of the second auxiliary functions.

4. The sound source separation method according to claim 3, characterized in that, The step of updating the first auxiliary function based on the probability of the speech presence to obtain multiple second auxiliary functions corresponding to the current round of sound source separation includes: Based on Formula 1, the corresponding second auxiliary function is obtained according to the first auxiliary function corresponding to the preset observation window length, forgetting factor and the probability of the speech presence; Formula 1 is: Where L is the observation window length, α is the forgetting factor, and y k (ω, t) is the separation estimation signal corresponding to the previous round of sound source separation, V k (ω, τ) is the second auxiliary function, V k (ω, τ-L) is the first auxiliary function, k = 1, 2, ..., M, and X(ω, τ) is the sound acquisition signal x. i A matrix of M*1 (ω, τ) is formed, i = 1, 2, ..., M, where M is the total number of sound acquisition signals and the total number of sound sources, p k The separation estimation signal y corresponding to the previous round of sound source separation. k The probability of the existence of a speech sound at (ω, τ), where ω represents frequency and τ represents time.

5. The sound source separation method according to claim 4, characterized in that, The step of generating the updated separation matrix based on multiple second auxiliary functions includes: Based on Formulas 2 and 3, the corresponding separating sub-matrix is ​​updated according to multiple second auxiliary functions, wherein the separating sub-matrix is ​​a 1*M matrix; Formula 2 is as follows: w k ′(ω,τ)=(W(ω,τ-L)V k (oh, t)) -1 e k ; Formula 3 is as follows: Where W(ω, τ-L) is the separation matrix corresponding to the previous round of sound source separation, w k ′(ω, τ) is the intermediate matrix, w k (ω, τ) is the separating submatrix, e k For an M*1 matrix, the e k The k-th element is 1, and the rest are 0. k ′ H (ω, τ) is w k The conjugate transpose of ′(ω, τ); The updated separation matrix is ​​obtained based on multiple updated separation sub-matrices, and the updated separation matrix is ​​an M*M matrix.

6. The sound source separation method according to claim 1, characterized in that, The step of obtaining the speech presence probability corresponding to the separated estimated signal includes: Speech endpoint detection is performed on the separation estimation signal to obtain valid speech segments containing speech in the separation estimation signal; Based on the effective speech segment, the probability of the speech segment corresponding to the separation estimation signal is obtained.

7. The sound source separation method according to claim 1, characterized in that, Also includes: Select the separation estimation signal corresponding to the target sound source from the multiple separation estimation signals obtained after multiple rounds of sound source separation; Based on the separation estimation signal corresponding to the target sound source, a target estimation signal corresponding to the target sound source is generated.

8. A sound source separation device, characterized in that, include: The first acquisition module is used to perform sound source separation on multiple sound acquisition signals in the previous observation window based on the separation matrix corresponding to the previous round of sound source separation, and obtain multiple separation estimation signals corresponding to the previous round of sound source separation. The second acquisition module is used to acquire the speech presence probability corresponding to each of the separation estimation signals; The update module is used to update the separation matrix corresponding to the previous round of sound source separation based on the probability of the existence of multiple speech sources; The separation module is used to separate the sound sources of multiple sound acquisition signals in the current observation window based on the updated separation matrix, and obtain multiple separation estimation signals corresponding to the sound source separation in this round.

9. A vehicle, characterized in that, include: The sound source separation device as described in claim 8.

10. An electronic device, characterized in that, include: The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the sound source separation method as described in any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the sound source separation method as described in any one of claims 1-8.

Images