Sound source signal estimation device, sound source signal estimation method, program

By employing algorithms that bypass weighted covariance matrix calculations and utilize approximate separated signals and auxiliary separation matrices, the method addresses computational complexity issues in sound source tracking, enabling efficient real-time operation in devices with multiple microphones.

JP2024160451APending Publication Date: 2024-11-14NIPPON TELEGRAPH & TELEPHONE CORP +1
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2023075445
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-01
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

Existing online sound source separation algorithms for hearing aids face computational complexity issues that hinder real-time operation as the number of microphones increases, making it difficult to perform sound source tracking effectively.

Method used

The proposed method reduces computational complexity by omitting the calculation of weighted covariance matrices and using alternative algorithms (Algorithm 1 and Algorithm 2) that update separation filters based on approximate separated signals and auxiliary separation matrices, achieving a computational complexity of O(K^2) instead of O(K^3).

Benefits of technology

This approach enables high-speed sound source tracking even with a large number of microphones, ensuring real-time operation in devices with limited computing power.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024160451000001_ABST
    Figure 2024160451000001_ABST
Patent Text Reader

Abstract

To provide a sound source tracking technology that operates at high speed even when there are many microphones.SOLUTION: A device includes: a first calculation unit that calculates an approximate separation signal ^y(f, t), which is an approximation signal of the separation signal y(f, t) from an observed signal x(f, t), using a separation matrix W(f, t-1); a second calculation unit that updates a value rk(t), using the approximate separation signal ^y(f, t); a third calculation unit that updates a weights Gλ(f, t) for k=λ, using the approximate separation signal ^y(f, t) and the rk(t), and updates a weight Gk(f, t) and the weight gk(f, t) for k≠λ; and a fourth calculation unit that updates a separation matrix W(f, t) and the separation signal y(f, t), using the weights Gk(f, t) and the weight gk(f, t).SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an online sound source separation technique for sequentially separating signals from pre-mixed sound sources contained in a mixed acoustic signal observed using multiple microphones. [Background technology]

[0002] The signal collected by the microphone built into the hearing aid is mixed with multiple voice and acoustic signals. Online source separation technology is actively researched to enable hearing aid users to distinguish between multiple voice and acoustic signals with high accuracy and in real time. Online source separation technology is a technology that sequentially separates the signal from the sound source before mixing (hereinafter referred to as the sound source signal) from the mixed acoustic signal observed using multiple microphones (hereinafter simply referred to as the observed signal).

[0003] In the following, online sound source separation is formulated and explained as a problem in the short-time Fourier transform domain.

[0004] Consider a situation where signals from K sound sources are observed using K microphones (K is an integer equal to or greater than 2). However, it is assumed that the K sound sources may move. With f as an index representing a frequency bin and t as an index representing a time frame, the observed signal x(f, t) (f=1, …, F, t=1, …, T) in the short-time Fourier transform domain is expressed as follows:

number

number

number

[0005] y k (f, t) is the k-th source signal s k Let (f, t) be the estimated signal (hereinafter referred to as the k-th separated signal), and y(f, t) be the vector defined by equation (4) (hereinafter referred to as the separated signal).

number

number

number

[0006] Online source separation is formulated as a problem of sequentially estimating a separation matrix in the current time frame using observed signals in the current time frame and observed signals in previous time frames when K sound sources are moving. In particular, when only one sound source is moving and the other K-1 sound sources are not moving, this is called source tracking. The one moving sound source is called the target sound source, and the K-1 sound sources that do not move are called non-target sound sources. If the λ-th sound source (λ satisfies 1≦λ≦K) is considered to be the target sound source, the steering vector a of the λ-th sound source is λ (f, t) is a time-varying vector of the other sound sources, a k(f, t) (k ≠ λ) does not change over time. Therefore, the steering vector a of the non-target sound source k (f, t) (k ≠ λ) may be known.

[0007] An algorithm for realizing online sound source separation and sound source tracking is, for example, the algorithm disclosed in Non-Patent Document 1. The algorithm in Non-Patent Document 1 is shown below.

[0008] <<Algorithm in Non-Patent Document 1>> ------------------------------------------------------------------ 1: for t=1, …, T 2: Obtain the observed signal x(f, t) (f=1, …, F). 3: Calculate the approximate separated signal ^y(f, t) (f=1, …, F), which is an approximation of the separated signal y(f, t), using the following formula:

number

number

number

number

number

number

number

number

[0009] [Non-Patent Document 1] T. Nakashima and N. Ono, “Inverse-free online independent vector analysis with flexible iterative source steering,” in Proc. Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), pp.750-754, 2022. Summary of the Invention [Problem to be solved by the invention]

[0010] However, as can be seen from the calculation of the weighted covariance matrix in step 4, the computational complexity of the algorithm in Non-Patent Document 1 is O(K 3 ) and in devices with limited computing power such as hearing aids, as the number of microphones increases it becomes difficult to guarantee real-time operation.

[0011] Therefore, an object of the present invention is to provide a sound source tracking technology that operates at high speed even when a large number of microphones are used. [Means for solving the problem]

[0012] In one aspect of the present invention, x(f, t) (f=1, ..., F, t=T'+1, ..., T) (where f is an index representing a frequency bin, t is an index representing a time frame, and T' is an integer equal to or greater than 1) is defined as an observed signal of a mixture of sounds from K sound sources observed using K microphones (where K is an integer equal to or greater than 2), and y k (f, t)(k=1, …, K, f=1, …, F, t=T'+1, …, T) is the k-th separated signal, which is the estimated signal of the signal from the k-th sound source (hereafter referred to as the k-th sound source signal), and y(f, t)=[y1(f, t), …, y K (f, t)] T (f=1, …, F, t=T'+1, …,T) is the separation signal, w k (f, t) (k=1, …, K, f=1, …, F, t=T'+1, …,T) is the k-th separation filter for separating the k-th source signal, W(f, t)=[w1(f, t), …, w K (f, t)] H(f=1, …, F, t=T'+1, …, T) is a separation matrix, and among the K sound sources, the λ-th sound source (where λ satisfies 1≦λ≦K) moves, and the K-1 sound sources other than the λ-th sound source do not move. The initial value W(f, T') (f=1, …, F) of the separation matrix W(f, t) is assumed to be obtained. The method includes an initialization unit that initializes the time frame t by t<-T'+1, a first calculation unit that calculates an approximate separation signal ^y(f, t) (f=1, …, F) that is an approximation signal of the separation signal y(f, t) from the observation signal x(f, t) (f=1, …, F) using the separation matrix W(f, t-1) (f=1, …, F), and a second calculation unit that calculates a value r k (t) (k=1, …, K), and the second calculation part updates the approximate separated signal ^y(f, t) (f=1, …, F) and the value r k (t) (k=1, …, K), for k=λ, the weight G λ (f, t) (f=1, …, F), and for k≠λ, update the weight G k (f, t) (f=1, …, F) and weight g k A third calculation part updates (f, t) (f=1, …, F) and the weights G k (f, t) (k=1, …, K, f=1, …, F) and weight g k and a fourth calculation unit that updates the separation matrix W(f, t) (f=1, ..., F) and the separated signals y(f, t) (f=1, ..., F) using (f, t) (k=1, ..., λ-1, λ+1, ..., K, f=1, ..., F), and a termination condition determination unit that updates the time frame t by t<-t+1 and terminates the processing if t>T is satisfied, and returns to the processing of the first calculation unit otherwise.

[0013] In one aspect of the present invention, x(f, t) (f=1, ..., F, t=T'+1, ..., T) (where f is an index representing a frequency bin, t is an index representing a time frame, and T' is an integer equal to or greater than 1) is defined as an observed signal of a mixture of sounds from K sound sources observed using K microphones (where K is an integer equal to or greater than 2), and y k(f, t)(k=1, …, K, f=1, …, F, t=T'+1, …, T) is the k-th separated signal, which is the estimated signal of the signal from the k-th sound source (hereafter referred to as the k-th sound source signal), and y(f, t)=[y1(f, t), …, y K (f, t)] T (f=1, …, F, t=T'+1, …,T) is the separation signal, w k (f, t) (k=1, …, K, f=1, …, F, t=T'+1, …,T) is the k-th separation filter for separating the k-th source signal, W(f, t)=[w1(f, t), …, w K (f, t)] H The separation matrix is ​​(f=1, …, F, t=T'+1, …, T), and among the K sound sources, the λ-th sound source (where λ satisfies 1≦λ≦K) moves, and the K-1 sound sources other than the λ-th sound source do not move. The initial value W(f, T') (f=1, …, F) of the separation matrix W(f, t) is assumed to be obtained. The separation matrix W(f, t) includes an initialization unit that initializes the time frame t by t<-T'+1, a first calculation unit that calculates an approximate separation signal ^y(f, t) (f=1, …, F) that is an approximation signal of the separation signal y(f, t) from the observation signal x(f, t) (f=1, …, F) using an auxiliary separation matrix ~Q(f, t-1) (f=1, …, F) that satisfies W(f, t-1)=~Q(f, t-1)W(f, T'), and a calculation unit that calculates the approximate separation signal ^y(f, t) (f=1, …, F) to find the value r k (t) (k=1, …, K), and the second calculation part updates the approximate separated signal ^y(f, t) (f=1, …, F) and the value r k (t) (k=1, …, K), for k=λ, the weight G λ (f, t) (f=1, …, F), and for k≠λ, update the weight G k (f, t) (f=1, …, F) and weight g k A third calculation part updates (f, t) (f=1, …, F) and the weights G k (f, t) (k=1, …, K, f=1, …, F) and weight g kand a fourth calculation unit that updates the auxiliary separation matrix ∼Q(f, t) (f=1, ..., F) and the separated signals y(f, t) (f=1, ..., F) using (f, t) (k=1, ..., λ-1, λ+1, ..., K, f=1, ..., F), and a termination condition determination unit that updates the time frame t by t<-t+1 and terminates the processing if t>T is satisfied, and returns to the processing of the first calculation unit otherwise. Effect of the Invention

[0014] According to the present invention, it is possible to estimate a sound source signal at high speed when a large number of microphones are used. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a block diagram showing a configuration of a sound source signal estimation device 100 / 200. [Diagram 2] 4 is a flowchart showing an operation of the sound source signal estimation apparatus 100. [Diagram 3] 4 is a flowchart showing the operation of the sound source signal estimation apparatus 200. [Figure 4] FIG. 2 is a diagram illustrating an example of a functional configuration of a computer that realizes each device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, an embodiment of the present invention will be described in detail. Note that components having the same functions are given the same reference numbers and duplicated explanations will be omitted.

[0017] Before describing each embodiment, the notation used in this specification will be described.

[0018] ^ (caret) represents a superscript. For example, x y^z Yes z is a superscript to x, and x y^z Yes z is a subscript to x. Also, _ (underscore) represents a subscript. For example, xy_z Yes z is a superscript to x, and x y_z Yes z is a subscript on x.

[0019] The superscripts "^" and "~" such as ^x and ~x for a certain letter x should actually be written directly above the "x", but due to the constraints of the description in the specification, they are written as ^x and ~x.

[0020] <Technical background> We consider reducing the amount of calculations by not calculating the weighted covariance matrix in the algorithm for realizing sound source tracking in Non-Patent Document 1. Both Algorithm 1 and Algorithm 2 shown below are equivalent to the algorithm for realizing sound source tracking in Non-Patent Document 1, and are algorithms that do not explicitly calculate the weighted covariance matrix. Since Algorithm 1 and Algorithm 2 do not explicitly calculate the weighted covariance matrix, the amount of calculations is O(K 2 )

[0021] First, we present Algorithm 1. Note that in both Algorithm 1 and Algorithm 2, it is assumed that the separation matrix W(f, T') (f=1, ..., F) has been obtained. <<Algorithm 1>> ------------------------------------------------------------------ 1: for t=T'+1, …, T 2: Obtain the observed signal x(f, t) (f=1, …, F). 3: Calculate the approximate separated signal ^y(f, t) (f=1, …, F) using the following formula:

number

number

number

number

number

number

[0022] Next, Algorithm 2 is shown. <<Algorithm 2>> ------------------------------------------------------------------ 1: for t=T'+1, …, T 2: Obtain the observed signal x(f, t) (f=1, …, F). 3: Calculate the approximate separated signal ^y(f, t) (f=1, …, F) using the following formula:

number

number

number

number

number

number

number

number

[0023] The sound source signal estimation device 100 estimates a separated signal y(f, t)=[y1(f, t), ..., y K (f, t)] T (f=1, …, F, t=T'+1, …,T) (where y k (f, t) (k=1, ..., K, f=1, ..., F, t=T'+1, ..., T) is the k-th separated signal, which is an estimated signal of the signal from the k-th sound source (hereinafter referred to as the k-th sound source signal). At that time, the sound source signal estimation device 100 estimates the separation matrix W(f, t)=[w1(f, t), ..., w K (f, t)] H (f=1, …, F, t=T'+1, …,T) (where w k (f, t) (k=1, …, K, f=1, …, F, t=T'+1, …, T) is the k-th separation filter for separating the k-th excitation signal. Among the K excitation sources, the λ-th excitation source (where λ satisfies 1≦λ≦K) moves, and the K-1 excitation sources other than the λ-th excitation source do not move.

[0024] Hereinafter, the source signal estimation device 100 will be described with reference to Figs. 1 and 2. Fig. 1 is a block diagram showing the configuration of the source signal estimation device 100. Fig. 2 is a flowchart showing the operation of the source signal estimation device 100. As shown in Fig. 1, the source signal estimation device 100 includes an initialization section 110, a first calculation section 120, a second calculation section 130, a third calculation section 140, a fourth calculation section 150, a termination condition determination section 160, and a recording section 190. The recording section 190 is a component section that appropriately records information required for the processing of the source signal estimation device 100. The recording section 190 records, for example, an initial value W(f, T') (f=1, ..., F) of a separation matrix W(f, t) obtained in advance by a predetermined estimation method.

[0025] The operation of the sound source signal estimation device 100 will be described with reference to FIG.

[0026] In S110, the initialization unit 110 initializes a time frame t such that t<-T'+1.

[0027] In S120, the first calculation unit 120 calculates approximate separation signals ^y(f,t) (f=1, ...,F) which are approximation signals of the separation signals y(f,t) from the observed signals x(f,t) (f=1, ...,F) using the separation matrix W(f,t-1) (f=1, ...,F). The first calculation unit 120 calculates the approximate separation signals ^y(f,t) (f=1, ...,F) by the following equation.

number

[0028] In S130, the second calculation unit 130 calculates a value r k (t) (k=1, ..., K). The second calculation unit 130 updates the value r k(t) Update (k=1, …, K).

number

number

number

[0029] In S150, the fourth calculation unit 150 calculates the weight G k (f, t) (k=1, …, K, f=1, …, F) and weight g k The fourth calculation unit 150 updates the separation matrix W(f, t) (f=1, ..., F) and the separated signals y(f, t) (f=1, ..., F) using (f, t) (k=1, ..., λ-1, λ+1, ..., K, f=1, ..., F). The fourth calculation unit 150 updates the separation matrix W(f, t) (f=1, ..., F) and the separated signals y(f, t) (f=1, ..., F) by the following equation.

number

number

[0030] According to an embodiment of the present invention, it is possible to estimate a sound source signal at high speed when there are a large number of microphones.

[0031] <Second embodiment> In this embodiment, a form of sound source tracking based on algorithm 2 will be described.

[0032] The sound source signal estimation device 200 estimates a separated signal y(f, t)=[y1(f, t), ..., y K (f, t)] T (f=1, …, F, t=T'+1, …,T) (where y k (f, t) (k=1, ..., K, f=1, ..., F, t=T'+1, ..., T) is the k-th separated signal, which is an estimated signal of the signal from the k-th sound source (hereinafter referred to as the k-th sound source signal). At that time, the sound source signal estimation device 200 estimates the separation matrix W(f, t)=[w1(f, t), ..., w K (f, t)] H (f=1, …, F, t=T'+1, …,T) (where w kInstead of (f, t) (k=1, …, K, f=1, …, F, t=T'+1, …, T) which is the k-th separation filter for separating the k-th excitation signal), an auxiliary separation matrix ~Q(f, t) (f=1, …, F, t=T'+1, …, T) which satisfies W(f, t)=~Q(f, t)W(f, T') is used. Note that, among the K excitation sources, the λ-th excitation source (where λ satisfies 1≦λ≦K) moves, and the K-1 excitation sources other than the λ-th excitation source do not move.

[0033] Hereinafter, the source signal estimation device 200 will be described with reference to Figs. 1 and 3. Fig. 1 is a block diagram showing the configuration of the source signal estimation device 200. Fig. 3 is a flowchart showing the operation of the source signal estimation device 200. As shown in Fig. 1, the source signal estimation device 200 includes an initialization unit 110, a first calculation unit 220, a second calculation unit 130, a third calculation unit 140, a fourth calculation unit 250, a termination condition determination unit 160, and a recording unit 190. The recording unit 190 is a component that appropriately records information required for the processing of the source signal estimation device 200. The recording unit 190 records, for example, an initial value W(f, T') (f=1, ..., F) of a separation matrix W(f, t) obtained in advance by a predetermined estimation method. The source signal estimation apparatus 200 differs from the source signal estimation apparatus 100 only in that it includes a first calculation unit 220 instead of the first calculation unit 120 and a fourth calculation unit 250 instead of the fourth calculation unit 150 .

[0034] The operation of the source signal estimation apparatus 200 will be described with reference to Fig. 3. Here, only the operations of the first calculation unit 220 and the fourth calculation unit 250, which are different between the source signal estimation apparatus 200 and the source signal estimation apparatus 100, will be described.

[0035] In S220, the first calculation unit 220 calculates an approximate separated signal ^y(f,t) (f=1, ...,F) which is an approximation signal of the separated signal y(f,t) from the observed signal x(f,t) (f=1, ...,F) using an auxiliary separation matrix ~Q(f,t-1) (f=1, ...,F) which satisfies W(f,t-1)=~Q(f,t-1)W(f,T'). The first calculation unit 220 calculates the approximate separated signal ^y(f,t) (f=1, ...,F) by the following formula.

number

[0036] In S250, the fourth calculation unit 250 calculates the weight G k (f, t) (k=1, …, K, f=1, …, F) and weight g k The fourth calculation unit 250 updates the auxiliary separation matrix ∼Q(f, t) (f=1, ..., F) and the separated signals y(f, t) (f=1, ..., F) using (f, t) (k=1, ..., λ-1, λ+1, ..., K, f=1, ..., F). The fourth calculation unit 250 updates the auxiliary separation matrix ∼Q(f, t) (f=1, ..., F) and the separated signals y(f, t) (f=1, ..., F) by the following equation.

number

number

number

number

[0037] <Additional Notes> The processing of each unit of each of the above-mentioned devices may be realized by a computer, in which case the processing contents of the functions that each device should have are described by a program. Then, this program is loaded into the recording unit 2020 of the computer 2000 shown in Fig. 4, and the arithmetic processing unit 2010, the input unit 2030, the output unit 2040, the auxiliary recording unit 2025, etc. are operated, whereby the processing functions of each of the above-mentioned devices are realized on the computer.

[0038] The device of the present invention, for example, as a single hardware entity, has an input unit capable of inputting a signal from outside the hardware entity, an output unit capable of outputting a signal to the outside of the hardware entity, a communication unit to which a communication device (e.g., a communication cable) capable of communicating with the outside of the hardware entity can be connected, a CPU (Central Processing Unit, which may have a cache memory or a register, etc.) which is an arithmetic processing unit, RAM and ROM which are memories, an external storage device which is a hard disk, and a bus which connects the input unit, output unit, communication unit, CPU, RAM, ROM, and external storage device so that data can be exchanged between them. If necessary, the hardware entity may be provided with a device (drive) capable of reading and writing a recording medium such as a CD-ROM. A physical entity equipped with such hardware resources is, for example, a general-purpose computer.

[0039] The external storage device of the hardware entity stores the programs required to realize the above-mentioned functions and the data required in the processing of these programs (not limited to the external storage device, but for example the programs may be stored in a ROM, which is a read-only storage device). Data obtained by the processing of these programs is stored appropriately in the RAM, the external storage device, etc.

[0040] In the hardware entity, each program stored in an external storage device (or ROM, etc.) and data required for processing each program are loaded into memory as necessary, and interpreted, executed, and processed by the CPU as appropriate. As a result, the CPU realizes a predetermined function (each component represented as the above, "unit," "means," etc.). In other words, each component of the embodiment of the present invention may be configured by a processing circuit.

[0041] As described above, when the processing functions of the hardware entities (the devices of the present invention) described in the above embodiments are realized by a computer, the processing contents of the functions that the hardware entities should have are described by a program. Then, by executing this program on a computer, the processing functions of the hardware entities are realized on the computer.

[0042] The program describing the processing contents can be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a non-transitory recording medium, specifically, a magnetic recording device, an optical disk, or the like.

[0043] The program may be distributed, for example, by selling, transferring, lending, etc. portable recording media such as DVDs and CD-ROMs on which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of a server computer and transferring the program from the server computer to other computers via a network.

[0044] A computer that executes such a program, for example, first stores a program recorded on a portable recording medium or a program transferred from a server computer in the auxiliary recording unit 2025, which is a non-transient storage device of the computer. Then, when executing a process, the computer reads the program stored in the auxiliary recording unit 2025, which is a non-transient storage device of the computer, into the recording unit 2020, and executes a process according to the read program. As another execution form of the program, the computer may read the program directly from the portable recording medium into the recording unit 2020 and execute a process according to the program, or may execute a process according to the received program each time a program is transferred from the server computer to the computer. Alternatively, the server computer may not transfer the program to the computer, but may execute the above-mentioned process by a so-called ASP (Application Service Provider) type service that realizes a processing function only by issuing an execution instruction and obtaining a result. Note that the program in this embodiment includes information used for processing by an electronic computer and equivalent to a program (data that is not a direct instruction to a computer but has a nature that specifies the processing of a computer, etc.).

[0045] In addition, in this embodiment, the present device is configured by executing a specific program on a computer, but at least a part of the processing contents may be realized by hardware.

[0046] The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention.

Claims

1. Let x(f, t)(f=1, …, F, t=T'+1, …, T) (where f is an index representing a frequency bin, t is an index representing a time frame, and T' is an integer equal to or greater than 1) be the observed signal of a mixture of sounds from K sound sources observed using K microphones (where K is an integer equal to or greater than 2), and y k Let (f, t)(k=1, …, K, f=1, …, F, t=T'+1, …, T) be the k-th separated signal, which is the estimated signal of the signal from the k-th sound source (hereafter referred to as the k-th sound source signal), and y(f, t)=[y 1 (f, t),…, y K (f, t)] T (f=1, …, F, t=T'+1, …,T) is the separation signal, w k (f, t) (k=1, …, K, f=1, …, F, t=T'+1, …, T) is the k-th separation filter for separating the k-th source signal, W(f, t)=[w 1 (f, t),…, w K (f, t)] H Let (f=1, …, F, t=T'+1, …, T) be the separation matrix. Among the K sound sources, the λ-th sound source (where λ satisfies 1≦λ≦K) moves, and the K-1 sound sources other than the λ-th sound source do not move. The initial value W(f, T') (f = 1, ..., F) of the separation matrix W(f, t) is assumed to be obtained. an initialization unit that initializes a time frame t by t<-T'+1; a first calculation unit that calculates an approximate separation signal ^y(f,t) (f=1, ...,F) that is an approximation signal of a separation signal y(f,t) from an observation signal x(f,t) (f=1, ...,F) using a separation matrix W(f,t-1) (f=1, ...,F); Using the approximate separation signal ^y(f, t) (f=1, …, F), the value r k (t) a second calculation unit for updating (k=1, …, K); Approximate separated signal ^y(f, t) (f=1, …, F) and value r k (t) (k=1, …, K), for k=λ, the weight G λ (f, t) (f=1, …, F), and for k≠λ, update the weight G k (f, t) (f=1, …, F) and weight g k a third calculation unit for updating (f, t) (f=1, …, F); Weight G k (f, t) (k=1, …, K, f=1, …, F) and weight g k a fourth calculation unit that updates a separation matrix W(f, t) (f=1, ..., F) and a separated signal y(f, t) (f=1, ..., F) using (f, t) (k=1, ..., λ-1, λ+1, ..., K, f=1, ..., F); a termination condition determination unit that updates the time frame t by t<-t+1, and terminates the process if t>T is satisfied, and otherwise returns to the process of the first calculation unit; A sound source signal estimation device comprising:

2. The sound source signal estimation device according to claim 1 , The first calculation unit calculates an approximate separated signal ^y(f, t) (f=1, ..., F) by the following formula: [0040] The second calculation unit calculates a value r k (t) (k=1, …, K) is updated. [0041] Let α be a constant satisfying 0<α≦1, and φ(r) be φ(r)=ψ'(r) / 2r (where ψ(r) is the set R of real numbers ≥ 0). ≧0 is a differentiable function from ψ to a set R of real numbers, and ψ' represents the derivative of ψ. The third calculation unit calculates the weight G k (f, t) (k=1, …, K, f=1, …, F), g k (f, t) (k=1, …, λ-1, λ+1, …, K, f=1, …, F), [0042] [0043] The fourth calculation unit updates the separation matrix W(f, t) (f=1, ..., F) and the separated signal y(f, t) (f=1, ..., F) by the following formula: [0044] [0045] A sound source signal estimation device comprising:

3. Let x(f, t)(f=1, …, F, t=T'+1, …, T) (where f is an index representing a frequency bin, t is an index representing a time frame, and T' is an integer equal to or greater than 1) be the observed signal of a mixture of sounds from K sound sources observed using K microphones (where K is an integer equal to or greater than 2), and y k Let (f, t)(k=1, …, K, f=1, …, F, t=T'+1, …, T) be the k-th separated signal, which is the estimated signal of the signal from the k-th sound source (hereafter referred to as the k-th sound source signal), and y(f, t)=[y 1 (f, t),…, y K (f, t)] T (f=1, …, F, t=T'+1, …,T) is the separation signal, w k (f, t) (k=1, …, K, f=1, …, F, t=T'+1, …, T) is the k-th separation filter for separating the k-th source signal, W(f, t)=[w 1 (f, t),…, w K (f, t)] H Let (f=1, …, F, t=T'+1, …, T) be the separation matrix. Among the K sound sources, the λ-th sound source (where λ satisfies 1≦λ≦K) moves, and the K-1 sound sources other than the λ-th sound source do not move. The initial value W(f, T') (f = 1, ..., F) of the separation matrix W(f, t) is assumed to be obtained. an initialization unit that initializes a time frame t by t<-T'+1; a first calculation unit that calculates an approximate separation signal ^y(f,t) (f=1, ...,F) that is an approximation signal of a separation signal y(f,t) from an observation signal x(f,t) (f=1, ...,F) by using an auxiliary separation matrix ~Q(f,t-1) (f=1, ...,F) that satisfies W(f,t-1)=~Q(f,t-1)W(f,T'); Using the approximate separation signal ^y(f, t) (f=1, …, F), the value r k (t) a second calculation unit for updating (k=1, …, K); Approximate separated signal ^y(f, t) (f=1, …, F) and value r k (t) (k=1, …, K), for k=λ, the weight G λ (f, t) (f=1, …, F), and for k≠λ, update the weight G k (f, t) (f=1, …, F) and weight g k a third calculation unit for updating (f, t) (f=1, …, F); Weight G k (f, t) (k=1, …, K, f=1, …, F) and weight g k a fourth calculation unit that updates an auxiliary separation matrix ∼Q(f, t) (f=1, ..., F) and a separated signal y(f, t) (f=1, ..., F) using (f, t) (k=1, ..., λ-1, λ+1, ..., K, f=1, ..., F); a termination condition determination unit that updates the time frame t by t<-t+1, and terminates the process if t>T is satisfied, and otherwise returns to the process of the first calculation unit; A sound source signal estimation device comprising:

4. The sound source signal estimation device according to claim 3, The first calculation unit calculates an approximate separated signal ^y(f, t) (f=1, ..., F) by the following formula: [0046] The second calculation unit calculates a value r k (t) (k=1, …, K) is updated. [0047] Let α be a constant satisfying 0<α≦1, and φ(r) be φ(r)=ψ'(r) / 2r (where ψ(r) is the set R of real numbers ≥ 0). ≧0 is a differentiable function from ψ to a set R of real numbers, and ψ' represents the derivative of ψ. The third calculation unit calculates the weight G k (f, t) (k=1, …, K, f=1, …, F), g k (f, t) (k=1, …, λ-1, λ+1, …, K, f=1, …, F), [0048] [0049] The fourth calculation unit updates the auxiliary separation matrix ?Q(f, t) (f=1, ..., F) and the separated signal y(f, t) (f=1, ..., F) by the following formula: [Number 50] [0051] (However, e k is a unit vector whose kth element is 1 and the other elements are 0) [0052] (where ~Q(f, T') is the identity matrix) [0,53] A sound source signal estimation device comprising:

5. Let x(f, t)(f=1, …, F, t=T'+1, …, T) (where f is an index representing a frequency bin, t is an index representing a time frame, and T' is an integer equal to or greater than 1) be the observed signal of a mixture of sounds from K sound sources observed using K microphones (where K is an integer equal to or greater than 2), and y k Let (f, t)(k=1, …, K, f=1, …, F, t=T'+1, …, T) be the k-th separated signal, which is the estimated signal of the signal from the k-th sound source (hereafter referred to as the k-th sound source signal), and y(f, t)=[y 1 (f, t),…, y K (f, t)] T (f=1, …, F, t=T'+1, …,T) is the separation signal, w k (f, t) (k=1, …, K, f=1, …, F, t=T'+1, …, T) is the k-th separation filter for separating the k-th source signal, W(f, t)=[w 1 (f, t),…, w K (f, t)] H Let (f=1, …, F, t=T'+1, …, T) be the separation matrix. Among the K sound sources, the λ-th sound source (where λ satisfies 1≦λ≦K) moves, and the K-1 sound sources other than the λ-th sound source do not move. The initial value W(f, T') (f = 1, ..., F) of the separation matrix W(f, t) is assumed to be obtained. an initialization step in which the source signal estimation apparatus initializes a time frame t by t<-T'+1; A first calculation step in which the sound source signal estimation device calculates an approximate separated signal ^y(f,t) (f=1, ...,F) which is an approximate signal of a separated signal y(f,t) from an observed signal x(f,t) (f=1, ...,F) by using a separation matrix W(f,t-1) (f=1, ...,F); The sound source signal estimation device calculates a value r k (t) a second computational step of updating (k=1, …, K); The sound source signal estimation device estimates an approximate separated signal ^y(f, t) (f=1, ..., F) and a value r k (t) (k=1, …, K), for k=λ, the weight G λ (f, t) (f=1, …, F), and for k≠λ, update the weight G k (f, t) (f=1, …, F) and weight g k a third computational step of updating (f, t) (f=1, …, F); The sound source signal estimation device is k (f, t) (k=1, …, K, f=1, …, F) and weight g k a fourth calculation step of updating the separation matrix W(f, t) (f=1, ..., F) and the separated signals y(f, t) (f=1, ..., F) using (f, t) (k=1, ..., λ-1, λ+1, ..., K, f=1, ..., F); a termination condition determination step in which the sound source signal estimation device updates a time frame t by t<-t+1, and terminates the process if t>T is satisfied, and returns to the process of the first calculation unit otherwise; A sound source signal estimation method comprising:

6. Let x(f, t)(f=1, …, F, t=T'+1, …, T) (where f is an index representing a frequency bin, t is an index representing a time frame, and T' is an integer equal to or greater than 1) be the observed signal of a mixture of sounds from K sound sources observed using K microphones (where K is an integer equal to or greater than 2), and y k Let (f, t)(k=1, …, K, f=1, …, F, t=T'+1, …, T) be the k-th separated signal, which is the estimated signal of the signal from the k-th sound source (hereafter referred to as the k-th sound source signal), and y(f, t)=[y 1 (f, t),…, y K (f, t)] T (f=1, …, F, t=T'+1, …,T) is the separation signal, w k (f, t) (k=1, …, K, f=1, …, F, t=T'+1, …, T) is the k-th separation filter for separating the k-th source signal, W(f, t)=[w 1 (f, t),…, w K (f, t)] H Let (f=1, …, F, t=T'+1, …, T) be the separation matrix. Among the K sound sources, the λ-th sound source (where λ satisfies 1≦λ≦K) moves, and the K-1 sound sources other than the λ-th sound source do not move. The initial value W(f, T') (f = 1, ..., F) of the separation matrix W(f, t) is assumed to be obtained. an initialization step in which the source signal estimation apparatus initializes a time frame t by t<-T'+1; a first calculation step in which the sound source signal estimation device calculates an approximate separated signal ^y(f,t) (f=1, ...,F) which is an approximation signal of a separated signal y(f,t) from an observed signal x(f,t) (f=1, ...,F) by using an auxiliary separation matrix ~Q(f,t-1) (f=1, ...,F) satisfying W(f,t-1)=~Q(f,t-1)W(f,T'); The sound source signal estimation device calculates a value r k (t) a second computational step of updating (k=1, …, K); The sound source signal estimation device estimates an approximate separated signal ^y(f, t) (f=1, ..., F) and a value r k (t) (k=1, …, K), for k=λ, the weight G λ (f, t) (f=1, …, F), and for k≠λ, update the weight G k (f, t) (f=1, …, F) and weight g k a third computational step of updating (f, t) (f=1, …, F); The sound source signal estimation device is k (f, t) (k=1, …, K, f=1, …, F) and weight g k a fourth calculation step of updating the auxiliary separation matrix ∼Q(f, t) (f=1, …, F) and the separated signals y(f, t) (f=1, …, F) using (f, t) (k=1, …, λ-1, λ+1, …, K, f=1, …, F); a termination condition determination step in which the sound source signal estimation device updates a time frame t by t<-t+1, and terminates the process if t>T is satisfied, and returns to the process of the first calculation unit otherwise; A sound source signal estimation method comprising:

7. A program for causing a computer to function as the sound source signal estimation device according to any one of claims 1 to 4.

Citation Information

Patent Citations

  • Device, program, and method for sound source separation

    JP2006154314A

  • Signal processing apparatus, method, and program

    JP2014041308A

  • Signal processing system, signal processing method, and signal processing program

    JP2018156052A