Signal extraction device and signal extraction method

Abstract

In a signal extraction device (10), an acquisition unit (15a) acquires a mixed signal for a plurality of sound sources and clue information for a target sound source. An estimation unit (15c) inputs, into a trained model (14a), the acquired mixed signal and clue information for a target sound source as well as data in a sampled step, and estimates a score corresponding to the data. An extraction unit (15d) extracts a signal for the target sound source from the mixed signal for the plurality of sound sources by using the estimated score.

Description

Signal extraction device and signal extraction method

[0001] The present invention relates to a signal extraction device and a signal extraction method.

[0002] In recent years, technologies for extracting the voice of a specific speaker from a mixed voice signal have been studied in order to improve the performance of voice recognition and speaker recognition, make voices easier to hear in conversation applications, etc. For example, a discriminative target sound source extraction system has been proposed in which an observed signal and some kind of clue that can identify a target sound, such as a pre-recorded voice of a target speaker, are input into a DNN (Deep Neural Network) model, and a target sound source signal is output (see Non-Patent Document 1).

[0003] Also, a method for removing noise from observed signals has been disclosed, which uses a diffusion model to generate data by inversely changing a Markov process that changes data to Gaussian noise (see Non-Patent Document 2).

[0004] Katerina Zmolikova, “SpeakerBeam: Speaker Aware Neural Network for Target Speaker Extraction in Speech Mixtures”, IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, VOL. 13, NO. 4, August 2019 Julius Richter, “Speech Enhancement and Dereverberation with Diffusion-based Generative Models”, IEEE / ACM Transactions on Audio, Speech and Language Processing, 2023

[0005] However, with conventional techniques, it has been difficult to extract a target sound source signal from a mixed signal with high accuracy. For example, the auditory quality of the extracted target sound source signal is insufficient. Furthermore, it has been difficult to improve the quality even when the target sound source signal includes a failure to extract a signal different from the target signal.

[0006] The present invention has been made in view of the above, and has an object to extract a target sound source signal from a mixed signal with high accuracy in the prior art.

[0007] In order to solve the above-mentioned problems and achieve the object, the signal extraction device according to the present invention is characterized by having an acquisition unit that acquires a mixed signal of multiple sound sources and clue information of a target sound source, an estimation unit that inputs the acquired mixed signal, the clues of the target sound source, and data at a sampled step into a trained model and estimates a score corresponding to the data, and an extraction unit that extracts a signal of the target sound source from the mixed signal of the multiple sound sources using the estimated score.

[0008] According to the present invention, it is possible to extract a target sound source signal from a mixed signal with high accuracy.

[0009] FIG. 1 is a diagram for explaining an overview of the signal extraction device of this embodiment. FIG. 2 is a diagram for explaining an overview of the signal extraction device of this embodiment. FIG. 3 is a diagram for explaining an overview of the signal extraction device of this embodiment. FIG. 4 is a diagram for explaining an overview of the signal extraction device of this embodiment. FIG. 5 is a diagram for explaining an overview of the signal extraction device of this embodiment. FIG. 6 is a schematic diagram illustrating an example of the general configuration of the signal extraction device of this embodiment. FIG. 7 is a diagram for explaining signal extraction processing. FIG. 8 is a diagram for explaining signal extraction processing. FIG. 9 is a diagram for explaining processing by the extraction unit. FIG. 10 is a diagram for explaining processing by the extraction unit. FIG. 11 is a diagram for explaining processing by the extraction unit. FIG. 12 is a flowchart showing a signal extraction processing procedure. FIG. 13 is a diagram for explaining an example. FIG. 14 is a diagram for explaining an example. FIG. 15 is a diagram showing an example of a computer that executes a signal extraction program.

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are designated by the same reference numerals.

[0011] [Outline of Signal Extraction Device] FIGS. 1 to 5 are diagrams for explaining an outline of the signal extraction device of this embodiment. First, FIG. 1 illustrates a conventional deterministic target sound source extraction process. As shown in FIG. 1, in deterministic target sound source extraction, an observed signal y and some kind of clue c capable of identifying target sound, such as pre-recorded speech from a target speaker, are input to a sound source extraction model, and a target sound source signal is output. This sound source extraction model is trained using a DNN to minimize the difference between the correct signal and the output signal. Because this sound source extraction model is not a data generation model that follows a probability distribution, it can only generate deterministic speech that outputs the same result regardless of timing. Therefore, there is no mechanism for improving when a failure, such as a signal that is different from the target, is included.

[0012] On the other hand, Fig. 2 illustrates the processing of a conventional diffusion model. As shown in Fig. 2, in the diffusion model, the data x ０ to Gaussian noise x Ｔ For a Markov process (forward process) that changes x Ｔ From x ０ The data x is then transformed in the reverse direction using a Markov process. ０ In this case, the DNN generates the first step p(x Ｔ |x Ｔ－１ ) is learned to achieve this.

[0013] FIG. 3 also illustrates processing of a Score Based Diffusion model that uses a value called Score in the Diffusion model shown in FIG. 2. Specifically, as shown in FIG. 3(a), the Score Based Diffusion model uses Stochastic Differential Equations (SDE) that describe continuous state transitions. In this case, the DNN (Score model) is trained using the correct Score value corresponding to the data to be generated as training data, as shown in FIG. 3(b). Then, as shown in FIG. 3(c), the trained Score model is used to calculate one-step transitions (data x at step t) ｔ ) is repeated to generate data.

[0014] 4 illustrates a process for removing noise from a speech signal (see Non-Patent Document 2) using the Score Based Diffusion model shown in FIG. 3. As shown in FIGS. 4(a) and 4(b), this technology uses a DNN (Score model) similar to that shown in FIG. 3 to generate clean speech from which noise has been removed under the conditions of the observed signal y. This DNN is a noise removal model and cannot extract the target sound source. Furthermore, there is no mechanism for improving the model even if a failure is included.

[0015] Therefore, the signal extraction device of this embodiment uses the Score Based Diffusion model (Score model) shown in FIG. 3 to generate a signal of the target sound source under the conditions of the observed signal y and the clue c of the target sound source.

[0016] 5 illustrates the signal extraction process of this embodiment. Specifically, the score model of the signal extraction device is generated by generating a score x as shown in FIG. ０ to Gaussian noise x Ｔ For the forward process shown in the following equation (1), Ｔ From x ０Then, the data x at step t is changed in the reverse direction as shown in the following equation (2). ｔ By repeating the inference, data x ０ Generate.

[0017]

[0018] In this way, the signal extraction device generates a speech signal of a target sound source using a diffusion model, which is a generative model, thereby enabling improvements in auditory quality, such as improvements in the performance of PESQ (Perceptual Evaluation of Speech Quality).

[0019] Furthermore, in the signal extraction device, the inference results change probabilistically depending on the random number seed used when sampling the inference step number t, etc., as shown in Figures 5(b) and 5(c). Therefore, the signal extraction device performs ensemble of the inference results, which makes it possible to improve the accuracy of the inference results.

[0020] [Configuration of the Signal Extraction Device] Fig. 6 is a schematic diagram illustrating the general configuration of a signal extraction device of this embodiment. As illustrated in Fig. 6, a signal extraction device 10 of this embodiment is realized by a general-purpose computer such as a personal computer, and includes an input unit 11, an output unit 12, a communication control unit 13, a storage unit 14, and a control unit 15.

[0021] The input unit 11 is realized using input devices such as a microphone, keyboard, and mouse, and inputs various instruction information such as a command to start processing to the control unit 15 in response to input operations by an operator. The output unit 12 is realized by a display device such as a liquid crystal display, a printing device such as a printer, etc. For example, the output unit 12 displays the results of a signal extraction process described below.

[0022] The communication control unit 13 is realized by a NIC (Network Interface Card) or the like, and controls communication between the control unit 15 and external devices via telecommunication lines such as a LAN (Local Area Network) or the Internet. For example, the communication control unit 13 controls communication between the control unit 15 and a management device or the like that manages various types of information.

[0023] The storage unit 14 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 14 stores in advance processing programs for operating the signal extraction device 10 and data used during execution of the processing programs, or temporarily stores the data each time processing is performed. The storage unit 14 may be configured to communicate with the control unit 15 via the communication control unit 13.

[0024] In this embodiment, the storage unit 14 stores a model 14a or a cascade model 140a, etc., which are used in the signal extraction process described later.

[0025] The control unit 15 is realized using a CPU (Central Processing Unit), NP (Network Processor), FPGA (Field Programmable Gate Array), etc., and executes processing programs stored in memory. As a result, the control unit 15 functions as an acquisition unit 15a, a learning unit 15b, an estimation unit 15c, and an extraction unit 15d, as exemplified in FIG. 6, and executes signal extraction processing. Note that these functional units may each be implemented in different hardware, or some of them may be implemented in different hardware. The control unit 15 may also include other functional units.

[0026] The acquisition unit 15a acquires a mixed signal of multiple sound sources and clue information of a target sound source. For example, the acquisition unit 15a acquires an observed signal y, which is a mixed signal of multiple sound sources, and some kind of clue c that can identify the target sound source, such as a pre-recorded voice of a target speaker, from a microphone, a management device that manages various information, or the like, via the input unit 11 or the communication control unit 13. The acquisition unit 15a may store the acquired observed signal y and clue information c of the target sound source in the storage unit 14.

[0027] The learning unit 15b uses the acquired mixed signal y, the clue c of the target sound source, and the sampled data x at the t-th step. ｔ When you input the data x ｔSpecifically, as shown in FIG. 5B, the model 14a is generated by learning to estimate a score corresponding to data x ｔ When the observed signal y and the clue c are input, x ｔ A model (Score model) 14a that outputs a score corresponding to the score is generated by learning.

[0028] This model 14a is a Gaussian noise x Ｔ Data to be generated from x ０ The data x is then converted into ０ Specifically, the model 14a is a diffusion model that generates the data x at step t using the score, as illustrated in FIG. ｔ By repeating the inference, data x ０ This is a score-based diffusion model that generates

[0029] The estimation unit 15c estimates the mixed signal y, the clue c of the target sound source, and the sampled data x at the t-th step. ｔ Specifically, as shown in FIG. 5C, the mixed signal y and the clues c and x of the target sound source are input to the trained model 14a, and the score corresponding to the data is estimated. ｔ and get the score output.

[0030] The extraction unit 15d extracts the signal of the target sound source from the mixed signal y of the multiple sound sources by using the estimated score. Specifically, as illustrated in FIG. 5C, the extraction unit 15d extracts x Ｔ－Δｔ The process of estimating the inference result x ０ By repeating this process until we obtain the signal x of the target sound source, ０ Extract.

[0031] Here, if the random number seed used during sampling is changed, the inference result will change probabilistically. Therefore, the extraction unit 15d may use the average value of the target sound source signals extracted in each of a predetermined number of processes as the extraction result. In other words, the extraction unit 15d may perform ensemble to change the random number seed, repeat inference a predetermined number of times, and calculate the average value of the inference results. This makes it possible to improve the accuracy of the inference.

[0032] 7 and 8 are diagrams for explaining the signal extraction process. First, Fig. 7 illustrates the process of the model 14a (DNN, Score model) shown in Fig. 5.

[0033] Specifically, during training (see FIG. 5B), the training unit 15b first receives the mixed speech y and the target speaker speech x as observed signals. ０ , using the speech of a randomly selected target speaker c as a cue for the target sound source, data (y, x ０ , c) is created.

[0034] The learning unit 15b also samples step number t from a uniform distribution of 0≦t≦T, and calculates x ０ Using x ｔ Sampling is performed.

[0035]

[0036] Then, the learning unit 15b applies the data (y, x ｔ , c) and outputs a score. The learning unit 15b also calculates a loss function to minimize the difference between the correct score and the output score, and performs a backward calculation.

[0037] During inference (see FIG. 5C), the estimation unit 15c calculates x Ｔ Also, [0, T] is divided into N parts, and Δt = T / N. (x Ｔ , x Ｔ－Δｔ , ..., x ０Specifically, the estimation unit 15c applies discontinuous state transitions to the model 14a using the data (y, x ｔ , c) and obtains the score output. The extraction unit 15d also uses the output score to extract x Ｔ－Δｔ The process of estimating the inference result x ０ Repeat until you get .

[0038] Note that changing the random number seed during sampling will probabilistically change the inference results. Therefore, the extraction unit 15d may perform ensemble, changing the random number seed and repeating the inference a predetermined number of times to calculate the average value of the inference results. This makes it possible to improve the accuracy of the inference.

[0039] 8A illustrates the processing of a model (Discriminative TSE (Target Source Extraction)) 141a of the deterministic target sound source extraction processing illustrated in Fig. 1. 8B illustrates the processing of a model (cascade model) 140a that combines the model 141a of the deterministic target sound source extraction processing of Fig. 8A and the model 14a of the signal extraction processing illustrated in Fig. 7.

[0040] In the cascade model 140a, the learning unit 15b performs multitask learning using two loss functions: a loss function of the score and a loss function of the target speaker's voice x. The cascade model 140a can be used in two ways: as a diffusion model that performs inference using the score, and as a deterministic model that uses a direct estimate of x.

[0041] Specifically, when training the cascade model 140a, the training unit 15b receives the mixed speech y and the target speaker speech x as observed signals, in the same manner as when training the model 14a in FIG. ０ , using the speech of a randomly selected target speaker c as a cue for the target sound source, data (y, x ０ , c) is created.

[0042] The learning unit 15b also samples step number t from a uniform distribution of 0≦t≦T, and calculates x ０ Using x ｔ Sampling is performed.

[0043] The learning unit 15b inputs data (y, c) to the model 141a of the model 140a in FIG. 8B to obtain an output of x^. The learning unit 15b also inputs data (x^) to the model 142a. ｔ , x^) is input to obtain an output score. Then, the learning unit 15b calculates a loss function of the difference between the correct score and the output score, and calculates the relationship between x^ and x ０ and the loss function of the difference between them are calculated to minimize each other, and learning is performed by performing backward calculations.

[0044] When the cascade model 140a is estimated, the estimation unit 15c calculates x according to the above formulas (3) to (5), in the same way as when the model 14a of FIG. Ｔ The estimation unit 15c divides [0, T] into N parts to obtain Δt=T / N, and then performs sampling of (x Ｔ , x Ｔ－Δｔ , ..., x ０ Specifically, the estimation unit 15c inputs data (y, x) to the cascade model 140a, and performs discontinuous state transitions by a reverse process. ｔ , c) and obtains the score output. The extraction unit 15d uses the output score to extract x Ｔ－Δｔ The process of estimating the inference result x ０ Repeat until you get .

[0045] As in the example shown in Fig. 7, changing the random number seed during sampling causes the inference result to change probabilistically. Therefore, the extraction unit 15d may use the average value of the target sound source signals extracted in each of a predetermined number of processes as the extraction result. In other words, an ensemble may be performed in which the random number seed is changed and inference is repeated a predetermined number of times to obtain the average value of the inference results. This makes it possible to improve the accuracy of the inference.

[0046] 9 to 11 are diagrams for explaining the processing of the extraction unit. In Fig. 9, black circles indicate the states output in each process. Since random numbers are referenced to generate each state, different states are generated by using different random numbers.

[0047] The extraction unit 15d generates voice data by repeatedly using the model (14a, 140a) about 30 times to generate one piece of data. In addition, the extraction unit 15d generates multiple pieces of voice data by changing the seed of the random number to perform ensemble.

[0048] For example, in the model 14a illustrated in FIG. 9A, the extraction unit 15d generates data by changing the seed of the random number in the first process of about 30 repeated processes for generating one data. Ｎ １ , x Ｎ ２ , ..., x Ｎ Ｍ Different seeds are generated in

[0049] Alternatively, the extraction unit 15d may generate and change multiple random number seeds during each predetermined number of processes, and branch the process according to each random number seed, thereby extracting signals of multiple target sound sources. That is, the extraction unit 15d may generate and branch multiple random number seeds during the repeated process by the process (Split tree) illustrated in FIG. 9(b). In the example illustrated in FIG. 9(b), for example, x ｎ’－１ １ , x ｎ’－１ ２ Different seeds are generated corresponding to the models (14a, 140a). As a result, when performing ensembling using data generated by the models (14a, 140a), the extraction unit 15d can reduce the number of calls (repeated processes) of the models (14a, 140a) to generate the same number of data. Therefore, it is possible to reduce the overall calculation load.

[0050] Here, Figure 10 illustrates the effect of Split Tree. The experiment shown in Figure 10 was conducted on a noise reduction dataset. In Figure 10, Split points indicate where branching occurs in the repeated processing from N (= 30) to 0. #Splits indicates the number of branches at each Split point. In the example shown in Figure 9(b), all #Splits are 2. PESQ and SI-SDR are evaluation indices for speech enhancement, with higher values ​​indicating better evaluation. #Calls indicates the total number of times the model was called (the number of black circles shown in Figure 9) and corresponds to the computational cost.

[0051] 10(2) corresponds to the case where multiple data sets (number of data sets M=8) to be ensembled are generated by the process shown in FIG. 9(a) (#Calls=240). Also, FIGS. 10(3) to 10(6) correspond to the case where the same number of data sets are generated by the process shown in FIG. 9(b) (Split tree). As shown in FIGS. 10(3) to 10(6), it can be seen that Split tree can reduce #Calls while maintaining performance equivalent to that of FIG. 10(2).

[0052] Furthermore, when the extraction unit 15d generates multiple data, poor performance data may be generated depending on the random number seed, and ensembling does not necessarily improve performance. Therefore, the extraction unit 15d may exclude signals that deviate from the average value of the target sound source signals extracted in each of a predetermined number of processes by a predetermined threshold or more, and calculate the average value of the target sound source signals other than the excluded signals as the extraction result. In other words, the extraction unit 15d may define data that deviates from the average value of the generated data by a predetermined threshold or more as outliers and perform a process (outliner removal) to exclude the data from the ensemble. Then, the extraction unit 15d calculates the average value of the remaining data other than the outliers from the generated data, and sets this as the extraction result.

[0053] Specifically, the extraction unit 15d calculates the average shown in the following formula (8) according to the following formulas (6) to (7).

[0054]

[0055] where i is the time index of the time series signal, m is the index of the generated sample, and x ０ ｍ [i] is the value of each sample, x ０ ̂[i] represents the mean value, and l represents the index of each interval when the time series signal is divided into short intervals.

[0056] The above formula (7) represents the variance. Furthermore, the above formula (6) represents the ratio of the squared error from the mean value of each sample to the variance. Furthermore, since the signal is a time series, the extraction unit 15d calculates the above formulas (6) and (7) for each short interval and calculates the average thereof using the above formula (8). The extraction unit 15d then uses the calculated average to define data that deviates from the mean by a predetermined threshold or more as outliers, and excludes the outliers from the ensemble. This makes it possible to improve estimation accuracy even when data with poor performance is generated depending on the random number seed.

[0057] FIG. 11 illustrates the effect of outliner removal. FIG. 11 shows a plot of performance (SI-SDR) for each emphasized speech in the evaluation set when outliers that are more than a predetermined threshold (=2.5) away from the mean are removed. The horizontal axis of the plot in FIG. 11 represents the performance when ensembling is performed without outlier removal, while the vertical axis represents the change in performance when outlier removal is performed. As shown in FIG. 11, samples with values ​​on the vertical axis greater than 0 show that performance is improved by outlier removal. In this case, the number of samples with improved performance is greater, confirming that overall performance is improved. In this way, outlier removal makes it possible to improve estimation accuracy by removing data by setting a certain threshold, even if poor-performing data is generated depending on the random number seed.

[0058] On the other hand, it was confirmed that for samples with poor original performance, whose horizontal axis values ​​were less than 0, it was difficult to improve performance through outlier removal.

[0059] [Signal Extraction Processing] Next, the signal extraction processing performed by the signal extraction device 10 according to this embodiment will be described with reference to Fig. 12. Fig. 12 is a flowchart showing the procedure of the signal extraction processing. The flowchart in Fig. 12 starts, for example, when the user performs an operation input to instruct the start of the processing.

[0060] First, the acquisition unit 15a acquires an observed signal y, which is a mixed signal of multiple sound sources, and a clue c of a target sound source (step S1).

[0061] Next, the estimation unit 15c estimates the obtained observed signal y, the clue c of the target sound source, and the sampled data x at the t-th step. ｔ The mixed signal y and the clues c and x of the target sound source are input to the trained model 14a, and the score corresponding to the data is estimated (step S2). ｔ and get the score output.

[0062] Then, the extraction unit 15d extracts the signal of the target sound source from the mixed signal y of the multiple sound sources using the estimated score (step S3). Specifically, the extraction unit 15d extracts x Ｔ－Δｔ The process of estimating the inference result x ０ By repeating this process until we obtain the signal x of the target sound source, ０ This completes the signal extraction process.

[0063] [Effects] As described above, in the signal extraction device 10 of this embodiment, the acquisition unit 15a acquires a mixed signal of multiple sound sources and clue information of a target sound source. The estimation unit 15c inputs the acquired mixed signal, the clue information of the target sound source, and sampled data at the t-th step into the trained model 14a and estimates a score corresponding to the data. The extraction unit 15d extracts a signal of the target sound source from the mixed signal of multiple sound sources using the estimated score.

[0064] Specifically, the trained model 14a is a diffusion model that generates the data to be generated by an inverse process that changes the data from Gaussian noise to the data to be generated step by step.

[0065] In this way, by using a generative model based on a probability distribution, it is possible to improve the performance of PESQ and other auditory quality. Therefore, it becomes possible to extract a target sound source signal from a mixed signal with high accuracy.

[0066] Furthermore, the extraction unit 15d sets the average value of the target sound source signals extracted in each of the predetermined number of processes as the extraction result. In this way, by utilizing the fact that the inference result changes probabilistically when the random number seed is changed, it is possible to improve the accuracy of the inference.

[0067] Furthermore, the extraction unit 15d may extract signals of multiple target sound sources by generating and changing multiple random number seeds during each of a predetermined number of processes and branching the process according to each random number seed. In this way, by generating multiple random number seeds during the repeated process and branching, the number of times the model (14a, 140a) is called to generate the same number of data (the number of repeated processes) can be reduced, thereby reducing the overall calculation load.

[0068] Furthermore, the extraction unit 15d may exclude signals that are different from the average value of the target sound source signals extracted in each processing operation a predetermined number of times by a predetermined threshold or more, and calculate the average value of the target sound source signals other than the excluded signals as the extraction result. In this way, even if data with poor performance is generated depending on the random number seed, it is possible to improve the estimation accuracy by removing the data by setting a certain threshold.

[0069] [Example] Figures 13 and 14 are diagrams for explaining an example. First, in the example shown in Figure 13, the model 14a of the above embodiment was used, and the inference process was repeated 10 times with different random number seeds. Figure 13 illustrates each inference result and an ensemble result obtained by simply averaging the inference results in the time-frequency domain. As illustrated in Figure 13(a), it was confirmed that the SDR (Source to Distortion Ratio) of the ensemble result (Ensemble) indicated by "x" was better than the SDR of each inference result (Extracted Samples) indicated by "·".

[0070] An example of the ensemble is shown in Fig. 13(b). As can be seen from the area enclosed by a bold frame in the figure, it was confirmed that the ensemble result (Ensemble) better represents the inference result of the target sound source (Target) for the mixed signal (Mixture).

[0071] 14 illustrates an example of a perceptual evaluation index of speech quality (PESQ) as an evaluation result of each of the above methods. As shown in FIG. 14, it was confirmed that the model 14a of the present invention illustrated in FIG. 7, using ensembles, performed better than the conventional discriminative TSE methods illustrated in FIGS. 1 and 8(a). It was also confirmed that the cascade model 140a of the present invention illustrated in FIG. 8(b) performed better than the conventional discriminative TSE methods, regardless of whether or not an ensemble was used.

[0072] [Program] A program describing the processing performed by the signal extraction device 10 according to the above embodiment in a computer-executable language can also be created. In one embodiment, the signal extraction device 10 can be implemented by installing a signal extraction program that executes the above signal extraction processing as package software or online software on a desired computer. For example, by executing the signal extraction program on an information processing device, the information processing device can function as the signal extraction device 10. The information processing device referred to here includes desktop and notebook personal computers. Other examples of information processing devices include mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handyphone System) phones, as well as slate terminals such as PDAs (Personal Digital Assistants). The functions of the signal extraction device 10 may also be implemented on a cloud server.

[0073] 15 is a diagram showing an example of a computer that executes a signal extraction program. The computer 1000 includes, for example, a memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0074] The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1031. The disk drive interface 1040 is connected to a disk drive 1041. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1041. The serial port interface 1050 is connected to a mouse 1051 and a keyboard 1052, for example. The video adapter 1060 is connected to a display 1061, for example.

[0075] Here, the hard disk drive 1031 stores, for example, an OS (Operating System) 1091, an application program 1092, a program module 1093, and program data 1094. The various pieces of information described in the above embodiments are stored in the hard disk drive 1031 or the memory 1010, for example.

[0076] The signal extraction program is stored in the hard disk drive 1031 as, for example, a program module 1093 in which instructions to be executed by the computer 1000 are written. Specifically, the program module 1093 in which each process executed by the signal extraction device 10 described in the above embodiment is written is stored in the hard disk drive 1031.

[0077] Furthermore, data used for information processing by the signal extraction program is stored as program data 1094, for example, in the hard disk drive 1031. Then, the CPU 1020 reads the program module 1093 and the program data 1094 stored in the hard disk drive 1031 into the RAM 1012 as necessary, and executes each of the above-described procedures.

[0078] Note that the program module 1093 and program data 1094 related to the signal extraction program are not limited to being stored in the hard disk drive 1031, and may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1041. Alternatively, the program module 1093 and program data 1094 related to the signal extraction program may be stored in another computer connected via a network such as a LAN or a WAN (Wide Area Network), and read by the CPU 1020 via the network interface 1070.

[0079] Although the present invention has been described above as an embodiment, the present invention is not limited to the description and drawings that form part of the disclosure of the present invention. In other words, other embodiments, examples, and operational techniques that can be made by those skilled in the art based on the present invention are all included in the scope of the present invention.

[0080] REFERENCE SIGNS LIST 10 Signal extraction device 11 Input unit 12 Output unit 13 Communication control unit 14 Storage unit 14a, 140a, 141a, 142a Model 15 Control unit 15a Acquisition unit 15b Learning unit 15c Estimation unit 15d Extraction unit

Claims

1. A signal extraction device comprising: an acquisition unit that acquires a mixed signal of multiple sound sources and clue information of a target sound source; an estimation unit that inputs the acquired mixed signal, the clue information of the target sound source, and data at a sampled step into a trained model and estimates a score corresponding to the data; and an extraction unit that extracts a signal of the target sound source from the mixed signal of multiple sound sources using the estimated score.

2. The signal extraction device according to claim 1, characterized in that the trained model is a diffusion model that generates the data by an inverse process of changing the data to be generated stepwise from Gaussian noise.

3. The signal extraction device according to claim 1, wherein the extraction section takes as an extraction result an average value of the signal of the target sound source extracted in each processing step a predetermined number of times.

4. The signal extraction device described in claim 3, characterized in that the extraction unit generates and changes multiple random number seeds during each of the processes, and extracts signals of the multiple target sound sources by branching the process corresponding to each random number seed.

5. The signal extraction device according to claim 3, characterized in that the extraction unit excludes signals of the target sound source extracted in each process that are away from the average value by a predetermined threshold or more, and calculates the average value of the signals of the target sound source other than the excluded signals as the extraction result.

6. A signal extraction method executed by a signal extraction device, comprising: an acquisition step of acquiring a mixed signal of multiple sound sources and clue information of a target sound source; an estimation step of inputting the acquired mixed signal, the clue information of the target sound source, and data at a sampled step into a trained model and estimating a score corresponding to the data; and an extraction step of extracting a signal of the target sound source from the mixed signal of multiple sound sources using the estimated score.

Images