Data generation system

Abstract

New data can be generated.SOLUTION: A synthesis unit that synthesizes the first acoustic signal data and the second acoustic signal data with a plurality of weights to generate a plurality of synthesis spectra, a state synthesis unit that synthesizes a state quantity corresponding to the first acoustic signal data and a state quantity corresponding to the second acoustic signal data with a plurality of weights to generate a plurality of synthesis state quantities, and a diffusion model including a down-sampling block to which the synthesis spectrum is input and an up-sampling block to which an output of the down-sampling block and the synthesis state quantity are input.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a data generation system. [Background technology]

[0002] A technique for detecting abnormalities using acoustic signals is known. Patent Document 1 discloses an abnormal sound detection device that detects whether an input sound is an abnormal sound, and includes an acoustic feature extraction unit that extracts acoustic features of the input sound using a feature extraction function, an abnormality degree calculation unit that calculates the abnormality level of the input sound using the extracted acoustic features, and a determination unit that determines whether the input sound is an abnormal sound based on the obtained abnormality level and a threshold value, wherein the feature extraction function is based on a probability distribution that models normal sounds and sounds that may include abnormal sounds, a probability distribution that models normal sounds, and a probability distribution that models the input abnormal sound, and the threshold value is set using the abnormality level obtained from the normal sound. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] European Patent Application Publication No. 003584573 Summary of the Invention [Problem to be solved by the invention]

[0004] The invention described in Patent Document 1 does not allow new data to be generated. [Means for solving the problem]

[0005] A data generation system according to a first aspect of the present invention includes a synthesis unit that synthesizes first acoustic signal data and second acoustic signal data using a plurality of weights to generate a plurality of synthetic spectra; a state synthesis unit that synthesizes a state quantity corresponding to the first acoustic signal data and a state quantity corresponding to the second acoustic signal data using the plurality of weights to generate a plurality of synthetic state quantities; a downsampling block that receives as input the synthetic spectra; and an upsampling block that receives as input the output of the downsampling block and the synthetic state quantities. [Effects of the Invention]

[0006] According to the present invention, new data can be generated. [Brief explanation of the drawings]

[0007] [Figure 1] Functional configuration diagram of the data generation system [Figure 2] Diagram showing the structure of the diffusion model [Figure 3] Flowchart showing the learning process of the diffusion model [Figure 4] Flowchart showing data generation processing [Figure 5] Hardware configuration diagram of the data generation system DETAILED DESCRIPTION OF THE INVENTION

[0008] -First embodiment- A first embodiment of a data generation system will be described below with reference to FIGS.

[0009] FIG. 1 is a functional configuration diagram of a data generation system 1. The data generation system 1 generates data for testing an anomaly detection system 36. There are no particular limitations on the anomalies detected by the anomaly detection system 36, but in this embodiment, an example will be described in which a motor anomaly is detected based on the sound emitted by the motor. The motor is operated in multiple domains, for example, with different applied voltages. In this embodiment, the applied voltage is described as being two, high voltage and low voltage, but the motor may have three or more domains.

[0010] The data generation system 1 includes a diffusion model 10, an acoustic signal acquisition unit 21, a spectrum generation unit 22, a spectrum synthesis unit 23, an acoustic signal generation unit 24, a weight management unit 25, an FAD score calculation unit 26, a matching data extraction unit 27, a state vector generation unit 30, and an anomaly detection evaluation system 35. The state vector generation unit 30 includes a state acquisition unit 31, a one-hot encoder 32, a state synthesis unit 33, and an embedded layer 34. The anomaly detection evaluation system 35 includes an anomaly detection system 36 and an evaluation unit 37.

[0011] The anomaly detection system 36 receives the sound to be evaluated and outputs whether or not there is an abnormality and the degree of the abnormality. The sound to be evaluated that is input to the anomaly detection system 36 is, for example, data obtained by recording the sound emitted by a gearbox with a microphone and saving the data in any file format. The frequency band of the sound to be evaluated is not particularly limited, and may exceed the range of human hearing. The sound to be evaluated can be saved in WAVE or MP3 format, for example.

[0012] The acoustic signal acquisition unit 21 acquires input acoustic signal data 81, which is the same type of data as the evaluation target sound. The evaluation target sound, the input acoustic signal data 81, and the output acoustic signal data 85 described below are the same type of data, and are given different names for ease of explanation. That is, the input acoustic signal data 81 and the output acoustic signal data 85 can all be the evaluation target sound. The input acoustic signal data 81 includes a low-pressure normal acoustic signal 81-1, a low-pressure abnormal acoustic signal 81-2, and a high-pressure normal acoustic signal 81-3. That is, it is assumed that almost no acoustic signal data was acquired for a combination of high pressure and abnormality. The few combinations of high pressure and abnormality that were acquired are used for the evaluation described below.

[0013] The input acoustic signal data 81 may be acquired actively or passively. For example, the input acoustic signal data 81 may be uploaded to the data generation system 1 from outside, and the acoustic signal acquisition unit 21 may receive the uploaded input acoustic signal data 81. Alternatively, for example, the acoustic signal acquisition unit 21 may read the input acoustic signal data 81 from a predetermined location. The predetermined location may be a predetermined URI or file path.

[0014] The spectrum generation unit 22 converts the input acoustic signal data 81 into an acoustic signal spectrum 82 based on a Fourier transform or the like. The acoustic signal spectrum 82 is data indicating the relationship between time and frequency, i.e., a spectrogram. As described above, the input acoustic signal data 81 includes three types of data, and the spectrum generation unit 22 converts all three types of data. That is, the spectrum generation unit 22 generates a low-pressure normal spectrum 82-1, a low-pressure abnormal spectrum 82-2, and a high-pressure normal spectrum 82-3.

[0015] The spectrum synthesis unit 23 synthesizes two acoustic signal spectra 82 based on the weight λ output by the weight management unit 25 to generate a synthesized spectrum 83. In other words, the two acoustic signal spectra 82 are input to the spectrum synthesis unit 23, and the spectrum synthesis unit 23 outputs a synthesized spectrum 83. This synthesized spectrum 83 is input to the diffusion model 10. Since multiple weights λ are set, multiple synthesized spectra 83 are generated from one set of acoustic signal spectra 82. The spectrum synthesis unit 23 performs calculations using two spectrograms and the weight λ according to the following Equation 1.

[0016] mix_up(xi, xj, λ) = λ*xi + (1-λ)* xj (Formula 1)

[0017] where xi and xj are different spectrograms in Equation 1.

[0018] The weight management unit 25 outputs variations of the weight λ to the spectrum synthesis unit 23 and the state synthesis unit 33. The weight λ is a value between 0 and 1, for example, "0, 0.1, 0.2, . . . , 0.9, 1.0." In this case, the weight management unit 25 may output these values ​​themselves, or may output information indicating that there are 11 values ​​from 0 to 1.0 in increments of 0.1. The weight management unit 25 outputs data so that the spectrum synthesis unit 23 and the state synthesis unit 33 use the same weight λ. However, the format in which the weight management unit 25 outputs the weight λ to the spectrum synthesis unit 23 and the state synthesis unit 33 may differ.

[0019] The state acquisition unit 31 acquires data indicating a state corresponding to the input acoustic signal data 81. In this embodiment, a combination of a domain variable and a positive / negative value is defined as a "state." In the above example, a "state" is a combination of the high / low voltage applied to the motor and the presence or absence of an abnormality in the motor. Specifically, the combinations are low voltage and normal, low voltage and abnormal, and high voltage and normal. As described above, it is assumed that almost no acoustic signal data was acquired for the combination of high voltage and abnormal. The data indicating the state may be a string of characters that is easily understandable to humans, such as "low voltage, normal," or may be data that is difficult for humans to distinguish.

[0020] The one-hot encoder 32 one-hot encodes the data indicating the state acquired by the state acquisition unit 31 and outputs a one-hot encoded value. Specifically, the one-hot encoder 32 converts each state into a number based on a predetermined conversion rule. In the following, the one-hot encoded value output by the one-hot encoder 32 is assumed to be "1" for the low-pressure normal acoustic signal 81-1, "2" for the low-pressure abnormal acoustic signal 81-2, and "3" for the high-pressure normal acoustic signal 81-3.

[0021] The state combining unit 33 combines the outputs of the one-hot encoder 32 based on the weight λ output by the weight management unit 25 to generate a combined state value 86. Specifically, the state combining unit 33 generates the combined state value 86 based on the following Equation 2.

[0022] mix_up(Coni, Conj, λ) = λ*Coni + (1-λ)* Conj (Formula 2)

[0023] In Equation 2, Coni and Conj are one-hot encoded values ​​of the states corresponding to xi and xj in Equation 1. As described above, multiple weights λ are set, and therefore multiple composite state values ​​86 are also generated.

[0024] The state synthesis unit 33 outputs the generated synthesis state value 86 to the embedding layer 34 and the evaluation unit 37. Since the output spectrum 84 output by the diffusion model 10 does not explicitly include data indicating the presence or absence of an abnormality, the value that is expected to be output by the anomaly detection system 36, that is, a value indicating the correct answer, is output to the evaluation unit 37.

[0025] The embedding layer 34 converts the composite state value 86 output by the state synthesis unit 33 into an embedding vector 87 and inputs it to the diffusion model 10 .

[0026] The diffusion model 10 receives as input a synthesized spectrum 83 output by the spectrum synthesis unit 23 and an embedding vector 87 output by the state vector generation unit 30. The diffusion model 10 outputs an output spectrum 84 based on a calculation that will be described in detail later. As described above, multiple synthesized spectra 83 are generated, and therefore multiple output spectra 84 are also generated.

[0027] The acoustic signal generation unit 24 converts the output spectrum 84, which is a spectrogram, into output acoustic signal data 85, which is the same type of data as the input acoustic signal data 81. In other words, the acoustic signal generation unit 24 executes the reverse process of the spectrum generation unit 22. Since there are multiple output spectra 84, multiple pieces of output acoustic signal data 85 are also generated.

[0028] The FAD score calculation unit 26 calculates the FAD score of each piece of output acoustic signal data 85. The FAD score calculation unit 26 calculates the FAD score indicating the Frechet Audio Distance based on the following Equation 3.

[0029]

number

[0030] In Equation 3, Xg represents the output acoustic signal data 85, which is a synthesized acoustic signal; Xr represents the reference data set obtained from actual recordings; Σg represents the covariance matrix of Xg; Σr represents the covariance matrix of Xr; μg represents the mean matrix of Xg; and μr represents the mean matrix of Xr. Xr is actual data that meets the conditions of Xg and is created in advance for each condition of Xg. For example, Xr could be a combination of high pressure and abnormality that was only obtained in small amounts. The smaller the FAD score, the better.

[0031] The matching data extraction unit 27 outputs, to the anomaly detection evaluation system 35, output acoustic signal data 85 whose FAD score satisfies a predetermined condition from among the multiple pieces of output acoustic signal data 85 output by the acoustic signal generation unit 24. The predetermined condition is that the FAD score is the smallest or that the FAD score is smaller than a predetermined threshold. The weight management unit 25 changes the weight λ to generate various pieces of output acoustic signal data 85, and the matching data extraction unit 27 extracts output acoustic signal data 85 that is similar to the few pieces of acquired Xr, specifically, output acoustic signal data 85 with a suitable FAD score. Therefore, the newly generated output acoustic signal data 85 whose FAD score satisfies the predetermined condition can be provided to the anomaly detection evaluation system 35.

[0032] The anomaly detection system 36 determines the presence or absence of an anomaly and the degree of the anomaly using the output acoustic signal data 85 output by the matching data extraction unit 27. The anomaly detection system 36 is the subject of evaluation in the anomaly detection evaluation system 35. The details of the operation of the anomaly detection system 36 are not important. The anomaly detection system 36 outputs the determination result to the evaluation unit 37. The evaluation unit 37 compares the output of the anomaly detection system 36 with the output of the state vector generation unit 30 to evaluate the anomaly detection system 36.

[0033] 2 is a diagram showing the configuration of the diffusion model 10. The diffusion model 10 includes a first downsampling block 11, a second downsampling block 12, a third downsampling block 13, a first upsampling block 14, a second upsampling block 15, and a third upsampling block 16.

[0034] A synthetic spectrum 83 is input to the first downsampling block 11 from outside the diffusion model 10. The first downsampling block 11, the second downsampling block 12, and the third downsampling block 13 perform downsampling processing. The downsampling processing is realized by, for example, a convolution function and a ReLu function. The output of the first downsampling block 11 is input to the second downsampling block 12 and the third upsampling block 16.

[0035] The output of the second downsampling block 12 is input to the third downsampling block 13 and the second upsampling block 15. The output of the third downsampling block 13 is input to the first upsampling block 14. The first upsampling block 14, the second upsampling block 15, and the third upsampling block 16 perform upsampling processing. The upsampling processing is realized by, for example, a convolution function, a ReLu function, and a transposed convolution function.

[0036] The first upsampling block 14 receives as input the output of the third downsampling block 13 and the embedding vector 87 output by the embedding layer 34. The second upsampling block 15 receives as input the output of the first upsampling block 14, the output of the second downsampling block 12, and the embedding vector 87 output by the embedding layer 34. The third upsampling block 16 receives as input the output of the second upsampling block 15, the output of the first downsampling block 11, and the embedding vector 87 output by the embedding layer 34. Note that the values ​​output from the embedding layer 34 to each of the first upsampling block 14 to third upsampling block 16 are the same.

[0037] 3 is a flowchart showing the learning process of the diffusion model 10 executed by the learning unit 28. First, in step S401, the learning unit 28 causes the acoustic signal acquisition unit 21 to acquire input acoustic signal data 81. In the following step S402, the learning unit 28 determines hyperparameters to be used by the spectrum generation unit 22. These hyperparameters include, for example, the data length and the number of hops for FFT. In the following step S403, the learning unit 28 causes the spectrum generation unit 22 to perform spectral transformation on the input acoustic signal data 81 to generate an acoustic signal spectrum 82. In the following step S404, the learning unit 28 causes the spectrum synthesis unit 23 to synthesize spectra using multiple weights λ.

[0038] In the following step S411, the learning unit 28 causes the state acquisition unit 31 to acquire the state of the input acoustic signal data 81 acquired by the acoustic signal acquisition unit 21. In the following step S412, the learning unit 28 causes the one-hot encoder 32 to one-hot encode the state acquired by the state acquisition unit 31. In the following step S413, the learning unit 28 causes the state synthesis unit 33 to synthesize states using multiple weights λ. In the following step S414, the learning unit 28 causes the embedding layer 34 to generate an embedding vector 87.

[0039] In the following step S421, the learning unit 28 inputs the synthetic spectrum 83 generated in step S404 and the embedding vector 87 generated in step S414 into the diffusion model 10, and updates the parameters of the diffusion model 10. In the following step S422, the learning unit 28 determines whether the number of epochs has reached a prescribed number. If the learning unit 28 determines that the number of epochs has reached the prescribed number, the process proceeds to step S424, and if the learning unit 28 determines that the number of epochs has not reached the prescribed number, the process proceeds to step S423.

[0040] In this step, a comparison is made between the value of variable E, which is initialized to zero at the start of execution of this flowchart, and a specified value. In step S423, the learning unit 28 increments variable E, which indicates the number of epochs, and returns to step S421. In step S424, the learning unit 28 saves the parameters of the diffusion model 10 and ends the processing shown in FIG. 3.

[0041] 4 is a flowchart showing the data generation process executed by the data generation unit 29. First, in step S431, the data generation unit 29 determines a state. In the following step S432, the data generation unit 29 one-hot encodes the state determined in step S431 using the one-hot encoder 32. In the following step S433, the data generation unit 29 acquires a weight λ from the weight management unit 25. In the following step S434, the data generation unit 29 causes the state synthesis unit 33 to generate a synthesized state value 86 using the weight λ acquired in step S433. In the following step S435, the data generation unit 29 causes the embedding layer 34 to generate an embedding vector 87.

[0042] In the following step S436, the data generation unit 29 loads the parameters of the diffusion model 10 generated or updated by the learning process into the diffusion model 10, and causes the diffusion model 10 to perform inference. In the following step S437, the data generation unit 29 acquires the output spectrum 84 generated by the diffusion model 10. In the following step S438, the data generation unit 29 converts the spectrum into an acoustic signal, that is, generates output acoustic signal data 85 from the output spectrum 84 using the acoustic signal generation unit 24, and the flowchart shown in FIG. 4 ends.

[0043] 5 is a hardware configuration diagram of the data generation system 1. The data generation system 1 includes a CPU 41, which is a central processing unit, a ROM 42, which is a read-only storage device, a RAM 43, which is a read / write storage device, an input / output device 44, which is a user interface, and a communication device 45. The CPU 41 loads a program stored in the ROM 42 into the RAM 43 and executes it to perform the various calculations described above.

[0044] The data generation system 1 may be realized by a field programmable gate array (FPGA), which is a rewritable logic circuit, or an application specific integrated circuit (ASIC), which is an integrated circuit for a specific application, instead of the combination of the CPU 41, the ROM 42, and the RAM 43. Furthermore, the data generation system 1 may be realized by a combination of different configurations, for example, a combination of the CPU 41, the ROM 42, the RAM 43, and the FPGA, instead of the combination of the CPU 41, the ROM 42, and the RAM 43.

[0045] The input / output device 44 includes a mouse and keyboard for receiving input from the user, and a display for presenting information to the user. The input / output device 44 may include a microphone for directly capturing input acoustic signal data 81. The communication device 45 can be used to exchange data with other devices, for example, to acquire the input acoustic signal data 81.

[0046] 5, for convenience, the data generation system 1 is illustrated as being configured from one hardware device, but the data generation system 1 may be configured from multiple hardware devices. In this case, the hardware devices may be installed adjacent to each other, or may be connected via a local area network or the Internet.

[0047] According to the first embodiment described above, the following advantageous effects can be obtained. (1) The data generation system 1 includes a spectrum synthesis unit 23 that synthesizes two acoustic signal spectra 82 using a plurality of weights λ to generate a plurality of synthetic spectra 83, a state synthesis unit 33 that synthesizes a state quantity corresponding to first acoustic signal data and a state quantity corresponding to second acoustic signal data using a plurality of weights λ to generate a plurality of synthetic state quantities, and a diffusion model 10 that has a downsampling block that receives as input the synthetic spectrum 83 and an upsampling block that receives as input the output of the downsampling block and the synthetic state quantity. Therefore, new data can be generated using the diffusion model 10.

[0048] (2) Data generation system 1 includes FAD score calculation unit 26 that calculates the Fréchet phonetic distance for data output by diffusion model 10, and matching data extraction unit 27 that selects data output by diffusion model 10 based on the calculation result of FAD score calculation unit 26. Therefore, generated data with a low FAD score and close to an actual statistical distribution can be extracted.

[0049] (Variation 1) The FAD score calculation unit 26 and the matching data extraction unit 27 are not essential components. The data generation system 1 may input all of the output acoustic signal data 85 output by the acoustic signal generation unit 24 to the anomaly detection evaluation system 35.

[0050] In each of the above-described embodiments and modifications, the functional block configurations are merely examples. Some functional configurations shown as separate functional blocks may be configured as an integrated unit, or a configuration shown in a single functional block diagram may be divided into two or more functions. Furthermore, some of the functions of each functional block may be provided by other functional blocks.

[0051] In the above-described embodiments and modifications, the program is stored in ROM 42. However, the program may be stored in a non-volatile storage device (not shown). Furthermore, the data generation system 1 may include an input / output interface (not shown), and the program may be loaded from another device as needed via the input / output interface and a medium available to the data generation system 1. Here, the medium refers to, for example, a storage medium detachable from the input / output interface, or a communication medium, i.e., a wired, wireless, or optical network, or a carrier wave or digital signal propagating through the network. Furthermore, some or all of the functions realized by the program may be realized by a hardware circuit or FPGA.

[0052] The above-described embodiments and modifications may be combined with each other. Although various embodiments and modifications have been described above, the present invention is not limited to these. Other embodiments conceivable within the scope of the technical concept of the present invention are also included within the scope of the present invention. [Explanation of symbols]

[0053] 1: Data generation system 10: Diffusion model 21: Acoustic signal acquisition unit 22: Spectrum generation unit 23: Spectral synthesis unit 24:Acoustic signal generation section 25: Weight management section 26: FAD score calculation section 27: Relevant data extraction section 28: Learning Department 29: Data generation section 30: State vector generation unit 31: Status acquisition unit 32: One-hot encoder 33: State synthesis unit 34: Embedded layer 35: Anomaly detection evaluation system 36: Anomaly detection system 37: Evaluation section 81: Input acoustic signal data 81-1: Low pressure normal acoustic signal 81-2: Low pressure abnormal acoustic signal 81-3: High voltage normal sound signal 82: Acoustic signal spectrum 82-1: Low pressure normal spectrum 82-2: Low pressure anomaly spectrum 82-3: High-pressure normal spectrum 83: Composite spectrum 84: Output spectrum 85: Output acoustic signal data 86: Composite state value 87: Embedding vector λ: weight

Claims

1. a synthesis unit that synthesizes the first acoustic signal data and the second acoustic signal data using a plurality of weights to generate a plurality of synthetic spectra; a state synthesis unit that synthesizes a state quantity corresponding to the first acoustic signal data and a state quantity corresponding to the second acoustic signal data using the plurality of weights to generate a plurality of synthesized state quantities; A data generation system comprising: a diffusion model having a downsampling block that receives the synthetic spectrum as an input; and an upsampling block that receives the output of the downsampling block and the synthetic state quantity as an input.

2. 2. The data generation system of claim 1, an FAD score calculation unit that calculates a Frechet acoustic distance for generated data that is data output by the diffusion model; The data generation system further comprises a matching data extraction unit that selects the generated data based on the calculation result of the FAD score calculation unit.

3. 3. The data generation system of claim 2, an anomaly detection system to be evaluated; An evaluation unit that evaluates an output of the anomaly detection system, A data generation system, wherein the generated data selected by the matching data extraction unit is input to the anomaly detection system.

Images