Abnormal sound detection device and computer program product

Abstract

The abnormal sound detection device of the present invention detects whether an input sound is an abnormal sound, and includes: a sound feature extraction unit, which extracts the sound feature of the input sound using a feature extraction function; an abnormality calculation unit, which calculates the abnormality from the sound feature of the input sound; and a determination unit, which determines whether the input sound is an abnormal sound based on the acquired abnormality and a predetermined threshold. The feature extraction function is optimized by associating the probability distribution of the abnormality calculated from the feature extracted from the normal sound and the probability distribution of the abnormality calculated from the feature extracted from the abnormal sound, so that in the case of the normal sound, a small abnormality is output, and in the case of the abnormal sound, a large abnormality is output.

Description

[0001] This application is a divisional application of an invention patent application with an application date of March 31, 2017, application number 201780020698.5, and invention name "Abnormal sound detection learning device, sound feature extraction device, abnormal sound sampling device, method thereof and recording medium". Technical Field

[0002] The present invention relates to a technique for detecting abnormal sound of a machine from an acoustic signal, for example, to a technique for generating an acoustic feature extraction function for abnormal sound detection. Background Art

[0003] Large-scale manufacturing machines, molding machines and other business equipment installed in factories and the like will cause great trouble to the business if they stop operating due to a malfunction. Therefore, it is necessary to monitor their operating conditions on a daily basis and respond immediately after an abnormality occurs. As a solution, there is a method in which the manager of business equipment regularly dispatches maintenance personnel to the site to check the wear of parts. However, it is difficult to implement this method in all business equipment or factories because it costs a lot of labor costs, moving costs and labor.

[0004] As a solution, there is a method of installing a microphone inside the machine to monitor its operating sound on a daily basis. The operating sound is analyzed, and a sound considered abnormal (for example, an abnormal sound) is detected after it occurs, and it is solved by issuing an alarm. However, in setting the type of abnormal sound or its detection method for each type or individual of the machine, it is costly to monitor manually. Therefore, it is necessary to automatically design a rule for detecting abnormal sounds in an automatic manner.

[0005] As a method for solving this problem, abnormal sound detection based on statistical methods is known (for example, refer to non-patent document 1). Abnormal sound detection based on statistical methods can be roughly divided into abnormal sound detection with a teacher and abnormal sound detection without a teacher. In abnormal sound detection with a teacher, the identifier is learned from the learning data of normal sound and abnormal sound, while in abnormal sound detection without a teacher, the identifier is learned only from the learning data of normal sound. In industrial applications, because it is difficult to collect learning data of abnormal sound, abnormal sound detection without a teacher is adopted in most cases.

[0006] The learning and detection process of abnormal sound detection without a teacher is as follows Fig. 9Like that. During learning, the sound feature quantity obtained from the sound data (learning data) during normal operation is extracted. After that, the normal sound model (probability density function) is learned from the sound feature quantity. Then, in the judgment, the sound feature quantity is extracted in relation to the newly obtained observation, and the negative log likelihood (for example, abnormality) is evaluated by the learned normal sound model. If the value is less than the threshold, it is judged to be normal, and if it is greater than the threshold, it is judged to be abnormal. This is to evaluate the goodness of the agreement between the observed sound and the normal sound model. This is based on the way of thinking that if the observation is a normal sound, a sound "similar" to the learning data of the normal sound should be emitted, and if it is abnormal, a sound "dissimilar" to the learning data of the normal sound should be emitted.

[0007] What becomes a problem in unsupervised anomaly detection is the design of the sound feature extraction function F(·). In abnormal sound detection with a teacher, a sound feature that can be well identified by the discrimination object is manually designed. For example, if it is known that the normal sound is a sine wave of 1000Hz and the abnormal sound is a sine wave of 2000Hz, then because the timbre is different, the logarithmic power (log-MFBO) of the Mel filter bank is extracted for each frame. If the normal sound is an unchanging engine sound and the abnormal sound is a sound like a "click" collision between devices, the abnormal sound is a sudden sound, so the time difference (ΔMFBO) of the power of the Mel filter bank is extracted. Moreover, in "deep learning", which is one of the learning with a teacher, it is also called the ability to automatically design sound features from learning data.

[0008] Prior art literature

[0009] Non-patent literature

[0010] Non-patent document 1: Tsuyoshi Ide, Masaru Sugiyama, "Exceptional knowledge and changing knowledge," Kodansha, pp. 6-7, 2015. Summary of the invention

[0011] Problems to be solved by the invention

[0012] However, in anomaly detection without a teacher, it is unknown what kind of sound characteristics the abnormal sound has. Therefore, it is difficult to manually design a sound feature extraction function, and it is also difficult to use deep learning. For example, if the normal sound is a sine wave of 1000Hz, and if the abnormal sound is assumed to be a sine wave of 2000Hz, and log-MFBO is used as the sound feature, abnormal sounds such as "clicks" between devices cannot be detected. And vice versa. Therefore, only the Mel filter bank cepstral coefficients (MFCC) as a general sound feature can be used, and the detection accuracy is poor compared to learning with a teacher.

[0013] An object of the present invention is to provide an abnormal sound detection learning device that can generate an acoustic feature extraction function for abnormal sound detection regardless of the presence or absence of abnormal sound learning data, as well as an acoustic feature extraction device, an abnormal sound sampling device, a method and a program associated with the abnormal sound detection learning device.

[0014] Means for solving problems

[0015] An abnormal sound detection learning device according to one embodiment of the present invention includes: a first sound feature extraction unit that uses a sound feature extraction function to extract sound features of normal sound based on learning data of normal sound; a normal sound model updating unit that uses the sound features extracted by the first sound feature extraction unit to update a normal sound model; a second sound feature extraction unit that uses the sound feature extraction function to extract sound features of abnormal sound based on simulated abnormal sound and extracts sound features of normal sound based on learning data of normal sound; and an sound feature extraction function updating unit that updates the sound feature extraction function using the sound features of abnormal sound extracted by the second sound feature extraction unit and the sound features of normal sound, and repeatedly performs the processing of the first sound feature extraction unit, the normal sound model updating unit, the second sound feature extraction unit, and the sound feature extraction function updating unit.

[0016] An abnormal sound detection learning device according to one embodiment of the present invention includes: a first sound feature extraction unit for extracting sound features of normal sound based on learning data of normal sound using a sound feature extraction function; a normal sound model updating unit for updating a normal sound model using the sound features extracted by the first sound feature extraction unit; a second sound feature extraction unit for extracting sound features of abnormal sound based on learning data of abnormal sound using the sound feature extraction function and extracting sound features of normal sound based on learning data of normal sound; and an sound feature extraction function updating unit for updating the sound feature extraction function using the sound features of abnormal sound extracted by the second sound feature extraction unit and the sound features of normal sound, and repeating the processing of the first sound feature extraction unit, the normal sound model updating unit, the second sound feature extraction unit, and the sound feature extraction function updating unit.

[0017] An abnormal sound detection device according to one embodiment of the present invention detects whether an input sound is an abnormal sound, and includes: a sound feature extraction unit for extracting the sound feature of the input sound using a feature extraction function; an abnormality calculation unit for calculating the abnormality from the sound feature of the input sound; and a determination unit for determining whether the input sound is an abnormal sound based on the acquired abnormality and a specified threshold value, wherein the feature extraction function is optimized by associating a probability distribution of the abnormality calculated from a feature extracted from a normal sound with a probability distribution of the abnormality calculated from a feature extracted from an abnormal sound, so that in the case of a normal sound, a small abnormality is output, and in the case of the abnormal sound, a large abnormality is output.

[0018] An abnormal sound sampling device according to one embodiment of the present invention comprises: an abnormal sound sampling unit, wherein an amplitude spectrum of the abnormal sound is based on the normal sound, is different for each abnormal sound, and is induced by sparsity, and an average spectrum of a stable noise included in an observation signal is added to generate an abnormal sound smoothed in the time-frequency direction.

[0019] Effects of the Invention

[0020] An acoustic feature extraction function for abnormal sound detection can be generated regardless of the presence or absence of abnormal sound learning data. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a block diagram for explaining an example of an abnormal sound detection learning device.

[0022] Figure 2 This is a flowchart for explaining an example of an abnormal sound detection learning method.

[0023] Figure 3 This is a block diagram for explaining an example of an abnormal sound detection device.

[0024] Figure 4 This is a flowchart for explaining an example of an abnormal sound detection method.

[0025] Figure 5 It is a diagram for explaining an example of the structure of the present invention.

[0026] Figure 6 It is a diagram for explaining an example of an acoustic feature quantity function.

[0027] Figure 7 This is a diagram showing an example of the frequency distribution of the amplitude spectrum of a single tone of an acoustic signal.

[0028] Figure 8 It is a diagram showing an example of the sparseness of the amplitude spectrum.

[0029] Fig. 9It is a diagram for explaining the conventional technology. DETAILED DESCRIPTION [Summary of the invention]

[0031] [[About the structure of the invention]]

[0032] Figure 5 Represents the hierarchical structure of the present invention.

[0033] (I) Regardless of the presence or absence of a teacher, the "normal sound model" and the "acoustic feature extraction function" are optimized simultaneously from the learning data.

[0034] (II) Learning data of sampling abnormal sounds without a teacher.

[0035] (III) Regardless of the presence or absence of a teacher, the "sound feature extraction function" is expressed, for example, using matrix product and logarithmic operations, or a multilayer perceptron.

[0036] (IV) Provide a learning / detection algorithm that simultaneously implements (I) to (III).

[0037] Hereinafter, (I) to (III) will be described. For the sake of convenience, please understand that the order of description is (III), (I), and (II). After that, the "abnormal sound detection learning device and method" for executing them will be described.

[0038] [[About (III) Definition of sound feature extraction function]]

[0039] Perform Fourier transform on the observed signal and set the sequence of extracted power spectrum or amplitude spectrum as X ω,t ∈R + Ω×T Here, ω = {1, 2, ..., Ω} and t = {1, 2, ..., T} represent the frequency and time indices, Ω represents the number of frequency bins, and T represents the number of observation frames. + Represents the set of non-negative real numbers.

[0040] The acoustic feature extraction can basically be expressed as the result of applying logarithmic functions and matrix operations to the sequence. For example, log-MFBO can be expressed as the spectrum X of all frequencies for a certain frame. t =(X 1,t ,...,X Ω,t ) T Multiply by M Mel filter banks mm and take the logarithm of the result.

[0041]

[0042] Here, T represents transpose. If ΔMFBO is defined as the time difference of the output of the Mel filter bank, the difference matrix D can be used as described below.

[0043]

[0044] Here, E M is the identity matrix of dimension M.

[0045] Here, we extend this concept and define an acoustic feature extraction function. First, we extract the past P from the target frame t. 1 Frame to the future P 2 frame, and set the result of vertically concatenating it to x t .x t like Figure 6 And it can be expressed as the following formula.

[0046]

[0047] Where, Q = Ω × (P 1 +P 2 +1). The change from the first line to the second line of formula (1') is a rewrite for the sake of simplified notation, and the content and order of the elements remain unchanged. t Multiply by the matrix A∈R D×Q , the logarithmic operation is set as the acoustic feature extraction function. That is, as follows.

[0048]

[0049]

[0050] Therefore, it can be said that the optimization of the acoustic feature extraction function is the optimization of the matrix A.

[0051] [[(I) Simultaneous optimization of normal sound model and acoustic feature extraction function]]

[0052] First, let the normal sound model (the probability density function followed by the feature quantity extracted from the normal sound) be p(F(A, x)|z=0). Then, the negative log likelihood (abnormality) can be calculated as follows.

[0053] g(A,x)=-lnp(F(A,x)|z=0) (5)

[0054] Here, various models can be used as the normal sound model, but if it is set as a mixed Gaussian distribution, for example, it can be described as:

[0055]

[0056]

[0057] Here, K is a mixed number, k, Σ k 、w k are the mean vector, covariance matrix, and weight parameter of the kth distribution respectively.

[0058] In abnormal sound detection, if the value of g(A,x) is greater than the threshold If it is less than , it is judged as "abnormal". This judgment can be described as a step function, but for the convenience of subsequent calculations, the sigmoid function is used.

[0059]

[0060] The sigmoid function 0≦σ{α}≦1, the smaller α is, the closer the output is to 0, and the larger α is, the closer the output is to 1. That is, if If it is 0.5 or more, it is judged as abnormal, and if it is 0.5 or less, it is judged as normal.

[0061] The goal of the abnormal sound detection algorithm is to output a large When the sound is normal, the output is small. That is, minimizing the following equation is the goal of optimizing abnormal sound detection.

[0062] L=-∫s{g(A,x)-f}(p(F(A,x)|z=1)-p(F(A,x)|z=0))dx (9)

[0063] Here, p(F(A,x)|z=1) is an abnormal sound model (a probability density function to which a feature quantity extracted from an abnormal sound conforms).

[0064] From the above, the normal sound model and the sound feature extraction function are optimized simultaneously to satisfy

[0065]

[0066]

[0067] It can be said that the optimization of acoustic feature extraction is the design of the matrix A that optimizes the formula (11).

[0068] The optimization of equation (10) can be performed by the maximum likelihood method, etc. Therefore, the optimization of equation (11) is considered. The most difficult operation in calculating equation (9) is the integral operation. Therefore, the integral is replaced by the arithmetic mean of the learning data. First, the objective function is set to,

[0069] L=-∫s{g(A,x)-f}p(F(A,x)|z=1)dx+∫s{g(A,x)-f}p(F(A,x)|z=0)dx (12).

[0070] Thus, each item is considered The expected value calculation of , so it can be described as follows.

[0071] L=-E[s{g(A,x)-f}] x|z＝1 +E[s{g(A,x)-f}] x|z＝0 (13)

[0072] Here, E[·] x|z＝0 It is related to the normal sound and is the expected value operation, E[·] x|z＝1 This is the expected value calculation related to abnormal sound.

[0073] Here, if the expected value calculation is replaced by the arithmetic mean, equation (11) can be approximated as follows.

[0074]

[0075] Here, N is the time index of the learning data of abnormal sound. That is, Ψ is calculated using the learning data of normal sound. 1,...,T The arithmetic mean of the outliers is calculated using the learning data of the outliers. 1,...,T If A is designed so that the sum of the two arithmetic means is minimized, the accuracy of abnormal sound detection is maximized. Here, in the case of abnormal sound detection with a teacher who obtains the learning data of abnormal sound, it is sufficient to execute it as it is. In the case of abnormal sound detection without a teacher, it is sufficient to perform N sampling (simulation) of the abnormal sound data according to the sampling algorithm explained from now on. At this time, it should be set to the extent of N = T.

[0076] The objective function can be set by equation (15), so it can be minimized. The slope method can be used in minimization. If the steepest descent method is used in the slope method,

[0077]

[0078] , for each dimension d, the matrix can be optimized. Here, λ is the step size. Of course, the improved algorithm of the steepest descent method, such as AdaGrad, can also be used in this optimization.

[0079] When the normal sound model is expressed by a mixed Gaussian distribution, the slope vector The chain rule of partial differentials can be used to perform the calculation as follows.

[0080]

[0081] Among them, each component can be

[0082]

[0083]

[0084]

[0085]

[0086]

[0087]

[0088]

[0089] Perform calculations.

[0090] [[(II) Sampling of Learning Data on Abnormal Sounds]]

[0091] In the unsupervised abnormal sound detection, a method of sampling abnormal sound learning data is described in order to calculate the equation (15).

[0092] The distribution used in the abnormal sound model p(F(A,x)|z=1) is arbitrary. In unsupervised learning, there is no prior knowledge about abnormal sounds. In order to detect all abnormal sounds, it is desirable to construct an abnormal sound model with as few assumptions as possible. In other words, it is desirable to sample abnormal sounds only based on generally available knowledge about the physical properties or occurrence frequency of the amplitude spectrum or power spectrum.

[0093] In the following, an abnormal sound model is constructed by setting assumptions only on the frequency, sparseness, and continuity in the time-frequency direction of the amplitude value.

[0094] First, consider X ω,n size. Figure 7 It is the frequency distribution of the amplitude spectrum of a single tone of the sound signal. The horizontal axis is the intensity and the vertical axis is the frequency of occurrence. Figure 7 It can be seen that the amplitude spectrum of the sound signal is most likely to appear near 0, and as the intensity increases, its frequency of occurrence decreases exponentially. ω,n Sampling should be done from a probability distribution that satisfies such properties.

[0095] An example of a distribution that satisfies this characteristic is the exponential distribution.

[0096]

[0097] The distribution of is that the larger the value of o is, the lower the frequency of occurrence is. And β is the average value. ω,n The sampling situation is as follows.

[0098]

[0099] Β is a parameter related to the volume of the observed sound, so it can be set based on the learning data of normal sound.

[0100] For example, you can

[0101]

[0102] Wait for setting.

[0103] If there is prior knowledge of the stationary noise contained in the observed signal, the sampling can be changed appropriately. For example, if the average spectrum Y of the stationary noise is ω is known, then as

[0104] X w,n =U w +E w,n (28)

[0105]

[0106] At this time, β can be obtained by

[0107]

[0108] And other calculations.

[0109] Furthermore, when sampling is performed using equation (29), the average volume of the abnormal sound β ω It is not time-dependent and is fixed. The amplitude value may increase or decrease depending on the distance from the microphone or the type of abnormal sound. Therefore, it is desirable to make β ω,n To express this, for example, equation (29) can be changed as follows.

[0110]

[0111] β w,n =g w ×b w 10 θ / 20 (32)

[0112]

[0113] b ω ~Uniform(0,1) (34)

[0114] Here, θ is a parameter for correcting the maximum volume of abnormal sound, and its unit is dB. Uniform(a,b) is a continuous uniform distribution with a maximum value a and a minimum value b. For example, θ can be set to a value from 0 to 12. In addition, θ is sampled from a normal distribution, etc., and b is not used. ω The same effect can be expected by always setting the sampling to 1.

[0115] Furthermore, we can focus on the sparseness of the amplitude spectrum (most of the elements are 0). Figure 8 Is the X that draws a certain frame 1 ,..., Ω,t Example. Figure 8 In many cases, the sound signal has only one to several frequency components in a single frame. In view of this, the sampling of equation (31) can be changed as follows.

[0116]

[0117]

[0118] z n ~Cat(1 Ω / W) (37)

[0119] Here, Cat(π) is the category distribution, 1 Ω is a vector of length Ω whose elements are all 1. This is done by z n The number of activated spectral bins is set to one at each time point, and the abnormal audio spectrum E is sparsely induced.

[0120] Moreover, when sampling through equation (35), E ω,n Generates completely independent values ​​in the time and frequency directions. Usually, the sound signal changes smoothly over time, so its spectrum changes smoothly in the time and frequency directions. To achieve this, you can also use E ω,n Smoothing in the time-frequency direction.

[0121] Various methods are considered for smoothing, but for example, using a Gaussian filter, consider

[0122] Xw,n=Yw+conv2(Ew,n,G(F,T)) (38)

[0123] Etc. Here, conv2 is a two-dimensional convolution operation, and G(F,T) is a two-dimensional Gaussian filter of size F×T. Each parameter should be controlled experimentally, but is set to, for example, F=11 and T=31.

[0124] [Abnormal sound detection learning device and method]

[0125] like Figure 1 As shown in the example, the abnormal sound detection learning device includes: a spectrum calculation unit 1; an initialization unit 2; a first sound feature extraction unit 3; a normal sound model update unit 4; an abnormal sound sampling unit 5; a second sound feature extraction unit 6; a sound feature extraction function update unit 7; and a control unit 8. The abnormal sound detection learning method is executed by each unit of the abnormal sound detection learning device. Figure 2 And the processing of steps S1 to S82 described below is implemented.

[0126] <Spectrum calculation unit 1>

[0127] The learning data of normal sounds is input to the spectrum calculation unit 1 .

[0128] The learning data of normal sound is the sound signal collected by a microphone installed near (or inside) the device to be detected as abnormal sound. The sampling frequency when collecting the sound signal should be set to a frequency that can record normal sound, and can be set to, for example, 16kHz.

[0129] The spectrum calculation unit 1 transforms the learning data into the frequency domain by a method such as short-time Fourier transform (STFT).

[0130] After that, take the absolute value of the complex spectrum to get the amplitude spectrum X ω,t (Step S1) Alternatively, the power spectrum obtained by square the amplitude spectrum may be set as X ω,t Here, the Fourier transform length can be set to 32ms, the offset width can be set to 16ms, and so on.

[0131] Below, X, which is the amplitude spectrum or power spectrum, ω,t Abbreviated as spectrum X ω,t . Spectrum X ω,t The signal is output to the first acoustic feature extraction unit 3 and the second acoustic feature extraction unit 6 .

[0132] Furthermore, when there is learning data of abnormal sound, in other words, when there is learning by a teacher, the spectrum calculation unit 1 calculates the spectrum of the abnormal sound based on the learning data of the abnormal sound. The spectrum of the abnormal sound is, for example, an amplitude spectrum or a power spectrum of the abnormal sound. The calculated spectrum of the abnormal sound is output to the first acoustic feature extraction unit 3 and the second acoustic feature extraction unit 6.

[0133] <Initialization Unit 2>

[0134] Preset parameters of the learning algorithm and parameters of the acoustic feature extraction function are input to the initialization unit 2 .

[0135] Various parameters can be set as P 1 =3,P2 =3,ε=min(X ω,t ) / 2, λ=5.0×10 -3 wait.

[0136] The initialization unit 2 initializes the matrix A of the acoustic feature extraction function and the normal sound model (step S2 ). The initialized matrix A and the normal sound model are output to the first acoustic feature extraction unit 3 .

[0137] The matrix A can be initialized by uniform random numbers, for example. The normal sound model differs according to the order of the probability density function, but when a mixed Gaussian distribution is used, it can be initialized by, for example, a k-means algorithm or the like.

[0138] <First sound feature extraction unit 3>

[0139] The first acoustic feature quantity extraction unit 3 receives the acoustic feature quantity function initialized by the initialization unit 2 and the learning data of the normal sound.

[0140] The first acoustic feature extraction unit 3 extracts the acoustic feature of the normal sound from the learning data of the normal sound using the acoustic feature extraction function (step S3). In this example, the first acoustic feature extraction unit 3 uses the spectrum X calculated by the spectrum calculation unit 1 based on the learning data of the normal sound as the learning data of the normal sound. ω,t , extracting the acoustic feature of the normal sound. The extracted acoustic feature is input to the normal sound model updating unit 4.

[0141] The first acoustic feature extraction unit 3 performs the first processing using the acoustic feature extraction function initialized by the initialization unit 2. The second and subsequent processing by the first acoustic feature extraction unit 3 uses the acoustic feature extraction function updated by the acoustic feature extraction function update unit 7.

[0142] When the spectrum calculation unit 1 calculates the spectrum of the abnormal sound based on the learning data of the abnormal sound, the first acoustic feature extraction unit 3 extracts the acoustic feature of the abnormal sound based on the calculated spectrum of the abnormal sound using an acoustic feature extraction function.

[0143] <Normal sound model update unit 4>

[0144] The normal sound model updating unit 4 updates the normal sound model using the acoustic feature extracted by the first acoustic feature extraction unit 3 (step S4).

[0145] The updating method varies according to the type of probability density function of the normal sound model, but it can be updated by, for example, maximum likelihood estimation. Moreover, when a mixed Gaussian distribution is used as the normal sound model, the updating can be equivalent to one to several steps by the EM algorithm (for example, see reference 1).

[0146] [Reference 1] Konishi Sadatori, "Introduction to Multi-Quantity Analysis, Fulu C EMアルゴリズム" pp.294-298, Iwanami Shoten, 2010.

[0147] <Abnormal sound sampling unit 5>

[0148] The abnormal sound sampling unit 5 samples the simulated abnormal sound (step S5 ). The sampled abnormal sound is output to the second acoustic feature extraction unit 6 .

[0149] The abnormal sound sampling unit 5 performs the sampling of the simulated abnormal sound according to, for example, equation (26). The parameter β in equation (26) can be determined according to, for example, equation (27). In this case, X in equation (27) is ω,t , the spectrum X of the normal sound calculated by the spectrum calculation unit 1 can be used ω,t .

[0150] Furthermore, when learning data of abnormal sounds exist, in other words, when learning is done by a teacher, sampling is not performed. That is, the learning data of abnormal sounds may be used as sampling results for subsequent processing.

[0151] <Second sound feature extraction unit 6>

[0152] The second acoustic feature extraction unit 6 uses an acoustic feature extraction function to extract the acoustic feature of the abnormal sound based on the simulated abnormal sound and extract the acoustic feature of the normal sound based on the learning data of the normal sound (step S6). The extracted acoustic feature of the abnormal sound and the acoustic feature of the normal sound are output to the acoustic feature extraction function updating unit 7.

[0153] In this example, the second acoustic feature extraction unit 6 uses the spectrum X calculated by the spectrum calculation unit 1 based on the normal sound learning data as the normal sound learning data. ω,t , extract the sound features of normal sound.

[0154] The first processing of the second acoustic feature extraction unit 6 is performed using the acoustic feature extraction function initialized by the initialization unit 2. The second and subsequent processing of the second acoustic feature extraction unit 6 is performed using the acoustic feature extraction function updated by the acoustic feature extraction function update unit 7.

[0155] <Acoustic feature extraction function updating unit 7>

[0156] The acoustic feature extraction function updating unit 7 updates the acoustic feature extraction function using the acoustic feature of the abnormal sound and the acoustic feature of the normal sound extracted by the second acoustic feature extraction unit 6 (step S7). The updated acoustic feature extraction function is output to the first acoustic feature extraction unit 3 and the second acoustic feature extraction unit 6. Furthermore, the acoustic feature extraction function that is finally updated after the control unit 8 repeatedly performs control is output as the final learning result of the abnormal sound detection learning device and method.

[0157] The acoustic feature extraction function updating unit 7 updates the acoustic feature extraction function according to, for example, equation (16). In this case, the learning data x of normal sound may be used before the update. 1 ,…,x T and the sampled abnormal sound data x 1 ,…,x N , update the threshold by the following formula Thus, the convergence is stable.

[0158]

[0159] <Control unit 8>

[0160] The control unit 8 controls to repeat the processing of the first acoustic feature extraction unit 3, the normal sound model updating unit 4, the second acoustic feature extraction unit 6, and the acoustic feature extraction function updating unit 7 (steps S81 and S82). This repeated processing is performed until the acoustic feature extraction function and the normal sound model converge.

[0161] For example, the control unit 8 controls to repeatedly perform the processing of the second acoustic feature extraction unit 6 and the acoustic feature extraction function update unit 7 (hereinafter referred to as the first iterative processing). The first convergence determination condition is set to whether the number of times the processing has been performed reaches a certain number (for example, 50 times), or whether the update norm (norm) of the matrix A is fixed or less, and the control unit 8 controls to repeatedly perform the first iterative processing until the first convergence condition is satisfied, in other words, until it is determined that convergence has been achieved. Other conditions may also be used as the first convergence determination condition.

[0162] After that, the control unit 8 controls to repeatedly perform the processing of the first acoustic feature extraction unit 3 and the normal sound model updating unit 4 and the first iterative processing (hereinafter referred to as the second iterative processing). The second convergence condition is set to whether the number of times the processing is performed reaches a certain number of times (for example, 200 times), and the control unit 8 controls to repeatedly perform the second iterative processing until the second convergence condition is satisfied, in other words, it is determined that convergence has been achieved. Other conditions may also be used as the second convergence determination condition.

[0163] [Abnormal sound detection device and method]

[0164] like Figure 3 As shown in the example, the abnormal sound detection device includes: a spectrum calculation unit 11; an acoustic feature extraction unit 12; an abnormality calculation unit 13 and a determination unit 14. The abnormal sound detection method is performed by each unit of the abnormal sound detection device. Figure 4 And the processing of steps S11 to S14 described below is implemented.

[0165] <Spectrum calculation unit 11>

[0166] The operating sound of the machine to be detected as an abnormal sound is collected by a microphone. The sampling rate used in this case is the same as that used in learning. The collected sound signal is input to the spectrum calculation unit 11.

[0167] The spectrum calculation unit 11 is similar to the spectrum calculation unit 1, and obtains the spectrum X according to the received sound signal. ω,t (Step S11). The obtained spectrum X ω,t The signal is output to the acoustic feature extraction unit 12 .

[0168] <Acoustic feature extraction unit 12>

[0169] The sound feature extraction unit 12 uses the obtained spectrum X ω,t According to the sound feature extraction function output as the final learning result of the abnormal sound detection learning device and method, the sound feature of the received sound signal is extracted (step S12). The extracted sound feature is output to the abnormality calculation unit 13.

[0170] <Abnormality degree calculation unit 13>

[0171] The abnormality calculation unit 13 uses the extracted acoustic feature and the normal sound model output as the final learning result by the abnormal sound detection learning device and method to calculate the negative log likelihood g(A, x) (step S13). Hereinafter, the negative log likelihood is referred to as the abnormality. The calculated abnormality is output to the determination unit 14.

[0172] <Determination unit 14>

[0173] If the abnormality of the current frame is the threshold If the above is true, the determination unit 14 outputs "abnormal" (step S14). The threshold value should be adjusted to match the machine or environment, and should be set to about 1500, for example.

[0174] Furthermore, similar to the sound section determination, a "hangover" that suppresses determination errors by a heuristic rule may be used. The hangover process to be applied should be set in accordance with the type of erroneous detection of abnormal sound in consideration of various circumstances.

[0175] As an example, sometimes musical noise generated during noise suppression is judged as a sudden abnormal sound. Sudden collision sounds often have a change in the spectrum shape over 100ms, so the abnormality level is above the threshold for (100 / STFT frame offset width) frames continuously. However, musical noise only has abnormal amplitude spectrum values ​​in this frame, so the abnormality level is above the threshold for at most several frames continuously. Therefore, the rule for abnormality judgment can be set as "if there are continuous F 1 If the abnormality degree of the frame is above the threshold, "abnormal" is output.

[0176] As another example, since the volume of the abnormal sound is low, it is considered that the abnormality level continues for a long time at a level slightly lower than the threshold. In such a case, as a judgment rule for continuous abnormal sound, it is possible to add "if the recent F 2 The sum of the abnormality of the frames is The above is detected as an abnormal sound. " and other rules. Here, This should be determined by tuning, but can be set, for example, to about.

[0177] By using such an abnormal sound detection device and method, by detecting abnormal sounds from large-scale manufacturing machines and molding machines installed in factories, etc., it is possible to respond to failures quickly and predict failures. This can contribute to the efficiency of industry, especially manufacturing, for example.

[0178] [Program and recording medium]

[0179] When implementing each process in the abnormal sound detection learning device or the abnormal sound detection device by a computer, the process contents of the functions that the abnormal sound detection learning device or the abnormal sound detection device should have are described in a program, and then the program is executed by the computer to implement each process on the computer.

[0180] The program describing the processing contents may be recorded in a computer-readable recording medium, such as a magnetic recording device, an optical disk, a magneto-optical recording medium, or a semiconductor memory.

[0181] Furthermore, each processing unit may be configured by executing a predetermined program on a computer, and at least a part of the processing contents may be realized as hardware.

[0182] [Modifications]

[0183] If the optimization index of the acoustic feature quantity described above is used, the multilayer perceptron can be learned in an unsupervised manner. Specifically, the acoustic feature quantity extraction function is described using equation (3), but it can also be actually implemented in the multilayer perceptron. For example, when the simplest fully combined multilayer perceptron is used, the acoustic feature quantity extraction function is as follows. In this way, the acoustic feature quantity extraction function can also be expressed by the multilayer perceptron.

[0184] F(x τ )=W (L) g τ (L-1) +b (L) (40)

[0185]

[0186] Here, L, W(k), and b(k) represent the number of layers, weight matrix, and bias vector of the neural network, respectively. L is a specified positive integer greater than 2. Moreover, σ θ is an activation function, which can be actually implemented in, for example, a sigmoid function. The input to the input layer is set to g τ (1) =x τ or τ (1) =ln x τ That's it. τ (k) is the input to the k-th layer of the multilayer perceptron. In this case, the optimization of the acoustic feature extraction function is the optimization of Θ={W(k), b(k)}, (k∈{2, ..., L}).

[0187] This optimization is the same as equation (16), and the slope method can be used. If the steepest descent method is used in the slope method, then

[0188]

[0189]

[0190] Of course, in this optimization, an improved algorithm of the steepest descent method, such as AdaGrad, can be used. The objective function J is obtained by

[0191]

[0192] is provided, equations (42) and (43) can be expressed as

[0193]

[0194]

[0195] Each slope

[0196]

[0197] From formula (17), using the chain rule of partial differentials, it can be calculated as follows.

[0198]

[0199]

[0200] In a multilayer perceptron, the calculation of the slope in the form of equations (47) and (48) can be efficiently calculated by using error back propagation (for example, see reference 2).

[0201] [Reference 2] Hideki Aso, No. 7, ""Deep Learning," Society for Artificial Intelligence, Modern Science Society, pp. 14, 2015.

[0202] The processes described in the abnormal sound detection learning device or the abnormal sound detection device may be executed not only in time series according to the order described but also in parallel or individually according to the processing capability or need of the device executing the processes.

[0203] In addition, it is needless to say that appropriate changes can be made without departing from the spirit of the present invention.

Claims

1. An abnormal sound detection device, detecting whether an input sound is an abnormal sound, comprising: A sound feature extraction unit, which extracts the sound feature of the input sound using a feature extraction function; an abnormality calculation unit, which calculates the abnormality according to the acoustic feature quantity of the input sound; and a determination unit, which determines whether the input sound is an abnormal sound according to the obtained abnormality degree and a predetermined threshold value, The feature extraction function is optimized based on the probability density function followed by the feature extracted from the normal sound and the probability density function followed by the feature extracted from the abnormal sound. The optimization is performed so that the abnormality calculated based on the feature amount extracted from the normal sound is less than the threshold value, and the abnormality calculated based on the feature amount extracted from the abnormal sound is greater than the threshold value, Let A be a matrix, let the feature extraction function be F(A,x), let the probability density function followed by the feature extracted from the normal sound be p(F(A,x)|z=0), let the probability density function followed by the feature extracted from the abnormal sound be p(F(A,x)|z=1), let σ be a sigmoid function, let the threshold be The abnormality calculated based on the feature extracted from the normal sound is defined as g(A, x). g(A, x) is defined by the following equation (5): g(A,x)=-lnp(F(A,x)|z=0) (5) The probability density function followed by the feature extracted from the normal sound and the feature extraction function are simultaneously optimized in such a way that the following equations (10) and (11) are simultaneously satisfied. L=-∫s{g(A,x)-f}(p(F(A,x)z=1)-p(F(A,x)z=0))dx (9).

2. The abnormal sound detection device according to claim 1, When the abnormality exceeds a predetermined threshold, the input sound is determined to be an abnormal sound. The threshold value is obtained based on the abnormality degree of the normal sound and the abnormality degree of the abnormal sound.

3. The abnormal sound detection device according to claim 1, The probability density function followed by the feature quantity extracted from the normal sound is obtained from the first number of normal sounds, The feature quantity extracted from the abnormal sound follows a probability density function obtained from the abnormal sound whose number is the same as the first number and which has been sampled.

4. A computer program product, comprising a computer program, wherein the computer program causes a computer to function as each unit of the apparatus of any one of claims 1 to 3.

Images