Abnormal sound detection system and method

Abstract

To provide an abnormal sound detection system which can detect abnormal sound in high accuracy, even when background noise is mixed into input sound.SOLUTION: An abnormal sound detection system performs abnormality determination for an input signal of an abnormal sound detection object of an object machine, by using a learning result obtained by learning a set of pairs of a normal sound signal of the object machine and operation section information indicating an actual operation section of the object machine (1000). The system: estimates the operation section of the object machine by using the learning result from the input signal of the abnormal sound detection object, when receiving the input signal of the abnormal sound detection object of the object machine (21); and determines the abnormality of the input signal of the abnormal sound detection object of the object machine, on the basis of a degree of divergence between the estimated operation section, and the actual operation section indicated in the operation section information (2000).SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an abnormal sound detection system and method. [Background technology]

[0002] Abnormal sound detection technology is a technology that determines whether a sound is normal or abnormal. An abnormal sound is a sound that differs from normal sounds. Abnormal sound detection technology can detect abnormal sounds caused by malfunctions in factory machinery, etc., and is useful for automating machine maintenance and conservation. Some machines operate continuously, while others alternate between operating and not operating.

[0003] In Non-Patent Document 1, an autoencoder is trained on the features of normal sounds. Next, in Non-Patent Document 1, abnormal sounds are detected based on the reconstruction error of the autoencoder, and sounds that differ from the trained sound features are detected as abnormal sounds. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] E. Marchi, “A novel approach for automatic acoustic novelty detection using a denoising autoencoder with Bidirectional LSTM neural networks,” in Proc of ICASSP, 2015 Summary of the Invention [Problem to be solved by the invention]

[0005] In cases where a variety of machines are operating simultaneously in a factory, the input sound to an abnormal sound detection system may include background noise other than the sound from the machine being detected (hereinafter referred to as the target machine). For example, if the operating sound of the target machine is quieter than the background noise, the difference between the normal operating sound of the target machine (hereinafter referred to as the normal operating sound) plus background noise and the abnormal operating sound of the target machine (hereinafter referred to as the abnormal operating sound) plus background noise becomes small. Therefore, the abnormal sound detection system faces a first problem: the difference between the abnormality level obtained by inputting a mixture of normal operating sound and background noise and the abnormality level obtained by inputting a mixture of abnormal operating sound and background noise becomes small, making it difficult to accurately determine whether a sound is normal or abnormal. Furthermore, there is a second problem: if background noise is mixed in a frequency band different from the main frequency band of normal operating sound, the abnormal sound detection system may erroneously determine the input sound as abnormal.

[0006] An abnormal sound detection system (hereinafter referred to as an example of an abnormal sound detection system) that serves as a comparative example for comparison with the abnormal sound detection system of the present disclosure, rather than as prior art, could include, for example, a feature extraction unit, a likelihood calculation unit, and an abnormality determination unit.

[0007] The feature extraction unit extracts time-frequency domain features from the time waveform of the input sound. The likelihood calculation unit estimates negative log-likelihood of the time-frequency domain features. The likelihood calculation unit is pre-trained to output a low negative log-likelihood for time-frequency domain features of normal sound samples. The likelihood calculation unit then outputs the estimated negative log-likelihood as the degree of abnormality of the sound. Finally, if the degree of abnormality is equal to or greater than a predetermined threshold, the abnormality determination unit determines the input sound to be an abnormal sound and outputs a determination result. On the other hand, if the degree of abnormality is less than the threshold, the abnormality determination unit determines the input sound to be a normal sound and outputs a determination result.

[0008] As an example of the second problem, consider a case where, while a target machine is operating normally, repair work, such as hammering metal, is occasionally performed near the target machine. In this example, the input sound to the abnormal sound detection system includes not only the operating sound of the target machine but also a mixture of high-frequency sounds from hammering metal. If sounds due to repair work occur only occasionally, the likelihood calculation unit estimates the negative log likelihood of the input sound to be high. Therefore, the example of the abnormal sound detection system described above has the problem of erroneously determining that the target machine is abnormal, even when the target machine is operating normally.

[0009] Therefore, the present invention provides an abnormal sound detection system and method that outputs a low level of abnormality for normal sounds and a high level of abnormality for abnormal sounds even when the operating sound of the target machine is quieter than background noise.The present invention also provides an abnormal sound detection system and method that outputs a low level of abnormality for normal sounds and a high level of abnormality for abnormal sounds even when the background noise occurs at a frequency unrelated to normal operating sounds. [Means for solving the problem]

[0010] In order to solve at least one of the above problems, a representative example of the invention disclosed in the present application is an abnormal sound detection system comprising an operating section detection subsystem that estimates the operating section and non-operating section of a target machine in response to an input of a normal sound signal of the target machine, using a learning result obtained by learning a set of pairs of a normal sound signal of the target machine and operating section information indicating the actual operating and non-operating sections of the target machine, and an abnormality judgment unit that judges whether an input signal of an abnormal sound detection target of the target machine is normal or abnormal, wherein when the input signal of the abnormal sound detection target of the target machine and the operating section information indicating the actual operating and non-operating sections of the target machine corresponding to the input signal are received, the operating section detection subsystem estimates the operating and non-operating sections of the target machine from the input signal of the abnormal sound detection target using the learning result, and the abnormality judgment unit judges whether the input signal of the abnormal sound detection target of the target machine is normal or abnormal based on the degree of deviation between the estimated operating and non-operating sections and the actual operating and non-operating sections indicated in the operating section information. [Effects of the Invention]

[0011] According to the present invention, abnormal sounds can be detected with high accuracy even when background noise is mixed with input sounds. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing the overall configuration of an abnormal sound detection system according to a first embodiment. FIG. [Figure 2] FIG. 2 is a block diagram showing a learning subsystem in the first embodiment. [Figure 3] FIG. 2 is a diagram showing a processing flow of a learning subsystem in the first embodiment. [Figure 4] FIG. 2 is a block diagram showing an abnormal sound detection subsystem according to the first embodiment. [Figure 5] FIG. 4 is a diagram showing a processing flow of an abnormal sound detection subsystem in the first embodiment. [Figure 6] FIG. 1 is a diagram illustrating an example of a hardware configuration of an abnormal sound detection system according to a first embodiment. [Figure 7] FIG. 10 is a block diagram showing the overall configuration of an abnormal sound detection system according to a second embodiment. [Figure 8] FIG. 10 is a block diagram showing a learning subsystem in the second embodiment. [Figure 9] FIG. 10 is a diagram showing a processing flow of a learning subsystem in the second embodiment. [Figure 10] FIG. 10 is a block diagram showing an abnormal sound detection subsystem in a second embodiment. [Figure 11] FIG. 10 is a diagram showing a processing flow of an abnormal sound detection subsystem in the second embodiment. [Figure 12] FIG. 10 is a diagram illustrating an example of a hardware configuration of an abnormal sound detection system according to a second embodiment. [Figure 13] FIG. 11 is a block diagram showing a learning subsystem in the third embodiment. [Figure 14] FIG. 11 is a diagram showing a processing flow of a learning subsystem in the third embodiment. [Figure 15]FIG. 10 is a diagram showing a block configuration of a learning subsystem in a fourth embodiment. [Figure 16] FIG. 10 is a diagram illustrating an example of operation section information. [Figure 17] FIG. 10 is a diagram showing an example of an action label. DETAILED DESCRIPTION OF THE INVENTION

[0013] An embodiment of the present invention will be described in detail below. The abnormal sound detection system of this embodiment determines an abnormality in a target machine based on the operating state of the target machine and an input signal (input sound) obtained for the target machine. The abnormal sound detection system of this embodiment detects abnormal sounds even when background noise is mixed with the input sound. In this embodiment, an abnormality is determined for the input signal that is the target for abnormal sound detection of the target machine using the learning results obtained by learning a set of pairs of normal sound signals of the target machine and operation section information that indicates the actual operation section of the target machine.

[0014] In this embodiment, when an input signal of the target machine that is the target for abnormal sound detection is received, the learning results are used to estimate the operating section of the target machine from the input signal of the target for abnormal sound detection, and an abnormality in the input signal of the target machine that is the target for abnormal sound detection is determined based on the degree of deviation between the estimated operating section and the actual operating section indicated in the operating section information.

[0015] The abnormal sound detection system of this embodiment can be used to detect abnormalities in target machines M used in various places, such as power plants, steel plants, chemical plants, food plants, machine assembly plants, etc. The target machines M are various machines that generate sounds during operation, such as motors, valves, pumps, solenoids, compressors, boilers, conveyors, agitators, and heating furnaces. [Example]

[0016] Example 1 will be described with reference to Figures 1 to 6, 16 and 17. Figure 1 is a block diagram of an abnormal sound detection system in Example 1. The abnormal sound detection system reads a time waveform 100 of a normal sound signal and movement section information 200 from a training data set database (DB) 1, and inputs them to a learning subsystem 1000.

[0017] The normal sound signal includes normal operation sounds of the target machine M and background noise. The operation interval information 200 is information indicating at what time in the time waveform 100 of the corresponding normal sound signal the target machine M was operating. As shown in FIG. 16, for example, the operation interval information 200 is a collection of pairs of operation start times and operation end times. The time when the target machine M is not operating is the non-operation interval of the target machine M. In the following explanation, the symbol M for the target machine will be omitted as appropriate.

[0018] Returning to Figure 1, the learning subsystem 1000 performs learning processing based on the normal sound time waveform 100 and the movement section information 200, and outputs parameters for the feature embedding unit 10 and parameters for the movement / non-movement estimation unit 11. The movement / non-movement estimation unit 11 has the function of estimating whether or not the target machine is moving. The parameters for the feature embedding unit 10 are stored in a feature embedding parameter DB2. The parameters for the movement / non-movement estimation unit 11 are stored in a movement / non-movement estimation parameter DB3.

[0019] The feature embedding unit 10 and the action / non-action estimation unit 11 are cascaded to form the action segment detection subsystem 1100, which estimates action segment information from the input signal. Therefore, by using the parameters of the feature embedding unit 10 stored in DB2 and the parameters of the action / non-action estimation unit 11 stored in DB3, the action segment detection subsystem 1100 can be obtained, which estimates correct action segments for normal sound signals.

[0020] The abnormal sound detection subsystem 2000 reads the parameters of the feature embedding unit 20 and the parameters of the activity / non-activity estimation unit 21 from DBs 2 and 3. Activity section information 400 and a signal time waveform 300 are input to the abnormal sound detection subsystem 2000. The abnormal sound detection subsystem 2000 determines whether the input signal is normal or abnormal.

[0021] Details will be explained in FIG. 4, but the determination of whether the input waveform signal 300 is normal or abnormal is performed as follows. First, the operation interval of the target machine is estimated from the signal time waveform 300 using the operation interval detection subsystem 2100 composed of the feature embedding unit 20 and the operation / non-operation estimation unit 21. Next, the degree of abnormality of the waveform signal 300 is calculated according to the degree of deviation between the estimated operation interval and the input operation interval information 400. Finally, a determination signal 500 is output according to the magnitude of the abnormality. The determination signal 500 is sent to another system (not shown). In the first embodiment, a signal for which the estimation of the operation interval of the target machine has failed is determined to be abnormal.

[0022] How the first embodiment solves the above problem will be described. First, estimating the operating interval of the target machine is equivalent to distinguishing between sounds in non-operating intervals and sounds in operating intervals. The sounds in non-operating intervals are background noise only. On the other hand, the sounds in operating intervals are a mixture of the operating sounds of the target machine and background noise. Therefore, the operating interval detection subsystem 2100 is a model that distinguishes between background noise and mixed sounds.

[0023] When the background noise is louder than the normal operation sound, the only difference between the background noise and the mixed sound is the presence or absence of a slight normal operation sound. Therefore, the operation section detection subsystem 2100 becomes a model that captures the slight characteristics of normal operation sounds. Abnormal operation sounds that deviate from these slight characteristics cannot be detected as operation sections, making it possible to detect abnormal sounds. On the other hand, the example of the abnormal sound detection system described above as a comparative example learns only the characteristics of the mixed sound and detects abnormal sounds based on this. Since Example 1 detects abnormal sounds using a model that directly captures the characteristics of normal operation sounds, it can more accurately detect sounds that deviate from the characteristics of normal operation sounds than the abnormal sound detection system of the comparative example.

[0024] As shown in FIG. 2, the learning subsystem 1000 in the first embodiment includes a time-frequency feature generator 12, a feature embedding unit 10, an action / non-action estimator 11, an action label generator 13, and a learning unit .

[0025] 3 is a processing flow of the learning subsystem 1000 in Example 1. The learning subsystem 1000 reads out a set of a time waveform 100 of a normal sound signal and movement section information 200 from the training dataset DB1. In the subsequent processing, in step S301, steps S302 to S306 (described later) are repeated until the number of iterations C1 becomes equal to or greater than a threshold value ThC, which is set as a learning termination condition.

[0026] In step S302, the time-frequency feature generation unit 12 converts the time waveform 100 of the normal sound signal into time-frequency domain features 110. For example, the time-frequency domain features 110 may be a power spectrogram or a logarithmic Mel-power spectrogram obtained by performing a short-time Fourier transform on the time waveform.

[0027] In step S303, the action label generation unit 13 generates an action label 210 from the action section information 200. The action label 210 is, for example, a binary vector with the same length as the number of time frames in the time-frequency domain feature 110. The value of the binary vector is 1 if the target machine is operating in each time frame, and 0 if it is not operating. An example of this is shown in FIG. 17. Alternatively, a scalar value may be used instead of a vector, and the central time frame of the time-frequency domain feature 110 may be set to 1 if the target machine is operating and 0 if it is not operating. For example, if the number of time frames in the time-frequency domain feature 110 is five, the action label for the third time frame may be determined depending on whether the target machine is operating or not operating. Furthermore, for both of the above two action labels, if the machine switches between operating and not operating during one time frame, the value of the corresponding action label may be set to the proportion of the time frame length during which the machine was operating. In other words, if the machine was operating for half the time frame length, the value of the corresponding action label may be set to 0.5.

[0028] In step S304, the feature embedding unit 10 converts the time-frequency domain features 110 into intermediate features 120 for estimating action labels. The feature embedding unit 10 may be a stack of fully connected layers and activation functions, a stack of multiple convolutional layers and activation functions, or a recurrent neural network.

[0029] In step S305, the action / non-action estimation unit 11 outputs an action label estimation value 130 using the intermediate feature amount 120. The action / non-action estimation unit 11 may be, for example, a stack of a fully connected layer and a softmax function.

[0030] In step S306, the learning unit 14 updates the parameters of the feature embedding unit 10 and the parameters of the action / non-action estimation unit 11 by minimizing the value of a loss function defined by the error between the action label estimate 130 and the action label 210. The error between the action label estimate 130 and the action label 210 may be, for example, the cross entropy between them. To minimize the value of the loss function, an optimization algorithm such as SGD, Momentum SGD, RMSProp, AdaGrad, AdaDelta, or Adam can be used.

[0031] If the number of iterations C1 is equal to or greater than ThC in step S301 (S301: YES), in step S307, the parameters of the feature embedding unit 10 are stored in DB2 and the parameters of the motion / non-motion estimation unit 11 are stored in DB3, and the processing flow of the learning subsystem 1000 is terminated.

[0032] As shown in FIG. 4 , the abnormal sound detection subsystem 2000 in the first embodiment includes a time-frequency feature generation unit 22, a feature embedding unit 20, an action / non-action estimation unit 21, an action label generation unit 23, an abnormality degree calculation unit 24, and an abnormality determination unit 25.

[0033] 5 shows the processing flow of the abnormal sound detection subsystem 2000 in Example 1. In step S501, the time-frequency feature generator 22 generates time-frequency domain features 310 from the signal time waveform 300. The time-frequency feature generator 22 is the same as the time-frequency feature generator 12 of the learning subsystem 1000 shown in FIG.

[0034] In step S502, the action label generation unit 23 generates action labels 410 from the action section information 400. The action label generation unit 23 is the same as the action label generation unit 13 of the learning subsystem 1000 shown in FIG.

[0035] In step S503, the feature embedding unit 20 converts the time-frequency domain features 310 into intermediate features 320 for estimating an action label by using the parameters read from the feature embedding parameter DB2.

[0036] In step S504, the action / non-action estimation unit 21 outputs an action label estimation value 330 from the intermediate feature amount 320 by using the parameters read from the action / non-action estimation parameter DB3.

[0037] In step S505, the abnormality degree calculation unit 24 calculates a predefined degree of deviation between the action label 410 and the action label estimated value 330, and outputs the calculated degree of deviation as the abnormality degree 340.

[0038] The discrepancy may be, for example, the cross-entropy error between the action label and the action label estimate. Alternatively, only the labels in the section where the target machine is operating may be extracted from the action labels and action label estimates, and the cross-entropy between them may be used as the discrepancy. In other words, the discrepancy between the action label and the action label estimate in the section where the target machine is not operating may be ignored. Furthermore, the discrepancy between the action label and the action label estimate in several frames before and after the switching between the operating and non-operating states of the target machine may also be ignored, and the cross-entropy error between the action label and the non-operation label in the remaining section may be used.

[0039] In step S506, the abnormality determination unit 25 determines whether the signal time waveform 300 is normal or abnormal based on the abnormality degree 340, and outputs the determination result 500. If the abnormality degree exceeds a predetermined threshold, it is determined to be abnormal, and if not, it is determined to be normal.

[0040] 6 is a diagram showing an example of the configuration of an abnormal sound detection system H1 according to Example 1. The abnormal sound detection system H1 includes a computer H2 and a microphone (hereinafter referred to as a microphone) H7. The number of computers H2 and microphones H7 may be two or more.

[0041] The microphone H7 is a device that collects operating sounds generated by the target machine for anomaly detection. The microphone H7 transmits audio data from the collected sounds to a computer. The microphone H7 is placed near the target machine. The microphone H7 may be attached to the target machine or may be attached at a location remote from the target machine.

[0042] The computer H2 determines whether the sound collected from the target machine is abnormal or normal. The computer H2 includes, for example, a processor H5, a main memory device H3, a sub-memory device H4, and a connection interface H6. Each piece of hardware is connected to each other via an internal path.

[0043] The connection interface H6 is an interface for connecting to an external device. The connection interface H6 is, for example, a network interface and an I / O interface. Sound captured by the microphone H7 is input to the computer H2 via the connection interface H6.

[0044] The processor H5 executes a computer program stored in the main memory device H3. The processor H5 executes processing in accordance with the computer program, thereby operating as a functional unit (module) that realizes a specific function. In the following description, when a process is described using a functional unit as the subject, this indicates that the processor H5 is executing a program that realizes the functional unit.

[0045] The main memory device H3 stores the computer programs executed by the processor H5 and data used by the computer programs. The main memory device H3 also includes a work area temporarily used by the computer programs. The main memory device H3 is, for example, a dynamic random access memory (DRAM). The computer programs stored in the main memory device H3 will be described later.

[0046] The computer programs and data stored in the main memory device H3 may be stored in the secondary memory device H4, in which case the processor H5 reads the computer programs and data from the secondary memory device H4 and stores them in the main memory device H3.

[0047] The secondary storage device H4 permanently stores data. The secondary storage device H4 is, for example, a hard disk drive (HDD) or a solid state drive (SSD). The data stored in the secondary storage device H4 will be described later.

[0048] The computer programs and data held by computer H2 will now be described. The components of the learning subsystem 1000, the block configuration of which is shown in Figure 2, and the components of the abnormal sound detection subsystem 2000, the block configuration of which is shown in Figure 4, are all computer programs, and are stored in main memory device H3.

[0049] A training dataset DB1 that stores data read by the learning subsystem 1000 is stored in the secondary storage device H4. A feature embedding parameter DB2 that holds parameters of the feature embedding unit and is stored by the learning subsystem 1000 and read by the abnormal sound detection subsystem 2000 is also stored in the secondary storage device H4. Similarly, an action / non-action estimation parameter DB3 that holds parameters of the action / non-action estimation unit and is stored by the learning subsystem 1000 and read by the abnormal sound detection subsystem 2000 is also stored in the secondary storage device H4.

[0050] According to the first embodiment, even when background noise is input together with the operating sound of the target machine, it is possible to accurately determine whether the target machine is normal or abnormal. [Example]

[0051] A second embodiment will be described using FIGS. 7 to 12. In the following description, including this embodiment, differences from the first embodiment will be mainly described. In the first embodiment, the simultaneous operation information 200 input to the learning subsystem 1000 and the operation section information 400 input to the abnormal sound detection subsystem 2000 can be automatically generated, for example, from the control signal of the target machine. Therefore, if the control signal of the target machine can be acquired, the operation section information 200, 400 can also be acquired, and the method described in the first embodiment can be implemented. On the other hand, if the control signal of the target machine cannot be acquired, the operation section information 200, 400 must be generated manually. However, generating the operation section information 200, 400 requires time and effort. Furthermore, generating the operation section information 400 each time the abnormal sound detection subsystem 2000 is used to calculate the degree of anomaly is a heavy workload.

[0052] Therefore, a second embodiment that addresses this problem is shown in Figure 7. As shown in Figure 7, abnormal sound detection subsystem 2000A determines whether time waveform 300 of the abnormal sound detection target signal is normal or abnormal, based on parameters from outlier detection parameter DB4 and parameters from feature embedding parameter DB2, without using operation section information.

[0053] As shown in FIG. 8 , the learning subsystem 1000A in the second embodiment includes a time-frequency feature generation unit 12, a feature embedding unit 10, an action / non-action estimation unit 11, an action label generation unit 13, an action segment learning unit 15, an outlier detection unit 16, and an outlier learning unit 17.

[0054] 9 shows the processing flow of the learning subsystem 1000A in the second embodiment. The learning subsystem 1000A reads out a set of a time waveform 100 of a normal sound signal and operation section information 200 from the training dataset DB1. The operation section information 200 is information indicating at what time in the time waveform 100 of a normal sound signal the target machine was operating. For example, the operation section information 200 is a set of pairs of an operation start time and an operation end time.

[0055] In step S901, the following processes are sequentially repeated until the number of iterations C1 becomes greater than the threshold value ThC1: time-frequency feature generation (S902), action label generation (S903), feature embedding (S904), action / non-action estimation (S905), parameter update of the feature embedding unit and parameter update of action / non-action (S906). These steps are performed in the same manner as in the first embodiment.

[0056] If the learning end condition is met in step S901 (S901: YES), the process proceeds to step S907. The following steps S908 to S910 are repeated until it is determined in step S907 that the number of iterations C2 is greater than the threshold value ThC2.

[0057] In step S908, the time-frequency feature generation unit 12 generates time-frequency domain features 110 of a normal sound signal from the time waveform 100 of the normal sound signal. In step S909, the feature embedding unit 10 generates intermediate features 120 from the time-frequency domain features 110. In step S910, the outlier learning unit 17 updates the parameters of the outlier detection unit 16 so that outlier detection is performed using the intermediate features 120 as normal data.

[0058] The outlier detector 16 can use well-known models used for outlier detection, such as a Gaussian mixture distribution, an autoencoder, or a variational autoencoder. The parameter update algorithm can be any well-known algorithm used for updating the parameters of each model. As will be described later, by using the outlier detector 16 in the abnormal sound detection subsystem 2000A, it is possible to determine whether a signal subject to abnormal sound detection is normal or abnormal without using an action label.

[0059] If it is determined in step S907 that the learning termination condition is met (S907: YES), in step S911, the parameters of the feature embedding unit 10 are stored in DB2, and the parameters of the outlier detection unit 16 are stored in DB4, and the processing flow of the learning subsystem 1000A is terminated.

[0060] As shown in FIG. 10, an abnormal sound detection subsystem 2000A in the second embodiment includes a time-frequency feature generator 22, a feature embedding unit 20, an outlier detector 26, and an abnormality determiner 25.

[0061] 11 shows the processing flow of the abnormal sound detection subsystem 2000A in Example 2. In step S1101, the time-frequency feature generator 22 generates time-frequency domain features 310 of a signal from the signal time waveform 300. This time-frequency feature generator 22 is the same as the time-frequency feature generator 22 of the learning subsystem 1000 shown in FIG.

[0062] In step S1102, the feature embedding unit 20 reads the parameters of the feature embedding unit from the feature embedding parameter DB2, and converts the time-frequency domain features 310 into intermediate features 320.

[0063] In step S1103, the outlier detector 26 reads parameters from the outlier detection parameter DB 4, and outputs the degree of anomaly 340 using the intermediate feature 320.

[0064] In step S1104, the abnormality determination unit 25 determines whether the signal time waveform 300 is normal or abnormal based on the abnormality degree 340, and outputs the determination result 500. If the abnormality degree exceeds a predetermined threshold, it is determined to be abnormal, and if not, it is determined to be normal.

[0065] As described above, in the second embodiment, by performing outlier detection on the intermediate features 320 output by the feature embedding unit 20, it is possible to determine whether the abnormal sound detection target signal is normal or abnormal without using an action label.

[0066] The parameters of the feature embedding unit 20 are parameters that have been trained in the learning subsystem 1000A so that the action label 210 can be correctly estimated. Therefore, in order to correctly distinguish between action intervals and non-action intervals, the intermediate feature obtained by inputting the time-frequency feature in the action interval into the feature embedding unit 20 is significantly different from the intermediate feature obtained by inputting the time-frequency feature in the non-action interval into the feature embedding unit 20. This means that the feature embedding unit 20 changes its output in response to subtle features of normal operation sounds. Therefore, like the first embodiment, the second embodiment detects abnormal sounds using a model that directly captures the features of normal operation sounds, and therefore can correctly detect sounds that deviate from the features of normal operation sounds compared to the example of the abnormal sound detection system described as a comparative example.

[0067] Fig. 12 is a diagram showing an example of the configuration of an abnormal sound detection system H1A of Example 2. Similar to the configuration of the abnormal sound detection system H1 of Example 1 shown in Fig. 6, the abnormal sound detection system H1A includes a computer H2 and a microphone H7. The computer H2 includes a connection interface H6, a processor H5, a main storage device H3, and a secondary storage device H4. The roles of the microphone H7, connection interface H6, processor H5, main storage device H3, and secondary storage device H4 are as described in Fig. 6. [Example]

[0068] A third embodiment will be described with reference to Figures 13 and 14. Figure 13 shows a block configuration of a learning subsystem 1000B in the third embodiment that correctly detects abnormal sounds even when rare background noise appears in a frequency band where normal operation sounds do not exist. The learning subsystem 1000B can be used as the learning subsystem 1000 of the first embodiment shown in Figure 1. Alternatively, by adding the outlier detection unit 16 and the outlier learning unit 17 included in the learning subsystem 1000A of the second embodiment shown in Figure 8 to Figure 13, it may be used as the learning subsystem 1000A of the second embodiment shown in Figure 7.

[0069] The learning subsystem 1000B of the third embodiment shown in FIG. 13 includes a time-frequency feature generation unit 12, a feature embedding unit 10, a motion / non-motion estimation unit 11, a frequency band utilization calculation unit 18, an motion label generation unit 13, and a learning unit 14.

[0070] 14 shows the processing flow of the learning subsystem 1000B in Example 3. In step S1401, the following steps S1402 to S1407 are executed until the learning end condition, that is, the number of iterations C3 becomes larger than the threshold value ThC3, is met.

[0071] In step S1402, the time-frequency feature generator 12 converts the time waveform 100 of the normal sound signal into time-frequency domain features 110 of the signal.

[0072] In step S 1403 , the action label generation unit 13 generates the action label 210 from the action section information 200 .

[0073] In step S1404, the feature embedding unit 10 converts the time-frequency domain features 110 into intermediate features 120 for estimating an action label. The feature embedding unit 10 also outputs model parameters 140 for the feature embedding unit 10. The feature embedding unit 10 is a neural network that includes a fully connected layer. For example, it may be a neural network that stacks a fully connected layer and an activation function, or a neural network that stacks multiple convolutional layers and activation functions, with a fully connected layer added at the end.

[0074] In step S1405, the action / non-action estimation unit 11 uses the intermediate feature amount 120 to output the action label estimation value 130.

[0075] In step S1406, the utilization frequency band calculation unit 18 extracts the weight matrix of the fully connected layer included in the feature embedding unit 10 from the model parameters 140, and outputs a predefined norm of the weight matrix as the learning regularization term 150. The norm is one that induces the weight matrix of the fully connected layer so that the feature embedding unit 10 has frequency band selectivity during learning.

[0076] The frequency band selectivity of the feature embedding unit 10 refers to the property of the feature embedding unit 10 to calculate the intermediate feature 120 only from information on a limited frequency band of the time-frequency domain feature 110. The regularization term 150 may be, for example, a mixed norm of the weight matrix of the fully connected layer that first appears in the feature embedding unit 10, defined as follows: First, each element of the weight matrix is ​​grouped according to the frequency of the element of the time-frequency domain feature 110 to be multiplied by that element. Next, the root mean square of the set of elements included in each group is calculated. Finally, the calculation results for all groups are summed to obtain the mixed norm value. Alternatively, other methods may be used to group the elements of the weight matrix. For example, frequency band groups may be defined in advance, and each element of the weight matrix may be assigned to a frequency band group that includes the frequency of the time-frequency domain feature 110 to be multiplied, and then a similar norm calculation may be performed. In any case, the calculated regularization term 150 becomes smaller as the feature embedding unit 10 embeds the intermediate feature 120 only from features in a narrow frequency band of the time-frequency domain feature 110.

[0077] In step S1407, the learning unit 14 learns the parameters of the feature embedding unit 10 and the parameters of the action / non-action estimation unit 11 by minimizing a loss function defined by the sum of the error between the action label estimation value 130 and the action label 210 and the regularization term 150.

[0078] The error between the action label estimate 130 and the action label 210 may be, for example, the cross entropy between them. To minimize the value of the loss function, an optimization algorithm such as SGD, Momentum SGD, RMSProp, AdaGrad, AdaDelta, or Adam may be used.

[0079] According to the definition of the loss function, the learning unit 14 learns the parameters of the feature embedding unit 10 and the parameters of the action / non-action estimation unit 11 so as to give the feature embedding unit 10 frequency band selectivity while also correctly estimating the action label 210. Therefore, as the learning progresses, the feature embedding unit 10 and the action / non-action estimation unit 11 come to correctly estimate the action label 210 only from a limited frequency band of the time-frequency domain feature 110.

[0080] At this time, to correctly estimate the operation label 210, it is necessary to correctly detect the normal operation sound. Therefore, the frequency band that can most accurately estimate the operation label 210 is the main frequency band of the normal operation sound. Therefore, through this learning, the feature embedding unit 10 uses the main frequency band of the normal operation sound as the frequency band to be used for calculating the intermediate feature 120. Therefore, the intermediate feature 120 is not affected by rare background noise that appears at a frequency different from the main frequency band of the normal operation sound. Similarly, the operation label estimate 130 is also not affected by the rare background noise. Therefore, erroneous determination of abnormal sound detection due to background noise does not occur.

[0081] In step S1401, if the number of iterations C3 becomes greater than the threshold value ThC3 and the learning termination condition is met (S1401: YES), in step S1408, the parameters of the feature embedding unit 10 are saved in DB2 and the parameters of the movement / non-movement estimation unit 11 are saved in DB3, and the processing flow of learning subsystem 1000B is terminated.

[0082] The utilization frequency band calculation unit 18 of the third embodiment is not limited to application to the parameters of the model output by the feature embedding unit 10, but may also be applied to the parameters of a model of another abnormal sound detection system. For example, it may be applied to the parameters of a model used in the abnormal sound detection system of "R. Giri, "Self-supervised classification for detecting anomalous sounds," in Proc of DCASE Workshop, 2020." [Example]

[0083] Example 4 will be described with reference to Figure 15. In Example 3, in order to deal with background noise that rarely appears in a frequency different from the main frequency band of normal operation sounds, the learning of the feature embedding unit 10 is guided, and intermediate features 120 are created from only the features in the main frequency band of normal operation sounds among the time-frequency domain features 110. On the other hand, if the main frequency band of normal operation sounds is known in advance, the same problem can also be addressed by creating the time-frequency domain features 110 themselves from only that frequency band. Example 4 shows this method.

[0084] Fig. 15 shows a block configuration of a learning subsystem 1000C in the fourth embodiment. This subsystem 1000C can be used as the learning subsystem 1000 in the first embodiment shown in Fig. 1. Alternatively, by adding the outlier detection unit 16 and the outlier learning unit 17 included in the learning subsystem 1000A in the second embodiment shown in Fig. 8 to Fig. 15, it can also be used as the learning subsystem 1000A in the second embodiment shown in Fig. 7.

[0085] As shown in FIG. 15, the learning subsystem 1000C in the fourth embodiment includes a time-frequency feature generator 12, a feature embedding unit 10, an action / non-action estimator 11, an action label generator 13, and a learning unit .

[0086] The processing flow of the learning subsystem 1000C in the fourth embodiment is the same as the processing flow of the learning subsystem 1000 in the first embodiment shown in FIG.

[0087] Of the specific processes performed in the processing flow of the learning subsystem 1000C in the fourth embodiment, only the process performed in the "generation of time-frequency features" step in S302 of Fig. 3 differs from the processes performed in the processing flow of the learning subsystem 1000 in the first embodiment. In the fourth embodiment, in S302, the time-frequency feature generator 12 outputs time-frequency domain features 110 only within the frequency band of use 160 from the time waveform 100 of a normal sound signal and the frequency band of use 160. For example, the time-frequency domain features 110 may be obtained by extracting only the elements included in the frequency band of use from a power spectrogram obtained by short-time Fourier transform of the time waveform or a logarithmic Mel power spectrogram.

[0088] The abnormal sound detection system described above may be configured as follows.

[0089] (1) An abnormal sound detection system comprising an operating section detection subsystem that estimates the operating / non-operating sections of a target machine in response to an input of a normal sound signal of the target machine using the learning results of a set of pairs of the target machine's normal sound signal and operating section information indicating the target machine's actual operating / non-operating section, and an abnormality judgment unit that judges whether the input signal of the target machine's abnormal sound detection target is normal or abnormal, wherein when the input signal of the target machine's abnormal sound detection target and operating section information indicating the actual operating / non-operating section of the target machine corresponding to the input signal are received, the operating section detection subsystem estimates the operating / non-operating section of the target machine from the input signal of the abnormal sound detection target using the learning results, and the abnormality judgment unit judges whether the input signal of the target machine's abnormal sound detection target is normal or abnormal based on the degree of deviation between the estimated operating / non-operating section and the actual operating / non-operating section indicated in the operating section information.

[0090] This makes it possible to suppress the influence of background noise even when the background noise is mixed with the input sound, and to detect abnormal sounds with high accuracy.

[0091] (2) In the abnormal sound detection system of (1) above, the learning results include feature embedding parameters used to generate features from the normal sound signals of the target machine and operation / non-operation estimation parameters used to estimate whether the target machine is operating or not from the extracted features, and the operation section detection subsystem uses the feature embedding parameters and the operation / non-operation estimation parameters to estimate whether the target machine is operating or not from the normal sound signals of the target machine or the input signal of the abnormal sound detection target.

[0092] (3) The abnormal sound detection system of (2) above further comprises an outlier detection unit that calculates the degree of abnormality of the intermediate features of the input signal of the abnormal sound detection target of the target machine, which is generated using the feature embedding parameters, with respect to the distribution of the intermediate features of the normal sound signal, based on the learning result of learning the intermediate features of the normal sound signal of the target machine, which is generated using the feature embedding parameters included in the learning result, and the abnormality determination unit determines whether the input signal of the abnormal sound detection target of the target machine is normal or abnormal, based on the calculated outlier.

[0093] This makes it possible to suppress the influence of background noise and detect abnormal sounds with high accuracy, even when it is not possible to obtain operation section information corresponding to the input signal that is the target of abnormal sound detection.

[0094] (4) In the abnormal sound detection system described in (1) above, the normal sound signal of the target machine includes the normal operating sound of the target machine and background noise, and the learning result guides the system to estimate the operating / non-operating section from the frequency band corresponding to the normal operating sound of the target machine among the normal sound signal or the input signal of the abnormal sound detection target.

[0095] This provides an abnormal sound detection system and method that outputs a low level of abnormality for normal sounds and a high level of abnormality for abnormal sounds, even when background noise occurs at a frequency unrelated to normal operation sounds.

[0096] The present invention is not limited to the above-described examples, but includes various modifications. The above-described examples have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those having all of the described configurations. The configurations and processing methods of the embodiments can be modified as appropriate within the scope of achieving the object of the present invention.

[0097] Furthermore, the components of the present invention can be selected arbitrarily, and the invention including the selected components is also included in the present invention. Furthermore, the components described in the claims can be combined in combinations other than those explicitly stated in the claims. [Explanation of symbols]

[0098] 1. Training dataset DB 2. Feature embedding parameter DB 3. Motion and non-motion estimation parameter DB 4. Outlier detection parameter DB 10. Feature embedding part 11...Movement / non-motion estimator 12. Time-frequency feature generation unit 13. Action label generation unit 14. Learning Department 15. Operating interval learning section 16. Outlier detection section 17. Outlier learning part 18. Frequency standby calculation unit 20. Feature embedding part 21...Movement / non-motion estimator 22. Time-frequency feature generation unit 23 Action label generation unit 24... Abnormality calculation section 25...Abnormality determination section 26. Outlier detection unit 100...Time waveform of normal sound signal 110···Time-frequency features of normal sound signals 120 Intermediate features 130...Movement / non-motion estimator 140...Model parameters 150...Regularization term 160...Frequency band used 200···Operation section information 210... Operation Label 300...Time waveform of the signal to be detected for abnormal sound 310···Signal time-frequency domain features 320···Intermediate features 330···Estimated action label 340... Abnormality level 400···Operation section information 410···Operation label 500...Normal / Abnormal 1000, 1000A, 1000B, 1000C...Learning subsystem 1100: Operation section detection subsystem 2000 Abnormal Sound Detection Subsystem 2100: Operation section detection subsystem H1: Anomaly detection system H2...Calculator H3...Main memory H4: Secondary storage device H5 Processor H6... Connection Interface H7...Microphone S301: Step to check the conditions for completing the learning S302: Time-frequency feature generation step S303: Action label generation step S304: Feature embedding step S305: Action / non-action estimation step S306: Parameter update step for the feature embedding unit and the motion / non-motion estimation unit S307: A step of storing parameters of the feature embedding unit and the motion / non-motion estimation unit in a database. S501: Time-frequency feature generation step S502: Action label generation step S503: Feature embedding step S504: Action / non-action estimation step S505: Anomaly calculation step S506: Abnormality determination step S901: Step to check the conditions for completing the learning S902: Time-frequency feature generation step S903: Action label generation step S904: Feature embedding step S905: Action / non-action estimation step S906: Parameter update step for the feature embedding unit and the motion / non-motion estimation unit S907: Step to check the conditions for completing the learning S908: Time-frequency feature generation step S909: Feature embedding step S910: Parameter update step for outlier detection S911: A step of storing parameters of the feature embedding unit and the motion / non-motion estimation unit in a database. S1101: Time-frequency feature generation step S1102: Feature embedding step S1103: Anomaly calculation step S1104: Abnormality determination step S1401: Step to check the conditions for completing learning S1402: Time-frequency feature generation step S1403: Action label generation step S1404: Feature embedding step S1405: Action / non-action estimation step S1406: Frequency band calculation step S1407: Parameter update step for the feature embedding unit and the motion / non-motion estimation unit S1408: A step of storing parameters of the feature embedding unit and the motion / non-motion estimation unit in a database.

Claims

1. An abnormal sound detection system comprising: an operating section detection subsystem that estimates an operating section and a non-operating section of a target machine in response to an input of a normal sound signal of the target machine, using a learning result obtained by learning a set of pairs of normal sound signals of the target machine and operating section information indicating the actual operating and non-operating sections of the target machine; and an abnormality determination unit that determines whether an input signal of an abnormal sound detection target of the target machine is normal or abnormal, When receiving an input signal of the target machine that is the target of abnormal sound detection and the operation section information that indicates the actual operation and non-operation sections of the target machine that correspond to the input signal, the operating section detection subsystem estimates operating and non-operating sections of the target machine from the input signal of the abnormal sound detection target using the learning result, the abnormality determination unit determines whether an input signal of an abnormal sound detection target of the target machine is normal or abnormal based on a degree of deviation between the estimated operating and non-operating sections and the actual operating and non-operating sections indicated in the operating section information. Abnormal sound detection system.

2. The abnormal sound detection system according to claim 1, the learning result includes a feature embedding parameter used to generate a feature from a normal sound signal of the target machine, and an action / non-action estimation parameter used to estimate an action / non-action of the target machine from the feature, The operating section detection subsystem uses the feature embedding parameters and the operating / non-operating estimation parameters to estimate whether the target machine is operating or non-operating from a normal sound signal of the target machine or an input signal of an abnormal sound detection target. Abnormal sound detection system.

3. An abnormal sound detection method using an abnormal sound detection system that uses a learning result obtained by learning a set of pairs of normal sound signals of a target machine and operation section information indicating the actual operation and non-operation sections of the target machine to estimate operation and non-operation sections of the target machine in response to an input of a normal sound signal of the target machine, and determines whether an input signal of an abnormal sound detection target of the target machine is normal or abnormal, When receiving an input signal of the target machine that is the target of abnormal sound detection and the operation section information that indicates the actual operation and non-operation sections of the target machine that correspond to the input signal, estimating an operating section and an inoperating section of the target machine from the input signal of the abnormal sound detection target using the learning result; determining whether an input signal of an abnormal sound detection target of the target machine is normal or abnormal based on the degree of deviation between the estimated operating and non-operating sections and the actual operating and non-operating sections indicated in the operating section information; Abnormal sound detection method.

4. The abnormal sound detection method according to claim 3, the learning result includes a feature embedding parameter used to generate a feature from a normal sound signal of the target machine, and an action / non-action estimation parameter used to estimate an action / non-action of the target machine from the feature, Using the feature embedding parameters and the operation / non-operation estimation parameters, the operation / non-operation of the target machine is estimated from the normal sound signal of the target machine or the input signal of the abnormal sound detection target. Abnormal sound detection method.

Images