Learning device and anomaly detection device

Abstract

An object of the present invention is to provide a learning device and an anomaly detection device that can improve the reconstruction accuracy of an autoencoder model. [Solution] The learning device includes a time-frequency conversion unit that converts the sensor time waveform of normal data into a time-frequency image, a pre-processing unit that normalizes the time-frequency image to a non-negative value to generate a normalized time-frequency image, a reconstruction unit that inputs the normalized time-frequency image to an autoencoder model and generates a reconstructed image, a post-processing unit that normalizes the reconstructed image to a non-negative value to generate a normalized reconstructed image, and a first weight update unit that updates a first weight coefficient of the autoencoder model based on the normalized time-frequency image and the normalized reconstructed image.

Description

[Technical Field]

[0001] The present invention relates to a learning device that learns weighting coefficients of an autoencoder model such as a VAE, and an anomaly detection device that detects anomalies using an autoencoder model. [Background technology]

[0002] One of the deep learning models used in the field of anomaly detection is the variational autoencoder (hereinafter referred to as "VAE"). Techniques for anomaly detection using VAEs are widely known. For example, Patent Document 1 describes a technique in which data acquired by a sensor is represented as a probability density distribution using an encoder and a decoder, and an anomaly is detected when the probability of newly acquired data occurring is lower than a predetermined threshold. Patent Document 2 also describes a technique in which an encoder is used to extract features of a group to which input data belongs and features specific to the input data, a decoder is used to reconstruct the input data from the group features and the specific features, and an anomaly is detected when the reconstruction error is equal to or greater than a threshold. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-219915 [Patent Document 2] Japanese Patent Publication No. 2022-66957 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when anomaly detection is performed using an autoencoder model such as a conventional VAE, the reconstruction accuracy during decoding of the autoencoder model may be reduced depending on the characteristics of the training signal during training or the type of abnormal pattern in the evaluation data input during evaluation.

[0005] The present invention has been made to solve the above-mentioned problems, and aims to provide a learning device and an anomaly detection device that can improve the reconstruction accuracy in an autoencoder model. [Means for solving the problem]

[0006] The learning device according to the present invention includes a time-frequency conversion unit that converts a sensor time waveform of normal data into a time-frequency image, a preprocessing unit that normalizes the time-frequency image to a non-negative value to generate a normalized time-frequency image, a reconstruction unit that inputs the normalized time-frequency image to an autoencoder model and generates a reconstructed image, a postprocessing unit that normalizes the reconstructed image to a non-negative value to generate a normalized reconstructed image, and a first weight update unit that updates a first weight coefficient of the autoencoder model based on the normalized time-frequency image and the normalized reconstructed image.

[0007] The anomaly detection device according to the present invention includes an anomaly determination unit that determines whether evaluation data is normal or abnormal. The anomaly determination unit includes a time-frequency conversion unit that converts a sensor time waveform of the evaluation data into a time-frequency image, a pre-processing unit that normalizes the time-frequency image to a non-negative value to generate a normalized time-frequency image, a reconstruction unit that inputs the normalized time-frequency image to a trained autoencoder model to generate a reconstructed image, and a post-processing unit that normalizes the reconstructed image to a non-negative value to generate a normalized reconstructed image. The anomaly detection device determines whether the evaluation data is normal or abnormal using the normalized time-frequency image and the normalized reconstructed image. [Effects of the Invention]

[0008] According to the learning device and anomaly detection device of the present invention, the reconstruction accuracy of the autoencoder model can be improved by converting the time-frequency image and the reconstructed image into non-negative values. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a schematic configuration diagram of an abnormality detection device according to a first embodiment. [Figure 2]FIG. 2 is a functional block diagram of a learning unit according to the first embodiment. [Figure 3] FIG. 2 is a functional block diagram of an optimum model determination unit according to the first embodiment. [Figure 4] 3 is a functional block diagram of an abnormality determination unit according to the first embodiment. FIG. [Figure 5] FIG. 3 is a functional block diagram of a background unwhitening input / output unit of a learning unit according to the first embodiment. [Figure 6] FIG. 3 is a functional block diagram of a background whitening input / output unit of a learning unit according to the first embodiment. [Figure 7] FIG. 3 is a functional block diagram of a first weight update unit of a learning unit according to the first embodiment. [Figure 8] FIG. 10 is a diagram illustrating patch parameter candidates. [Figure 9] 10A and 10B are diagrams illustrating an example of a procedure for normalizing a time-frequency image to a non-negative value. [Figure 10] 10A and 10B are diagrams illustrating the procedure of background whitening processing by LOFAR. [Figure 11] 10 is an example of a time-frequency image of normal data. [Figure 12] FIG. 4 is a diagram illustrating an effect of learning by a learning unit according to the first embodiment. [Figure 13] FIG. 10 is a diagram illustrating an area to which an abnormality is added. [Figure 14] FIG. 10 is a diagram illustrating an example of a true positive rate and a false positive rate for each determination threshold. [Figure 15] FIG. 10 is a schematic configuration diagram of an abnormality detection device according to a modified example. [Figure 16] FIG. 1 is a schematic diagram illustrating the configuration of an anomaly detection device according to a conventional technique. [Figure 17] FIG. 10 is a functional block diagram of a learning unit according to the prior art. [Figure 18] FIG. 10 is a functional block diagram of an abnormality determination unit according to the prior art. [Figure 19] FIG. 1 is a schematic diagram illustrating the configuration of an anomaly detection device according to a conventional technique using SSIM-AE. [Figure 20] FIG. 1 is a functional block diagram of a learning unit according to a conventional technique using SSIM-AE. [Figure 21]FIG. 1 is a diagram illustrating a procedure for calculating an average SSIM value according to the prior art. DETAILED DESCRIPTION OF THE INVENTION

[0010] First, as a conventional technique, an anomaly detection device 600 that performs anomaly detection using a general VAE will be described with reference to Figs. 16 to 18. Fig. 16 is a schematic configuration diagram of the conventional anomaly detection device 600. As shown in Fig. 16, the conventional anomaly detection device 600 is composed of a learning unit 61, an anomaly determination unit 62, and a storage unit 63.

[0011] The learning unit 61 receives as input the acoustic sensor time waveform Twn of normal data used for learning and weighting coefficients 630 stored in the storage unit 63. The weighting coefficients 630 are weighting coefficients in a VAE model. A single VAE model includes multiple weighting coefficients, and multiple (a set of) weighting coefficients for a single VAE model are referred to as the weighting coefficients 630. The learning unit 61 updates the weighting coefficients 630 by learning based on the input data. The learned weighting coefficients 630 learned by the learning unit 61 are output to the storage unit 63 and stored in the storage unit 63.

[0012] The abnormality determination unit 62 receives the learned weighting coefficient 630 from the storage unit 63, the determination threshold value Th from the outside, and the acoustic sensor time waveform Twe of the evaluation data. Based on the learned weighting coefficient 630, the abnormality determination unit 62 determines whether the acoustic sensor time waveform Twe of the evaluation data is normal or abnormal.

[0013] 17 is a functional block diagram of a learning unit 61 according to the prior art. As shown in FIG. 17, the learning unit 61 includes a time-frequency transform unit 611, a standardization unit 612, a reconstruction unit 613, a reconstruction error calculation unit 614, a loss function calculation unit 615, and an error backpropagation unit 616.

[0014] The symbol u denotes a time waveform obtained by converting a response from an acoustic sensor into a digital signal (hereinafter referred to as "acoustic sensor time waveform"). i denotes an index of a small section. u iis a short-time signal in a small section of the acoustic sensor time waveform u that is subjected to a Short Time Fourier Transform (hereinafter referred to as "STFT"), and N is the short-time signal u i The acoustic sensor time waveform u is the above acoustic sensor time waveform TWn. The time-frequency conversion unit 611 converts the acoustic sensor time waveform u of the input normal data into a short-time signal u i The frequency spectrum U obtained by frequency analysis for each i and calculate the frequency spectrum U i The time-frequency image X is output to the standardization unit 612.

[0015] First, the short-term signal u i is defined as equation (1).

number

[0016] Next, the short-time signal u i is Fourier transformed to obtain the frequency spectrum U i get.

number

[0017] Next, as shown in equation (3), the frequency spectrum U i Extract (N / 2+1) values ​​from j=0 to j=N / 2 corresponding to the range of positive frequencies and the frequency spectrum obtained by taking the absolute values ​​of complex numbers from U i Rewrite it as:

number

[0018] where N F (=N / 2+1) is the number of elements in the frequency direction in the time-frequency image X, and || represents the operation of the absolute value. i The values ​​of each element of are explained using the absolute values ​​as in equation (3), but the squares of the absolute values ​​may also be calculated and expressed in decibels.

[0019] Finally, as shown in equation (4), the frequency spectrum U calculated at each time i is i are arranged to form the time-frequency image X.

number

[0020] The standardization unit 612 standardizes the time-frequency image X to a mean of "0" and a standard deviation of "1" as shown in Equation (5) to generate a time-frequency image X (hereinafter referred to as a "standardized time-frequency image") standard and convert it to the normalized time-frequency image X standard is output to the reconstruction unit 613.

number

[0021] The reconstruction unit 613 receives the standardized time-frequency image X from the standardization unit 612. standard and the weighting coefficients 630 from the storage unit 63. The reconstruction unit 613 uses the input weighting coefficients 630 to reconstruct the time-frequency image X standardis converted into a feature, and an image (hereinafter referred to as the "reconstructed image") Y is generated by reconstructing the image from the feature to an image close to the input image, and a standardized time-frequency image X standard and the reconstructed image Y are output to the reconstruction error calculation unit 614. The standardized time-frequency image X standard The transformation from the normalized time-frequency image X to the reconstructed image Y is performed by the VAE model. The operation of the VAE model using the weighting coefficients 630 is represented as function f(), and the weighting coefficients 630 are represented as w. standard The transformation from the image to the reconstructed image Y is expressed as equation (6).

number

[0022] Here, the initial value of the weighting coefficient w used in equation (6) is generally initialized with a random number such as the "initial value of Xavier" or the "initial value of He" in order to perform learning efficiently.

[0023] The reconstruction error calculation unit 614 calculates the standardized time-frequency image X standard and the reconstruction error E between the reconstructed image Y rec and outputs it to the loss function calculation unit 615. The reconstruction error E rec Although the absolute error or the sum of squares error may be used, the sum of squares error will be explained as an example here. The sum of squares error is calculated as shown in Equation (7).

number

[0024] The loss function calculation unit 615 receives the reconstruction error E rec The loss function calculation unit 615 calculates how similar the distribution of the feature values ​​obtained by converting the time-frequency image into feature values ​​is to the standard normal distribution using the known Kullback-Leibler divergence (hereinafter referred to as "D KLThe loss function calculation unit 615 calculates the Kullback-Leibler divergence D KL and the reconstruction error E rec The loss function value L is calculated by adding the above and output to the error backpropagation unit 616.

number

[0025] The error backpropagation unit 616 updates the weighting coefficient w used in the function f() in equation (6) based on the error backpropagation method so that the loss function value L input from the loss function calculation unit 615 becomes smaller, and outputs the weighting coefficient w to the storage unit 63 as a learned weighting coefficient 630. Here, the operation of the VAE model expressed by the function f() in equation (6) and the method of updating the weighting coefficients of the VAE model do not directly affect the present invention, so a detailed description of the configuration or operation will be omitted. The principle of the error backpropagation method, which is one method of updating the weighting coefficients of the VAE model, is described in, for example, Patent Document 2. Hereinafter, the reconstruction error E rec The VAE model using the weighting coefficient w updated by the loss function value L of equation (8) based on the above is referred to as a “general VAE model.”

[0026] The frequency and amplitude of the narrowband component contained in the acoustic sensor time waveform u of normal data used for learning by the learning unit 61 may vary depending on the time and operating conditions, even during normal operation. The range in which the frequency and amplitude of the narrowband component fluctuates during normal operation can be investigated in advance. When the acoustic sensor time waveform in the actual environment can be obtained, the acoustic sensor time waveform in the actual environment can be used as Q. train The acquired data are used as the acoustic sensor time waveform Twn of normal data for learning by the learning unit 61. train is the number of data points required to learn fluctuations within the range of normal operation, and is hereinafter referred to as the "number of data points in the learning section."

[0027] On the other hand, when it is difficult to obtain the time waveform of an acoustic sensor in an actual environment, it is necessary to use an electrically generated Q train These waveforms are assumed to be acoustic sensor time waveforms Twn of normal data used for learning by the learning unit 61. Then, learning consisting of the processes of the time-frequency conversion unit 611, the standardization unit 612, the reconstruction unit 613, the reconstruction error calculation unit 614, the loss function calculation unit 615, and the error backpropagation unit 616 is called Q train For each of the learning acoustic sensor time waveforms Twn, the learning update is repeated a predetermined number of times.

[0028] Fig. 18 is a functional block diagram of an abnormality determination unit 62 according to the prior art. As shown in Fig. 18, the abnormality determination unit 62 includes a time-frequency conversion unit 621, a standardization unit 622, a reconstruction unit 623, a reconstruction error calculation unit 624, and a threshold determination unit 625. The acoustic sensor time waveform Twe of the evaluation data is input to the abnormality determination unit 62, and the presence or absence of an abnormality in the acoustic sensor time waveform Twe of the evaluation data is determined.

[0029] The operations of the time-frequency conversion unit 621, standardization unit 622, reconstruction unit 623, and reconstruction error calculation unit 624 of the abnormality determination unit 62 are the same as the operations of the time-frequency conversion unit 611, standardization unit 612, reconstruction unit 613, and reconstruction error calculation unit 614 of the learning unit 61. However, the reconstruction unit 623 generates a reconstructed image Y using a weighting coefficient 630 learned by the learning unit 61, and the reconstruction error calculation unit 624 calculates a reconstruction error E rec and outputs it to the threshold determination unit 625.

[0030] The threshold value determination unit 625 determines the threshold value Th input from the outside and the reconstruction error E rec Based on this, a decision is made on the acoustic sensor time waveform Twe of the evaluation data, and the decision result decide is output. Here, the decision result decide is determined as in equation (9), and the reconstruction error E recIf it is less than the threshold, "0" (normal) is output, and if it is greater than or equal to the threshold, "1" (abnormal) is output.

number

[0031] Here, in the field of general anomaly detection, problems are likely to occur with frequency fluctuations, especially within the normal range. That is, the frequency of the narrowband component contained in the acoustic sensor time waveform u to be monitored may vary depending on the time or operating conditions, and therefore the position of the narrowband component in the time-frequency image X obtained by time-frequency transforming the acoustic sensor time waveform u may shift slightly for each data. If the position of the narrowband component in the training data shifts for each data, a general VAE model that incorporates the square sum error of equation (7) into the loss function learns to reconstruct an average image of all data, and therefore the narrowband component in the reconstructed image Y becomes blurred. This blurring is called the reconstruction error E rec This increases the error, resulting in an increased number of cases where normal signals are mistakenly classified as abnormal. If the decision threshold is increased to avoid this, the number of cases where abnormal signals are mistakenly classified as normal increases, resulting in a blurring of the narrowband components of the reconstructed image and a deterioration in anomaly detection performance. To address this issue, conventional techniques have been known to use structural similarity (hereinafter referred to as "SSIM") instead of the reconstruction error. Hereinafter, the use of a VAE model using weighting coefficients updated by a loss function based on structural similarity will be referred to as "SSIM-AE."

[0032] Fig. 19 is a schematic diagram of a conventional anomaly detection device 700 using SSIM-AE. The conventional anomaly detection device 700 using SSIM-AE shown in Fig. 19 differs from the anomaly detection device 600 using a general VAE model shown in Fig. 16 in the operation of a learning unit 71 and in that patch parameters PP are input to the learning unit 71.

[0033] Fig. 20 is a functional block diagram of a learning unit 71 according to the prior art that uses SSIM-AE. As shown in Fig. 20, the learning unit 71 includes a time-frequency conversion unit 711, a standardization unit 712, a reconstruction unit 713, an SSIM calculation unit 714, a loss function calculation unit 715, and an error backpropagation unit 716. The operations of the time-frequency conversion unit 711, the standardization unit 712, the reconstruction unit 713, and the error backpropagation unit 716 of the learning unit 71 are the same as those of the time-frequency conversion unit 611, the standardization unit 612, the reconstruction unit 613, and the error backpropagation unit 616 of the learning unit 61 that uses the general VAE model shown in Fig. 17.

[0034] The SSIM calculation unit 714 of the learning unit 71 calculates the SSIM value using the patch parameters PP. The loss function calculation unit 715 performs calculations different from those performed by the loss function calculation unit 615 of the learning unit 61 of a general VAE. Below, we will explain the SSIM calculation unit 714 and the loss function calculation unit 715, which are different from those of a general VAE.

[0035] The SSIM calculation unit 714 calculates the time-frequency image X standard The average SSIM value E is calculated using the reconstructed image Y and the patch parameters PP input from the outside. SSIM and output it to the loss function calculation unit 715. Here, the parameters included in the patch parameters PP are the padding width P pad and patch size P size and the shift width P stride The padding width P pad and patch size P size and the shift width P stride Normally, independent values ​​can be selected for the vertical and horizontal directions of the image, but for simplicity, the following explanation will focus on the case where the values ​​of each parameter are changed in tandem in the vertical and horizontal directions of the image.

[0036] FIG. 21 shows the average SSIM value E SSIM First, as shown in FIG. 21(a), a padding width P is calculated around the original image Z before padding. padBy adding a region with a specific value within the range of Z, the image is enlarged, and the original image Z after padding is padding The value of the added region can be set to zero or the nearest value among the outermost values ​​of the original image Z before padding.

[0037] Next, as shown in FIG. 21(b) and Equation (10), the padded original image Z padding A patch image Z is created by cutting out a small section of the image from p However, the patch image Z p is the time-frequency image X obtained by normalizing the original image Z before padding. standard If the input patch image X p , if the reconstructed image Y is the output patch image Y p is.

number

[0038] Here, p is an index representing the patch image number, i is an index number representing the time of the time-frequency image, j is an index number representing the frequency of the time-frequency image, and i1, i2, j1, and j2 are arbitrary coordinates in the image. p =Z[i1:i2,j1:j2] represents an operation of selecting the range of coordinates in the time direction of the pre-padding original image Z from i1 to i2 and the range of coordinates in the frequency direction from j1 to j2, as shown in FIG. 21(b).

[0039] Next, the brightness l in the pth patch image p , contrast c p , structure s p , p=1,...,P (where P is the total number of patch images) is calculated using equations (11), (12), and (13).

number

number

number

[0040] where μ Xp , μ Yp is the pth patch image X p , Y p is the average value in σ Xp 2 , σ Yp 2 is the pth patch image X p , Y p is the variance in σ Xp , σ Yp is σ Xp 2 , σ Yp 2 is the square root of σ XpYp is the pth patch image X p , Y p where C1 and C2 are constants to ensure numerical stability of the calculation. p , contrast c p , structure s p The closer the value to 1, the more similar the two images are.

[0041] Next, the SSIM value SSIM in the pth patch image p is calculated using equation (14).

number

[0042] In this case, the SSIM value SSIM in equation (14) p is simplified as equation (15).

number

[0043] Finally, the SSIM value calculated for all patch images in Equation (16) is p Average SSIM value E SSIM Calculate.

number

[0044] The loss function calculation unit 715 receives the average SSIM value E SSIM The loss function calculation unit 715 calculates the average SSIM value E SSIM and the separately calculated Kullback-Leibler divergence D KL The loss function value L is calculated using equation (17) and the result is output to the error backpropagation unit 716.

number

[0045] As mentioned above, the average SSIM value E SSIM The closer to 1, the more similar the two images are, and the closer to -1, the less similar they are. On the other hand, the smaller the loss function value L, the more progress has been made in training the VAE model. The average SSIM value E SSIM To express as a loss in training a VAE model, we use the equation (17) to convert from 1 to E SSIM The loss function value L is calculated by subtracting the error backpropagation unit 716. The operation of the error backpropagation unit 716 and subsequent units is exactly the same as in the case of a general VAE model, and therefore a description thereof will be omitted.

[0046] The anomaly detection device 700 that performs anomaly detection using the conventional SSIM-AE described above has the following five problems.

[0047] (1 / 5) As described above, the SSIM calculation unit 714 calculates the average SSIM value E SSIMThe mean and variance values ​​of the input and output for each patch image are calculated and trained so that they approach each other. As a result, if the normal signal contains narrow-band components, the narrow-band components in the reconstructed image Y are standard On the other hand, the intensity of each pixel in the patch image is reconstructed without blurring at the same position as the narrowband component in the normalized time-frequency image X standard Since there is no guarantee that the reconstructed image Y will be close to the rec There is a limit to how small , and the number of cases where normal signals are judged as abnormal increases. If the judgment threshold is increased to avoid this, the problem arises that the anomaly detection performance deteriorates.

[0048] (2 / 5) Average SSIM value E SSIM The optimal value of the patch parameter PP for calculating σ may vary depending on the abnormality pattern of the system to which the anomaly detection method is applied and the parameters that represent that abnormality. Therefore, it is necessary to estimate in advance the optimal value of the patch parameter PP that will improve anomaly detection performance. To estimate the optimal value of the patch parameter PP in advance, it is necessary to measure the anomaly detection performance in advance using normal and abnormal data and select the patch parameter PP that maximizes the anomaly detection performance. However, it is difficult to obtain abnormal data in the actual environment in which the anomaly detection method is applied. This is because, while a large amount of data on the situation can be collected as "normal data" while the machine or system is operating normally, it is expected that situations in which the machine or system behaves abnormally will rarely occur.

[0049] (3 / 5) The standardized time-frequency image X to be input to the VAE model by the operation of the standardization unit 712 standard There is a possibility that some pixels in the reconstructed image Y will have negative values. Furthermore, the SSIM value SSIM in the p-th patch image calculated by the SSIM calculation unit 714 in the early stage of learning is p is generally calculated using equation (15) with α=β=γ=1. Therefore, μ in equation (15) Xp and μ Yp and σXpYp If any of the values ​​is negative, the SSIM value SSIM p becomes negative, and the average SSIM value E SSIM In this way, the SSIM value SSIM p As a result of taking negative values ​​at the beginning of learning, the average SSIM value E SSIM If SSIM is 0, the intensity in the patch image cannot be reconstructed correctly. p As a result of taking negative values ​​in the early stages of learning, the average SSIM value E SSIM When the loss function value L expressed by Equation (17) becomes small due to the fact that it becomes 0, the SSIM value SSIM p Constituting the brightness l p , contrast c p , structure s p is still in the middle of convergence, and as a result, the reconstructed image is not correctly reconstructed. Furthermore, since the loss function value is already small, even if the number of iterations of weight update is increased, the reconstruction does not progress any further, and the reconstruction accuracy does not improve.

[0050] (4 / 5) Generally, the amplitude spectrum of ambient noise (hereinafter referred to as "noise amplitude spectrum") contained in an acoustic signal recorded in the air or underwater, especially underwater, is broadband, with the amplitude (power) increasing at lower frequencies and decreasing at higher frequencies. In the standardization method of equation (1) in the prior art, the frequency spectrum at each time in the time-frequency image X is standardized by the standardization unit 612 while maintaining the shape of the noise amplitude spectrum. Furthermore, the reconstructed image Y by the VAE model output by the reconstruction unit 613 in FIG. 17 or the like is trained to reconstruct the input image including the shape of the noise amplitude spectrum. Therefore, when the acoustic sensor time waveform Twe of normal evaluation data is input, the standardized time-frequency image X calculated by the reconstruction error calculation unit 614 in FIG. 18 standard and the reconstruction error E between the reconstructed image Yrec In the case of a signal with a low frequency, the error variance is larger, and the error variance decreases as the frequency increases. Generally, the judgment threshold Th of the threshold judgment unit 625 needs to be set based on the error variance value so as to limit the number of erroneous judgments to a certain value. However, the fact that the error variance varies depending on the frequency means that threshold judgment cannot be performed using a constant judgment threshold. This makes it difficult to set the value of the judgment threshold Th input to the threshold judgment unit 625 in FIG. 18. To address the issue of the error variance varying depending on the frequency, if threshold judgment is performed using a constant judgment threshold, the judgment threshold must be increased to accommodate the error variance on the low-frequency side where the error variance is large. This results in a problem of reduced detection capability when a signal condition indicating an abnormality occurs in the high-frequency region. Changing the threshold for each frequency is one possible solution, but this would increase the amount of processing required by the threshold judgment unit 625.

[0051] (5 / 5) As a solution to problem (4 / 5), when the noise amplitude spectrum is not white, a method can be considered to whiten the background noise while preserving the frequency and amplitude spectrum of the narrowband signal. However, commonly known methods for whitening background noise flatten the frequency characteristics of non-narrowband signals, such as wideband signals that have a wide spread in the frequency direction, and remove the signals. Therefore, when anomaly detection is performed using an anomaly detection method that incorporates processing to whiten background noise, the wideband signal cannot be recognized.

[0052] Embodiment 1 Next, an anomaly detection device 100 according to a first embodiment that solves the problems of the conventional technology will be described. FIG. 1 is a schematic configuration diagram of the anomaly detection device 100 according to the first embodiment. The anomaly detection device 100 according to the present embodiment detects the presence or absence of an anomaly in a signal that is emitted from an object in the ocean and received by an acoustic sensor provided in a sonar or the like, for example. As shown in FIG. 1, the anomaly detection device 100 according to the present embodiment includes a learning unit 1, an optimal model determination unit 2, an anomaly determination unit 3, a first storage unit 4, and a second storage unit 5.

[0053] The anomaly detection device 100 includes a processor such as a CPU or GPU and a memory, and the learning unit 1, optimal model determination unit 2, and anomaly determination unit 3 are functional units that are realized by the processor executing programs stored in the memory. Note that at least one of the learning unit 1, optimal model determination unit 2, and anomaly determination unit 3 may be realized by a dedicated processing circuit such as an ASIC or FPGA.

[0054] The first storage unit 4 and the second storage unit 5 are composed of volatile or non-volatile semiconductor memory such as RAM or ROM, a magnetic disk, an optical disk, or a flash memory. The first storage unit 4 and the second storage unit 5 may be a single storage device or may be composed of multiple storage devices. Furthermore, the first storage unit 4 and the second storage unit 5 are not limited to being included in the anomaly detection device 100, and may be external devices such as online storage connected via a network.

[0055] The learning unit 1 learns a plurality of first weighting coefficient candidates 41 stored in the first storage unit 4 and a plurality of second weighting coefficient candidates 51 stored in the second storage unit 5. The learning unit 1 receives input of the acoustic sensor time waveform Twn and a plurality of patch parameter candidates PPc of normal data from the outside, the plurality of first weighting coefficient candidates 41 from the first storage unit 4, and the plurality of second weighting coefficient candidates 51 from the second storage unit 5. In the following description, the term "input from the outside" refers to input from an external device communicatively connected to the anomaly detection device 100. However, the data referred to as "input from the outside" is not limited to being input from the outside and may be input from a storage device (not shown) included in the anomaly detection device 100. The learning unit 1 learns a plurality of first weighting coefficient candidates 41 and a plurality of second weighting coefficient candidates 51 based on the input data and stores them in the first storage unit 4 and the second storage unit 5.

[0056] The optimal model determination unit 2 determines optimal first weighting coefficients 42 and second weighting coefficients 52 from the multiple first weighting coefficient candidates 41 and multiple second weighting coefficient candidates 51 learned by the learning unit 1. The optimal model determination unit 2 receives, from the outside, an acoustic sensor time waveform Tw1 of normal data in the optimal model determination unit 2, pseudo-abnormal information FI, a background non-whitening determination threshold Thn, and a background whitening determination threshold Thw, and also receives, from the first storage unit 4, the multiple first weighting coefficient candidates 41, and from the second storage unit 5, the multiple second weighting coefficient candidates 51. The optimal model determination unit 2 determines one of the multiple first weighting coefficient candidates 41 and the multiple second weighting coefficient candidates 51 as the first weighting coefficient 42 and the second weighting coefficient 52 based on the input data, and stores them in the first storage unit 4 and the second storage unit 5.

[0057] The abnormality determination unit 3 determines whether or not there is an abnormality in the acoustic sensor time waveform Twe of the evaluation data, using the first weighting coefficient 42 and the second weighting coefficient 52 determined by the optimum model determination unit 2. The acoustic sensor time waveform Twe of the evaluation data, the background non-whitening determination threshold Thn, and the background whitening determination threshold Thw are input to the abnormality determination unit 3 from the outside, and the first weighting coefficient 42 is input from the first storage unit 4 and the second weighting coefficient 52 is input from the second storage unit 5. The abnormality determination unit 3 outputs a determination result as to whether the acoustic sensor time waveform Twe of the evaluation data is normal or abnormal, based on the input data.

[0058] The first storage unit 4 stores a plurality of first weighting coefficient candidates 41 and first weighting coefficients 42. The first weighting coefficient candidates 41 are weighting coefficients of a VAE model. Although a plurality of weighting coefficients are included for one VAE model, the description will be given with a plurality (a set of) weighting coefficients for one VAE model as one first weighting coefficient candidate 41. The number of first weighting coefficient candidates 41 is the same as the number of parameter set candidates included in the input patch parameter candidates PPc. The first weighting coefficients 42 are a set of weighting coefficients selected by the optimal model determination unit 2 from the plurality of first weighting coefficient candidates 41, and are weighting coefficients used for determination by the abnormality determination unit 3.

[0059] The second storage unit 5 stores a plurality of second weighting coefficient candidates 51 and second weighting coefficients 52. The plurality of second weighting coefficient candidates 51 are weighting coefficients of a VAE model. Although a plurality of weighting coefficients are included for one VAE model, the description will be given by assuming that a plurality (a set of) weighting coefficients for one VAE model is one second weighting coefficient candidate 51. The number of second weighting coefficient candidates 51 is the same as the number of parameter set candidates included in the input patch parameter candidates PPc. The second weighting coefficients 52 are a set of weighting coefficients selected by the optimal model determination unit 2 from the plurality of second weighting coefficient candidates 51, and are weighting coefficients used for determination by the abnormality determination unit 3.

[0060] The following describes in detail the internal configurations of the learning unit 1, the optimal model determination unit 2, and the anomaly determination unit 3. Fig. 2 is a functional block diagram of the learning unit 1 according to embodiment 1. As shown in Fig. 2, the learning unit 1 of this embodiment includes a time-frequency conversion unit 101, a background unwhitening input / output unit 102, a background whitening input / output unit 103, a first weight update unit 104, and a second weight update unit 105.

[0061] The time-frequency conversion unit 101 converts the acoustic sensor time waveform Twn of normal data input from outside into a time-frequency image, and outputs it to the background unwhitening input / output unit 102 and the background whitening input / output unit 103 .

[0062] The background unwhitening input / output unit 102 is connected to the time-frequency transform unit 101, the first weight update unit 104, and the first storage unit 4, and receives as input the time-frequency image from the time-frequency transform unit 101 and the plurality of first weight coefficient candidates 41 from the first storage unit 4. The background unwhitening input / output unit 102 generates a normalized time-frequency image and a plurality of normalized reconstructed images based on the input data, and outputs them to the first weight update unit 104.

[0063] The first weight update unit 104 is connected to the background unwhitening input / output unit 102 and the first storage unit 4. The first weight update unit 104 receives as input the patch parameter candidate PPc from outside, the normalized time-frequency image and the plurality of normalized reconstructed images from the background unwhitening input / output unit 102, and the plurality of first weight coefficient candidates 41 from the first storage unit 4. The first weight update unit 104 updates the plurality of first weight coefficient candidates 41 stored in the first storage unit 4 through learning.

[0064] The background whitening input / output unit 103 is connected to the time-frequency transform unit 101, the second weight update unit 105, and the second storage unit 5. The background whitening input / output unit 103 receives as input the time-frequency image from the time-frequency transform unit 101 and the plurality of second weight coefficient candidates 51 from the second storage unit 5. The background whitening input / output unit 103 generates a background-whitened time-frequency image and a plurality of background-whitened normalized reconstructed images based on the input data, and outputs them to the second weight update unit 105.

[0065] The second weight update unit 105 is connected to the background whitening input / output unit 103 and the second storage unit 5. The second weight update unit 105 receives as input the patch parameter candidate PPc from outside, the background whitened time-frequency image and multiple background white reconstruction images from the background whitening input / output unit 103, and multiple second weight coefficient candidates 51 from the second storage unit 5. The second weight update unit 105 updates the multiple second weight coefficient candidates 51 stored in the second storage unit 5 through learning.

[0066] Fig. 3 is a functional block diagram of the optimum model determination unit 2 according to Embodiment 1. As shown in Fig. 3, the optimum model determination unit 2 includes a time-frequency conversion unit 201, a pseudo anomaly generation unit 202, a background unwhitening input / output unit 203, a background whitening input / output unit 204, a first reconstruction error calculation unit 205, a second reconstruction error calculation unit 206, a first threshold determination unit 207, a second threshold determination unit 208, a first comparison unit 209, and a second comparison unit 210.

[0067] The time-frequency conversion unit 201 is connected to the pseudo-abnormal generation unit 202. The time-frequency conversion unit 201 generates a time-frequency image based on the acoustic sensor time waveform Tw1 of normal data in the optimal model determination unit 2 input from the outside, and outputs the generated image to the pseudo-abnormal generation unit 202.

[0068] The pseudo anomaly generation unit 202 is connected to the time-frequency conversion unit 201, the background unwhitening input / output unit 203, and the background whitening input / output unit 204. The pseudo anomaly generation unit 202 receives as input the time-frequency image from the time-frequency conversion unit 201 and pseudo anomaly information FI from outside. The pseudo anomaly generation unit 202 generates two types of time-frequency images, one for normal data and one for pseudo anomaly data (hereinafter, the term "two types" refers to the two types of normal data and pseudo anomaly data), and outputs these to the background unwhitening input / output unit 203 and the background whitening input / output unit 204. In the following description, the time-frequency image of the normal data will be referred to as a "normal time-frequency image," and the time-frequency image of the pseudo anomaly data will be referred to as a "pseudo anomaly time-frequency image."

[0069] The background unwhitening input / output unit 203 is connected to the pseudo anomaly generator 202 and the first reconstruction error calculator 205. The background unwhitening input / output unit 203 receives as input the two types of time-frequency images from the pseudo anomaly generator 202 and the plurality of first weighting coefficient candidates 41 from the first storage unit 4. The background unwhitening input / output unit 203 generates two types of normalized time-frequency images and two types of normalized reconstruction images based on the input data, and outputs them to the first reconstruction error calculator 205.

[0070] The first reconstruction error calculation unit 205 is connected to the background unwhitening input / output unit 203 and the first threshold determination unit 207. The first reconstruction error calculation unit 205 receives as input two types of normalized time-frequency images and two types of multiple normalized reconstruction images from the background unwhitening input / output unit 203. The first reconstruction error calculation unit 205 calculates reconstruction errors for the two types of normalized time-frequency images based on the input data, and outputs the calculated errors to the first threshold determination unit 207.

[0071] The first threshold determination unit 207 is connected to both the first reconstruction error calculation unit 205 and the first comparison unit 209. The first threshold determination unit 207 receives as input a plurality of reconstruction errors for the normalized time-frequency image from the first reconstruction error calculation unit 205 and an external background non-whitening determination threshold Thn. The first threshold determination unit 207 performs determination on the two types of normalized time-frequency images based on the input data, and outputs the determination result to the first comparison unit 209.

[0072] The first comparing unit 209 is connected to the first storage unit 4 and the first threshold determining unit 207. The first comparing unit 209 receives as input the plurality of first weighting coefficient candidates 41 from the first storage unit 4 and the determination results for the two types of normalized time-frequency images from the first threshold determining unit 207. The first comparing unit 209 determines a first weighting coefficient 42 based on the input data and outputs it to the first storage unit 4.

[0073] The background whitening input / output unit 204 is connected to the pseudo anomaly generator 202 and the second reconstruction error calculator 206. The background whitening input / output unit 204 receives as input two types of time-frequency images from the pseudo anomaly generator 202 and a plurality of second weighting coefficient candidates 51 from the second storage unit 5. The background whitening input / output unit 204 generates two types of background-whitened time-frequency images and two types of a plurality of background-whitened normalized reconstruction images based on the input data, and outputs them to the second reconstruction error calculator 206.

[0074] The second reconstruction error calculation unit 206 is connected to the background whitening input / output unit 204 and the second threshold determination unit 208. The two types of background whitened time-frequency images and two types of multiple background whitened normalized reconstruction images are input to the second reconstruction error calculation unit 206 from the background whitening input / output unit 204. The second reconstruction error calculation unit 206 calculates reconstruction errors for the two types of multiple background whitened time-frequency images based on the input data, and outputs the calculated reconstruction errors to the second threshold determination unit 208.

[0075] The second threshold determination unit 208 is connected to the second reconstruction error calculation unit 206 and the second comparison unit 210. The second threshold determination unit 208 receives as input the reconstruction errors for the two types and multiple background-whitened time-frequency images from the second reconstruction error calculation unit 206 and an external background whitening determination threshold Thw. The second threshold determination unit 208 performs determination on the two types and multiple background-whitened time-frequency images based on the input data, and outputs the determination result to the second comparison unit 210.

[0076] The second comparing unit 210 is connected to the second threshold determining unit 208. The second comparing unit 210 receives as input a plurality of second weighting coefficient candidates 51 from the second storage unit 5 and the determination results for the two types and plurality of background-whitened time-frequency images from the second threshold determining unit 208. The second comparing unit 210 determines a second weighting coefficient 52 based on the input data and outputs it to the second storage unit 5.

[0077] Fig. 4 is a functional block diagram of the abnormality determination unit 3 according to Embodiment 1. As shown in Fig. 4, the abnormality determination unit 3 includes a time-frequency conversion unit 301, a background unwhitening input / output unit 302, a background whitening input / output unit 303, a first reconstruction error calculation unit 304, a second reconstruction error calculation unit 305, a first threshold determination unit 306, a second threshold determination unit 307, and a determination result integration unit 308.

[0078] The time-frequency conversion unit 301 is connected to the background unwhitening input / output unit 302 and the background whitening input / output unit 303. An acoustic sensor time waveform Twe, which is evaluation data, is input from outside to the time-frequency conversion unit 301. The time-frequency conversion unit 301 generates a time-frequency image based on the input data and outputs it to the background unwhitening input / output unit 302 and the background whitening input / output unit 303.

[0079] The background unwhitening input / output unit 302 is connected to the time-frequency transform unit 301 and the first reconstruction error calculation unit 304. The background unwhitening input / output unit 302 receives as input the time-frequency image from the time-frequency transform unit 301 and the first weighting coefficient 42 from the first storage unit 4. The background unwhitening input / output unit 302 generates a normalized time-frequency image and a normalized reconstruction image based on the input data, and outputs them to the first reconstruction error calculation unit 304.

[0080] The first reconstruction error calculation unit 304 is connected to the background unwhitening input / output unit 302 and the first threshold determination unit 306. The first reconstruction error calculation unit 304 receives as input the normalized time-frequency image and the normalized reconstructed image from the background unwhitening input / output unit 302. The first reconstruction error calculation unit 304 calculates a reconstruction error for the normalized time-frequency image based on the input data, and outputs the calculated error to the first threshold determination unit 306.

[0081] The first threshold determination unit 306 is connected to the first reconstruction error calculation unit 304 and the determination result integration unit 308. The first threshold determination unit 306 receives as input the reconstruction error for the normalized time-frequency image from the first reconstruction error calculation unit 304 and an external background non-whitening determination threshold Thn. The first threshold determination unit 306 performs determination on the normalized time-frequency image based on the input data, and outputs the determination result to the determination result integration unit 308.

[0082] The background whitening input / output unit 303 is connected to the time-frequency transform unit 301 and the second reconstruction error calculation unit 305. The background whitening input / output unit 303 receives as input the time-frequency image from the time-frequency transform unit 301 and the second weighting coefficient 52 from the second storage unit 5. The background whitening input / output unit 303 generates a background-whitened time-frequency image and a background-whitened normalized reconstruction image based on the input data, and outputs them to the second reconstruction error calculation unit 305.

[0083] The second reconstruction error calculation unit 305 is connected to the background whitening input / output unit 103 and the second threshold determination unit 307. The second reconstruction error calculation unit 305 receives as input the background whitened time-frequency image and the background whitened normalized reconstruction image from the background whitening input / output unit 303. The second reconstruction error calculation unit 305 calculates a reconstruction error for the background whitened time-frequency image based on the input data, and outputs the calculated error to the second threshold determination unit 307.

[0084] The second threshold determination unit 307 is connected to the second reconstruction error calculation unit 305 and the determination result integration unit 308. The second threshold determination unit 307 receives as input the reconstruction error for the background-whitened time-frequency image from the second reconstruction error calculation unit 305 and an external background whitening determination threshold Thw. The second threshold determination unit 307 performs determination on the background-whitened time-frequency image based on the input data, and outputs the determination result to the determination result integration unit 308.

[0085] The judgment result integrating unit 308 is connected to the first threshold judgment unit 306 and the second threshold judgment unit 307. The judgment result integrating unit 308 receives as input the judgment result for the normalized time-frequency image from the first threshold judgment unit 306 and the judgment result for the background-whitened time-frequency image from the second threshold judgment unit 307. The judgment result integrating unit 308 performs a logical OR on the input judgment results to generate a final judgment result, and outputs it as an abnormality judgment result.

[0086] 5 is a functional block diagram of the background unwhitening input / output unit 102 of the learning unit 1 according to embodiment 1. The background unwhitening input / output unit 203 of the optimal model determination unit 2 and the background unwhitening input / output unit 302 of the anomaly determination unit 3 have the same configuration as the background unwhitening input / output unit 102 of the learning unit 1. As shown in FIG. 5, the background unwhitening input / output unit 102 has a pre-processing unit 121, a reconstruction unit 122, and a post-processing unit 123.

[0087] The pre-processing unit 121 is connected to a first input terminal 81 and a reconstruction unit 122. The first input terminal 81 is connected to the time-frequency transform unit 101, and a time-frequency image is input to the pre-processing unit 121 via the first input terminal 81. The pre-processing unit 121 transforms the input time-frequency image into a normalized time-frequency image and outputs the normalized time-frequency image to the reconstruction unit 122.

[0088] In the case of the background unwhitening input / output unit 203 of the optimal model determination unit 2, the first input terminal 81 is connected to the pseudo anomaly generation unit 202, and two types of time-frequency images are input to the pre-processing unit 121 via the first input terminal 81. The pre-processing unit 121 converts the two types of input time-frequency images into two types of normalized time-frequency images and outputs them to the reconstruction unit 122.

[0089] In the case of the background non-whitening input / output unit 302 of the anomaly determination unit 3, the first input terminal 81 is connected to the time-frequency conversion unit 301, and a time-frequency image corresponding to the acoustic sensor time waveform Twe for evaluation is input to the pre-processing unit 121 via the first input terminal 81. The pre-processing unit 121 converts the input time-frequency image into a normalized time-frequency image and outputs it to the reconstruction unit 122.

[0090] The reconstruction unit 122 is connected to the pre-processing unit 121, the post-processing unit 123, and a second input terminal 82. The second input terminal 82 is connected to the first storage unit 4. The reconstruction unit 122 receives input of the plurality of first weighting coefficient candidates 41 from the first storage unit 4 and the normalized time-frequency image from the pre-processing unit 121 via the second input terminal 82. The reconstruction unit 122 generates a plurality of reconstructed images based on the input normalized time-frequency image and the plurality of first weighting coefficient candidates 41, and outputs the reconstructed images to the post-processing unit 123 together with the normalized time-frequency image.

[0091] In the case of the background unwhitening input / output unit 203 of the optimal model determination unit 2, the second input terminal 82 is connected to the first storage unit 4, and the reconstruction unit 122 receives the plurality of first weighting coefficient candidates 41 from the first storage unit 4 and the two types of normalized time-frequency images from the pre-processing unit 121 via the second input terminal 82. The reconstruction unit 122 generates two types of multiple reconstructed images based on the two types of normalized time-frequency images and the plurality of first weighting coefficient candidates 41 that have been input, and outputs them to the post-processing unit 123 together with the two types of normalized time-frequency images.

[0092] In the case of the background unwhitening input / output unit 302 of the anomaly determination unit 3, the second input terminal 82 is connected to the first storage unit 4, and the first weighting coefficients 42 from the first storage unit 4 and the normalized time-frequency image from the pre-processing unit 121 are input via the second input terminal 82. The reconstruction unit 122 generates a reconstructed image based on the input normalized time-frequency image and first weighting coefficients 42, and outputs the reconstructed image together with the normalized time-frequency image to the post-processing unit 123.

[0093] The post-processing unit 123 is connected to the reconstruction unit 122 and the output terminal 83. The normalized time-frequency image and a plurality of reconstructed images from the reconstruction unit 122 are input to the post-processing unit 123. The post-processing unit 123 generates a plurality of normalized reconstructed images from the input reconstructed images, and outputs them to the output terminal 83 together with the normalized time-frequency image.

[0094] In the case of the background unwhitening input / output unit 203 of the optimal model determination unit 2, the two types of normalized time-frequency images and two types of multiple reconstructed images from the reconstruction unit 122 are input to the post-processing unit 123. The post-processing unit 123 generates two types of multiple normalized reconstructed images from the two types of multiple reconstructed images that have been input, and outputs them to the output terminal 83 together with the two types of normalized time-frequency images.

[0095] In the case of the background unwhitening input / output unit 302 of the abnormality determination unit 3, the normalized time-frequency image and one reconstructed image from the reconstruction unit 122 are input to the post-processing unit 123. The post-processing unit 123 generates one normalized reconstructed image from the one input reconstructed image, and outputs it to the output terminal 83 together with the normalized time-frequency image.

[0096] The output terminal 83 outputs the normalized time-frequency image and the plurality of normalized reconstructed images from the post-processing unit 123 to the first weight updating unit 104. In the case of the background unwhitening input / output unit 203 of the optimal model determination unit 2, the output terminal 83 outputs the two types of normalized time-frequency images and the two types of normalized reconstructed images from the post-processing unit 123 to the first reconstruction error calculation unit 205. In the case of the background unwhitening input / output unit 302 of the abnormality determination unit 3, the output terminal 83 outputs the normalized time-frequency image and one normalized reconstructed image from the post-processing unit 123 to the first reconstruction error calculation unit 304.

[0097] 6 is a functional block diagram of the background whitening input / output unit 103 of the learning unit 1 according to Embodiment 1. The background whitening input / output unit 204 of the optimal model determination unit 2 and the background whitening input / output unit 303 of the abnormality determination unit 3 have the same configuration as the background whitening input / output unit 103 of the learning unit 1. As shown in FIG. 6, the background whitening input / output unit 103 has a background whitening unit 131, a reconstruction unit 132, and a post-processing unit 133.

[0098] The background whitening unit 131 is connected to a third input terminal 84 and the reconstruction unit 132. The third input terminal 84 is connected to the time-frequency transform unit 101, and a time-frequency image is input to the background whitening unit 131 via the third input terminal 84. The background whitening unit 131 transforms the input time-frequency image into a background-whitened time-frequency image and outputs it to the reconstruction unit 132.

[0099] In the case of the background whitening input / output unit 204 of the optimal model determination unit 2, the third input terminal 84 is connected to the pseudo anomaly generation unit 202, and two types of time-frequency images are input to the background whitening unit 131 via the third input terminal 84. The background whitening unit 131 converts the two types of input time-frequency images into two types of background-whitened time-frequency images and outputs them to the reconstruction unit 132.

[0100] In the case of the background whitening input / output unit 303 of the anomaly determination unit 3, the third input terminal 84 is connected to the time-frequency conversion unit 301, and a time-frequency image corresponding to the acoustic sensor time waveform Twe for evaluation is input to the background whitening unit 131 via the third input terminal 84. The background whitening unit 131 converts the input time-frequency image into a background-whitened time-frequency image and outputs it to the reconstruction unit 132.

[0101] The reconstruction unit 132 is connected to the background whitening unit 131, the post-processing unit 133, and a fourth input terminal 85. The fourth input terminal 85 is connected to the second storage unit 5. The reconstruction unit 132 receives, via the fourth input terminal 85, the plurality of second weighting coefficient candidates 51 from the second storage unit 5 and the background whitened time-frequency image from the background whitening unit 131. The reconstruction unit 132 generates a plurality of background whitened reconstructed images based on the input background whitened time-frequency image and the plurality of second weighting coefficient candidates 51, and outputs them to the post-processing unit 133 together with the background whitened time-frequency image.

[0102] In the case of the background unwhitening input / output unit 203 of the optimal model determination unit 2, the fourth input terminal 85 is connected to the second storage unit 5, and the reconstruction unit 132 receives, via the fourth input terminal 85, the plurality of second weighting coefficient candidates 51 from the second storage unit 5 and the two types of background whitened time-frequency images from the background whitening unit 131. The reconstruction unit 132 generates two types of background whitened reconstructed images based on the two types of background whitened time-frequency images and the plurality of second weighting coefficient candidates 51 that have been input, and outputs them to the post-processing unit 133 together with the two types of background whitened time-frequency images.

[0103] In the case of the background unwhitening input / output unit 302 of the abnormality determination unit 3, the fourth input terminal 85 is connected to the second storage unit 5, and the second weighting coefficients 52 from the second storage unit 5 and the background whitening time-frequency image from the background whitening unit 131 are input via the fourth input terminal 85. The reconstruction unit 132 generates a background whitened reconstructed image based on the input background whitened time-frequency image and second weighting coefficients 52, and outputs the generated image together with the background whitened time-frequency image to the post-processing unit 133.

[0104] The post-processing unit 133 is connected to the reconstruction unit 132 and the output terminal 86. The post-processing unit 133 receives the background-whitened time-frequency image and the plurality of background-whitened reconstructed images from the reconstruction unit 122. The post-processing unit 133 generates a plurality of background-whitened normalized reconstructed images from the plurality of input background-whitened reconstructed images, and outputs the generated images to the output terminal 86 together with the background-whitened time-frequency image.

[0105] In the case of the background unwhitening input / output unit 203 of the optimal model determination unit 2, the two types of background-whitened time-frequency images and two types of background-whitened reconstructed images from the reconstruction unit 132 are input to the post-processing unit 133. The post-processing unit 133 generates two types of background-whitened normalized reconstructed images from the two types of background-whitened reconstructed images that have been input, and outputs them to the output terminal 86 together with the two types of background-whitened time-frequency images.

[0106] In the case of the background unwhitening input / output unit 302 of the abnormality determination unit 3, the background-whitened time-frequency image and one background-whitened reconstructed image from the reconstruction unit 132 are input to the post-processing unit 133. The post-processing unit 133 generates one background-whitened normalized reconstructed image from the one background-whitened reconstructed image that was input, and outputs it to the output terminal 86 together with the background-whitened time-frequency image.

[0107] The output terminal 86 outputs the background-whitened time-frequency image and the plurality of background-whitened normalized reconstructed images from the post-processing unit 133 to the second weight updating unit 105. In the case of the background whitening input / output unit 204 in the optimal model determination unit 2, the output terminal 86 outputs the two types of background-whitened time-frequency images and the two types of plurality of background-whitened normalized reconstructed images from the post-processing unit 133 to the second reconstruction error calculation unit 206. In the case of the background whitening input / output unit 303 in the abnormality determination unit 3, the output terminal 86 outputs the background-whitened time-frequency image and the background-whitened normalized reconstructed image from the post-processing unit 133 to the second reconstruction error calculation unit 305.

[0108] 7 is a functional block diagram of the first weight update unit 104 of the learning unit 1 according to Embodiment 1. The second weight update unit 105 of the learning unit 1 has the same configuration as the first weight update unit 104. As shown in FIG. 7, the first weight update unit 104 has a reconstruction error calculation unit 141, an SSIM calculation unit 142, a loss function calculation unit 143, and an error backpropagation unit 144.

[0109] The reconstruction error calculation unit 141 is connected to a fifth input terminal 87 and a loss function calculation unit 143. The fifth input terminal 87 is connected to the background unwhitening input / output unit 102, and the normalized time-frequency image and the plurality of normalized reconstructed images are input to the reconstruction error calculation unit 141 via the fifth input terminal 87. The reconstruction error calculation unit 141 calculates the reconstruction errors of the plurality of normalized reconstructed images based on the input normalized time-frequency images, and outputs the calculated reconstruction errors to the loss function calculation unit 143.

[0110] In the case of the second weight update unit 105, the fifth input terminal 87 is connected to the background whitening input / output unit 103, and the background whitened time-frequency image and multiple background whitened normalized reconstructed images are input to the reconstruction error calculation unit 141 via the fifth input terminal 87. Then, the reconstruction error calculation unit 141 calculates the reconstruction errors of the multiple background whitened normalized reconstructed images based on the input background whitened time-frequency image, and outputs the calculated reconstruction errors to the loss function calculation unit 143.

[0111] The SSIM calculation unit 142 is connected to the fifth input terminal 87 and the loss function calculation unit 143. The fifth input terminal 87 is connected to the background unwhitening input / output unit 102. The SSIM calculation unit 142 receives as input the normalized time-frequency image and multiple normalized reconstructed images from the background unwhitening input / output unit 102, and multiple patch parameter candidates PPc from outside. The SSIM calculation unit 142 calculates multiple average SSIM values ​​corresponding to the multiple normalized reconstructed images using the multiple input patch parameter candidates PPc, and outputs the average SSIM values ​​to the loss function calculation unit 143.

[0112] In the case of the second weight update unit 105, the background-whitened time-frequency image and multiple background-whitened normalized reconstructed images from the background-whitening input / output unit 103, and external patch parameter candidates PPc are input to the SSIM calculation unit 142. Then, the SSIM calculation unit 142 calculates multiple average SSIM values ​​corresponding to the multiple background-whitened normalized reconstructed images using the multiple input patch parameter candidates PPc, and outputs the average SSIM values ​​to the loss function calculation unit 143.

[0113] The loss function calculation unit 143 is connected to the reconstruction error calculation unit 141, the SSIM calculation unit 142, and the error backpropagation unit 144. The loss function calculation unit 143 receives as input a plurality of reconstruction errors from the reconstruction error calculation unit 141 corresponding to a plurality of normalized reconstructed images and a plurality of average SSIM values ​​from the SSIM calculation unit 142. The loss function calculation unit 143 calculates a loss function value from the input reconstruction error and average SSIM value for each of the plurality of normalized reconstructed images, and outputs the loss function value to the error backpropagation unit 144.

[0114] In the case of the second weight update unit 105, the loss function calculation unit 143 receives as input a plurality of reconstruction errors from the reconstruction error calculation unit 141 corresponding to a plurality of background-whitened normalized reconstructed images and a plurality of average SSIM values ​​from the SSIM calculation unit 142. The loss function calculation unit 143 calculates a loss function value from the input reconstruction error and average SSIM value for each of the plurality of background-whitened normalized reconstructed images, and outputs the loss function value to the error backpropagation unit 144.

[0115] The error backpropagation unit 144 is connected to the loss function calculation unit 143 and the input / output terminal 88. The input / output terminal 88 is connected to the first storage unit 4. The error backpropagation unit 144 receives as input a plurality of loss function values ​​from the loss function calculation unit 143 corresponding to a plurality of standardized reconstructed images, and a plurality of first weighting coefficient candidates 41 from the first storage unit 4 via the input / output terminal 88. The error backpropagation unit 144 updates the corresponding first weighting coefficient candidates 41 so as to reduce the loss function value of each standardized reconstructed image, and outputs the updated first weighting coefficient candidates 41 to the first storage unit 4 via the input / output terminal 88.

[0116] In the case of the second weight update unit 105, the input / output terminal 88 is connected to the second storage unit 5. The back-propagation unit 144 receives as input a plurality of loss function values ​​from the loss function calculation unit 143 corresponding to a plurality of background-whitened normalized reconstructed images, and a plurality of second weight coefficient candidates 51 from the second storage unit 5 via the input / output terminal 88. The back-propagation unit 144 updates the corresponding second weight coefficient candidates 51 so as to reduce the loss function value of each background-whitened normalized reconstructed image, and outputs the updated second weight coefficient candidates 51 to the second storage unit 5 via the input / output terminal 88.

[0117] The operation of the learning unit 1 of this embodiment will be described below. Prior to describing the detailed operation of each unit of the learning unit 1 shown in Fig. 2, the patch parameter candidate PPc, the plurality of first weighting coefficient candidates 41, the plurality of second weighting coefficient candidates 51, and the acoustic sensor time waveform Twn of normal data in the learning unit 1 will be described.

[0118] 8 is a diagram illustrating the patch parameter candidate PPc. As shown in FIG. 8, the patch parameter candidate PPc is a candidate having an average SSIM value E SSIM The padding width P is a parameter used to calculate pad and patch size P size and the shift width P stride It is a list with elements that are combinations of the values ​​of s, s1, s2, and s3. s1 pad , P s2 size , P s3 strideLet be the padding width, patch size, and shift width values ​​for the sth element of the list, respectively.

[0119] Let pad_max, size_max, and stride_max be the number of candidates for padding width, patch size, and shift width in the list, respectively. The list of patch parameter candidates PPc is max = pad_max × size_max × stride_max elements (s=1,2,…,S max )

[0120] Next, the plurality of first weighting coefficient candidates 41 and the plurality of second weighting coefficient candidates 51 will be described. In this embodiment, the same number of different weighting coefficients as the number of parameter candidates included in the patch parameter candidate PPc are prepared. Furthermore, different weighting coefficients are prepared for cases where the time-frequency image is background whitened and cases where it is not. Here, as shown in equation (18), an area list in which the plurality of weighting coefficient candidates are stored is model o Define

number

[0121] Here, the superscript o becomes "NotWhite" when it corresponds to the first storage unit 4, and becomes "White" when it corresponds to the second storage unit 5. s o (s=1,…,S max ) represents a plurality of first weighting coefficient candidates 41 or a plurality of second weighting coefficient candidates 51. s o (s=1,…,S max ) is initialized with a random number value such as "Xavier's initial value" or "He's initial value" as in the conventional technique, but the initial values ​​of the weighting coefficients for each s are different values.

[0122] Next, we will describe the acoustic sensor time waveform Twn of normal data in the learning unit 1. As with the prior art, when the acoustic sensor time waveform in the actual environment can be obtained, Q train(However, Q train is the number of data in learning unit 1) is taken as the acoustic sensor time waveform Twn of normal data in learning unit 1. If it is difficult to obtain the acoustic sensor time waveform in the actual environment, the waveform electrically generated as follows is taken as the acoustic sensor time waveform Twn of normal data in learning unit 1.

[0123] Q train When waveforms are generated from time series with different noise types and the acoustic signal to be monitored is a narrowband signal, Q train The waveform should contain different frequencies and amplitudes within the range in which the frequency and amplitude of the narrowband component normally fluctuate. If the acoustic signal to be monitored is a wideband signal, Q train The waveform contains different center frequencies, amplitudes and bandwidths within the range in which the center frequencies, amplitudes and bandwidths of the wideband components operate normally, and in the case of a mixture of narrowband and wideband signals, contains both of the above.

[0124] Next, detailed operations of each part of the learning unit 1 will be described. The time-frequency conversion unit 101 of the learning unit 1 converts the input acoustic sensor time waveform Twn of normal data into a time-frequency image, and outputs the result to the background unwhitening input / output unit 102 and the background whitening input / output unit 103. The operation of the time-frequency conversion unit 101 is the same as that of the time-frequency conversion unit 611 of the prior art, so a description thereof will be omitted. The number of output time-frequency images is Q train There are individuals.

[0125] The pre-processing unit 121 of the background unwhitening input / output unit 102 receives the time-frequency image X from the time-frequency transform unit 101 through the first input terminal 81. q (q=1,…,q M , but q M is the number of input time-frequency images, and in learning unit 1, q M =Q train ) is input to the pre-processing unit 121 of the background unwhitening input / output unit 102. q is normalized so that all pixel values ​​are non-negative, and the resulting normalized time-frequency image X qNotWhite (q=1,…,q M ) to the reconstruction unit 122 of the background unwhitening input / output unit 102.

[0126] 9 is a diagram illustrating an example of a procedure for normalizing a time-frequency image to a non-negative value. First, a time-frequency image X is normalized using Equation (5) as in the conventional technique. q standard (Fig. 9(a)). Next, the normalized time-frequency image X q standard 9(b)), and then multiplying it by the ReLU function of equation (19) to normalize it to a non-negative value (FIG. 9(c)).

number

[0127] Normalized time-frequency image X q NotWhite (q=1,…,q M ) can be calculated as shown in equation (20).

number

[0128] The reconstruction unit 122 converts the normalized time-frequency image X q NotWhite (q=1,…,Q train ) is input to a VAE model of a plurality of first weighting coefficient candidates 41 input from the first storage unit 4 through the second input terminal 82, and a plurality of reconstructed images Y qs NotWhite , q=1,…,Q train , s=1,…,S max Then, multiple reconstructed images Y qsNotWhite , q=1,…,Q train , s=1,…,S max and the input normalized time-frequency image X q NotWhite , q=1,…,Q train and output to the post-processing unit 123.

[0129] The operation of the reconstruction unit 122 is similar to that of the reconstruction unit 613 of the prior art, but differs in that it outputs a plurality of reconstructed images in response to the input of a plurality of first weighting coefficient candidates 41. If the operation of the VAE model using a plurality of first weighting coefficient candidates 41 is expressed as a function f(), and q is used as an index for the time-frequency image, and the reconstruction units 122 of the background unwhitening input / output unit 102 and the background whitening input / output unit 103 are described by a common formula, the time-frequency image X q o From multiple weight coefficient candidates w s o Multiple reconstructed images Y qs o The conversion to is expressed as in equation (21).

number

[0130] where q M is the number of input time-frequency images, S max is the total number of parameter candidates included in the patch parameter candidate PPc, and the superscript o is "NotWhite" indicating that the reconstruction unit 122 of the background unwhitening input / output unit 102 corresponds to the background unwhitening input / output unit 102. In the reconstruction unit 122 of the background unwhitening input / output unit 102 in the learning unit 1, q M =Q train is.

[0131] The post-processing unit 123 processes the multiple reconstructed images Y qs NotWhite , q=1,…,q M , s=1,…,S max The post-processing unit 123 normalizes the normalized reconstructed images Yqs (NotWhite)´ , q=1,…,q M , s=1,…,S max and the input normalized time-frequency image X q NotWhite , q=1,…,q M and is output to the first weight update unit 104 via the output terminal 83.

[0132] Multiple normalized reconstructed images Y qs (NotWhite)´ , q=1,…,q M , s=1,…,S max is calculated using equation (22).

number

[0133] This concludes the description of the background unwhitening input / output unit 102 in the learning unit 1. By normalizing the time-frequency image and the reconstructed image to non-negative values ​​in the background unwhitening input / output unit 102, the two images input to the SSIM calculation unit 142 always have non-negative values.

[0134] The operation of the first weight update unit 104 of the learning unit 1 will be described. The reconstruction error calculation unit 141 of the first weight update unit 104 receives the normalized time-frequency image X q NotWhite , q=1,…,q M and multiple normalized reconstructed images Y qs (NotWhite)´ , q=1,…,q M , s=1,…,S max The first weight update unit 104 receives a plurality of reconstruction errors E rec,(q,s) NotWhite , q=1,…,q M , s=1,…,S max and outputs the result to the loss function calculation unit 143.

[0135] The operation of the reconstruction error calculation unit 141 is the same as that of the reconstruction error calculation unit 614 of the prior art, but differs in that it accepts input of a plurality of normalized reconstructed images and outputs a plurality of reconstruction errors. rec,(q,s) o , q=1,…,q M , s=1,…,S max is calculated by equation (23).

number

[0136] Here, the superscript o is "NotWhite" in the first weight update unit 104, indicating that it is input from the background non-whitening input / output unit 102, and is "White" in the second weight update unit 105, indicating that it is input from the background whitening input / output unit 103. M =Q train is.

[0137] The SSIM calculation unit 142 of the first weight update unit 104 receives the normalized time-frequency image X from the background unwhitening input / output unit 102 via the fifth input terminal 87. q NotWhite , q=1,…,Q train and multiple normalized reconstructed images Y qs (NotWhite)´ , q=1,…,Q train , s=1,…,S max and patch parameter candidates PPc from the outside. The SSIM calculation unit 142 calculates a plurality of average SSIM values ​​E SSIM,(q,s) NotWhite , q=1,…,Q train , s=1,…,S max and outputs the result to the loss function calculation unit 143. The operation of the SSIM calculation unit 142 is similar to that of the SSIM calculation unit 714 of the prior art, except that it outputs multiple average SSIM values ​​in response to the input of a normalized time-frequency image and multiple normalized reconstructed images.

[0138] Here, the operation of the SSIM calculation unit 142 of the first weight update unit 104 and the second weight update unit 105 is made common by the calculation formula, which is used to calculate the average SSIM value E SSIM,(q,s) o , q=1,…,Q train , s=1,…,S max First, as shown in FIG. 20(a) of the prior art, a plurality of unpadding original images Z q,s , q=1,…,Q train , s=1,…,S max Padding width P around s pad , s=1,…,S max By adding a region with a specific value within the range of Z, the image is enlarged, and after multiple padding, the original image Z q,s padding , q=1,…,Q train , s=1,…,S max get.

[0139] Next, as shown in Equation (24) and Figure 20(b), we apply multiple paddings to the original image Z q,s padding , q=1,…,Q train , s=1,…,S max By cutting out small sections of the image from q,s,p , q=1,…,Q train , s=1,…,S max (P is the total number of patch images) is obtained. However, multiple patch images Z q,s,p is the original image Z before padding q,s is the time-frequency image X q o If the input patch image X q,p o and the original image Z before padding q,s are multiple reconstructed images Y q,s,p (o)´ If the output patch image Y q,s,p oHere, the superscript o becomes “NotWhite” in the first weight update unit 104, indicating that the input is from the background non-whitening input / output unit 102, and becomes “White” in the second weight update unit 105, indicating that the input is from the background whitening input / output unit 103.

number

[0140] Next, the qth training data, the pth patch image, and the sth patch parameter candidate PPc are calculated using the brightness l q,s,p o , contrast c q,s,p o , structure s q,s,p o , q=1,…,Q train , s=1,…,S max , p=1,...,P are calculated using equations (25), (26), and (27).

number

number

number

[0141] Here, μ in equations (25) to (27) Xq,po , μ Yq,s,po , σ Xq,po , σ Yq,s,po , σ Xq,poYq,s,po (o is a subscript X q,p , Y q,s,p The superscript of ( ) is the patch image X q,p o , Y q,s,p (o)´are the mean, square root of the variance, and covariance calculated for the above weights, C1 and C2 are constants for ensuring the numerical stability of the calculation, and the superscript o is “White” in the first weight update unit 104, indicating that the input is from the background unwhitening input / output unit 102, and is “White” in the second weight update unit 105, indicating that the input is from the background whitening input / output unit 103.

[0142] Next, multiple SSIM valuesSSIM q,s,p o , q=1,…,Q train , s=1,…,S max , p=1,...,P are calculated using equation (28).

number

[0143] Here, the constants α, β, and γ (α and β are arbitrary real numbers, and γ is an integer equal to or greater than 1) are generally set to α=β=γ=1, as in the prior art. In this case, multiple SSIM values ​​SSIM q,s,p o , q=1,…,Q train , s=1,…,S max , p=1,...,P is simplified as equation (29).

number

[0144] Finally, the average SSIM values ​​E SSIM,(q,s) o , q=1,…,Q train , s=1,…,S max Calculate.

number

[0145] Here, the superscript o becomes “NotWhite” in the first weight update unit 104, indicating that the input is from the background non-whitening input / output unit 102, and becomes “White” in the second weight update unit 105, indicating that the input is from the background whitening input / output unit 103.

[0146] The loss function calculation unit 143 of the first weight update unit 104 receives a plurality of reconstruction errors E from the reconstruction error calculation unit 141 corresponding to a plurality of normalized reconstructed images. rec,(q,s) NotWhite , q=1,…,q M , s=1,…,S max and a plurality of average SSIM values ​​E from the SSIM calculation unit 142. SSIM,(q,s) NotWhite , q=1,…,Q train , s=1,…,S max The loss function calculation unit 143 calculates a plurality of loss function values ​​L corresponding to a plurality of normalized reconstructed images. q,s NotWhite , q=1,…,Q train , s=1,…,S max and outputs the result to the error backpropagation unit 144. The operation of the loss function calculation unit 143 of the first weight update unit 104 and the second weight update unit 105 is made common to calculate a plurality of loss function values ​​L q,s o , q=1,…,Q train , s=1,…,S max is calculated using equation (31).

number

[0147] where D KL,(q,s) o , q=1,…,Q train , s=1,…,S maxis the Kullback-Leibler divergence calculated separately for the time-frequency image and the plurality of first weighting coefficient candidates 41 and second weighting coefficient candidates 51. The superscript o in the first weighting update unit 104 becomes “NotWhite” indicating that the input is from the background non-whitening input / output unit 102, and in the second weighting update unit 105 becomes “White” indicating that the input is from the background whitening input / output unit 103.

[0148] The error backpropagation unit 144 of the first weight update unit 104 receives a plurality of loss function values ​​L q,s NotWhite , q=1,…,Q train , s=1,…,S max and a plurality of first weighting coefficient candidates 41w from the first storage unit 4 via the input / output terminal 88. s NotWhite , s=1,…,S max and are input. A plurality of first weighting coefficient candidates 41w s NotWhite , s=1,…,S max The total number of first weighting coefficient candidates 41 is calculated from the number of elements of max Then, read S max Each of the first weighting coefficient candidates 41 is train Loss function values ​​L q,s NotWhite , q=1,…,Q train are updated based on the backpropagation algorithm and output as learned first weight coefficient candidates 41 to the corresponding s-th positions in the first storage unit 4 via the input / output terminal 88 and stored. This concludes the description of the first weight update unit 104.

[0149] Next, the operation of the background whitening input / output unit 103 of the learning unit 1 shown in Fig. 6 will be described. First, the background whitening unit 131 of the background whitening input / output unit 103 receives the time-frequency image X q , q=1,…,q M (However, q M is the number of input time-frequency images, and in learning unit 1, q M =Q train) is input to the background whitening unit 131. q , the noise amplitude spectrum is estimated at each frequency from the frequency spectrum at each time in , and the ambient noise is normalized by dividing the frequency spectrum at that frequency using the noise amplitude spectrum, and the resulting background-whitened time-frequency image X q White , q=1,…,q M to the reconstruction unit 132 of the background whitening input / output unit 103. Hereinafter, the process of estimating a noise amplitude spectrum at each frequency from the frequency spectrum at each time and dividing the frequency spectrum at that frequency using the noise amplitude spectrum to normalize the ambient noise will be referred to as "background whitening processing."

[0150] There are various known methods for background whitening processing, but here we will explain the principles of noise normalization processing used in a technology called "Low Frequency Analysis Recorder (hereinafter referred to as "LOFAR")" described in the non-patent document "Sonar signal processing," Richard O. Nielsen, Artech House, 1991.

[0151] 10A and 10B are diagrams illustrating the procedure of background whitening processing by LOFAR. q The frequency spectrum U at time i of i and Fig. 10(b) shows the expanded frequency spectrum U i ´´ , and Fig. 10(c) shows the background whitening frequency spectrum U i White is.

[0152] First, as shown in equation (32), the time-frequency image X q from the frequency spectrum U at time i i Take out.

number

[0153] Next, the frequency spectrum U i The noise amplitude spectrum E for the jth frequency of noise To calculate the frequency spectrum U i Expand the expanded frequency spectrum U i ´´ The expanded frequency spectrum U i ´´ is the frequency spectrum U i The first value of U i [0] to N White Duplicate and U i Before joining, the frequency spectrum U i The terminal value of U i [N F -1] to N White Duplicate and U i is obtained by joining it to the end of

number

[0154] where k is the index number and N White is the noise amplitude spectrum E noise is half the length of the noise estimation interval in the frequency direction for which U i [j] is U i represents the operation of extracting the j-th frequency element of the frequency spectrum U i The number of elements in F While the expanded frequency spectrum U i ´´ The number of elements is N F +2·N White The frequency spectrum U i The j-th element in is the expanded frequency spectrum U i ´´ j+N in White corresponds to the th element.

[0155] Next, the expanded frequency spectrum U i ´´ Using the noise amplitude spectrum E noiseCalculate the noise amplitude spectrum E noise jth element of E noise [j] is the expanded frequency spectrum U i ´´ j+N White Centering on the j+N element White 2×N excluding the first White The average of elements is calculated.

number

[0156] Next, as shown in equation (35), the noise amplitude spectrum E noise jth element of E noise [j] and frequency spectrum U i jth element of U i [j] and E noise Correct [j].

number

[0157] Next, the background whitened time-frequency image X white The background whitening frequency spectrum U at the i-th time i white Calculate.

number

[0158] The background whitening frequency spectrum U at the jth frequency obtained by equation (36) i white is the frequency spectrum U i The noise amplitude spectrum E estimated at each frequency noiseBy normalizing with , the value becomes close to "1" at frequencies containing only noise, and the value becomes the amount of signal protrusion relative to the ambient noise at frequencies containing signal.

[0159] Finally, the background-whitened time-frequency image X q White , q=1,…,q M is the background whitening frequency spectrum U obtained from Eq. (36) for all time instants. i white , i=0,1,…,N T By arranging -1 in the time direction, it is calculated as shown in equation (37) for each q.

number

[0160] In learning part 1, q M =Q train However, the noise normalization process explained in equations (32) to (37) is the simplest procedure, and various improved methods are known. The noise normalization process of the background whitening unit 131 may use such an improved method. Furthermore, when using the squared absolute values ​​expressed in decibels as the values ​​of each element of the time-frequency image, rather than the absolute values ​​as in equation (3), the noise amplitude spectrum E noise When calculating the expanded frequency spectrum U i ´´ After converting each element of into antilogarithms, we apply Equation (34) and express the result in decibels again. Furthermore, the background whitening frequency spectrum U i white When calculating, the value of each element is calculated by subtraction, not by division as in equation (36).

[0161] The reconstruction unit 132 of the background whitening input / output unit 103 receives the background whitened time-frequency image X q White , q=1,…,q Mand a plurality of second weighting coefficient candidates 51 from the second storage unit 5 through the fourth input terminal 85. The reconstruction unit 132 inputs the background-whitened time-frequency image to the VAE model and generates a plurality of background-whitened reconstructed images Y qs White , q=1,…,q M , s=1,…,S max and generate multiple background-whitened reconstructed images Y qs White , q=1,…,q M , s=1,…,S max and the input background-whitened time-frequency image X q White , q=1,…, Q train and output them to the post-processing unit 133. The operation of the reconstruction unit 132 in the background whitening input / output unit 103 is exactly the same as that of the reconstruction unit 122 in the background unwhitening input / output unit 102 except for the input, and the superscript o in equation (21) can be processed as "White" indicating that it corresponds to the background whitening input / output unit 103, so detailed explanation will be omitted.

[0162] The post-processing unit 133 receives the background-whitened time-frequency image X from the reconstruction unit 132. q White , q=1,…,q M and multiple background-whitened reconstructed images Y qs White , q=1,…,q M , s=1,…,S max The post-processing unit 133 receives the inputs Y qs White , q=1,…,q M , s=1,…,S max The resulting background-whitened normalized reconstructed images Y qs (White)´ , q=1,…,q M , s=1,…,S max and the background-whitened time-frequency image X q White , q=1,…,q Mand output to the second weight update unit 105 via the output terminal 86. The operation of the post-processing unit 133 is the same as that of the post-processing unit 123 in the background unwhitening input / output unit 102, but the background-whitened time-frequency image X q White , q=1,…,q M Since the noise frequency value is close to "1", instead of using Equation (22), we add "1" to each element of the multiple background-whitened reconstructed images for the background-whitened time-frequency image as in Equation (38), and then apply the ReLU function to obtain multiple background-whitened normalized reconstructed images Y qs (White)´ , q=1,…,q M , s=1,…,S max Ask for.

number

[0163] This concludes the description of the background whitening input / output unit 103 shown in Fig. 6. The background whitening unit 131 performs background whitening on the time-frequency image and normalizes the background-whitened reconstructed image to a non-negative value, so that the two images input to the SSIM calculation unit 142 always have non-negative values.

[0164] The operation of the second weight update unit 105 is the same as that of the first weight update unit 104. The reconstruction error calculation unit 141 receives the background whitened time-frequency image X from the background whitening input / output unit 103 via the fifth input terminal 87. q White , q=1,…,Q train and multiple background-whitened normalized reconstructed images Y qs (White)´ , q=1,…,Q train , s=1,…,S max The reconstruction error calculation unit 141 calculates a plurality of reconstruction errors E rec,(q,s) White , q=1,…,Q train , s=1,…,S maxand outputs the result to the loss function calculation unit 143.

[0165] The SSIM calculation unit 142 receives the background whitened time-frequency image X from the background whitening input / output unit 103 via a fifth input terminal 87. q White , q=1,…,Q train and multiple background-whitened normalized reconstructed images Y qs (White)´ , q=1,…,Q train , s=1,…,S max and patch parameter candidates PPc from the outside. The SSIM calculation unit 142 calculates a plurality of average SSIM values ​​E corresponding to a plurality of background-whitened normalized reconstructed images using equations (24) to (30). SSIM,(q,s) White , q=1,…,Q train , s=1,…,S max and outputs the result to the loss function calculation unit 143.

[0166] The loss function calculation unit 143 receives the multiple reconstruction errors E rec,(q,s) White , q=1,…,Q train , s=1,…,S max and a plurality of average SSIM values ​​E from the SSIM calculation unit 142. SSIM,(q,s) White , q=1,…,Q train , s=1,…,S max The loss function calculation unit 143 calculates the loss function value L for the plurality of background-whitened reconstructed images using Equation (31). q,s White , q=1,…,Q train , s=1,…,S max and outputs the result to the error backpropagation unit 144.

[0167] Detailed explanation of the operations of each unit up to this point will be omitted, since it is sufficient to process the superscript o in equations (23) to (31) described in the first weight update unit 104 as “White,” which indicates that it corresponds to the background whitening input / output unit 103.

[0168] The error backpropagation unit 144 receives the multiple loss function values ​​L from the loss function calculation unit 143. q,s White , q=1,…,Q train , s=1,…,S max and a plurality of second weighting coefficient candidates 51w from the second storage unit 5 through the input / output terminal 88. s white , s=1,…,S max The backpropagation unit 144 receives S max Each of the second weighting coefficient candidates 51 is updated based on the backpropagation algorithm, and is output as a learned second weighting coefficient candidate 51 to the corresponding s-th position in the second storage unit 5 via the input / output terminal 88 and stored therein. This completes the description of the operation of the learning unit 1 shown in FIG. 2.

[0169] Hereinafter, for the conventional technology, multiple loss function values ​​L corresponding to the normalized time-frequency image of Equation (31) are used. q,s NotWhite We will explain the difference in behavior when learning using this method. Figure 11 shows examples of time-frequency images of normal data. Figure 11(a) is a time-frequency image when the signal frequency is 5 Hz, Figure 11(b) is a time-frequency image when the signal frequency is 6 Hz, and Figure 11(c) is a time-frequency image when the signal frequency is 4.5 Hz.

[0170] FIG. 12 is a diagram illustrating the effect of learning by the learning unit 1 according to the first embodiment. FIG. 12 illustrates an example of a reconstructed image obtained when normal data including the example time-frequency image shown in FIG. 11 is learned. FIG. 12(a) illustrates an example of a reconstructed image (hereinafter referred to as an "ideal reconstructed image") of a general VAE model trained to reconstruct a time-frequency image under ideal conditions in which there is no frequency fluctuation in the narrowband components. FIG. 12(b) illustrates an example of a reconstructed image of a general VAE model trained using the loss function of Equation (8). FIG. 12(c) illustrates an example of a reconstructed image of an SSIM-AE trained using the loss function of Equation (17). FIG. 12(d) illustrates an example of a reconstructed image of a VAE model trained by the learning unit 1 according to the present embodiment trained using the loss function of Equation (31). The horizontal axis of each image represents the time index, the vertical axis represents the frequency index, and the color represents the intensity of the image, with brighter colors representing higher intensity.

[0171] A typical VAE trained using the loss function of Equation (8) reconstructs an image in which the signal is spread over the range of the narrowband component fluctuations, as shown in Figure 12(b). The average intensity of the narrowband component (approximately 1.5) is smaller than the average intensity of the ideally reconstructed image shown in Figure 12(a) (approximately 3.1), resulting in a large reconstruction error around the frequency index where the narrowband component is located. An SSIM-AE trained using the loss function of Equation (17) reconstructs an image with clear narrowband components, as shown in Figure 12(c), but the average intensity of the narrowband component (approximately 3.8) is larger than the ideally reconstructed image shown in Figure 12(a), resulting in a large reconstruction error at the frequency index where the narrowband component is located.

[0172] In contrast, the VAE model of this embodiment, trained using the loss function of Equation (31), reconstructs clear narrowband components as shown in Fig. 12(d), and the average intensity of the narrowband components (approximately 3.2) is closer to the ideal reconstructed image in Fig. 12(a) than the reconstructed image of SSIM-AE in Fig. 12(c). Therefore, the reconstruction error at the frequency index where the narrowband components are located is reduced.

[0173] Next, the operation of the optimum model determination unit 2 shown in Figure 3 will be described. Prior to a detailed description of each block in Figure 3, the pseudo-abnormal information FI and the acoustic sensor time waveform Tw1 of normal data in the optimum model determination unit 2 will be described. As a premise, when an acoustic signal obtained from an acoustic sensor is used for anomaly detection, if the acoustic signal to be monitored is a narrowband signal, the frequency or amplitude of the acoustic signal contained in the normal data does not change significantly, and narrowband components with approximately the same frequency or approximately the same amplitude occur due to fluctuations within the range of normal operation. For this reason, the abnormality patterns expected as abnormal data in anomaly detection based on acoustic signals that monitor narrowband signals can be roughly divided into the following three types. (1) The level of the narrow-band components contained in normal data increases. (2) The narrowband components contained in the normal data disappear. (3) Narrowband or wideband components occur at frequencies different from those of the narrowband components contained in normal data.

[0174] On the other hand, when the signal to be monitored is a wideband signal, the frequency bandwidth or amplitude of the acoustic signal contained in the normal data does not change significantly, and wideband components with approximately the same center frequency, approximately the same amplitude, or approximately the same bandwidth occur due to fluctuations within the range of normal operation.For this reason, the abnormal patterns expected as abnormal data in anomaly detection based on acoustic signals that monitor wideband signals can be broadly divided into the following three types. (4) The levels of some or all of the broadband components contained in the normal data increase. (5) A part of the broadband components contained in the normal data disappears or the level of a part of the broadband components decreases. (6) Narrowband or wideband components occur at frequencies different from the wideband components contained in normal data.

[0175] Furthermore, when the acoustic signal to be monitored is a mixture of narrowband and wideband signals, the abnormal pattern expected as abnormal data will be the mixture described above.

[0176] The pseudo anomaly information FI is composed of an identification number that specifies one of these types of anomaly and parameters used to generate a pseudo anomaly time-frequency image of the anomaly data in the pseudo anomaly generator 202. The parameters used to generate the time-frequency image include a frequency index j that indicates the position of the narrowband component, mask , the width of the region P mask , an arbitrary constant a representing the degree of the anomaly to be added, and a frequency direction index j representing the position of the anomaly to be added. add , the width of the abnormal region to be added P add is.

[0177] Next, we will discuss the acoustic sensor time waveform Tw1 of normal data in the optimal model determination unit 2. The acoustic sensor time waveform Tw1 of normal data in the optimal model determination unit 2 is a plurality of waveforms similar to the acoustic sensor time waveform Twn of normal data in the learning unit 1, but it is data at a different time from the acoustic sensor time waveform Twn of normal data in the learning unit 1, or 2·Q obtained from a time series generated from a different noise species. validation The waveforms are (Q validation is the number of data used to determine the anomaly detection performance, hereafter referred to as the "number of data in the optimal model determination unit 2"). validation is the number of data in learning part 1, Q train However, the number of data points must be sufficient to determine the anomaly detection performance.

[0178] The time-frequency transform unit 201 of the optimal model determination unit 2 outputs the time-frequency image X to the pseudo anomaly generator 202 and calculates the time-frequency image X by 2×Q validation The acoustic sensor time waveform Tw1 of normal data in the optimal model determination unit 2 is input, and 2×Q validation The time-frequency transform unit 611 performs the same operation as the time-frequency transform unit 611 of the prior art, except that it outputs time-frequency images.

[0179] Next, detailed operations of each part of the optimum model determination unit 2 shown in Fig. 3 will be described. The pseudo anomaly generation unit 202 generates the 2 × Qvalidation From the time-frequency images, Q validation Pseudo anomalous time-frequency images X q anom , q=1,…,Q validation Generate the remaining Q validation The time-frequency images are converted into normal time-frequency images X q norm , q=1,…,Q validation The pseudo anomaly generation unit 202 outputs the two types of generated time-frequency images to the background unwhitening input / output unit 203 and the background whitening input / output unit 204. The procedure for generating pseudo anomaly time-frequency images will be described in detail below.

[0180] There are various methods for generating a pseudo-abnormal time-frequency image for a narrowband signal. Here, as an example, a method for adding an abnormality to a partial region of the time-frequency image X based on the time-frequency image X and pseudo-abnormal information FI input from outside will be described.

[0181] First, the input 2×Q validation Let X be the time-frequency image, and Q be the validation Then, divide one of the two Q validation For each of the time-frequency images X, the frequency index j included in the pseudo abnormality information FI is calculated as shown in Equation (39). mask and the width of the region P mask From Area X mask Specify.

number

[0182] where j mask and P mask is set to a value that includes only the narrowband components in each time-frequency image, and P mask = 0. FIG. 13 is a diagram for explaining the region where an anomaly is added. X mask =X[0:N T -1,j mask -P mask :j mask +Pmask ] is the time coordinate of the original image X from 0 to N as shown in Figure 13. T -1, and the coordinate in the frequency direction is j mask -P mask From j mask +P mask Represents the operation of cutting the range up to

[0183] Next, area X mask The abnormality specified by the identification number included in the pseudo-abnormality information FI is embedded using the above. Below, a method for adding an abnormality for each type of abnormality will be explained. When the type of abnormality is "(1) The level of the narrow band component included in the normal data increases", the corresponding area X in the original image X is embedded using an arbitrary constant a as in equation (40). mask Increase the value of .

number

[0184] If the type of anomaly is "(2) the narrowband component contained in the normal data disappears", then the corresponding region X in the original image X' can be calculated by using an arbitrary constant a in the same way as in equation (40). mask However, the arbitrary constant a is set to a value that is judged to be "abnormal if it becomes less than this value" based on statistical data of the amplitude fluctuation of the corresponding frequency component in the normal time-frequency image.

[0185] If the type of anomaly is "(3) Narrowband or wideband components of a different frequency not included in normal data occur" and a narrowband signal is generated as an anomaly, the region X selected to include the narrowband component is mask is used to replace another frequency interval that does not contain narrowband components, as in equation (41).

number

[0186] where j add is an index in the frequency direction on the image that indicates the position where an anomaly is to be added, and is an index of a position outside the frequency range that the narrowband components of normal data can take. Furthermore, the constant a is set to a value that determines that "a value above this value indicates an anomaly" based on statistical data of the amplitude fluctuations of the corresponding frequency component in the time-frequency image of normal data.

[0187] When a wideband signal is generated as an anomaly, P mask The width of the abnormal region to be added is P add Replace with j add -P add :j add +P add For the frequency range specified by a, we set all frequency indices to a·X mask It is generated by replacing

[0188] When generating a pseudo anomalous time-frequency image for a wideband signal, the region X is calculated as in the case of a narrowband signal, as shown in Equation (39). mask where the frequency index j included in the pseudo anomaly information FI in equation (39) is specified. mask is set to a value that indicates the center frequency of the broadband component in the time-frequency image, and the width of the region P mask is set to a value that includes some or all of the broadband components in the time-frequency image.

[0189] If the type of abnormality is "(4) the level of the broadband component contained in the normal data increases", then, as in the case of a narrowband signal, the corresponding area X in the original image X can be calculated using an arbitrary constant a as in equation (40). mask The arbitrary constant a is set to a value that is judged to be "abnormal if it exceeds this value" based on statistical data of the amplitude fluctuation of the corresponding frequency component in the normal time-frequency image.

[0190] If the type of anomaly is "(5) the wideband component contained in the normal data disappears", as in the case of a narrowband signal, the corresponding area X in the original image X' can be calculated by using an arbitrary constant a as in equation (40). mask However, the arbitrary constant a is set to a value that determines that "a value below this value is abnormal" based on statistical data of amplitude fluctuations of the corresponding frequency component in the time-frequency image of normal data.

[0191] When the type of anomaly is "(6) A narrowband component or a wideband component occurs at a frequency different from the wideband component contained in normal data," when a wideband signal is generated as an anomaly, the region X selected to include the wideband component in equation (39) is used, as in the case of a narrowband signal. mask To generate a narrowband signal as an anomaly, the frequency index j is used to replace the interval of another frequency that does not contain the normal wideband component, as in equation (41). mask and the width of the region P mask Set the area X of equation (39) mask is selected and replaced with another frequency interval that does not contain normal wideband components, as in equation (41).

[0192] When both narrowband signals and wideband signals are targeted, a pseudo-abnormal time-frequency image is generated using the above combination. The above method of generating an abnormal signal is merely an example, and any method can be used as long as it can generate a signal that is judged to be abnormal beyond the fluctuation range of a normal signal in a time-frequency image.

[0193] The background unwhitening input / output unit 203 of the optimal model determination unit 2 operates in the same manner as the background unwhitening input / output unit 102 of the learning unit 1, but differs in that two types of time-frequency images (pseudo abnormal time-frequency image / normal time-frequency image) are input to the background unwhitening input / output unit 203. The pseudo abnormality generation unit 202 outputs two types of time-frequency images X q anom / norm , q=1,…,Q validation and a plurality of first weighting coefficient candidates 41w from the first storage unit 4. sNotWhite , s=1,…,S max (S max is the total number of parameter sets included in the patch parameter candidate PPc) and are input. The background unwhitening input / output unit 203 receives two types of time-frequency images X q anom / norm , q=1,…,Q validation For each of these, two normalized time-frequency images X q anom / norm,NotWhite , q=1,…,Q validation and two or more normalized reconstructed images Y q,s anom / norm,(NotWhite)´ , q=1,…,Q validation , s=1,…,S max and output them to the first reconstruction error calculation unit 205. The operation of obtaining two types of normalized time-frequency images and two types of multiple normalized reconstruction images is the same as the operation of the background unwhitening input / output unit 102 of the learning unit 1, except that it corresponds to multiple time-frequency images (pseudo abnormal time-frequency images / normal time-frequency images).

[0194] The background whitening input / output unit 204 of the optimal model determination unit 2 receives two types of time-frequency images X q anom / norm , q=1,…,Q validation and a plurality of second weighting coefficient candidates 51w from the second storage unit 5. s White , s=1,…,S max The background whitening input / output unit 204 receives two types of time-frequency images X q anom / norm , q=1,…,Q validation For each of these, two types of background-whitened time-frequency images X q anom / norm,White , q=1,…,Q validation and two types of background-whitened normalized reconstructed images Y q,s anom / norm,(White)´ , q=1,…,Q validation , s=1,…,S maxand output them to the second reconstruction error calculation unit 206. The operation of obtaining two types of background-whitened time-frequency images and two types of multiple background-whitened normalized reconstruction images is the same as the operation of the background whitening input / output unit 103 of the learning unit 1, except that it corresponds to multiple time-frequency images (pseudo abnormal time-frequency images / normal time-frequency images).

[0195] The operations of the first reconstruction error calculation unit 205 and the second reconstruction error calculation unit 206 of the optimal model determination unit 2 are the same as those of the reconstruction error calculation unit 141 described in the first weight update unit 104 of the learning unit 1, but the number of input time-frequency images is two types (pseudo abnormal time-frequency image / normal time-frequency image) × (number of data Q used to calculate abnormality detection performance). validation ) and the number of reconstructed images is two (pseudo-abnormal data / normal data) × Q validation × (number of multiple weighting coefficient candidates S max ) and the number of output reconstruction errors is 2 types (pseudo abnormal data / normal data) × Q validation ×S max The difference is that they are individuals.

[0196] The first reconstruction error calculation unit 205 of the optimal model determination unit 2 receives two types of normalized time-frequency images X q anom / norm,NotWhite , q=1,…,Q validation and two or more normalized reconstructed images Y q,s anom / norm,(NotWhite)´ , q=1,…,Q validation , s=1,…,S max The first reconstruction error calculation unit 205 receives two types of normalized time-frequency images X q anom / norm,NotWhite , q=1,…,Q validation Multiple reconstruction errors E for two normalized time-frequency images for each rec,(q,s) anom / norm,NotWhite , q=1,…,Q validation , s=1,…,S max and outputs the result to the first threshold value determination unit 207.

[0197] Similarly, the second reconstruction error calculation unit 206 of the optimal model determination unit 2 calculates two types of background whitened time-frequency images X q anom / norm,White , q=1,…,Q validation and two or more background-whitened normalized reconstructed images Y q,s anom / norm,(White)´ , q=1,…,Q validation , s=1,…,S max The second reconstruction error calculation unit 206 receives two types of background-whitened time-frequency images X q anom / norm,White , q=1,…,Q validation Multiple reconstruction errors E for two types of normalized reconstruction images with multiple background whitening rec,(q,s) anom / norm,White , q=1,…,Q validation , s=1,…,S max and outputs the result to the second threshold value determination unit 208.

[0198] The operation of calculating the reconstruction error for two types and a plurality of (normalized time-frequency images / background-whitened time-frequency images) is performed using equation (23) in the same manner as the operation of the reconstruction error calculation unit 141 of the first weight update unit 104 of the learning unit 1, except that it corresponds to two types and a plurality of (normalized / background-whitened) reconstruction images.

[0199] The first threshold determination unit 207 (or the second threshold determination unit 208) of the optimal model determination unit 2 operates in the same manner as the threshold determination unit 625 in the prior art, but differs in the following respects. The first threshold determination unit 207 receives as input a plurality of reconstruction errors for two or more types of standardized reconstructed images (or a plurality of reconstruction errors for two or more types of background-whitened standardized reconstructed images) and an external background non-whitening determination threshold Thn (or a background whitening determination threshold Thw). The first threshold determination unit 207 determines determination results for the plurality of standardized reconstructed images (or determination results for the plurality of background-whitened standardized reconstructed images), and outputs them to the first comparison unit 209 (or the second comparison unit 210).

[0200] Here, the operation of the first threshold value determination unit 207 and the second threshold value determination unit 208 will be described in the case where the anomaly detection performance calculated by the first comparison unit 209 and the second comparison unit 210 is AUC (Area Under the Curve). The first threshold value determination unit 207 first prepares a background unwhitening determination threshold Thn. In the following, the background unwhitening determination threshold Thn is referred to as the background unwhitening determination threshold Th r NotWhite , r=1,…,R threshold It is expressed as R threshold is the number of thresholds, and the decision thresholds are set to be sufficiently fine intervals to represent the ROC (Receiver Operating Characteristic) curve required for calculating the AUC. Then, multiple decision results for the normalized time-frequency image are decided. q,s,r t,NotWhite , q=1,…, Q validation , s=1,…,S max , r=1,…,R threshold Here, the decision result for the multiple normalized reconstructed images is decided. q,s,r t,NotWhite , q=1,…, Q validation , s=1,…,S max , r=1,…,R threshold is determined as shown in equation (42) in a format that combines the first comparing section 209 and the second comparing section 210.

number

[0201] Here, the superscript t stands for "norm" or "anom", and the superscript o stands for "NotWhite" or "White". The threshold decision of Equation (42) is performed as follows: t = norm / anom, q = 1, ..., Q validation , s=1,…,S max , r=1,…,R threshold The above is performed for each of the pseudo-abnormal data and the normal data. Similarly, the second threshold value determination unit 208 prepares a background whitening determination threshold value Thw. In the following, the background whitening determination threshold value Thw is referred to as the background whitening determination threshold value Th r White , r=1,…,Rthreshold Then, the decision result for multiple background whitened normalized reconstructed images is q,s,r t,White , q=1,…,Q validation , s=1,…,S max , r=1,…,R threshold is calculated for each of the pseudo abnormal data and normal data using equation (42).

[0202] The first comparison unit 209 of the optimum model determination unit 2 decides whether or not the first threshold determination unit 207 outputs two or more normalized reconstructed images. q,s,r t,NotWhite , q=1,…, Q validation , s=1,…,S max , r=1,…,R threshold (The superscript t stands for "norm" or "anom"), and multiple anomaly detection performances corresponding to multiple normalized reconstructed images are s NotWhite , s=1,…,S max Calculate the multiple anomaly detection performance b s O is obtained as shown in equation (43) in a format in which the first comparing section 209 and the second comparing section 210 are common to each other.

number

[0203] Here, F() is the specified number of evaluation samples Q validation This is a function that calculates the anomaly detection performance from the judgment results for each of the normal and abnormal data, and any index such as "AUC" or "F1 score" can be used. The superscript o stands for "NotWhite" or "White". The procedure for calculating AUC is explained below.

[0204] First, the background dewhitening threshold Th r NotWhite , r=1,…,R threshold The true positive rate Pr for the r-th threshold s,r TP,NotWhite and the false positive rate Pr s,r FP,NotWhite, s=1,…,S max , r=1,…,R threshold Calculate the true positive rate Pr s,r TP,NotWhite and the false positive rate Pr s,r FP,NotWhite , s=1,…,S max , r=1,…,R threshold is calculated as shown in equations (44) and (45) in a format that combines the first comparing section 209 and the second comparing section 210.

number

number

[0205] Here, the superscript o is either "NotWhite" or "White". The true positive rate of equation (44) is Q validation represents the proportion of pseudo-anomalous data that are judged to be "anomalous." The false positive rate of Equation (45) is Q validation This represents the percentage of normal data that are judged to be "abnormal."

[0206] Figure 14 shows examples of true positive rates and false positive rates for each determination threshold. Figure 14(a) shows an example of the true positive rate for each determination threshold, and Figure 14(b) shows an example of the false positive rate for each determination threshold. The horizontal axis of Figure 14 represents the determination threshold, and the vertical axis represents the true positive rate or false positive rate corresponding to the determination threshold.

[0207] The true positive rate in Figure 14(a) and the false positive rate in Figure 14(b) are connected via the decision threshold. In the example shown, when the decision threshold is set so that the false positive rate is 50%, the true positive rate is 98%, while when the decision threshold is set so that the true positive rate is 50%, the false positive rate is 2%.

[0208] The ROC curve can be drawn by taking the false positive rate on the horizontal axis and the true positive rate on the vertical axis, as shown in Figure 14(c). The AUC is calculated from the area of ​​the region enclosed by the ROC curve, the line representing the true positive rate = 0%, and the line representing the false positive rate = 100% (the hatched area in Figure 14(c)). This value is used to calculate the anomaly detection performance b corresponding to multiple normalized reconstructed images. s Notwhite , s=1,…,S max The value of the multiple anomaly detection performance b corresponding to multiple normalized reconstructed images is s Notwhite , s=1,…,S max Arrange the above to create a list of anomaly detection performance for normalized time-frequency images. performance NotWhite Anomaly detection performance list performance NotWhite is obtained by arranging the first comparing section 209 and the second comparing section 210 in a common format as shown in equation (46).

number

[0209] Next, as shown in Equation (47) where the first comparison unit 209 and the second comparison unit 210 are used in common, the index s^ corresponding to the patch parameter candidate PPc that maximizes the anomaly detection performance is calculated. NotWhite The symbol "^" next to s is a hat indicating an estimated value, and is written above s as shown in equation (47).

number

[0210] Next, as in equation (48) in which the first comparing unit 209 and the second comparing unit 210 are common, the storage area list of the first storage unit 4 defined in equation (18) is model NotWhite From the index s^ NotWhite The optimal learned first weight coefficient w^ is calculated using NotWhite The symbol "^" next to w is a hat indicating an estimated value, and is written above w as shown in equation (48).

number

[0211] In formulas (46) to (48), the superscript o is either "NotWhite" or "White". model o [s^ o ] is s^ o Represents the operation to extract the th element.

[0212] The optimal first weighting factor w^ selected in equation (48) NotWhite is output as the first weighting coefficient 42 to the first storage unit 4. Similarly, the second comparison unit 210 determines the plurality of determination results for the two types and plurality of background-whitened normalized reconstructed images output from the second threshold determination unit 208. q,s,r t,White , q=1,…, Q validation , s=1,…,S max , r=1,…,R threshold Based on this, the anomaly detection performance b s white , s=1,…,S max and obtain the optimal learned second weighting coefficient w^ from equations (44) to (48). NotWhite is selected and output as the second weighting coefficient 52 to the second storage unit 5. This completes the description of the operation of the optimum model determination unit 2 shown in FIG.

[0213] Next, the configuration of the anomaly determination unit 3 shown in Fig. 4 will be described. The acoustic sensor time waveform Twe of the evaluation data is input to the time-frequency conversion unit 301 of the anomaly determination unit 3. The time-frequency conversion unit 301 converts the acoustic sensor time waveform Twe of the evaluation data into a time-frequency image, and outputs the result to the background unwhitening input / output unit 302 and the background whitening input / output unit 303. The operation of the time-frequency conversion unit 301 is the same as that of the time-frequency conversion unit 611 of the prior art, so a description thereof will be omitted. The number of time-frequency images output from the time-frequency conversion unit 301 of the anomaly determination unit 3 is one.

[0214] The operation of the background unwhitening input / output unit 302 of the anomaly determination unit 3 is similar to the operation of the background unwhitening input / output unit 102 of the learning unit 1, except that a time-frequency image corresponding to the evaluation data is input to the background unwhitening input / output unit 302. The time-frequency image X eval and the first weighting coefficient 42w^ from the first storage unit 4. NotWhite The background unwhitening input / output unit 302 receives the time-frequency image X eval to the normalized time-frequency image X eval,NotWhite and generate a normalized time-frequency image X eval,NotWhite Normalized reconstructed image Y eval,(NotWhite)´ and output these to the first reconstruction error calculation unit 304. The normalized time-frequency image X eval,NotWhite and the normalized reconstructed image Y eval,(NotWhite)´ The operation of obtaining is the same as the operation of the background unwhitening input / output unit 102 of the learning unit 1, except that it corresponds to a time-frequency image of one evaluation data.

[0215] The operation of the background whitening input / output unit 303 of the abnormality determination unit 3 is the same as the operation of the background whitening input / output unit 103 of the learning unit 1, except that a time-frequency image corresponding to the evaluation data is input. The background whitening input / output unit 303 receives the time-frequency image X eval and the second weighting coefficient 52w^ from the second storage unit 5. White The background whitening input / output unit 303 receives the time-frequency image X eval background-whitened time-frequency image X eval,White and generate a background-whitened time-frequency image X eval,White background whitened normalized reconstructed image Y eval,(White)´ and output these to the second reconstruction error calculation unit 305. The background-whitened time-frequency image X eval,White and the background-whitened normalized reconstructed image Y of the evaluation data eval,(White)´ The operation of obtaining is the same as the operation of the background whitening input / output unit 103 of the learning unit 1, except that it corresponds to a time-frequency image of one evaluation data.

[0216] The first reconstruction error calculation unit 304 of the abnormality determination unit 3 receives the normalized time-frequency image X eval,NotWhite and the normalized reconstructed image Y eval,(NotWhite)´ The first reconstruction error calculation unit 304 calculates the reconstruction error E rec eval,NotWhite is calculated and output to the first threshold decision unit 306. The reconstruction error E rec eval,NotWhite In the operation of calculating the error, equation (7) is used, as in the reconstruction error calculation unit 614 of the prior art.

[0217] The second reconstruction error calculation unit 305 of the abnormality determination unit 3 receives the background whitened time-frequency image X eval,White and the background whitened normalized reconstructed image Y eval,(White)´ The second reconstruction error calculation unit 305 calculates the reconstruction error E rec eval,White is calculated and output to the second threshold decision unit 307. The reconstruction error E rec eval,White The operation of calculating the error is performed using equation (7) in the same manner as in the reconstruction error calculation unit 614 of the prior art.

[0218] The operation of the first threshold determination unit 306 (or the second threshold determination unit 307) of the abnormality determination unit 3 is similar to the operation of the threshold determination unit 625 of the prior art, except for the following differences. The first threshold determination unit 306 (or the second threshold determination unit 307) receives as input the reconstruction error for the normalized time-frequency image (or the reconstruction error for the background-whitened time-frequency image) from the first reconstruction error calculation unit 304 (or the second reconstruction error calculation unit 305) and an external background non-whitening determination threshold Thn (or background whitening determination threshold Thw). The first threshold determination unit 306 (or the second threshold determination unit 307) determines a determination result for the normalized time-frequency image of the evaluation data (or a determination result for the background-whitened time-frequency image of the evaluation data), and outputs the determination result to the determination result integration unit 308.

[0219] The first threshold decision unit 306 decides the decision result for the normalized time-frequency image of the evaluation data. NotWhite Here, the evaluation data decision result decide o is determined as shown in equation (49) in a format that combines the first threshold value determination unit 306 and the second threshold value determination unit 307.

number

[0220] Here, the superscript o stands for "NotWhite" or "White." E rec eval,o is the reconstruction error for the normalized time frequency or the reconstruction error for the background-whitened time frequency image, and Th o is the background non-whitening determination threshold Thn or the background whitening determination threshold Thw. Similarly, the second threshold determination unit 307 determines the determination result for the background whitening time-frequency image of the evaluation data by White is determined using equation (49).

[0221] The judgment result integration unit 308 of the abnormality judgment unit 3 receives the judgment result for the normalized time-frequency image from the first threshold judgment unit 306 and the judgment result for the background-whitened time-frequency image from the second threshold judgment unit 307. The judgment result integration unit 308 determines a final judgment result by taking the logical sum of the judgment results of each judgment unit as shown in equation (50). final Output.

number

[0222] Here, the symbol + inside a circle represents a logical sum operation. Equation (50) means that if the acoustic sensor time waveform Twe of the evaluation data is determined to be "1" (abnormal) by either the first threshold value determination unit 306 corresponding to the normalized time-frequency image or the second threshold value determination unit 307 corresponding to the background-whitened time-frequency image, "1" (abnormal) is output. This concludes the description of the operation of the abnormality determination unit 3 shown in FIG. 4.

[0223] The effects of the anomaly detection device 100 of this embodiment will be described. First, in the first weight update unit 104 and the second weight update unit 105 of the learning unit 1 of this embodiment, the reconstruction error E rec and the average SSIM value E SSIM This makes it possible to generate a reconstructed image that has a response at the frequency position where the narrowband component of the evaluation data exists, even if the frequency position of the narrowband component contained in the time-frequency image of the evaluation data is slightly shifted from the frequency position of the narrowband component contained in the time-frequency image of the training data, and to make the intensity of the narrowband component of the reconstructed image closer to that of the time-frequency image of the evaluation data. In other words, the reconstruction performance of normal data can be improved.

[0224] As a result, even if the evaluation data is normal and the frequency position of the narrowband component contained in the evaluation data deviates within the normal operation variation range, the VAE model operates to reconstruct the frequency position of the narrowband component contained in the evaluation data as a result of learning by adding the average SSIM value to the loss function, and further operates to reconstruct the intensity of the narrowband component contained in the evaluation data as a result of adding the reconstruction error to the loss function, thereby reducing the reconstruction error and preventing an increase in cases where normal signals are judged to be abnormal. Furthermore, since the reconstruction error is included in the loss function, learning progresses to reduce the reconstruction error, making it possible to keep the judgment threshold low and avoiding the problem of deterioration in anomaly detection performance.

[0225] Furthermore, in the anomaly detection device 100 of this embodiment, the optimal model determination unit 2 specifies an abnormality pattern of a system to which this anomaly detection method is applied as pseudo-abnormality information FI, generates abnormal data by creating an abnormality pattern based on the pseudo-abnormality information FI as pseudo-abnormal data from normal data that can be acquired in large quantities, and uses this normal data and abnormal data to select an optimal weighting factor from multiple weighting factor candidates based on multiple patch parameter candidates. This makes it possible to determine an optimal weighting factor for the abnormality pattern specified by the pseudo-abnormality information FI.

[0226] Furthermore, in the anomaly detection device 100 of this embodiment, the pre-processing unit 121 of the background unwhitening input / output unit 102 and the background whitening unit 131 of the background whitening input / output unit 103 in the learning unit 1 perform pre-processing to normalize the time-frequency image X to be input to the VAE model so that it becomes a non-negative value, and the post-processing unit 123 of the background unwhitening input / output unit 102 and the post-processing unit 133 of the background whitening input / output unit 103 perform post-processing to normalize the reconstructed image output by the VAE model so that it also becomes a non-negative value. As a result, the SSIM calculation unit 142 of the first weight update unit 104 and the second weight update unit 105 in the learning unit 1 calculates a plurality of SSIM values ​​SSIM q,s,p o (The superscript o stands for "NotWhite" or "White") The input and output images used in the calculation are always output as non-negative values.

[0227] As a result, a plurality of SSIM values ​​SSIM for the plurality of first weighting coefficient candidates 41 or the plurality of second weighting coefficient candidates 51 are obtained. q,s,p o Furthermore, the frequency of multiple SSIM values ​​being negative is reduced, so that the average SSIM value E SSIM,(s,p) o This can prevent the loss function value from becoming 0, and it is possible to avoid the problem that the reconstruction does not progress any further because the loss function value is already small even though learning has not progressed, and the reconstruction accuracy does not improve.

[0228] Furthermore, in the anomaly detection device 100 of this embodiment, in the background whitening process of the background whitening unit 131 of the learning unit 1, the noise amplitude spectrum is estimated from the frequency spectrum at each time in the time-frequency image X and normalized to white. This allows learning to be performed so that the noise amplitude spectrum of the reconstructed image Y, which is the output of the reconstruction unit 122, is reconstructed as a flat characteristic. As a result, even in cases where the signal to be monitored is a narrowband signal and the shape of the noise amplitude spectrum is not flat, the reconstruction error E rec White In this case, the error variance becomes constant across all frequencies, which has the effect of improving the anomaly detection performance, and it also becomes easier to set the background whitening determination threshold value Thw to be input to the second threshold value determination unit 307.

[0229] On the other hand, the reconstruction error E rec NotWhite However, because the error variance varies depending on the frequency, the judgment threshold must be increased in order to keep the number of erroneous judgments to a certain level, which results in a deterioration of anomaly detection performance. However, because the judgment result integration unit 308 takes the logical sum of both results, the integrated judgment result has the effect of improving anomaly detection performance on the background whitening input / output unit 303 side.

[0230] Furthermore, the background whitening process in the background whitening unit 131 flattens the frequency characteristics of non-narrowband signals, such as wideband signals that have a wide spread in the frequency direction, thereby eliminating the signals. In contrast, the background unwhitening input / output unit 102 effectively generates normalized reconstruction images for these wideband signals, enabling anomaly detection for the wideband signals based on the reconstruction error. The anomaly detection unit 3 of this embodiment uses the first weighting coefficient 42 learned using the normalized time-frequency image and the second weighting coefficient 52 learned using the background-whitened time-frequency image to independently determine anomalies in the first threshold determination unit 306 and the second threshold determination unit 307, and then integrates the two determination results in the determination result integration unit 308. This effectively enables anomaly detection for signals containing both wideband and narrowband signals.

[0231] The above is a description of the embodiments of the present invention, but the present invention is not limited to the configurations of the above embodiments and various modifications and combinations are possible within the scope of the technical concept. For example, the above embodiments have been described using a VAE as a deep learning model for anomaly detection, but other autoencoder models may also be used. In particular, other derivative models of VAE may be used as long as they can learn using only normal data and handle structural similarity (SSIM) in a form in which an image is input and reconstructed.

[0232] Furthermore, in the above embodiment, the pre-processing unit 121 of the background dewhitening input / output unit 102 normalizes the input image to a non-negative value by standardizing the input image as shown in Equation (5), shifting the average value of the entire image by 3σ as shown in Equation (20), and then clipping negative values ​​using a ReLU function. However, other methods may be used for normalization as long as all values ​​of the input image are limited to non-negative values. For example, the maximum and minimum values ​​of the input image may be obtained and compressed to a range of 0 to 1. Alternatively, the variance value of the standardized image, "1," may be set to the value of 3σ, and the variance may be reduced to "1 / 3," after which the average may be shifted to 0.5, and then the ReLU function may be applied to normalize to a non-negative value.

[0233] Furthermore, in the above embodiment, the background whitening unit 131 has been described as using noise normalization processing by LOFAR as the background whitening processing method. However, other background whitening methods may be used as long as they can whiten the background noise amplitude spectrum while sufficiently preserving the amplitude spectrum of the narrowband signal.

[0234] In the above embodiment, the post-processing unit 123 (or the post-processing unit 133) in the background unwhitening input / output unit 102 (or the background whitening input / output unit 103) has been described as a means for obtaining a normalized reconstructed image (or a background whitened normalized reconstructed image) by shifting the average value by "3" (or "1") as in equation (22) (or equation (38)) and then clipping negative values ​​using the ReLU function. However, normalization may be performed by other means as long as all values ​​of the reconstructed image are limited to non-negative values. However, the reconstruction error E calculated by equation (23) rec In order to prevent bias error, it is necessary to match the mean value of the intensity distribution with the normalized time-frequency image obtained by the pre-processing unit 121 (or background whitening unit 131) applied to the time-frequency image.

[0235] In the above embodiment, the pseudo anomaly generator 202 generates a pseudo anomaly data set by using a region X mask Although the method of adding an abnormality based on the pseudo abnormality information FI has been described, other means may be used as long as they generate abnormality data suitable for the application to which the abnormality is applied.

[0236] Furthermore, in the above embodiment, the pseudo-anomaly information FI has one identification number for identifying the type of anomaly, and a case has been described in which one type of anomaly is handled. However, when handling multiple types of anomalies, an anomaly detection device 100 that can handle multiple types of anomalies can be configured by arranging multiple configurations of this embodiment in parallel and assigning different identification numbers to each piece of pseudo-anomaly information FI.

[0237] Fig. 15 is a schematic configuration diagram of an anomaly detection device 100A according to a modified example. For example, if the number of expected types of anomalies is v, v sets of first weighting coefficient candidates 41-1 to 41-v and v sets of second weighting coefficient candidates 51-1 to 51-v are prepared as shown in Fig. 15. A learning unit 1 learns each of the v sets of first weighting coefficient candidates 41-1 to 41-v and the v sets of second weighting coefficient candidates 51-1 to 51-v.

[0238] The optimal model determination unit 2 uses pseudo anomaly information FI-1 to FI-v, which stores a different identification number for each type of anomaly, to select the first weighting coefficient candidate that provides the best anomaly detection performance from each of v sets of multiple first weighting coefficient candidates 41-1 to 41-v, and stores these in the first memory unit 4 as v sets of first weighting coefficients 42-1 to 42-v.

[0239] Similarly, the second weighting coefficient candidate that provides the best anomaly detection performance is selected from each of the v sets of second weighting coefficient candidates 51-1 to 51-v, and stored in the second storage unit 5 as the v sets of second weighting coefficients 52-1 to 52-v.

[0240] The abnormality detection device 100A of this modified example has v abnormality judgment units 3-1 to 3-v, and each of the v abnormality judgment units 3-1 to 3-v outputs an abnormality judgment result using first weighting coefficients 42-1 to 42-v and second weighting coefficients 52-1 to 52-v stored in the first memory unit 4 and the second memory unit 5.

[0241] Furthermore, in the above embodiment, the anomaly determination unit 3 is described as being configured in parallel with a background unwhitening processing system consisting of the background unwhitening input / output unit 302, the first reconstruction error calculation unit 304, and the first threshold determination unit 306, and a background whitening processing system consisting of the background whitening input / output unit 303, the second reconstruction error calculation unit 305, and the second threshold determination unit 307. However, the anomaly detection device 100 can also be configured using a separate background unwhitening processing system and a separate background whitening processing system.

[0242] Furthermore, in the above embodiment, the anomaly detection device 100 is configured to include the learning unit 1. However, the learning unit 1 may be configured as an independent learning device, and the learning unit 1 may be omitted from the anomaly detection device 100. In this case, the anomaly detection device 100 is configured with an optimal model determination unit 2 and an anomaly judgment unit 3, and performs anomaly judgment using a plurality of first weighting coefficient candidates 41 and second weighting coefficient candidates 51 of a VAE model trained by a learning device having the functions of the learning unit 1. Furthermore, the optimal model determination unit 2 of the anomaly detection device 100 may be configured as a separate independent device, and the optimal model determination unit 2 may be omitted from the anomaly detection device 100. In this case, the anomaly detection device 100 is configured with the anomaly judgment unit 3, and performs anomaly judgment using the first weighting coefficient 42 and the second weighting coefficient 52 of the trained VAE model determined by the optimal model determination unit 2. [Explanation of symbols]

[0243] 1 Learning unit, 2 Optimal model determination unit, 3, 3-1 to 3-v Abnormality determination unit, 4 First memory unit, 5 Second memory unit, 41, 41-1 to 41-v First weighting coefficient candidate, 42, 42-1 to 42-v First weighting coefficient, 51, 51-1 to 51-v Second weighting coefficient candidate, 52, 52-1 to 52-v Second weighting coefficient, 61 Learning unit, 62 Abnormality determination unit, 63 Memory unit, 71 Learning unit, 81 First input terminal, 82 Second input terminal, 83 Output terminal, 84 Third input terminal, 85 Fourth input terminal, 86 Output terminal, 87 Fifth input terminal, 88 Input / output terminal, 100, 100A Abnormality detection device, 101 Time-frequency conversion unit, 102 Background unwhitening input / output unit, 103 Background whitening input / output unit, 104 First weight update unit, 105 Second weight update unit, 121 pre-processing unit, 122 reconstruction unit, 123 post-processing unit, 131 background whitening unit, 132 reconstruction unit, 133 post-processing unit, 141 reconstruction error calculation unit, 142 SSIM calculation unit, 143 loss function calculation unit, 144 error backpropagation unit, 201 time-frequency conversion unit, 202 pseudo anomaly generation unit, 203 background unwhitening input / output unit, 204 background whitening input / output unit, 205 first reconstruction error calculation unit, 206 second reconstruction error calculation unit, 207 first threshold determination unit, 208 second threshold determination unit, 209 first comparison unit, 210 second comparison unit, 301 time-frequency conversion unit, 302 background unwhitening input / output unit, 303 background whitening input / output unit, 304 first reconstruction error calculation unit, 305 second reconstruction error calculation unit, 306 first threshold determination unit, 307 Second threshold judgment unit, 308 judgment result integration unit, 600 anomaly detection device, 611 time-frequency conversion unit, 612 standardization unit, 613 reconstruction unit, 614 reconstruction error calculation unit, 615 loss function calculation unit, 616 error back propagation unit, 621 time-frequency conversion unit, 622 standardization unit, 623 reconstruction unit, 624 reconstruction error calculation unit, 625 threshold judgment unit, 630 weighting coefficient, 700 anomaly detection device, 711 time-frequency conversion unit, 712 standardization unit, 713 reconstruction unit, 714 SSIM calculation unit, 715 loss function calculation unit, 716 error back propagation unit.

Claims

1. a time-frequency conversion unit that converts the sensor time waveform of normal data into a time-frequency image; a pre-processing unit that normalizes the time-frequency image to a non-negative value to generate a normalized time-frequency image; a reconstruction unit that inputs the normalized time-frequency image into an autoencoder model and generates a reconstructed image; a post-processing unit that normalizes the reconstructed image to a non-negative value to generate a normalized reconstructed image; a first weight update unit that updates a first weight coefficient of the autoencoder model based on the normalized time-frequency image and the normalized reconstructed image; A learning device comprising:

2. a background unwhitening input / output unit having the pre-processing unit, the reconstruction unit, and the post-processing unit; a background whitening input / output unit; a second weight update unit; The background whitening input / output unit a background whitening unit that performs background whitening processing on the time-frequency image to flatten a noise amplitude spectrum and generates a background-whitened time-frequency image; a reconstructor that inputs the background-whitened time-frequency image to the autoencoder model and generates a background-whitened reconstructed image; a post-processing unit that normalizes the reconstructed image to a non-negative value to generate a background-whitened normalized reconstructed image; The learning device according to claim 1 , wherein the second weight update unit updates a second weight coefficient of the autoencoder model based on the background-whitened time-frequency image and the background-whitened normalized reconstructed image.

3. The first weight update unit an SSIM calculation unit that calculates an average SSIM value using the normalized time-frequency image, the normalized reconstructed image, and patch parameter candidates; a reconstruction error calculation unit that calculates a reconstruction error, which is an error between the normalized time-frequency image and the normalized reconstructed image; a loss function calculation unit that calculates a loss function using the reconstruction error and the average SSIM value; an error backpropagation unit that updates the first weighting coefficient using the loss function; The learning device according to claim 1 , comprising:

4. the SSIM calculation unit calculates a plurality of average SSIM values ​​using a plurality of the patch parameter candidates; the first weight update unit updates the first weight coefficients corresponding to the patch parameter candidates; The learning device according to claim 3 , wherein the number of the plurality of first weighting coefficients is the same as the number of the plurality of patch parameter candidates.

5. The second weight update unit an SSIM calculation unit that calculates an average SSIM value using the background-whitened time-frequency image, the background-whitened normalized reconstructed image, and patch parameter candidates; a reconstruction error calculation unit that calculates a reconstruction error, which is an error between the background-whitened time-frequency image and the background-whitened normalized reconstruction image; a loss function calculation unit that calculates a loss function using the reconstruction error and the average SSIM value; an error backpropagation unit that updates the second weighting coefficient using the loss function; 3. The learning device according to claim 2, comprising:

6. the SSIM calculation unit calculates a plurality of average SSIM values ​​using a plurality of the patch parameter candidates; the second weight update unit updates the second weight coefficients corresponding to the patch parameter candidates; The learning device according to claim 5 , wherein the number of the plurality of second weighting coefficients is the same as the number of the plurality of patch parameter candidates.

7. an abnormality determination unit that determines whether the evaluation data is normal or abnormal; The abnormality determination unit a time-frequency conversion unit that converts a sensor time waveform of the evaluation data into a time-frequency image; a pre-processing unit that normalizes the time-frequency image to a non-negative value to generate a normalized time-frequency image; a reconstruction unit that inputs the normalized time-frequency image into a trained autoencoder model and generates a reconstructed image; a post-processing unit that normalizes the reconstructed image to a non-negative value to generate a normalized reconstructed image, An anomaly detection device that determines whether the evaluation data is normal or abnormal using the normalized time-frequency image and the normalized reconstructed image.

8. an optimal model determination unit that determines a weighting coefficient to be used in the abnormality determination unit from a plurality of weighting coefficient candidates of the trained autoencoder model, The optimal model determination unit a time-frequency conversion unit that converts the sensor time waveform of normal data into a normal time-frequency image; a pseudo-abnormality generating unit that generates a pseudo-abnormal time-frequency image, which is a time-frequency image of pseudo-abnormal data, using the normal time-frequency image and the pseudo-abnormal information; a comparison unit that calculates anomaly detection performances of a plurality of weighting factor candidates based on the normal time-frequency image and the pseudo-abnormal time-frequency image, and determines, as the weighting factor, the weighting factor candidate that maximizes the anomaly detection performance from among the plurality of weighting factor candidates.

9. the plurality of weighting factor candidates include a plurality of first weighting factor candidates and a plurality of second weighting factor candidates; the optimal model determination unit determines an optimal first weighting factor from among the plurality of first weighting factor candidates, and determines an optimal second weighting factor from among the plurality of second weighting factor candidates; The abnormality determination unit 9. The anomaly detection device according to claim 8, wherein the logical sum of the judgment result of the evaluation data using the determined first weighting coefficient and the judgment result of the evaluation data using the determined second weighting coefficient is output as the final judgment result.

10. The abnormality determination unit a background unwhitening input / output unit having the pre-processing unit, the reconstruction unit, and the post-processing unit; a background whitening input / output unit; The background whitening input / output unit a background whitening unit that performs background whitening processing on the time-frequency image to flatten a noise amplitude spectrum and generates a background-whitened time-frequency image; a reconstruction unit that inputs the background-whitened time-frequency image to the trained autoencoder model and generates a background-whitened reconstructed image; The anomaly detection device according to claim 9 , further comprising a post-processing unit that normalizes the reconstructed image to a non-negative value to generate a background-whitened normalized reconstructed image.

11. a learning unit configured to learn the autoencoder model; The learning unit a time-frequency conversion unit that converts the sensor time waveform of normal data into a time-frequency image; a pre-processing unit that normalizes the time-frequency image to a non-negative value to generate a normalized time-frequency image; a reconstruction unit that inputs the normalized time-frequency image to the autoencoder model and generates a reconstructed image; a post-processing unit that normalizes the reconstructed image to a non-negative value to generate a normalized reconstructed image; The anomaly detection device according to any one of claims 7 to 10, further comprising: a weight update unit that updates the plurality of weight coefficient candidates of the autoencoder model based on the normalized time-frequency image and the normalized reconstructed image.

Images