Trained autoencoder, trained autoencoder generation method, non-steady-state sound detection method, non-steady-state sound detection device, and computer program

Abstract

Provided is a trained artificial intelligence for detecting the non-steady-state nature of an object, and which functions accurately even in the presence of ambient sound.　In the present invention, an autoencoder is given input of steady-state sound feature data, which is generated from steady-state sound data and which is data about features of steady-state sound specified by said steady-state sound data, which is data about steady-state sound that includes sound produced from an object at steady-state times, and the autoencoder is made to output estimated steady-state sound feature data. The loss between the steady-state sound feature data and the estimated steady-state sound feature data is generated, and the autoencoder is trained to minimize the differences between the two. A loss function for obtaining the loss is modified with a denoising function that removes noise from the differences and a trend extraction function, which is a function that extracts the trend of the differences over a given time period.

Description

Trained autoencoder, trained autoencoder generation method, non-stationary sound detection method, non-stationary sound detection device, and computer program

[0001] The present invention mainly relates to a technique for detecting non-stationarity of an object based on the sound generated by the object.

[0002] As will be explained in detail later, the present invention detects the non-stationarity of an object based on sound, whereas most of the prior art detects the abnormality of an object based on sound or determines whether the object is normal or abnormal.

[0003] We will discuss later how determining the non-steady state of an object (whether the object is in a non-steady state) differs from determining the abnormality of an object (or whether the object is normal or abnormal), and will now explain the prior art in order. For example, let's assume the object is a machine. The idea of ​​detecting machine malfunctions, or malfunctions in structures, equipment, etc., by sound has existed for a long time as a non-destructive testing method and can be considered classic. It is well known that machines that are malfunctioning or about to malfunction will emit abnormal noises, and it is a common technique to detect machine malfunctions or impending malfunctions by using such abnormal noises. It is also well known that when an unfavorable condition occurs in a structure, for example, when a crack appears in a wall, abnormal noises will be mixed in with the sounds produced by a hammering test. It is also common knowledge that the generation of such abnormal noises indicates that a malfunction has occurred or is about to occur in the structure. In the past, such abnormal noise detection was performed by skilled personnel.

[0004] With the remarkable development of artificial intelligence in recent years, there has naturally been a movement to replace skilled humans who can distinguish between machine malfunctions and abnormal sounds that occur during tapping tests with artificial intelligence. For example, if the target object is a machine, a large amount of normal sound data, which is the sound of the machine in a normal state, is sampled, along with a large amount of abnormal sound data, which is the sound of the machine in a state where a malfunction has occurred or will occur in the near future. Both the normal and abnormal sound data are labeled as normal or abnormal sounds, and the AI ​​is then trained to learn them. Then, when the trained AI receives sound data from the target object (i.e., the machine being detected or determined to be in a normal or abnormal state), it determines whether the sound identified by the sound data is normal or abnormal. Of course, if the AI ​​determines that the sound is an abnormal sound, it can then determine whether the machine is malfunctioning or will malfunction in the near future. This determination can be divided into more than two stages, depending on the type of labeling.

[0005] The technology for detecting abnormal sounds using artificial intelligence as described above can be said to be a simple replacement of skilled human abilities with artificial intelligence. The artificial intelligence determines whether a sound identified by input sound data is normal or abnormal based on whether the input sound data is closer to already-learned normal sound data or abnormal sound data.

[0006] As mentioned above, the above-mentioned artificial intelligence requires a large amount of normal and abnormal sound data for its training, which means that it takes a lot of effort and cost to create a trained artificial intelligence. If the object to be judged is a machine, after confirming that the machine is normal, a large amount of normal sound data, which is the sound produced by the machine in a normal state, must be sampled, and after confirming that the machine is abnormal, a large amount of abnormal sound data must also be sampled. If the abnormality level is divided into multiple levels, for example, from mild to severe, rather than just two levels, normal and abnormal, the amount of data that must be sampled increases even more. In addition, the sampled sound data must be correctly labeled in order for the artificial intelligence to learn. This also increases the effort and cost. However, these efforts and costs are unavoidable in order to obtain the above-mentioned trained artificial intelligence.

[0007] To address the need to reduce the effort and cost required to obtain such trained AI, the inventors of the present application have already invented and filed a patent application based on the following concept. This invention will be described below, but before that, the definition of the term "non-stationarity" as used herein will be explained. In this application, "non-stationarity (or non-stationary state)" refers to a state in which sounds other than those generated by the object when pre-training was performed, as described below. In other words, in this invention, sounds generated in a stationary state are trained by an autoencoder, as described below. From the perspective of an anthropomorphized autoencoder, a stationary state is a state in which only sounds that have been learned are generated (or input), while a non-stationary state is a state in which sounds that have not been learned are generated.

[0008] For example, let's say the target object is a machine. If sounds (including ambient sounds, i.e., environmental sounds that can be called noise) emitted by the machine when it is operating normally are considered to be sounds generated in a steady state, and if the machine is experiencing a breakdown or other malfunction (or is in a state where a malfunction will occur in the near future) and the machine is emitting an abnormal sound, then the state in which sounds including the abnormal sound are being generated is a non-steady state. In other words, in this example, if the non-steadiness of the target object can be detected, it can be detected that the machine is experiencing a malfunction or that a malfunction will occur in the near future.

[0009] In other words, a trained AI can be obtained by machine learning data on steady sounds, which are sounds generated by an object in a steady state. Then, by inputting measurement sound data, which is sound necessary for determining the state of an object, into such a trained AI, it becomes possible to detect the non-stationarity of the object. When adopting this concept, the only data required to obtain the trained AI is steady sound data, including sounds emitted when the object is in a steady state, and labeling is not required. Therefore, this concept significantly reduces the effort and cost required to obtain the trained AI. However, in order for a trained AI trained only using steady sound data to correctly detect non-stationary states, a different method is required than the one used in the AI ​​described above that trains using both steady and non-stationary sounds.

[0010] The present inventors utilized an autoencoder as a different approach. The trained autoencoder, which the present inventors have already developed and applied for a patent for, is created from an autoencoder that encodes input data, which is predetermined data, and then decodes the encoded data to restore data of the same dimensions as the input data. Such autoencoders are well known in the field of artificial intelligence. The trained autoencoder developed by the present inventors is generated based on such a pre-trained autoencoder. The input data is steady-state sound feature data, which is data on steady-state sound features identified from steady-state sound data, which is data for a specific duration of steady-state sound including sounds generated in a steady state from an object whose non-stationarity is to be detected based on the sound. The output data is estimated steady-state sound feature data, and the trained autoencoder is obtained by inputting a plurality of steady-state sound feature data and pre-training the trained autoencoder so as to minimize loss in the difference between the steady-state sound feature data as input data and the estimated steady-state sound feature data as output data for a specific duration of the input data.

[0011] The trained autoencoder described above is incorporated into a non-stationary sound detection device as follows: The non-stationary sound detection device includes a first recording unit that records the trained autoencoder described above, a receiving unit that receives measurement sound data that is data for a specific time length about measurement sound including sound generated from an object for which sound-based non-stationarity detection is being performed, a measurement sound feature data generation unit that generates, from the measurement sound data received by the receiving unit, measurement sound feature data that is data about the feature amounts of the measurement sound identified by the measurement sound data in the same manner as the stationary sound feature data was generated from the stationary sound data in pre-learning, and a unit that reads the trained autoencoder recorded in the first recording unit, inputs the measurement sound feature data generated by the measurement sound feature data generation unit into the trained autoencoder, and performs a process of generating measurement sound feature data. and a second calculation unit that calculates a loss for a difference between the measurement sound feature data generated by the measurement sound feature data generation unit and estimated measurement sound feature data generated from the measurement sound feature data generated by the first calculation unit, using the same loss function as used when training the trained autoencoder, and that, if the generated data for the loss satisfies a predetermined condition, determines that the measurement sound identified by the measurement sound data that is the source of the measurement sound feature data is a non-stationary sound and generates result data indicating that a non-stationary sound has occurred.

[0012] International Publication No. 2022 / 064590

[0013] The above-described unsteady sound detection device using an autoencoder can detect the occurrence of unsteady sounds or the presence of an unsteady state in an object, even though it uses only an artificial intelligence (autoencoder) trained only on steady sounds. This reduces the effort and cost required to develop an artificial intelligence capable of detecting the unsteadiness of an object. However, when attempting to detect the unsteadiness of a certain type of device based on the sound it emits, the above-described unsteady sound detection device including an autoencoder can produce somewhat inaccurate results. The machine in question is a machine having a rotating mechanism. A rotating mechanism is a mechanism that rotates around an axis. A typical example of a machine having a rotating mechanism is a motor. Therefore, it has been difficult to accurately detect the unsteadiness of motor-equipped machines, such as propeller drives in wind power generation facilities, motor drives in other power plants, and rotary drives in manufacturing plants such as glass factories, using the above-described unsteady sound detection device including an autoencoder.

[0014] The objective of the present invention is to improve a non-stationary sound detection device that includes an autoencoder trained only on stationary sounds so that it can detect non-stationarity based on sounds generated by an object that is a machine with a rotating mechanism with greater accuracy than before.

[0015] The present inventors propose the following trained autoencoder as one aspect of the present invention. This trained autoencoder forms the core of the artificial intelligence in the present invention. The trained autoencoder in the present invention is created from an autoencoder that encodes input data, which is predetermined data, and then decodes the encoded data to restore it to data of the same dimension as the input data. Such autoencoders are well known or well-known in the field of artificial intelligence, and although they have been used to some extent in training artificial intelligence with training data, they are widely used in training artificial intelligence without training data. The trained autoencoder of the present invention is generated based on such an autoencoder, and is a trained autoencoder obtained by inputting steady-state sound feature data, which is data on the features of steady-state sound identified by steady-state sound data generated from steady-state sound data of a specific time length, the steady-state sound being data including sounds generated in a steady state from an object that is a machine including a rotating mechanism, and by pre-training the autoencoder so that the loss is minimized, by inputting a plurality of steady-state sound feature data and calculating the loss between the steady-state sound feature data that is the input data and the estimated steady-state sound feature data that is the output data for the input data using a loss function. When generating this trained autoencoder, the loss function is calculated based on noise-removed data, which is the output when a plurality of difference data, which is data based on the difference between the steady sound feature data and the estimated steady sound feature data, is input into a noise removal function, which is a function for removing noise from the difference data, and trend data, which is the output when the difference data is input into a trend extraction function, which is a function for extracting a trend in the difference data over a predetermined time period. In other words, by repeating input and output many times, this autoencoder is trained so that the steady sound feature data as input and the estimated steady sound feature data as output become as close to identical as possible.In other words, the trained autoencoder of the present invention is tuned so that when the input data is steady-state sound feature data, the input and output data are nearly identical. Roughly speaking, it is an autoencoder that can make the input data and output data nearly identical for data related to sounds occurring in a steady state. The loss function described above is used in the training process.

[0016] The unsteady sound detection device described below, which is created using the trained autoencoder described above, can detect unsteadiness based on sounds produced by an object, such as a machine having a rotating mechanism, with greater accuracy than before. Furthermore, because the data trained by this trained autoencoder is only steady sound feature data generated based on steady sound data, the effect of reducing the effort and cost required to generate a trained autoencoder is also maintained. The reason why the trained autoencoder of the present invention can detect unsteadiness of an object, such as a machine having a rotating mechanism, with greater accuracy than before is as follows. The inventors of the present invention conducted extensive research into why unsteady sound detection devices incorporating conventional trained autoencoders tend to have lower accuracy in detecting unsteadiness based on sounds produced by objects having a rotating mechanism, and found the following: Objects including rotating mechanisms generally include rotating bodies such as rotating shafts, as well as gears, bearings, etc. When an object including a rotating mechanism enters an unsteady state, shaft misalignment, eccentricity of the rotating body, gear loss, abnormal gear contact, bearing loss, etc., are often present. Such defects in an object manifest as extremely short-lived spikes in the sound produced by the object. These abnormal sounds manifest as spikes in the error score as the output (loss) of the loss function. Objects including rotating mechanisms are constantly in a state of inertial vibration, resulting in minute fluctuations in the error score due to, for example, the engagement of adjacent gears. These fluctuations in the error score manifest as noise in the loss. Furthermore, in rotating mechanisms, the same state occurs repeatedly as the rotating body rotates, so the error score corresponding to the loss may contain a relatively long-term temporal trend, which could be called a trend. It has been discovered that unless the effects of the above-mentioned noise and trend in the loss calculated using the loss function are suppressed, it is impossible to obtain a trained autoencoder capable of matching the steady sound feature data (input data) with the estimated steady sound feature data (output data) as closely as possible. This finding was discovered by the present inventors.Based on this finding, in the present invention, the loss function used when training the encoder is generated based on the noise-removed data, which is the output when the differential data is input to a noise removal function that removes noise from the differential data, and is calculated after generating a plurality of differential data, which is data based on the difference between steady sound feature data and estimated steady sound feature data, and trend data, which is the output when the differential data is input to a trend extraction function that extracts a trend in the differential data over a predetermined time period. By using such a loss function, noise and components corresponding to the trend are removed from the calculated loss, although this may not be completely removed. Therefore, the non-stationary sound detection device described below, which is created using the trained autoencoder according to the present application, can detect non-stationarity based on sounds generated by an object that is a machine having a rotating mechanism with greater accuracy than before.

[0017] As described above, the loss function used in the process of obtaining the trained autoencoder of the present invention includes a noise removal function to remove noise that appears in the differential data, and a trend extraction function to remove trends that appear in the differential data. Here, the noise removal function can be one of three functions: a function that functions as a low-pass filter, a function that calculates a moving average, or a function that performs a Fourier transform, removes frequency band signals exceeding a predetermined threshold, and then performs an inverse Fourier transform to restore the data. In any case, using one of these three functions makes it possible to correct extreme fluctuations that occur in a short period of time between differential data that are generated continuously, which is noise specific to machines with rotating mechanisms. Furthermore, the trend extraction function can be a function that calculates the median of the differential data and multiple data before, after, or before and after the differential data, or a function that takes the difference series of the differential data. Here, the function that calculates the median of multiple data is literally a function that extracts trends. Although a large number of differential data are generated continuously, this does not affect the detection of non-stationarity. However, trends specific to machines with rotating mechanisms, such as long-term sustained changes or gradual periodic changes, may occur. By using a median function to calculate the median of multiple consecutive differential data, including one differential data of interest, it is possible to extract the above-mentioned trend or trend data representing the trend. Using the trend data, the influence of the trend can be removed from the differential data. On the other hand, a function that takes a difference series of differential data removes the influence of the trend from the differential data, and its nature is slightly different from that of a median function that extracts the trend itself. While trend data generated using a median function represents a trend as described above, trend data generated using a difference series corresponds to differential data after the influence of the above-mentioned trend has been removed.

[0018] As described above, the loss function used in the training process of the trained autoencoder of the present invention calculates the loss based on the noise-removed data and trend data. More specifically, if the trend extraction function is a function that calculates the median of the difference data and multiple pieces of data before, after, or both before and after the difference data, the loss function may calculate the loss as the difference between the noise-removed data and the trend data. Conceptually, the noise-removed data is obtained by removing the influence of noise from the difference data between the steady sound feature data and the estimated steady sound feature data paired with that steady sound feature data. When the difference between data obtained by removing the influence of noise from such difference data and trend data that conceptually corresponds to a trend is calculated, the resulting data is conceptually obtained by further removing the influence of the trend from the data obtained by removing the influence of noise from the difference data. Therefore, by using such a loss function, the influence of noise and trends specific to sounds generated by objects having rotating mechanisms can be effectively removed from the loss between the steady sound feature data and the estimated steady sound feature data. When the trend extraction function is a function that takes a difference series of the difference data, the loss function may be determined by applying the noise removal function to the trend data. In this case, the trend data is conceptually obtained by removing the influence of trends from difference data regarding the difference between steady sound feature data and estimated steady sound feature data that pairs with that steady sound feature data. Data obtained by applying the noise removal function to such difference data is conceptually obtained by further removing the influence of noise from data obtained by removing the influence of trends from the difference data. Therefore, by using such a loss function, it is possible to effectively remove the influence of noise and trends that are specific to sounds generated by objects having rotating mechanisms from the loss between steady sound feature data and estimated steady sound feature data.

[0019] The present inventor also proposes a method for generating the trained autoencoder described above as one aspect of the present invention. The effects of this method are similar to those of the trained autoencoder described above. One example of this method is a method for converting an autoencoder that encodes input data, which is predetermined data, and then decodes the encoded data to restore data of the same dimension as the input data, into a trained autoencoder, in which the input data is steady-state sound feature data that is data for a specific length of time about steady-state sound that is generated in a steady state from an object, the object being a target of which non-stationarity is to be detected based on sound, and the output data is estimated steady-state sound feature data for the specific length of time, and a plurality of steady-state sound feature data is input, and the trained autoencoder is generated from the steady-state sound feature data and the output data is estimated steady-state sound feature data for the specific length of time. The method includes a step of calculating a loss for the estimated steady sound feature data, which is output data for the input data, using a loss function, and performing pre-learning so as to minimize the loss function, wherein the loss function is calculated based on noise-removed data, which is an output when a plurality of difference data, which is data based on the difference between the steady sound feature data and the estimated steady sound feature data, is input into a noise removal function, which is a function for removing noise from the difference data, and trend data, which is an output when the difference data is input into a trend extraction function, which is a function for extracting a trend in the difference data within a predetermined time period.

[0020] The present inventors also propose a non-stationary sound detection device using the trained autoencoder described above as one aspect of the present invention. The non-stationary sound detection device (hereinafter, sometimes simply referred to as a "detection device") includes a first recording unit that records one of the trained autoencoders described above, a reception unit that receives measurement sound data, which is data for a specific time length about measurement sound including sound generated from the object for which sound-based non-stationarity detection is being performed, a measurement sound feature data generation unit that generates, from the measurement sound data received by the reception unit, measurement sound feature data, which is data about the feature amounts of the measurement sound identified by the measurement sound data, in the same manner as the stationary sound feature data was generated from the stationary sound data in pre-learning, and a measurement sound feature data generation unit that reads the trained autoencoder recorded in the first recording unit and reads the measurement sound feature data generated by the measurement sound feature data generation unit into the trained autoencoder. a first calculation unit that inputs the signal to a coder to obtain estimated measurement sound feature data for the predetermined time length that is output from the trained autoencoder for the measurement sound feature data; and a second calculation unit that calculates the loss for the difference between the measurement sound feature data generated by the measurement sound feature data generation unit and the estimated measurement sound feature data created from the measurement sound feature data generated by the first calculation unit, using the same loss function as used when training the trained autoencoder, and that, if the generated data for the loss satisfies a predetermined condition, determines that the measurement sound identified by the measurement sound data that is the basis of the measurement sound feature data is a non-stationary sound and generates result data indicating that a non-stationary sound has occurred.

[0021] This non-stationary sound detection device includes a first recording unit that records the above-mentioned trained autoencoder. The trained autoencoder recorded in the first recording unit is used as described below. The detection device also includes a receiving unit that receives measurement sound data, which is data for a specific time length about measurement sound including sound generated from an object for which sound-based non-stationary detection is being performed. The data received by the receiving unit is measurement sound data, which is data for a specific time length about measurement sound including sound generated from an object for which sound-based non-stationary detection is being performed. In other words, the detection device detects non-stationary sound based on measurement sound including sound generated from the object. The detection device also includes a measurement sound feature data generation unit that generates measurement sound feature data from the measurement sound data received by the receiving unit, which is data about feature quantities of the measurement sound identified by the measurement sound data. The measurement sound data has the same data format and type as the steady sound data used in the pre-learning stage, and the measurement sound feature data is generated by the same method as the method used when generating the steady sound feature data from the steady sound data in the pre-learning stage. The measured sound feature data is input to the trained autoencoder, but since both the measured sound feature data and the steady sound feature data input to the autoencoder during training for training the trained autoencoder are data about sound, the format or type of both data can be the same. For example, when the steady sound feature data is a Mel frequency spectrogram generated from the steady sound data, the measured sound feature data is also a Mel frequency spectrogram generated from the measured sound data.

[0022] This non-stationary sound detection device also includes a first calculation unit. The first calculation unit reads the trained autoencoder recorded in the first recording unit and operates the trained autoencoder. The first calculation unit inputs the measurement sound feature data generated by the measurement sound feature data generation unit into the trained autoencoder read from the first recording unit, and obtains estimated measurement sound feature data as its output. As already mentioned, the trained autoencoder is tuned so that when data on the characteristics of a stationary sound (stationary sound feature data if training is in progress) is input, it outputs estimated stationary sound feature data that is nearly identical to the input data. Therefore, if the measurement sound identified by the measurement sound data from which the measurement sound feature data is derived is a stationary sound (sound including sound from an object in a stationary state), the estimated measurement sound feature data output by the trained autoencoder is almost identical to the measurement sound feature data from which the estimated measurement sound feature data is derived. This conclusion remains true even if the measurement sound identified by the measurement sound data from which the measurement sound feature data is derived includes environmental sound. This is because, just as environmental sounds may be included in the measurement sounds, the steady sounds identified by the steady sound data that was the source of the sound feature data input to the trained autoencoder during the learning process may also include environmental sounds. Therefore, the components derived from environmental sounds contained in the measurement sound feature data can be regarded as components derived from steady sounds that have already been learned from the trained autoencoder, and are not perceived as components that deviate from steady sounds. On the other hand, the trained autoencoder used in the first calculation unit is tuned so that the input and output approximately match only when the input is data related to steady sounds. In other words, the trained autoencoder described above can be said to be an autoencoder dedicated to steady states or steady sounds that has been trained specifically for steady states or steady sounds, and can demonstrate the function of approximately matching the input and output only when the object is in a steady state or when a steady sound including a sound from an object in a steady state is input.Therefore, when the measurement sound identified by the measurement sound data from which the measurement sound feature data is derived is a non-stationary sound, including sound from an object in a non-stationary state, the trained autoencoder will not function in the same way as when the object is in a stationary state (in other words, it will not function as expected, or, to put it crudely, will malfunction). Therefore, in this case, the estimated measurement sound feature data output by the trained autoencoder will be significantly different from the measurement sound feature data from which the estimated measurement sound feature data is derived. For the reasons already mentioned, this conclusion remains the same even if the measurement sound includes environmental sound. This detection device includes a second calculation unit. The second calculation unit, in cooperation with the first calculation unit or in cooperation with the measurement sound feature data generation unit and the first calculation unit, functions as an artificial intelligence that detects that the object is in a non-stationary state based on measurement sound data received from an external source. The second calculation unit is configured to calculate a loss for the difference between the measurement sound feature data generated by the measurement sound feature data generation unit and the estimated measurement sound feature data generated from the measurement sound feature data generated by the first calculation unit. The loss function used to calculate the loss is the same loss function as that used when training the trained autoencoder. In other words, the method used to calculate the loss is the same as that used when training the trained autoencoder. As described above, when the measurement sound data is a stationary sound, the measurement sound feature data as input to the trained autoencoder and the estimated measurement sound feature data as output from the trained autoencoder will be approximately the same, and when the measurement sound data is a non-stationary sound, the measurement sound feature data as input to the trained autoencoder and the estimated measurement sound feature data as output from the trained autoencoder will be significantly different.Therefore, when the measurement sound data is a stationary sound, the loss between the measurement sound feature data as input to the trained autoencoder and the estimated measurement sound feature data as output from the trained autoencoder is relatively small. When the measurement sound data is a non-stationary sound, the loss between the measurement sound feature data as input to the trained autoencoder and the estimated measurement sound feature data as output from the trained autoencoder is relatively large. Because the loss function used to generate the loss for the difference between the measurement sound feature data and the estimated measurement sound feature data is the same loss function used during training of the trained autoencoder, the effects of noise and trends are removed from the resulting loss. Therefore, the characteristic that the loss obtained using the loss function is relatively small when the measurement sound data is a stationary sound and relatively large when the measurement sound data is a non-stationary sound is guaranteed with a certain level of accuracy. Furthermore, when the generated loss data satisfies a predetermined condition, the second calculation unit determines that the measurement sound identified by the measurement sound data that is the basis of the measurement sound feature data is a non-stationary sound and generates result data indicating that a non-stationary sound has occurred. Based on the difference in the magnitude of the loss, the second calculation unit can determine whether the object that emitted the sound included in the measurement sound identified in the measurement sound data received by the detection device is in a steady state or an unsteady state, and can generate result data that indicates that the object is in an unsteady state. The result data that indicates that the object is in an unsteady state may be used in any manner. For example, once the result data is generated, the detection device or a predetermined other device that receives the result data from the detection device may perform an appropriate operation to notify a user of the detection device that the object is in an unsteady state.For example, the detection device or other device may notify a predetermined user of the occurrence of the event by email or other message, or by displaying the occurrence on a display connected to the detection device or other device, sounding a patrol lamp, or other method that allows the user to detect the occurrence using one of the five senses. Furthermore, by recording a large amount of result data in chronological order, for example, together with a timestamp, on a recording medium inside or outside the detection device, it becomes possible to predict how the state of the object will change in the future based on the trends in the accumulated result data, or to verify the history of the object's state in the past. The detection device may be equipped with a predetermined display device. In this case, the second calculation unit may generate the result data as image data for displaying on the display a graph image indicating that the measurement sound identified by the measurement sound data from which the measurement sound feature data was derived is a non-stationary sound. This allows the user to intuitively grasp the state of the object generating the input measurement sound data, for example, whether it is in a steady state or a non-stationary state.

[0023] As is clear from the above explanation, the non-stationary sound detection device according to this application can detect that the object is in a non-stationary state even if the measurement sound data includes environmental sound. This detection device also has the advantage that it does not require the application of noise canceling technology to the measurement sound data.

[0024] As described above, the second calculation unit in the detection device of the present invention calculates the difference between the measurement sound feature data generated by the measurement sound feature data generation unit and the estimated measurement sound feature data created from the measurement sound feature data generated by the first calculation unit. In this case, the method for detecting the difference can be determined as appropriate. For example, the second calculation unit may determine that the measurement sound identified by the measurement sound data from which the measurement sound feature data is derived is a non-stationary sound when the loss data contains a predetermined number of values ​​exceeding a predetermined threshold. In this manner, the second calculation unit can detect whether the sound from the object contains a non-stationary sound, i.e., whether the object is in a non-stationary state, simply by counting the number of values ​​in the loss function that exceed the threshold. In addition, using the above-described threshold has the following advantages: In the case of conventional artificial intelligence, the type of value output from the artificial intelligence basically depends on the type of data that the artificial intelligence has been pre-trained with. For example, if the AI ​​outputs four levels of abnormal noise generated by an object: normal, average, bad, and worst, the type of data pre-trained by the AI ​​must conform to these four levels, and each data must be labeled accordingly. However, if the second calculation unit uses the threshold described above, even if the data trained by the trained autoencoder of the present invention is only steady-state data, simply changing the threshold can produce different judgment results from the same loss function. This means that the above example allows for four-level judgment without complicating the pre-training of the trained autoencoder. Needless to say, to achieve this effect, the detection device of the present invention can change the threshold used by the second calculation unit. Such threshold changes can be made, for example, by input from a specified input device (keyboard, mouse, etc.) connected to the detection device. Furthermore, the detection device itself can automatically change the threshold according to predetermined rules.

[0025] The present inventor also proposes, as one aspect of the present invention, a method for detecting non-stationary sounds that applies the trained autoencoder of the present invention. The effects of this method are similar to those of the non-stationary sound detection device of the present invention. One example of the method is a method for detecting non-stationary sounds that is executed by a computer having a first recording unit that records the trained autoencoder described above, and includes the following steps, all of which are executed by a computer. The steps include a first step of receiving measurement sound data, which is data for a specific time length about measurement sound that includes sound generated from the object for which sound-based non-stationarity detection is being performed; a second step of generating, from the measurement sound data received in the first step, measurement sound feature data, which is data about the feature quantities of the measurement sound identified by the measurement sound data, using the same method as used to generate the stationary sound feature data from the stationary sound data in pre-learning; and a second step of reading the trained autoencoder recorded in the first recording unit, inputting the measurement sound feature data generated in the second step into the trained autoencoder, and performing the trained autoencoder on the measurement sound feature data. a third step of obtaining estimated measurement sound feature data for the predetermined time length that is output from an autoencoder; and a fourth step of calculating the loss for the difference between the measurement sound feature data generated in the second step and the estimated measurement sound feature data created from the measurement sound feature data generated in the third step, using the same loss function as used when training the trained autoencoder, and, if the generated data for the loss satisfies a predetermined condition, determining that the measurement sound identified by the measurement sound data that is the basis of the measurement sound feature data is a non-stationary sound and generating result data indicating that a non-stationary sound has occurred.

[0026] The present inventors also propose, as one aspect of the present invention, a computer program for causing a specified computer to function as a non-stationary sound detection device that applies the trained autoencoder of the present invention. The effects of this computer program are similar to those of the non-stationary sound detection device of the present invention, and also have the effect of causing a general-purpose computer to function as the non-stationary sound detection device of the present invention. One example of the computer program is a computer program for causing a specified computer to function as the non-stationary sound detection device.This computer program includes an autoencoder that encodes input data, which is predetermined data, and then decodes the encoded data to restore it to data of the same dimension as the input data, the input data being steady sound feature data that is data on features of steady sound identified by steady sound data generated from steady sound data, which is data for a specific length of time about steady sound including sound generated in a steady state from an object that is an object for which non-stationarity based on sound is to be detected, and output data being estimated steady sound feature data, and the computer program includes a first recording unit that records a trained autoencoder that is obtained by inputting a plurality of steady sound feature data and performing pre-training so as to minimize a difference between the steady sound feature data that is input data and the estimated steady sound feature data that is output data for the input data; a receiving unit that receives measurement sound data that is data for a specific length of time about measurement sound including sound generated from the object for which sound-based non-stationarity detection is being performed; and a receiving unit that receives measurement sound data that is data for a specific length of time about measurement sound including sound generated from the object for which sound-based non-stationarity detection is being performed, the receiving unit receiving data on the features of the measurement sound identified by the measurement sound data. a measurement sound feature data generation unit that generates measurement sound feature data, which is data for the measurement sound feature data, in the same manner as the steady sound feature data was generated from the steady sound data in pre-learning; a first calculation unit that reads the trained autoencoder recorded in the first recording unit, inputs the measurement sound feature data generated by the measurement sound feature data generation unit into the trained autoencoder, and obtains estimated measurement sound feature data that is output from the trained autoencoder for the measurement sound feature data; and a second calculation unit that calculates a loss for a difference between the measurement sound feature data generated by the measurement sound feature data generation unit and the estimated measurement sound feature data created from the measurement sound feature data generated by the first calculation unit, using the same loss function as used when training the trained autoencoder, and, if the generated data for the loss satisfies a predetermined condition, determines that the measurement sound identified by the measurement sound data that is the basis of the measurement sound feature data is a non-stationary sound, and generates result data indicating that a non-stationary sound has occurred.

[0027] 1 is a perspective view showing an outline of the overall configuration of a learning device according to an embodiment. A diagram showing the hardware configuration of a computer device included in the learning device shown in FIG. 1. A block diagram showing functional blocks generated inside the computer device shown in FIG. 2. A diagram conceptually showing the configuration of an autoencoder included in the computer device shown in FIG. 2. A diagram conceptually showing an example of a loss generated by the loss generation unit shown in FIG. 3. A block diagram showing functional blocks generated inside a computer device included in a detection device according to an embodiment. A diagram showing the relationship between loss and a threshold when determination is made by a state determination unit. A diagram showing an example of a histogram when an object is in a steady state. A diagram showing an example of a histogram when an object is in a non-steady state.

[0028] A preferred embodiment of the present invention will be described below with reference to the drawings. Test examples performed using the detection device of the embodiment will also be described later.

[0029] In this embodiment, a non-stationary sound detection device (hereinafter, sometimes simply referred to as a "detection device") will be described. The non-stationary sound detection device includes a trained autoencoder, as described below. Therefore, to obtain the non-stationary sound detection device, it is first necessary to obtain a trained autoencoder. In this embodiment, for convenience, the device necessary to obtain the trained autoencoder will be referred to as a learning device. This detection device is designed to detect whether a machine having a rotating mechanism is in an unsteady state based on the sound from the machine. A typical example of a machine having a rotating mechanism is a motor. Devices incorporating motors, such as the propeller drive unit of a wind power generation facility, the motor drive unit of other power plants, and the rotary drive unit of a manufacturing factory such as a glass factory, can be objects whose unsteady state is detected by the detection device of this embodiment. Furthermore, the non-stationary sound detection device of this embodiment uses a threshold value, described below, when detecting unsteady sounds, although this is not necessarily required. In this embodiment, the threshold value is generated by the above-mentioned learning device used to obtain the trained autoencoder. However, the threshold may be generated by the detection device instead of the learning device, or may be generated by a device other than the learning device or the detection device.

[0030] As described above, to establish a non-stationary sound detection device, in addition to the detection device, a learning device and, in some cases, another device for determining a threshold value, separate from the detection device and the learning device, are required. However, the detection device, learning device, and other device, which are required up to three times, can have the same required hardware configuration, so they can be combined into a single device by installing an appropriate computer program on them. Therefore, naturally, any two of the detection device, learning device, and other device, which are required up to three times, can be combined into a single device. As already described, in this embodiment, the threshold value is determined by the learning device. In other words, this can be said to be an example of an aspect in which the learning device and other device are combined into a single device.

[0031] Both the detection device and the learning device include a computer. The computer included in the detection device and the learning device can be the same, and in this embodiment, they are the same. In addition, both the detection device and the learning device include peripheral devices connected to the computer. These peripheral devices can also be the same for the detection device and the learning device, and in this embodiment, they are the same, but are not limited to this.

[0032] <About the Learning Device> From a hardware perspective, the configuration of the detection device is the same, but for now, the hardware configuration of the learning device is shown in Figure 1. The learning device includes a computer device and peripheral devices, which are also shown in Figure 1.

[0033] In the figure, reference numeral 100 denotes a computer device. A display 101 and an input device 102 are connected to the computer device 100. Furthermore, as will be described later, a microphone 103 is also connected to the computer device 100. The display 101 is for displaying still images or moving images, and a publicly known or well-known display device can be used. In this embodiment, the display 101 is capable of displaying both still images and moving images, but this is not a requirement. The display 101 may be a publicly known or well-known display device, and commercially available products such as a liquid crystal display or an organic electroluminescence (EL) display are sufficient. In this embodiment, the display 101 is connected to the computer device 100 via a cable, but may also be connected to the computer device 100 wirelessly. The technology used to connect the computer device 100 and the display 101 may also be publicly known or well-known. The input device 102 allows a user to input desired information to the computer device 100. A publicly known or well-known display device can be used as the input device 102. In this embodiment, the input device 102 of the computer device 100 is a keyboard; however, the input device 102 is not limited to this, and publicly known or well-known voice input devices using a numeric keypad, trackball, mouse, or microphone terminal can also be used. If the display 101 is a touch panel, the display 101 also functions as the input device 102. A microphone 103 is connected to the computer device 100. The microphone 103 has the function of collecting sounds, including sounds emitted by objects described below, and generating sound data, which is data on the collected sounds. As long as the microphone 103 has such a function, it may be a publicly known or well-known device, and a commercially available product may suffice. The sound data of the sound generated by the microphone 103 is sent from the microphone 103 to the computer device 100 via a cable. The method for sending the sound data from the microphone 103 to the computer device 100 may be wired or wireless.Furthermore, the transmission of sound data from the microphone 103 to the computer device 100 may be via the Internet. Furthermore, the transmission of sound data from the microphone 103 to the computer device 100 via wired, wireless, or Internet communication does not have to be performed in substantially real time. For example, the sound data generated by the microphone 103 may be temporarily recorded on a recording medium within a device unrelated to the computer device 100, and the sound data recorded on that recording medium may then be read into the computer device 100 directly or via another recording medium, thereby being supplied to the computer device 100. In short, while the sound data generated by the microphone 103 must be sent to the computer device 100, the method and timing of providing the sound data may be determined appropriately depending on the circumstances. An amplifier for amplifying the sound data may be present between the microphone 103 and the computer device 100. Amplifiers are, of course, well-known and commercially available in large quantities, so if an amplifier is to be used, an appropriate one may be selected from these. Although an amplifier is used in this embodiment, it is a common one and will not be described or illustrated in detail.

[0034] Next, the configuration of the computer device 100 constituting the learning device will be described. The hardware configuration of the computer device 100 is shown in FIG. 2 . The hardware includes a central processing unit (CPU) 111, a read-only memory (ROM) 112, a random access memory (RAM) 113, an interface 114, and a mass storage medium 115, which are interconnected by a bus 116. The CPU 111 is a computing device that performs calculations. The CPU 111, which is a computing device, can be replaced with a graphics processing unit (GPU) or a general purpose computing unit (GPGPU), which are also computing devices. The CPU 111 executes the processes described below by executing a computer program stored in, for example, the ROM 112 or the RAM 113. The mass storage medium 115 is a known or commonly used device for storing large amounts of data, such as a hard disk drive (HDD) or a solid state drive (SSD). The computer program described above may be stored in the mass storage device 115, and this is actually more common. The computer program referred to here includes a computer program for causing the computer device 100 to execute the processes described below that are necessary for the computer device 100 to function as a learning device. This computer program may be pre-installed on the computer device 100 or may be post-installed. This computer program may be installed on the computer device 100 via a predetermined recording medium (not shown), such as a memory card, or via a network such as a LAN or the Internet. Of course, the computer program may include an OS and other necessary computer programs in addition to the above computer programs. The ROM 112 stores computer programs and data necessary for the CPU 111 to execute the processes described below. The RAM 113 provides a work area necessary for the CPU 111 to perform processing.In some cases, the above-mentioned computer programs and data (at least a portion thereof) may be recorded or temporarily recorded. The interface 114 exchanges data between the CPU 111, RAM 113, etc., which are connected via a bus 116, and the outside. The interface 114 is connected to the above-mentioned display 101, input device 102, and microphone 103. Data on operation content input from the input device 102 and sound data sent from the microphone 103 are input from the interface 114 to the bus 116. As is well known, data for displaying an image on the display 101 is sent from the bus 116 to the interface 114 and output from the interface 114 to the display 101.

[0035] The entire system including the computer device 100 described above executes the processes necessary for the computer device 100 to function as a learning device using the following functional blocks. When the CPU 111 executes a computer program, functional blocks such as those shown in FIG. 3 are generated within the computer device 100. The following functional blocks may be generated by the functions of the computer program alone, which causes the computer device 100 to execute the processes described below necessary for the computer device 100 to function as a learning device. Alternatively, the following functional blocks may be generated by cooperation between the computer program and an OS or other computer programs installed on the computer device 100. Within the computer device 100, an input unit 121, a main control unit 122, a feature detection unit 123, an autoencoder 124, an autoencoder recording unit 124A, a loss generation unit 125, a loss function recording unit 125A, a loss recording unit 125B, an optimization unit 126, a threshold determination unit 127, and an output unit 128 are generated in relation to the functions of the present invention. Among these, the functional blocks that perform information processing, i.e., the main control unit 122, the feature detection unit 123, the autoencoder 124, the loss generation unit 125, the optimization unit 126, and the threshold determination unit 127, correspond to a computing device (the CPU 111 in this embodiment) in terms of hardware, or are realized by the functions of the computing device that executes information processing in accordance with the instructions of the computer program. The input unit 121 and the output unit 128 correspond to the interface 114 in terms of hardware, or are realized by the functions of the interface 114. More specifically, the input unit 121 and the output unit 128 conceptually correspond to the connection between the interface 114 and the bus 116. Furthermore, the functional blocks that record data, i.e., the autoencoder recording unit 124A, the loss function recording unit 125A, and the loss recording unit 125B, correspond to a recording device, such as the RAM 113 or the large-capacity recording medium 115, in terms of hardware, or are realized as a part of the recording device.

[0036] The input unit 121 receives input from the interface 114. Inputs from the interface 114 to the input unit 121 include input from the input device 102. The input from the input device 102, which will be described in detail later, includes, for example, mode selection data. When mode selection data or the like is input from the input device 102, the mode selection data or the like input from the input device 102 is sent from the input unit 121 to the main control unit 122. Data input from the interface 114 to the input unit 121 also includes sound data from the microphone 103. When the sound data is received by the input unit 121, the input unit 121 sends the sound data to the feature detection unit 123. The main control unit 122 performs overall control of the above-mentioned functional blocks generated within the computer device 100. For example, the main control unit 122 may receive mode selection data as described above. In this embodiment, the learning device functions in a mode alternatively selected from two modes: a learning mode in which the computer device 100 functions as a learning device, and a threshold determination mode in which the computer device 100 functions as a threshold determination device. The mode selection data determines (specifies) which of the two modes the computer device 100 should function in. Upon receiving this data, the main control unit 122 instructs appropriate functional blocks to execute the selected mode. In this embodiment, at least the main control unit 122 instructs the loss generation unit 125 as to the output destination of the loss (or, more precisely, the data) generated by the loss generation unit 125, as described below. Specifically, when the learning device executes the learning mode, the main control unit 122 instructs, for example, the optimization unit 126 to update the data (described below) for generating the autoencoder 124 recorded in the autoencoder recording unit 124A. When the learning device executes the threshold determination mode, the main control unit 122 instructs, for example, the threshold determination unit 127 to determine a threshold (described below).

[0037] The feature detection unit 123 detects features of the received sound data and generates sound feature data, which is data regarding the sound features. In this embodiment, the feature detection unit 123 performs necessary processing on the sound data before generating the sound feature data, such as converting the sound data from analog data to digital data in a predetermined format (e.g., digital data in WAV format) and adjusting the length of the sound data to a predetermined length (e.g., a planned length such as 10 seconds or 30 seconds). While not necessarily limited to this, in this embodiment, the feature detection unit 123 generates sound feature data regarding the features of the sound data adjusted in this manner. The sound feature data will be described in detail later, but in this embodiment, it is a Mel frequency spectrogram (or, more precisely, the data thereof), although it is not limited to this. How the Mel frequency spectrogram is generated will be described later. The feature detection unit 123 sends the generated sound feature data to the autoencoder 124 and the loss generation unit 125.

[0038] The autoencoder recording unit 124A stores data for generating an autoencoder. The CPU 111, which serves as a computing device, reads the data for generating an autoencoder from the autoencoder recording unit 124A to generate the autoencoder 124. The autoencoder 124 is an autoencoder that encodes input data, which is predetermined data, and then decodes the encoded data to restore data of the same dimensions as the input data. Such autoencoders are publicly known or well-known in the field of artificial intelligence. The autoencoder 124 (or data for generating an autoencoder) of this embodiment can be selected from such known autoencoders (or data for generating an autoencoder). The autoencoder 124 is conceptually shown in FIG. 4. The autoencoder 124 is configured by combining an input layer (Input Layer) that accepts an input x, an intermediate layer that encodes the input x to compress features and decodes the compressed data to expand the features, and an output layer (Output Layer) that outputs an output x'. As described above, the autoencoder 124 may receive sound feature data from the feature detection unit 123. Upon receiving the sound feature data, the autoencoder 124 encodes the received sound feature data and subsequently decodes the encoded data to restore the sound feature data and output it. The output sound feature data will be referred to as estimated sound feature data. The autoencoder 124 sends the generated estimated sound feature data to the loss generation unit 125. The data for generating the autoencoder 124, which is recorded in the autoencoder recording unit 124A, is updated by the optimization unit 126, as described below. The data for generating the autoencoder 124, which is recorded in the autoencoder recording unit 124A, is sent to the output unit 128 at a timing described below.

[0039] As described above, the loss generation unit 125 receives sound feature data from the feature detection unit 123 and estimated sound feature data from the autoencoder 124. The sound feature data and estimated sound feature data are a set of data that correspond to the input and output, respectively, of a single encoding and decoding process performed by the autoencoder 124. The loss generation unit 125 generates a loss for the difference between the set of sound feature data and the estimated sound feature data using a loss function. There are several known or well-known loss functions, but this embodiment uses the following loss function, although this is not limited to these. Note that data specifying the loss function is recorded in the loss function recording unit 125A. When the loss generation unit 125 generates data regarding the loss, it first reads the data specifying the loss function recorded in the loss function recording unit 125A and performs calculations using the loss function specified by the data specifying the loss function.

[0040] The loss function used in this embodiment is a function combining the following (a) to (d) mathematically expressed: (a) generating a plurality of difference data that is data based on the difference between sound feature data and estimated sound feature data; (b) generating noise-removed data that is the output when the difference data is input to a noise removal function that is a function for removing noise from the difference data; (c) generating trend data that is the output when the difference data is input to a trend extraction function that is a function for extracting a trend in the difference data for a predetermined time period; (d) calculating the loss based on the noise-removed data and the trend data. Here, the loss function is a combination of mathematical expressions equivalent to (a) to (d) in that order, but (b) and (c) can be in any order.

[0041] An example of a formula that can be used in (a) above, "generating a plurality of difference data, which are data based on the differences between sound feature data and steady sound feature data," is the mean squared error (MSE). The reason for using this is to reduce the inference error when the difference between the sound feature data and the estimated sound feature data is extremely large. Specifically, the formula shown in (Equation 1) below is a formula for calculating the mean squared error. If it is important to make the value of the loss function during learning easier to intuitively recognize, the root mean squared error (RMSE) may be used instead of the mean squared error. The formula shown in (Equation 2) below is a formula for calculating the root mean squared error. As an alternative loss function, if the goal is to reduce the average error between the sound feature data and the estimated sound feature data, the mean absolute error (MAE) may also be used. The formula shown in (Equation 3) below is a formula for calculating the mean absolute error. In (Equation 1) to (Equation 3), "i" means the ith number. The meaning of "i" is the same in (Equation 4) and subsequent equations described later. As described above, sound data is sound data of a specific time length. From this, a large number of sets of sound feature data and estimated sound feature data are created for each short period of time along the flow of time. The "y hat" in (Equation 1) to (Equation 3) ｉ (The hat symbol is written this way for convenience because it cannot be used in the text of the application software.) is the i-th estimated sound feature data, and "y ｉ " means the i-th sound feature data. Also, "MSE ｉ " means the mean square error between the i-th sound feature data and the estimated sound feature data. Similarly, "RMSE" in (Equation 2) ｉ " is the root mean square error between the i-th sound feature data and the estimated sound feature data, expressed as "MAE" in (Equation 3). ｉ" means the mean absolute error between the i-th sound feature data and the estimated sound feature data. Although not limited to this, in this embodiment, as a mathematical formula that can be used in the above-mentioned (a) of "generating a plurality of difference data that are data based on the difference between the steady sound feature data and the estimated steady sound feature data", (Math. 1) for the mean square error is used. As a result, a large number of MSE ｉ (where i is a natural number from 1 to n (n is 2 or more)) will be generated. ｉ Each of these is differential data.

[0042] The mathematical formula that can be used in (b) above, "generating noise-removed data, which is the output when differential data is input to a noise removal function that removes noise from differential data," is, for example, a function that functions as a low-pass filter, a function that calculates a moving average, or a function that performs a Fourier transform, removes frequency band signals that exceed a predetermined threshold, and then performs an inverse Fourier transform to return the data. All three of these functions are known to be usable to remove noise from certain data. An example of a mathematical formula that functions as a low-pass filter is shown below (Math 4). An example of a mathematical formula that calculates a moving average is shown below (Math 5). f(MSE ｉ ) = α × MSE ｉ + (1-α) × MSE ｉ－１ ... (Equation 4) *α is a parameter and a constant between 0 and 1. * m is a parameter and is an integer value of 2 or more. In the example expressed by (Equation 5), the difference data is MSE ｉ is MSE ｉ－（ｍ－１） , …, MSE ｉ－１ , MSE ｉ That is, MSE ｉ MSE including ｉ f(MSE ｉ ) is corrected to MSE when calculating the moving average. ｉ You can also use the m MSEs after ｉ MSE including ｉIt is obvious that m MSEs before and after f(MSE) may be used. Although not limited to this, in this embodiment, a function that can be used as a low-pass filter is used as the mathematical expression that can be used in the above-mentioned (b) of "generating noise-removed data, which is the output when differential data is input to a noise removal function that is a function that removes noise from differential data." As a result, a large number of f(MSE ｉ ) (where i is 1 to n (n is a natural number greater than or equal to 2)) will be generated. ｉ ) are the denoised data.

[0043] The formula that can be used in (c) above, "generating trend data, which is the output when differential data is input into a trend extraction function that is a function that extracts the trend of differential data over a predetermined time period," can be, for example, a function that finds the median of multiple pieces of data before, after, or both before and after the differential data, or a function that takes the difference series of differential data. It is known that all three of these functions can be used to extract the trend over time of a large amount of data. An example of a formula for finding the median is shown below (Equation 6): g(MSE ｉ ) = Median (MSE ｉ ,MSE ｉ－１ , ...MSE ｉ－（ｍ－１） ) ... (Equation 6) * m is a parameter, an integer value of 2 or more. The g (MSE ｉ ) represents the trend of the i-th MSE. In this formula, ｉ To find the trend, we use MSE ｉ From MSE ｉ－（ｍ－１） MSE ｉ The data for the previous m-1 MSEs are used, and the median of the total m data is called Median (MSE ｉ , MSE ｉ－１ , ...MSE ｉ－（ｍ－１） However, the data used to calculate the median is MSE ｉ and MSE ｉNot limited to data older than MSE ｉ and MSE ｉ Data after this is also acceptable, and MSE ｉ and MSE ｉ Data before and after the MSE ｉ When calculating the median using data before and after g, (Equation 6) becomes, for example, the following (Equation 6x): g(MSE ｉ ) = Median (MSE ｉ＋ｊ ,MSE ｉ＋ｊ－１ ,…, MSE ｉ ,MSE ｉ－１ ,…, , MSE ｉ－（ｍ－１） ) ... (Equation 6x) *j and m are parameters, each an integer value of 2 or more. Next, an example of a formula for finding the difference sequence is shown below (Equation 7). g(MSE ｉ ) = MSE ｉ -MSE ｉ－１ ... (Equation 7) g (MSE ｉ ) is obtained by removing the influence of the trend from the i-th MSE. In this case, the difference series is the difference between adjacent MSE values, but the difference series does not necessarily have to be the difference between adjacent MSE values; for example, it can be the difference between the MSE and the MSE two steps before or after. Although not limited to this, in this embodiment, the function shown in (Equation 6) for finding the median is used as the mathematical formula that can be used in (c) above, which is to "generate trend data, which is the output when differential data is input into a trend extraction function, which is a function that extracts the trend of differential data in a predetermined time period." As a result, a large number of g(MSE ｉ ) (where i is 1 to n (n is a natural number greater than or equal to 2)) will be generated. ｉ ) are trend data.

[0044] After obtaining the noise-removed data and trend data, the loss generator 125 executes the calculation (d) described above, which is to "obtain the loss based on the noise-removed data and trend data." In (d), for example, the loss can be obtained by finding the difference between the noise-removed data and the trend data. The formula used in this case is to find the difference between the noise-removed data and the trend data. Specifically, this calculation is as shown in the following (Equation 8). Loss ｉ = f (MSE ｉ ) - g (MSE ｉ ) ... (Number 8) Here, Loss ｉ is the loss calculated as the difference between the i-th noise-removed data and the i-th trend data. i If is negative, replace it with 0 to get Loss i can be obtained as a value equal to or greater than 0, and although not limited to this, this is the case in this embodiment. Conceptually, the noise-removed data is obtained by removing the influence of noise from difference data regarding the difference between sound feature data and estimated sound feature data paired with that sound feature data. When the difference between data obtained by removing the influence of noise from such difference data and trend data corresponding to the trend is calculated, the data is conceptually obtained by further removing the influence of the trend from data obtained by removing the influence of noise from the difference data. As is clear from the above explanation, the loss function in this embodiment is defined as a combination of (Equation 1), (Equation 4), (Equation 6), and (Equation 8). Data defining such a loss function is recorded in the loss function recording unit 125A. In any case, Loss ｉ (where i is a natural number greater than or equal to 2)) will be generated. ｉ The loss between the sound feature data and the estimated sound feature data is finally obtained as a set. Note that, when a function for obtaining a difference series such as that shown in (Equation 7) is used as the trend extraction function, the loss between the sound feature data and the estimated sound feature data is ｉ) is obtained by removing the influence of the trend from the i-th MSE. In this case, the calculation (d) above, performed by the loss generator 125 to "obtain a loss based on noise-removed data and trend data," may be, for example, to obtain a loss by applying one of the noise removal functions described above to the trend data. In this case, the trend data is conceptually obtained by removing the influence of the trend from the difference data regarding the difference between the steady sound feature data and the estimated steady sound feature data paired with that steady sound feature data. The data obtained by applying the noise removal function to such difference data is conceptually obtained by further removing the influence of noise from the data obtained by removing the influence of the trend from the difference data. This calculation is specifically shown in Equation 8x below. Loss ｉ = f(g(MSE ｉ )) ... (Equation 8x) In this case, f is a noise removal function, for example, any of the noise removal functions already described. The loss can also be determined by such a calculation. Therefore, by using such a loss function, it is possible to effectively remove the effects of noise and trends specific to sounds generated from objects having rotating mechanisms from the loss between the steady sound feature data and the estimated steady sound feature data. The loss function in this case is defined as a combination of, for example, (Equation 1), (Equation 4), (Equation 7), and (Equation 8x). In this case, data defining such a loss function is recorded in the loss function recording unit 125A.

[0045] The loss generator 125 generates a large number of losses. ｉ The loss recording unit 125B is configured to send data about losses, which is a set of data about losses, to the loss recording unit 125B. Data about losses is sequentially additionally written to the loss recording unit 125B, and thereby the data about losses is recorded.

[0046] The optimization unit 126 has a function of adjusting or updating data for generating the autoencoder 124, i.e., data recorded in the autoencoder recording unit 124A, so as to minimize the loss function. ｉ Each time the loss generation unit 125 generates data about losses and records it in the loss recording unit 125B, it updates the data for generating the autoencoder 124 recorded in the autoencoder recording unit 124A. For example, each time the loss generation unit 125 generates data about losses, it sends data to the optimization unit 126 that prompts the update of the data for generating the autoencoder 124 recorded in the autoencoder recording unit 124A. Upon receiving the data prompting the update, the optimization unit 126 updates the data for generating the autoencoder 124 recorded in the autoencoder recording unit 124A. The optimization unit 126 updates the data for generating the autoencoder 124 recorded in the autoencoder recording unit 124A, for example, as follows. In the initial state, the autoencoder recording unit 124A records a dataset for generating the autoencoder 124 that has not yet been trained. As is known, the autoencoder 124 compresses the features of the input data by encoding the input data, and expands the minimized features by decoding the minimized feature data. In the process, it uses multiple weighting coefficients. This data set is the data for generating the autoencoder 124. When performing the first process of optimizing the data for generating the autoencoder 124, the optimization unit 126 receives the first loss, Loss １The optimization unit 126 reads out the Loss from the loss recording unit 125B, and also reads out data for generating the autoencoder 124 from the autoencoder recording unit 124A. Then, the optimization unit 126 adjusts the multiple coefficients that are the data for generating the autoencoder 124 by changing them so as to minimize the loss. Then, the adjusted coefficients are recorded in the autoencoder recording unit 124A as new data for generating the autoencoder 124. This updates the data for generating the autoencoder 124 that was recorded in the autoencoder recording unit 124A. When performing a second process for optimizing the data for generating the autoencoder 124, the optimization unit 126 reads out the Loss from the loss recording unit 125B, and also reads out data for generating the autoencoder 124 from the loss recording unit 125B. １ and Loss ２ and data for generating the autoencoder 124 from the autoencoder recording unit 124A. １ and Loss ２ The optimization unit 126 adjusts the coefficients so that the sum of the Loss σ is minimized. Then, each adjusted coefficient is recorded in the autoencoder recording unit 124A as new data for generating the autoencoder 124. As a result, the data for generating the autoencoder 124 recorded in the autoencoder recording unit 124A is updated. This is the same for subsequent cases. When performing the i-th process of optimizing the data for generating the autoencoder 124, the optimization unit 126 reads the Loss σ from the loss recording unit 125B. １ , Loss ２ ,……Loss ｉ and also reads data for generating the autoencoder 124 from the autoencoder recording unit 124A. １ , Loss ２ ,……Loss ｉ The multiple coefficients are adjusted so that the sum of the Loss is minimized. Then, each adjusted coefficient is recorded in the autoencoder recording unit 124A as new data for generating the autoencoder 124. As a result, the data for generating the autoencoder 124 recorded in the autoencoder recording unit 124A is updated. １ , Loss２ ,……Loss ｉ Without using all of the above, １ Alternatively, the data for generating the autoencoder 124 may be sequentially updated using only the estimated sound feature data. In this manner, the learning of the autoencoder 124 proceeds. As a result, when the learning device is executing the learning mode, the autoencoder 124 is tuned so that the difference between the sound feature data and the estimated sound feature data is minimized.

[0047] The threshold determination unit 127 may receive data from the main control unit 122 instructing it to generate threshold data. Upon receiving this data, the threshold determination unit 127 determines a threshold value using a method described below, and sends threshold setting data, which is data specifying the threshold value, to the output unit 128. Details of the threshold value and how the threshold determination unit 127 determines the threshold value will be described later. The threshold determination unit 127 sends the threshold setting data to the output unit 128 at a timing described below.

[0048] The output unit 128 has a function of outputting necessary data generated by the functional blocks in the computer device 100 to the outside via the interface 114. As described above, the output unit 128 may receive trained autoencoder data and threshold setting data. Upon receiving these, the output unit 128 outputs the data to the interface 114.

[0049] The following describes how to use and the operation of the learning device. As described above, the learning device functions alternatively in one of two modes: a learning mode and a threshold determination mode. Therefore, to use the learning device, it is first necessary to determine in which mode the learning device will function. As will be described later, the threshold is determined depending on the characteristics of the trained autoencoder after training in the learning mode, and therefore the threshold cannot be determined before the trained autoencoder is completed. Therefore, the learning device will first execute the learning mode, and then execute the threshold determination mode.

[0050] To cause the learning device or computer device 100 to execute the learning mode, mode selection data is first input from the input device 102 of the computer device 100. The mode selection data selects the mode to be executed by the learning device from the two modes mentioned above. In this case, the user operates the input device 102 to input data from the input device 102 indicating that the learning device is to function in learning mode. This data is sent from the input device 102 via the interface 114 to the input unit 121, and from the input unit 121 to the main control unit 122. Upon receiving this data, the main control unit 122 instructs each functional block to execute the learning mode.

[0051] In this state, the learning device executes the learning mode. The microphone 103 has the function of collecting sounds, including sounds emitted by the object, and generating sound data, which is data on the collected sounds. Therefore, the microphone 103 is disposed in a position where it can at least sense the sounds emitted by the object. For example, if the object is a motor, the microphone 103 is disposed in a position where it can collect the sounds generated by the motor. However, the sounds collected by the microphone 103 when the learning mode is executed are sounds when the object is in a steady state. These sounds may include environmental sounds. This is acceptable as long as the sounds collected by the microphone when the detection device is later used also include environmental sounds. This means that the sound collection environment for the microphone 103 when the learning mode is executed is the same as when the detection device is used. The definition of a steady state can be determined as appropriate by the user operating the learning device. The microphone 103 generates sound data, which is data on collected or sensed sounds. The sound data is amplified by an amplifier as necessary and sent to the computer device 100 in the learning device, which is connected via a wire in this embodiment. The sound data is sent from the interface 114 to the input unit 121. The sound data is sent from the input unit 121 to the feature detection unit 123.

[0052] The feature detection unit 123 detects features of the received sound data and generates sound feature data, which is data regarding the sound features. The sound feature data may be in any format; however, in this embodiment, the sound feature data is a Mel frequency spectrogram, which is a publicly known or well-known data format for expressing sound features. The feature detection unit 123 first arranges the received sound data. In this embodiment, but not limited to, the feature detection unit 123 converts the sound data sent as analog data into digital data in a predetermined format (in this embodiment, but not limited to, WAV format), and arranges the time length of the converted data to a predetermined length (for example, 10 seconds or 30 seconds, or, in this embodiment, 60 seconds, but not limited to). A publicly known or well-known method can be used to obtain sound feature data, which is a Mel frequency spectrogram, from the arranged sound data. For example, a Mel frequency spectrogram can be obtained by converting a sound data file into power spectrum data by performing a short-time Fourier transform (STFT), applying a Mel filter bank to the converted data, and smoothing the data to which the Mel filter bank has been applied by performing a logarithmic operation. Of course, a Mel frequency spectrogram can also be obtained by other known methods. Note that the feature detection unit 123 may detect a delta Mel frequency spectrogram, which extracts the difference between previous and next frames of a Mel frequency spectrogram, in order to capture dynamic temporal changes in sound, rather than the sound itself at frequencies in the audible range. In any case, once sound feature data as a Mel frequency spectrogram has been generated, the feature detection unit 123 sends the generated sound feature data to the autoencoder 124 and the loss generation unit 125.

[0053] The autoencoder 124 receives input of sound feature data sent from the feature detection unit 123. More specifically, the sound feature data is input to the input layer of the autoencoder 124. The autoencoder 124 generates estimated sound feature data by encoding the input sound feature data and subsequently decoding the encoded data. Before encoding and decoding the sound feature data, the autoencoder 124 reads a dataset for generating the autoencoder 124 from the autoencoder recording unit 124A. This data reading is performed each time sound feature data is encoded and decoded. Therefore, the autoencoder 124 is in the latest state based on the latest data for generating the autoencoder 124 recorded in the autoencoder recording unit 124A. Before encoding and decoding the first sound feature data, the autoencoder 124 is a known, appropriate autoencoder 124. When sound feature data is input for the second or subsequent times, the autoencoder 124 is in a state where learning has progressed. As described above, the autoencoder 124 outputs estimated sound feature data. The autoencoder 124 sends the generated estimated sound feature data to the loss generation unit 125 .

[0054] As described above, the loss generation unit 125 receives sound feature data from the feature detection unit 123 and estimated sound feature data from the autoencoder 124. The sound feature data and estimated sound feature data are a set of data that correspond to the input and output, respectively, when the autoencoder 124 performs a single encoding and decoding process. The loss generation unit 125 calculates the loss by taking the difference between the set of sound feature data and the estimated sound feature data. When calculating the loss, the loss generation unit 125 uses a loss function, and the loss generation unit 125 reads data specifying the loss function from the loss function recording unit 125A at an appropriate timing before calculating the loss. It is sufficient to calculate the data specifying the loss function only once, before the first loss is calculated. The method of generating data regarding the loss between sound feature data and estimated sound feature data using a loss function has already been described. Using the loss function as already described, the loss generation unit 125 calculates the loss Loss ｉ As mentioned above, data on a large number of losses is generated. ｉ The set of data is data about loss generated from one sound data. ｉ An example of losses arranged in the x-axis direction of time is shown in Fig. 5. The loss generating unit 125 sends the loss data generated as described above to the loss recording unit 125B.

[0055] Every time new loss data is recorded in loss recording unit 125B, optimization unit 126 corrects autoencoder 124 so as to minimize the loss. More specifically, every time new loss data is recorded in loss recording unit 125B, optimization unit 126 in this embodiment reads from autoencoder recording unit 124A the data for generating autoencoder 124 that was recorded in autoencoder recording unit 124A at that time, i.e., data of multiple coefficients, and also reads from loss recording unit 125B all loss data that was previously recorded in loss recording unit 125B. Then, in this embodiment, optimization unit 126 calculates the read loss Loss ｉThe data for generating the autoencoder 124 that was previously recorded in the autoencoder recording unit 124A is then overwritten with the data for generating the autoencoder 124 newly generated by the correction, so that the sum of the coefficients is minimized.

[0056] While the learning device is running in the learning mode, the computer device 100 constituting the learning device repeatedly executes each process from the process of inputting sound data from the input unit 121 to the process of optimizing multiple coefficients included in the data for generating the autoencoder 124 recorded in the autoencoder recording unit 124A in order to minimize loss by the optimization unit 126, that is, the additional learning process. By repeating this process multiple times, for example, but not limited to, about 200 times (one time is defined as when all of the sound feature data provided in the input process has been used at least once), the loss function, which is the difference between the sound feature data as input to the autoencoder 124 and the estimated sound feature data as output from the autoencoder 124, will not be the same, but will become small enough to fall within a certain range. Since the sound data used for training is for sounds including sounds when the object is in a steady state, the autoencoder 124 is dedicated to the time when the object is in a steady state and has a function of making the sound feature data when the object is in a steady state and the estimated sound feature data created from that sound feature data almost identical. When the input-output relationship when the object is in a steady state satisfies the above-described relationship, the autoencoder 124 can be said to be completed as a trained autoencoder. As described above, a large amount of sound data is required to obtain a trained autoencoder, but this sound data only needs to be for sounds when the object is in a steady state. Since the time the object is in a steady state is generally overwhelmingly longer than the time the object is in a non-steady state, collecting such sound data is relatively easy. Furthermore, when collecting sound data, there is no need to collect sound data for sounds in a non-steady state, nor is there any need to collect sound data for sounds in multiple different stages of non-steady state. Furthermore, since the sound data used for training the autoencoder 124 is only steady-state data, there is no need to label the sound data. Furthermore, so-called noise canceling processing for removing, for example, environmental sounds from the sound data is not essential.In this embodiment, the sound feature data created from sound data input from the microphone 103 is sequentially input to the autoencoder 124. However, if a large amount of sound data is generated in advance and recorded on a recording medium (which may be built into the computer device 100 or external to the computer device 100), the sound data read by the computer device 100 from the recording medium can be sequentially sent to the feature detection unit 123, allowing the autoencoder 124 to perform learning in the same manner as described above. Similarly, if a large amount of sound feature data is recorded on a recording medium, the sound feature data can be sequentially supplied to the autoencoder 124, allowing the autoencoder 124 to perform learning in the same manner as described above. In other words, the generation of sound data by the microphone 103 and the learning of the autoencoder 124 do not need to be performed consecutively in time.

[0057] Once the trained autoencoder is generated as described above, the next step is to determine a threshold value. The threshold value is used by the detection device, as described below, when determining whether a sound identified by sound data input to the detection device as described below has non-stationary characteristics (whether it includes non-stationary sound). When the threshold value is determined, the learning device or the computer device 100 constituting the learning device executes the threshold determination mode, as described above. The user inputs mode selection data indicating the selection of the threshold determination mode from the input device 102 of the computer device 100. As described above, the mode selection data is sent from the input device 102 to the input unit 121 via the interface 114, and then from the input unit 121 to the main control unit 122. Upon receiving this data, the main control unit 122 instructs each functional block to execute the threshold determination mode. For example, the main control unit 122 instructs the threshold determination unit 127 to determine a threshold value.

[0058] In this state, the learning device executes the threshold determination mode. The processing when the threshold determination mode is executed is as follows. The threshold determination unit 127 reads out from the loss recording unit 125B data on all losses that were generated in the learning process of the autoencoder 124 and recorded in the loss recording unit 125B. As described above, each piece of data on losses is stored in a number of Loss ｉ In this embodiment, the loss recording unit 125B is a set of data for multiple losses. ｉ Among these, the maximum loss that occurs when the object is in a steady state is ｉ In other words, the maximum loss that can be tolerated from the loss obtained from the sound emitted by the object is determined so that the object can be considered to be in a steady state. The value of such a loss can be determined, for example, by statistical processing. In this embodiment, a large number of Loss ｉ The third quartile of the Loss ｉ When the values ​​are arranged from smallest to largest, the value of the value in the order of 3 / 4 from smallest to largest is multiplied by 1.5 to determine the maximum loss value that occurs when such an object is in a steady state. ｉ The value obtained in this way can be used as the reference threshold (Loss max ) in this embodiment. Although not limited to this, the threshold value determination unit 127 further determines several threshold values ​​based on this reference threshold value. Next, a new threshold value is determined by multiplying the reference threshold value by a certain coefficient. The coefficient by which the reference threshold value is multiplied is set to a value greater than 1. Therefore, this new threshold value is greater than the reference threshold value. This new threshold value is referred to as an upper limit loss threshold (Loss limit) The upper loss threshold conceptually corresponds to the value of the largest expected loss among losses occurring in an unsteady state (naturally, losses occurring in an unsteady state are generally larger than losses occurring in a steady state). The above-mentioned coefficient by which the reference threshold is multiplied is determined so that the value of the upper loss threshold corresponds to this conceptual meaning. This coefficient is generally set to a value between 1.5 and 3.0. However, the value of the coefficient does not need to be within the above range as it may depend on the environment of the measurement target. In this embodiment, although not limited to this, a further threshold is set between the reference threshold and the upper loss threshold. This threshold is called the unsteady intermediate threshold (Loss interH ) The conceptual meaning of the non-steady state intermediate threshold is as follows: As mentioned above, the reference threshold is the maximum loss value that occurs when the object is in a steady state. Therefore, a loss value that exceeds the reference threshold suggests that the object is in a non-steady state. However, the strength of the indication that the object is in a non-steady state varies depending on whether the loss value exceeds the reference threshold slightly or greatly. The non-steady state intermediate threshold has the following implication: a loss value that falls between the reference threshold and the non-steady state intermediate threshold is a relatively weak indication that the object is in a non-steady state, while a loss value that falls between the non-steady state intermediate threshold and the upper loss threshold is a relatively strong indication that the object is in a non-steady state. The non-steady state intermediate threshold that has this conceptual meaning can be calculated, for example, using the following equation (Equation 9): Loss interH = Loss max +1 / 2 (Loss limit -Loss max ) (Equation 9) In this embodiment, there is one non-stationary intermediate threshold. max That's all, Loss limit The following region is divided into two regions with a non-stationary intermediate threshold in between. max That's all, Loss limitThe following regions can also be divided into two or more. For example, if the region is divided into three regions, two unsteady intermediate thresholds should be set; if the region is divided into four regions, three unsteady intermediate thresholds should be set; if the region is divided into n regions, n-1 unsteady intermediate thresholds should be set. Although not limited to this, in this embodiment, the threshold determination unit 127 also determines a threshold for a region where the value is smaller than the reference threshold. If the loss value is smaller than the reference threshold, it is suggested that the object is in a steady state. However, the strength of the suggestion that the object is in a steady state changes depending on whether the loss is slightly smaller than the reference threshold or small enough to be close to 0. Just as the unsteady intermediate threshold is used to divide the region between the reference threshold and the upper loss threshold into multiple regions, a threshold is set to divide the region where the loss is between 0 and the reference threshold. This threshold is called the steady intermediate threshold (Loss interL ) The significance of the steady-state intermediate threshold in this embodiment is as follows. As mentioned above, a loss whose value is smaller than the reference threshold suggests that the object that emitted the sound that generated the loss is in a steady state. However, if the loss value is smaller than the reference threshold but is, for example, large enough to be close to the reference threshold, it raises the suspicion that the object is in a non-steady state. A reasonable threshold for raising such suspicion is set as the steady-state intermediate threshold. A steady-state intermediate threshold that conceptually has such significance can be calculated, for example, by the following equation (Equation 10). Loss interL = Loss max × 0.75 (Equation 10) As described above, in this embodiment, the threshold value determination unit 127 generates the reference threshold value, the upper limit loss threshold value, the unsteady intermediate threshold value, and the steady intermediate threshold value. The threshold value determination unit 127 generates threshold value setting data, which is data for setting each of these threshold values ​​in the detection device.

[0059] In this way, the autoencoder 124 becomes a trained autoencoder 124, and the thresholds used in the trained autoencoder 124 are determined. These are then transferred to the detection device described below for use. The process of transferring this data from the training device to the detection device, or the process of transferring this data from the computer device 100 included in the training device to the computer device included in the detection device, may be performed appropriately using publicly known or well-known techniques. For example, in accordance with input input from the input device 102, through the interface 114, and the input unit 121 to the main control unit 122, the main control unit 122 may send an instruction to the optimization unit 126 to read a set of data for generating the trained autoencoder 124 from the autoencoder recording unit 124A and send it to the output unit 128, and may also send an instruction to the threshold determination unit 127 to send data for the determined thresholds to the output unit 128. If this is the case, data from the trained autoencoder 124 and threshold data are sent from the output unit 128 to the interface 114. For example, this data is sent from the interface 114 to the computer device 100 included in the detection device via a predetermined cable. Alternatively, this data is recorded from the interface 114 to a recording medium connected to the interface 114 and sent to the computer device 100 included in the detection device via the recording medium.

[0060] <Regarding the Detection Device> As already mentioned, the learning device and the detection device may be identical in terms of hardware configuration, even including their peripheral devices. While not limited to this, this is the case in this embodiment. The detection device includes a computer device 100X having a hardware configuration equivalent to that of the computer device 100 in the learning device, a display 101X equivalent to the display 101 in the learning device, an input device 102X equivalent to the input device 102 in the learning device, and a microphone 103X equivalent to the microphone 103 in the learning device, each connected to the computer device 100X. The computer devices 100 and 100X, the displays 101 and 101X, the input devices 102 and 102X, and the microphones 103 and 103X can all be equivalent, but may also be completely identical. Furthermore, if the learning device includes an amplifier for amplifying sound data, the detection device also includes an amplifier. These amplifiers may also be equivalent, or may be completely identical. If the detection device were illustrated, it would be the same as if each symbol in FIG. 1 had an X added to it, so the detection device is not shown.

[0061] As described above, the hardware configuration of the computer 100X of the detection device is equivalent to the hardware configuration of the computer 100 of the learning device. The computer X of the detection device includes a CPU 111X, a ROM 112X, a RAM 113X, an interface 114X, and a large-capacity recording medium 115X, which are interconnected by a bus 116X. From a hardware perspective, the functions of the CPU 111X, ROM 112X, RAM 113X, interface 114X, large-capacity recording medium 115X, and bus 116X provided in the computer 100 of the detection device are equivalent to or identical to the functions of the CPU 111, ROM 112, RAM 113, interface 114, large-capacity recording medium 115, and bus 116 provided in the computer 100 of the learning device. Note that if the hardware configuration of the computer device 100X of the detection device were illustrated, it would simply be the same as adding an X to each reference symbol in Figure 2, and therefore the illustration of the hardware configuration of the computer device 100X of the detection device will be omitted. The only difference between the computer device 100X of the detection device and the computer device 100 of the learning device is that the computer programs recorded in the ROM 112X or large-capacity recording medium 115X of the computer device 100X of the detection device are different from the computer programs recorded in the computer device 100 of the learning device. The computer programs recorded in the computer device 100X of the detection device include computer programs for causing the computer device 100X to execute the processing described below that is necessary for the computer device 100X to function as a detection device.However, just as in the case of computer device 100 or a computer program installed on computer device 100, this computer program may be pre-installed or post-installed on computer device 100X, and the computer program may be installed on computer device 100X via a recording medium or via a network, and further, in addition to the above-mentioned computer program, an OS and other necessary computer programs may be installed on computer device 100X.

[0062] When the CPU 111X executes the computer program, functional blocks such as those shown in Fig. 6 are generated within the computer device 100X. Note that the functional blocks below may be generated by the functions of the above-described computer program alone, which causes the computer device 100X to execute the processes described below that are necessary for the computer device 100X to function as a detection device, or may be generated by cooperation between the above-described computer program and an OS or other computer programs installed in the computer device 100. Within the computer device 100, an input unit 121X, a main control unit 122X, a feature detection unit 123X, a first calculation unit 124X, a loss generation unit 125X, a state determination unit 126X, a first recording unit 127X, and an output unit 128X are generated in relation to the functions of the present invention (Fig. 6). Among these, the functional blocks that perform information processing, i.e., the main control unit 122X, the feature detection unit 123X, the first calculation unit 124X, the loss generation unit 125X, and the state determination unit 126X, correspond to a calculation device (CPU 111X in this embodiment) in terms of hardware, or are realized by the functions of the calculation device that executes information processing in accordance with the instructions of the computer program described above. The input unit 121X and the output unit 128X correspond to the interface 114X in terms of hardware, or are realized by the functions of the interface 114X. More specifically, the input unit 121X and the output unit 128X conceptually correspond to the connection between the interface 114X and the bus 116X. Furthermore, the functional block that records data, i.e., the first recording unit 127X, corresponds to a recording device, such as RAM 113X or a large-capacity recording medium 115X, in terms of hardware, or is realized as a part of such a recording device. Furthermore, the combination of the loss generating unit 125X and the state determining unit 126X corresponds to the second calculating unit in this application.

[0063] The input unit 121X receives input from the interface 114X. Inputs from the interface 114X to the input unit 121X include input from the input device 102X. The input from the input device 102X, which will be described in detail later, includes, for example, threshold setting data. When threshold setting data is input from the input device 102X, the threshold setting data is sent from the input unit 121X to the main control unit 122X. Data input from the interface 114X to the input unit 121X also includes sound data from the microphone 103. When sound data is received by the input unit 121X, the input unit 121X sends the sound data to the feature detection unit 123X. The main control unit 122X performs overall control of the above-mentioned functional blocks generated within the computer device 100X. For example, the main control unit 122X may receive threshold setting data as described above. The main control unit 122X that receives the threshold setting data sends it to the state determination unit 126X.

[0064] The feature detection unit 123X has functions equivalent to those of the feature detection unit 123 of the computer device 100 of the learning device. That is, it detects features of the received sound data and generates sound feature data, which is data on the sound features. Like the feature detection unit 123 of the learning device, the feature detection unit 123X prepares the sound data as necessary and then executes feature detection processing. The sound feature data generated by the feature detection unit 123X is in the same format as the sound feature data created by the feature detection unit 123. Therefore, in this embodiment, the sound feature data generated by the feature detection unit 123X is, but is not limited to, a Mel frequency spectrogram. The feature detection unit 123X sends the generated sound feature data to the first calculation unit 124X and the loss generation unit 125X.

[0065] The first calculation unit 124X effectively functions as a trained autoencoder 124 created by a learning device. The first recording unit 127X records a set of data, i.e., a set of multiple coefficients, for generating the trained autoencoder 124 created by the learning device. When performing the calculations described below, the first calculation unit 124X reads trained autoencoder data from the first recording unit 127X and functions as a trained autoencoder itself. Since the first calculation unit 124X functions as a trained autoencoder completed by a learning device, it has the function of encoding input data, which is predetermined data, and then decoding the encoded data to restore it to data of the same dimension as the input data. In other words, the first calculation unit 124X functions as an autoencoder. In this embodiment, the data input to the virtual trained autoencoder present in the first calculation unit 124X is sound feature data input from the feature detection unit 123X. The first calculation unit 124X functioning as a trained autoencoder is configured to output estimated sound feature data when sound feature data is input, just like the trained autoencoder 124 of the learning device. The first calculation unit 124X is configured to send the generated estimated sound feature data to the loss generation unit 125X.

[0066] The loss generation unit 125X has the same functions as the loss generation unit 125 present in the computer device 100 of the learning device. The first recording unit 127X records data for specifying a loss function that is the same as that recorded in the loss function recording unit 125A. When calculating a loss as described below, the loss generation unit 125X reads data for specifying a loss function from the first recording unit 127X at least once prior to the calculation, and calculates a loss using the same loss function as that used in the learning device based on the data. As described above, the loss generation unit 125X receives sound feature data from the feature detection unit 123X and estimated sound feature data from the first calculation unit 124X. The sound feature data and estimated sound feature data are, in short, a set of data that corresponds to the input and output, respectively, of a single encoding and decoding process performed by an autoencoder generated by the function of the first calculation unit 124X. The loss generation unit 125X is configured to calculate the loss between the set of sound feature data and the estimated sound feature data, and generate data regarding the loss, using the same loss function as in the case of the loss generation unit 125 in the computer device 100 of the learning device. The loss generation unit 125X is configured to send the generated loss data to the state determination unit 126X.

[0067] The state determination unit 126X has a function of determining, based on the loss data received from the loss generation unit 125X, whether the sound identified by the sound data that caused the loss included a sound indicating that the object was in an unsteady state (whether it was an unsteady sound) or whether the object that emitted the sound identified by the sound data was in an unsteady state. How such a determination is made will be described later. Note that threshold setting data generated by a learning device is recorded in the first recording unit 127X. Prior to making the above-mentioned determination, the state determination unit 126X reads the threshold setting data from the first recording unit 127X and uses the thresholds generated by the threshold setting data, i.e., the reference threshold, upper limit loss threshold, unsteady intermediate threshold, and steady intermediate threshold, for the above-mentioned determination. If the state determination unit 126X determines that the sound identified by the sound data that caused the loss included a sound indicating that the object was in an unsteady state or that the object that emitted the sound identified by the sound data was in an unsteady state, the state determination unit 126X generates result data indicating that fact. Note that result data may be generated only when such a determination is made. However, result data indicating this may also be generated when it is determined that the sound identified by the sound data that caused the loss does not include a sound indicating that the object is in an unsteady state (i.e., a steady sound), or that the object that emitted the sound identified by the sound data is not in an unsteady state (i.e., is in a steady state). This is not limited to this, but is the case in this embodiment. The result data generated in this case is, of course, different from the result data generated when it is determined that the object that emitted the sound identified by the sound data is in an unsteady state. The state determination unit 126X outputs the generated result data to the output unit 128X.

[0068] The output unit 128X has a function of outputting necessary data generated by the functional blocks in the computer device 100X to the outside via the interface 114X. As described above, the output unit 128X may receive result data. Upon receiving the result data, the output unit 128X outputs the result data to the interface 114X.

[0069] The following describes the use and operation of the detection device. Like the microphone 103 included in the learning device, the microphone 103X collects sounds, including sounds emitted by an object, and generates sound data, which is data on the collected sounds. The location of the microphone 103X corresponds to the location of the microphone 103 of the learning device. When the microphone 103X of the detection device collects sounds, it is not clear whether the object is in a steady state or an unsteady state. Therefore, the sound collected by the microphone 103X may be either a steady sound or an unsteady sound. This is natural given the nature of the detection device, since the detection device detects when an object that is originally expected to be in a steady state enters an unsteady state. Therefore, the sound collected by the microphone 103X may include sounds emitted by an object in a steady state or sounds emitted by an object in an unsteady state. The microphone 103X generates sound data, which is data on collected or sensed sounds. The sound data is amplified by an amplifier as needed and sent to the computer device 100X, which in this embodiment is connected by wire.

[0070] From this point on, until the loss function is generated by the loss generation unit 125X, the processing is the same as the processing up to the time when the loss function is generated by the loss generation unit 125 in the computer device 100 of the learning device, and the processing conditions such as the specifications of each data, the method of generating sound feature data, and the method of generating the loss function are all the same.

[0071] The sound data is sent from the interface 114X to the input unit 121X and then to the feature detection unit 123X. The sound data is arranged in the feature detection unit 123X. In this embodiment, the sound data is converted into a 60-second wav-format file, as was done in the learning device. The arranged sound data is sent from the input unit 121X to the feature detection unit 123X. The feature detection unit 123X detects features of the arranged sound data and generates sound feature data, which is data on the sound features. The feature detection unit 123X generates sound feature data, which is a Mel-frequency spectrogram, from the sound data in the same way as the feature detection unit 123 in the computer device 100 in the learning device. If what is detected by the learning device is not the sound itself at a frequency in the audible range, but a delta mel frequency spectrogram that extracts the difference between previous and next frames of a mel frequency spectrogram in order to capture dynamic temporal changes in sound, the feature detected by the feature detection unit 123X may be a delta mel frequency spectrogram. Once the sound feature data as a mel frequency spectrogram has been generated, the feature detection unit 123X sends the generated sound feature data to a first calculation unit 124X that functions as a trained autoencoder, and also sends it to a loss generation unit 125X.

[0072] The sound feature data sent from the feature detection unit 123X is input to the trained autoencoder in the first calculation unit 124X. Then, the first calculation unit 124X serving as a trained autoencoder outputs estimated sound feature data. The estimated sound feature data output from the first calculation unit 124X is sent to the loss generation unit 125X.

[0073] The loss generation unit 125X receives sound feature data from the feature detection unit 123X and estimated sound feature data from the first calculation unit 124X. The sound feature data and estimated sound feature data are a set of data that corresponds to the input and output when an encoding and decoding process is performed once by the autoencoder in the first calculation unit 124X. The loss generation unit 125X generates a loss for the set of sound feature data and estimated sound feature data. The method by which the loss generation unit 125X generates a loss is the same as the method by which the loss generation unit 125 of the computer device 100 in the learning device generates a loss. The loss generation unit 125X sends the loss data generated as described above to the state determination unit 126X.

[0074] Based on the received loss data, the state determination unit 126X determines whether the sound identified by the sound data that is the source of the loss includes a sound that indicates that the object is in an unsteady state, or whether the object that emitted the sound identified by the sound data is in an unsteady state. The determination method is as follows. As described above, the trained autoencoder included in the first calculation unit 124X is trained only on sounds (steady sounds) that include sounds produced by objects in a steady state, and when sound feature data generated from a steady sound (a Mel-frequency spectrogram in this embodiment) is input, it outputs estimated sound feature data that is almost identical to the input sound feature data (a Mel-frequency spectrogram that could be called an "estimated Mel-frequency spectrogram"). On the other hand, the trained autoencoder included in the first calculation unit 124X has not been trained on sounds (unsteady sounds) including sounds generated from objects in an unsteady state, and when sound feature data generated from unsteady sound is input, the output estimated sound feature data is significantly different from the input sound feature data. Therefore, the loss corresponding to the difference between the sound feature data and the estimated sound feature data is relatively small in the former case and relatively large in the latter case. Therefore, based on the magnitude of the loss, the state determination unit 126X can distinguish whether the object is in a steady state (whether the sound input to the microphone 103X is a steady sound) or whether the object is in an unsteady state (whether the sound input to the microphone 103X is a unsteady sound).

[0075] In this embodiment, although not limited thereto, the distinction is made using thresholds set in the state determination unit 126X as described above: a reference threshold, an upper limit loss threshold, an unsteady intermediate threshold, and a steady intermediate threshold. To use these thresholds, the state determination unit 126X reads threshold setting data from the first recording unit 127X before making a determination. In this embodiment, the state determination unit 126X uses the reference threshold, the upper limit loss threshold, the unsteady intermediate threshold, and the steady intermediate threshold to determine whether sound data about a sound emitted by an object includes a sound indicating that the object is in an unsteady state, or whether the object that emitted a sound identified by the sound data is in an unsteady state, as follows. Assume that the loss according to the loss data received by the state determination unit 126X is the same as that shown in FIG. 5. Then, the reference threshold (Loss max ), upper loss threshold (Loss limit ), non-stationary intermediate threshold (Loss interH ), stationary intermediate threshold (Loss interL ) is plotted in FIG. 7. As a result, the vertical axis relating to loss is divided into two sections in the range above the reference threshold and below the upper loss threshold, and into two sections in the range below the reference threshold (see the thick dashed lines in FIG. 7). The state determination unit 126X also finely divides the vertical axis direction of the graph into range values ​​indicated by thin lines that separate the regions. The intervals between adjacent range values ​​above and below, and the intervals between a certain range value and its adjacent reference threshold, upper loss threshold, unsteady intermediate threshold, or steady intermediate threshold are generally constant (see the thick dashed lines and thin dashed lines in FIG. 7). The range values ​​are determined, for example, as follows. In this embodiment, the steady intermediate threshold (Loss interL ) into which the area below the Loss threshold is to be divided is determined. For example, in this embodiment, seven range values ​​are determined to divide the range into eight equal parts. Then, the steady-state intermediate threshold (Loss interL The interval between adjacent range values ​​is determined below the range of the LossinterL ) and the reference threshold (Loss max ) and the reference threshold (Loss max ) and the non-stationary intermediate threshold (Loss interH ) and the non-stationary intermediate threshold (Loss interH ) and upper loss threshold (Loss limit ) and the stationary intermediate threshold (Loss interL ) and the reference threshold (Loss max ) are divided into three by two range values ​​(Range value), and the reference threshold (Loss max ) and the non-stationary intermediate threshold (Loss interH ) and the non-stationary intermediate threshold (Loss interH ) and upper loss threshold (Loss limit ) is divided into two regions by one range value. In this way, the region in FIG. 7 is divided into two regions by one range value. max ), upper loss threshold (Loss limit ), non-stationary intermediate threshold (Loss interH ), stationary intermediate threshold (Loss interL ) and range values ​​(Range values) into 15 regions (ranges). In Fig. 7, each region sandwiched between adjacent thick and thin lines is a range. Naturally, the intervals between range values ​​and the number of range divisions can be set using parameters according to the characteristics of the target device.

[0076] In this way, the reference threshold (Loss max ), upper loss threshold (Loss limit ), non-stationary intermediate threshold (Loss interH ), stationary intermediate threshold (Loss interLOnce the upper limit loss threshold (Loss ) and the ranges are determined, the state determination unit 126X counts the number of losses of magnitudes that belong to each range. For example, in FIG. 7, losses marked with an X belong to the second range from the top. It is possible to determine whether a loss at the boundary value of an adjacent region (i.e., a value equal to any threshold or range value) should be counted in the upper or lower range, as appropriate, and although not limited to this, in this embodiment, it is counted in the upper range where the loss is large. In addition, an upper limit loss threshold (Loss ) is also determined. limit The rules for counting the number of losses can be determined as appropriate, such as counting in the top range if there is a loss greater than 0, or counting in the bottom range if there is a loss less than 0.

[0077] The number of losses counted in this way is shown in graphs in Figures 8 and 9. The horizontal axis of this graph represents the magnitude of the loss, and the vertical axis represents the number of losses. max ), upper loss threshold (Loss limit ), non-stationary intermediate threshold (Loss interH ), stationary intermediate threshold (Loss interL ) is shown. Scales without annotations are range values. The range values ​​may be generated by the detection device, or may be generated by the learning device, for example, by including them in the threshold setting data. Figure 8 is a typical histogram when the object is in a steady state. Figure 9 is a typical histogram when the object is in a non-steady state. In the case of Figure 8, most of the losses are below the reference threshold (Loss max ) and the values ​​are concentrated below the reference threshold (Loss max) there is almost no loss greater than the reference threshold (Loss max ) in the range of values ​​higher than the non-stationary intermediate threshold (Loss interH ) is also present in a considerable number. This means that there are many losses of a certain magnitude or more, in other words, that there is a large difference between the sound feature data used to calculate the loss and the estimated sound feature data. More specifically, this means that the sound data that was the basis for the sound feature data generated within the detection device is different from the sound data used when the first calculation unit 124X functioning as an encoder learned (i.e., sound data based on the sound emitted by an object in a steady state). This means that the sound from the object currently being determined by the detection device is sound from an object in a non-steady state. The state determination unit 126X can determine whether the object is in a steady state or a non-steady state, for example, by the following arrangement. 1. Reference threshold (Loss max ) or more than XX losses, or XX% or more of the total number of losses. 2. Non-stationary intermediate threshold (Loss interH ) or more than XX losses, or XX% or more of the total number of losses. 3. Criteria Threshold (Loss max ) is XX or more, or XX% or more of the total number of losses, and the steady-state intermediate threshold (Loss interL) or more, or YY% or more of the total number of losses. The state determination unit 126X may determine whether the object is in a steady state or an unsteady state based on the received losses using the above algorithm, or may perform this determination using a predetermined artificial intelligence. In either case, the state determination unit 126X generates image data for displaying a histogram such as that shown in FIG. 8 or FIG. 9 on a predetermined display together with text such as "The object is in a steady state" or "The object is in an unsteady state." This corresponds to the result data referred to herein. While not limited to this, in this embodiment, the state determination unit 126X performs a two-stage determination. However, the state determination unit 126X may also perform a three-stage or five-stage determination, ranging from whether the object is in a steady state to whether it is in an unsteady state. Furthermore, it is not essential that the result data include the above-described text data or image data for displaying a histogram on a predetermined display.

[0078] As described above, the state determination unit 126X generates result data according to the determination result. The result data may be generated only when it is determined that the object is in an unsteady state or that the sound input to the microphone 103X includes unsteady sound, or it may not be generated otherwise. In this embodiment, although not limited thereto, result data is generated both when it is determined that the object is in a steady state or that the sound input to the microphone 103X is a steady sound and when it is determined that the sound is an unsteady sound.

[0079] The result data is used as needed. In this embodiment, but not limited to, the state determination unit 126X sends the result data to the output unit 128X. The output unit 128X outputs the result data to another device external to the computer device 100X via the interface 114X. The other device is, for example, a display (not shown) connected to the computer device 100X. The display screen displays the above-described text and histogram corresponding to the result data. Another example of the other device may be a patrol lamp connected to the computer device 100X that sounds upon receiving the result data to notify nearby people that the object is in an unsteady state. In this case, it is obvious that the result data is data for operating the patrol lamp and does not need to include the above-described text or image data of the histogram. It goes without saying that the means for notifying nearby people that the object is in an unsteady state is not limited to a patrol lamp. The above-described display and patrol lamp do not need to be connected to the computer device 100X. For example, the above-mentioned display and patrol lamp may be connected to another computer device that is located remotely from computer device 100X and can communicate with computer device 100X, for example, via the Internet. In this case, it will be obvious to those skilled in the art that the above-mentioned text and histogram images can be displayed on the display that receives the result data, and that the patrol lamp that receives the result data can be sounded.

[0080] In this embodiment, the threshold value set in the state determination unit 126X of the detection device using the threshold setting data generated by the learning device and used by the state determination unit 126X to make the above-mentioned determination based on the received loss data is a reference threshold value (Loss max ), upper loss threshold (Loss limit ), non-stationary intermediate threshold (Loss interH ), stationary intermediate threshold (Loss interL) and furthermore, range values ​​similar to the thresholds are also used. However, this is not limited to this. For example, if the thresholds generated by the learning device and specified by the threshold setting data are, for example, a reference threshold (Loss max In this case, the state determination unit 126X of the detection device may simply count the number of losses that exceed a reference threshold, and if that number (or a percentage of the total losses) is greater than a certain number (or percentage), determine that the sound data about the sounds emitted by the object includes a sound that indicates that the object is in an unsteady state.

[0081] <Test Example> A test example will be described below. The test example described below was conducted using the detection device described in the first embodiment, which had data from a trained autoencoder that had been trained by executing the learning mode using the learning device described in one embodiment. In the test example, a test jig (hereinafter referred to as the test device) simulating an electric drill with a rotation mechanism and capable of intentionally generating abnormal vibrations was used as the object for which steady-state and unsteady-state determination was performed. This test device can reproduce not only the normal (steady) state of an electric drill with a rotation mechanism, but also an abnormal (unsteady) state. This test device is commonly used as a training device for workers to learn factory equipment maintenance techniques, for purposes such as predictive maintenance. The types of abnormalities that the test device can reproduce will be described later.

[0082] Prior to determining whether the test device was in a steady state or non-steady state using the detection device, the autoencoder was trained using a learning device. Naturally, the normal environment of the test device was used during training using the learning device. The sound generated by a normal test device was used as the steady sound. For the steady sound sampled, the sampling rate was 44,100 Hz, the FFT frame length was 4,096 sample data pieces, the FFT frame interval was 2,048 sample data pieces, the sound feature data was a mel frequency spectrogram, and the number of mel band filters for the mel frequency spectrogram was 8. The FFT frame length refers to a parameter that controls frequency resolution by the number of reference data pieces processed when performing fast Fourier transform processing, and the FFT frame interval refers to the number of sample data pieces that take a time interval when performing the same processing, and is a parameter that controls time resolution. Steady sound was recorded for three 30-minute periods for each of three rotation speed patterns of the rotary drive unit: low speed (100 rpm), medium speed (500 rpm), and high speed (1000 rpm). By using this steady sound data for learning, three types of trained autoencoders were obtained for equipment with different rotation speeds.

[0083] Next, the data for generating the trained autoencoder was transferred to a detection device to complete the detection device, and the steady state and non-steady state of the test device were determined using the detection device. The threshold value used in the detection device was determined by executing the threshold determination mode in the learning device described in the embodiment. The threshold value used in the test example was the reference threshold (Loss) described in the above embodiment. max ) is only one type. However, the reference threshold (Loss max) was determined as the third quartile of all losses obtained when the steady sound data used was input into the learning device and inference was performed, i.e., the value of the value in the order of 3 / 4 from the smallest to the largest when all losses were sorted from the smallest to the largest, multiplied by 1.5. Furthermore, the state determination unit 126X of the detection device determines whether the object is in a steady state or an unsteady state based on whether the value of losses exceeding the reference threshold is greater or less than a predetermined number. In other words, in the detection device in the test example, if the number of losses exceeding the reference threshold is greater than a predetermined number, the object is determined to be in an unsteady state. Here, the "predetermined number" beyond which the object is determined to be in an unsteady state when the number of losses exceeding the reference threshold exceeds that number was determined to be a predetermined ratio (for example, but not limited to, 10%) of the number of excess samples to the number of input data to be diagnosed. That is, when there are Z pieces of loss value data, if the number of loss value data exceeding the reference threshold is 0.1Z or more, the state determination unit 126X determines that the object is in an unsteady state.

[0084] As mentioned above, the test equipment used for judgment by the detection device can reproduce both a normal state, that is, a steady state, and an abnormal state, that is, a non-steady state, including the sounds emitted. The following five types of non-steady state test equipment were prepared. 1. Gear abnormality: A test equipment with a defect in a gear included in the rotation mechanism was used. 2. Bearing abnormality: A test equipment with a defect in a bearing included in the rotation mechanism was used. 3. Rotation mechanism angle abnormality: The angle of the rotation axis of the rotation mechanism was shifted from the original rotation axis 4. Gear contact abnormality: The distance between meshing gears was slightly shifted from the original distance 5. Eccentricity abnormality: The center of gravity of a rotating member included in the rotation mechanism was shifted from the rotation axis, causing an unbalanced load and imbalance.

[0085] First, the sound generated by a normal test device was sampled as the measurement sound to determine whether the test device was in a steady state or an unsteady state. The sampling conditions were the same as those for sampling steady sounds during learning. The measurement sound was sampled three times for 10 minutes each for the rotation speeds of the rotary drive unit: low (100 rpm), medium (500 rpm), and high (1000 rpm). The measurement sound feature data was a Mel frequency spectrogram, and the Mel frequency spectrogram had eight Mel band filters. As a result, based on the measurement sound generated by the normal test device, the detection device determined that the test device was in a steady state for all three samplings of the measurement sound, regardless of the rotation speed of the test device: low (100 rpm), medium (500 rpm), or high (1000 rpm). As in the following examples, the trained autoencoder used by the detection device when determining whether the test device is in a steady state or a non-steady state at low speed is a trained autoencoder that has been trained using the sound emitted by the test device at low speed, and similarly, the trained autoencoder used by the detection device when determining whether the test device is in a steady state or a non-steady state at medium and high speed is a trained autoencoder that has been trained using the sound emitted by the test device at medium and high speed, respectively.

[0086] Next, the sound generated by the test device with any of the above-mentioned abnormalities 1 to 5 was sampled as the measurement sound to determine whether the test device was in a steady state or an unsteady state. The sampling conditions were the same as those for sampling the steady sound during learning. Measurement sound sampling was performed for 10 minutes three times for each of the test devices with abnormalities 1 to 5, with the rotation speeds of the rotary drive unit being low (100 rpm), medium (500 rpm), and high (1000 rpm). The measurement sound feature data was a Mel frequency spectrogram, and the number of Mel band filters for the Mel frequency spectrogram was eight. As a result, based on the measurement sound generated by the test device with abnormality 1, the detection device determined that the test device was in an unsteady state for all three times the measurement sound was sampled, regardless of the rotation speed of the rotary drive unit being low (100 rpm), medium (500 rpm), or high (1000 rpm). Similarly, based on the measurement sounds generated by the test device with abnormalities 2, 3, and 4, the detection device determined that the test device was in an unsteady state all three times the measurement sounds were sampled, regardless of whether the rotation speed of the test device was low (100 rpm), medium (500 rpm), or high (1000 rpm). Based on the measurement sounds generated by the test device with abnormality 5, the detection device determined that the test device was in an unsteady state all three times the measurement sounds were sampled, regardless of whether the rotation speed of the rotary drive unit was medium (500 rpm) or high (1000 rpm). On the other hand, when the rotation speed of the rotary drive unit was low (100 rpm), based on the measurement sounds generated by the test device with abnormality 5, the detection device determined that the test device was in a steady state all three times the measurement sounds were sampled, and was unable to detect that the test device was in an unsteady state.

[0087] REFERENCE SIGNS LIST 100 Computer device 101 Display 102 Input device 103 Microphone 100X Computer device 101X Display 102X Input device 103X Microphone 121 Input unit 122 Main control unit 123 Feature amount detection unit 124 Autoencoder 125 Loss generation unit 126 Optimization unit 127 Threshold determination unit 128 Output unit 121X Input unit 122X Main control unit 123X Feature amount detection unit 124X First calculation unit 125X Loss generation unit 126X State determination unit 127X First recording unit 128X Output unit

Claims

1. A trained autoencoder that encodes input data, which is predetermined data, and then decodes the encoded data to restore it to data of the same dimension as the input data, wherein the input data is steady sound feature data that is data on the feature of steady sound identified by steady sound data generated from steady sound data, which is data for a specific length of time about steady sound including sounds generated in a steady state from an object, including a rotating object, whose non-stationarity is to be detected based on the sound, and the output data is estimated steady sound feature data for the specific length of time, and wherein a plurality of steady sound feature data are input, and a loss between the steady sound feature data that is the input data and the estimated steady sound feature data that is the output data for the input data is calculated using a loss function, and the autoencoder is pre-trained to minimize the loss, the loss function is calculated based on noise-removed data, which is an output when a plurality of difference data, which is data based on the difference between the steady sound feature data and the estimated steady sound feature data, is input into a noise removal function, which is a function for removing noise from the difference data, and on trend data, which is an output when the difference data is input into a trend extraction function, which is a function for extracting a trend in a predetermined time period of the difference data.

2. The trained autoencoder according to claim 1, wherein the steady-state sound feature data is a Mel-frequency spectrogram generated from the steady-state sound data.

3. The trained autoencoder according to claim 1, wherein the noise removal function is one of a function that functions as a low-pass filter, a function that calculates a moving average, and a function that performs a Fourier transform, removes frequency band signals that exceed a predetermined threshold, and then performs an inverse Fourier transform to return the signal.

4. The trained autoencoder according to claim 1, wherein the trend extraction function is a function that calculates the median of the difference data and multiple pieces of data before, after, or both before and after the difference data.

5. The trained autoencoder according to claim 1, wherein the trend extraction function is a function that takes a difference series of the difference data.

6. A trained autoencoder according to any one of claims 1 to 3, wherein the trend extraction function is a function that calculates the median of the difference data and multiple pieces of data before, after, or both before and after the difference data, and the loss function calculates the loss as the difference between the noise-removed data and the trend data.

7. A trained autoencoder according to any one of claims 1 to 3, wherein the trend extraction function is a function that takes a difference series of the difference data, and the loss function calculates the loss by applying the noise removal function to the trend data.

8. A method for converting an autoencoder that encodes input data, which is predetermined data, and then decodes the encoded data to restore data of the same dimension as the input data into a trained autoencoder, the method comprising the steps of: as input data, steady sound feature data, which is data on features of steady sound identified by steady sound data generated from steady sound data, which is data for a specific length of time about steady sound that is sound generated in a steady state from an object, the object being a target of sound-based detection of non-stationarity, and output data, which is estimated steady sound feature data for the specific length of time; inputting a plurality of steady sound feature data, and calculating a loss between the steady sound feature data that is input data and the estimated steady sound feature data that is output data for the input data using a loss function, and performing pre-training so as to minimize the loss; the loss function is determined based on noise-removed data, which is an output when a plurality of difference data, which is data based on the difference between the steady sound feature data and the estimated steady sound feature data, is input into a noise removal function, which is a function for removing noise from the difference data, and on trend data, which is an output when the difference data is input into a trend extraction function, which is a function for extracting a trend of the difference data in a predetermined time period.

9. A first recording unit that records the trained autoencoder according to any one of claims 1 to 7; a receiving unit that receives measurement sound data, which is data for a specific time length about measurement sound including sound generated from the object for which sound-based non-stationarity detection is being performed; a measurement sound feature data generating unit that generates, from the measurement sound data received by the receiving unit, measurement sound feature data, which is data about the features of the measurement sound identified by the measurement sound data, using the same method as used to generate the steady sound feature data from the steady sound data in pre-training; and a first calculation unit that reads the trained autoencoder recorded in the first recording unit, inputs the measurement sound feature data generated by the measurement sound feature data generating unit into the trained autoencoder, and obtains estimated measurement sound feature data for the specific time length that is output from the trained autoencoder for the measurement sound feature data. a second calculation unit that calculates the loss for the difference between the measurement sound feature data generated by the measurement sound feature data generation unit and the estimated measurement sound feature data created from the measurement sound feature data generated by the first calculation unit, using the same loss function as used when training the trained autoencoder, and that, if the generated data for the loss satisfies a predetermined condition, determines that the measurement sound identified by the measurement sound data that is the basis of the measurement sound feature data is a non-stationary sound, and generates result data indicating that a non-stationary sound has occurred.

10. The non-stationary sound detection device according to claim 9, wherein the second calculation unit is configured to determine that the measurement sound identified by the measurement sound data that is the basis of the measurement sound feature data is a non-stationary sound when the data about the loss includes a predetermined number of values ​​that exceed a predetermined threshold.

11. A non-stationary sound detection device according to claim 9, further comprising a predetermined display device, wherein the second calculation unit generates the result data as image data for displaying on the display an image of a graph indicating that the measured vibration sound identified by the measured vibration sound data on which the measured vibration sound feature data is based is non-stationary vibration sound.

12. The non-stationary sound detection device according to claim 9, wherein the measurement sound feature data is a Mel-frequency spectrogram generated from the measurement sound data.

13. A method for detecting non-stationary sounds executed by a computer having a first recording unit that records the trained autoencoder according to any one of claims 1 to 7, all of which are executed by a computer, comprising: a first step of receiving measurement sound data, which is data for a specific time length about measurement sound including sound generated from the object for which sound-based non-stationarity detection is being performed; a second step of generating, from the measurement sound data received in the first step, measurement sound feature data, which is data about the features of the measurement sound identified by the measurement sound data, using the same method as used to generate the stationary sound feature data from the stationary sound data in pre-training; and a third step of reading the trained autoencoder recorded in the first recording unit, inputting the measurement sound feature data generated in the second step into the trained autoencoder, and obtaining estimated measurement sound feature data for the specific time length that is output from the trained autoencoder for the measurement sound feature data. a fourth step of calculating the loss for a difference between the measurement sound feature data generated in the second step and the estimated measurement sound feature data created from the measurement sound feature data generated in the third step, using the same loss function as used when training the trained autoencoder, and, if the generated data for the loss satisfies a predetermined condition, determining that the measurement sound identified by the measurement sound data on which the measurement sound feature data is based is a non-stationary sound, and generating result data indicating that a non-stationary sound has occurred.

14. A computer program for causing a specified computer to function as a non-stationary sound detection device, comprising: a first recording unit that records a trained autoencoder that encodes input data, which is specified data, and then decodes the encoded data to restore it to data of the same dimension as the input data, the input data being steady sound feature data that is data for a specific length of time about steady sound including sound generated in a steady state from an object whose non-stationarity is to be detected based on sound, and the output data being estimated steady sound feature data, the trained autoencoder being obtained by inputting a plurality of steady sound feature data and performing pre-training so that the difference between the steady sound feature data that is the input data and the estimated steady sound feature data that is the output data for the input data is minimized; and a receiving unit that receives measurement sound data that is data for a specific length of time about measurement sound including sound generated from the object whose non-stationarity is being detected based on sound. a measurement sound feature data generation unit that generates, from the measurement sound data received by the reception unit, measurement sound feature data, which is data on features of the measurement sound identified by the measurement sound data, using the same method as used to generate the steady sound feature data from the steady sound data in pre-learning; a first calculation unit that reads the trained autoencoder recorded in the first recording unit, inputs the measurement sound feature data generated by the measurement sound feature data generation unit into the trained autoencoder, and obtains estimated measurement sound feature data that is an output from the trained autoencoder for the measurement sound feature data; a second calculation unit that calculates the loss for the difference between the measurement sound feature data generated by the measurement sound feature data generation unit and the estimated measurement sound feature data created from the measurement sound feature data generated by the first calculation unit, using the same loss function as used when training the trained autoencoder, and, if the generated data on the loss satisfies a predetermined condition, determines that the measurement sound identified by the measurement sound data that is the basis of the measurement sound feature data is a non-stationary sound, and generates result data indicating that a non-stationary sound has occurred.A computer program that makes something function.

Images