Trained autoencoder, trained autoencoder generation method, non-stationary sound detection method, non-stationary sound detection device, and computer program

Abstract

It provides trained artificial intelligence for detecting object non-stationarity that works accurately even in the presence of environmental noise. Steady sound feature data, which is data on the features of steady sound identified by steady sound data generated from steady sound data, is input to an autoencoder, and estimated steady sound feature data is output. Loss between the steady sound feature data and the estimated steady sound feature data is generated, and the autoencoder is trained to minimize the difference between the two. The loss function for determining the loss is modified using a noise removal function that removes noise from the difference, and a trend extraction function that extracts the trend of the difference in a specified time period.

Description

[Technical field]

[0001] The present invention primarily relates to a technique for detecting non-stationarity of an object based on the sound generated by the object. [Background technology]

[0002] As will be explained in detail later, the present invention detects the non-stationarity of an object based on sound, whereas most of the conventional techniques detect the abnormality of an object or judge whether the object is normal or abnormal based on sound.

[0003] We will explain later how determining the non-stationarity of an object (that an object is in a non-stationary state) differs from determining the abnormality of an object (or whether it has normality or abnormality), but for now we will explain the conventional technology in order. For example, if the target object is a machine, the idea of ​​detecting malfunctions in the machine by sound, or malfunctions in structures, facilities, etc. by sound, has existed for a long time as a non-destructive testing method, and can already be said to be classic. It is well known that machines that are out of order or about to break down will make abnormal noises, and it is a common method to detect machine failures or impending failures by using these abnormal noises. It is also well known that when an unfavorable condition occurs in a structure, for example when a crack appears in a wall, the sound produced by a hammering test will be mixed with abnormal noises, and it is common knowledge that the occurrence of such abnormal noises indicates that a problem has occurred or is about to occur in the structure. In the past, detection of such abnormal sounds was performed by skilled humans.

[0004] Incidentally, with the remarkable development of artificial intelligence in recent years, there has naturally been a movement to replace skilled humans who can detect machine malfunctions and abnormal sounds that occur during tapping inspections with artificial intelligence. For example, if the target object is a machine, a large amount of normal sound data, which is the sound of the machine in a normal state, is sampled, and a large amount of abnormal sound data, which is the sound of the machine when a malfunction occurs or will occur in the near future, is sampled, and both the normal and abnormal sound data are labeled as the sound identified by the data as normal or abnormal, and then the artificial intelligence is made to learn them. Then, when sound data emitted by the target object (i.e., the machine that is the target of detection or judgment of whether the current state is normal or abnormal) is input to the trained artificial intelligence, the artificial intelligence judges whether the sound identified by the sound data is a normal sound or an abnormal sound. Of course, once the sound is judged to be abnormal, it can be judged that the machine is out of order or will be out of order in the near future. This judgment can be divided into more stages than just two, depending on the type of labeling.

[0005] The technology for detecting abnormal sounds using artificial intelligence as described above can be said to be a simple replacement of skilled human abilities with artificial intelligence. The artificial intelligence described above determines whether a sound identified by input sound data is a normal sound or an abnormal sound, based on whether the input sound data is closer to already-learned normal sound data or abnormal sound data.

[0006] As mentioned above, the above-mentioned artificial intelligence requires a huge amount of data on normal and abnormal sounds for its training, which means that it takes a lot of effort and cost to create a trained artificial intelligence. If the object to be judged is a machine, after confirming that the machine is normal, a large amount of normal sound data, which is the sound the machine produces in a normal state, must be sampled, and after confirming that the machine is abnormal, a large amount of abnormal sound data must also be sampled. If abnormalities are divided into multiple stages, for example, from mild to severe, rather than just normal and abnormal, the amount of data that must be sampled increases. In addition, the sampled sound data must be properly labeled in order for the AI ​​to learn it, which also increases the effort and cost. However, in order to obtain the trained artificial intelligence described above, these efforts and costs are unavoidable.

[0007] In order to reduce the effort and costs involved in obtaining such trained artificial intelligence, the inventors of the present application have already invented the following concept and filed a patent application. The invention will be described below, but before that, the definition of the term "non-stationarity" in this application will be described. In this application, "non-stationarity (or non-stationary state)" means a state in which sounds other than the sounds generated from the object when pre-learning was performed as described below are generated from the object. In other words, in this invention, sounds generated in a stationary state are learned by an autoencoder, which will be described later. From the perspective of an anthropomorphized autoencoder, a stationary state is a state in which only sounds that have been learned are generated (or input), and a non-stationary state is a state in which sounds that have not been learned are generated.

[0008] For example, let us assume that a certain machine is the target object. If the sound (including the sound emitted by the machine when it is operating normally, which may include ambient sound, i.e., noise) is the sound generated in a stationary state, and if the machine is experiencing a breakdown or other malfunction (or is in a state where a malfunction will occur in the near future) and the machine is emitting an abnormal sound, then the state in which the sound including the abnormal sound is generated is a non-stationary state. In other words, in this example, if the non-stationarity of the target object can be detected, it can be detected that the machine is experiencing a malfunction or that a malfunction will occur in the near future.

[0009] In other words, it is possible to obtain trained artificial intelligence by machine learning data on steady sounds, which are sounds generated from an object in a steady state, and by inputting measurement sound data, which are sounds necessary when determining the state of an object, into such trained artificial intelligence, it is possible to detect non-stationarity of the object. When adopting such a concept, the only data required to obtain trained AI is steady sound data, including sounds emitted when the target object is in a steady state, and no labeling is required. Therefore, according to such a concept, it is possible to greatly reduce the effort and cost required to obtain trained AI. However, in order for a trained artificial intelligence that has been trained using only steady sound data to correctly detect non-steady states that are not steady, a different method is required than that used in the above-mentioned artificial intelligence that trains using both steady and non-steady sounds.

[0010] The inventors of this application have used autoencoders as a different approach to address the problem. The trained autoencoder that the present inventor has already developed and filed a patent application for is made from an autoencoder that encodes input data, which is predetermined data, and then decodes the encoded data to restore it to data of the same dimension as the input data. Such autoencoders are publicly known or well-known in the field of artificial intelligence. The trained autoencoder developed by the inventor of the present application is generated based on such an autoencoder before training, and is a trained autoencoder obtained by inputting a plurality of steady sound feature data, the trained autoencoder being generated based on steady sound data, the steady sound feature data being data on the features of steady sound identified by the steady sound data, the steady sound data being data for a specific length of time about steady sound including sounds generated in a steady state from an object whose non-stationarity is to be detected based on sound, and by pre-training the trained autoencoder so as to minimize loss in the difference between the steady sound feature data that is the input data and the estimated steady sound feature data that is the output data for a specific length of time for the input data.

[0011] The trained autoencoder described above is incorporated into a non-stationary sound detection device as follows. The non-stationary sound detection device includes a first recording unit that records the above-mentioned trained autoencoder, a reception unit that receives measurement sound data, which is data for a specific time length about measurement sound including sound generated from an object for which sound-based non-stationarity detection is being performed, a measurement sound feature data generation unit that generates, from the measurement sound data received by the reception unit, measurement sound feature data, which is data about the features of the measurement sound identified by the measurement sound data, in the same manner as the stationary sound feature data was generated from the stationary sound data in pre-learning, and a trained autoencoder that reads the trained autoencoder recorded in the first recording unit, inputs the measurement sound feature data generated by the measurement sound feature data generation unit into the trained autoencoder, and performs a trained autoencoder for the measurement sound feature data. and a second calculation unit that calculates a loss for a difference between the measurement sound feature data generated by the measurement sound feature data generation unit and estimated measurement sound feature data generated from the measurement sound feature data generated by the first calculation unit, using the same loss function as used when training the trained autoencoder, and, if the generated data for the loss satisfies a predetermined condition, determines that the measurement sound identified by the measurement sound data that is the basis of the measurement sound feature data is a non-stationary sound, and generates result data indicating that a non-stationary sound has occurred. [Prior art documents] [Patent documents]

[0012] [Patent Document 1] International Publication No. 2022 / 064590 Summary of the Invention [Problem to be solved by the invention]

[0013] The non-stationary sound detection device using the above-mentioned autoencoder uses only artificial intelligence (autoencoder) trained only on stationary sounds, yet is able to detect the occurrence of non-stationary sounds or that an object is in a non-stationary state, thereby reducing the effort and cost required to develop artificial intelligence that detects the non-stationarity of an object. However, when trying to detect the unsteadiness of a certain type of machine based on the sound it emits, the above-mentioned unsteadiness detection device including the autoencoder may lack accuracy in its judgment results. The machine is a machine having a rotating mechanism. A rotating mechanism is a mechanism that rotates around an axis. A typical example of a machine having a rotating mechanism is a motor. Therefore, it is difficult to accurately detect the unsteadiness of a machine incorporating a motor, such as a propeller drive unit of a wind power generation facility, a motor drive unit of other power plants, or a rotary drive unit of a manufacturing plant such as a glass factory, using the above-mentioned unsteadiness detection device including the autoencoder.

[0014] The objective of the present invention is to improve a non-stationary sound detection device, including an autoencoder trained only on stationary sounds, so that it can detect non-stationarity based on sounds generated by an object, which is a machine having a rotating mechanism, with greater accuracy than before. [Means for solving the problem]

[0015] The present inventors propose the following trained autoencoder as one aspect of the present invention. This trained autoencoder forms the core of the artificial intelligence in the present invention. The trained autoencoder in the present invention is made from an autoencoder that encodes input data, which is predetermined data, and then decodes the encoded data to restore the data to the same dimension as the input data. Such autoencoders are known or well-known in the field of artificial intelligence and are used to some extent in training artificial intelligence with training data, but are widely used in training artificial intelligence without training data. The trained autoencoder in the present invention is generated based on such an autoencoder, and is a trained autoencoder obtained by inputting steady-state sound feature data, which is data about the features of steady-state sound identified by steady-state sound data generated from steady-state sound data, which is data for a specific length of time about steady-state sound including sounds generated in a steady state from an object that is a machine including a rotating mechanism, and whose non-stationarity is to be detected based on sound, and by inputting estimated steady-state sound feature data for the specific length of time, and by using a loss function to calculate the loss between the steady-state sound feature data, which is the input data, and the estimated steady-state sound feature data, which is the output data for the input data, and by pre-training so that the loss is minimized. When generating this trained autoencoder, the loss function is calculated based on noise-removed data, which is the output when a plurality of difference data, which is data based on the difference between the steady sound feature data and the estimated steady sound feature data, is input into a noise removal function, which is a function for removing noise from the difference data, and trend data, which is the output when the difference data is input into a trend extraction function, which is a function for extracting a trend in a predetermined time period from the difference data. In other words, this autoencoder is trained by repeatedly inputting and outputting so that the steady sound feature data as input and the estimated steady sound feature data as output are as close to identical as possible. In other words, the trained autoencoder according to the present invention is tuned so that when the input data is steady sound feature data, the input and output data are as close to identical as possible. Roughly speaking, it is an autoencoder that can make the input data and output data almost identical for data related to sounds occurring in a steady state. In the learning process, the loss function described above is used.

[0016] The non-stationary sound detection device described below, which is created using the trained autoencoder described above, can detect non-stationarity based on sounds generated by an object that is a machine having a rotating mechanism with higher accuracy than ever before. Note that the data trained by this trained autoencoder is only stationary sound feature data generated based on stationary sound data, so the effect of reducing the effort and cost required to generate a trained autoencoder is also maintained. The reason why the trained autoencoder according to the present invention can detect the non-stationarity of an object, which is a machine having a rotating mechanism, more accurately than before based on the sound generated by the object is as follows. The inventors of the present application conducted extensive research into why a non-stationary sound detection device incorporating a conventional trained autoencoder tends to have low accuracy in detecting non-stationarity based on sounds generated by objects having a rotating mechanism, and discovered the following. An object including a rotating mechanism generally includes gears, bearings, etc., in addition to a rotating body that rotates such as a rotating shaft. When an object including a rotating mechanism is in an unsteady state, there are often cases where the shaft is misaligned, the rotating body is eccentric, a gear is missing, the gear contact is abnormal, a bearing is missing, etc. Such a malfunction of the object appears as a spike-like abnormal sound that lasts for a very short time in the sound generated by the object. This abnormal sound appears as a spike-like error score in the output (loss) of the loss function. However, an object including a rotating mechanism is always in an inertial vibration state, and for example, small fluctuations in the error score are always generated due to the joint of adjacent gears. This fluctuation in the error score appears as noise in the loss. In addition, in a rotating mechanism, the same state occurs repeatedly due to the rotation of the rotating body, so the error score corresponding to the loss may include a relatively long-term temporal tendency that could be called a trend. The inventors of the present application have found that unless the effects of the above-mentioned noise and trends in the loss calculated using the loss function are suppressed, it is impossible to obtain a trained autoencoder that has a function of matching the steady sound feature data, which is the input data, with the estimated steady sound feature data, which is the output data, as closely as possible. This is a finding that they have obtained. Based on this knowledge, in the present invention, a loss function used when causing an encoder to perform learning is generated based on noise-removed data, which is the output when the difference data is input to a noise removal function, which is a function for removing noise from the difference data, and which is obtained after generating a plurality of difference data, which is data based on the difference between steady sound feature data and estimated steady sound feature data, and trend data, which is the output when the difference data is input to a trend extraction function, which is a function for extracting a trend in a predetermined time period from the difference data. By using such a loss function, noise and components corresponding to trends are removed from the calculated loss, although they may not be completely removed. Therefore, according to the non-stationary sound detection device described below that is created using the trained autoencoder of the present application, it becomes possible to detect non-stationarity based on sounds generated by an object that is a machine having a rotating mechanism with higher accuracy than before.

[0017] As described above, the loss function used in the process of obtaining the trained autoencoder of the present invention uses a noise removal function to remove noise that appears in the differential data, and a trend extraction function to remove trends that appear in the differential data. Here, the noise removal function can be any of the following three functions: a function that functions as a low-pass filter, a function that finds a moving average, and a function that performs a Fourier transform, removes frequency band signals exceeding a predetermined threshold, and returns the signal by performing an inverse Fourier transform. In any case, by using any of these three functions, it becomes possible to correct extremely violent fluctuations that occur in a short period of time from previous and next differential data among a large number of differential data that are generated continuously, which is noise specific to the target object that is a machine having a rotating mechanism. The trend extraction function may be a function that obtains the difference data and the median of a plurality of pieces of data before, after, or before and after the difference data, or a function that takes a difference series of the difference data. Here, the function for finding the median of multiple data is literally a function for extracting a trend. Since a large number of differential data are generated continuously, it does not affect the detection of non-stationarity, but there are cases where trends specific to the target object, which is a machine having a rotating mechanism, occur, which are long-term continuous changes or gradual changes with periodicity. By using the function for finding the median to find the median of multiple consecutive differential data including one differential data of interest, it becomes possible to extract the above-mentioned trend or trend data expressing the trend, and the influence of the trend can be removed from the differential data by using the trend data. On the other hand, a function that takes the difference series of difference data is a function that removes the influence of trends from the difference data, and its nature is slightly different from a function that finds a median, which extracts the trend itself. While trend data generated using a function that finds a median represents a trend as described above, trend data generated using a difference series corresponds to difference data after the influence of the above-mentioned trend has been removed.

[0018] As described above, the loss function used in the training process of the trained autoencoder of the present invention is one that calculates the loss based on the noise-removed data and trend data. More specifically, when the trend extraction function is a function for obtaining a median of the difference data and a plurality of pieces of data before, after, or before and after the difference data, the loss function may obtain the loss as a difference between the noise-removed data and the trend data. The noise-removed data is conceptually obtained by removing the influence of noise from difference data regarding the difference between steady sound feature data and estimated steady sound feature data paired with the steady sound feature data. When the difference between data obtained by removing the influence of noise from such difference data and trend data conceptually corresponding to a trend is calculated, the resulting data is conceptually obtained by further removing the influence of the trend from data obtained by removing the influence of noise from the difference data. Therefore, by using such a loss function, the effects of noise and trends specific to sounds generated from an object having a rotating mechanism can be effectively removed from the loss between the steady-state sound feature data and the estimated steady-state sound feature data. When the trend extraction function is a function that takes a difference series of the difference data, the loss function may be one that determines the loss by applying the noise removal function to the trend data. In this case, the trend data is conceptually obtained by removing the influence of trends from difference data regarding the difference between the steady sound feature data and the estimated steady sound feature data paired with that steady sound feature data. Data obtained by applying a noise removal function to such difference data is conceptually obtained by further removing the influence of noise from data obtained by removing the influence of trends from the difference data. Therefore, by using such a loss function, the effects of noise and trends specific to sounds generated from an object having a rotating mechanism can be effectively removed from the loss between the steady-state sound feature data and the estimated steady-state sound feature data.

[0019] The present inventor also proposes a method for generating the trained autoencoder described above as one aspect of the present invention. The effect of this method is similar to that of the trained autoencoder described above. One example of the method is a method for converting an autoencoder that encodes input data, which is predetermined data, and then decodes the encoded data to return it to data of the same dimension as the input data into a trained autoencoder, the method comprising the steps of: generating stationary sound feature data, which is data for a specific time length about stationary sound, which is sound generated in a stationary state from an object, which is an object for which non-stationarity is to be detected based on sound, from the stationary sound data; generating estimated stationary sound feature data for the specific time length from the stationary sound data; and outputting the estimated stationary sound feature data for the specific time length by inputting a plurality of stationary sound feature data, and The method includes a step of calculating a loss for the estimated steady sound feature data, which is output data for the input data, by using a loss function, and performing pre-learning so that the loss function is minimized, in which the loss function is calculated based on noise-removed data that is an output when a plurality of difference data, which is data based on a difference between the steady sound feature data and the estimated steady sound feature data, is input into a noise removal function that is a function for removing noise from the difference data, and trend data that is an output when the difference data is input into a trend extraction function that is a function for extracting a trend in a predetermined time period of the difference data.

[0020] The present inventors also propose, as one aspect of the present invention, a non-stationary sound detection device that uses the above-mentioned trained autoencoder. An example of the non-stationary sound detection device (hereinafter, may be simply referred to as a "detection device") includes a first recording unit that records any of the trained autoencoders described above, a reception unit that receives measurement sound data that is data for a specific time length about a measurement sound including a sound generated from the object for which sound-based non-stationarity detection is being performed, a measurement sound feature data generation unit that generates, from the measurement sound data received by the reception unit, measurement sound feature data that is data about the feature of the measurement sound identified by the measurement sound data in the same manner as the stationary sound feature data was generated from the stationary sound data in pre-learning, and a trained autoencoder that reads in the trained autoencoder recorded in the first recording unit and reads out the measurement sound feature data generated by the measurement sound feature data generation unit into the trained autoencoder. a first calculation unit that inputs the measured sound feature data to a decoder and obtains estimated measurement sound feature data for the predetermined time length which is output from the trained autoencoder for the measurement sound feature data; and a second calculation unit that calculates the loss for the difference between the measurement sound feature data generated by the measurement sound feature data generation unit and the estimated measurement sound feature data created from the measurement sound feature data generated by the first calculation unit, using the same loss function as used when training the trained autoencoder, and when the generated data for the loss satisfies a predetermined condition, determines that the measurement sound identified by the measurement sound data that is the basis of the measurement sound feature data is a non-stationary sound, and generates result data indicating that a non-stationary sound has occurred.

[0021] This non-stationary sound detection device includes a first recording unit that records the trained autoencoder described above. The trained autoencoder recorded in the first recording unit is used as described below. This detection device includes a receiving unit that receives measurement sound data, which is data for a specific time length about a measurement sound including a sound generated from an object for which sound-based non-stationary detection is being performed. The data received by the receiving unit is the measurement sound data, which is data for a specific time length about a measurement sound including a sound generated from an object for which sound-based non-stationary detection is being performed. In other words, this detection device detects non-stationary sound based on the measurement sound including a sound generated from the object. This detection device includes a measurement sound feature data generating unit. The measurement sound feature data generating unit generates measurement sound feature data, which is data on the feature of the measurement sound specified by the measurement sound data, from the measurement sound data received by the receiving unit. The measurement sound data has the same data format and type as the steady sound data used in the pre-learning stage, and the measurement sound feature data is generated by the same method as the method used when the steady sound feature data was generated from the steady sound data in the pre-learning stage. The measurement sound feature data is input to the trained autoencoder, but since both the measurement sound feature data and the steady sound feature data input to the autoencoder being trained for training the trained autoencoder are data on sound, the format or type of both data can be the same. For example, when the steady sound feature data is a Mel frequency spectrogram generated from the steady sound data, the measurement sound feature data is a Mel frequency spectrogram generated from the measurement sound data.

[0022] This non-stationary sound detection device includes a first calculation unit. The first calculation unit reads the trained autoencoder recorded in the first recording unit and causes the trained autoencoder to function. The first calculation unit inputs the measurement sound feature data generated by the measurement sound feature data generation unit to the trained autoencoder read from the first recording unit, and obtains estimated measurement sound feature data as the output. As already mentioned, the trained autoencoder is tuned so that when data on the characteristics of stationary sounds (steady sound feature data if the trained autoencoder is in the process of learning) is input, it outputs estimated stationary sound feature data that is nearly identical to the data. Therefore, when the measurement sound specified by the measurement sound data that is the source of the measurement sound feature data is a stationary sound (sound including a sound from an object in a stationary state), the estimated measurement sound feature data output by the trained autoencoder is almost the same as the measurement sound feature data that is the source of the estimated measurement sound feature data. This conclusion remains the same even when the measurement sound specified by the measurement sound data that is the source of the measurement sound feature data contains an environmental sound. This is because, just as the measurement sound may contain an environmental sound, the stationary sound specified by the stationary sound data that is the source of the sound feature data input to the trained autoencoder in the learning process may also contain an environmental sound. Therefore, from the perspective of the trained autoencoder, the components derived from the environmental sound contained in the measurement sound feature data can be regarded as components derived from the stationary sound that have already been learned, and are not recognized as components that deviate from the stationary sound. On the other hand, the trained autoencoder used in the first calculation unit is tuned so that the input and output are approximately the same only when the input is data related to stationary sound. In other words, the trained autoencoder described above can be said to be an autoencoder dedicated to stationary states or stationary sounds, which has been trained specifically for stationary states or stationary sounds, and which can exhibit the function of approximately matching the input and output only when the object is in a stationary state or when a stationary sound including a sound from an object in a stationary state is input. Therefore, when the measurement sound identified by the measurement sound data on which the measurement sound feature data is based is a non-stationary sound including a sound from an object in a non-stationary state, the trained autoencoder does not function in the same way as when the object is in a stationary state (in other words, it does not function as expected, or, to put it bluntly, it malfunctions). Therefore, in this case, the estimated measurement sound feature data output by the trained autoencoder is significantly different from the measurement sound feature data on which the estimated measurement sound feature data is based. And, for the reasons already mentioned, this conclusion also remains the same even when the measurement sound includes an environmental sound. This detection device includes a second calculation unit that functions as an artificial intelligence that detects that the object is in a non-steady state based on the measurement sound data received from the outside, in cooperation with the first calculation unit or in cooperation with the measurement sound feature data generation unit and the first calculation unit. The second calculation unit is configured to calculate a loss for the difference between the measurement sound feature data generated by the measurement sound feature data generation unit and the estimated measurement sound feature data generated from the measurement sound feature data generated by the first calculation unit. The loss function used to calculate the loss is the same loss function used when the trained autoencoder is trained. In other words, the method used to calculate the loss is the same method used when the trained autoencoder is trained. As described above, when the measurement sound data is a stationary sound, the measurement sound feature data as an input to the trained autoencoder and the estimated measurement sound feature data as an output from the trained autoencoder are approximately the same, and when the measurement sound data is a non-stationary sound, the measurement sound feature data as an input to the trained autoencoder and the estimated measurement sound feature data as an output from the trained autoencoder are significantly different. Therefore, when the measurement sound data is stationary sound, the loss between the measurement sound feature data as the input to the trained autoencoder and the estimated measurement sound feature data as the output from the trained autoencoder is relatively small, and when the measurement sound data is non-stationary sound, the loss between the measurement sound feature data as the input to the trained autoencoder and the estimated measurement sound feature data as the output from the trained autoencoder is relatively large. Since the loss function used when generating the loss for the difference between the measurement sound feature data and the estimated measurement sound feature data is the same loss function used when training the trained autoencoder, the effects of noise and trends are removed from the resulting loss. Therefore, the characteristic that the loss obtained using the loss function is relatively small when the measurement sound data is stationary sound and the loss obtained using the loss function is relatively large when the measurement sound data is non-stationary sound is guaranteed with a certain level of accuracy. The second calculation unit is also configured to determine that the measurement sound identified by the measurement sound data on which the measurement sound feature data is based is a non-stationary sound when the generated data on the loss satisfies a predetermined condition, and generate result data indicating that a non-stationary sound has occurred. Based on the difference in magnitude of the loss described above, the second calculation unit is able to determine whether the object that emitted the sound contained in the measurement sound identified by the measurement sound data received by the detection device is in a steady state or a non-stationary state, and is therefore able to generate result data indicating that the object is in a non-stationary state. The result data, which is data indicating that the object is in an unsteady state, may be used in any manner. For example, when the result data is generated, the detection device, or a predetermined other device that receives the result data from the detection device, may perform an appropriate operation to notify a user of the detection device of the fact that the object is in an unsteady state. For example, the detection device or other device may notify a predetermined user of the occurrence of the above-mentioned fact by email or other message, or may notify the user of the occurrence of the above-mentioned fact in a manner that the user can detect with any of the five senses, such as by displaying the occurrence of the above-mentioned fact on a display connected to the detection device or other device, or by sounding a police light. In addition, by recording a large amount of result data in chronological order, for example along with timestamps, on a recording medium inside or outside the detection device, it is possible to predict how the condition of the object will change in the future based on the trends in the accumulated result data, and also to verify how the condition of the object has changed in the past. The detection device may include a predetermined display device. In this case, the second calculation unit may generate the result data as image data for displaying on the display an image of a graph indicating that the test sound identified by the test sound data on which the test sound feature data is based is a non-stationary sound. This allows the user to intuitively grasp the state of the object generating the input test sound data, for example, whether it is in a stationary state or a non-stationary state.

[0023] As is clear from the above description, the non-stationary sound detection device according to this application can detect that the object is in a non-stationary state even if the measurement sound data includes environmental sound. This detection device also has the advantage that it is not necessary to apply noise canceling technology to the measurement sound data.

[0024] As described above, the second calculation unit in the detection device of the present invention calculates the difference between the measurement sound feature data generated by the measurement sound feature data generation unit and the estimated measurement sound feature data generated from the measurement sound feature data generated by the first calculation unit. At this time, how to detect the difference can be appropriately determined. For example, the second calculation unit may be configured to determine that a measurement sound identified by the measurement sound data on which the measurement sound feature data is based is a non-stationary sound when the data on the loss includes a predetermined number or more of values ​​exceeding a predetermined threshold value. By doing this, the second calculation unit can detect whether the sound from the object contains unsteady sound, i.e., whether the object is in an unsteady state, simply by counting the number of values ​​in the loss function that exceed the threshold. In addition, using the threshold as described above has the following advantages. In the case of conventional artificial intelligence, the type of value output from the artificial intelligence basically depends on the type of data pre-trained by the artificial intelligence. For example, if the abnormal sound generated from the object output by the artificial intelligence has four stages, normal, normal, bad, and worst, the type of data pre-trained by the artificial intelligence must conform to the above four stages and be labeled according to each data. However, if the second calculation unit uses the threshold as described above, even if the data trained by the trained autoencoder of the present invention is only steady-state data, it is possible to obtain different judgment results from the same loss function by simply changing the above threshold. This means that in the above example, four-stage judgment can be performed without complicating the pre-learning in the trained autoencoder. Needless to say, in order to obtain this effect, the detection device in the present invention can change the threshold used in the second calculation unit. Such a change in threshold can be performed, for example, by input from a predetermined input device (keyboard, mouse, etc.) connected to the detection device. In addition, the detection device itself can automatically change the threshold according to a predetermined rule.

[0025] The present inventor also proposes, as one aspect of the present invention, a method for detecting non-stationary sounds that applies the trained autoencoder of the present invention. The effects of this method are similar to those of the non-stationary sound detection device of the present invention. One example of the method is a method for detecting non-stationary sounds executed by a computer having a first recording unit that records the trained autoencoder described above, and includes the following steps, all of which are executed by a computer: The process includes a first process of receiving measurement sound data, which is data for a specific time length regarding measurement sound including sound generated from the object for which sound-based non-stationarity detection is being performed; a second process of generating measurement sound feature quantity data, which is data regarding the feature quantity of the measurement sound identified by the measurement sound data, from the measurement sound data received in the first process in the same manner as the stationary sound feature quantity data was generated from the stationary sound data in pre-learning; and a second process of reading a trained autoencoder recorded in the first recording unit, inputting the measurement sound feature quantity data generated in the second process into the trained autoencoder, and performing a trained autoencoder for the measurement sound feature quantity data. a third step of obtaining estimated measurement sound feature data for the predetermined time length which is output from an autoencoder; and a fourth step of calculating the loss for the difference between the measurement sound feature data generated in the second step and the estimated measurement sound feature data created from the measurement sound feature data generated in the third step, using the same loss function as used when training the trained autoencoder, and, if the generated data for the loss satisfies a predetermined condition, determining that the measurement sound identified by the measurement sound data on which the measurement sound feature data is based is a non-stationary sound, and generating result data indicating that a non-stationary sound has occurred.

[0026] The inventor of the present application also proposes, as one aspect of the present invention, a computer program for causing a specific computer to function as a non-stationary sound detection device that applies the trained autoencoder of the present invention. The effects of this computer program are similar to those of the non-stationary sound detection device of the present invention, and also have the effect of making a general-purpose computer function as the non-stationary sound detection device of the present invention. An example of the computer program is a computer program for causing a specific computer to function as a non-stationary sound detection device. This computer program includes an autoencoder that causes the computer to encode input data, which is predetermined data, and then decodes the encoded data to return it to data of the same dimension as the input data, the input data being steady sound feature data that is data for a specific time length about steady sound including sound generated in a steady state from an object that is an object for which non-stationarity based on sound is to be detected, the input data being steady sound feature data that is data about a feature of steady sound identified by the steady sound data, and output data being estimated steady sound feature data, and the computer program includes an autoencoder that causes a plurality of steady sound feature data to be input and pre-trained so that a difference between the steady sound feature data that is the input data and the estimated steady sound feature data that is output data for the input data is minimized, the computer program includes a first recording unit that records a trained autoencoder that receives measurement sound data that is data for a specific time length about measurement sound including sound generated from the object for which non-stationarity based on sound is to be detected, and a reception unit that receives measurement sound data that is data for a specific time length about measurement sound including sound generated from the object for which sound-based non-stationarity is to be detected, and a reception unit that receives measurement sound data received by the reception unit and performs pre-training on the feature of the measurement sound identified by the measurement sound data. a measurement sound feature data generation unit that generates measurement sound feature data, which is data for the measurement sound feature data, in the same manner as the steady sound feature data was generated from the steady sound data in pre-learning; a first calculation unit that reads the trained autoencoder recorded in the first recording unit, inputs the measurement sound feature data generated by the measurement sound feature data generation unit to the trained autoencoder, and obtains estimated measurement sound feature data that is an output from the trained autoencoder for the measurement sound feature data; and a second calculation unit that calculates a loss for a difference between the measurement sound feature data generated by the measurement sound feature data generation unit and the estimated measurement sound feature data created from the measurement sound feature data generated by the first calculation unit, using the same loss function as used when training the trained autoencoder, and, if the generated data for the loss satisfies a predetermined condition, determines that the measurement sound identified by the measurement sound data that is the basis of the measurement sound feature data is a non-stationary sound, and generates result data indicating that a non-stationary sound has occurred. [Brief description of the drawings]

[0027] [Figure 1] 1 is a perspective view showing an outline of the overall configuration of a learning device according to an embodiment; [Diagram 2] FIG. 2 is a diagram showing the hardware configuration of a computer device included in the learning device shown in FIG. [Diagram 3] 3 is a block diagram showing functional blocks generated within the computer device shown in FIG. 2; [Figure 4] FIG. 3 is a diagram conceptually illustrating the configuration of an autoencoder included in the computer device illustrated in FIG. 2. [Diagram 5] 4 is a diagram conceptually illustrating an example of loss generated in a loss generating unit illustrated in FIG. 3. [Figure 6] FIG. 2 is a block diagram showing functional blocks generated within a computer device included in the detection device of an embodiment. [Figure 7] FIG. 13 is a diagram showing the relationship between loss and a threshold value when a determination is made by a state determination unit. [Figure 8] FIG. 13 is a diagram showing an example of a histogram when the object is in a steady state. [Figure 9] FIG. 13 is a diagram showing an example of a histogram when the object is in a non-stationary state. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] A preferred embodiment of the present invention will now be described with reference to the drawings. Test examples performed using the detection device of the embodiment will also be described.

[0029] In this embodiment, a detection device for a non-stationary sound (hereinafter, may be simply referred to as a "detection device") will be described. The non-stationary sound detection device includes a trained autoencoder, as described below. Therefore, in order to obtain a non-stationary sound detection device, it is first necessary to obtain a trained autoencoder. In this embodiment, the device necessary to obtain a trained autoencoder is referred to as a learning device for convenience. This detection device is for detecting that an object, which is a machine having a rotating mechanism, is in a non-stationary state based on the sound from the object. A typical example of a machine having a rotating mechanism is a motor. Devices incorporating a motor, such as a propeller drive unit of a wind power generation facility, a motor drive unit of other power plants, and a rotation drive unit of a manufacturing plant such as a glass factory, can be objects that are detected to be in a non-stationary state by the detection device of this embodiment. Furthermore, the non-stationary sound detection device in this embodiment uses a threshold value, which will be described later, when detecting a non-stationary sound, although this is not necessarily required. In this embodiment, the threshold value is generated by the above-mentioned learning device used to obtain the trained autoencoder. However, the threshold value may be generated by the detection device instead of the learning device, or may be generated by another device that is neither a learning device nor a detection device.

[0030] As described above, in order to establish a detection device for non-stationary sounds, in addition to the detection device, a learning device and, in some cases, another device for determining a threshold value other than the detection device and the learning device are required. However, the detection device, the learning device, and the other device, which are required up to three, can have the same required hardware configuration, so they can be integrated into one device by installing appropriate computer programs in them. Therefore, it is natural that any two of the detection device, the learning device, and the other device, which are required up to three, can be integrated into one device. As already described, in this embodiment, the learning device determines the threshold value. In other words, this is an example of a configuration in which the learning device and the other device are integrated into one device.

[0031] Both the detection device and the learning device include a computer device. The computer devices included in the detection device and the learning device can be the same, and in this embodiment are the same. In addition, both the detection device and the learning device include peripheral devices connected to the computer device. The peripheral devices can also be the same in the detection device and the learning device, and in this embodiment are the same, but are not limited to this.

[0032] <About the learning device> From a hardware perspective, the configuration of the detection device is the same, but for the time being, the hardware configuration of the learning device is shown in Figure 1. The learning device includes a computer device and peripheral devices, which are also shown in Figure 1.

[0033] In the figure, a computer device is designated by 100. A display 101 and an input device 102 are connected to the computer device 100. In addition, a microphone 103 is connected to the computer device 100, as will be described later. The display 101 is for displaying still images or moving images, and may be a publicly known or well-known display. The display 101 in this embodiment is capable of displaying both still images and moving images, but this is not essential. The display 101 may be a publicly known or well-known display, and may be a commercially available product, such as a liquid crystal display or an organic electro-luminescence (EL) display. The display 101 in this embodiment is connected to the computer device 100 by a cable, but may be connected to the computer device 100 wirelessly. The technology used for connecting the computer device 100 and the display 101 may also be a publicly known or well-known technology. The input device 102 is for a user to input a desired value to the computer device 100. The input device 102 may be a known or publicly known device. In this embodiment, the input device 102 of the computer device 100 is a keyboard, but the input device 102 is not limited to this, and a known or publicly known voice input using a numeric keypad, a trackball, a mouse, or a microphone terminal may also be used. If the display 101 is a touch panel, the display 101 also functions as the input device 102. A microphone 103 is connected to the computer device 100. The microphone 103 has a function of collecting sounds including sounds emitted by an object to be described later, and generating sound data which is data of the collected sounds. As long as the microphone 103 has such a function, it may be a publicly known or well-known one, and a commercially available product may suffice. The sound data on the sound generated by the microphone 103 is sent from the microphone 103 to the computer device 100 via a cable. The method for sending the sound data from the microphone 103 to the computer device 100 may be wireless or not wired. In addition, the transmission of the sound data from the microphone 103 to the computer device 100 may be via the Internet. In addition, the above-mentioned transmission of the sound data from the microphone 103 to the computer device 100 via a wired, wireless, or Internet does not need to be performed in approximately real time. For example, sound data generated by microphone 103 may be temporarily recorded on a recording medium in a device unrelated to computer device 100, and the sound data recorded on that recording medium may be read directly or via another recording medium into computer device 100, thereby being supplied to computer device 100. In short, although it is essential that sound data generated by microphone 103 be sent to computer device 100, the method and timing of providing the data may be determined appropriately depending on the circumstances. An amplifier for amplifying sound data may be provided between the microphone 103 and the computer device 100. Amplifiers are of course publicly known or well-known, and are commercially available in large quantities, so if an amplifier is to be used, an appropriate one may be selected and used. Although an amplifier is used in this embodiment, it is a common one, so detailed explanation and illustrations will be omitted.

[0034] Next, a description will be given of the configuration of the computer device 100 that constitutes the learning device. The hardware configuration of the computer device 100 is shown in FIG. The hardware includes a CPU (central processing unit) 111, a ROM (read only memory) 112, a RAM (random access memory) 113, an interface 114, and a large-capacity recording medium 115, which are interconnected by a bus 116. The CPU 111 is a computing device that performs calculations. The CPU 111, which is a computing device, can be replaced with a GPU (graphics processing unit) or GPGPU (general purpose computing on GPU), which are also computing devices. The CPU 111 executes a computer program recorded in, for example, the ROM 112 or the RAM 113, thereby executing the process described below. The large-capacity recording medium 115 is a publicly known or well-known device for recording a large amount of data, such as a hard disk drive (HDD) or a solid state drive (SSD). The above-mentioned computer program may be recorded in the large-capacity recording device 115, and is more common. The computer program here includes a computer program for causing the computer device 100 to execute the processes described below that are necessary for the computer device 100 to function as a learning device. This computer program may be pre-installed in the computer device 100, or may be post-installed. This computer program may be installed in the computer device 100 via a predetermined recording medium (not shown), such as a memory card, or via a network such as a LAN or the Internet. Of course, the computer program may include an OS and other necessary computer programs in addition to the above computer programs. The ROM 112 stores computer programs and data necessary for the CPU 111 to execute the processes described below. The RAM 113 provides a work area necessary for the CPU 111 to perform processing. In some cases, the above-mentioned computer programs and data (at least a part of them) may be recorded or temporarily recorded therein. The interface 114 exchanges data between the CPU 111, RAM 113, etc., which are connected via a bus 116, and the outside. The interface 114 is connected to the above-mentioned display 101, input device 102, and microphone 103. Data on the operation contents input from the input device 102 and sound data sent from the microphone 103 are input from an interface 114 to a bus 116 . As is well known, data for displaying an image on the display 101 is sent from the bus 116 to the interface 114 and output from the interface 114 to the display 101 .

[0035] The entire system including the computer device 100 described above executes the processes required to function as a learning device using the following functional blocks. 3 are generated within computer device 100 as a result of CPU 111 executing the computer program. Note that the following functional blocks may be generated by the functions of the above-mentioned computer program alone for causing computer device 100 to execute the processes described below that are necessary for computer device 100 to function as a learning device, or may be generated by cooperation between the above-mentioned computer program and an OS or other computer programs installed in computer device 100. Within the computer device 100, in relation to the functions of the present invention, an input unit 121, a main control unit 122, a feature detection unit 123, an autoencoder 124, an autoencoder recording unit 124A, a loss generation unit 125, a loss function recording unit 125A, a loss recording unit 125B, an optimization unit 126, a threshold determination unit 127, and an output unit 128 are generated. Among these, the functional blocks that perform information processing, i.e., the main control unit 122, the feature detection unit 123, the autoencoder 124, the loss generation unit 125, the optimization unit 126, and the threshold determination unit 127, correspond to a calculation device (CPU 111 in this embodiment) in terms of hardware, or are realized by the functions of a calculation device that performs information processing in accordance with the instructions of the above-mentioned computer program. When viewed as hardware, the input unit 121 and the output unit 128 correspond to the interface 114, or are realized by the function of the interface 114. More specifically, the input unit 121 and the output unit 128 conceptually correspond to the connection portion between the interface 114 and the bus 116. In addition, the functional blocks that record data, i.e., the autoencoder recording unit 124A, the loss function recording unit 125A, and the loss recording unit 125B, correspond in hardware to a recording device, for example, the RAM 113 or the large-capacity recording medium 115, or are realized as a part of the recording device.

[0036] The input unit 121 receives an input from the interface 114 . The input from the interface 114 to the input unit 121 includes an input from the input device 102. The input from the input device 102 includes, for example, mode selection data, the details of which will be described later. When mode selection data or the like is input from the input device 102, the mode selection data or the like input from the input device 102 is sent from the input unit 121 to the main control unit 122. The data input from the interface 114 to the input unit 121 also includes sound data from the microphone 103. When the sound data is received by the input unit 121, the input unit 121 sends the sound data to the feature detection unit 123. The main control unit 122 performs overall control of the above-mentioned functional blocks generated in the computer device 100. For example, the main control unit 122 may receive mode selection data as described above. The learning device in this embodiment is configured to function in a mode alternatively selected from two modes: a learning mode in which the learning device functions as a learning device, and a threshold determination mode in which the device functions as a device that determines a threshold. The mode selection data is data that determines (specifies) which of the two modes the computer device 100 should function in, and the main control unit 122 that has received the data instructs appropriate ones of the other functional blocks to execute the selected mode. In this embodiment, at least the main control unit 122 instructs the loss generation unit 125 as to the output destination of the loss (or, more precisely, the data) generated by the loss generation unit 125, as described below. Specifically, when the learning device executes the learning mode, the main control unit 122 instructs, for example, the optimization unit 126 to update the data described below for generating the autoencoder 124 recorded in the autoencoder recording unit 124A, and when the learning device executes the threshold determination mode, for example, the threshold determination unit 127 to determine a threshold described below.

[0037] The feature amount detection unit 123 detects the feature amount of the received sound data, and generates sound feature amount data that is data regarding the feature amount of the sound. In this embodiment, prior to generating sound feature data, the feature detection unit 123 performs necessary processing on the sound data, such as converting the sound data from analog data to digital data in a predetermined format (e.g., digital data in wav format) and adjusting the length of the sound data to a predetermined length (e.g., a planned length such as 10 seconds or 30 seconds), as necessary. Although not necessarily limited to this, in this embodiment, the feature detection unit 123 generates sound feature data for the features of the sound data adjusted in this manner. The sound feature data will be described in detail later, but in this embodiment, it is a Mel frequency spectrogram (more precisely, the data thereof), although it is not limited to this. How to generate the Mel frequency spectrogram will be described later. The feature detection unit 123 sends the generated sound feature data to the autoencoder 124 and the loss generation unit 125.

[0038] Data for generating an autoencoder is recorded in the autoencoder recording unit 124A. The CPU 111, which is a calculation device, reads the data for generating an autoencoder from the autoencoder recording unit 124A, and generates the autoencoder 124. The autoencoder 124 is an autoencoder that encodes input data, which is predetermined data, and then decodes the encoded data to return it to data of the same dimension as the input data. In the field of artificial intelligence, such autoencoders are publicly known or well-known, and the autoencoder 124 (or data for generating an autoencoder) of this embodiment may be selected from such known autoencoders (or data for generating an autoencoder). The autoencoder 124 is conceptually shown in FIG. 4. The autoencoder 124 is configured by a combination of an input layer that receives an input x, an intermediate layer that encodes the input x to compress the feature and decodes the compressed data to expand the feature, and an output layer that outputs an output x'. As described above, the autoencoder 124 may receive sound feature data from the feature detection unit 123. Upon receiving the sound feature data, the autoencoder 124 encodes it and subsequently decodes the encoded data to return it to sound feature data and output it. The output sound feature data is referred to as estimated sound feature data. The autoencoder 124 sends the generated estimated sound feature data to a loss generation unit 125. The data for generating the autoencoder 124, which is recorded in the autoencoder recording unit 124A, is updated by the optimization unit 126, as described below. Moreover, the data for generating the autoencoder 124 recorded in the autoencoder recording unit 124A is sent to the output unit 128 at a timing to be described later.

[0039] As described above, the loss generation unit 125 receives the sound feature data from the feature detection unit 123 and the estimated sound feature data from the autoencoder 124. In essence, the sound feature data and the estimated sound feature data are a set of data that respectively correspond to the input and output when encoding and decoding are performed once by the autoencoder 124. The loss generation unit 125 generates a loss for the difference between the set of sound feature data and the estimated sound feature data by using a loss function. There are several types of publicly known or well-known loss functions, and although this is not limited to these, the following loss function is used in this embodiment. The data specifying the loss function is recorded in the loss function recording unit 125A. When the loss generation unit 125 generates data on the loss, the loss generation unit 125 reads the data specifying the loss function recorded in the loss function recording unit 125A before that, and executes a calculation using the loss function specified by the data specifying the loss function.

[0040] The loss function used in this embodiment is a function that combines the following (a) to (d) mathematically expressed. (a) generating a plurality of difference data based on the difference between the sound feature data and the estimated sound feature data; (b) Generate denoised data, which is the output when the difference data is input to a noise removal function, which is a function that removes noise from the difference data. (c) generating trend data, which is the output when differential data is input to a trend extraction function, which is a function that extracts trends in differential data for a specified time period; (d) Calculate the loss based on the noise-removed data and the trend data. Here, the loss function is a combination of the equivalent formulas (a) through (d) in that order, but (b) and (c) can be in any order.

[0041] An example of a formula that can be used in (a) above, "generating a plurality of difference data that are data based on the difference between the sound feature data and the steady sound feature data," is the Mean Squared Error (MSE). The reason for using this formula is to reduce the inference error when the difference between the sound feature data and the estimated sound feature data is extremely large. Specifically, the following formula (1) is used to calculate the mean square error. If it is important to make the loss function value during learning easier to intuitively recognize, the root mean squared error (RMSE) may be used instead of the mean squared error. The formula for calculating the root mean squared error is shown in the following (Equation 2). As another loss function, if the objective is to reduce the average error between the sound feature data and the estimated sound feature data, the Mean Absolute Error (MAE) may be used. The formula for calculating the Mean Absolute Error is shown in the following (Equation 3). TIFF0007525137000001.tif1973TIFF0007525137000002.tif2084TIFF0007525137000003.tif1777In (Equation 1) to (Equation 3), "i" means the i-th number. The meaning of "i" is the same in (Equation 4) and onwards, which will be described later. As described above, the sound data is data of a specific length of time. From the data, a large number of sets of sound feature data and estimated sound feature data are created for each short period of time. i (We use this notation for convenience because hats cannot be used in the text of the application software.) is the i-th estimated sound feature data, and "y i " means the i-th sound feature data. In addition, the "MSE i " means the mean square error between the i-th sound feature data and the estimated sound feature data. Similarly, "RMSE" in (Equation 2) i" is the root mean square error between the i-th sound feature data and the estimated sound feature data, which is "MAE" in (Equation 3). i " means the mean absolute error between the i-th sound feature data and the estimated sound feature data. Although not limited thereto, in this embodiment, the formula (1) for the mean square error is used as the formula that can be used in the above-mentioned (a) of "generating a plurality of difference data, which is data based on the difference between the steady sound feature data and the estimated steady sound feature data". This results in a large number of MSE i (where i ranges from 1 to n (n is a natural number greater than or equal to 2)). i Each of these is differential data.

[0042] The formula that can be used in (b) above, "generating noise-removed data, which is the output when differential data is input to a noise removal function that removes noise from differential data," is, for example, a function that functions as a low-pass filter, a function that calculates a moving average, or a function that performs a Fourier transform, removes frequency band signals that exceed a predetermined threshold, and returns the result by performing an inverse Fourier transform. All three of these functions are known to be usable for removing noise from certain data. An example of a formula that functions as a low-pass filter is shown below (Formula 4). An example of a formula for calculating a moving average is shown below (Formula 5). f(MSE i ) = α × MSE i +(1-α)×MSE i-1 …(number 4) *α is a parameter and a constant between 0 and 1. TIFF0007525137000004.tif1979 *m is a parameter and is an integer value of 2 or greater. In the example shown in (Equation 5), the difference data, MSE i is MSE i-(m-1) , …, MSE i-1 , M.S.E. i That is, MSE i MSE including if(MSE i ) is used to calculate the moving average. i You can use the m MSEs after the MSE i MSE including i It is obvious that m MSEs before and after the above may also be used. Although not limited to this, in this embodiment, a function that can be used as a low-pass filter is used as the formula that can be used in the above-mentioned (b) of "generating noise-removed data, which is the output when differential data is input into a noise removal function, which is a function that removes noise from differential data." This allows many f(MSE i ) (where i is 1 to n (n is a natural number greater than or equal to 2)) will be generated. i ) are the denoised data.

[0043] The formula that can be used in (c) above, "generating trend data, which is the output when differential data is input to a trend extraction function, which is a function that extracts the trend of differential data in a specified time period," can be, for example, a function that finds the median of multiple data before, after, or before and after the differential data, or a function that takes the difference series of the differential data. It is known that any of these three functions can be used to extract trends over time for a large amount of data. An example of a formula for calculating the median is shown below (Equation 6). g(MSE i )=Median(MSE i ,MSE i-1 ,···MSE i-(m-1) )……(number 6) *m is a parameter, an integer value of 2 or more The g(MSE i ) represents the trend of the i-th MSE. In this formula, a certain MSE i To find the trend, we use MSE i From MSEi-(m-1) MSE i The data for the m-1 previous MSEs are used, and the median of the m data is calculated as Median(MSE i , MSE i-1 ,···MSE i-(m-1) However, the data used to calculate the median is MSE i and MSE i Not limited to data older than MSE i and MSE i Data after MSE is also acceptable. i and MSE i Data before and after the MSE is also acceptable. i When calculating the median using data before and after, (Equation 6) becomes, for example, the following (Equation 6x). g(MSE i )=Median(MSE i+j ,MSE i+j-1 ,…,MSE i ,MSE i-1 ,…, , MSE i-(m-1) )……(Number 6x) *j and m are parameters, each an integer value of 2 or more Next, an example of a formula for determining a difference sequence is shown in the following (Equation 7). g(MSE i )=MSE i -MSE i-1 …(number 7) g(MSE i ) is the i-th MSE with the effect of trend removed. In this case, the difference sequence is the difference between adjacent MSE values, but the difference sequence does not necessarily have to be the difference between adjacent MSE values; for example, it could be the difference between the MSE and the MSE two values ​​before or after it. Although not limited to this, in this embodiment, the function shown in (Equation 6) for finding the median is used as the formula that can be used in (c) above, which is to "generate trend data, which is the output when differential data is input into a trend extraction function, which is a function that extracts a trend in a specified time period of differential data." This results in a large number of g(MSE i ) (where i is 1 to n (n is a natural number greater than or equal to 2)) will be generated. i ) are trend data.

[0044] After obtaining the noise-removed data and trend data, the loss generating unit 125 executes the above-mentioned calculation (d) of "obtaining a loss based on the noise-removed data and trend data." In (d), for example, the loss can be obtained by obtaining the difference between the noise-removed data and the trend data. The formula used in this case is one that obtains the difference between the noise-removed data and the trend data. Specifically, this calculation is as shown in the following (Equation 8). Loss i =f(MSE i )-g(MSE i )……(number 8) Here, Loss i is the loss calculated as the difference between the i-th noise-removed data and the i-th trend data. i If is negative, replace it with 0 to get Loss i can be obtained as a value equal to or greater than 0, although this is not limited to this in this embodiment. The noise-removed data is conceptually obtained by removing the influence of noise from difference data regarding the difference between sound feature data and estimated sound feature data paired with that sound feature data. When a difference is calculated between data obtained by removing the influence of noise from such difference data and trend data conceptually corresponding to a trend, the data is conceptually obtained by further removing the influence of the trend from data obtained by removing the influence of noise from the difference data. As is clear from the above description, the loss function in this embodiment is defined as a combination of (Equation 1), (Equation 4), (Equation 6), and (Equation 8). Data defining such a loss function is recorded in loss function recording unit 125A. Either way, Loss i (where i is a number between i and n (n is a natural number greater than or equal to 2).) i is the loss between the sound feature data and the estimated sound feature data. In addition, when a function for obtaining a difference series such as that shown in (Equation 7) is used as the trend extraction function, the g(MSE i ) is the i-th MSE with the effect of trend removed. In this case, the above-mentioned calculation (d) executed by the loss generation unit 125 to “obtain a loss based on the noise removed data and trend data” may be, for example, to obtain a loss by applying any of the noise removal functions described above to the trend data. In this case, the trend data is conceptually obtained by removing the influence of trends from difference data regarding the difference between the steady sound feature data and the estimated steady sound feature data paired with that steady sound feature data. Data required by applying a noise removal function to such difference data is conceptually obtained by further removing the influence of noise from data obtained by removing the influence of trends from the difference data. Specifically, this calculation is as follows (number 8x). Loss i =f(g(MSE i ))……(number 8x) In this case, f is a noise removal function, for example, any of the noise removal functions already described. The loss can also be calculated by such a calculation. Therefore, by using such a loss function, the effects of noise and trends specific to sounds generated from an object having a rotating mechanism can be effectively removed from the loss between the steady-state sound feature data and the estimated steady-state sound feature data. In this case, the loss function is defined as a combination of, for example, (Equation 1), (Equation 4), (Equation 7), and (Equation 8x). In this case, data defining such a loss function is recorded in the loss function recording unit 125A.

[0045] The loss generator 125 generates a large number of Losses i The loss recorder 125B is adapted to transmit data on losses, which is a set of data on losses, to the loss recorder 125B. Data on losses is sequentially additionally written to the loss recorder 125B, thereby recording the data on losses.

[0046] The optimization unit 126 has a function of adjusting or updating the data for generating the autoencoder 124, that is, the data recorded in the autoencoder recording unit 124A, so as to minimize the loss function. The optimization unit 126 receives data on losses from the loss generator 125, i.e., Loss i Each time the loss generating unit 125 generates data on the loss and records it in the loss recording unit 125B, it updates the data for generating the autoencoder 124 recorded in the autoencoder recording unit 124A. For example, each time the loss generating unit 125 generates data on the loss, it sends data for prompting the optimization unit 126 to update the data for generating the autoencoder 124 recorded in the autoencoder recording unit 124A, and the optimization unit 126 that receives the data for prompting the update updates the data for generating the autoencoder 124 recorded in the autoencoder recording unit 124A. The optimization unit 126 updates the data for generating the autoencoder 124 recorded in the autoencoder recording unit 124A, for example, in the following manner. In the initial state, the autoencoder recording unit 124A stores a data set that has not been trained but is used to generate the autoencoder 124. As is known, the autoencoder 124 compresses the features of the input data by encoding the input data, and expands the minimized features by decoding the minimized feature data, using multiple weighting coefficients in the process. This data set is the data used to generate the autoencoder 124. When performing the first process of optimizing data for generating the autoencoder 124, the optimization unit 126 reads out the first loss, Loss1, from the loss recording unit 125B, and reads out data for generating the autoencoder 124 from the autoencoder recording unit 124A. Then, the optimization unit 126 adjusts the multiple coefficients, which are data for generating the autoencoder 124, by changing the multiple coefficients so as to minimize the loss. Then, the adjusted coefficients are recorded in the autoencoder recording unit 124A as new data for generating the autoencoder 124. As a result, the data for generating the autoencoder 124 recorded in the autoencoder recording unit 124A is updated. When performing a second process of optimizing the data for generating the autoencoder 124, the optimization unit 126 reads out Loss1 and Loss2 from the loss recording unit 125B and reads out the data for generating the autoencoder 124 from the autoencoder recording unit 124A. Then, for example, the optimization unit 126 adjusts a plurality of coefficients so that the sum of Loss1 and Loss2 is minimized. Then, the adjusted coefficients are recorded in the autoencoder recording unit 124A as new data for generating the autoencoder 124. As a result, the data for generating the autoencoder 124 recorded in the autoencoder recording unit 124A is updated. The same applies to the following cases. When performing the i-th process of optimizing data for generating the autoencoder 124, the optimization unit 126 receives Loss1, Loss2, ... Loss from the loss recording unit 125B. i, and also reads data for generating the autoencoder 124 from the autoencoder recording unit 124A. i The multiple coefficients are adjusted so that the sum of is minimized. Then, each adjusted coefficient is recorded in the autoencoder recording unit 124A as new data for generating the autoencoder 124. As a result, the data for generating the autoencoder 124 recorded in the autoencoder recording unit 124A is updated. In addition, Loss1, Loss2, ... Loss i Alternatively, instead of using all of Loss1, data for generating the autoencoder 124 may be updated sequentially using only Loss1. In this manner, the learning of the autoencoder 124 proceeds. As a result, when the learning device is executing the learning mode, the autoencoder 124 is tuned so as to minimize the difference between the sound feature data and the estimated sound feature data.

[0047] The threshold determination unit 127 may receive data regarding an instruction to generate threshold data from the main control unit 122. Upon receiving the data, the threshold determination unit 127 determines a threshold value by a method to be described later, and sends threshold setting data, which is data specifying the threshold value, to the output unit 128. Details of the threshold value and how the threshold value determining unit 127 determines the threshold value will be described later. The threshold value determining unit 127 is configured to send the threshold value setting data to the output unit 128 at a timing to be described later.

[0048] The output unit 128 has a function of outputting necessary data generated by the functional blocks in the computer device 100 to the outside via the interface 114 . As described above, the output unit 128 may receive the trained autoencoder data and the threshold setting data. Upon receiving the data, the output unit 128 outputs the data to the interface 114.

[0049] The use and operation of the learning device will now be described. As described above, the learning device functions alternatively in either of two modes, the learning mode and the threshold determination mode. Therefore, in order to use the learning device, it is first necessary to determine in which mode the learning device is to function. As will be described later, the threshold is determined depending on the characteristics of the trained autoencoder after training in the learning mode, so that the threshold cannot be determined before the trained autoencoder is completed. Therefore, in the learning device, the learning mode is executed first, and then the threshold determination mode is executed.

[0050] To cause the learning device or computer device 100 to execute the learning mode, first, mode selection data is input from the input device 102 of the computer device 100. The mode selection data is used to select the mode to be executed by the learning device from the two modes mentioned above. In this case, the user operates the input device 102 to input data from the input device 102 as mode selection data indicating that the learning device is to function in the learning mode. This data is sent from the input device 102 to the input section 121 via the interface 114, and then from the input section 121 to the main control section 122. Upon receiving this data, the main control section 122 issues an instruction to each functional block to execute the learning mode.

[0051] In this state, the learning device executes the learning mode. Microphone 103 has a function of collecting sounds including sounds emitted by an object and generating sound data, which is data of the collected sounds. Therefore, microphone 103 is placed at a position where it can at least sense the sounds emitted by the object. For example, if the object is a motor, microphone 103 is placed at a position where it can collect the sounds generated by the motor. However, the sound collected by the microphone 103 when the learning mode is executed is the sound when the object is in a stationary state. This sound may include environmental sound. This is acceptable as long as the sound collected by the microphone when the detection device is used later also includes environmental sound. This means that the sound collection environment of the microphone 103 when the learning mode is executed and when the detection device is used may be the same. The definition of the steady state can be appropriately determined by the user who operates the learning device. The microphone 103 generates sound data, which is data of collected or sensed sound. The sound data is amplified by an amplifier as necessary, and in this embodiment, is sent to the computer device 100 in the learning device, which is connected by wire. The sound data is sent from the interface 114 to the input unit 121. The sound data is sent from the input unit 121 to the feature amount detection unit 123.

[0052] The feature amount detection unit 123 detects the feature amount of the received sound data, and generates sound feature amount data that is data regarding the feature amount of the sound. The sound feature data may be in any format, but in this embodiment, the data format is a Mel-frequency spectrogram, which is a known or well-known data format for expressing sound features. First, the feature detection unit 123 arranges the received sound data. In this embodiment, but not limited to this, the feature detection unit 123 converts the sound data sent as analog data into digital data of a predetermined format (in this embodiment, but not limited to this, wav format), and arranges the time length of the converted data to data of a predetermined time (for example, 10 seconds or 30 seconds, but not limited to this, 60 seconds in this embodiment). A publicly known or well-known method can be adopted as a method for obtaining sound feature data, which is a Mel frequency spectrogram, from the adjusted sound data. For example, a Mel frequency spectrogram can be obtained by converting a sound data file into power spectrum data by performing a short-time Fourier transform (STFT), applying a Mel filter bank to the converted data, and smoothing the data to which the Mel filter bank has been applied by performing a logarithmic operation. Of course, a Mel frequency spectrogram can be obtained by other publicly known or well-known methods. Note that in order to capture dynamic temporal changes in sound, rather than the sound itself with a frequency in the audible range, the feature detection unit 123 may detect a delta Mel frequency spectrogram that extracts the difference between previous and subsequent frames of a Mel frequency spectrogram. In any case, once the sound feature data is generated as a Mel-frequency spectrogram, the feature detection unit 123 sends the generated sound feature data to the autoencoder 124 and also to the loss generation unit 125.

[0053] The autoencoder 124 receives the sound feature data sent from the feature detection unit 123. More specifically, the sound feature data is input to an input layer of the autoencoder 124. The autoencoder 124 generates estimated sound feature data by encoding the input sound feature data and subsequently decoding the encoded data. Before encoding and decoding the sound feature data, the autoencoder 124 reads a data set for generating the autoencoder 124 from the autoencoder recording unit 124A. This data reading is performed every time sound feature data is encoded and decoded. Therefore, the autoencoder 124 is in the latest state based on the latest data for generating the autoencoder 124 recorded in the autoencoder recording unit 124A. Before encoding and decoding the first sound feature data, the autoencoder 124 is a known appropriate autoencoder 124. When the second or subsequent sound feature data is input, the autoencoder 124 is in a state where learning has progressed. As described above, the autoencoder 124 outputs estimated sound feature data. The autoencoder 124 sends the generated estimated sound feature data to the loss generating unit 125.

[0054] As described above, the loss generation unit 125 receives the sound feature data from the feature detection unit 123 and the estimated sound feature data from the autoencoder 124. In short, the sound feature data and the estimated sound feature data are a set of data that respectively correspond to the input and output when the autoencoder 124 performs one encoding and decoding process. The loss generation unit 125 calculates a loss by taking the difference between the set of sound feature data and the estimated sound feature data. When calculating the loss, the loss generation unit 125 uses a loss function, and the loss generation unit 125 reads data specifying the loss function from the loss function recording unit 125A at an appropriate timing before calculating the loss. It is sufficient that the data specifying the loss function is calculated only once before the first loss is calculated. The method of generating data on the loss between the sound feature data and the estimated sound feature data using the loss function has already been described. iAs mentioned above, data on a number of losses are generated. i The set of data is data on loss generated from one sound data. A series of Loss i An example of losses arranged along the x-axis with respect to time is shown in FIG. The loss generating unit 125 sends the loss data generated as described above to the loss recording unit 125B.

[0055] Each time new loss data is recorded in the loss recorder 125B, the optimizer 126 modifies the autoencoder 124 to minimize the loss. More specifically, in this embodiment, each time new loss data is recorded in the loss recording unit 125B, the optimization unit 126 reads from the autoencoder recording unit 124A the data for generating the autoencoder 124 that was recorded in the autoencoder recording unit 124A at that time, i.e., data on multiple coefficients, and reads from the loss recording unit 125B all loss data that was previously recorded in the loss recording unit 125B. In this embodiment, the optimization unit 126 calculates the read loss i The data for generating the autoencoder 124 that was read out is then corrected so that the sum of the coefficients is minimized. Then, the data for generating the autoencoder 124 that was previously recorded in the autoencoder recording unit 124A is overwritten with the data for generating the autoencoder 124 that was newly generated by the correction.

[0056] While the learning device is executing the learning mode, in the computer device 100 constituting the learning device, each process from the process of inputting sound data from the input unit 121 to the process of optimizing multiple coefficients contained in the data for generating the autoencoder 124 recorded in the autoencoder recording unit 124A in order to minimize loss, i.e., the additional learning process, is repeatedly executed. By repeating this process multiple times, for example, but not limited to, about 200 times (each time is counted as one time when all the sound feature data provided in the input process has been used at least once), the loss function, which is the difference between the sound feature data as the input to the autoencoder 124 and the estimated sound feature data as the output from the autoencoder 124, will not be the same, but will be small enough to fall within a certain range. Since the sound data used for learning is about sounds including sounds when the object is in a stationary state, the autoencoder 124 is dedicated to the time when the object is in a stationary state, and has a function of making the sound feature data when the object is in a stationary state and the estimated sound feature data created from that sound feature data almost the same. When the input and output relationship when the object is in a stationary state satisfies the above-mentioned relationship, the autoencoder 124 can be said to be completed as a trained autoencoder. As described above, a large amount of sound data is required to obtain a trained autoencoder, but the sound data need only be about sounds when the object is in a stationary state. In general, the time when the object is in a stationary state is overwhelmingly longer than the time when the object is in a non-stationary state, so it is relatively easy to collect such sound data. In addition, when collecting sound data, it is not necessary to collect sound data about sounds in a non-stationary state, and it is also not necessary to collect sound data about sounds in multiple stages of different types of non-stationary states. In addition, since the sound data used for training the autoencoder 124 is only data in a stationary state, labeling of the sound data is not necessary. In addition, so-called noise canceling processing for removing, for example, environmental sounds from the sound data is at least not essential. In this embodiment, the sound feature data created from the sound data input from the microphone 103 is input to the autoencoder 124 one after another. However, if a large number of sound data are generated in advance and recorded in a recording medium (which may be built in the computer device 100 or may be present outside the computer device 100), the sound data read by the computer device 100 from the recording medium can be sent to the feature detection unit 123 one after another, so that the autoencoder 124 can learn in the same manner as described above. Similarly, if a large number of sound feature data are recorded in a recording medium, the sound feature data can be supplied to the autoencoder 124 one after another, so that the autoencoder 124 can learn in the same manner as described above. In other words, the generation of sound data by the microphone 103 and the learning of the autoencoder 124 do not need to be performed consecutively in time.

[0057] Once a trained autoencoder has been generated in this manner, the next step is to determine a threshold. The threshold value is used by the detection device as described below, and is used when determining whether a sound identified by sound data input to the detection device as described below has non-stationary characteristics (whether it includes a non-stationary sound). When the threshold is determined, the learning device or the computer device 100 constituting the learning device executes the threshold determination mode as described above. The user inputs mode selection data to select the threshold determination mode from the input device 102 of the computer device 100. As described above, the mode selection data is sent from the input device 102 to the input unit 121 via the interface 114, and then from the input unit 121 to the main control unit 122. The main control unit 122 receives this data and instructs each functional block to execute the threshold determination mode. For example, the main control unit 122 instructs the threshold determination unit 127 to determine a threshold.

[0058] In this state, the learning device executes the threshold determination mode. The process when the threshold determination mode is executed is as follows. The threshold determination unit 127 reads, from the loss recording unit 125B, data on all losses that have been generated in the course of learning by the autoencoder 124 and recorded in the loss recording unit 125B. As mentioned above, each loss data is a set of multiple Losses. i It is a collection of data. In this embodiment, the loss recorder 125B records a number of Losses included in the data on multiple losses. i Among these, the maximum Loss that occurs when the object is in a steady state is i In other words, the maximum loss that can be tolerated from the sound emitted by the object while considering the object to be in a steady state is determined. The method for determining such a loss value can be, for example, by statistical processing. In this embodiment, a large number of Loss i The third quartile of the Loss i The maximum loss value that occurs when such an object is in a steady state is determined by multiplying the value of the smallest one by 1.5 when the smallest one is arranged in ascending order. i The value obtained in this way can be conveniently referred to as the reference threshold (Loss max ) Although not limited to this, the threshold value determining unit 127 in this embodiment further determines several threshold values ​​based on this reference threshold value. Next, a new threshold is determined by multiplying the standard threshold by a certain coefficient. The coefficient multiplied by the standard threshold is a value greater than 1. Therefore, this new threshold is greater than the standard threshold. This new threshold is called the upper loss threshold (Loss limit) The upper loss threshold conceptually corresponds to the maximum expected loss value among losses occurring in a non-steady state (naturally, losses occurring in a non-steady state are generally greater than losses occurring in a steady state). The above-mentioned coefficient to be multiplied by the standard threshold is determined so that the value of the upper loss threshold corresponds to this conceptual meaning. This coefficient is generally set to a value between 1.5 and 3.0. However, the value of the coefficient does not have to be within the above range as it may depend on the environment of the measurement target. Although not limited to this, in this embodiment, a further threshold value is set between the reference threshold value and the upper loss threshold value. This threshold value is called a non-stationary intermediate threshold value (Loss interH ) The conceptual meaning of the non-stationary intermediate threshold is as follows. As described above, the reference threshold is the maximum loss value that occurs when the object is in a steady state. Therefore, a loss value exceeding the reference threshold suggests that the object is in a non-stationary state. However, the strength of the suggestion that the object is in a non-stationary state varies depending on whether the loss value exceeds the reference threshold slightly or greatly. The non-stationary intermediate threshold has the meaning that a loss value located between the reference threshold and the non-stationary intermediate threshold relatively weakly suggests that the object is in a non-stationary state, and a loss value located between the non-stationary intermediate threshold and the upper loss threshold relatively strongly suggests that the object is in a non-stationary state. The non-stationary intermediate threshold having such conceptual meaning can be calculated, for example, by the following formula (Mathematical Expression 9). Loss interH =Loss max +1 / 2(Loss limit -Loss max )……(Number 9) In this embodiment, the non-stationary intermediate threshold is one. max That's all, Loss limit The following region is divided into two regions, with a non-stationary intermediate threshold in between. max That's all, Loss limitThe following regions can also be divided into two or more regions. For example, if the region is divided into three regions, two non-stationary intermediate thresholds should be set, if the region is divided into four regions, three non-stationary intermediate thresholds should be set, and if the region is divided into n regions, n-1 non-stationary intermediate thresholds should be set. Although not limited to this, in this embodiment, the threshold determination unit 127 further determines a threshold in a region where the value is smaller than the reference threshold. When the loss value is smaller than the reference threshold, it is suggested that the object is in a steady state. However, the strength of the suggestion that the object is in a steady state changes depending on whether the loss is slightly smaller than the reference threshold or small enough to be close to 0. Just as the non-stationary intermediate threshold is used to divide the range between the reference threshold and the upper loss threshold into multiple regions, a threshold is set to divide the range where the loss is between 0 and the reference threshold. This threshold is called the stationary intermediate threshold (Loss interL ) The significance of the steady-state intermediate threshold in this embodiment is as follows. As described above, a loss whose value is smaller than the reference threshold suggests that the object that emitted the sound that generated the loss is in a steady state. However, if the loss value is smaller than the reference threshold but is large enough to be close to the reference threshold, for example, it is suspected that the object is in a non-steady state. A reasonable threshold for raising such suspicion is set as the steady-state intermediate threshold. A steady-state intermediate threshold that conceptually has this significance can be calculated, for example, by the following formula (Mathematical Expression 10). Loss interL =Loss max ×0.75……(number 10) As described above, in this embodiment, the threshold determination unit 127 generates the reference threshold, the upper limit loss threshold, the unsteady intermediate threshold, and the steady intermediate threshold. The threshold determination unit 127 generates threshold setting data, which is data for setting each of these thresholds in the detection device.

[0059] In this manner, the autoencoder 124 becomes a trained autoencoder 124, and the threshold used in the trained autoencoder 124 is determined. These are transplanted and used in the detection device described later. The process of transferring these data from the learning device to the detection device, or the process of transferring these data from the computer device 100 included in the learning device to the computer device included in the detection device, may be appropriately performed using publicly known or well-known techniques. For example, in accordance with the input input from the input device 102, through the interface 114, and then through the input unit 121 to the main control unit 122, the main control unit 122 may send an instruction to the optimization unit 126 to read a set of data for generating the trained autoencoder 124 from the autoencoder recording unit 124A and send it to the output unit 128, and may also send an instruction to the threshold determination unit 127 to send data of the determined threshold to the output unit 128. If this is the case, the output unit 128 sends the data of the trained autoencoder 124 and the threshold data to the interface 114. For example, the data is sent from the interface 114 to the computer device 100 included in the detection device via a predetermined cable. Alternatively, the data is recorded from the interface 114 to a recording medium connected to the interface 114, and sent to the computer device 100 included in the detection device via the recording medium.

[0060] <About the detector> As already mentioned, in terms of hardware configuration, including the peripherals, the learning device and the detection device may be the same, and although not limited to this, this is effectively the case in this embodiment. The detection device includes a computer device 100X having a hardware configuration equivalent to that of the computer device 100 in the learning device, a display 101X equivalent to the display 101 in the learning device, an input device 102X equivalent to the input device 102 in the learning device, and a microphone 103X equivalent to the microphone 103 of the learning device, which are each connected to the computer device 100X. The computer device 100 and the computer device 100X, the display 101 and the display 101X, the input device 102 and the input device 102X, and the microphone 103 and the microphone 103X can be equivalent, or may be completely identical. Also, if the learning device has an amplifier that amplifies the sound data, the detection device also includes an amplifier. These two amplifiers may be equivalent, or may be completely identical. Incidentally, if the detection device were illustrated, it would simply be the same as each reference numeral in FIG. 1 with an X added, and therefore the detection device is omitted from the illustration.

[0061] As described above, the hardware configuration of the computer device 100X of the detection device is equivalent to the hardware configuration of the computer device 100 of the learning device. The computer device X of the detection device includes a CPU 111X, a ROM 112X, a RAM 113X, an interface 114X, and a large-capacity recording medium 115X, which are interconnected by a bus 116X. When viewed as hardware, the functions of the CPU 111X, ROM 112X, RAM 113X, interface 114X, mass storage medium 115X, and bus 116X of the computer device 100X of the detection device are equivalent to or identical to the functions of the CPU 111, ROM 112, RAM 113, interface 114, mass storage medium 115, and bus 116 of the computer device 100 of the learning device. Note that if the hardware configuration of the computer device 100X of the detection device were illustrated, it would simply be the same as that of FIG. 2 with an X added to each of the reference characters, and therefore the illustration of the hardware configuration of the computer device 100X of the detection device is omitted. The difference between the computer device 100X of the detection device and the computer device 100 of the learning device is, at most, that the computer program recorded in the ROM 112X or the large-capacity recording medium 115X of the computer device 100X of the detection device is different from the computer program recorded in the computer device 100 of the learning device. The computer program recorded in the computer device 100X of the detection device includes a computer program for causing the computer device 100X of the detection device to execute the processing described below that is necessary for the computer device 100X to function as a detection device. However, just as in the case of computer device 100 or a computer program installed on computer device 100, this computer program may be pre-installed or post-installed in computer device 100X, and the computer program may be installed in computer device 100X via a recording medium or via a network. Furthermore, computer device 100X may have an OS and other necessary computer programs installed in addition to the above-mentioned computer program.

[0062] 6 are generated in the computer device 100X by the CPU 111X executing the computer program. Note that the following functional blocks may be generated by the functions of the above-mentioned computer program alone for causing the computer device 100X to execute the processes described below that are necessary for the computer device 100X to function as a detection device, or may be generated by cooperation between the above-mentioned computer program and an OS or other computer programs installed in the computer device 100. Within the computer device 100, in relation to the functions of the present invention, an input unit 121X, a main control unit 122X, a feature detection unit 123X, a first calculation unit 124X, a loss generation unit 125X, a state determination unit 126X, a first recording unit 127X, and an output unit 128X are generated (FIG. 6). Among these, the functional blocks that execute information processing, i.e., the main control unit 122X, the feature detection unit 123X, the first calculation unit 124X, the loss generation unit 125X, and the state determination unit 126X, correspond to a calculation device (CPU 111X in this embodiment) in terms of hardware, or are realized by the functions of a calculation device that executes information processing in accordance with the instructions of the above-mentioned computer program. When viewed as hardware, the input unit 121X and the output unit 128X correspond to the interface 114X, or are realized by the function of the interface 114X. More specifically, the input unit 121X and the output unit 128X conceptually correspond to the connection portion between the interface 114X and the bus 116X. Moreover, the functional block for recording data, that is, the first recording unit 127X, corresponds in terms of hardware to a recording device, for example, the RAM 113X or the large-capacity recording medium 115X, or is realized as one area of ​​such a recording device. Furthermore, a combination of the loss generating unit 125X and the state determining unit 126X corresponds to the second calculation unit referred to in the present application.

[0063] The input unit 121X receives an input from the interface 114X. The input from the interface 114X to the input unit 121X includes an input from the input device 102X. The input from the input device 102X includes, for example, threshold setting data, which will be described in detail later. When threshold setting data is input from the input device 102X, the threshold setting data is sent from the input unit 121X to the main control unit 122X. The data input from the interface 114X to the input unit 121X also includes sound data from the microphone 103. When the sound data is received by the input unit 121X, the input unit 121X sends the sound data to the feature detection unit 123X. The main control unit 122X performs overall control of the above-mentioned functional blocks generated in the computer device 100X. For example, the main control unit 122X may receive threshold setting data as described above. The main control unit 122X that receives the threshold setting data sends it to the state determination unit 126X.

[0064] The feature detection unit 123X has a function equivalent to that of the feature detection unit 123 of the computer device 100 of the learning device. That is, the feature detection unit 123X detects the feature of the received sound data and generates sound feature data, which is data on the feature of the sound. Like the feature detection unit 123 of the learning device, the feature detection unit 123X prepares the sound data as necessary and then executes the feature detection process. The sound feature data generated by the feature detection unit 123X is in the same format as the sound feature data created by the feature detection unit 123. Therefore, in this embodiment, the sound feature data generated by the feature detection unit 123X is a Mel frequency spectrogram, although it is not limited to this. The feature detection unit 123X is configured to send the generated sound feature data to the first calculation unit 124X and the loss generation unit 125X.

[0065] The first calculation unit 124X actually functions as a trained autoencoder 124 created by a learning device. The first recording unit 127X records a set of data for generating the trained autoencoder 124 created by the learning device, that is, a set of multiple coefficients. When performing the calculations described below, the first calculation unit 124X reads the trained autoencoder data from the first recording unit 127X each time, and itself functions as a trained autoencoder. The first calculation unit 124X functions as a trained autoencoder completed by a learning device, and therefore has a function of encoding input data, which is predetermined data, and then decoding the encoded data to restore it to data of the same dimension as the input data. That is, the first calculation unit 124X functions as an autoencoder. In this embodiment, the data input to the virtual trained autoencoder present in the first calculation unit 124X is sound feature data input from the feature detection unit 123X. The first calculation unit 124X functioning as a trained autoencoder is configured to output estimated sound feature data when sound feature data is input, just like the trained autoencoder 124 of the learning device. The first calculation unit 124X is configured to send the generated estimated sound feature data to a loss generation unit 125X.

[0066] The loss generator 125X has the same functions as the loss generator 125 present in the computer device 100 of the learning device. The first recording unit 127X records data for specifying a loss function that is the same as that recorded in the loss function recording unit 125A. When calculating a loss as described below, the loss generating unit 125X reads data for specifying a loss function from the first recording unit 127X at least once prior to the calculation, and calculates a loss based on the data using the same loss function as that used in the learning device. As described above, the loss generation unit 125X is configured to receive sound feature data from the feature detection unit 123X and estimated sound feature data from the first calculation unit 124X. In short, the sound feature data and estimated sound feature data are a set of data that respectively correspond to the input and output when encoding and decoding are performed once by an autoencoder generated by the function of the first calculation unit 124X. The loss generation unit 125X is configured to calculate the loss between the set of sound feature data and the estimated sound feature data and generate data regarding the loss using the same loss function as that used in the loss generation unit 125 in the computer device 100 of the learning device. The loss generating unit 125X is configured to send the generated loss data to the state determining unit 126X.

[0067] The state determination unit 126X has a function of determining whether the sound specified by the sound data that is the source of the loss includes a sound indicating that the object is in an unsteady state (whether it was an unsteady sound) or whether the object that emitted the sound specified by the sound data was in an unsteady state, based on the loss data received from the loss generation unit 125X. How such a determination is made will be described later. The first recording unit 127X records threshold setting data generated by the learning device. Prior to making the above-mentioned judgment, the state judgment unit 126X reads the threshold setting data from the first recording unit 127X and uses the thresholds generated by the threshold setting data, that is, the reference threshold, the upper limit loss threshold, the unsteady intermediate threshold, and the steady intermediate threshold, for the above-mentioned judgment. When the sound specified by the sound data that caused the loss includes a sound indicating that the object is in an unsteady state, or when the object that emitted the sound specified by the sound data is determined to be in an unsteady state, the state determination unit 126X generates result data indicating that. Note that the result data may be generated only when such a determination is made, but when the sound specified by the sound data that caused the loss does not include a sound indicating that the object is in an unsteady state (i.e., it is a steady sound), or when the object that emitted the sound specified by the sound data is not in an unsteady state (i.e., it is in a steady state), result data indicating that may be generated, although not limited thereto, is so in this embodiment. The result data generated in this case is of course different from the result data generated when the object that emitted the sound specified by the sound data is determined to be in an unsteady state. The state determination unit 126X outputs the generated result data to the output unit 128X.

[0068] The output unit 128X has a function of outputting necessary data generated by the functional blocks in the computer device 100X to the outside via the interface 114X. The output unit 128X may receive the result data, as described above. Upon receiving the result data, the output unit 128X is configured to output the result data to the interface 114X.

[0069] The use and operation of the detection device will now be described. Microphone 103X, like microphone 103 included in the learning device, collects sounds including sounds emitted by objects and generates sound data, which is data on the collected sounds. The position at which microphone 103X is placed corresponds to the position at which microphone 103 of the learning device is placed. When the microphone 103X of the detection device collects sound, it is not clear whether the object is in a steady state or a non-steady state, and therefore the sound collected by the microphone 103X may be a steady sound or a non-steady sound. This is natural due to the nature of the detection device, since the detection device detects that an object that is originally expected to be in a steady state has become a non-steady state. Therefore, the sound collected by the microphone 103X may include a sound emitted by an object in a steady state, or a sound emitted by an object in a non-steady state. The microphone 103X generates sound data, which is data of collected or sensed sound. The sound data is amplified by an amplifier as necessary, and is sent to the computer device 100X, which is connected by wire in this embodiment.

[0070] From this point on, until a loss function is generated by the loss generation unit 125X, the processing is the same as the processing until a loss function is generated by the loss generation unit 125 in the computer device 100 of the learning device, and the processing conditions such as the specifications of each data, the method of generating sound feature data, the method of generating a loss function, etc. are all the same.

[0071] The sound data is sent from the interface 114X to an input unit 121X and further to a feature amount detection unit 123X. The sound data is prepared by the feature detection unit 123X. In this embodiment, the sound data is converted into a 60-second wav file, as was done in the learning device. The prepared sound data is sent from the input unit 121X to the feature detection unit 123X. The feature detection unit 123X detects the feature of the arranged sound data and generates sound feature data which is data on the feature of the sound. The feature detection unit 123X generates sound feature data which is a Mel frequency spectrogram from the sound data in the same manner as that executed by the feature detection unit 123 in the computer device 100 in the learning device. If what is detected by the learning device is not the sound itself of a frequency in the audible range, but a delta Mel frequency spectrogram which extracts the difference between previous and subsequent frames of the Mel frequency spectrogram in order to capture dynamic temporal changes in the sound, the feature detected by the feature detection unit 123X may be a delta Mel frequency spectrogram. When the sound feature data is generated as a Mel-frequency spectrogram, the feature detection unit 123X sends the generated sound feature data to a first calculation unit 124X functioning as a trained autoencoder, and also sends it to a loss generation unit 125X.

[0072] The sound feature data sent from the feature detection unit 123X is input to the trained autoencoder in the first calculation unit 124X. Then, the first calculation unit 124X serving as the trained autoencoder outputs estimated sound feature data. The estimated sound feature data output from first calculation unit 124X is sent to loss generating unit 125X.

[0073] The loss generation unit 125X receives sound feature data from the feature detection unit 123X and estimated sound feature data from the first calculation unit 124X. The sound feature data and estimated sound feature data are a pair of data corresponding to the input and output when encoding and decoding are performed once by the autoencoder in the first calculation unit 124X. The loss generation unit 125X generates a loss for the pair of sound feature data and estimated sound feature data. The method in which the loss generation unit 125X generates a loss is the same as the method in which the loss generation unit 125 of the computer device 100 in the learning device generates a loss. The loss generating unit 125X sends the loss data generated as described above to the state determining unit 126X.

[0074] The state determination unit 126X determines, based on the received loss data, whether the sound identified by the sound data that is the source of the loss includes a sound indicating that the object is in an unsteady state, or whether the object that emitted the sound identified by the sound data is in an unsteady state. The method of judgment is as follows. As described above, the trained autoencoder contained in the first calculation unit 124X is trained only on sounds (steady sounds) including sounds generated from objects in a steady state, and when sound feature data generated from a stationary sound (in this embodiment, a Mel frequency spectrogram) is input, it outputs estimated sound feature data that is almost the same as the sound feature data (a Mel frequency spectrogram that may be called an "estimated Mel frequency spectrogram"). On the other hand, the trained autoencoder contained in the first calculation unit 124X is not trained on sounds (unsteady sounds) including sounds generated from objects in a non-steady state, and when sound feature data generated from a non-steady sound is input, the estimated sound feature data that is output is significantly different from the input sound feature data. Therefore, the loss corresponding to the difference between the sound feature data and the estimated sound feature data is relatively small in the former case and relatively large in the latter case. Therefore, based on the magnitude of the loss, state determination unit 126X can distinguish whether the object is in a steady state (whether the sound input to microphone 103X is a steady sound) or whether the object is in a non-steady state (whether the sound input to microphone 103X is a non-steady sound).

[0075] In this embodiment, although not limited thereto, the thresholds set in the state determination unit 126X as described above, that is, the reference threshold, the upper limit loss threshold, the non-steady intermediate threshold, and the steady intermediate threshold, are used for the distinction. In order to use these thresholds, the state determination unit 126X reads the threshold setting data from the first recording unit 127X before making a determination. In this embodiment, the state determination unit 126X uses a reference threshold, an upper limit loss threshold, a non-steady intermediate threshold, and a steady intermediate threshold to determine whether sound data regarding a sound emitted by an object includes a sound indicating that the object is in a non-steady state, or whether the object that emitted the sound identified by the sound data is in a non-steady state, as follows. It is assumed that the loss due to the loss data received by the state determination unit 126X is the same as that shown in FIG. 5. Then, the reference threshold (Loss max ), upper loss threshold (Loss limit ), non-stationary intermediate threshold (Loss interH ), stationary intermediate threshold (Loss interL ) is depicted in Figure 7. The vertical axis for losses is thus divided into two segments above the reference threshold and below the upper loss threshold, and into two segments below the reference threshold (see the thick dashed lines in Figure 7). Moreover, the state determination unit 126X finely divides the vertical axis direction of the graph into range values ​​indicated by thin lines that are regions separating the graphs. The interval between adjacent range values ​​above and below, and the interval between a certain range value and the reference threshold, upper limit loss threshold, unsteady intermediate threshold, or steady intermediate threshold adjacent to that range value, are generally constant (see the thick and thin dashed lines in FIG. 7). For example, the range value is determined as follows. In this embodiment, first, a stationary intermediate threshold (Loss interL ) into which the area below the Loss threshold is to be divided is determined. For example, in this embodiment, seven range values ​​are determined to divide the range into eight equal parts. Then, the steady-state intermediate threshold (Loss interL ) determines the spacing between adjacent range values. Then, the stationary intermediate threshold (Loss interL ) and the reference threshold (Loss max ) and the reference threshold (Loss max ) and the non-stationary intermediate threshold (Loss interH ) and the non-stationary intermediate threshold (Loss interH ) and the upper loss threshold (Loss limit ) to separate the stationary intermediate threshold (Loss interL ) and the reference threshold (Loss max ) are divided into three by two range values ​​(Range value), and the reference threshold (Loss max ) and the non-stationary intermediate threshold (Loss interH ) and the non-stationary intermediate threshold (Loss interH ) and the upper loss threshold (Loss limit ) is divided into two by a range value. In this way, the area in FIG. 7 is divided into a vertical axis along the reference threshold (Loss max ), upper loss threshold (Loss limit ), non-stationary intermediate threshold (Loss interH ), stationary intermediate threshold (Loss interL ) and range values ​​(Rangevalue) into 15 regions (ranges). In Figure 7, each region between adjacent thick and thin lines is a range. Naturally, the interval between range values ​​and the number of range divisions can be set by parameters according to the characteristics of the target device.

[0076] In this way, the reference threshold (Loss max ), upper loss threshold (Losslimit ), non-stationary intermediate threshold (Loss interH ), stationary intermediate threshold (Loss interL ) and the ranges are determined, the state determination unit 126X counts the number of losses with magnitudes that belong to each range. For example, in FIG. 7, the loss marked with X belongs to the second range from the top. In addition, whether the loss at the boundary value of the adjacent region (i.e., the same value as any of the threshold values ​​or range values) is counted in the adjacent upper or lower range can be determined appropriately, and although not limited to this, in this embodiment, it is counted in the upper range where the loss is large. limit The rules for counting the number of losses can be determined appropriately, such as counting the top range if there is a loss greater than 0, or counting the bottom range if there is a loss less than 0.

[0077] The number of losses counted in this way is shown in graphs in Fig. 8 and Fig. 9. The horizontal axis of this graph is the magnitude of the loss, and the vertical axis is the number of losses. max ), upper loss threshold (Loss limit ), non-stationary intermediate threshold (Loss interH ), stationary intermediate threshold (Loss interL ) is shown. Scales without annotations are range values. The range values ​​may be generated by the detection device, or may be generated by the learning device, for example, in a form that is included in the threshold setting data. Figure 8 shows a typical histogram when the object is in a stationary state, and Figure 9 shows a typical histogram when the object is in a non-stationary state. In the case of Figure 8, most of the losses are below the reference threshold (Loss max ) and are concentrated below the reference threshold (Loss max) is hardly present. This means that most of the losses are extremely small, that is, the difference between the sound feature data used to calculate the losses and the estimated sound feature data is extremely small. More specifically, this means that the sound data that is the source of the sound feature data generated in the detection device is extremely close to the sound data when the first calculation unit 124X functioning as an encoder learned (i.e., sound data based on the sound emitted by an object in a steady state). This means that the sound from the object currently being determined by the detection device is the sound from an object in a steady state. In the case of Figure 9, the reference threshold (Loss max ) are found to have many losses. In particular, the non-stationary intermediate threshold (Loss interH ) is also present in a considerable number of cases. This means that there are many losses of a certain magnitude or more, in other words, that there is a large difference between the sound feature data used to calculate the loss and the estimated sound feature data. More specifically, this means that the sound data that is the source of the sound feature data generated in the detection device is different from the sound data when the first calculation unit 124X functioning as an encoder learned (i.e., sound data based on a sound emitted by an object in a steady state). This means that the sound from the object currently being determined by the detection device is a sound from an object in a non-steady state. The state determination unit 126X can determine whether the object is in a steady state or a non-steady state, for example, according to the following rules. 1. Criterion threshold (Loss max ) or more than XX% of the total number of losses 2. Non-stationary intermediate threshold (Loss interH ) or more than XX% of the total number of losses 3. Criterion Threshold (Loss max ) is greater than XX, or is greater than XX% of the total number of losses, and is greater than the stationary intermediate threshold (Loss interL) or more than YY losses, or YY% or more of the total number of losses The state determination unit 126X may be configured to determine whether the object is in a steady state or a non-steady state from the loss received using the above algorithm, but may also be configured to make the determination using a predetermined artificial intelligence. In any case, the state determination unit 126X generates image data for displaying on a predetermined display a histogram such as that shown in Fig. 8 or 9 together with text such as "The object is in a steady state" or "The object is in an unsteady state." This corresponds to the result data referred to in this application. Although not limited to this, in this embodiment, the determination made by the state determination unit 126X is assumed to be in two stages. However, the determination made by the state determination unit 126X may be in, for example, three stages or five stages between the object being in a steady state and the object being in a non-steady state. Also, it is not essential that the result data includes the above-mentioned text data or image data for displaying a histogram on a specified display.

[0078] As described above, the state determination unit 126X generates result data according to the determination result. The result data may or may not be generated only when it is determined that the object is in a non-stationary state or the sound input to the microphone 103X includes a non-stationary sound. In this embodiment, although not limited thereto, it is assumed that the result data is generated both when it is determined that the object is in a stationary state or the sound input to the microphone 103X is a stationary sound and when it is determined that the sound is a non-stationary sound.

[0079] The resulting data will be used as needed. In this embodiment, but not limited to this, the state determination unit 126X sends the result data to the output unit 128X. The output unit 128X outputs the result data to another device outside the computing device 100X via the interface 114X. The other device is, for example, a display (not shown) connected to the computer device 100X. The screen of the display displays the above-mentioned text and histogram according to the result data. Another example of the other device may be a patrol lamp connected to the computer device 100X, which rings when it receives the result data to notify people around that the object is in an unsteady state. In this case, the result data is data for operating the patrol lamp, and it is self-evident that it does not need to include the above-mentioned text or image data of the histogram. It is obvious that the means for notifying people around that the object is in an unsteady state is not limited to the patrol lamp. The above-mentioned display and patrol lamp do not need to be connected to the computer device 100X. For example, the above-mentioned display and patrol lamp may be connected to another computer device that is located remotely from the computer device 100X and can communicate with the computer device 100X, for example, via the Internet. Even in this case, it will be obvious to those skilled in the art that it is possible to display the above-mentioned text and histogram images on the display that has received the result data, and that it is possible to sound the patrol lamp that has received the result data.

[0080] In this embodiment, the threshold value set in the state determination unit 126X of the detection device using the threshold setting data generated by the learning device and used by the state determination unit 126X to make the above-mentioned determination based on the received loss data is a reference threshold value (Loss max ), upper loss threshold (Loss limit ), non-stationary intermediate threshold (Loss interH ), stationary intermediate threshold (Loss interL ) and also range values ​​similar to threshold values ​​were used. This is not limited to the above. For example, the threshold value generated by the learning device and specified by the threshold setting data may be, for example, a reference threshold value (Loss max ) may be used. In this case, the state determination unit 126X of the detection device may simply count the number of losses that exceed a reference threshold, and if that number (or the percentage of the total losses) is greater than a certain number (or percentage), determine that the sound data regarding the sound emitted by the object includes a sound indicating that the object is in an unsteady state.

[0081] <Test Example> Test examples will be described below. The test example described below was carried out using the detection device described in the first embodiment, which has data from a trained autoencoder that was trained by executing the training mode using the training device described in one embodiment. In the test example, a test jig (hereafter referred to as the test device) simulating an electric drill that has a rotation mechanism and can intentionally generate abnormal vibrations was used as the object for determining whether it was in a steady state or an unsteady state, because the frequency of abnormalities occurring in actual equipment is extremely low and the causes of the abnormalities may be limited. This test device can reproduce not only normal (steady) conditions of an electric drill with a rotation mechanism, but also abnormal (unsteady) conditions. This test device is generally used for the purpose of predictive maintenance, etc., as a training device for workers to learn maintenance techniques for factory equipment. The types of abnormalities that the test device can reproduce will be described later.

[0082] Prior to determining whether the test device was in a steady state or a non-steady state using the detection device, the autoencoder was trained using the training device. During training on the training device, the normal environment of the test device was used as a matter of course. The sound generated by the normal test device was used as stationary sound for training on the training device. For the stationary sound to be sampled, the sampling rate was 44100Hz, the FFT frame length was 4096 sample data, the FFT frame interval was 2048 sample data, the sound feature data was a Mel frequency spectrogram, and the number of Mel band filters for the Mel frequency spectrogram was 8. The FFT frame length refers to a parameter that controls the frequency resolution by the number of reference data to be processed when performing Fast Fourier Transform processing, and the FFT frame interval refers to the number of sample data that are spaced apart in time when performing the same processing, and is a parameter that controls the time resolution. Steady-state sounds were recorded for three rotation speed patterns of the rotary drive part: low speed (100 revolutions / min), medium speed (500 revolutions / min), and high speed (1000 revolutions / min), for 30 minutes x 3 times each. By training using this steady-state sound data, three types of trained autoencoders targeting equipment with different rotation speeds were obtained.

[0083] Next, the data for generating the trained autoencoder was transferred to a detection device to complete the detection device, and the detection device was used to determine whether the test device was in a steady state or a non-steady state. The threshold value used in the detection device was determined by executing the threshold determination mode in the learning device described in the embodiment. The threshold value used in the test example was the reference threshold value (Loss max )There is only one type. However, the reference threshold (Loss max ) was determined as the third quartile of the total losses obtained when the steady sound data used was input into the learning device and inference was performed, that is, the value of the value in the order of 3 / 4 from the smallest when the total losses were arranged from the smallest to the largest, multiplied by 1.5. Furthermore, the state determination unit 126X of the detection device determines whether the object is in a steady state or an unsteady state based on whether the value of the loss exceeding the reference threshold is more or less than a predetermined number. That is, in the detection device of the test example, when the number of losses exceeding the reference threshold is more than a predetermined number, the object is determined to be in an unsteady state. Here, the "predetermined number" beyond which the number of losses exceeding the reference threshold is judged to be in an unsteady state is determined as a number equivalent to a predetermined ratio (for example, but not limited to, 10%) of the number of excess samples to the number of data of the input to be diagnosed. In other words, when there are Z pieces of loss value data, if the number of data of loss values ​​exceeding the reference threshold is 0.1Z or more, the state judgment unit 126X judges that the object is in an unsteady state.

[0084] As described above, the test device used in the judgment by the detection device can reproduce a normal state, i.e., a steady state, and an abnormal state, i.e., an unsteady state, including the sounds emitted. The following five types of unsteady test equipment conditions were prepared: 1. Gear malfunction The gear included in the rotating mechanism had a defect. 2. Products with abnormal bearings The bearing included in the rotating mechanism had a defect. 3. Rotation mechanism angle is abnormal The angle of the rotation axis of a rotating mechanism is shifted from the original rotation axis. 4. Gear contact abnormality The distance between meshing gears is slightly increased from the normal distance. 5. Those with eccentricity abnormality The center of gravity of the rotating parts included in the rotation mechanism is shifted from the rotation axis, causing an unbalanced load.

[0085] First, the sound generated by the normal test equipment was sampled as a measurement sound to determine whether the state of the test equipment was in a steady state or an unsteady state. The sampling conditions were the same as those for sampling the steady sound during learning. The measurement sound was sampled for 10 minutes three times for each of the rotation speeds of the rotary drive unit: low (100 rpm), medium (500 rpm), and high (1000 rpm). The measurement sound feature data was a Mel frequency spectrogram, and the number of Mel band filters in the Mel frequency spectrogram was eight. As a result, based on the measurement sounds generated by a normal test equipment, the detection device determined that the test equipment was in a steady state all three times the measurement sounds were sampled, regardless of whether the rotation speed of the test equipment was low (100 revolutions / min), medium (500 revolutions / min), or high (1,000 revolutions / min). As is the case in the following examples, the trained autoencoder used by the detection device when determining whether the test device is in a steady state or unsteady state at low speed is a trained autoencoder that has been trained using the sounds emitted by the test device at low speed, and similarly, the trained autoencoder used by the detection device when determining whether the test device is in a steady state or unsteady state at medium and high speed is a trained autoencoder that has been trained using the sounds emitted by the test device at medium and high speed, respectively.

[0086] Next, the sound generated by the test equipment with any of the above-mentioned abnormalities from 1 to 5 was sampled as the measurement sound, and it was determined whether the state of the test equipment was in a steady state or an unsteady state. The sampling conditions were the same as those for sampling the steady sound during learning. For each of the cases where the rotation speed of the rotary drive unit was low (100 rpm), medium (500 rpm), and high (1000 rpm), sampling of the measurement sound was performed for 10 minutes three times for the test equipment with abnormality 1 above, through the test equipment with abnormality 5. The measured sound feature data was a Mel frequency spectrogram, and the number of Mel band filters in the Mel frequency spectrogram was eight. As a result, based on the measurement sounds generated by the test equipment having abnormality 1, the detection device determined that the test equipment was in an unsteady state all three times that the measurement sounds were sampled, regardless of whether the rotation speed of the rotary drive unit was low (100 rpm), medium (500 rpm), or high (1,000 rpm). Similarly, based on the measured sounds produced by the test equipment having abnormalities 2, 3, and 4, the detection device determined that the test equipment was in an unsteady state all three times the measured sounds were sampled, regardless of whether the rotation speed of the test equipment was low (100 rpm), medium (500 rpm), or high (1,000 rpm). Based on the measurement sounds generated by the test equipment with abnormality number 5, the detection device determined that the test equipment was in an unsteady state all three times that the measurement sounds were sampled, both when the rotation speed of the rotary drive unit was medium (500 rpm) and when it was high (1,000 rpm). On the other hand, when the rotation speed of the rotary drive unit was low (100 rpm), based on the measurement sound generated by the test equipment having abnormality number 5, the detection device determined that the test equipment was in a steady state each of the three times that the measurement sound was sampled, and was unable to detect that the test equipment was in a non-steady state. [Explanation of symbols]

[0087] 100 Computer equipment 101 Display 102 Input Device 103 Microphone 100X Computer Equipment 101X Display 102X Input Device 103X Microphone 121 Input section 122 Main control unit 123 Feature detection unit 124 Autoencoder 125 Loss generation section 126 Optimization Department 127 Threshold Determination Unit 128 Output section 121X Input section 122X Main control unit 123X Feature detection unit 124X 1st calculation section 125X loss generator 126X Status Judgment Unit 127X First Recording Section 128X output section

Claims

1. An autoencoder that encodes input data, which is predetermined data, and then decodes the encoded data to return it to data of the same dimension as the input data. the input data is steady-state sound feature data, which is data on the feature of steady-state sound identified by steady-state sound data generated from steady-state sound data, which is data for a specific time length about steady-state sound including sounds generated in a steady state from an object, including a rotating object, whose unsteadiness is to be detected based on the sound; and the output data is estimated steady-state sound feature data for the specific time length, a plurality of pieces of steady sound feature data are input, and a loss is calculated using a loss function for the steady sound feature data that is input data and the estimated steady sound feature data that is output data for the input data, and the loss is obtained by performing pre-learning so as to minimize the loss; A trained autoencoder, The loss function is determined based on noise-removed data, which is an output when a plurality of pieces of difference data, which are data based on the difference between the steady sound feature data and the estimated steady sound feature data, are input into a noise removal function, which is a function for removing noise from the difference data, and on trend data, which is an output when the difference data is input into a trend extraction function, which is a function for extracting a trend of the difference data in a predetermined time period. A trained autoencoder.

2. the steady-state sound feature data is a Mel-frequency spectrogram generated from the steady-state sound data; The trained autoencoder of claim 1 .

3. The noise removal function is A function that functions as a low-pass filter, a function that calculates a moving average, or a function that performs a Fourier transform, removes frequency band signals that exceed a predetermined threshold, and then performs an inverse Fourier transform to return the signal. The trained autoencoder of claim 1 .

4. The trend extraction function is a function for calculating the median of the differential data and a plurality of data before, after, or both before and after the differential data; The trained autoencoder of claim 1 .

5. The trend extraction function is is a function that takes a difference series of the difference data, The trained autoencoder of claim 1 .

6. the trend extraction function is a function for obtaining the difference data and a median of a plurality of data before, after, or before and after the difference data, The loss function calculates the loss as a difference between the noise-removed data and the trend data. A trained autoencoder according to any one of claims 1 to 3.

7. the trend extraction function is a function that takes a difference series of the difference data, The loss function is one that calculates the loss by applying the noise removal function to the trend data. A trained autoencoder according to any one of claims 1 to 3.

8. A method for converting an autoencoder that encodes input data, which is predetermined data, and then decodes the encoded data to data of the same dimension as the input data into a trained autoencoder, comprising: the input data is steady-state sound feature data, which is data on the feature of steady-state sound identified by steady-state sound data generated from steady-state sound data, which is data for a specific time length about steady-state sound that is generated in a steady state from an object whose non-stationarity is to be detected based on the sound; and the output data is estimated steady-state sound feature data for the specific time length, a step of inputting a plurality of pieces of steady sound feature data, calculating a loss between the steady sound feature data as input data and the estimated steady sound feature data as output data for the input data using a loss function, and performing pre-learning so as to minimize the loss, the loss function is one that calculates the loss based on: noise-removed data, which is an output when a plurality of difference data, which is data based on the difference between the steady sound feature data and the estimated steady sound feature data, is input to a noise removal function that is a function for removing noise from the difference data, and is obtained after generating the difference data; and trend data, which is an output when the difference data is input to a trend extraction function that is a function for extracting a trend of the difference data in a predetermined time period. method.

9. a first recording unit that records the trained autoencoder according to any one of claims 1 to 7; a receiving unit that receives measurement sound data, which is data for a specific time period about a measurement sound including a sound generated from the object for which sound-based non-stationarity detection is being performed; a measurement sound feature data generation unit that generates measurement sound feature data, which is data on the feature of the measurement sound identified by the measurement sound data, from the measurement sound data received by the reception unit, in the same manner as the steady sound feature data was generated from the steady sound data in pre-learning; a first calculation unit that reads the trained autoencoder recorded in the first recording unit, inputs the measurement sound feature data generated by the measurement sound feature data generation unit into the trained autoencoder, and obtains estimated measurement sound feature data for the specific time length that is an output from the trained autoencoder for the measurement sound feature data; a second calculation unit that calculates the loss for the difference between the measurement sound feature data generated by the measurement sound feature data generation unit and the estimated measurement sound feature data created from the measurement sound feature data generated by the first calculation unit, using the same loss function as used when training the trained autoencoder, and that, if the generated data for the loss satisfies a predetermined condition, determines that the measurement sound identified by the measurement sound data that is the basis of the measurement sound feature data is a non-stationary sound and generates result data indicating that a non-stationary sound has occurred; A non-stationary sound detection device comprising:

10. the second calculation unit is configured to determine that the measurement sound identified by the measurement sound data on which the measurement sound feature data is based is a non-stationary sound when the data on the loss includes a predetermined number or more of values ​​exceeding a predetermined threshold. The non-stationary sound detection device according to claim 9.

11. A predetermined display device is provided, the second calculation unit generates, from the result data, image data for displaying on the display device an image of a graph indicating that the test sound identified by the test sound data on which the test sound feature data is based is a non-stationary sound. The non-stationary sound detection device according to claim 9.

12. the measurement sound feature data is a Mel-frequency spectrogram generated from the measurement sound data; The non-stationary sound detection device according to claim 9.

13. A non-stationary sound detection method executed by a computer having a first recording unit that records the trained autoencoder according to any one of claims 1 to 7, wherein the method is executed by a computer. a first step of receiving measurement sound data, the measurement sound data being data for a specific time length about the measurement sound including the sound generated from the object for which sound-based non-stationarity detection is being performed; a second step of generating, from the measurement sound data received in the first step, measurement sound feature data, which is data on the feature of the measurement sound identified by the measurement sound data, in the same manner as the steady sound feature data was generated from the steady sound data in pre-learning; a third step of reading the trained autoencoder recorded in the first recording unit, inputting the measurement sound feature data generated in the second step into the trained autoencoder, and obtaining estimated measurement sound feature data for the predetermined time length that is an output from the trained autoencoder for the measurement sound feature data; a fourth step of calculating the loss for the difference between the measurement sound feature data generated in the second step and the estimated measurement sound feature data created from the measurement sound feature data generated in the third step using the same loss function as used when training the trained autoencoder, and, if the generated data for the loss satisfies a predetermined condition, determining that the measurement sound identified by the measurement sound data on which the measurement sound feature data is based is a non-stationary sound and generating result data indicating that a non-stationary sound has occurred; A method for detecting non-stationary sounds, including:

14. A computer program for causing a predetermined computer to function as a non-stationary sound detection device, the computer comprising: a first recording unit that records the trained autoencoder according to any one of claims 1 to 7; a receiving unit that receives measurement sound data, which is data for a specific time period about a measurement sound including a sound generated from the object for which sound-based non-stationarity detection is being performed; a measurement sound feature data generation unit that generates measurement sound feature data, which is data on the feature of the measurement sound identified by the measurement sound data, from the measurement sound data received by the reception unit, in the same manner as the steady sound feature data was generated from the steady sound data in pre-learning; a first calculation unit that reads the trained autoencoder recorded in the first recording unit, inputs the measurement sound feature data generated by the measurement sound feature data generation unit into the trained autoencoder, and obtains estimated measurement sound feature data that is an output from the trained autoencoder for the measurement sound feature data; a second calculation unit that calculates the loss for the difference between the measurement sound feature data generated by the measurement sound feature data generation unit and the estimated measurement sound feature data created from the measurement sound feature data generated by the first calculation unit, using the same loss function as used when training the trained autoencoder, and that, when the generated data for the loss satisfies a predetermined condition, determines that the measurement sound identified by the measurement sound data that is the basis of the measurement sound feature data is a non-stationary sound, and generates result data indicating that a non-stationary sound has occurred; A computer program that makes something function.

Images