Abnormal sound determination system, abnormal sound determination device, and program product

Abstract

It includes: an action sound data acquisition unit for acquiring action sound data of a product; an ambient sound data acquisition unit for acquiring ambient sound data; a removal unit for correcting the ambient sound data from the action sound data; and a determination unit for determining the action sound of the product based on the action sound data with the corrected ambient sound data.

Description

Technical Field

[0001] This disclosure relates to an abnormal sound detection system, an abnormal sound detection device, and a program product. Background Technology

[0002] A previously known information processing apparatus includes: an abnormal sound detection unit that determines whether the acquired sound is an abnormal sound based on a sound signal generated from the acquired sound; an abandonment unit that determines whether the acquired sound is an abandonable sound and, based on the determination result, determines whether to abandon the abnormal sound detected by the abnormal sound detection unit; and an abnormality determination unit that determines that an abnormality has occurred when the abnormal sound detection unit determines that the acquired sound is an abnormal sound and the abandonment unit determines that the abnormal sound should not be abandoned (for example, see Patent Document 1).

[0003] In addition, the following anomaly detection device is known in the past: using time-varying waveform data obtained from the inherent motion of an object, performing learning based on an autoencoder, performing a persistent homology transformation on the waveform data to calculate the change in the number of connected components corresponding to the change in the threshold in the value direction, inputting the output of the autoencoder based on the waveform data and the output of the persistent homology transformation, and judging the anomaly based on the discrimination result of the learner that has performed machine learning on the waveform data and the anomaly (see, for example, Patent Document 2).

[0004] [Existing Technical Documents]

[0005] [Patent Documents]

[0006] [Patent Document 1] International Publication No. 2020 / 084680

[0007] [Patent Document 2] Japanese Patent Application Publication No. 2020-36633 Summary of the Invention

[0008] [The problem this invention aims to solve]

[0009] For example, during abnormal sound checks of product movement sounds in production lines, the results of judging product movement sounds can sometimes be biased due to the influence of ambient sounds, such as sudden or periodic sounds.

[0010] The purpose of this disclosure is to provide an abnormal sound determination system, an abnormal sound determination device, and a program product that can suppress deviations in the determination results when determining the sound of a product's operation.

[0011] [Methods for solving the problem]

[0012] The abnormal sound determination system disclosed herein includes: a motion sound data acquisition unit that acquires motion sound data of a product; an ambient sound data acquisition unit that acquires ambient sound data; a correction unit that corrects the ambient sound data based on the motion sound data; and a determination unit that determines the motion sound of the product based on the motion sound data with the corrected ambient sound data.

[0013] According to this disclosure, when determining the motion sound of a product, it is possible to correct the surrounding sound data based on the motion sound data of the product, and use the motion sound data with corrected surrounding sound data to determine the motion sound of the product, thereby suppressing the deviation of the determination result caused by the influence of surrounding sound data.

[0014] Its characteristic is that the motion sound data acquisition unit picks up the motion sound of the product set in the soundproof box through a microphone.

[0015] Its feature is that the ambient sound data acquisition unit picks up ambient sounds generated outside the soundproof box through a microphone.

[0016] Its characteristic is that the determination unit uses multiple machine learning models.

[0017] Its characteristic is that the correction unit corrects the motion sound data based on the surrounding sound data.

[0018] According to this disclosure, motion sound data can be modified based on ambient sound data to remove ambient sound data from the motion sound data of the product.

[0019] Its characteristic is that the determination unit uses a determination algorithm composed of a first machine learning model and a second machine learning model. The first machine learning model learns the normal sound data of the product and predicts the deviation value of the product's action sound relative to the normal sound data of the product. The second machine learning model learns the degree of deviation of the deviation value of the normal sound data of the product predicted by the first machine learning model. If the degree of deviation of the deviation value of the product's action sound is greater than a threshold, the action sound of the product is predicted to be abnormal.

[0020] According to this disclosure, an algorithm comprising a first machine learning model and a second machine learning model can automate the determination of abnormal sounds. The first machine learning model predicts the deviation of the product's movement sound from normal sound data, and the second machine learning model predicts the abnormality of the product's movement sound based on the degree of deviation. Furthermore, according to this disclosure, by utilizing the algorithm, the determination criteria for abnormal sound identification can be standardized.

[0021] Its characteristic may be that the first machine learning model uses an autoencoder.

[0022] According to this disclosure, by using an autoencoder learned from normal sound data in the first machine learning model, it is possible to predict the deviation of the product's motion sound from the normal sound data.

[0023] Its characteristic is that the second machine learning model uses the Local Outlier Factor (LOF) method.

[0024] According to this disclosure, by using the local outlier factor method in the second machine learning model, the degree of deviation of the deviation value of the normal sound data of the product predicted by the first machine learning model is learned. When the degree of deviation of the deviation value of the product's motion sound is greater than the threshold, the product's motion sound can be judged as abnormal.

[0025] The abnormal sound determination device disclosed herein comprises: a motion sound data acquisition unit that acquires motion sound data of a product; an ambient sound data acquisition unit that acquires ambient sound data; a correction unit that corrects the ambient sound data based on the motion sound data; and a determination unit that determines the motion sound of the product based on the motion sound data with the corrected ambient sound data.

[0026] According to this disclosure, when determining the motion sound of a product, it is possible to correct the surrounding sound data based on the motion sound data of the product, and use the motion sound data with corrected surrounding sound data to determine the motion sound of the product, thereby suppressing the deviation of the determination result caused by the influence of surrounding sound data.

[0027] The program product disclosed herein enables a computer to perform the following steps: an action sound data acquisition step, which acquires action sound data of the product; an ambient sound data acquisition step, which acquires ambient sound data; a correction step, which corrects the ambient sound data based on the action sound data; and a determination step, which determines the action sound of the product based on the action sound data with the corrected ambient sound data.

[0028] According to this disclosure, when determining the motion sound of a product, it is possible to correct the surrounding sound data based on the motion sound data of the product, and use the motion sound data with corrected surrounding sound data to determine the motion sound of the product, thereby suppressing the deviation of the determination result caused by the influence of surrounding sound data. Attached Figure Description

[0029] Figure 1 This is a structural diagram of an example of an abnormal sound determination system according to this embodiment.

[0030] Figure 2 This is a hardware structure diagram of an example of a computer according to this embodiment.

[0031] Figure 3 This is a functional block diagram of an example of an abnormal sound determination device according to this embodiment.

[0032] Figure 4A This is a schematic diagram illustrating an example of the ambient sound removal process.

[0033] Figure 4B This is a schematic diagram illustrating an example of the ambient sound removal process.

[0034] Figure 4C This is a schematic diagram illustrating an example of the ambient sound removal process.

[0035] Figure 5 This is a schematic diagram of an example of abnormal sound detection processing performed by the detection algorithm department.

[0036] Figure 6 This is a flowchart of an example of machine learning processing in the abnormal sound determination device according to this embodiment.

[0037] Figure 7 This is a schematic diagram illustrating an example of machine learning processing in the abnormal sound determination device according to this embodiment.

[0038] Figure 8 This is a flowchart of an example of abnormal sound determination processing in the abnormal sound determination device according to this embodiment.

[0039] Figure 9 This is a schematic diagram of an example of abnormal sound determination processing in the abnormal sound determination device according to this embodiment.

[0040] Figure 10 This is a schematic diagram illustrating an example of abnormal sound determination processing in the abnormal sound determination device according to this embodiment. Detailed Implementation

[0041] The embodiments of the present invention will now be described in detail.

[0042] [First Implementation]

[0043] <System Structure>

[0044] Figure 1This is a structural diagram of an example of the abnormal sound detection system according to this embodiment. The abnormal sound detection system 1 includes an abnormal sound detection device 10, a microphone 14A that acquires (picks up) motion sound data including the motion sound of a product 16 placed inside the inspection device 12, and a microphone 14B that acquires (picks up) ambient sound data including sounds outside the inspection device 12 (ambient sounds). The motion sound of the product 16 is a sound emitted from the product 16. The motion sound is a sound emitted from the motor, compressor, indoor unit, or outdoor unit of an air conditioner, etc. Ambient sounds are sounds generated outside the inspection device 12. Ambient sounds refer to sounds emitted by equipment or forklifts operating on the production line.

[0045] The inspection device 12 is a box in which the product 16, which generates movement sounds, is placed. Microphone 14A can measure movement sound data including the movement sounds of the product 16 placed in the inspection device 12. The movement sound data acquired by microphone 14A sometimes includes ambient sounds intruding into the inspection device 12. For example, by placing a normal product 16 in the inspection device 12, microphone 14A can measure movement sound data including the movement sounds of the normal product. Hereinafter, the movement sound data of the normal product 16 is sometimes referred to as normal sound data. Alternatively, for example, by placing the product 16, which is a target for abnormal sound determination, in the inspection device 12, microphone 14A can measure movement sound data including the movement sounds of the product 16, which is a target for abnormal sound determination. Microphone 14B acquires ambient sound data including ambient sounds generated outside the inspection device 12. The ambient sound data acquired by microphone 14B does not include the movement sounds of the product 16 placed in the inspection device 12. Furthermore, when measuring the movement sound data of the normal product 16, by using a soundproof box (soundproof box) with high sound insulation performance in the inspection device 12, normal sound data without the influence of ambient sound data can also be measured.

[0046] The abnormal sound detection device 10 receives motion sound data of the product 16 obtained by microphone 14A and ambient sound data obtained by microphone 14B. The abnormal sound detection device 10 learns the normal sound data of the product 16 using a learning algorithm described later, and creates a machine learning model (an example of a first machine learning model) that predicts the deviation value of the motion sound data of the product 16, the object of abnormal sound detection, relative to the normal sound data. Furthermore, the abnormal sound detection device 10 learns the degree of deviation of the deviation value predicted by the learning algorithm based on the normal sound data of the product 16, and generates a machine learning model (an example of a second machine learning model) that predicts whether a sound is abnormal based on the degree of deviation of the deviation value predicted from the motion sound data of the product 16, the object of abnormal sound detection. Hereinafter, the machine learning model that learns the normal sound data of the product 16 and predicts the deviation value of the motion sound data of the product 16, the object of abnormal sound detection, relative to the normal sound data, is sometimes referred to as a "machine learning model that predicts the deviation value relative to normal data." In addition, a machine learning model that learns the degree of deviation of the deviation value predicted based on the normal sound data of product 16 and the degree of deviation of the deviation value predicted based on the movement sound data of product 16, the object of abnormal sound judgment, is sometimes called a "machine learning model for predicting the degree of deviation".

[0047] In the abnormal sound detection device 10, the detection algorithm described later utilizes a machine learning model that predicts the deviation value relative to normal data and a machine learning model that predicts the degree of deviation to determine the abnormal sound of the product 16, which is the object of the abnormal sound detection. In addition, the abnormal sound detection device 10 has an ambient sound correction function, which corrects the ambient sound data based on the motion sound data of the product 16 obtained by the microphone 14A (correcting the motion sound data according to the ambient sound data to reduce the influence of the ambient sound data).

[0048] Furthermore, the name of the abnormal sound detection device 10 is just one example; it could be any other name. The communication connection between the abnormal sound detection device 10 and microphones 14A and 14B can be wired or wireless. Additionally, the abnormal sound detection device 10 is an information processing terminal such as a PC, smartphone, or tablet.

[0049] Figure 1 The structure of the abnormal sound detection system 1 is an example; for instance, the abnormal sound detection device 10 can also be implemented by one or more information processing terminals (computers). For example, the abnormal sound detection device 10 can also be a structure that separates the computer that performs the learning algorithm processing from the computer that performs the detection algorithm processing. Thus, Figure 1 The structure of the abnormal sound detection system 1 is an example, and of course there are various system structure examples depending on the purpose and use.

[0050] <Hardware Structure>

[0051] Figure 1 The abnormal sound detection device 10, for example, through Figure 2 The hardware structure shown is implemented by the computer 500.

[0052] Figure 2 This is a hardware structure diagram of an example of the computer described in this embodiment. Figure 2 The computer 500 shown includes an input device 501, a display device 502, an external I / F 503, RAM 504, ROM 505, CPU 506, a communication I / F 507, and an HDD 508. Additionally, the input device 501 and the display device 502 can also be connected and used as needed.

[0053] Input device 501 includes touch panels, operation keys or buttons, keyboards or mice, etc., used by the operator to input various signals. Display device 502 consists of a liquid crystal or organic EL display for showing images and speakers for outputting sound data such as sound or music. Communication I / F 507 is the interface for computer 500 to conduct data communication via a network.

[0054] HDD508 is an example of a non-volatile storage device for storing programs and data. The stored programs and data include the operating system (OS), which is the basic software controlling the entire computer 500, and applications that provide various functions on the OS. Alternatively, the computer 500 can utilize a drive device that uses flash memory as the storage medium (e.g., a solid-state drive: SSD, etc.) instead of HDD508.

[0055] External I / F 503 is an interface to external devices. External devices include recording media 503a, etc. Thus, computer 500 can read from and / or write to recording media 503a via external I / F 503. Recording media 503a may include floppy disks, CDs, DVDs, SD memory cards, USB storage devices, etc.

[0056] ROM 505 is an example of a non-volatile semiconductor memory (storage device) that can retain programs or data even when the power is cut off. ROM 505 stores programs and data such as BIOS, OS settings, and network settings executed when the computer 500 starts. RAM 504 is an example of a volatile semiconductor memory (storage device) that temporarily stores programs and data.

[0057] CPU 506 is a computing device that reads programs or data from storage devices such as ROM 505 or HDD 508 into RAM 504 and performs processing to realize the overall control or function of computer 500. The abnormal sound detection device 10 according to this embodiment can implement various functional blocks described later.

[0058] <Software Structure>

[0059] Function Blocks

[0060] The functional blocks of the abnormal sound determination device 10 of the abnormal sound determination system 1 according to this embodiment will be described. Figure 3 This is a functional block diagram of an example of the abnormal sound determination device of this embodiment. The abnormal sound determination device 10 implements the action sound data acquisition unit 50, the ambient sound data acquisition unit 52, the ambient sound removal unit 54, the abnormal sound determination unit 56, and the display control unit 58 by executing a program.

[0061] The motion sound data acquisition unit 50 receives motion sound data of the normal product 16 or the product 16 subject to abnormal sound determination, measured by microphone 14A. The ambient sound data acquisition unit 52 receives ambient sound data measured by microphone 14B.

[0062] The motion sound data acquisition unit 50 sends the acquired motion sound data of the product 16 to the ambient sound removal unit 54 and the abnormal sound determination unit 56. Additionally, the ambient sound data acquisition unit 52 sends the acquired ambient sound data to the ambient sound removal unit 54 and the abnormal sound determination unit 56.

[0063] The ambient sound removal unit 54 uses the motion sound data of the product 16 received from the motion sound data acquisition unit 50 and the ambient sound data received from the ambient sound data acquisition unit 52 to perform, for example... Figures 4A to 4C The corrections shown include removing the influence of ambient sound data from the motion sound data to reduce the impact of ambient sound data. Furthermore, although the following description states "removing ambient sound data from the motion sound data," it is not limited to completely removing the influence of ambient sound data; some influence may remain.

[0064] Figures 4A-4C This is a schematic diagram illustrating an example of the ambient sound removal process. Figure 4A The waveform representing the time-varying intensity of the sound data during the action of product 16. Additionally, Figure 4B A waveform representing the time-varying intensity of ambient sound data. For example... Figure 4C As shown, by using the basis Figure 4B Ambient sound data Figure 4A The abnormal parts of the motion sound data are corrected by reducing the intensity, thereby removing surrounding sound data from the product's motion sound data. Figure 4C The waveform shown is after data correction. Figure 4AThe waveform after correcting abnormal parts of the motion sound data. The ambient sound removal unit 54 sends the motion sound data with the influence of ambient sound data removed to the abnormal sound determination unit 56.

[0065] The abnormal sound determination unit 56 performs machine learning processing and abnormal sound determination processing, which will be described later. The abnormal sound determination unit 56 includes a learning algorithm unit 60 and a determination algorithm unit 62. The learning algorithm unit 60 learns normal sound data of the product 16 without the influence of ambient sound data, and generates a machine learning model (AI) that predicts the deviation value relative to the normal data, and a machine learning model (AI) that predicts the degree of deviation.

[0066] The judgment algorithm unit 62 uses a machine learning model created by the learning algorithm unit 60 to predict deviation values ​​relative to normal data, and a machine learning model to predict the degree of deviation, to determine whether the movement sound of the product 16, the object of the abnormal sound judgment, is an abnormal sound. For example, Figure 5 As shown, the abnormal sound determination process executed by the determination algorithm unit 62 is performed. Figure 5 This is a schematic diagram of an example of abnormal sound detection processing performed by the algorithm department.

[0067] like Figure 5 As shown, an autoencoder can be used in the machine learning model to predict deviations from normal data. The machine learning model using the autoencoder has learned a compression method capable of successfully reconstructing the normal sound data of product 16 through the learning algorithm 60. The machine learning model using the autoencoder is input with the motion sound data of product 16, the object of abnormal sound detection, and then reconstructed by compressing it using the learned compression method.

[0068] The judgment algorithm unit 62 uses the difference (deviation value) between the motion sound data input into the machine learning model using an autoencoder and the motion sound data restored by the machine learning model using an autoencoder to predict the deviation value of the motion sound data of the product 16, which is an abnormal sound judgment object relative to normal sound data.

[0069] Additionally, a machine learning model for predicting the degree of deviation can use the Local Outlier Factor (LOF) method. A machine learning model using LOF has already learned the characteristic distribution of deviation values ​​in normal sound data. As described later, the machine learning model using LOF can predict whether the movement sound of product 16, the object of the abnormal sound assessment, is normal or abnormal based on the relationship between the characteristic distribution of deviation values ​​in normal sound data and the characteristic distribution of deviation values ​​in the movement sound data of product 16, the object of the abnormal sound assessment.

[0070] For example, such as Figure 5As shown, the determination algorithm unit 62 compares the plot of the feature representing the deviation value of the motion sound data of product 16, which is the object of abnormal sound determination, with the distribution of the plot of the feature representing the deviation value of normal sound data on the graph representing the deviation value of the motion sound data "Feature 1" and "Feature 2". If the distribution of the plot of the feature representing the deviation value of normal sound data includes the plot of the feature representing the deviation value of the motion sound data of product 16, which is the object of abnormal sound determination, then the determination algorithm unit 62 predicts that the motion sound of product 16, which is the object of abnormal sound determination, is normal. Otherwise, if the distribution of the plot of the feature representing the deviation value of normal sound data does not include the plot of the feature representing the deviation value of the motion sound data of product 16, which is the object of abnormal sound determination, then the determination algorithm unit 62 predicts that the motion sound of product 16, which is the object of abnormal sound determination, is abnormal. The details of the processing of the learning algorithm unit 60 and the determination algorithm unit 62 will be described later.

[0071] return Figure 3 The display control unit 58 displays information that needs to be prompted to the operator on the display device 502. For example, the display control unit 58 displays on the display device 502 the judgment result of whether the operation sound of the product 16, which is the object of abnormal sound judgment, is normal, as predicted by the abnormal sound judgment unit 56. The display control unit 58 is not limited to the display on the display device 502, but may also prompt information to the operator through the output of a buzzer or alarm, the lighting of a lamp or light fixture, etc.

[0072] in addition, Figure 3 The functional block diagram appropriately omits functions that are not required in the description of the abnormal sound determination system 1 according to this embodiment.

[0073] <Processing>

[0074] Figure 6 This is a flowchart of an example of machine learning processing in the abnormal sound determination device according to this embodiment. Figure 7 This is a schematic diagram illustrating an example of machine learning processing in the abnormal sound detection device of this embodiment.

[0075] In step S10, the abnormal sound detection device 10 acquires normal sound data of the product 16 from the microphone 14A. In step S12, the abnormal sound detection device 10 performs a short-time Fourier transform (STFT) on the normal sound data to obtain, for example... Figure 7 The frequency domain characteristics of the normal sound data shown are illustrated. The abnormal sound determination device 10 can also perform preprocessing (normalization, standardization, regularization) as needed before performing the short-time Fourier transform.

[0076] In step S14, the abnormal sound determination device 10 teaches the machine learning model using an autoencoder to learn a compression method that can effectively reproduce the normal sound data of product 16. After the learning is completed, the process proceeds from step S16 to step S18, where the abnormal sound determination device 10 inputs the frequency domain characteristics of the normal sound data into the machine learning model using the autoencoder.

[0077] A machine learning model trained using an autoencoder is used to restore the normal sound data by compressing its frequency domain features (hereinafter referred to as normal sound input data) using a learned compression method. The output of the machine learning model trained using an autoencoder is, for example... Figure 7 The frequency domain characteristics of the restored normal sound data as shown (hereinafter referred to as normal sound restoration data).

[0078] In step S20, the abnormal sound determination device 10 obtains the relationship between the normal sound input data and the normal sound restoration data, such as... Figure 7 The differential data shown. In step S22, the abnormal sound determination device 10, for example, Figure 7 As shown, the difference data are averaged in both the time and frequency directions to calculate the average value in the time direction and the average value in the frequency direction.

[0079] In steps S24 to S30, the abnormal sound detection device 10 processes the calculated average value in the time direction. Additionally, in steps S32 to S38, the abnormal sound detection device 10 processes the calculated average value in the frequency direction.

[0080] In step S24, the abnormal sound detection device 10 teaches the machine learning model using LOF to learn the calculated average value in the time direction. After learning, the process proceeds from step S26 to step S28, where the abnormal sound detection device 10 inputs the average value in the time direction into the machine learning model using LOF and outputs a score (abnormality). In step S30, the abnormal sound detection device 10 obtains the distribution of the scores and determines the score value indicating the 3σ position as the threshold for anomaly detection (threshold 1).

[0081] Additionally, in step S32, the abnormal sound detection device 10 uses the calculated average value in the frequency direction to train the machine learning model using LOF. After training, the process proceeds from step S34 to step S36, where the abnormal sound detection device 10 inputs the average value in the frequency direction to the trained machine learning model using LOF and outputs a score (abnormality). In step S38, the abnormal sound detection device 10 obtains the score distribution and determines the score value indicating the 3σ position as the threshold for abnormality detection (threshold 2).

[0082] Abnormal sound detection device 10 in use Figure 6 as well as Figure 7 After machine learning processing, Figures 8-10 The abnormal sound detection and processing shown. Figure 8 This is a flowchart of an example of abnormal sound determination processing in the abnormal sound determination device of this embodiment. Figure 9 as well as Figure 10 This is a schematic diagram of an example of abnormal sound determination processing in the abnormal sound determination device of this embodiment.

[0083] In step S50, the abnormal sound determination device 10 acquires the motion sound data of the product 16, which is the object of abnormal sound determination, from the microphone 14A. In step S52, the abnormal sound determination device 10 performs a short-time Fourier transform on the motion sound data of the product 16, which is the object of abnormal sound determination, to obtain the characteristic quantities of the frequency domain performance of the motion sound data of the product 16.

[0084] In step S54, the abnormal sound determination device 10 inputs the frequency domain characteristic of the motion sound data of the product 16, the object of the abnormal sound determination, into the device that uses... Figure 6 and Figure 7 The machine learning model of the autoencoder is described in the machine learning process.

[0085] A machine learning model that has been trained using an autoencoder is used to compress the frequency domain features of the motion sound data of the product 16, the input abnormal sound detection object (hereinafter referred to as motion sound input data), using a learned compression method, and then restores the motion sound data. Additionally, the machine learning model that has been trained using an autoencoder outputs the frequency domain features of the restored motion sound data (hereinafter referred to as motion sound restored data).

[0086] In step S56, the abnormal sound detection device 10 acquires the difference data between the motion sound input data and the motion sound reconstruction data. In step S58, the abnormal sound detection device 10 averages the difference data in both the time and frequency directions, calculating the average value in the time direction and the average value in the frequency direction. The average value in the time direction is used to detect abnormal sounds that may occur intermittently. Furthermore, the average value in the frequency direction is used to detect abnormal sounds that may occur continuously.

[0087] In step S60, the abnormal sound detection device 10 inputs the average value in the time direction to the machine learning model that has completed learning using LOF and outputs a score (score1). In step S62, the abnormal sound detection device 10 compares the score (score1) output in step S60 with the anomaly detection threshold (threshold 1) determined in the machine learning process.

[0088] If the score (score1) output in step S60 is greater than the anomaly detection threshold (threshold 1) determined in the machine learning process, the abnormal sound detection device 10 performs steps S64 to S68 to determine the motion sound after removing the influence of ambient sound. In step S64, the abnormal sound detection device 10 confirms the ambient sound data obtained from the microphone 14B. The abnormal sound detection device 10 obtains, for example, a short-time Fourier transform of the ambient sound data. Figure 9 The frequency domain characteristics of the surrounding sound data are shown. Furthermore, as... Figure 9 As shown, the abnormal sound determination device 10 calculates the average value in the time direction through preprocessing.

[0089] In step S66, the abnormal sound determination device 10 compares the average value of the time direction calculated in step S58 with the average value of the time direction of the surrounding sound data calculated in step S64. For example... Figure 9 As shown, the abnormal sound detection device 10 corrects the average value of the time direction calculated in step S58 to reduce the uniform portion of peaks exceeding the threshold. Through the processing in step S66, if the abnormal sound detection device 10 has a sudden strong sound generated in the surroundings (such as a forklift horn or a knocking sound), it can eliminate the influence of the sound. In step S68, the abnormal sound detection device 10 inputs the average value of the time direction corrected in step S66 into the machine learning model that has completed learning using LOF and outputs a score (score1). In step S70, the abnormal sound detection device 10 compares the score (score1) output in step S68 with the anomaly detection threshold (threshold 1) determined in the machine learning process.

[0090] If the score (score1) output in step S68 is greater than the threshold (threshold 1) for anomaly determination determined in machine learning processing, then the abnormal sound determination device 10 determines in step S84 that the action of the product 16, which is the object of abnormal sound determination, is abnormal.

[0091] If the score (score1) output in step S60 is not greater than the threshold (threshold 1) for anomaly determination determined in the machine learning process, or if the score (score1) output in step S68 is not greater than the threshold (threshold 1) for anomaly determination determined in the machine learning process, then the abnormal sound determination device 10 performs the processing in step S72.

[0092] In step S72, the abnormal sound detection device 10 inputs the average value of the frequency direction to the machine learning model that has completed learning using LOF and outputs a score (score2). In step S74, the abnormal sound detection device 10 compares the score (score2) output in step S72 with the anomaly detection threshold (threshold 2) determined in the machine learning process.

[0093] If the score (score2) output in step S72 is greater than the anomaly detection threshold (threshold 2) determined in the machine learning process, the abnormal sound detection device 10 performs steps S76 to S80 of motion sound detection after removing the influence of ambient sound. In step S76, the abnormal sound detection device 10 confirms the ambient sound data obtained from the microphone 14B. The abnormal sound detection device 10 obtains, for example, a short-time Fourier transform of the ambient sound data. Figure 10 The frequency domain characteristics of the surrounding sound data are shown. Furthermore, as... Figure 10 As shown, the abnormal sound detection device 10 calculates the average value in the frequency direction through preprocessing.

[0094] In step S78, the abnormal sound determination device 10 compares the average frequency direction calculated in step S72 with the average frequency direction of the surrounding sound data calculated in step S76. Figure 10 As shown, the average value of the frequency direction calculated in step S72 is corrected to reduce the uniform portion of the peak value. Through the processing in step S78, if there is a periodically generated sound in the surroundings (such as an alarm, the sound of other equipment operating, etc.), the abnormal sound determination device 10 can remove the influence of the sound.

[0095] In step S80, the abnormal sound determination device 10 inputs the average value of the frequency direction corrected in step S78 into the machine learning model that has completed learning using LOF, and outputs a score (score2). In step S82, the abnormal sound determination device 10 compares the score (score2) output in step S80 with the anomaly determination threshold (threshold 2) determined in the machine learning process.

[0096] If the score (score2) output in step S80 is greater than the threshold (threshold 2) for anomaly determination determined in machine learning processing, then the abnormal sound determination device 10 determines in step S84 that the action of the product 16, the object of the abnormal sound determination, is abnormal.

[0097] If the score (score2) output in step S72 is not greater than the anomaly determination threshold (threshold 2) determined in the machine learning process, or if the score (score2) output in step S80 is not greater than the anomaly determination threshold (threshold 2) determined in the machine learning process, then the abnormal sound determination device 10 proceeds to step S86. In step S86, the abnormal sound determination device 10 determines that the operation of the product 16, the object of the abnormal sound determination, is normal.

[0098] The abnormal sound determination system 1 of this embodiment described above can be applied, for example, to the abnormal sound inspection of a production line. In abnormal sound inspections on a production line, operators sometimes listen to abnormal sounds caused by component interference or other factors to make a judgment. However, the operator's judgment criteria for abnormal sounds depend on the operator's own judgment. Therefore, in abnormal sound inspections performed by the operator, there is a possibility that the judgment results regarding the movement sound of the product 16 may deviate due to differences in the operator's judgment.

[0099] In the abnormal sound determination system 1 according to this embodiment, a machine learning model is learned by using normal product 16 motion sound data (normal sound data). Based on the degree of deviation between the characteristics of the normal sound data and the characteristics of the motion sound data of the product 16, the abnormal sound determination object can be automatically determined to be abnormal.

[0100] Therefore, in the abnormal sound determination system 1 of this embodiment, the abnormal sound inspection of product 16 can be automated, saving personnel for abnormal sound inspection.

[0101] The above description of the embodiments is provided, but it is understood that various changes in form and detail may be made without departing from the spirit and scope of the claims.

[0102] For example, in the above embodiment, the machine learning model using LOF is trained using both the time-based average and the frequency-based average, but only one method may be used. Furthermore, in the above embodiment, the influence of ambient noise is removed when the motion sound is abnormal, but the influence of ambient noise may also be removed from the motion sound data beforehand.

[0103] For example, the information obtained by the abnormal sound detection device 10 is not limited to the sound pressure of microphones 14A and 14B, but can also be data measured by vibration sensors, current and voltage waveforms, and other time series data.

[0104] A machine learning model using an autoencoder is one example; other deep learning methods, such as VAE (variational autoencoder) or GAN (generative adversarial networks), could also be used.

[0105] Furthermore, the average values ​​in the frequency and time directions can be calculated not after the differenced data is generated, but before the autoencoder receives the input and is then learned. After generating the differenced data, the data input into the machine learning model using LOF is not limited to the average value; other statistics (distributed, maximum, minimum, etc.) can also be used. Additionally, dimensionality compression of the average values ​​can be performed using methods such as principal component analysis.

[0106] Furthermore, using a machine learning model with LOF is one example; it is not limited to LOF, other machine learning methods such as Mahalanobis distance, one-class SVM, and isolation forest can also be used. Alternatively, these machine learning methods can be used in ensemble learning alongside an autoencoder. In addition, the judgment result for motion sounds can be a score representing the degree of anomalousness, or a label of 0 / 1 representing normal / abnormal. On-site personnel can change the threshold used to determine normal / abnormal as needed. Furthermore, in a sufficiently quiet environment, the abnormal sound judgment system 1 of this embodiment can also disable the function of the ambient sound removal unit 54.

[0107] The present invention has been described above based on embodiments, but the present invention is not limited to the above embodiments, and various modifications can be made within the scope of the claims. This application claims priority to basic application No. 2021-022070 filed with the Japan Patent Office on February 15, 2021, the entire contents of which are incorporated herein by reference.

[0108] [Symbol Explanation]

[0109] 1 Abnormal Sound Detection System

[0110] 10 Abnormal Sound Detection Device

[0111] 12 Inspection device

[0112] 14A and 14B microphones

[0113] 16 products

[0114] 50 Motion Sound Data Acquisition Department

[0115] 52 Ambient Sound Data Acquisition Department

[0116] 54 Ambient Sound Elimination Section

[0117] 56 Abnormal Sound Detection Department

[0118] 58 Display Controller

[0119] 60 Learning Algorithm Department

[0120] 62. Decision Algorithm Department.

Claims

1. An abnormal sound detection system, comprising: The motion sound data acquisition unit acquires the motion sound data of the product. Ambient sound data acquisition unit, which acquires ambient sound data; The correction unit corrects the motion sound data based on the ambient sound data. The determination unit determines the product's motion sound based on the motion sound data after correcting the ambient sound data. The determination unit utilizes a determination algorithm composed of a first machine learning model and a second machine learning model. The first machine learning model learns the normal sound data of the product and predicts the deviation value of the product's movement sound relative to the normal sound data of the product. The second machine learning model learns the degree of deviation of the deviation value of the normal sound data of the product predicted by the first machine learning model, and predicts that the product's movement sound is abnormal when the degree of deviation of the deviation value of the product's movement sound is greater than a threshold.

2. The abnormal sound detection system according to claim 1, characterized in that, The motion sound data acquisition unit uses a microphone to pick up the motion sounds of the product placed inside the soundproof box.

3. The abnormal sound detection system according to claim 1, characterized in that, The ambient sound data acquisition unit uses a microphone to pick up ambient sounds generated outside the soundproof box.

4. The abnormal sound detection system according to claim 1, characterized in that, The first machine learning model uses an autoencoder.

5. The abnormal sound detection system according to claim 1 or 4, characterized in that, The second machine learning model uses the local outlier method.

6. An abnormal sound detection device, comprising: The motion sound data acquisition unit acquires the motion sound data of the product. Ambient sound data acquisition unit, which acquires ambient sound data; The correction unit corrects the motion sound data based on the ambient sound data. The determination unit determines the product's motion sound based on the motion sound data after correcting the ambient sound data. The determination unit utilizes a determination algorithm composed of a first machine learning model and a second machine learning model. The first machine learning model learns the normal sound data of the product and predicts the deviation value of the product's movement sound relative to the normal sound data of the product. The second machine learning model learns the degree of deviation of the deviation value of the normal sound data of the product predicted by the first machine learning model, and predicts that the product's movement sound is abnormal when the degree of deviation of the deviation value of the product's movement sound is greater than a threshold.

7. A program product that causes a computer to perform the following steps: The step of acquiring motion sound data involves obtaining the motion sound data of the product. The step of acquiring ambient sound data involves acquiring ambient sound data. The correction step corrects the motion sound data based on the surrounding sound data; The determination step involves determining the product's motion sound based on the motion sound data after correcting the ambient sound data. The determination step utilizes a determination algorithm composed of a first machine learning model and a second machine learning model. The first machine learning model learns the normal sound data of the product and predicts the deviation value of the product's movement sound relative to the normal sound data of the product. The second machine learning model learns the degree of deviation of the deviation value of the normal sound data of the product predicted by the first machine learning model, and predicts that the product's movement sound is abnormal when the degree of deviation of the deviation value of the product's movement sound is greater than a threshold.

Images