Abnormal sound analysis system and industrial automation system comprising same

Abstract

The invention mainly discloses an abnormal sound analysis system and an industrial automation system comprising the same, which are used for acquiring a time domain sound signal from mechanical equipment and then converting the time domain sound signal into a frequency domain sound signal containing K frequencies. In particular, the abnormal sound analysis system is configured to individually separate the K frequencies from the frequency-domain sound signal, thereby obtaining K sub-frequency-domain sound signals, and then inversely convert the K sub-frequency-domain sound signals into K sub-sound signals. Therefore, the K sub-sound signals respectively correspond to K mechanical units and / or mechanical modules contained in the mechanical equipment, so that an engineer can judge whether the corresponding mechanical unit is abnormal or not by playing and listening to the specific sub-sound signals.

Description

Technical Field

[0001] The present invention relates to the technical field of mechanical abnormal noise analysis, and in particular to an abnormal noise analysis system and an industrial automation system including the same. Background Art

[0002] It's well known that mechanical equipment, such as conveyor belts and robotic arms, plays an indispensable role in industrial automation. However, after a period of use, these devices inevitably produce unusual noises, such as vibrations, high-frequency humming, metal collisions, hisses, and clattering. In light of this, Taiwan Patent No. TWI587294B discloses a method for detecting unusual noises in equipment, comprising the following steps:

[0003] A sound signal when the acquisition device is running;

[0004] performing a pre-processing on the sound signal to obtain a processed sound signal;

[0005] extracting a plurality of features from the processed sound signal;

[0006] Performing a cluster analysis and / or a SVM linear classification on the plurality of features and a plurality of reference features stored in a database; and

[0007] Determine whether the sound signal is an abnormal sound based on the analysis result.

[0008] Practical experience shows that performing feature extraction, cluster analysis, and / or SVM linear classification in real time consumes considerable computer processor power. In other words, the computer used to implement the device noise detection method described in Taiwan Patent No. TWI587294B must include a high-performance processor and a large cache memory (SRAM). This, in turn, limits the application of the device noise detection method.

[0009] As can be seen from the above description, conventional methods for detecting and analyzing abnormal noise in equipment still have room for improvement. In view of this, the inventors of this case have made great efforts to research and invent, and finally developed a system for analyzing abnormal noise. Summary of the Invention

[0010] The primary objective of the present invention is to provide an abnormal sound analysis system for acquiring a time-domain sound signal from a mechanical device and then converting the time-domain sound signal into a frequency-domain sound signal containing K frequencies. Specifically, the abnormal sound analysis system is configured to individually separate the K frequencies from the frequency-domain sound signal, thereby obtaining K sub-frequency-domain sound signals, and then deconvert the K sub-frequency-domain sound signals into K sub-sound signals. In this way, the K sub-sound signals correspond to the K mechanical units and / or mechanical modules included in the mechanical device, allowing engineers to determine whether a corresponding mechanical unit is experiencing an anomaly by playing or listening to a specific sub-sound signal. Of course, the abnormal sound analysis system can also be configured to automatically classify the K sub-sound signals as abnormal using an AI model, and issue an alert to engineers when at least one sub-sound signal is classified as abnormal.

[0011] To achieve the above objectives, the present invention provides an embodiment of the abnormal sound analysis system, which includes:

[0012] a sound capturing device for performing a sound capturing process on a mechanical device to obtain a time-domain sound signal; and

[0013] An electronic device is coupled to the sound capture device and includes a processor and a memory, wherein the memory stores an application program, and the processor executes the application program and is configured to perform:

[0014] Performing a time-domain-to-frequency-domain conversion process on the time-domain sound signal to obtain a frequency-domain sound signal; wherein the frequency-domain sound signal contains K frequencies, and K is a positive integer;

[0015] performing a sub-signal separation process to separate the K frequencies from the frequency-domain sound signal individually, thereby obtaining K sub-frequency-domain sound signals; and

[0016] A frequency domain-time domain conversion process is performed on the K sub-frequency domain sound signals to obtain K sub-sound signals.

[0017] In one embodiment, the recording device is any one selected from the group consisting of a recorder, a smart phone, a tablet computer, a notebook computer, and an all-in-one computer.

[0018] In one embodiment, the electronic device is any one selected from the group consisting of a smart phone, a tablet computer, a notebook computer, a desktop computer, an all-in-one computer, and an industrial computer.

[0019] In a possible embodiment, the electronic device further includes or is coupled to a speaker, and the processor executes the application program and is further configured to perform: controlling the speaker to play the sub-sound signal.

[0020] In another possible embodiment, the memory stores a plurality of reference features simultaneously, and the processor executes the application program and is further configured to perform:

[0021] extracting at least one feature from the sub-frequency domain sound signal or the sub-sound signal; and

[0022] inputting the plurality of reference features and the at least one feature into a pre-trained abnormal sound recognition model;

[0023] The abnormal sound recognition model performs a feature matching on the at least one feature and the plurality of reference features, thereby identifying the sub-sound signal as a normal sound signal or an abnormal sound signal emitted by a mechanical unit.

[0024] In another possible embodiment, the processor 12P executes the application and is further configured to: merge at least two sub-sound signals having the same characteristics into a combined sub-sound signal.

[0025] In another possible embodiment, the processor 12P executes the application and is further configured to: control the electronic device to emit an alarm signal when the sub-sound signal is identified as an abnormal sound signal.

[0026] In one embodiment, each of the frequencies has an amplitude, and the sub-signal separation operation includes the following steps:

[0027] Sort the K frequencies according to the amplitudes; and

[0028] The K frequencies are separated from the frequency domain sound signal in sequence according to the order.

[0029] In one embodiment, the processor correspondingly generates an i-th bandpass Butterworth filter to separate the frequency ranked i from the i-th residual frequency domain sound signal, and correspondingly generates an i-th bandreject Butterworth filter to separate the i+1-th residual frequency domain sound signal from the i-th residual frequency domain sound signal; wherein i is a positive integer, the minimum value of i is 1, the maximum value of i is K, and the first residual frequency domain sound signal is the frequency domain sound signal.

[0030] Furthermore, the present invention also discloses an industrial automation system having a plurality of mechanism units and / or mechanism modules, and is characterized in that it also has at least one abnormal sound analysis system of the present invention as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A schematic three-dimensional diagram of an abnormal sound analysis system of the present invention;

[0032] Figure 2 is a block diagram of the abnormal sound analysis system of the present invention;

[0033] Figure 3 Schematic diagram of the operation of time domain-frequency domain conversion processing;

[0034] Figure 4A 、 Figure 4B and Figure 4C This is a schematic diagram of the operation of sub-signal separation processing;

[0035] Figure 5A is the waveform of the first sub-sound signal;

[0036] Figure 5B is the waveform diagram of the second sub-sound signal;

[0037] Figure 5C is a waveform diagram of the third sub-sound signal; and

[0038] Figure 6 A schematic diagram of the operation of sub-signal separation processing and frequency domain-time domain conversion processing; and

[0039] Figure 7 Schematic diagram of the merging operation of two sub-sound signals.

[0040] Description of reference numerals:

[0041] 1: Abnormal sound analysis system

[0042] 11: Sound Capture Device

[0043] 12: Electronic devices

[0044] 12P: Processor

[0045] 12B: Speaker

[0046] 12C: Communication interface

[0047] 12D: Display

[0048] 12M: Memory

[0049] 12M1: Control module

[0050] 12M2:Signal processing module

[0051] 12M3: Feature extraction module

[0052] 12M4: foreign sound recognition model module

[0053] 2: Mechanical equipment DETAILED DESCRIPTION

[0054] In order to more clearly describe the abnormal sound analysis system proposed by the present invention, the following will be used in conjunction with the drawings to fully illustrate the preferred embodiments of the present invention.

[0055] See also Figure 1 , which is a schematic three-dimensional diagram of an abnormal sound analysis system of the present invention. And, Figure 2 FIG. 1 is a block diagram of the abnormal sound analysis system of the present invention. Figure 1 and Figure 2 As shown, the abnormal sound analysis system 1 of the present invention is applied to an industrial automation system, and is used to monitor whether a mechanical device 2 included in the industrial automation system emits abnormal sounds, and to determine, based on the abnormal sounds, whether at least one of the multiple mechanical units / modules included in the mechanical device 2 is in an abnormal working state.

[0056] like Figure 1 and Figure 2 As shown, the abnormal sound analysis system 1 of the present invention mainly includes a sound capture device 11 and an electronic device 12, wherein the sound capture device 11 is arranged adjacent to a mechanical device 2, and is used to perform a sound capture process on the mechanical device 2 to obtain a time domain sound signal. In a feasible embodiment, the sound capture device 11 can be but not limited to a recorder, a smart phone, a tablet computer, a laptop computer, or an all-in-one computer. In addition, the electronic device 12 can be but not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, an all-in-one computer, or an industrial computer. As Figure 1 and Figure 2As shown, the electronic device 12 is coupled to the sound capturing device 11 and includes a processor 12P and a memory 12M, wherein the time domain sound signal is stored in the memory 12M in the form of a recording file, and the memory 12M stores an application, so that the processor 12P is configured to perform multiple functions after executing the application.

[0057] In one embodiment, if Figure 2 As shown, the application includes multiple subroutines (i.e., subroutine modules), and the multiple subroutines include: a control module 12M1, a signal processing module 12M2, a feature extraction module 12M3, and a strange sound recognition model module 12M4. Computer science (CS) engineers familiar with AI recognition models should know that the strange sound recognition model module 12M4 uses multiple training samples to perform sound feature recognition machine learning training on a feature classifier, where the multiple training samples include: multiple normal sound features (i.e., golden samples) extracted from multiple different mechanical units / modules and multiple abnormal sound features extracted from multiple different mechanical units / modules, and these normal sound features and abnormal sound features have been pre-classified and labeled (i.e., pre-labeled).

[0058] Therefore, after accessing the memory 12M to execute the application, the processor 12P is configured to execute:

[0059] Performing a time-domain-to-frequency-domain conversion process on the time-domain sound signal to obtain a frequency-domain sound signal; wherein the frequency-domain sound signal contains K frequencies, and K is a positive integer;

[0060] performing a sub-signal separation process to separate the K frequencies from the frequency-domain sound signal individually, thereby obtaining K sub-frequency-domain sound signals; and

[0061] A frequency domain-time domain conversion process is performed on the K sub-frequency domain sound signals to obtain K sub-sound signals.

[0062] Figure 3 FIG. 1 is a schematic diagram of the operation of the time domain-frequency domain conversion process. Figure 3 In the example, the time domain sound signal is specifically labeled as “Waveform”, and the processor 12P uses the signal processing module 12M2 to perform a Fast Fourier Transform (FFT) on the time domain sound signal, so that the time domain sound signal is transformed into a frequency domain sound signal containing K frequencies after FFT processing. It can be found that Figure 3The frequency domain sound signal containing K frequencies is specifically labeled as "Spectrogram".

[0063] Furthermore, the processor 12P continues to utilize the signal processing module 12M2 to perform a sub-signal separation process on the frequency domain sound signal. Figures 4A to 4C The diagram is a schematic diagram of the sub-signal separation operation. Specifically, each frequency has an amplitude, and the sub-signal separation operation first sorts the K frequencies according to the amplitude, and then separates the K frequencies from the frequency domain sound signal in order according to the sorting. For example, in Figure 4A In the figure (a), the frequency domain sound signal has the maximum amplitude at the frequency of 810Hz. At this time, the processor 11P generates the first bandpass Butterworth filter to separate the frequency ranked as 1 (i.e., 810Hz) from the frequency domain sound signal, and obtains the first sub-frequency domain sound signal as shown in Figure (b). At the same time, the processor 11P correspondingly generates the first bandreject Butterworth filter to remove the frequency from the frequency domain sound signal (i.e., Figure 4A The first residual frequency domain sound signal is separated from Figure (a) in the figure.

[0064] exist Figure 4B , Figure (c) is the first residual frequency domain sound signal, and it is specifically marked as "Spectrogram remain". Figure 4B As shown in FIG. 1 , the frequency 550 Hz has the maximum amplitude. Therefore, the processor 11P generates a second bandpass Butterworth filter to separate the frequency (i.e., 550 Hz) ranked 2 in amplitude from the first residual frequency domain sound signal, thereby obtaining a second sub-frequency domain sound signal as shown in FIG. (d). At the same time, the processor 11P correspondingly generates a second bandreject Butterworth filter to remove the second sub-frequency domain sound signal from the first residual frequency domain sound signal (i.e., Figure 4B The second residual frequency domain sound signal is separated from Figure (c) in the figure.

[0065] exist Figure 4C In FIG, FIG (e) is the second residual frequency domain sound signal, and it is also labeled as “Spectrogram remain”. Figure 4CAs shown, the frequency 817 Hz has the largest amplitude. Therefore, the processor 11P generates a third bandpass Butterworth filter to separate the frequency (i.e., 817 Hz) ranked 3rd in amplitude from the second residual frequency domain sound signal, obtaining a third sub-frequency domain sound signal as shown in FIG (f). At the same time, the processor 11P correspondingly generates a third bandreject Butterworth filter to remove the second residual frequency domain sound signal (i.e., Figure 4C The third residual frequency domain sound signal is separated from Figure (e) in the figure.

[0066] like Figures 4A to 4C As shown, in short, when performing the sub-signal separation operation, the processor correspondingly generates an i-th bandpass Butterworth filter to separate the frequency ranked i from the i-th residual frequency domain sound signal, and correspondingly generates an i-th bandreject Butterworth filter to separate the i+1-th residual frequency domain sound signal from the i-th residual frequency domain sound signal; wherein i is a positive integer, the minimum value of i is 1, the maximum value of i is K, and the first residual frequency domain sound signal is the frequency domain sound signal.

[0067] On the other hand, the processor 12P continues to use the signal processing module 12M2 to perform a short-time Fourier transform (STFT) on the K sub-frequency domain sound signals, so that the sub-frequency domain sound signals are converted into a sub-sound signal after the STFT processing. In addition, the K sub-sound signals corresponding to the K frequencies can also be stored in the memory 12M in the form of a recording file. Furthermore, Figure 5A 、 Figure 5B and Figure 5C The waveforms of the first, second and third sub-sound signals are shown in Figure 2. Figure 5A 、 Figure 5B and Figure 5C Among them, the waveforms of the first, second and third sub-sound signals are marked as "Waveform peak i 810Hz", "Waveform peak i 550Hz" and "Waveform peak i817Hz" respectively. In short, Figures 4A to 4C as well as Figures 5A to 5C The signal processing flow can be integrated as Figure 6As shown, the processor 12P executes the signal processing module 12M2 to perform the sub-signal separation processing to separate K sub-frequency domain sound signals from the frequency domain sound signal, and then performs the frequency domain-time domain conversion processing to convert the K sub-frequency domain sound signals into K sub-sound signals.

[0068] It is worth noting that, in a feasible embodiment, the electronic device 12 further includes or is coupled to a speaker 12B, and the processor 12P executes the control module 12M1 of the application program and is further configured to control the speaker 12B to play the sub-sound signal. Figure 5A In the example, the sub-frequency domain sound signal of 810 Hz is transformed into a sub-sound signal after IFFT processing, and when the sub-sound signal is played by the speaker 12B, an obvious beat feature can be heard, so the sub-sound signal can be determined to be an abnormal sound. Figure 5B In the example, the sub-frequency domain sound signal of 550 Hz is transformed into a sub-sound signal after STFT processing, and when the sub-sound signal is played by the speaker 12B, no obvious beat characteristics can be heard, so it can be determined that the sub-sound signal is a motor operation sound. Figure 5C In the example, the sub-frequency domain sound signal of the frequency 817 Hz is transformed into a sub-sound signal after STFT processing, and when the sub-sound signal is played by the speaker 12B, the same beat characteristics can be heard. Therefore, the sub-sound signal can be determined to be an abnormal sound. In this case, if Figure 7 As shown, the processor 12P can use the signal processing module 12M2 to Figure 5A sub-sound signal and Figure 5C In other words, the processor 12P executes the application program and is further configured to perform the following steps: merging at least two of the sub-sound signals having the same characteristics into a combined sub-sound signal.

[0069] The above description indicates that the K sub-frequency domain sound signals are inversely converted into K sub-sound signals. In this way, the K sub-sound signals correspond to the K mechanical units and / or mechanical modules included in the mechanical device 2, so that engineers can judge whether the corresponding mechanical unit has an abnormality by playing and listening to specific sub-sound signals. Of course, the abnormal sound analysis system of the present invention can also be configured through an AI model (i.e., Figure 2The abnormal sound recognition model 12M4 shown automatically classifies K sub-sound signals as abnormal sounds and issues an alert to the engineer when at least one sub-sound signal is classified as abnormal. Therefore, in a feasible embodiment, the memory 12M simultaneously stores multiple reference features, and the processor 12P executes the feature extraction module 12M3 of the application program and is further configured to extract at least one feature from the sub-frequency domain sound signal or the sub-sound signal. Furthermore, the processor 12P inputs the multiple reference features and the at least one feature into a pre-trained abnormal sound recognition model 12M4, thereby utilizing the abnormal sound recognition model 12M4 to perform a feature matching on the at least one feature and the multiple reference features, thereby identifying the sub-sound signal as a normal sound signal or an abnormal sound signal emitted by a mechanical unit.

[0070] Furthermore, the processor 12P executes the control module 12M1 of the application program and is configured to execute: when the sub-sound signal is identified as an abnormal sound signal, control the electronic device 12 to issue an alarm signal. Figure 1 and Figure 2 As shown, the electronic device 12 includes a display 12D and a communication interface 12C, and further includes or is coupled to a speaker 12B. Therefore, the processor 12P can control the display 12D to display an alert signal containing images and / or text, and / or control the speaker 12B to emit an alert sound. Furthermore, the processor 12P can also send the alert signal containing images and / or text via the communication interface 12C via a text message or email.

[0071] In summary, the present invention provides an abnormal sound analysis system 1, which is applied to an industrial automation system and is used to monitor whether a mechanical device 2 included in the industrial automation system emits abnormal sounds. Based on the abnormal sounds, the system determines whether at least one of the multiple mechanical units / modules within the mechanical device 2 is operating abnormally. Specifically, the system 1 is configured to record the mechanical device 2 to obtain a time-domain sound signal, and then convert the time-domain sound signal into a frequency-domain sound signal containing K frequencies. Furthermore, the system 1 is configured to individually separate the K frequencies from the frequency-domain sound signal to obtain K sub-frequency-domain sound signals, and then deconvert the K sub-frequency-domain sound signals into K sub-sound signals. Thus, the K sub-sound signals correspond to the K mechanical units and / or modules within the mechanical device, allowing engineers to determine whether a corresponding mechanical unit is experiencing an abnormality by playing or listening to a specific sub-sound signal. Of course, the abnormal sound analysis system can also be configured to automatically classify the K sub-sound signals as abnormal sounds through an AI model, and issue an alarm notification to the engineering staff when at least one sub-sound signal is classified as an abnormal sound.

[0072] The above description thus fully and clearly describes the noise analysis system and method of the present invention. However, it must be emphasized that the above disclosure of this patent is merely a preferred embodiment, and any partial changes or modifications that are derived from the technical principles of this patent and are readily inferred by those skilled in the art are within the scope of this patent.

Claims

1. A system for analyzing abnormal sound, characterized in that: include: A sound capturing device is used to perform a sound capture process on a mechanical device to obtain a time domain sound signal; as well as An electronic device is coupled to the sound capture device and includes a processor and a memory, wherein the memory stores an application program, and the processor executes the application program and is configured to perform: Performing a time-domain-to-frequency-domain conversion process on the time-domain sound signal to obtain a frequency-domain sound signal; wherein the frequency-domain sound signal contains K frequencies, and K is a positive integer; performing a sub-signal separation process to separate the K frequencies from the frequency-domain sound signal individually, thereby obtaining K sub-frequency-domain sound signals; and A frequency domain-time domain conversion process is performed on the K sub-frequency domain sound signals to obtain K sub-sound signals.

2. The abnormal sound analysis system according to claim 1, characterized in that: The recording device is any one selected from the group consisting of a recorder, a smart phone, a tablet computer, a notebook computer, and an all-in-one computer.

3. The abnormal sound analysis system according to claim 1, characterized in that: The electronic device is any one selected from the group consisting of a smart phone, a tablet computer, a notebook computer, a desktop computer, an all-in-one computer, and an industrial computer.

4. The abnormal sound analysis system according to claim 1, characterized in that: The electronic device further includes or is coupled to a speaker, and the processor executing the application is further configured to perform: The speaker is controlled to play the sub-sound signal.

5. The abnormal sound analysis system according to claim 1, characterized in that: The memory concurrently stores a plurality of reference signatures, and the processor executes the application program and is further configured to perform: extracting at least one feature from the sub-frequency domain sound signal or the sub-sound signal; and Inputting the plurality of reference features and the at least one feature into a pre-trained abnormal sound recognition model; The abnormal sound recognition model performs a feature matching on the at least one feature and the plurality of reference features, thereby identifying the sub-sound signal as a normal sound signal or an abnormal sound signal emitted by a mechanical unit.

6. The abnormal sound analysis system according to claim 5, characterized in that: The processor executing the application is further configured to perform: At least two of the sub-sound signals having the same characteristics are combined into a combined sub-sound signal.

7. The abnormal sound analysis system according to claim 5, characterized in that: The processor executing the application is further configured to perform: When the sub-sound signal is identified as an abnormal sound signal, the electronic device is controlled to emit a warning signal.

8. The abnormal sound analysis system according to claim 5, characterized in that: Each of the frequencies has an amplitude, and the sub-signal separation operation includes the following steps: Sort the K frequencies according to the amplitudes; and The K frequencies are separated from the frequency domain sound signal in sequence according to the order.

9. The abnormal sound analysis system according to claim 8, characterized in that: The processor correspondingly generates an i-th bandpass Butterworth filter to separate the frequency ranked i from the i-th residual frequency domain sound signal, and correspondingly generates an i-th bandreject Butterworth filter to separate the i+1-th residual frequency domain sound signal from the i-th residual frequency domain sound signal; wherein i is a positive integer, the minimum value of i is 1, the maximum value of i is K, and the first residual frequency domain sound signal is the frequency domain sound signal.

10. An industrial automation system having a plurality of mechanism units and / or mechanism modules, characterized in that: At the same time, there is at least one abnormal sound analysis system according to any one of claims 1 to 9.

Images