Device sound anomaly detection method and apparatus, and electronic device

Abstract

The application relates to the technical field of audio analysis, and discloses a device sound anomaly detection method and device and electronic equipment, the method comprising the following steps: mixing acquired device running sound and environmental noise according to different signal-to-noise ratios to construct an enhanced data set; training a self-encoder model based on the enhanced data set to obtain an audio detection model; acquiring the spectral features of to-be-detected audio, inputting the spectral features into the audio detection model, and calculating a reconstruction error; based on the reconstruction error and the matching result of the spectral features and a preset audio feature library, identifying whether the to-be-detected audio is abnormal audio, and classifying the abnormal audio; the application mixes device running sound and environmental noise according to different signal-to-noise ratios to construct an enhanced data set, thereby improving the generalization capability of the model in a complex noise environment; a two-stage diagnosis strategy combining the self-encoder reconstruction error and the matching of the preset audio feature library can effectively identify known abnormal types and detect unknown abnormalities.

Description

Technical Field

[0001] This invention relates to the field of audio analysis technology, specifically to methods, devices, and electronic equipment for detecting abnormal sound in devices. Background Technology

[0002] As a core piece of equipment for bulk material conveying, belt conveyors often initially manifest as faint, audible abnormal noises when early mechanical defects appear in their idlers, drums, or belt surfaces. Sound monitoring is a crucial method for identifying anomalies in components such as idlers and belts. Traditional acoustic detection methods, such as vibration and temperature monitoring, typically rely on fixed thresholds or simple signal processing, making it difficult to effectively separate the conveyor's operating noise from environmental interference and achieve reliable identification in complex noise environments. Manual inspection methods, limited by cost and subjective experience, cannot perform real-time, high-intensity inspections to capture these transient sound signals under continuous, strong background noise.

[0003] Existing machine learning-based fault diagnosis systems often face the following difficulties: First, it is difficult to obtain a large number of abnormal sound samples in actual production, resulting in unbalanced model training data and insufficient generalization ability; second, conventional methods can only identify preset known fault types (such as bearing wear), and cannot autonomously detect sudden unknown anomalies (such as roller breakage and abnormal noise), which must rely on manual verification, resulting in serious response delays; third, after the model is deployed, its judgment logic is usually fixed and lacks a dynamic update mechanism, making it unable to adapt to equipment aging or seasonal changes in operating conditions, and the recognition accuracy is prone to decline after long-term use. Summary of the Invention

[0004] This invention provides a method, apparatus, and electronic device for detecting abnormal sounds in equipment, in order to solve the problems of low accuracy and difficulty in identifying unknown abnormality types in existing belt conveyor sound abnormality detection methods under complex noise environments.

[0005] In a first aspect, the present invention provides a method for detecting abnormal sound in a device, the method comprising: The acquired device operating sounds and environmental noise are mixed according to different signal-to-noise ratios to construct an enhanced dataset; An audio detection model is obtained by training an autoencoder model based on an augmented dataset. The spectral features of the audio to be detected are obtained and input into the audio detection model to calculate the reconstruction error. Based on the reconstruction error and the matching results between the spectral features and the preset audio feature library, the system identifies whether the audio to be detected is abnormal and classifies the abnormal audio.

[0006] This invention provides a method for detecting abnormal device sounds. By mixing device operating sounds with environmental noise at different signal-to-noise ratios to construct an enhanced dataset, the generalization ability of the model in complex noise environments is improved. By using a two-stage diagnostic strategy that combines autoencoder reconstruction error with matching of a preset audio feature library, it can effectively identify known anomaly types and detect unknown anomalies, thereby solving the technical problems of low recognition rate and difficulty in identifying unknown anomalies in noisy environments in existing methods.

[0007] In one alternative implementation, the acquired device operating sound is mixed with ambient noise at different signal-to-noise ratios to construct an enhanced dataset, including: The acquired device operating sound and environmental noise are used as audio samples, and then mixed according to different preset signal-to-noise ratios to obtain synthetic audio samples. After standard preprocessing of the synthesized audio samples, the augmented dataset is obtained.

[0008] In the above technical solution, by mixing the device operating sound with environmental noise at different signal-to-noise ratios, synthetic audio samples covering a variety of noise intensities are generated, which effectively expands the diversity of training data and solves the problem of scarce abnormal samples in real-world scenarios. After standardized preprocessing, an augmented dataset is constructed, which significantly improves the model's generalization ability and detection robustness in complex noise environments.

[0009] In one optional implementation, the augmented dataset includes normal audio samples and abnormal audio samples. An autoencoder model is trained based on the augmented dataset to obtain an audio detection model, including: Spectral features of audio samples are extracted from the augmented dataset, and the spectral features are divided into training and validation sets. The spectral features include normal sample features extracted from normal audio samples and abnormal sample features extracted from abnormal audio samples. The features of normal samples in the training set are input into the autoencoder model for iterative training to learn the compression and reconstruction of normal sound patterns, thus obtaining a trained autoencoder model. The reconstruction error distribution of the validation set is calculated based on the trained autoencoder model, and the anomaly detection threshold is determined based on the reconstruction error distribution. The trained autoencoder model and the anomaly detection threshold are encapsulated into an audio detection model.

[0010] In the above technical solution, the autoencoder is trained using the features of normal samples in the augmented dataset, so that it focuses on learning normal sound patterns, thus overcoming the problem of scarce abnormal samples; the discrimination threshold is adaptively determined by reconstructing the error distribution through the validation set, thus avoiding the subjectivity of manually setting the threshold; finally, the trained model and the threshold are encapsulated to form a complete audio detection model, providing a reliable foundation for subsequent anomaly recognition.

[0011] In one alternative implementation, the preset audio feature library is obtained in the following manner: A normal feature library is constructed based on the features of normal samples, and an abnormal feature library is constructed based on the features of abnormal samples. Use the normal feature library and the abnormal feature library as the preset audio feature library.

[0012] In the above technical solution, by constructing normal feature libraries and abnormal feature libraries respectively, normal and abnormal sound patterns are structurally separated and stored, so that the preset audio feature library has a clear category differentiation ability; in subsequent detection, the spectral features of the audio to be tested can be compared with the two feature libraries respectively, which can not only identify known abnormal types, but also provide a comparison benchmark for judging unknown abnormalities.

[0013] In one optional implementation, based on the reconstruction error and the matching results of the spectral features with a preset audio feature library, it is determined whether the audio to be detected is abnormal audio, and the abnormal audio is classified, including: The spectral features of the audio to be detected are matched with the normal feature library and the abnormal feature library respectively for similarity. The category of the audio to be detected is determined by comprehensively considering the matching results and whether the reconstruction error exceeds the anomaly detection threshold.

[0014] In the above technical solution, by simultaneously matching the spectral features of the audio to be detected with the similarity databases of normal and abnormal features, and combining this with a comprehensive judgment based on whether the reconstruction error exceeds the discrimination threshold, it is possible to effectively distinguish between normal audio, known abnormal audio, and unknown abnormal audio, thereby achieving accurate identification and classification of abnormal audio and significantly improving the accuracy and reliability of the detection results.

[0015] In one optional implementation, the category of the audio to be detected is determined comprehensively based on the matching results and whether the reconstruction error exceeds the anomaly detection threshold, including: If the spectral characteristics of the audio to be detected match those of the normal feature library, it is determined to be normal audio. If the spectral characteristics of the audio to be detected match the abnormal feature database, it is determined to be a known abnormal audio. If the spectral features of the audio to be detected do not match the normal feature library and the abnormal feature library, and the reconstruction error exceeds the anomaly discrimination threshold, it is determined to be an unknown abnormal audio.

[0016] The above technical solution, through clear matching and error judgment rules, can accurately identify normal audio, known abnormal audio and unknown abnormal audio, and achieve fine classification of abnormal audio; in particular, the ability to identify unknown abnormalities effectively covers sudden fault types that traditional methods cannot detect, and improves the comprehensiveness and adaptability of the detection system.

[0017] In one alternative implementation, the method further includes: Based on the judgment result, the following tiered alarm triggers will be executed: If the audio is a known abnormal audio, trigger the corresponding abnormality type alert. If the audio is an unknown abnormality, a preset high-priority alarm will be triggered, and a manual review will be requested.

[0018] In the above technical solution, differentiated responses to known and unknown anomalies are achieved by executing tiered alarms based on the judgment results: corresponding type of prompt alarms are triggered for known anomalies to facilitate rapid fault location; high-priority alarms are triggered for unknown anomalies and prompts for manual review to ensure that abnormal situations are promptly addressed and accurately judged.

[0019] In one alternative implementation, the method further includes: Add the feature vectors of the audio samples that have been manually reviewed and confirmed to the corresponding normal feature library or abnormal feature library. The autoencoder model is incrementally trained periodically using audio samples that have been manually reviewed and confirmed, and the parameters of the autoencoder model are updated accordingly.

[0020] In the above technical solution, the feature library is continuously enriched and updated by dynamically adding the features of manually reviewed audio samples to the corresponding normal or abnormal feature library. At the same time, the autoencoder model is incrementally trained using the reviewed samples regularly, so that the model parameters can adapt to new data distributions and changes in working conditions, thereby improving the long-term adaptability and recognition accuracy of the detection system.

[0021] In a second aspect, the present invention provides a device for detecting abnormal sound in a device, the device comprising: The augmented dataset building module is used to mix the acquired device operating sounds with environmental noise according to different signal-to-noise ratios to build an augmented dataset. The model training module is used to train an autoencoder model based on an augmented dataset to obtain an audio detection model. The reconstruction error calculation module is used to obtain the spectral features of the audio to be detected, and input the spectral features into the audio detection model to calculate the reconstruction error; The audio anomaly detection module is used to identify whether the audio to be detected is abnormal based on the reconstruction error and the matching results of the spectral features with the preset audio feature library, and to classify the abnormal audio.

[0022] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the device sound anomaly detection method described in the first aspect or any corresponding embodiment.

[0023] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the device sound anomaly detection method described in the first aspect or any corresponding embodiment thereof.

[0024] Fifthly, the present invention provides a computer program product, including computer instructions, which are used to cause a computer to execute the device sound anomaly detection method described in the first aspect or any corresponding embodiment. Attached Figure Description

[0025] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of an application scenario according to an embodiment of the present invention; Figure 2 This is a schematic flowchart of a first method for detecting abnormal device sound according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a second process for a device sound anomaly detection method according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the third process of the device sound abnormality detection method according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the third process of the device sound abnormality detection method according to an embodiment of the present invention; Figure 6 This is a structural block diagram of a device for detecting abnormal sound in an apparatus according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0029] As an optional application scenario of this invention, such as Figure 1 As shown, this invention provides a device sound anomaly detection system. This system is used for device sound anomaly detection methods. The method involves acquiring on-site audio through dedicated hardware, performing preliminary anomaly detection using a deep learning model, then achieving fault classification and alarm through high-precision feature matching, and finally continuously optimizing the system through a closed-loop feedback mechanism. Specifically, this system is implemented collaboratively by the following three core layers: Perception Layer 110: This part is responsible for acquiring high-quality raw data, including high-fidelity industrial microphone arrays deployed at key points on the equipment (such as near drive rollers and redirecting rollers) or integrated on the inspection robot. The system uses a high-fidelity industrial microphone array as the core pickup unit to collect raw audio data during the operation of the conveyor belt. The sound sensors support contact or non-contact installation and can be directly placed near the bearing seats of key idler rollers or integrated onto the inspection robot carrier, with an effective pickup distance of 0.5 to 2 meters.

[0030] Number of channels: No less than 4 channels, supporting synchronous acquisition; Sampling frequency: ≥44.1 kHz; Audio precision: 24-bit; Frequency response range: 20 Hz - 20 kHz; Protection rating: not lower than IP65, with anti-electromagnetic interference and dust and moisture protection capabilities.

[0031] Edge Algorithm Layer 120 (Edge Computing Device or Central Server): This part is the core of the system, responsible for signal analysis and anomaly detection. It includes feature extraction, semi-supervised training, initial reconstruction screening, and similarity calculation fine screening. It mainly includes two important detection algorithms: feature library establishment and matching, and model training and inference. Among them: Feature extraction module: The system first converts the enhanced dataset signal into a log-Mel spectrum, which can effectively simulate human auditory perception and is more sensitive to specific frequency components presented by equipment failure.

[0032] Model Module: The core of this system is to train the model using only a large number of normal sound samples, enabling the autoencoder to learn efficient compressed representation and reconstruction capabilities of normal sound features. During the inference phase, when a sound is input, the model attempts to reconstruct it; if the sound is abnormal (not previously learned by the model), the reconstruction result will deviate significantly from the original input, resulting in a high reconstruction error. By setting a reasonable error threshold, the system can initially screen out potentially abnormal audio, completing the first layer of efficient screening.

[0033] At the edge algorithm layer 120, a dynamic knowledge base, namely the feature vector library, is constructed: a structured sound feature database that transforms abstract audio signals into measurable and comparable mathematical vectors. The system maintains two sub-libraries: the "normal feature library" stores the sound pattern benchmark under the health state of the device, while the "abnormal feature library" archives the sound features and corresponding types of confirmed faults.

[0034] Similarity matching is also performed: cosine similarity matching is a mathematical method to measure the directional consistency between two feature vectors. Its value range is [-1, 1]. The closer the value is to 1, the more similar the patterns of the two vectors are.

[0035] Interaction layer 130 is responsible for executing preset operations based on the detection results. It receives the judgment results from the intelligent detection module in real time: if the detection result is "normal", only data is recorded; if it is "known anomaly", the corresponding level of alarm is automatically triggered to notify the operation and maintenance personnel; if it is "unknown anomaly", while triggering the alarm, it is marked as requiring "manual review" to prompt professional personnel to intervene for analysis and judgment.

[0036] Dynamic Optimization Layer 140: This part systematically collects samples and their labels that have been manually verified in the alarm response module, and uses this newly added "knowledge" to update the system: on the one hand, it expands the feature vectors of the samples into the corresponding feature library to enrich the system's comparison benchmark; on the other hand, it periodically initiates incremental training to fine-tune the autoencoder model using newly labeled data. Through this mechanism, the system can continuously adapt to changes in operating conditions and continuously improve the accuracy and generalization ability of anomaly identification and classification.

[0037] The workflow of the entire system is as follows: First, the system collects raw audio containing equipment operating sounds and background noise. Then, by mixing normal sounds and noise at different signal-to-noise ratios, an augmented dataset is constructed for model training. Based on this, the system establishes a "normal feature library" and an "abnormal feature library" for comparison, using known normal and abnormal sound samples respectively. In the model recognition stage, the system extracts the log-Mel spectrum of the audio and performs semi-supervised training based on an autoencoder to initially screen for anomalies. Then, it performs precise classification by calculating the similarity between the test signal and the two feature libraries: if it matches the normal feature library, it is judged as normal, and no alarm is triggered; if it matches the abnormal feature library, it is judged as a known anomaly, and an alarm is triggered based on the anomaly type determined by the matching result; if neither matches and the reconstruction error significantly deviates from the normal range, it is judged as an unknown anomaly, an alarm is triggered, and manual judgment is prompted. Finally, the system introduces a dynamic optimization mechanism: samples from manual feedback are used to expand or correct the feature libraries, and newly labeled data is used for periodic incremental model training, thereby achieving adaptive evolution and continuous improvement of diagnostic capabilities.

[0038] According to an embodiment of the present invention, a method for detecting abnormal sound in a device is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0039] This embodiment provides a method for detecting abnormal device sound, which can be used in the aforementioned device sound anomaly detection system. Figure 2 This is a flowchart of a device sound anomaly detection method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: Mix the acquired device operating sound with the ambient noise according to different signal-to-noise ratios to construct an enhanced dataset.

[0040] In this embodiment, the equipment includes a belt conveyor. Taking the belt conveyor as an example, the equipment operating sound refers to the mechanical vibration sound generated when the belt conveyor is running, and the environmental noise refers to the background interference sound present in the belt conveyor operating scene.

[0041] Different signal-to-noise ratios refer to using various signal-to-noise energy ratios when mixing equipment operating sounds with environmental noise to simulate various real-world operating conditions ranging from low to high noise.

[0042] Augmented datasets refer to a collection of synthetic audio samples generated by mixing the original device operating sound with environmental noise at various signal-to-noise ratios and then performing standardized preprocessing. These samples are used to expand the diversity of training data.

[0043] Specifically, an array of sound sensors deployed on-site continuously collects the operating audio of the conveyor belt, as well as environmental samples from various working environments, and transmits this data to an edge computing device or a central server. The collected clean equipment operating sound is then mixed with background noise at preset signal-to-noise ratios to generate a series of synthetic audio samples simulating real-world working conditions. The mixed audio undergoes standardization preprocessing, including segmentation and zero-padding, to obtain an enhanced dataset.

[0044] Step S202: Train an autoencoder model based on the augmented dataset to obtain an audio detection model.

[0045] Among them, the autoencoder model is an unsupervised learning neural network that learns normal data patterns by compressing the input data into a low-dimensional code and then reconstructing the output. In anomaly detection, after training with only normal samples, the reconstruction error of abnormal inputs is large, thus it is used to identify anomalies.

[0046] An audio detection model refers to an autoencoder model trained on an augmented dataset, used to calculate the reconstruction error of the audio to be detected, and its associated anomaly detection threshold. Its core function is to output the reconstruction error as the basis for subsequent anomaly judgment.

[0047] Specifically, the autoencoder model is trained using enhanced feature sequences (the audio signals in the enhanced dataset are converted into log-Mel spectra, which are then used as standard feature inputs to characterize the time-frequency properties of sound). The model is semi-supervised training using a mixture of normal and environmental samples, the reconstruction error of the input features is calculated, and a reasonable anomaly detection threshold is set based on the training results for subsequent use. Once training is complete, the audio detection model is obtained.

[0048] Step S203: Obtain the spectral features of the audio to be detected, and input the spectral features into the audio detection model to calculate the reconstruction error.

[0049] The audio to be detected refers to the real-time sound of the belt conveyor.

[0050] The spectral features are log-Mel spectral features. Reconstruction error refers to the degree of difference between the reconstructed result output by the model and the original input after the spectral features of the audio to be detected are input into the autoencoder model; the larger the error value, the greater the deviation of the audio to be detected from the normal sound pattern learned by the model.

[0051] Specifically, after the spectral features of the audio to be detected are input into the trained autoencoder model, the model first compresses the input into a low-dimensional code through the encoder, and then reconstructs the corresponding spectral features through the decoder; the degree of difference between the reconstruction result and the original input is calculated, which is the reconstruction error. This error quantifies the degree of deviation between the audio to be detected and the normal sound pattern learned by the model.

[0052] Step S204: Based on the reconstruction error and the matching results of the spectral features and the preset audio feature library, identify whether the audio to be detected is abnormal audio and classify the abnormal audio.

[0053] Specifically, this step involves abnormal audio classification based on an audio detection model and a pre-set audio feature library, including: Based on the reconstruction error and the feature library matching results, the system first determines whether the spectral features of the audio to be detected match the normal feature library. If they match, the audio is judged as normal and no alarm is triggered. If they do not match, the system continues to determine whether they match the abnormal feature library. If they match, the audio is judged as known abnormal and an alarm is triggered. If they still do not match, the system further determines whether the reconstruction error exceeds a preset threshold. If it does, the audio is judged as unknown abnormal and a manual judgment is requested. Otherwise, the audio is judged as normal and no alarm is triggered.

[0054] The device sound anomaly detection method provided in this embodiment improves the model's generalization ability in complex noise environments by constructing an enhanced dataset by mixing device operating sounds with environmental noise at different signal-to-noise ratios. By using a two-stage diagnostic strategy that combines autoencoder reconstruction error with matching of a preset audio feature library, it can effectively identify known anomaly types and detect unknown anomalies, thereby solving the technical problems of low recognition rate and difficulty in identifying unknown anomalies in existing methods under noisy environments.

[0055] This embodiment provides a method for detecting abnormal device sound, which can be used in the aforementioned device sound anomaly detection system. Figure 3 This is a flowchart of a device sound anomaly detection method according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps: Step S301: Mix the acquired device operating sound with the ambient noise according to different signal-to-noise ratios to construct an enhanced dataset.

[0056] Specifically, step S301 includes: Step S3011: The acquired device operating sound and environmental noise are used as audio samples and mixed according to different preset signal-to-noise ratios to obtain synthesized audio samples.

[0057] Specifically, the collected clean equipment operating sound and the ambient noise are used as raw audio samples. They are linearly superimposed and mixed according to a variety of preset signal-to-noise ratios (i.e., the energy ratio of signal to noise) to generate a series of synthetic audio samples covering different noise intensities, thereby simulating the operating sound scenarios of the equipment under various actual working conditions such as low noise, medium noise, and high noise.

[0058] Step S3012: After standard preprocessing of the synthesized audio samples, the augmented dataset is obtained.

[0059] The generated synthetic audio samples undergo uniform and standardized preprocessing, including audio segmentation (dividing long audio into short segments of fixed length) and zero-padding alignment (ensuring that all samples have the same length), to ensure that all samples have a uniform format and standardized quality, ultimately forming an augmented dataset that can be used for model training.

[0060] Step S302: Train an autoencoder model based on the augmented dataset to obtain an audio detection model.

[0061] Specifically, step S302 includes: Step S3021: Extract spectral features of audio samples from the augmented dataset and divide the spectral features into training set and validation set; the spectral features include normal sample features extracted from normal audio samples and abnormal sample features extracted from abnormal audio samples.

[0062] Specifically, spectral analysis is performed on each audio sample in the augmented dataset to extract the log-Mel spectrum as a feature vector representing the time-frequency characteristics of the sound, resulting in a complete feature set containing features of normal and abnormal audio samples. Subsequently, the feature set is randomly divided into a training set and a validation set according to a preset ratio to ensure that both sets contain features of normal and abnormal samples, providing a data foundation for subsequent model training and threshold determination.

[0063] Step S3022: Input the features of normal samples in the training set into the autoencoder model for iterative training, learn the compression and reconstruction of normal sound patterns, and obtain the trained autoencoder model.

[0064] Specifically, all normal sample features are selected from the training set and input into the autoencoder network for iterative training. During the training process, the model compresses the input into a low-dimensional code through the encoder and then reconstructs the original input through the decoder. By minimizing the reconstruction loss function, the model learns the efficient compression representation and accurate reconstruction capability of normal sound patterns, and finally obtains an autoencoder model that can accurately reconstruct normal sounds.

[0065] Step S3023: Calculate the reconstruction error distribution of the validation set based on the trained autoencoder model, and determine the anomaly detection threshold based on the reconstruction error distribution.

[0066] Specifically, the features of all samples in the validation set are input into the trained autoencoder model, and the reconstruction error of each sample (i.e., the degree of difference between the reconstruction result and the original input) is calculated to obtain the set of reconstruction errors of the validation set. The distribution characteristics of this set (such as mean, variance or percentile) are statistically analyzed. According to the preset confidence level or false alarm rate requirement, a certain quantile in the distribution is selected as the anomaly discrimination threshold. This threshold is used to distinguish normal and abnormal audio in subsequent detection.

[0067] Step S3024: The trained autoencoder model and the anomaly detection threshold are encapsulated into an audio detection model.

[0068] Specifically, the trained autoencoder network structure is integrated with the anomaly detection threshold determined by the validation set to form a complete audio detection model. This model has a dual function: it outputs the reconstruction error for the input spectral features and provides a preset error threshold as an anomaly judgment benchmark, providing a standardized anomaly detection tool for the subsequent real-time detection stage.

[0069] Step S303: Obtain the spectral features of the audio to be detected, and input the spectral features into the audio detection model to calculate the reconstruction error. For details, please refer to [link to relevant documentation]. Figure 2 Step S203 of the illustrated embodiment will not be described again here.

[0070] Step S304: Based on the reconstruction error and the matching results between the spectral features and the preset audio feature library, identify whether the audio to be detected is abnormal audio, and classify the abnormal audio. For details, please refer to [link to relevant documentation]. Figure 2 Step S204 of the illustrated embodiment will not be described again here.

[0071] The device sound anomaly detection method provided in this embodiment uses the features of normal samples in the augmented dataset to train an autoencoder, enabling it to focus on learning normal sound patterns and overcoming the problem of scarce abnormal samples. The method adaptively determines the discrimination threshold by reconstructing the error distribution through the validation set, avoiding the subjectivity of manually setting the threshold. Finally, the trained model and the threshold are encapsulated to form a complete audio detection model, providing a reliable foundation for subsequent anomaly recognition.

[0072] This embodiment provides a method for detecting abnormal device sound, which can be used in the aforementioned device sound anomaly detection system. Figure 4 This is a flowchart of a device sound anomaly detection method according to an embodiment of the present invention, such as... Figure 4 As shown, the process includes the following steps: Step S401 involves mixing the acquired device operating sound with ambient noise according to different signal-to-noise ratios to construct an enhanced dataset. For details, please refer to [link to relevant documentation]. Figure 3 Step S301 of the illustrated embodiment will not be described again here.

[0073] Step S402: Train the autoencoder model based on the augmented dataset to obtain the audio detection model. For details, please refer to [link to relevant documentation]. Figure 3 Step S302 of the illustrated embodiment will not be described again here.

[0074] Step S403: Obtain the spectral features of the audio to be detected, and input the spectral features into the audio detection model to calculate the reconstruction error. For details, please refer to [link to relevant documentation]. Figure 3Step S303 of the illustrated embodiment will not be described again here.

[0075] Step S404: Based on the reconstruction error and the matching results of the spectral features and the preset audio feature library, identify whether the audio to be detected is abnormal audio, and classify the abnormal audio.

[0076] In one alternative implementation, the preset audio feature library is obtained in the following manner: A normal feature library is constructed based on the features of normal samples, and an abnormal feature library is constructed based on the features of abnormal samples; the normal feature library and the abnormal feature library are used as a preset audio feature library.

[0077] Specifically, all normal sample features are selected from the spectral features extracted from the enhanced dataset, and the feature vector of each normal sample is stored in a structured manner according to a unified format to form a normal feature library. This feature library serves as a benchmark set of sound patterns under the device's health state and is used for similarity comparison with the audio to be tested in subsequent detection.

[0078] All anomalous sample features are selected from the spectral features extracted from the augmented dataset. The feature vector of each anomalous sample is associated with its corresponding anomalous type label and stored to form an anomalous feature library. This feature library serves as a set of sound patterns for known fault types and is used to identify specific anomalous categories in subsequent detection.

[0079] The completed normal feature library and abnormal feature library are integrated together to form a preset audio feature library. This feature library contains feature data of both normal and abnormal sound patterns, providing a complete comparison benchmark for similarity matching in the subsequent two-stage diagnosis.

[0080] Step S404 above includes: Step S4041: Perform similarity matching between the spectral features of the audio to be detected and the normal feature library and the abnormal feature library, respectively.

[0081] Specifically, the cosine similarity between the spectral feature vector of the audio to be detected and each feature vector in the normal feature library is calculated to obtain a set of similarity scores; Calculate the cosine similarity between the spectral feature vector of the audio to be detected and each feature vector in the abnormal feature database to obtain a set of similarity scores.

[0082] Step S4042: Based on the matching results and whether the reconstruction error exceeds the anomaly detection threshold, the category of the audio to be detected is determined comprehensively.

[0083] In an optional implementation, step S4042 includes: Step a1: If the spectral features of the audio to be detected match those of the normal feature library, then it is determined to be normal audio.

[0084] Specifically, if the highest similarity between the spectral feature vector and the normal feature library exceeds a preset similarity threshold (e.g., 0.8), then the feature is determined to match the normal feature library, indicating that the audio to be detected conforms to the normal sound pattern of a healthy device.

[0085] Step a2: If the spectral features of the audio to be detected match the abnormal feature database, it is determined to be a known abnormal audio.

[0086] Specifically, if the highest similarity between the spectral feature vector and the abnormal feature library exceeds a preset threshold (e.g., 0.8), then the feature is determined to match the abnormal feature library, and the specific abnormal category corresponding to the audio to be detected is identified based on the type label associated with the matched abnormal feature.

[0087] Step a3: If the spectral features of the audio to be detected do not match the normal feature library and the abnormal feature library, and the reconstruction error exceeds the abnormality discrimination threshold, then it is determined to be an unknown abnormal audio.

[0088] It should be noted that if the spectral features of the audio to be detected do not match the normal feature library and the abnormal feature library, and the reconstruction error does not exceed the abnormality discrimination threshold, then it is determined to be normal audio.

[0089] In step a3, the spectral features of the audio to be detected are compared with the normal feature library and the abnormal feature library respectively. If no matching item exceeding the preset similarity threshold appears in either feature library, it is confirmed that the audio feature does not belong to the known normal pattern library or the known abnormal pattern library, thus completing the first layer of screening.

[0090] When the audio to be detected simultaneously meets the conditions of not matching any feature library and the reconstruction error exceeds the anomaly discrimination threshold, the system comprehensively determines that the audio is an unknown abnormal audio, that is, it is neither a healthy state nor a registered fault type, indicating that there may be a new, unlearned abnormal pattern.

[0091] Step S405: Based on the judgment result, the following graded alarm triggering is performed: if it is a known abnormal audio, trigger the corresponding abnormal type prompt alarm; if it is an unknown abnormal audio, trigger the preset high-priority alarm and prompt manual review.

[0092] Specifically, perform the corresponding operation based on the audio classification result of step S404: For "known abnormal audio", the system automatically triggers the corresponding preset alarm that indicates the suspected fault type.

[0093] For “unknown abnormal audio”, the system triggers a high-priority alarm and clearly indicates that “manual intervention diagnosis” is required.

[0094] Maintenance personnel respond to alarms, conduct on-site or remote diagnostics, and perform final confirmation and manual annotation of system alarm results.

[0095] Step S406: Add the feature vectors of the audio samples confirmed by manual review to the corresponding normal feature library or abnormal feature library; periodically use the audio samples confirmed by manual review to incrementally train the autoencoder model and update the parameters of the autoencoder model.

[0096] Specifically, the system archives samples and their labels that have been confirmed by the system and by humans. The feature vectors of new samples are added to the corresponding normal or abnormal feature libraries, enabling online expansion of the knowledge base. Simultaneously, when the newly labeled data accumulates to a certain scale, the system uses the newly added labeled samples to incrementally train the autoencoder model, fine-tuning the model parameters to adapt to slow changes in device status and environment, thus completing a full "perception-decision-learning" closed loop.

[0097] The device sound anomaly detection method provided in this embodiment can accurately identify normal audio, known abnormal audio, and unknown abnormal audio through clear matching and error judgment rules, and achieve refined classification of abnormal audio. In particular, its ability to identify unknown anomalies effectively covers sudden fault types that traditional methods cannot detect, thereby improving the comprehensiveness and adaptability of the detection system.

[0098] As one or more specific application embodiments of the present invention, combined with Figure 5 The method for detecting abnormal sound in a device provided by the present invention will be further described in detail, such as... Figure 5 As shown, the specific steps include: S501, Data Acquisition and Upload: Continuously collects the operating audio of the belt conveyor system and environmental noise samples from various operating backgrounds using a sensor array deployed on-site, and transmits them to edge computing devices or a central server.

[0099] S502, Data Augmentation and Extraction: The collected clean equipment operating sound is mixed with background noise at different preset signal-to-noise ratios to generate a series of synthetic audio samples simulating real-world operating conditions. The mixed audio undergoes standardization preprocessing, including segmentation and zero-padding. Subsequently, the audio signal is converted into log-Melogram spectral characteristics, which serve as the standard feature input characterizing the time-frequency properties of the sound.

[0100] S503, Establish a feature library: A normal feature library is built using known features of normal samples, and an abnormal feature library is built using known features of abnormal samples. The feature library consists of feature vectors derived from preprocessed audio samples after conversion to Mel-spectrum analysis.

[0101] S504, Model Training: Spectral features of audio samples are extracted from the augmented dataset and divided into training and validation sets. These spectral features include normal sample features extracted from normal audio samples and abnormal sample features extracted from abnormal audio samples. An autoencoder model is trained using the normal sample features from the training set. The model undergoes semi-supervised training using a mixture of normal sample features and environmental samples. The reconstruction error of the input features is calculated, and a reasonable threshold is set based on the training results for subsequent use.

[0102] S505, Abnormal Audio Classification Based on Model and Audio Feature Library: Based on the reconstruction error and the matching result of the feature library, firstly determine whether the spectral features of the audio to be detected match the normal feature library. If they match, it is judged as normal audio and no alarm is triggered. If they do not match, continue to determine whether they match the abnormal feature library. If they match, it is judged as known abnormal audio and an alarm is triggered. If they still do not match, further determine whether the reconstruction error exceeds a preset threshold. If it exceeds the threshold, it is judged as unknown abnormal audio and manual judgment is requested. Otherwise, it is judged as normal audio and no alarm is triggered.

[0103] S506, graded alarm trigger: Perform the corresponding operation based on the audio classification result of step S505: For "known abnormal audio", the system automatically triggers the corresponding preset alarm that indicates the suspected fault type.

[0104] For “unknown abnormal audio”, the system triggers a high-priority alarm and clearly indicates that “manual intervention diagnosis” is required.

[0105] Maintenance personnel respond to alarms, conduct on-site or remote diagnostics, and perform final confirmation and manual annotation of system alarm results.

[0106] S507, Dynamic Optimization: The system archives abnormal audio samples and their labels, both confirmed by the system and manually. The feature vectors of new samples are added to the corresponding normal or abnormal feature libraries, enabling online expansion of the knowledge base. Simultaneously, when newly labeled data (i.e., audio samples confirmed after manual review) accumulates to a certain scale, the system uses these newly labeled samples to incrementally train the autoencoder model, fine-tuning its parameters to adapt to slow changes in device status and environment, thus completing a full "perception-decision-learning" closed loop.

[0107] The device sound anomaly detection method provided in this embodiment has the following beneficial effects: 1. Integration of data augmentation and two-stage diagnosis: The signal-to-noise ratio hybrid technology is used to simulate complex working conditions for data augmentation. Combined with the two-stage process of initial screening by autoencoder and fine screening by feature library, it takes into account the dual needs of discovering unknown anomalies and accurately identifying known faults, thus significantly improving the robustness of the model.

[0108] 2. Establish an evolutionary, dynamic dual-feature library: A dynamically updated library of normal and abnormal feature vectors was constructed, and interpretable matching and comparison were performed using cosine similarity. This mechanism not only clearly distinguishes abnormal types but also enables the system's knowledge to evolve autonomously by continuously incorporating new samples.

[0109] 3. Forming a closed-loop self-optimizing system paradigm: By integrating a dynamic optimization module, the results of manual review are automatically converted into training data, driving the periodic collaborative updates of the feature library and diagnostic model, thereby achieving autonomous and continuous improvement of the system's diagnostic capabilities and reducing long-term maintenance costs.

[0110] This embodiment also provides a device for detecting abnormal device sound, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0111] This embodiment provides a device for detecting abnormal device sound, such as... Figure 6 As shown, it includes: The augmented dataset construction module 601 is used to mix the acquired device operating sound with environmental noise according to different signal-to-noise ratios to construct an augmented dataset.

[0112] Model training module 602 is used to train an autoencoder model based on an augmented dataset to obtain an audio detection model.

[0113] The reconstruction error calculation module 603 is used to obtain the spectral features of the audio to be detected, and input the spectral features into the audio detection model to calculate the reconstruction error.

[0114] The audio anomaly detection module 604 is used to identify whether the audio to be detected is abnormal based on the reconstruction error and the matching results of the spectral features with the preset audio feature library, and to classify the abnormal audio.

[0115] In some alternative implementations, the augmented dataset construction module 601 includes: Different signal-to-noise ratio mixing units are used to take the acquired device operating sound and environmental noise as audio samples and mix them according to preset different signal-to-noise ratios to obtain synthesized audio samples.

[0116] The standard preprocessing unit is used to perform standard preprocessing on the synthesized audio samples to obtain the augmented dataset.

[0117] In some optional implementations, the augmented dataset includes normal audio samples and abnormal audio samples, and the model training module 602 includes: The spectral feature extraction unit is used to extract spectral features of audio samples from the augmented dataset and divide the spectral features into training and validation sets. The spectral features include normal sample features extracted from normal audio samples and abnormal sample features extracted from abnormal audio samples.

[0118] The model training unit is used to input the features of normal samples in the training set into the autoencoder model for iterative training, learn the compression and reconstruction of normal sound patterns, and obtain a trained autoencoder model.

[0119] The reconstruction error calculation unit is used to calculate the reconstruction error distribution of the validation set based on the trained autoencoder model, and to determine the anomaly detection threshold based on the reconstruction error distribution.

[0120] The audio detection model building unit is used to encapsulate the trained autoencoder model and the anomaly detection threshold into an audio detection model.

[0121] In one alternative implementation, the preset audio feature library is obtained in the following manner: A normal feature library is constructed based on the features of normal samples, and an abnormal feature library is constructed based on the features of abnormal samples. Use the normal feature library and the abnormal feature library as the preset audio feature library.

[0122] In one optional implementation, the audio anomaly detection module 604 includes: The similarity matching unit is used to perform similarity matching between the spectral features of the audio to be detected and the normal feature library and the abnormal feature library, respectively.

[0123] The audio category determination unit is used to comprehensively determine the category of the audio to be detected based on the matching results and whether the reconstruction error exceeds the anomaly detection threshold.

[0124] In one optional implementation, the audio category determination unit includes: The normal audio judgment subunit is used to determine that if the spectral features of the audio to be detected match the normal feature library, it is normal audio.

[0125] The known abnormal audio judgment subunit is used to determine that if the spectral characteristics of the audio to be detected match the abnormal feature library, it is a known abnormal audio.

[0126] The unknown abnormal audio judgment subunit is used to determine that if the spectral features of the audio to be detected do not match the normal feature library and the abnormal feature library, and the reconstruction error exceeds the abnormal discrimination threshold, then the audio is considered to be unknown abnormal audio.

[0127] In one alternative embodiment, the device further includes: The graded alarm triggering module is used to execute the following graded alarm triggering based on the judgment result: if it is a known abnormal audio, trigger the corresponding abnormal type prompt alarm; if it is an unknown abnormal audio, trigger the preset high-priority alarm and prompt manual review.

[0128] In one alternative embodiment, the device further includes: The dynamic optimization module is used to add the feature vectors of audio samples that have been manually reviewed and confirmed to the corresponding normal feature library or abnormal feature library; and periodically use the audio samples that have been manually reviewed and confirmed to incrementally train the autoencoder model and update the parameters of the autoencoder model.

[0129] The device sound anomaly detection apparatus provided in this embodiment of the invention can execute the device sound anomaly detection method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as in the corresponding embodiments described above, and will not be repeated here.

[0130] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0131] The following is a detailed reference. Figure 7 This diagram illustrates a suitable structural schematic for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 701, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 702 or a program loaded from memory 708 into random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device. The processor 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0132] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 7 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0133] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 709, or installed from a memory 708, or installed from a ROM 702. When the computer program is executed by the processor 701, it performs the functions defined in the device sound anomaly detection method of the embodiments of the present invention.

[0134] Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0135] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the device sound anomaly detection method shown in the above embodiments is implemented.

[0136] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0137] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for detecting abnormal sound in equipment, characterized in that, The method includes: The acquired device operating sounds and environmental noise are mixed according to different signal-to-noise ratios to construct an enhanced dataset; An autoencoder model is trained based on the augmented dataset to obtain an audio detection model; The spectral features of the audio to be detected are obtained, and the spectral features are input into the audio detection model to calculate the reconstruction error. Based on the reconstruction error and the matching results between the spectral features and the preset audio feature library, it is determined whether the audio to be detected is abnormal audio, and the abnormal audio is classified.

2. The method according to claim 1, characterized in that, The process of mixing the acquired device operating sound with environmental noise at different signal-to-noise ratios to construct an enhanced dataset includes: The acquired device operating sound and environmental noise are used as audio samples, and then mixed according to different preset signal-to-noise ratios to obtain synthetic audio samples. After standard preprocessing, the synthesized audio samples are used to obtain the augmented dataset.

3. The method according to claim 2, characterized in that, The augmented dataset includes normal audio samples and abnormal audio samples. An autoencoder model is trained based on the augmented dataset to obtain an audio detection model, including: Spectral features of audio samples are extracted from the augmented dataset, and the spectral features are divided into training and validation sets; the spectral features include normal sample features extracted from normal audio samples and abnormal sample features extracted from abnormal audio samples. The normal sample features in the training set are input into the autoencoder model for iterative training to learn the compression and reconstruction of normal sound patterns, and a trained autoencoder model is obtained. The reconstruction error distribution of the validation set is calculated based on the trained autoencoder model, and the anomaly detection threshold is determined based on the reconstruction error distribution. The trained autoencoder model and the anomaly detection threshold are encapsulated into an audio detection model.

4. The method according to claim 3, characterized in that, The preset audio feature library is obtained through the following method: A normal feature library is constructed based on the normal sample features, and an abnormal feature library is constructed based on the abnormal sample features; The normal feature library and the abnormal feature library are used as preset audio feature libraries.

5. The method according to claim 4, characterized in that, Based on the reconstruction error and the matching results between the spectral features and the preset audio feature library, the system identifies whether the audio to be detected is abnormal and classifies abnormal audio, including: The spectral features of the audio to be detected are matched with the normal feature library and the abnormal feature library for similarity. The category of the audio to be detected is determined by comprehensively considering the matching results and whether the reconstruction error exceeds the anomaly detection threshold.

6. The method according to claim 5, characterized in that, Based on the matching results and whether the reconstruction error exceeds the anomaly detection threshold, the category of the audio to be detected is comprehensively determined, including: If the spectral features of the audio to be detected match the normal feature library, it is determined to be normal audio. If the spectral characteristics of the audio to be detected match the abnormal feature database, it is determined to be a known abnormal audio. If the spectral features of the audio to be detected do not match the normal feature library and the abnormal feature library, and the reconstruction error exceeds the anomaly discrimination threshold, then it is determined to be an unknown abnormal audio.

7. The method according to claim 6, characterized in that, The method further includes: Based on the judgment result, the following tiered alarm triggers will be executed: If the audio is a known abnormal audio, trigger the corresponding abnormality type alert. If the audio is an unknown abnormality, a preset high-priority alarm will be triggered, and a manual review will be requested.

8. The method according to claim 7, characterized in that, The method further includes: Add the feature vectors of the audio samples that have been manually reviewed and confirmed to the corresponding normal feature library or abnormal feature library. The autoencoder model is incrementally trained periodically using audio samples that have been manually reviewed and confirmed, and the parameters of the autoencoder model are updated accordingly.

9. A device for detecting abnormal sound in equipment, characterized in that, The device includes: The augmented dataset building module is used to mix the acquired device operating sounds with environmental noise according to different signal-to-noise ratios to build an augmented dataset. The model training module is used to train an autoencoder model based on the augmented dataset to obtain an audio detection model; The reconstruction error calculation module is used to obtain the spectral features of the audio to be detected, and input the spectral features into the audio detection model to calculate the reconstruction error; The audio anomaly detection module is used to identify whether the audio to be detected is abnormal based on the reconstruction error and the matching result between the spectral features and the preset audio feature library, and to classify the abnormal audio.

10. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the device sound anomaly detection method according to any one of claims 1 to 8.

Images