Acoustic detection model training method and device, industrial equipment detection method and device and electronic equipment
By using multiple acoustic models in the acoustic detection model and combining modules such as ConvBank and Conv Block, audio data are classified and trained according to weather, location and time, the problem of low detection accuracy in the existing technology is solved, and industrialization is achieved High accuracy detection and adaptability of equipment operating status are improved.
Patent Information
- Application Number
- CN202411138459.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2024-11-26
AI Technical Summary
The existing acoustic detection model has low detection accuracy for industrial equipment operating status, making it difficult to accurately distinguish between normal and abnormal operating status.
By obtaining the normal and abnormal audio data generated by the industrial equipment to be detected in the area to be detected, audio data is classified according to weather, location and time, multiple acoustic models are trained to adapt to different environmental conditions, using ConvBank, Conv Block, and adaptive averaging The pooling layer and Dense Block module perform feature extraction and deep learning to improve the adaptability and accuracy of the model.
Improve the accuracy of the detection of industrial equipment operation status by the acoustic detection model, enhance the targetedness and adaptability of the model, maintain high detection accuracy in complex and changeable industrial environments, and has flexibility and scalability. .
Smart Images

Figure CN119028326A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of industrial equipment, for example, to a training method for an acoustic detection model, a detection method, device and electronic device for industrial equipment. Background Art
[0002] Industrial equipment refers to various equipment used in the industrial production process, such as lathes, grinders, and belt conveyors. Some industrial scenarios often have multiple industrial equipment or large-scale industrial equipment with a wide coverage, such as blast furnaces, rolling mills, and casting machines in automobile manufacturing plants, and tubular belt conveyors that pass through multiple production plants in the industrial raw material transportation system. For such industrial scenarios, staff are required to frequently inspect various locations to check the operating status of industrial equipment and ensure the normal operation of industrial equipment.
[0003] Since there are differences in the sounds of industrial equipment under normal operation and abnormal operation, a method for detecting abnormalities of industrial equipment through sound is proposed in the related art, including: installing a sound collection device near the equipment to collect the sound of equipment operation; using a feature extractor to extract sound feature options selected according to the parameter function configuration, and calculating the eigenvalues of the time domain, frequency domain and cepstrum domain of the audio data for training or discrimination; saving the sound feature data generated by the feature extractor through a feature database; using a model trainer to train the training database in the feature database according to the model algorithm and parameters selected by the parameter function configuration, and saving the trained model to the model library; using a model determiner to load the model library data to construct a model, and when the current operating state is a discrimination state, determining whether the current sound (determined according to the sound feature items after the feature extractor) is normal; presenting the prediction results of the model to the user, and then collecting the user's feedback data, and then using it to train the model to form a closed-loop system.
[0004] In the process of implementing the above embodiments, it is found that although the relevant technology can reduce labor intensity and improve inspection efficiency, the trained model has low detection accuracy for the operating status of industrial equipment. Summary of the invention
[0005] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical elements or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.
[0006] The embodiments of the present disclosure provide a training method for an acoustic detection model, a detection method, device and electronic device for industrial equipment, which can improve the detection accuracy of the acoustic detection model on the operating status of industrial equipment.
[0007] In some embodiments, a training method for an acoustic detection model is provided, wherein the acoustic detection model includes multiple first acoustic models; the training method includes: obtaining a first training audio set; the first training audio set is an audio data set including normal audio generated by the industrial equipment to be detected during normal operation and abnormal audio generated during abnormal operation in the area to be detected; the audio data in the first training audio set is classified according to the marking information of the audio data to obtain multiple training subsets; the marking information includes weather, location and time; and the multiple first acoustic models are trained respectively using the multiple training subsets to obtain a trained acoustic detection model.
[0008] Optionally, the first acoustic model includes: a ConvBank module, used to extract features from input audio data and output first feature data; a Conv Block module, used to perform deep learning on the input first feature data and output second feature data; an adaptive average pooling layer, used to adjust the feature size of the input second feature data and generate third feature data; and a Dense Block module, used to perform feature reuse and classification on the input third feature data to achieve audio data recognition.
[0009] Optionally, the first training audio set is obtained in the following manner: obtain real-time audio of the industrial equipment to be inspected during operation; input the real-time audio into the trained second acoustic model to obtain the audio type; mark and save the real-time audio to obtain the first training audio set; the marking information includes weather, location, time and audio type, and the audio type includes normal audio or abnormal audio.
[0010] Optionally, when the audio type is abnormal audio, the marking information also includes the abnormal type of industrial equipment; marking the real-time audio includes: inputting the abnormal audio into a trained third acoustic model, and identifying the abnormal type of the abnormal audio; if the identification is successful, obtaining the abnormal type of the industrial equipment and marking the real-time audio with the abnormal type; if the identification fails, performing time domain and frequency domain analysis on the abnormal audio, obtaining and feeding back analysis data; receiving and responding to the abnormal type of the industrial equipment based on the analysis data, and marking the real-time audio with the abnormal type.
[0011] Optionally, the third acoustic model is trained in the following manner: real-time audio is input into a pre-built third acoustic model for recognition to obtain recognition results; time domain and frequency domain analysis is performed on the real-time audio to obtain and feed back analysis data; analysis results based on the analysis data are received, and the third acoustic model is trained based on the recognition results and the analysis results.
[0012] Optionally, the third acoustic model is trained according to the recognition results and the analysis results, including: comparing the recognition results and the analysis results; when the recognition results and the analysis results are different, marking the real-time audio according to the analysis results; and using the marked real-time audio to train the third acoustic model.
[0013] Optionally, the third acoustic model is constructed in the following manner: the audio data of the obtained second training audio set is preprocessed; the second training audio set is an audio data set including normal audio generated by the same industrial equipment as the industrial equipment to be detected during normal operation and abnormal audio generated during abnormal operation; the preprocessing includes mean filtering, median filtering and adaptive filtering; the preprocessed audio data is used to train the Gaussian mixture model and the hidden Markov model in sequence; the trained Gaussian mixture model and the hidden Markov model are fused to obtain the constructed third acoustic model.
[0014] Optionally, the second acoustic model is trained in the following manner: the obtained real-time audio is recognized; when the real-time audio is normal audio, the real-time audio is marked and saved to obtain a normal audio training set; and the normal audio training set is used to train the second acoustic model.
[0015] Optionally, the audio data in the first training audio set is classified according to the marking information of the audio data to obtain multiple training subsets, including: classifying the audio data in the first training audio set once according to weather to obtain multiple first training sets; classifying the audio data in the multiple first training sets twice according to location to obtain multiple second training sets; and classifying the audio data in the multiple second training sets three times according to time to obtain multiple training subsets.
[0016] Optionally, multiple training subsets are used to train multiple first acoustic models respectively, including: extracting features from the audio data in each training subset to obtain Mel-cepstral coefficients corresponding to each audio data; inputting the Mel-cepstral coefficients corresponding to each audio data in each training subset into the corresponding first acoustic model to train the first acoustic model.
[0017] In some embodiments, a method for detecting industrial equipment is provided, including: acquiring environmental data and real-time audio of the industrial equipment to be detected during operation; the environmental data includes weather, location and time; determining a target first acoustic model in an acoustic detection model based on the environmental data; wherein the acoustic detection model is trained using the acoustic detection model training method described in the above embodiment; adjusting the target first acoustic model to an activated state, and adjusting other first acoustic models of non-target first acoustic models to a deactivated state; inputting the real-time audio into the adjusted acoustic detection model to obtain a detection result.
[0018] In some embodiments, a training device for an acoustic detection model is provided, including: an acquisition module, configured to acquire a first training audio set; the first training audio set is an audio data set including normal audio generated during normal operation and abnormal audio generated during abnormal operation of the industrial equipment to be detected in the detection area; a classification module, configured to classify the audio data in the first training audio set according to the labeling information of the audio data to obtain multiple training subsets; the labeling information includes weather, location and time; a training module, configured to use multiple training subsets to train multiple first acoustic models respectively to obtain trained acoustic detection models.
[0019] In some embodiments, a detection device for industrial equipment is provided, including: an acquisition module, configured to obtain environmental data and real-time audio of the industrial equipment to be detected during operation; the environmental data includes weather, location and time; a determination module, configured to determine the target first acoustic model in the acoustic detection model according to the environmental data; wherein the acoustic detection model is trained using the training method of the acoustic detection model as described in the above embodiment; an adjustment module, configured to adjust the target first acoustic model to an activated state, and adjust other first acoustic models of the non-target first acoustic model to a deactivated state; and a detection module, configured to input the real-time audio into the adjusted acoustic detection model to obtain a detection result.
[0020] In some embodiments, an electronic device is provided, comprising: a processor; and a memory storing program instructions, wherein the processor is configured to execute the training method of the acoustic detection model as described in the above embodiments, or the detection method of industrial equipment as described in the above embodiments.
[0021] The acoustic detection model training method, industrial equipment detection method, device and electronic device provided in the embodiments of the present disclosure can achieve the following technical effects: The training method of the acoustic detection model provided in the embodiment of the present disclosure uses the audio generated by the actual operation of the industrial equipment to be detected in the area to be detected, that is, the first training audio set, to train the acoustic detection model. The trained acoustic detection model is more adaptable to the industrial equipment to be detected, and the detection accuracy of the acoustic detection model for the operating status of the industrial equipment to be detected is improved. Before using the training, the audio data in the first training audio set is first classified according to the label information of the audio data (that is, weather, location and time) to consider the impact of environmental factors on the acoustic characteristics, so that each training subset can accurately reflect the acoustic characteristics of the industrial equipment to be detected in the area to be detected under a specific environment. In addition, each first acoustic model is optimized for its corresponding specific environment, so as to better capture and identify the acoustic characteristics in the specific environment, improve the pertinence of the acoustic detection model, and further improve the accuracy of the acoustic detection model.
[0022] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein: Figure 1 is a schematic diagram of an electronic device provided by an embodiment of the present disclosure; Figure 2 is a schematic diagram of a training method for an acoustic detection model provided by an embodiment of the present disclosure; Figure 3 is a schematic diagram of the structure of a first acoustic model provided by an embodiment of the present disclosure; Figure 4 is a schematic diagram of a Conv block layer provided by an embodiment of the present disclosure; Figure 5 is a schematic diagram of a Dense block layer provided by an embodiment of the present disclosure; Figure 6 is a schematic diagram of a training method for an acoustic detection model provided in another embodiment of the present disclosure; Figure 7 is a schematic diagram of a detection method for industrial equipment provided by an embodiment of the present disclosure; Figure 8 is a schematic diagram of a training device for an acoustic detection model provided by an embodiment of the present disclosure; Fig. 9 is a schematic diagram of a detection device for industrial equipment provided by an embodiment of the present disclosure; Fig.10is a schematic diagram of a controller provided by an embodiment of the present disclosure.
[0024] Reference numerals: 1. Electronic equipment; 10. Equipment body; 20. Controller; 200. Processor; 201. Memory; 202. Communication interface; 203. Bus; 80. Acoustic detection model training device; 810. Acquisition module; 820. Classification module; 830. Training module; 90. Detection device for industrial equipment; 910. Collection module; 920. Determination module; 930. Adjustment module; 940. Detection module. DETAILED DESCRIPTION
[0025] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.
[0026] The terms "first", "second", etc. in the specification and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.
[0027] Unless otherwise stated, the term "plurality" means two or more.
[0028] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.
[0029] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A, B, A and B.
[0030] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.
[0031] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.
[0032] Combination Figure 1 As shown, the embodiment of the present disclosure provides an electronic device 1, comprising: a device body 10 and a controller 20. The controller 20 is installed in the device body 10.
[0033] In the disclosed embodiment, the controller 20 is installed in the device body 10. The installation relationship described here is not limited to being placed inside the device body 10, but also includes installation connections with other components of the electronic device 1, including but not limited to physical connections, electrical connections, or signal transmission connections. It can be understood by those skilled in the art that the controller 20 can be adapted to a feasible electronic device 1 to achieve other feasible embodiments. In some embodiments, the electronic device 1 includes a robot or a computer, etc., and the specific type of the electronic device 1 is not limited in this application.
[0034] Optionally, combined Fig.10 As shown, the controller 20 includes a processor 200. The processor 200 can obtain a first training audio set; classify the audio data in the first training audio set according to the tag information of the audio data to obtain multiple training subsets; and use the multiple training subsets to train multiple first acoustic models respectively to obtain trained acoustic detection models.
[0035] Combination Figure 1 and Fig.10 As shown, the present disclosure provides a method for training an acoustic detection model, such as Figure 2 As shown, the acoustic detection model includes multiple first acoustic models. Specifically, the training method includes: S201: The processor obtains a first training audio set.
[0036] The first training audio set is an audio data set including normal audio generated when the to-be-detected industrial equipment is operating normally in the to-be-detected area and abnormal audio generated when the to-be-detected industrial equipment is operating abnormally.
[0037] S202: The processor classifies the audio data in the first training audio set according to the label information of the audio data to obtain multiple training subsets.
[0038] The labeled information includes weather, location, and time. Each training subset includes normal audio and abnormal audio.
[0039] In this embodiment, it is limited that each training subset includes normal audio and abnormal audio, so that each first acoustic model can learn the acoustic characteristics of the industrial equipment to be detected during normal operation and abnormal operation, thereby realizing the detection of two operating states, normal operation and abnormal operation of the industrial equipment to be detected.
[0040] Optionally, the audio data in the first training audio set is classified according to the marking information of the audio data to obtain multiple training subsets, including: classifying the audio data in the first training audio set once according to weather to obtain multiple first training sets; classifying the audio data in the multiple first training sets twice according to location to obtain multiple second training sets; and classifying the audio data in the multiple second training sets three times according to time to obtain multiple training subsets.
[0041] In this embodiment, the first training audio set is first classified according to weather conditions, such as sunny days, rainy days, snowy days, etc., to obtain multiple first training sets. Through one classification, the influence of different weather on acoustic features can be distinguished. Weather changes will change the propagation mode and characteristics of sound and cause changes in background noise. Then each first training set is classified according to location, such as the starting point and end point of the tubular belt conveyor (such as a railway station or a port), the passing point (such as a production plant), etc., to obtain multiple second training sets. Through secondary classification, the influence of spatial factors on acoustic features can be distinguished. Different locations will produce different acoustic effects due to environmental noise. Each second training set is further classified according to time, such as 7 to 9 in the morning, 9 to 11 in the morning, 11 to 13 at noon, 17 to 19 in the evening, and night, etc. Through three classifications, the influence of time factors on acoustic features can be distinguished. The environmental noise level and equipment operation status in different time periods will be different.
[0042] In this embodiment, by subdividing the first training audio set into multiple more specific training subsets, each subset corresponds to specific weather, location and time conditions, so that each first acoustic model trained by the training subset can be more focused on the acoustic features under its specific conditions, thereby more accurately capturing and identifying the acoustic features under specific environmental conditions, and improving the detection accuracy, reliability and robustness of the acoustic detection model. And when new environmental conditions or detection requirements arise, new training subsets can be added and new first acoustic models can be trained without reclassifying the first training audio set and retraining the acoustic detection model, thereby ensuring the flexibility and scalability of the acoustic detection model.
[0043] S203: The processor uses multiple training subsets to train multiple first acoustic models respectively to obtain trained acoustic detection models.
[0044] In the disclosed embodiment, the acoustic detection model is used to identify the input audio data and output the detection result, and the detection result includes normal operation or abnormal operation. The training method of the acoustic detection model provided in the disclosed embodiment can collect the normal audio generated by the industrial equipment to be detected in the area to be detected during normal operation and the abnormal audio generated during abnormal operation as the first training audio set for training, so that the acoustic detection model trained by the first training audio set can learn the acoustic characteristics of the industrial equipment to be detected during normal operation and abnormal operation, and then realize the operation status detection of the industrial equipment to be detected through the trained acoustic detection model. Since the audio data in the first training audio set is the audio generated by the actual operation of the industrial equipment to be detected in the area to be detected, the acoustic detection model trained by the first training audio set is more adaptable to the industrial equipment to be detected in the area to be detected, and customized training is realized. Through customized training, the accuracy of the acoustic detection model in detecting the operation status of the industrial equipment to be detected is improved.
[0045] In the disclosed embodiment, the acoustic detection model includes multiple first acoustic models. Before using the first training audio set to train the acoustic detection model, the audio data in the first training audio set is first classified according to the labeling information of the audio data (i.e., weather, location, and time) to generate multiple training subsets to consider the impact of environmental factors on acoustic features, so that each training subset can more accurately reflect the acoustic characteristics of the industrial equipment to be detected operating in the area to be detected under a specific environment. Then, multiple training subsets are used to train multiple first acoustic models respectively, so that each first acoustic model is optimized for its corresponding specific environment, so as to better capture and identify the acoustic features in the specific environment, thereby improving the pertinence of the acoustic detection model and further improving the accuracy of the acoustic detection model.
[0046] In addition, by classifying the audio data in the first training audio set according to environmental factors such as weather, location and time, and training different first acoustic models respectively, the adaptability of the acoustic detection model under different environmental conditions is improved, so that the acoustic detection model can maintain a high detection accuracy in complex and changeable industrial environments. Even if a first acoustic model performs poorly in a specific environment, other first acoustic models can still provide accurate detection results, thereby ensuring the overall performance of the acoustic detection model. And when new environmental conditions arise, the acoustic detection model can be updated by adding a new training subset and training a new first acoustic model without retraining the entire acoustic detection model, which improves the flexibility and scalability of the acoustic detection model and enables the acoustic detection model to quickly adapt to new industrial environments and detection needs.
[0047] Optionally, multiple training subsets are used to train multiple first acoustic models respectively, including: extracting features from the audio data in each training subset to obtain Mel-cepstral coefficients corresponding to each audio data; inputting the Mel-cepstral coefficients corresponding to each audio data in each training subset into the corresponding first acoustic model to train the first acoustic model.
[0048] Mel-Frequency Cepstral Coefficients (MFCC) are the coefficients that make up the Mel-Frequency Cepstrum, which is derived from the cepstrum of an audio clip. The Mel-Frequency Cepstrum is a linear transformation of the logarithmic power spectrum based on the nonlinear mel scale of the sound frequencies.
[0049] MFCC uses the Mel scale, which is consistent with the nonlinear frequency perception of the human ear. The frequency bands divided equally on the Mel scale can better approximate the human auditory system, which enables MFCC to more accurately reflect the way the human ear perceives audio signals, thereby providing a feature representation that is more in line with human auditory characteristics in audio classification and recognition, and more accurately capturing the detailed features of audio signals. MFCC has a certain degree of robustness to noise and channel changes, and can maintain good recognition performance in different environments, making MFCC more reliable and stable in audio classification and recognition tasks.
[0050] In this embodiment, the Mel-cepstral coefficients are input into the first acoustic model as the feature representation of each audio data, so that the first acoustic model can learn the features of each audio data by taking advantage of the Mel-cepstral coefficients, thereby improving the accuracy, reliability and stability of the first acoustic model in recognizing the audio data. In this embodiment, the Mel-cepstral coefficients corresponding to each audio data in each training subset are a set, and the Mel-cepstral coefficient set corresponding to each training subset is obtained, and then each Mel-cepstral coefficient set is respectively input into the first acoustic model corresponding to its training subset to train the first acoustic model.
[0051] In a practical application, the steps of extracting features from the audio data in each training subset and obtaining the Mel cepstral coefficients corresponding to each audio data include: preprocessing: including pre-emphasis, framing, windowing, etc., to remove noise, highlight high-frequency signals, and make the audio signal more suitable for subsequent processing; Fourier transform (FFT): transform the time domain signal to the frequency domain to obtain the spectrum of the audio signal; Mel filter bank: use a group of triangular bandpass filters equidistantly distributed on the Mel scale to filter the spectrum, and the frequency range covered by each filter is approximately a critical bandwidth of the human ear; logarithmic operation: calculate the logarithmic energy of the output of each filter to simulate the nonlinear perception of the human ear to the sound intensity; discrete cosine transform (DCT): perform DCT transform on the logarithmic energy to remove the correlation between the audio signals of each dimension, and reduce the dimension to obtain the final MFCC coefficients.
[0052] Optionally, combined Figure 3 As shown, the first acoustic model includes: a ConvBank module, a Conv Block module, an adaptive average pooling layer and a Dense Block module. The ConvBank module is used to extract features from the input audio data and output first feature data. The Conv Block module is used to perform deep learning on the input first feature data and output second feature data. The adaptive average pooling layer (such as Figure 3 The one-dimensional adaptive average pooling layer in the image processing module, Adaptive-AvgPool-1d, is used to adjust the feature size of the input second feature data to generate the third feature data. The Dense Block module is used to reuse and classify the input third feature data to realize audio data recognition.
[0053] In this embodiment, the ConvBank module is used to capture the long-term information of the input audio data to achieve feature extraction. The Conv Block module achieves deep learning by further extracting and transforming the first feature data. The adaptive average pooling layer can perform adaptive average pooling on the second feature data to generate third feature data of a fixed size so as to be compatible with the subsequent Dense Block module. The Dense Block module enhances feature reuse and gradient flow through dense connections (the core idea of DenseNet), improves the learning ability and efficiency of the first acoustic model, and achieves audio data recognition.
[0054] In this embodiment, the first acoustic model mainly combines the characteristics of convolutional neural networks (CNN) and densely connected networks (DenseNet), and realizes feature extraction and classification tasks through multiple levels and modules, forming a powerful network for processing one-dimensional sequence data (i.e., audio data). The first acoustic model not only utilizes the feature extraction capability of convolutional neural networks, but also combines the advantages of densely connected networks, improving its performance in processing complex sequence data, thereby improving the accuracy of its audio data recognition and classification. When using the first acoustic model to detect the operating status of the industrial equipment to be detected, the accuracy is higher.
[0055] Optionally, there are multiple Conv block modules. By increasing the number of Conv block modules, the learning ability and efficiency of the first acoustic model are further improved, and the recognition accuracy and reliability of the first acoustic model are improved.
[0056] Optionally, there are multiple Dense block modules. By increasing the number of Dense block modules, the learning ability and efficiency of the first acoustic model are further improved, and the recognition accuracy and reliability of the first acoustic model are improved.
[0057] Exemplarily, the number of Conv block modules is 3, and the number of Dense block modules is 3.
[0058] Optionally, combined Figure 3 As shown, the ConvBank module includes multiple one-dimensional convolutional layers. In this embodiment, the one-dimensional convolutional layer (Conv1d) can perform convolution operations along one axis of the input data (such as the sequence axis) to capture the local dependencies of the input data, thereby effectively capturing the local features in the one-dimensional data (such as the audio signal) and converting it into a higher-level feature representation. In this embodiment, the input data (i.e., audio data) is subjected to local feature extraction through multiple Conv1ds respectively, and the local features output by each Conv1d are concatenated (Concat) to capture the long-term information of the input audio data, and the concatenated feature data is output as the first feature data.
[0059] Optionally, combined Figure 3 As shown, the Conv block module includes a Conv block layer and a one-dimensional average pooling layer. The Conv block layer is used to perform deep learning on the input feature data. The one-dimensional average pooling layer (AvgPool-1d) is used to reduce the dimension of the feature data. In this embodiment, the feature extraction and conversion of the input feature data are realized through the cooperation of the Conv block layer and the one-dimensional average pooling layer, so as to realize deep learning.
[0060] Optionally, combined Figure 3 As shown, the number of Conv block layers is multiple. In this embodiment, the learning ability and efficiency of the Conv block module are improved by increasing the number of Convblock layers, thereby improving the learning ability and efficiency of the first acoustic model. Exemplarily, the number of Conv block layers is 4.
[0061] Optionally, as shown in combination 4, the Conv block layer includes a one-dimensional convolution layer. Conv1d is followed by a Relu activation function and Dropout, and a residual connection is made between the input of Conv1d and the output of Dropout. In this embodiment, learning of input feature data is achieved through Conv1d and Relu activation functions. Dropout is used to prevent overfitting. A residual connection is made between the input of Conv1d and the output of Dropout to alleviate the vanishing gradient in the deep network.
[0062] Optionally, combined Figure 3 As shown, the Dense block module includes a Dense block layer. The Dense block module enhances feature reuse through the Dense block layer to achieve audio data classification, thereby achieving audio data recognition.
[0063] Optionally, combined Figure 3 As shown, the number of Dense block layers is multiple. By increasing the number of Dense block layers, the feature reuse strength and classification accuracy of the Dense block module are enhanced, thereby improving the learning ability and recognition accuracy of the first acoustic model.
[0064] Optionally, combined Figure 5 As shown, the Dense block layer includes a linear layer. The linear layer (Linear) is followed by a Relu activation function and Dropout, and the input of the Linear is connected to the output residual of the Dropout. In this embodiment, the Linear of the last Denseblock layer is used to convert the features extracted by the previous layer into the final output to achieve the classification and recognition of audio data. The Linear of the non-last Dense block layer is used to further convert and combine features to improve the accuracy of subsequent classification.
[0065] Combination Figure 6 As shown, the present disclosure provides another method for training an acoustic detection model, including: S601: The processor obtains real-time audio of the industrial equipment to be detected during operation.
[0066] S602: The processor inputs the real-time audio into the trained second acoustic model to obtain the audio type.
[0067] S603: The processor marks and saves the real-time audio to obtain a first training audio set.
[0068] The tag information includes weather, location, time and audio type, and the audio type includes normal audio or abnormal audio. The first training audio set is an audio data set including normal audio generated when the industrial equipment to be detected is in the area to be detected and when it is operating normally and abnormal audio generated when it is operating abnormally.
[0069] S604: The processor classifies the audio data in the first training audio set according to the tag information of the audio data to obtain multiple training subsets.
[0070] In this embodiment, the marking information includes weather, location and time. Each training subset includes normal audio and abnormal audio. It can be understood that the marking information of the audio data includes information such as weather, location, time and audio type, but in the process of classifying audio data, classification is not performed according to audio type, and it is necessary to ensure that each training subset includes normal audio and abnormal audio.
[0071] S605: The processor uses multiple training subsets to train multiple first acoustic models respectively to obtain trained acoustic detection models.
[0072] The training method of the acoustic detection model provided in the embodiment of the present disclosure can automatically collect real-time audio of the industrial equipment to be detected during operation, and use the trained second acoustic model to mark the real-time audio, thereby automatically constructing a first training audio set, reducing the need for manual intervention, improving the efficiency and accuracy of data labeling, reducing labor intensity, and speeding up the construction of the data set.
[0073] Specifically, by installing a sound collection device such as a microphone or a microphone array in the area to be detected, the real-time audio of the industrial equipment to be detected during operation can be automatically collected in real time to achieve real-time audio acquisition, and the installation location of the sound collection device can be used as the location information for subsequent marking of the real-time audio. The time and weather information when the real-time audio is collected can be obtained by connecting to the Internet. The second acoustic model is used to identify and classify the input audio data and output the audio type, that is, normal audio or abnormal audio. By inputting the obtained real-time audio into the trained second acoustic model, the audio type information is acquired, and then the real-time audio is marked and saved with weather, location, time and audio type, so as to obtain the first training audio set.
[0074] In addition, by labeling the audio data with environmental information such as weather, location, and time, the richness and diversity of the first training audio set are improved, so that the acoustic detection model trained using the first training audio set can learn more comprehensive and detailed sound features, thereby improving its recognition ability and adaptability in different environments.
[0075] In some embodiments, when the audio type is abnormal audio, the marking information also includes the abnormal type of the industrial equipment. Marking the real-time audio includes: inputting the abnormal audio into the trained third acoustic model, identifying the abnormal type of the abnormal audio; in the case of successful identification, obtaining the abnormal type of the industrial equipment and marking the real-time audio as abnormal; in the case of failed identification, performing time domain and frequency domain analysis on the abnormal audio, obtaining and feeding back analysis data; receiving and responding to the abnormal type of the industrial equipment based on the analysis data, and marking the real-time audio as abnormal.
[0076] In this embodiment, the third acoustic model is used to identify and classify the input abnormal audio, and output the abnormal type of the industrial equipment, that is, the cause of the fault when the industrial equipment is operating abnormally, such as roller damage, local belt tearing, belt deflection, etc. of the tubular belt conveyor. By first inputting the abnormal audio into the pre-trained third acoustic model for abnormal type identification, when the identification is successful, the abnormal type of the industrial equipment is automatically acquired and then marked, further improving the richness and diversity of the first training audio set, so that the acoustic detection model trained using the first training audio set can identify the specific abnormal type, that is, the cause of the abnormal operation of the industrial equipment, and improve the functionality of the acoustic detection model. Using the trained acoustic model for abnormal identification further reduces the need for manual intervention, improves the efficiency and accuracy of data labeling, reduces labor intensity, and speeds up the construction of the first training audio set.
[0077] In this embodiment, in the case of recognition failure, the abnormal type of industrial equipment can be determined by performing time domain and frequency domain analysis on the abnormal audio, and the abnormal type can be obtained to make up for the shortcomings of the third acoustic model and ensure the reliability of the recognition of the abnormal type of industrial equipment. In this embodiment, after obtaining the analysis data, the analysis data can be fed back to the monitoring center where the technician is located and the technician is prompted to analyze the analysis data to determine the abnormal type of industrial equipment. After the technician determines the specific abnormal type based on the actual industrial equipment, the abnormal type of the industrial equipment can be fed back through voice input or manual input, so that the processor receives and responds to the abnormal type of the industrial equipment based on the analysis data, and marks the real-time audio with the abnormal type.
[0078] In some embodiments, the analysis data includes a time domain graph, a decibel graph, a frequency domain graph, and a spectrum graph of the abnormal audio. The time domain and frequency domain analysis of the abnormal audio includes: using audio processing software or a programming language (such as MATLAB, Python, etc.) to read the abnormal audio and obtain the sampling data and sampling rate of the abnormal audio; according to the sampling data and the sampling rate, drawing a time domain waveform graph of the abnormal audio to obtain a time domain graph; wherein the horizontal axis represents time and the vertical axis represents amplitude.
[0079] Calculate the power or energy of each sampling point or each data segment of the abnormal audio; use the decibel calculation formula to convert the power or energy value into a decibel value; draw and obtain a decibel graph based on the converted decibel value and time information; where the horizontal axis represents time and the vertical axis represents decibel value. For example, the decibel calculation formula is dB=20×log 10 (P / P0), where dB represents decibel, P represents the power of the audio signal, and P0 represents the reference power. The specific value of P0 is set by the technician according to the actual industrial equipment to be detected and the area to be detected, and is not limited in this application. For example, the specific value of P0 is 1 milliwatt, 2 milliwatts, or 3 milliwatts.
[0080] Perform Fourier transform on the abnormal audio to convert the time domain signal into a frequency domain signal; draw a frequency domain graph and a spectrum graph according to the result of Fourier transform to obtain a frequency domain graph and a spectrum graph; wherein the horizontal axis of the frequency domain graph represents frequency and the vertical axis represents amplitude; the horizontal axis of the spectrum graph represents time and the vertical axis represents frequency.
[0081] Optionally, the third acoustic model is trained in the following manner: real-time audio is input into a pre-built third acoustic model for recognition to obtain recognition results; time domain and frequency domain analysis is performed on the real-time audio to obtain and feed back analysis data; analysis results based on the analysis data are received, and the third acoustic model is trained based on the recognition results and the analysis results.
[0082] In this embodiment, a third acoustic model is pre-constructed to identify the input real-time audio and obtain the identification result, that is, the abnormal type of the industrial equipment. When the third acoustic model is initially constructed, it has low adaptability to the industrial equipment to be detected in the area to be detected. At this time, the real-time audio is analyzed in the time domain and frequency domain to obtain analysis data and the third acoustic model is trained based on the analysis results of the analysis data to further optimize the third acoustic model and improve the accuracy of the third acoustic model recognition. In this embodiment, the real-time audio is analyzed in the time domain and frequency domain to obtain and feedback analysis data; the specific process of receiving the analysis results based on the analysis data, that is, the abnormal type of the industrial equipment, refers to the above embodiment and will not be repeated here.
[0083] Optionally, the third acoustic model is trained according to the recognition results and the analysis results, including: comparing the recognition results and the analysis results; when the recognition results and the analysis results are different, marking the real-time audio according to the analysis results; and using the marked real-time audio to train the third acoustic model.
[0084] In this embodiment, the recognition result of the third acoustic model is compared with the analysis result to verify the recognition result of the third acoustic model. When the recognition result is different from the analysis result, it is confirmed that the third acoustic model predicts an error, and the real-time audio is correctly marked according to the analysis result, and the abnormal type of the actual industrial equipment is marked to create a new training sample with the correct mark. Then, the real-time audio with the new mark (i.e., the training sample with the correct mark) is used to retrain the third acoustic model, and the third acoustic model learns the association between the new audio features and the abnormal type, thereby gradually improving its recognition accuracy.
[0085] Optionally, the third acoustic model is constructed in the following manner: the audio data of the obtained second training audio set is preprocessed; the second training audio set is an audio data set including normal audio generated by the same industrial equipment as the industrial equipment to be detected during normal operation and abnormal audio generated during abnormal operation; the preprocessing includes mean filtering, median filtering and adaptive filtering; the preprocessed audio data is used to train the Gaussian mixture model and the hidden Markov model in sequence; the trained Gaussian mixture model and the hidden Markov model are fused to obtain the constructed third acoustic model.
[0086] Gaussian Mixture Model (GMM) is a clustering algorithm that assumes that all data points are generated by a mixture (i.e., superposition) of a finite number of Gaussian distributions. As a probabilistic model, GMM is used to represent the probability distribution of data, even if the data comes from multiple different sources or sub-distributions. In audio recognition, each acoustic unit (such as a phoneme or syllable) can be considered as a mixture of multiple Gaussian distributions, each of which represents the sound performance under different audio conditions (such as different industrial equipment, different operating states of industrial equipment, or different environmental noise). GMM can give a certain speech frame X t The probability P(X) of belonging to a certain acoustic state St t ∣S t ), which is calculated by calculating the frame (X t ) is achieved by taking the weighted sum of all possible Gaussian distributions.
[0087] Hidden Markov Model (HMM) is a probabilistic model of time series. It describes a process in which a hidden Markov chain randomly generates a random sequence of unobservable states, and then each state generates an observation to generate a random sequence of observations. HMM is used to describe a Markov process with hidden states. In audio recognition, these hidden states correspond to different acoustic units (such as phonemes), and the observed data is the speech signal itself. HMM assumes that the transition between states depends only on the previous state, and each state will produce an observable output according to a probability distribution. This output is the speech frame, and the transition probability P(S t+1 =j|S t =i) and the observation probability P(X t ∣S t =i) together define the entire HMM.
[0088] In this embodiment, the audio data set of normal audio generated during normal operation and abnormal audio generated during abnormal operation of the same industrial equipment as the industrial equipment to be detected can be collected by connecting to the Internet as the second training audio set for preliminary training of the subsequent model. The audio data in the second training audio set is then preprocessed to eliminate noise, improve the signal-to-noise ratio, and extract useful audio features. Among them, the audio signal is smoothed by mean filtering to reduce random noise. Pulse noise and salt and pepper noise are removed by median filtering. Adaptive filtering is used to dynamically adjust the filter parameters according to the characteristics of the audio signal to better suppress noise and interference. Preprocessing is performed to improve the accuracy and efficiency of subsequent model training.
[0089] By inputting the preprocessed audio data into the Gaussian mixture model and the hidden Markov model in turn for training (by first inputting a certain audio data into the GMM, obtaining the probability that the audio data belongs to a certain abnormal type, and then using the obtained probability as the observation probability of the HMM to perform HMM), the connection between the GMM and the HMM is improved. The trained GMM and HMM are fused to obtain a constructed third acoustic model, namely the GMM-HMM model. GMM and HMM fusion can adopt a variety of methods, such as weighted fusion, cascade fusion or fusion based on Bayesian network, etc. The specific fusion method adopted by GMM and HMM is not limited in this application. The fused GMM-HMM model, combining the advantages of GMM and HMM, can not only capture the static characteristics of the audio signal, but also process its temporal changes, thereby improving the accuracy and robustness of abnormal type detection.
[0090] Optionally, the second acoustic model is trained in the following manner: the obtained real-time audio is recognized; when the real-time audio is normal audio, the real-time audio is marked and saved to obtain a normal audio training set; and the normal audio training set is used to train the second acoustic model.
[0091] In this embodiment, the obtained real-time audio is recognized, and normal audio is screened out from the real-time audio to construct a training set, so as to ensure that the training data is highly relevant to the actual application scenario, improve the pertinence and adaptability of the second acoustic model in the application scenario, and improve the accuracy and reliability of the second acoustic model. By using the normal audio training set to train the second acoustic model, the second acoustic model can learn the characteristics of normal audio so that similar normal audio can be accurately identified. In actual applications, when the real-time audio does not match the prediction result of the second acoustic model, that is, when the second acoustic model fails to recognize, the real-time audio is abnormal audio, and the classification of normal audio and abnormal audio is achieved. The embodiment of the present disclosure improves the pertinence and accuracy of the second acoustic model in identifying normal audio by training the second acoustic model to learn only the characteristics of normal audio, thereby improving the classification accuracy of the second acoustic model, that is, the recognition accuracy of normal audio and abnormal audio.
[0092] In some embodiments, the obtained real-time audio is identified, including: inputting the real-time audio into a trained third acoustic model, and performing audio type identification on the real-time audio; in the case of successful identification, the audio type is obtained; in the case of failed identification, time domain and frequency domain analysis is performed on the real-time audio, and analysis data is obtained and fed back; receiving and responding to the audio type based on the analysis data, and audio type tagging is performed on the real-time audio. In this embodiment, audio type analysis is performed in combination with the trained third acoustic model and the time domain and frequency domain analysis of the real-time audio, so as to improve the efficiency and accuracy of audio type identification, reduce labor intensity, and ensure the reliability of audio type identification.
[0093] In some embodiments, the second acoustic model comprises a GMM-HMM model.
[0094] Combination Figure 1 and Fig.10 The electronic device 1 shown in the figure, the embodiment of the present disclosure provides a detection method for industrial equipment, such as Figure 7 As shown, the detection method includes: S701, the processor obtains environmental data and real-time audio of the industrial equipment to be detected during operation.
[0095] Environmental data includes weather, location, and time.
[0096] S702: The processor determines a target first acoustic model in the acoustic detection model according to the environmental data.
[0097] The acoustic detection model is obtained by training using the acoustic detection model training method described in the above embodiment.
[0098] In this embodiment, the target first acoustic model in the acoustic detection model is determined based on the environmental data, including: obtaining labeling information of a training subset used to train the first acoustic model; and taking the first acoustic model corresponding to the same labeling information as the environmental data as the target first acoustic model.
[0099] S703: The processor adjusts the target first acoustic model to be in an activated state, and adjusts other first acoustic models that are not the target first acoustic models to be in a deactivated state.
[0100] S704, the processor inputs the real-time audio into the adjusted acoustic detection model to obtain a detection result.
[0101] The detection method for industrial equipment provided by the embodiments of the present disclosure can use the acoustic detection model training method as described in the above embodiments to train an acoustic detection model to perform detection of the industrial equipment to be detected, so as to obtain a detection result, that is, to determine whether the industrial equipment to be detected is in normal operation or abnormal operation. Therefore, the technology possessed by the acoustic detection model in the acoustic detection model training method described in the above embodiments is also possessed by this application and will not be repeated here. In this embodiment, the operating status of the industrial equipment to be detected can be detected by using the trained acoustic detection model, which reduces the labor intensity while improving the detection efficiency and accuracy.
[0102] Optionally, the detection method further includes: performing time domain and frequency domain analysis on the real-time audio to obtain and feed back analysis data.
[0103] In this embodiment, while using the acoustic detection model to detect the operating status of the industrial equipment to be detected, the real-time audio is analyzed in the time domain and frequency domain to obtain and feedback analysis data, so that the technician can trace the operating status of the industrial equipment to be detected, and after the acoustic detection model has been running for a period of time, the technician can use the analysis data to evaluate and optimize the acoustic detection model.
[0104] With reference to the acoustic detection model training method described in the above embodiment, the present disclosure provides another industrial equipment detection method, which includes multiple stages, such as a first stage, a second stage, a third stage, and a fourth stage.
[0105] Specifically, the first stage: a second training audio set; preprocessing the audio data of the obtained second training audio set; using the preprocessed audio data to train the Gaussian mixture model and the hidden Markov model in turn; fusing the trained Gaussian mixture model and the hidden Markov model to obtain a constructed third acoustic model; deploying the constructed third acoustic model to the area to be detected; obtaining real-time audio of the industrial equipment to be detected during operation; inputting the real-time audio into the pre-constructed third acoustic model for recognition to obtain recognition results; performing time domain and frequency domain analysis on the real-time audio to obtain and feed back analysis data; receiving analysis results based on the analysis data, and training the third acoustic model according to the recognition results and the analysis results to obtain a trained third acoustic model.
[0106] In the first stage, the recognition accuracy and reliability of the third acoustic model are improved by performing time domain and frequency domain analysis on real-time audio.
[0107] The second stage: input the real-time audio into the trained third acoustic model, perform audio type recognition on the real-time audio, and when the audio type is abnormal audio, perform abnormal type recognition on the abnormal audio; if the recognition is successful, obtain the audio type, and if the audio type is abnormal audio, obtain the abnormal type of the industrial equipment; if the recognition fails, perform time domain and frequency domain analysis on the real-time audio, obtain and feedback the analysis data; receive the audio type or the audio type and the abnormal type based on the analysis data; mark the real-time audio according to the obtained audio type or the audio type and the abnormal type; obtain a normal audio training set and accumulate audio data for the first training audio set.
[0108] In the second stage, the need for manual intervention is reduced, and the trained third acoustic model is used to detect the operating status of the industrial equipment to be detected. When the third acoustic model fails to recognize, the real-time audio is analyzed in the time domain and frequency domain to ensure the completeness and reliability of the operating status detection of the industrial equipment to be detected. At the same time, a normal audio training set and audio data for the first training audio set are accumulated.
[0109] The third stage: use the normal audio training set to train the second acoustic model to obtain the trained second acoustic model; deploy the trained second acoustic model to the area to be detected; input the real-time audio into the trained second acoustic model to obtain the audio type; when the audio type is abnormal audio, input the abnormal audio into the trained third acoustic model to identify the abnormal type of the abnormal audio; if the recognition is successful, obtain the abnormal type of the industrial equipment and mark the real-time audio as abnormal; if the recognition fails, perform time domain and frequency domain analysis on the abnormal audio to obtain and feedback the analysis data; receive and respond to the abnormal type of the industrial equipment based on the analysis data, and mark the real-time audio as abnormal; obtain the first training audio set.
[0110] In the third stage, the trained second acoustic model avoids interference from external audio data (only real-time audio of the industrial equipment to be detected in the area to be detected is used for training), and focuses on learning the characteristics of normal audio, so the accuracy in audio type recognition is higher. The second acoustic model is used to first screen the real-time audio, and the abnormal audio is then input into the third acoustic model to identify the abnormal type, and the abnormal audio is analyzed in the time domain and frequency domain when the recognition fails. The first training audio set is obtained while the operating status detection of the industrial equipment to be detected is realized, which further improves the accuracy of the operating status detection and reduces the labor intensity.
[0111] The fourth stage: classify the audio data in the first training audio set according to the labeling information of the audio data to obtain multiple training subsets; use the multiple training subsets to train multiple first acoustic models respectively to obtain a trained acoustic detection model; deploy the trained acoustic detection model to the area to be detected or in a movable electronic device; obtain environmental data; determine the target first acoustic model in the acoustic detection model according to the environmental data; adjust the target first acoustic model to be in an activated state, and adjust other first acoustic models of the non-target first acoustic model to be in a deactivated state; input the real-time audio into the adjusted acoustic detection model to obtain the detection result; perform time domain and frequency domain analysis on the real-time audio to obtain and feedback the analysis data.
[0112] In the fourth stage, the acoustic detection model is trained using the first training audio set, and the trained acoustic detection model is deployed to the area to be detected or to a movable electronic device (such as a robot) to further improve the accuracy of operating status detection and reduce labor intensity.
[0113] In the first stage to the fourth stage, the acoustic model used for the running state detection of the industrial equipment to be detected is gradually replaced to improve the accuracy of the running state detection of the industrial equipment to be detected and reduce the labor intensity. In the first stage to the third stage, the third acoustic model and the second acoustic model used are constructed and trained based on the GMM-HMM model. The GMM-HMM model can be quickly trained with a small amount of training data, so as to deploy the running state detection of the industrial equipment to be detected as soon as possible and reduce the need for manual intervention. The first acoustic model used in the fourth stage is constructed and trained based on the convolutional neural network. The accuracy of the first acoustic model is higher, but it depends on a large amount of training data. The audio data of industrial equipment in abnormal operating state is difficult to obtain, so the audio data collected in the first stage to the third stage is used to realize the training of the first acoustic model, and then the trained acoustic detection model including multiple first acoustic models is deployed, and the real-time audio is detected to improve the accuracy of the running state detection of the industrial equipment to be detected.
[0114] Combination Figure 8 As shown, the embodiment of the present disclosure provides a training device 80 for an acoustic detection model, including an acquisition module 810, a classification module 820 and a training module 830. The acquisition module 810 is configured to acquire a first training audio set; the first training audio set is an audio data set including normal audio generated when the industrial equipment to be detected is in a normal operation in the area to be detected and abnormal audio generated when the industrial equipment to be detected is in an abnormal operation; the classification module 820 is configured to classify the audio data in the first training audio set according to the tag information of the audio data to obtain multiple training subsets; the tag information includes weather, location and time; the training module 830 is configured to use multiple training subsets to train multiple first acoustic models respectively to obtain trained acoustic detection models.
[0115] The acoustic detection model training device 80 provided in the embodiment of the present disclosure can execute the acoustic detection model training method described in the above embodiment. Therefore, the techniques of the acoustic detection model training method described in the above embodiment are also possessed by this application and will not be repeated here.
[0116] Optionally, the acquisition module 810 is also configured to acquire real-time audio of the industrial equipment to be detected during operation; input the real-time audio into the trained second acoustic model to obtain the audio type; mark and save the real-time audio to obtain the first training audio set.
[0117] Optionally, the acquisition module 810 is also configured to input the abnormal audio into a trained third acoustic model, and identify the abnormal type of the abnormal audio; if the identification is successful, obtain the abnormal type of the industrial equipment and mark the real-time audio as abnormal; if the identification fails, perform time domain and frequency domain analysis on the abnormal audio, obtain and feedback analysis data; receive and respond to the abnormal type of the industrial equipment based on the analysis data, and mark the real-time audio as abnormal.
[0118] Optionally, the acquisition module 810 is also configured to input the real-time audio into a pre-built third acoustic model for recognition to obtain a recognition result; perform time domain and frequency domain analysis on the real-time audio to obtain and feedback analysis data; receive analysis results based on the analysis data, and train the third acoustic model based on the recognition result and the analysis result.
[0119] Optionally, the acquisition module 810 is also configured to compare the recognition result and the analysis result; when the recognition result and the analysis result are different, mark the real-time audio according to the analysis result; and use the marked real-time audio to train the third acoustic model.
[0120] Optionally, the acquisition module 810 is also configured to preprocess the audio data of the obtained second training audio set; use the preprocessed audio data to train the Gaussian mixture model and the hidden Markov model in sequence; and fuse the trained Gaussian mixture model and the hidden Markov model to obtain a constructed third acoustic model.
[0121] Optionally, the acquisition module 810 is also configured to recognize the acquired real-time audio; when the real-time audio is normal audio, mark and save the real-time audio to obtain a normal audio training set; and use the normal audio training set to train the second acoustic model.
[0122] Optionally, the classification module 820 is also configured to classify the audio data in the first training audio set once according to the weather to obtain multiple first training sets; classify the audio data in the multiple first training sets twice according to the location to obtain multiple second training sets; and classify the audio data in the multiple second training sets three times according to the time to obtain multiple training subsets.
[0123] Optionally, the training module 830 is also configured to perform feature extraction on the audio data in each training subset to obtain the Mel-cepstral coefficients corresponding to each audio data; input the Mel-cepstral coefficients corresponding to each audio data in each training subset into the corresponding first acoustic model to train the first acoustic model.
[0124] Combination Fig. 9 As shown, the embodiment of the present disclosure provides a detection device 90 for industrial equipment, including a collection module 910, a determination module 920, an adjustment module 930 and a detection module 940. The collection module 910 is configured to obtain environmental data and real-time audio of the industrial equipment to be detected during operation; the environmental data includes weather, location and time; the determination module 920 is configured to determine the target first acoustic model in the acoustic detection model according to the environmental data; wherein the acoustic detection model is trained and obtained by using the training method of the acoustic detection model as described in the above embodiment; the adjustment module 930 is configured to adjust the target first acoustic model to be in an activated state, and adjust other first acoustic models of non-target first acoustic models to be in a deactivated state; the detection module 940 is configured to input the real-time audio into the adjusted acoustic detection model to obtain the detection result.
[0125] The detection device 90 for industrial equipment provided in the embodiment of the present disclosure can execute the detection method for industrial equipment described in the above embodiment. Therefore, the technology possessed by the detection method for industrial equipment described in the above embodiment is also possessed by the present application and will not be described in detail here.
[0126] In some embodiments, in combination Fig.10As shown, the controller 20 includes a processor 200 and a memory 201. Optionally, the controller 20 may also include a communication interface 202 and a bus 203. The processor 200, the communication interface 202, and the memory 201 may communicate with each other through the bus 203. The communication interface 202 may be used for information transmission. The processor 200 may call the logic instructions in the memory 201 to execute the training method of the acoustic detection model in the above-mentioned embodiment, or the detection method of the industrial equipment.
[0127] In addition, the logic instructions in the memory 201 described above may be implemented in the form of software functional units and when sold or used as independent products, may be stored in a computer-readable storage medium.
[0128] The memory 201 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the methods in the embodiments of the present disclosure. The processor 200 executes functional applications and data processing by running the program instructions / modules stored in the memory 201, that is, implementing the training method of the acoustic detection model in the above-mentioned embodiment, or the detection method of industrial equipment.
[0129] The memory 201 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 201 may include a high-speed random access memory and may also include a non-volatile memory.
[0130] An embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute the training method of the acoustic detection model in the above embodiment, or the detection method of industrial equipment.
[0131] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiment of the present disclosure. The aforementioned storage medium may be a non-transient storage medium, such as: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes.
[0132] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible changes. Unless explicitly required, separate components and functions are optional, and the order of operations may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and the claims, unless the context clearly indicates, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of one or more associated listings. In addition, when used in this application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, method or device including the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.
[0133] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods for each specific application to implement the described functions, but such implementations should not be considered to exceed the scope of the embodiments of the present disclosure. The technicians may clearly understand that, for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above may refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.
[0134] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.
[0135] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.
Claims
1. A training method for an acoustic detection model, It is characterized in that The acoustic detection model includes a plurality of first acoustic models; and the training method includes: Acquire a first training audio set; the first training audio set is an audio data set including normal audio generated when the industrial equipment to be detected is operating normally and abnormal audio generated when the industrial equipment to be detected is operating abnormally in the area to be detected; Classifying the audio data in the first training audio set according to the label information of the audio data to obtain multiple training subsets; the label information includes weather, location and time; A plurality of training subsets are used to train a plurality of first acoustic models respectively to obtain trained acoustic detection models.
2. The training method according to claim 1, It is characterized in that The first acoustic model includes: ConvBank module, used for extracting features from input audio data and outputting first feature data; Conv Block module, used for performing deep learning on the input first feature data and outputting second feature data; An adaptive average pooling layer, used to adjust the feature size of the input second feature data to generate third feature data; The Dense Block module is used to perform feature reuse and classification on the input third feature data to realize audio data recognition.
3. The training method according to claim 1 or 2, It is characterized in that Obtain the first training audio set as follows: Obtain real-time audio of the industrial equipment to be tested during operation; Input the real-time audio into the trained second acoustic model to obtain the audio type; The real-time audio is marked and saved to obtain a first training audio set; the marking information includes weather, location, time and audio type, and the audio type includes normal audio or abnormal audio.
4. The training method according to claim 3, It is characterized in that When the audio type is abnormal audio, the tag information also includes the abnormal type of the industrial equipment; Labeling live audio includes: Inputting the abnormal audio into the trained third acoustic model to identify the abnormal type of the abnormal audio; In case of successful identification, the abnormal type of the industrial equipment is obtained and the real-time audio is marked as abnormal; In case of recognition failure, the abnormal audio is analyzed in the time domain and frequency domain to obtain and feed back the analysis data; the abnormal type of the industrial equipment based on the analysis data is received and responded to, and the real-time audio is marked with the abnormal type.
5. The training method according to claim 4, It is characterized in that The third acoustic model is trained as follows: Input the real-time audio into a pre-built third acoustic model for recognition to obtain a recognition result; Perform time domain and frequency domain analysis on real-time audio to obtain and feedback analysis data; An analysis result based on the analysis data is received, and a third acoustic model is trained according to the recognition result and the analysis result.
6. The training method according to claim 5, It is characterized in that The third acoustic model is trained according to the recognition results and the analysis results, including: Compare the recognition results with the analysis results; When the recognition result and the analysis result are different, the real-time audio is marked according to the analysis result; The third acoustic model is trained using the labeled real-time audio.
7. The training method according to claim 5, It is characterized in that The third acoustic model is constructed as follows: Preprocessing the audio data of the obtained second training audio set; the second training audio set is an audio data set including normal audio generated by the same industrial equipment as the industrial equipment to be detected during normal operation and abnormal audio generated during abnormal operation; the preprocessing includes mean filtering, median filtering and adaptive filtering; The preprocessed audio data is used to train the Gaussian mixture model and the hidden Markov model in sequence; The trained Gaussian mixture model and the hidden Markov model are fused to obtain a constructed third acoustic model.
8. The training method according to claim 3, It is characterized in that Train the second acoustic model as follows: Recognize the acquired real-time audio; When the real-time audio is normal audio, the real-time audio is marked and saved to obtain a normal audio training set; The second acoustic model is trained using a normal audio training set.
9. The training method according to claim 1 or 2, It is characterized in that The audio data in the first training audio set is classified according to the label information of the audio data to obtain multiple training subsets, including: Classify the audio data in the first training audio set once according to the weather to obtain multiple first training sets; Perform secondary classification on the audio data in the plurality of first training sets according to the positions to obtain a plurality of second training sets; The audio data in the plurality of second training sets are classified three times according to time to obtain a plurality of training subsets.
10. The training method according to claim 1 or 2, It is characterized in that Using multiple training subsets to train multiple first acoustic models respectively includes: Perform feature extraction on the audio data in each training subset to obtain the Mel-frequency cepstral coefficients corresponding to each audio data; The Mel-cepstral coefficients corresponding to each audio data in each training subset are input into the corresponding first acoustic model to train the first acoustic model.
11. A method for detecting industrial equipment, It is characterized in that include: Obtain environmental data and real-time audio of the industrial equipment to be tested during operation; environmental data includes weather, location and time; Determine a target first acoustic model in the acoustic detection model according to the environmental data; wherein the acoustic detection model is obtained by training using the acoustic detection model training method according to any one of claims 1 to 10; Adjusting the target first acoustic model to be in an activated state, and adjusting other first acoustic models of non-target first acoustic models to be in a deactivated state; Input the real-time audio into the adjusted acoustic detection model to obtain the detection results.
12. A training device for an acoustic detection model, It is characterized in that include: An acquisition module is configured to acquire a first training audio set; the first training audio set is an audio data set including normal audio generated when the industrial equipment to be detected is operating normally and abnormal audio generated when the industrial equipment to be detected is operating abnormally in the area to be detected; A classification module is configured to classify the audio data in the first training audio set according to the label information of the audio data to obtain multiple training subsets; Tag information includes weather, location and time; The training module is configured to use multiple training subsets to train multiple first acoustic models respectively to obtain trained acoustic detection models.
13. A detection device for industrial equipment, It is characterized in that include: A collection module configured to obtain environmental data and real-time audio of the industrial equipment to be detected during operation; Environmental data includes weather, location, and time; A determination module, configured to determine a target first acoustic model in the acoustic detection model according to the environmental data; wherein the acoustic detection model is obtained by training using the acoustic detection model training method according to any one of claims 1 to 10; An adjustment module, configured to adjust the target first acoustic model to be in an activated state, and adjust other first acoustic models of non-target first acoustic models to be in a deactivated state; The detection module is configured to input the real-time audio into the adjusted acoustic detection model to obtain the detection result.
14. An electronic device, It is characterized in that include: processor; and A memory storing program instructions, wherein the processor is configured to execute the training method of the acoustic detection model according to any one of claims 1 to 10, or the detection method of industrial equipment according to claim 11.
