Fault identification method and device of mechanical equipment, storage medium and computer equipment

Abstract

The invention discloses a fault identification method and device for mechanical equipment, a storage medium and computer equipment, relates to the technical field of fault identification, and mainly aims to solve the problems of low accuracy and unstable identification effect of mechanical equipment fault identification in an existing small sample mechanical equipment fault identification scene. Comprising the following steps: acquiring equipment image data and sound data of mechanical equipment in normal and fault states, and preprocessing to obtain to-be-analyzed equipment image data and to-be-analyzed sound data; performing fusion processing on the to-be-analyzed equipment image data and the to-be-analyzed sound data to obtain a fusion feature data set; configuring a random disturbance weight for a fusion feature vector in the fusion feature data set, and training a fault recognition fusion model based on the data set configured with the random disturbance weight; and based on the fault identification fusion model and the equipment image monitoring data and the sound monitoring data of the mechanical equipment, performing fault judgment processing to obtain an operation state of the mechanical equipment.

Description

Technical Field

[0001] The present invention relates to the technical field of fault identification, and in particular to a method and device for identifying a fault of a mechanical device, a storage medium, and a device. Background Art

[0002] With the continuous development of artificial intelligence technology, its application fields are becoming more and more extensive. In mechanical equipment fault identification, artificial intelligence technology has also been widely used. For example, in a substation equipment sound fault detection and positioning method based on deep learning disclosed in patent publication number CN112183647A, convolutional neural network encoding and long short-term memory network detection are used to construct a substation equipment sound fault detection model. The substation equipment sound fault detection model can realize single-stage end-to-end detection and positioning of substation equipment sound faults. However, the existing method of using artificial intelligence technology to identify mechanical equipment faults requires a large number of mechanical equipment fault sound samples for model training and processing. However, in the fault identification scenario of small sample mechanical equipment, the recognition accuracy is often low. In addition, the signal stability of the fault sound sample of the mechanical equipment is poor, and a large amount of noise is mixed in. The existing fault identification technology that only relies on the equipment fault sound signal cannot guarantee the stability of mechanical equipment fault identification even if a large amount of preprocessing of the sound signal is performed before fault identification. Summary of the invention

[0003] In view of this, the present invention provides a mechanical equipment fault identification method and device, a storage medium, and a computer device, the main purpose of which is to solve the problems of low accuracy and unstable identification effect of mechanical equipment fault identification in existing small sample mechanical equipment fault identification scenarios.

[0004] According to one aspect of the present invention, a method for identifying a fault of a mechanical device is provided, comprising: Acquire device image data and sound data of the mechanical device in a normal state and a fault state, wherein the device image data and the sound data are marked with a state label; Preprocessing the device image data to obtain the device image data to be analyzed; and preprocessing the sound data to obtain the sound data to be analyzed; Performing fusion processing on the device image data to be analyzed and the sound data to be analyzed to obtain a fused feature data set; Configuring random disturbance weights for fused feature vectors in the fused feature data set; and training a long short-term memory artificial neural network model based on the fused feature data set after configuring the random disturbance weights to obtain a fault recognition fusion model; The equipment image monitoring data and the sound monitoring data of the mechanical equipment are acquired, and the fault judgment processing is performed based on the fault identification fusion model, the equipment image monitoring data and the sound monitoring data to obtain the operating status of the mechanical equipment.

[0005] Further, the preprocessing of the device image data to obtain the device image data to be analyzed; and the preprocessing of the sound data to obtain the sound data to be analyzed include: Cleaning, enhancing and standardizing the device image data to obtain the device image data to be analyzed; The sound data is filtered, denoised, framed and enhanced to obtain the sound data to be analyzed.

[0006] Furthermore, the enhancing process of the sound data includes: Performing Fourier short-time transform and Mel transform on the sound data after the filtering, the noise reduction and the framing processing to obtain logarithmic Mel spectrum feature data; A data enhancement method combining noise injection and Mixup is used to perform data enhancement processing on the logarithmic Mel spectrum feature data to obtain a sound enhancement data set.

[0007] Furthermore, the data enhancement method using a combination of noise injection and Mixup is used to perform data enhancement processing on the logarithmic Mel spectrum feature data to obtain a sound enhancement data set, including: Randomly obtain logarithmic Mel spectrum feature data x 1 and the corresponding status labels y 1 , and logarithmic Mel spectrum feature data x 2 and the corresponding status labels y 2 ; The Mixup generates sound enhancement data by injecting a convex combination of noise in the input and feature space. , and generate the sound enhancement data Corresponding new status labels , the specific formula is as follows:

[0008]

[0009] in, represents a value randomly sampled from a Beta distribution, represents the added noise term; The sound enhancement data And the corresponding new status label The sound enhancement data set is obtained by adding the logarithmic Mel spectrum feature data.

[0010] Furthermore, the fusing the device image data to be analyzed and the sound data to be analyzed to obtain a fused feature data set includes: Using a convolutional neural network model to perform feature extraction processing on the image data of the device to be analyzed to obtain an image feature vector; and using a convolutional neural network model to perform feature extraction processing on the sound data to be analyzed to obtain an acoustic feature vector; The image feature vector and the acoustic feature vector corresponding to the same sampling time are merged into a fused feature vector to obtain a fused feature data set.

[0011] Furthermore, the long short-term memory artificial neural network model is trained based on the fusion feature data set after configuring the random disturbance weights to obtain a fault recognition fusion model, including: Performing forward propagation on the fused feature vector based on the random perturbation weight to obtain a forward propagation result; The model loss is calculated based on the forward propagation result, and the parameters of the long short-term memory artificial neural network model are optimized according to the model loss to obtain the trained fault identification fusion model.

[0012] Furthermore, configuring random perturbation weights for the fused feature vectors in the fused feature data set includes: A disturbance value is randomly extracted from a normal distribution, and the disturbance value is determined as the random disturbance weight of the fused feature vector ; Determine the iterative value of the currently extracted disturbance value by iteration, and determine the maximum disturbance in the noise distribution based on the iterative value t ; Based on the maximum disturbance t The random perturbation weight of the fused feature vector Update the process and set the maximum disturbance t and the random perturbation weights Combining, we get the following:

[0013] Will and t The random perturbation weight is calculated by comparing the function The matching degree with the normal distribution, when the matching degree requirement is not met, the variance value in the normal distribution is adjusted so that and t Matched to an adjusted normal distribution.

[0014] According to another aspect of the present invention, a fault identification device for mechanical equipment is provided, comprising: A data acquisition module, used to acquire device image data and sound data of the mechanical device in a normal state and a fault state, wherein the device image data and the sound data are marked with a state label; A preprocessing module, used to preprocess the device image data to obtain the device image data to be analyzed; and to preprocess the sound data to obtain the sound data to be analyzed; A data fusion module, used for fusing the device image data to be analyzed and the sound data to be analyzed to obtain a fused feature data set; A training module is used to configure random perturbation weights for fused feature vectors in the fused feature data set; and to train a long short-term memory artificial neural network model based on the fused feature data set after the random perturbation weights are configured to obtain a fault recognition fusion model; The fault judgment module is used to obtain the equipment image monitoring data and sound monitoring data of the mechanical equipment, and perform fault judgment processing based on the fault identification fusion model, the equipment image monitoring data and the sound monitoring data to obtain the operating status of the mechanical equipment.

[0015] Furthermore, the preprocessing module comprises: An image data preprocessing unit, used for cleaning, enhancing and standardizing the device image data to obtain the device image data to be analyzed; The sound data preprocessing unit is used to filter, reduce noise, divide frames and enhance the sound data to obtain the sound data to be analyzed.

[0016] Furthermore, the sound data preprocessing unit is also used for: Performing Fourier short-time transform and Mel transform on the sound data after the filtering, the noise reduction and the framing processing to obtain logarithmic Mel spectrum feature data; A data enhancement method combining noise injection and Mixup is used to perform data enhancement processing on the logarithmic Mel spectrum feature data to obtain a sound enhancement data set.

[0017] Furthermore, the sound data preprocessing unit is also used for: Randomly obtain logarithmic Mel spectrum feature data x 1and the corresponding status labels y 1 , and logarithmic Mel spectrum feature data x 2 and the corresponding status labels y 2 ; The Mixup generates sound enhancement data by injecting a convex combination of noise in the input and feature space. , and generate the sound enhancement data Corresponding new status labels , the specific formula is as follows:

[0018]

[0019] in, represents a value randomly sampled from a Beta distribution, represents the added noise term; The sound enhancement data And the corresponding new status label The sound enhancement data set is obtained by adding the logarithmic Mel spectrum feature data.

[0020] Furthermore, the data fusion module includes: A feature extraction unit, configured to perform feature extraction processing on the image data of the device to be analyzed using a convolutional neural network model to obtain an image feature vector; and perform feature extraction processing on the sound data to be analyzed using a convolutional neural network model to obtain an acoustic feature vector; The feature fusion unit is used to merge the image feature vector and the acoustic feature vector corresponding to the same sampling time into a fused feature vector to obtain a fused feature data set.

[0021] Furthermore, the training module is also used for: Performing forward propagation on the fused feature vector based on the random perturbation weight to obtain a forward propagation result; The model loss is calculated based on the forward propagation result, and the parameters of the long short-term memory artificial neural network model are optimized according to the model loss to obtain the trained fault identification fusion model.

[0022] Furthermore, the training module is also used for: A disturbance value is randomly extracted from a normal distribution, and the disturbance value is determined as the random disturbance weight of the fused feature vector ; Determine the iterative value of the currently extracted disturbance value by iteration, and determine the maximum disturbance in the noise distribution based on the iterative value t ; Based on the maximum disturbance t The random perturbation weight of the fused feature vector Update the process and set the maximum disturbance t and the random perturbation weights Combining, we get the following:

[0023] Will and t The random perturbation weight is calculated by comparing the function The matching degree with the normal distribution, when the matching degree requirement is not met, the variance value in the normal distribution is adjusted so that and t Matched to an adjusted normal distribution.

[0024] According to another aspect of the present invention, a storage medium is provided, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to execute operations corresponding to the above-mentioned method for identifying faults of mechanical equipment.

[0025] According to another aspect of the present invention, there is provided a computer device, comprising a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute operations corresponding to the fault identification method of the mechanical equipment as described above.

[0026] By means of the above technical solution, the technical solution provided by the embodiment of the present invention has at least the following advantages: The present invention provides a method and device for mechanical equipment fault identification, a storage medium, and a computer device. Compared with the prior art, the present invention obtains feature vectors for model training by acquiring, preprocessing, and extracting features of equipment image data and sound data of mechanical equipment in normal and faulty states; then, based on the extracted feature vectors of different modes, a fault identification fusion model for mechanical equipment fault identification is trained, thereby realizing data analysis of multi-modal monitoring data of mechanical equipment, and enabling the trained fault identification fusion model to reduce dependence on a single mode and improve the robustness of the overall system. In addition, when performing model training, the present invention configures random perturbation weights for fused feature vectors in a fused feature data set, which helps the model select a suitable noise level and mixing ratio, not only avoiding the risk of overfitting the model but also effectively improving the accuracy of fault identification.

[0027] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings: Figure 1 A schematic diagram showing a flow chart of a method for identifying a mechanical device fault provided by an embodiment of the present invention; Figure 2 A schematic flow chart of another method for identifying a fault of a mechanical device provided by an embodiment of the present invention is shown; Figure 3 A schematic flow chart of another method for identifying a mechanical device fault provided by an embodiment of the present invention is shown; Figure 4 A schematic diagram showing the structure of a fault identification device for mechanical equipment provided by an embodiment of the present invention is shown; Figure 5 A schematic structural diagram of a computer device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0029] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0030] The embodiment of the present invention provides a method for identifying a fault of a mechanical device. Figure 1 As shown, the method includes: 101. Acquire device image data and sound data of a mechanical device in a normal state and a fault state, wherein the device image data and the sound data are marked with a state label; In an embodiment of the present invention, the current execution end obtains the device image data and sound data of the mechanical equipment in a normal state, and the device image data and sound data of the mechanical equipment in a faulty state. It should be noted that the acquired device image data and sound data are marked with a status label. Among them, the status label represents the normal state and faulty state of the mechanical equipment. In order to facilitate the subsequent feature fusion, the device image data and sound data at the same sampling time can also be associated and saved, which is not specifically limited in the embodiment of the present invention.

[0031] 102. Preprocess the device image data to obtain device image data to be analyzed; and preprocess the sound data to obtain sound data to be analyzed; In the embodiment of the present invention, the current execution end preprocesses the device image data obtained in step 101 to obtain the device image data to be analyzed. The image data preprocessing includes cleaning, enhancing and standardizing the image data to improve the robustness and accuracy of the model. The image data enhancement can be performed by rotation, scaling, cropping, resizing, graying, normalization, etc., which is not specifically limited in the embodiment of the present invention.

[0032] It should be noted that if the image data is extracted from multiple sources, it is also necessary to perform operations such as deduplication and deletion of invalid samples on the image data. It is also necessary to ensure that the size of all image data samples is consistent and the number of channels of all image data samples is consistent.

[0033] In an embodiment of the present invention, the current execution end also preprocesses the sound data obtained in step 101 to obtain the sound data to be analyzed. The sound data preprocessing includes filtering, noise reduction, framing and enhancement processing of the sound data to improve the robustness and accuracy of the model. The sound data enhancement can be enhanced by time offset, speed change, audio filling and cutting, mixed background sound and other technologies, which are not specifically limited in the embodiment of the present invention.

[0034] It should be noted that in order to facilitate subsequent processing and model training, the audio data needs to be divided into windows of fixed length, and the fixed-length audio data needs to be normalized to meet the requirements of the model input.

[0035] 103. Fusing the device image data to be analyzed and the sound data to be analyzed to obtain a fused feature data set; In an embodiment of the present invention, the current execution end extracts image features from the image data of the device to be analyzed obtained in step 102. The extracted image features may include edges, textures, shapes, etc., which are not specifically limited in the embodiment of the present invention. It should be noted that in order to improve the efficiency of fault identification, the embodiment of the present invention preferably uses a deep learning model to automatically extract image data features. In addition, the current execution end also extracts acoustic features from the sound data to be analyzed obtained in step 102. The extracted acoustic features can reflect important information of the sound signal of the mechanical equipment. It should be noted that in order to improve the efficiency of fault identification, the embodiment of the present invention preferably uses a deep learning model to automatically extract sound data features.

[0036] In the embodiment of the present invention, the current execution end performs data fusion processing based on the extracted image features and acoustic features to obtain a fused feature data set. The image features and acoustic features are two different modalities. By fusing the features of the two different modalities, the obtained fused feature data set can obtain a richer and more robust feature representation, which helps to improve the performance and generalization ability of the fault identification fusion model.

[0037] 104. Configuring random disturbance weights for the fused feature vectors in the fused feature data set; and training the long short-term memory artificial neural network model based on the fused feature data set after the random disturbance weights are configured to obtain a fault recognition fusion model; In an embodiment of the present invention, in order to reduce the risk of overfitting, a suitable parameter range is selected, and the current execution end configures a random perturbation weight for the fused feature vector in the fused feature data set, wherein the random perturbation weight is mainly used to determine a reasonable random weight value through the perturbation value.

[0038] The current execution end also trains and processes the long short-term memory artificial neural network model (Long Short-Term Memory, referred to as LSTM) based on the fusion feature data set after configuring the random perturbation weights to obtain a fault recognition fusion model. The LSTM network model is good at processing time series data. When the sound and image data change over time, the LSTM can capture these changes and effectively model them. LSTM can use the spatial features extracted by the convolution layer and combine the temporal information of the sequence data for modeling. In the LSTM layer, multiple LSTM units can be set to model the temporal information. LSTM can capture the long-term dependencies in the sequence data and learn the temporal patterns in the data; one or more fully connected layers can be added after the LSTM layer to output the final prediction results. The fault recognition fusion model trained by the present invention belongs to a multimodal model, which can adapt to different application scenarios (video understanding, intelligent monitoring), etc., which usually contain visual and auditory information. The information of one modality may be incomplete or unreliable. Multimodal fusion can reduce the dependence on a single modality and improve the robustness of the overall system. By processing images and audio simultaneously, the fault recognition fusion model can better understand the context of the scene, which is particularly important for fault recognition applications of mechanical equipment in complex environments.

[0039] It should be noted that after obtaining the fault identification fusion model, the trained model can also be evaluated using a validation set or a test set, and various evaluation indicators such as accuracy, precision, recall, F1 score, etc. can be used to evaluate the performance of the model.

[0040] 105. Obtain equipment image monitoring data and sound monitoring data of the mechanical equipment, and perform fault judgment processing based on the fault identification fusion model, the equipment image monitoring data and the sound monitoring data to obtain the operating status of the mechanical equipment.

[0041] In the embodiment of the present invention, the current execution end also obtains the device image monitoring data and sound monitoring data for real-time monitoring of the mechanical equipment. Based on the fault identification fusion model trained in step 104, the real-time monitoring device image detection data and sound monitoring data are processed for fault judgment to obtain the current operating status of the mechanical equipment, that is, to obtain the status analysis result of whether the current mechanical equipment is faulty.

[0042] Furthermore, as a refinement and extension of the specific implementation of the above embodiment, in order to improve the robustness of the fault identification fusion model to data disturbances and improve the generalization ability of the fault identification fusion model, another mechanical equipment fault identification method is provided, such as Figure 2 As shown, the step of enhancing the sound data includes: 201. Performing Fourier short-time transform and Mel transform on the sound data after the filtering, the denoising and the framing processing to obtain logarithmic Mel spectrum feature data; In the embodiment of the present invention, the current execution end performs Fourier short-time transform (STFT) and mel transform on the sound data after filtering, noise reduction and frame division processing to obtain logarithmic mel spectrum feature data (log-Mel).

[0043] 202. Use a data enhancement method that combines noise injection and Mixup to perform data enhancement processing on the logarithmic Mel spectrum feature data to obtain a sound enhancement data set.

[0044] In the embodiment of the present invention, the current execution end uses a data enhancement method combining noise injection and Mixup to perform data enhancement processing on the logarithmic Mel spectrum feature data obtained in step 201 to obtain a sound enhancement data set. The specific method is as follows: (1) Randomly obtain logarithmic Mel spectrum feature data x 1 and the corresponding status labels y 1 , and logarithmic Mel spectrum feature data x 2 and the corresponding status labels y 2 ; (2) Mixup generates sound enhancement data by injecting a convex combination of noise into the input and feature spaces , and generate the sound enhancement data Corresponding new status labels , the specific formula is as follows:

[0045]

[0046] in, represents a value randomly sampled from a Beta distribution, represents the added noise term; (3) The sound enhancement data And the corresponding new status label The sound enhancement data set is obtained by adding the logarithmic Mel spectrum feature data.

[0047] Further, as a refinement and extension of the specific implementation of the above embodiment, in order to train a model that can accurately identify the fault state of mechanical equipment based on multimodal features, another method for identifying mechanical equipment faults is provided, such as Figure 3As shown, the step of fusing the device image data to be analyzed and the sound data to be analyzed to obtain a fused feature data set includes: 301. Use a convolutional neural network model to perform feature extraction processing on the image data of the device to be analyzed to obtain an image feature vector; and use a convolutional neural network model to perform feature extraction processing on the sound data to be analyzed to obtain an acoustic feature vector; In the embodiment of the present invention, the current execution end uses a convolutional neural network model CNN to perform feature extraction processing on the image data of the device to be analyzed to obtain an image feature vector. Among them, the convolutional neural network model CNN used for image feature extraction can be obtained by transfer learning based on a pre-trained model, thereby improving the recognition accuracy of the model. The specific image feature extraction process is as follows: The image is input into the CNN model and first convolved by multiple convolutional layers to obtain:

[0048] in, f(x) is the output of the convolutional layer, x is the input device image data to be analyzed, W is the weight of the convolution kernel, ∗ represents the convolution operation, b is the bias term, σ is the activation function (such as ReLU); Then, the pooling layer is used for pooling processing to reduce the feature dimension and extract important features, such as maximum pooling or average pooling, and convert the pooled features into a one-dimensional vector. The specific formula is as follows: or

[0049] in, p(x) is the output of the pooling layer, k is the size of the pooling window; Finally, a fully connected layer is used to map the features to a feature vector of fixed length, as follows:

[0050] in, hi is the image feature vector, f is the output of the convolutional layer. W and b are the weight and bias of the fully connected layer respectively.

[0051] In addition, the current execution end also uses a convolutional neural network model CNN to perform feature extraction processing on the sound data to be analyzed to obtain an acoustic feature vector. Among them, the convolutional neural network model CNN used for acoustic feature extraction can also be obtained by transfer learning based on a pre-trained model, thereby improving the recognition accuracy of the model. The specific acoustic feature extraction process is: extract features such as Mel-frequency cepstral coefficients (MFCC), select the convolutional neural network model CNN for feature encoding, and encode the features into a vector of fixed length. The specific formula is as follows:

[0052] in, ha is the acoustic eigenvector, a is the sound data to be analyzed, E is the encoder function.

[0053] 302. Merge the image feature vector and the acoustic feature vector corresponding to the same sampling time into a fused feature vector to obtain a fused feature data set; In the embodiment of the present invention, the image feature vector and the acoustic feature vector corresponding to the same sampling time are currently merged into a fused feature vector. The specific formula is as follows:

[0054] in, hi is the image feature vector, ha is the acoustic feature vector, and [;] indicates vector connection.

[0055] In order to reduce the risk of overfitting and select a suitable parameter range, in an embodiment of the present invention, when performing model training, the current execution end configures a random perturbation weight for the fused feature vector in the fused feature data set; then, based on the random perturbation weight, the fused feature vector is forward propagated to obtain a forward propagation result; then, the model loss is calculated based on the obtained forward propagation result, and the parameters of the long short-term memory artificial neural network model are optimized according to the model loss to obtain the trained fault identification fusion model. Specific optimizable model parameters include learning rate, batch size, network structure, etc., which are not specifically limited in the embodiment of the present invention.

[0056] In order to make the added random disturbance within a reasonable range and ensure the stability of the fault identification fusion model, in the embodiment of the present invention, the current execution end randomly extracts a disturbance value from a normal distribution and determines the disturbance value as the random disturbance weight of the fusion feature vector. ; Among them, the random perturbation weight The mathematical expression is:

[0057] in, The mean is 0 and the variance is η The normal distribution of I is the identity matrix. Among them, the variance of the noise η is a key parameter that controls the intensity of noise injection. η The value of needs to be determined according to the complexity of the model. η Values ​​of mean smaller perturbations, while larger η Values ​​imply larger disturbances. For the sample data , random perturbation weight Is a random value The perturbation value of To determine a reasonable random value λ, and then to constrain and limit it, that is, only matching λ and It can be used as parameter input in the training model.

[0058] Next, the iteration value of the currently extracted disturbance value is determined by iteration, and the maximum disturbance in the noise distribution is determined based on the iteration value. t ; Then, based on the maximum disturbance t The random perturbation weight of the fused feature vector Update the process and set the maximum disturbance t and the random perturbation weights Combining, we get the following:

[0059] Will and t The random perturbation weight is calculated by comparing the function The degree of matching with the normal distribution. If the matching requirement is not met, the variance value in the normal distribution is adjusted. η , so that and t Matched to an adjusted normal distribution.

[0060] For small η value, minimize the objective function with weight noise J can be viewed as minimizing an objective function with a regularization term added, namely , to control the complexity of the model and reduce overfitting; thus deriving the random perturbation weight During the model training process, each input sample uses a weight with random perturbations for forward propagation and loss calculation. This process is training a perturbation model; the target output of the model after adding perturbations is equal to the output of the original model, that is:

[0061] in, For input data x The corresponding value θ Add perturbation weight The target output after disturbance, For input data x Corresponding value θ The output of the original model; in this way, the stability of the training model is maintained.

[0062] Through the above steps, random weight perturbations can be introduced in model training, which improves the generalization ability and robustness of the model and avoids overfitting.

[0063] The embodiment of the present invention provides a method for fault identification of mechanical equipment. Compared with the prior art, the present invention obtains feature vectors for model training by acquiring, preprocessing and extracting features of equipment image data and sound data of mechanical equipment in normal state and fault state; then, based on the extracted feature vectors of different modes, a fault identification fusion model for mechanical equipment fault identification is trained, thereby realizing data analysis of multi-modal monitoring data of mechanical equipment, and enabling the trained fault identification fusion model to reduce dependence on a single mode and improve the robustness of the overall system. In addition, the present invention configures random perturbation weights for fused feature vectors in the fused feature data set during model training, which helps the model select a suitable noise level and mixing ratio, not only avoiding the risk of overfitting the model but also effectively improving the accuracy of fault identification.

[0064] As the above Figure 1 The embodiment of the present invention provides a fault identification device for mechanical equipment, such as Figure 4 As shown, the device comprises: A data acquisition module 41 is used to acquire device image data and sound data of the mechanical device in a normal state and a fault state, wherein the device image data and the sound data are marked with a state label; The preprocessing module 42 is used to preprocess the device image data to obtain the device image data to be analyzed; and to preprocess the sound data to obtain the sound data to be analyzed; A data fusion module 43 is used to fuse the device image data to be analyzed and the sound data to be analyzed to obtain a fused feature data set; The training module 44 is used to configure random perturbation weights for the fused feature vectors in the fused feature data set; and to train the long short-term memory artificial neural network model based on the fused feature data set after the random perturbation weights are configured to obtain a fault recognition fusion model; The fault judgment module 45 is used to obtain the equipment image monitoring data and the sound monitoring data of the mechanical equipment, and perform fault judgment processing based on the fault identification fusion model, the equipment image monitoring data and the sound monitoring data to obtain the operating status of the mechanical equipment.

[0065] Furthermore, the preprocessing module 42 includes: An image data preprocessing unit, used for cleaning, enhancing and standardizing the device image data to obtain the device image data to be analyzed; The sound data preprocessing unit is used to filter, reduce noise, divide frames and enhance the sound data to obtain the sound data to be analyzed.

[0066] Furthermore, the sound data preprocessing unit is also used for: Performing Fourier short-time transform and Mel transform on the sound data after the filtering, the noise reduction and the framing processing to obtain logarithmic Mel spectrum feature data; A data enhancement method combining noise injection and Mixup is used to perform data enhancement processing on the logarithmic Mel spectrum feature data to obtain a sound enhancement data set.

[0067] Furthermore, the sound data preprocessing unit is also used for: Randomly obtain logarithmic Mel spectrum feature data x 1 and the corresponding status labels y 1 , and logarithmic Mel spectrum feature data x 2 and the corresponding status labels y 2 ; The Mixup generates sound enhancement data by injecting a convex combination of noise in the input and feature space. , and generate the sound enhancement data Corresponding new status labels , the specific formula is as follows:

[0068]

[0069] in, represents a value randomly sampled from a Beta distribution, represents the added noise term; The sound enhancement data And the corresponding new status label The sound enhancement data set is obtained by adding the logarithmic Mel spectrum feature data.

[0070] Furthermore, the data fusion module 43 includes: A feature extraction unit, configured to perform feature extraction processing on the image data of the device to be analyzed using a convolutional neural network model to obtain an image feature vector; and perform feature extraction processing on the sound data to be analyzed using a convolutional neural network model to obtain an acoustic feature vector; The feature fusion unit is used to merge the image feature vector and the acoustic feature vector corresponding to the same sampling time into a fused feature vector to obtain a fused feature data set.

[0071] Furthermore, the training module 44 is also used for: Performing forward propagation on the fused feature vector based on the random perturbation weight to obtain a forward propagation result; The model loss is calculated based on the forward propagation result, and the parameters of the long short-term memory artificial neural network model are optimized according to the model loss to obtain the trained fault identification fusion model.

[0072] Furthermore, the training module 44 is also used for: A disturbance value is randomly extracted from a normal distribution, and the disturbance value is determined as the random disturbance weight of the fused feature vector ; Determine the iterative value of the currently extracted disturbance value by iteration, and determine the maximum disturbance in the noise distribution based on the iterative value t ; Based on the maximum disturbance t The random perturbation weight of the fused feature vector Update the process and set the maximum disturbance t and the random perturbation weights Combining, we get the following:

[0073] Will and t The random perturbation weight is calculated by comparing the function The matching degree with the normal distribution, when the matching degree requirement is not met, the variance value in the normal distribution is adjusted so that and t Matched to an adjusted normal distribution.

[0074] The embodiment of the present invention provides a fault identification device for mechanical equipment. Compared with the prior art, the present invention obtains feature vectors for model training by acquiring, preprocessing and extracting features of equipment image data and sound data of mechanical equipment in normal state and fault state; then, based on the extracted feature vectors of different modes, a fault identification fusion model for mechanical equipment fault identification is trained, thereby realizing data analysis of multi-modal monitoring data of mechanical equipment, and enabling the trained fault identification fusion model to reduce dependence on a single mode and improve the robustness of the overall system. In addition, the present invention configures random perturbation weights for fused feature vectors in the fused feature data set during model training, which helps the model select a suitable noise level and mixing ratio, not only avoiding the risk of overfitting the model but also effectively improving the accuracy of fault identification.

[0075] According to one embodiment of the present invention, a storage medium is provided, wherein the storage medium stores at least one executable instruction, and the computer executable instruction can execute the fault identification method of mechanical equipment in any of the above method embodiments.

[0076] Figure 5 A schematic diagram of the structure of a computer device provided according to an embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the computer device.

[0077] like Figure 5 As shown, the computer device may include: a processor (processor) 502 , a communication interface (Communications Interface) 504 , a memory (memory) 506 , and a communication bus 508 .

[0078] The processor 502 , the communication interface 504 , and the memory 506 communicate with each other via a communication bus 508 .

[0079] The communication interface 504 is used to communicate with other devices such as clients or other servers.

[0080] The processor 502 is used to execute the program 510, and specifically can execute the relevant steps of the above-mentioned mechanical equipment fault identification method.

[0081] Specifically, the program 510 may include program codes, which include computer operation instructions.

[0082] The processor 502 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the computer device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0083] The memory 506 is used to store the program 510. The memory 506 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0084] The program 510 may be specifically configured to enable the processor 502 to perform the following operations: Acquire device image data and sound data of the mechanical device in a normal state and a fault state, wherein the device image data and the sound data are marked with a state label; Preprocessing the device image data to obtain the device image data to be analyzed; and preprocessing the sound data to obtain the sound data to be analyzed; Performing fusion processing on the device image data to be analyzed and the sound data to be analyzed to obtain a fused feature data set; Configuring random disturbance weights for fused feature vectors in the fused feature data set; and training a long short-term memory artificial neural network model based on the fused feature data set after configuring the random disturbance weights to obtain a fault recognition fusion model; The equipment image monitoring data and the sound monitoring data of the mechanical equipment are acquired, and the fault judgment processing is performed based on the fault identification fusion model, the equipment image monitoring data and the sound monitoring data to obtain the operating status of the mechanical equipment.

[0085] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0086] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for identifying a fault of a mechanical device, characterized in that: include: Acquire device image data and sound data of the mechanical device in a normal state and a fault state, wherein the device image data and the sound data are marked with a state label; Preprocessing the device image data to obtain the device image data to be analyzed; and preprocessing the sound data to obtain the sound data to be analyzed; Performing fusion processing on the device image data to be analyzed and the sound data to be analyzed to obtain a fusion feature data set; Configuring random disturbance weights for fused feature vectors in the fused feature data set; and training a long short-term memory artificial neural network model based on the fused feature data set after configuring the random disturbance weights to obtain a fault recognition fusion model; The equipment image monitoring data and the sound monitoring data of the mechanical equipment are acquired, and the fault judgment processing is performed based on the fault identification fusion model, the equipment image monitoring data and the sound monitoring data to obtain the operating status of the mechanical equipment.

2. The method according to claim 1, characterized in that The preprocessing of the device image data to obtain the device image data to be analyzed; and the preprocessing of the sound data to obtain the sound data to be analyzed include: Cleaning, enhancing and standardizing the device image data to obtain the device image data to be analyzed; The sound data is filtered, denoised, framed and enhanced to obtain the sound data to be analyzed.

3. The method according to claim 2, characterized in that The sound data is enhanced, including: Performing Fourier short-time transform and Mel transform on the sound data after the filtering, the noise reduction and the framing processing to obtain logarithmic Mel spectrum feature data; A data enhancement method combining noise injection and Mixup is used to perform data enhancement processing on the logarithmic Mel spectrum feature data to obtain a sound enhancement data set.

4. The method according to claim 3, characterized in that The data enhancement method using a combination of noise injection and Mixup is used to perform data enhancement processing on the logarithmic Mel spectrum feature data to obtain a sound enhancement data set, including: Randomly obtain logarithmic Mel spectrum feature data x 1 and the corresponding status label y 1, and logarithmic Mel spectrum feature data x 2 and the corresponding status label y 2; The Mixup generates sound enhancement data by injecting a convex combination of noise in the input and feature space. , and generate the sound enhancement data Corresponding new status labels , the specific formula is as follows: in, represents a value randomly sampled from a Beta distribution, represents the added noise term; The sound enhancement data And the corresponding new status label The sound enhancement data set is obtained by adding the logarithmic Mel spectrum feature data.

5. The method according to claim 1, characterized in that The fusing the device image data to be analyzed and the sound data to be analyzed to obtain a fused feature data set includes: Using a convolutional neural network model to perform feature extraction processing on the image data of the device to be analyzed to obtain an image feature vector; and using a convolutional neural network model to perform feature extraction processing on the sound data to be analyzed to obtain an acoustic feature vector; The image feature vector and the acoustic feature vector corresponding to the same sampling time are merged into a fused feature vector to obtain a fused feature data set.

6. The method according to claim 1, characterized in that The long short-term memory artificial neural network model is trained based on the fusion feature data set after configuring the random disturbance weights to obtain a fault recognition fusion model, including: Performing forward propagation on the fused feature vector based on the random perturbation weight to obtain a forward propagation result; The model loss is calculated based on the forward propagation result, and the parameters of the long short-term memory artificial neural network model are optimized according to the model loss to obtain the trained fault identification fusion model.

7. The method according to claim 1, characterized in that Configuring random perturbation weights for the fused feature vectors in the fused feature data set includes: A disturbance value is randomly extracted from a normal distribution, and the disturbance value is determined as the random disturbance weight of the fused feature vector ; Determine the iterative value of the currently extracted disturbance value by iteration, and determine the maximum disturbance in the noise distribution based on the iterative value t ; Based on the maximum disturbance t The random perturbation weight of the fused feature vector Update the process and set the maximum disturbance t and the random perturbation weights Combining, we get the following: Will and t The random perturbation weight is calculated by comparing the function The matching degree with the normal distribution, when the matching degree requirement is not met, the variance value in the normal distribution is adjusted so that and t Matched to an adjusted normal distribution.

8. A fault identification device for mechanical equipment, characterized in that: include: A data acquisition module, used to acquire device image data and sound data of the mechanical device in a normal state and a fault state, wherein the device image data and the sound data are marked with a state label; A preprocessing module, used to preprocess the device image data to obtain the device image data to be analyzed; and to preprocess the sound data to obtain the sound data to be analyzed; A data fusion module, used for fusing the device image data to be analyzed and the sound data to be analyzed to obtain a fused feature data set; A training module is used to configure random perturbation weights for fused feature vectors in the fused feature data set; and to train a long short-term memory artificial neural network model based on the fused feature data set after the random perturbation weights are configured to obtain a fault recognition fusion model; The fault judgment module is used to obtain the equipment image monitoring data and the sound monitoring data of the mechanical equipment, and perform fault judgment processing based on the fault identification fusion model, the equipment image monitoring data and the sound monitoring data to obtain the operating status of the mechanical equipment.

9. A storage medium, wherein at least one executable instruction is stored in the storage medium, and the executable instruction executes an operation corresponding to the fault identification method of mechanical equipment according to any one of claims 1 to 7.

10. A computer device, comprising a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the fault identification method of mechanical equipment as described in any one of claims 1-7.

Images