Electromechanical spot inspection equipment fault diagnosis method based on deep neural network

Abstract

The invention belongs to the technical field of electromechanical spot inspection equipment fault diagnosis based on a deep neural network, and discloses an electromechanical spot inspection equipment fault diagnosis method based on the deep neural network, and the method comprises the steps: collecting multi-time-period multi-mode signal data, attaching a timestamp, and building a training sample of a fault mark through combining operation and maintenance records; dividing equal-length windows according to timestamps, and counting the number of samples to construct a time density function; during pre-training, introducing a time density function as a regular factor, adjusting a sample weight in a feature embedding layer, compressing a sample weight in a high-density time period, and enhancing a sample weight in a low-density time period; based on a training sample and time distribution regularization embedding, using a multi-task loss function to supervise training, and dynamically adjusting a pre-training network of a time regularization coefficient; and obtaining to-be-diagnosed data, preprocessing the to-be-diagnosed data, inputting the preprocessed to-be-diagnosed data into According to the invention, the stability and robustness of the model across time periods are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of fault diagnosis of electromechanical inspection equipment based on deep neural networks, and in particular to a fault diagnosis method for electromechanical inspection equipment based on deep neural networks. Background Art

[0002] Currently, intelligent methods based on deep neural networks are gradually being adopted for spot inspection of electromechanical equipment at metallurgical production sites. However, due to the volatility of production cycles and the variability of team operations, the sampling data of equipment operating status exhibits non-uniform and skewed distribution characteristics in the time dimension. This leads to the following problems in the training process of existing diagnostic models: the models are easily dominated by densely sampled data within a certain time period, resulting in a decrease in their ability to identify operating characteristics in other time periods; when the models are deployed across different teams or used across time periods, their generalization ability is insufficient, and diagnostic results are prone to bias or false positives; and when the sampling frequency of inspection data is limited, the traditional time window sliding average method cannot effectively alleviate the structural offset caused by time series sample imbalance.

[0003] Therefore, there is an urgent need for a fault diagnosis method that improves the stability and robustness of the model across time periods. Summary of the Invention

[0004] In this regard, the present invention provides a fault diagnosis method, system, electronic device, medium and computer program product for electromechanical inspection equipment based on deep neural network to solve the above technical problems.

[0005] The present invention provides a fault diagnosis method for electromechanical inspection equipment based on a deep neural network, comprising the following method steps: Step S101, collecting multimodal signal data of electromechanical equipment during operation in multiple time periods, each sample data having timestamp information, establishing corresponding fault labels in combination with operation and maintenance records, and obtaining training sample data; Step S102: Divide the training sample data into equal time windows according to timestamps, count the number of samples in each window, and construct a time density function to reflect the distribution density of the samples on the time axis; Step S103, performing time distribution regularization embedding during deep neural network pre-training, specifically, introducing the time density function as a time regularization factor, acting on the sample weight adjustment process in the feature embedding layer, applying penalized weight compression to samples in a first high density time period, and enhancing the representation weight of samples in a second density time period during the training process, wherein the first density is greater than the second density; Step S104: Based on the training sample data and the time distribution regularization embedding, a multi-task loss function is used to perform supervised training on the fault category, and the time regularization coefficient is dynamically adjusted during the training process to obtain a pre-trained deep neural network; Step S105 , obtaining multimodal signal data of the electromechanical equipment to be diagnosed, pre-processing the multimodal signal data and inputting it into the pre-trained deep neural network, and predicting the fault type of the electromechanical equipment through the deep neural network.

[0006] Furthermore, in step S102, when the training sample data is divided into equal time windows according to timestamps, the time window length Δt is within a preset value range and is dynamically adjusted according to the characteristics of the equipment operation cycle; all time windows start from the earliest sample timestamp of the training data set and end at the latest sample timestamp, and each window is marked as W k ; When counting the number of samples in each window, all samples are traversed and the sample timestamp T is used. i Determine the window W to which it belongs k , get the total number of samples n in each window k ; When constructing the time density function ρ(t), first calculate the average sample density ρ of each window k =n k / Δt, and then use the kernel density estimation method to estimate ρ k Smoothing is performed, and the Gaussian kernel is selected as the kernel function. The bandwidth parameter is adaptively adjusted according to the density of the sample time distribution to obtain the continuous function ρ(t) defined on the time axis.

[0007] Furthermore, in step S103, the time distribution regularized embedding is performed, including the following steps: Step S201, calculate the time density coefficient α i Specifically, for the i-th sample in the training set, according to its timestamp T i Query the time density function ρ(t) to get the density value ρ(T i ), for ρ(T i ) is normalized to obtain ρ'(T i )=ρ(T i ) / max(ρ(t)), where max(ρ(t)) is the maximum value of ρ(t) on the entire time axis, , is a positive number between 1e-6 and 1e-4, so that α i ∈(0,1]; Step S202: Input weighted embedding, specifically, the original input feature x of the i-th sample i α i Element-by-element weighting to get x' i =α i ·x i , x' i Passed to subsequent network layers; Step S203: Dynamically weight the loss function. Specifically, the loss L of the i-th sample is i Multiply by α i , total loss L total =Σ(α i ·L i ) / Σα i , where Σα i is the normalization term.

[0008] Furthermore, in step S104, the deep neural network includes a feature processing layer, a temporal feature extraction layer, a feature fusion layer and a prediction output layer; the feature processing layer embeds the modulation feature x' output by the temporal distribution regularization i Perform batch normalization; the time series feature extraction layer uses a 1-3 layer bidirectional gated recurrent unit or a bidirectional long short-term memory network; the feature fusion layer compresses the time series dimension through global average pooling, and then performs feature mapping and fusion through 1-2 layers of fully connected layers; the prediction output layer is a fully connected layer using the Softmax activation function, and the output dimension is consistent with the total number of fault categories; The multi-task loss function is L total =λ1·(Σ(α i ·L cls _i) / Σα i )+λ2·L reg , where L cls _i is the cross entropy loss of the i-th sample, L reg is the time distribution consistency regularization term, λ1 and λ2 are the task weight coefficients in the multi-task loss function.

[0009] Furthermore, the rationality of the time density function ρ(t) is verified and the actual number of samples in each window n is calculated. k The relative error with the theoretical sample number obtained based on the integration of ρ(t) is adjusted if the error exceeds the preset threshold. If ρ(t) suddenly changes to 0 within a certain window, the ρ(t) of the window is corrected by interpolation to ensure the continuity of the density function.

[0010] Furthermore, the time distribution regularized embedding also includes step S204, dynamically adjusting the time regularization coefficient λ, specifically, setting λ to a first preset value at the initial stage of training to weaken the weighted effect of the time distribution regularized embedding, so that the model prioritizes learning the overall distribution characteristics of the data, and gradually increasing it to 1.0 in the middle stage of training to enhance the adjustment intensity of the time distribution regularized embedding on the sample weights, strengthen the learning of sparse time segment features, and maintain λ=1.0 in the later stage of training to ensure that the model converges under the constraint of balanced time distribution.

[0011] On the other hand, the present application also provides a fault diagnosis system for electromechanical inspection equipment based on a deep neural network, including: an acquisition module for collecting multimodal signal data of electromechanical equipment during operation in multiple time periods, each sample data having timestamp information, and establishing corresponding fault labels in combination with operation and maintenance records to obtain training sample data; A first construction module is used to divide the training sample data into equal time windows according to timestamps, count the number of samples in each window, and construct a time density function to reflect the distribution density of the samples on the time axis; a first embedding module for performing time-distributed regularized embedding during deep neural network pre-training, specifically, introducing the time density function as a time regularization factor to act on the sample weight adjustment process in the feature embedding layer, applying penalized weight compression to samples in a first density time period, and enhancing the representation weight of samples in a second density time period during the training process, wherein the first density is greater than the second density; A first training module is configured to perform supervised training on the fault category based on the training sample data and the time distribution regularized embedding using a multi-task loss function, and dynamically adjust the time regularization coefficient during the training process to obtain a pre-trained deep neural network; The output module is used to obtain multimodal signal data of the electromechanical equipment to be diagnosed, pre-process the multimodal signal data and input it into the pre-trained deep neural network, and predict the type of electromechanical equipment fault through the deep neural network.

[0012] The solution provided by the embodiment of the present application effectively solves the problem of uneven distribution of the time dimension of metallurgical electromechanical equipment inspection data due to fluctuations in production rhythm and differences in team operations by introducing a time distribution regularization embedding mechanism. By constructing a time density function to perceive the time distribution characteristics of samples, a punitive weight is imposed on samples in high-density time periods and the representation weight of samples in low-density time periods is enhanced to avoid the model being biased due to the dominance of dense data; the multi-task loss function and the dynamic regularization coefficient adjustment strategy take into account both the fault classification accuracy and the time generalization ability, and improve the diagnostic stability of the model in cross-time period and cross-team scenarios; the pre-processed multimodal signal is deeply integrated with the timestamp information, so that the model can more comprehensively capture the equipment operation characteristics, reduce false alarms and deviations caused by data distribution offset, and ultimately significantly enhance the robustness and reliability of deep neural networks in electromechanical equipment fault diagnosis in complex metallurgical production environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0014] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings: Figure 1 A flowchart of a method for fault diagnosis of electromechanical inspection equipment based on a deep neural network is provided in an embodiment of the present invention.

[0015] Figure 2 A schematic diagram of the time distribution regularization embedding process provided by an embodiment of the present invention.

[0016] Figure 3 A schematic diagram of a deep neural network architecture is provided for an embodiment of the present invention.

[0017] Figure 4 This is a structural diagram of a deep neural network-based electromechanical inspection equipment fault diagnosis system provided by an embodiment of the present invention.

[0018] Figure 5 It is a structural diagram of a device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0020] This application proposes a fault diagnosis method for electromechanical inspection equipment based on deep neural networks. The technical solution of this application is described in detail below in conjunction with various embodiments.

[0021] like Figure 1 As shown, the embodiment of the present invention discloses a fault diagnosis method 100 for electromechanical inspection equipment based on a deep neural network, comprising the following method steps: Step S101 : Collect multimodal signal data of electromechanical equipment during operation in multiple time periods. Each piece of sample data has timestamp information. Combined with operation and maintenance records, corresponding fault labels are established to obtain training sample data.

[0022] In some embodiments, for metallurgical electromechanical equipment (such as rolling mill main drive motors, conveyor belt motors, etc.), multiple types of sensors are deployed to collect multimodal signal data of the equipment during different operating time periods. The multimodal signals include but are not limited to: Vibration signal: collected by acceleration sensors installed in key parts of the equipment, such as the bearing seat and housing, with a sampling frequency of 1kHz to 10kHz, recording the vibration amplitude, frequency, and phase characteristics during equipment operation; Current signal: The three-phase current data of the equipment drive circuit is collected through the current transformer. The sampling frequency is 50Hz~200Hz, reflecting the changes in motor load and electromagnetic characteristics; Sound signal: The acoustic sensor collects the sound wave signal when the equipment is running, with a sampling frequency of 44.1kHz, to capture characteristic sounds such as abnormal friction and impact; Temperature signal: The temperature data of key components of the equipment (such as windings and bearings) is collected through infrared sensors or thermocouples with a sampling frequency of 1Hz~10Hz to monitor the thermal state changes of the equipment.

[0023] Each piece of collected signal data is accompanied by precise timestamp information, recording the specific moment when the signal was generated. Each piece of sample data also contains the corresponding device number, operating parameters (such as load rate, speed) and sampling sensor number.

[0024] In some embodiments, the collected multimodal signal data is fault-marked in combination with the operation and maintenance records of the metallurgical electromechanical equipment (including but not limited to equipment fault repair orders, maintenance work orders, and regular inspection reports): For fault events clearly recorded in the operation and maintenance records (such as bearing wear, motor overload, gear tooth breakage, etc.), the signal data collected during that time period is marked as the corresponding fault category (such as "bearing outer ring wear" and "stator winding short circuit"). For a period of time with no fault records in the operation and maintenance records and where the equipment operating parameters meet normal standards, the signal data during this period is marked as "normal operation"; Signal data with suspected abnormalities (such as parameter fluctuations but no fault alarm is triggered) and the fault type has not yet been confirmed is marked as "pending verification" and will be updated in the future based on manual review or equipment shutdown inspection results.

[0025] Finally, a structured dataset consisting of “timestamp-multimodal signal-fault label” is formed, where the fault label is encoded in one-hot or integer format and serves as supervisory information for model training.

[0026] The multimodal signal data is preprocessed and converted into a form suitable for model input. For example, signal filtering: for vibration, current, and sound signals, bandpass filtering of the corresponding frequency band is used to remove high-frequency noise and low-frequency interference. For example, the vibration signal retains the 10Hz~5kHz frequency band.

[0027] Signal segmentation: Segment the continuous signal based on a sliding window. For example, the window sliding step is 1 / 2 to 1 of the window length to obtain signal sequence segments of fixed dimensions. Based on the timestamp, the segmented segments of the vibration, current, sound, and temperature signals are matched one by one to form sample units containing multi-dimensional features. For example, the dimensions are unified as [time_steps, feature_dim] (where feature_dim is the total number of features of the multimodal signal).

[0028] Feature standardization: Normalize the segmented sample features to eliminate the influence of the difference in signal magnitude. Specifically, calculate the mean μ and standard deviation σ of each feature dimension in the training set; perform standardization transformation on the feature value of each sample: x train '=(x train -μ) / σ, where x train Represents the original training sample feature value, x train ' represents the feature value of the converted training sample, so that the processed data conforms to the normal distribution with a mean of 0 and a standard deviation of 1, improving the learning efficiency of the model for different modal features.

[0029] Step S102 : Divide the training sample data into equal time windows according to timestamps, count the number of samples in each window, and construct a time density function ρ(t) to reflect the distribution density of samples on the time axis.

[0030] In some embodiments, when the training sample data is divided into equal time windows according to timestamps, the time window length Δt is within a preset value range and is dynamically adjusted according to the characteristics of the equipment operation cycle; all time windows start from the earliest sample timestamp of the training data set and end at the latest sample timestamp, and each window is marked as W k ; When counting the number of samples in each window, traverse all samples and calculate the number of samples according to the sample timestamp T i Determine the window W to which it belongs k , get the total number of samples n in each window k ; When constructing the time density function ρ(t), first calculate the average sample density ρ of each window k =n k / Δt, and then use the kernel density estimation method to estimate ρ k Smoothing is performed, and the Gaussian kernel is selected as the kernel function. The bandwidth parameter is adaptively adjusted according to the density of the sample time distribution to obtain the continuous function ρ(t) defined on the time axis.

[0031] Exemplarily, specifically, based on the timestamp information of all samples in the training data, time windows of equal length are defined on the time axis, specifically including: Time window length setting: Based on the operating cycle characteristics of metallurgical electromechanical equipment (such as shift intervals and production takt cycles), the time axis is divided into continuous and non-overlapping time windows of length Δt. The value range of Δt is 5 minutes to 2 hours (it can be dynamically adjusted according to the actual operating rhythm of the equipment. For example, for rolling mill equipment, Δt can be set to 30 minutes to match the rolling batch interval). Window coverage: All time windows start from the earliest sample timestamp in the training dataset and end at the latest sample timestamp in the training dataset, ensuring that the time distribution interval of all training samples is covered. Window ID: uniquely identify each time window, denoted as W k (k=1,2,...,N, N is the total number of windows), where W k The time interval is [t {k-1} ,t k ), t0 is the earliest timestamp, t N is the latest timestamp, and t k -t {k-1} =Δt.

[0032] For example, for each time window W k The number of samples within the scope of the statistics includes: Traverse all samples in the training data set and calculate the timestamp T of the samples i Determine the time window W to which it belongs k (i.e. satisfying t {k-1} ≤T i <t k ); Statistics for each window W k The total number of samples n included k , forming a sample number sequence {n1,n2,...,n N}, where n k ≥0 (if there is no sample in the window, n k =0).

[0033] For example, based on the above sample quantity statistics, a time density function ρ(t) reflecting the distribution density of samples on the time axis is constructed. The specific steps are: Basic density calculation: For each time window W k , calculate its average sample density ρ k =n k / Δt, represents the number of samples per unit time in the window; Density function smoothing: Kernel Density Estimation (KDE) method is used to smooth the discrete ρ k Smoothing is performed to eliminate local fluctuations caused by window division. The kernel function uses a Gaussian kernel, and the bandwidth parameter is adaptively adjusted according to the density of the sample time distribution (for example, reducing the bandwidth in dense sample areas to preserve details, and increasing the bandwidth in sparse areas to smooth noise); Function form definition: After smoothing, the time density function ρ(t) is defined on the time axis [t0,t N ] is a continuous function on , for any time t, the value of ρ(t) represents the sample distribution density per unit time near that time, and ρ(t) ≥ 0.

[0034] Optionally, it also includes verifying the rationality of the time density function ρ(t) and calculating the actual number of samples n in each window. k The relative error with the theoretical sample number obtained based on the integration of ρ(t) is adjusted if the error exceeds the preset threshold. If ρ(t) suddenly changes to 0 within a certain window, the ρ(t) of the window is corrected by interpolation to ensure the continuity of the density function.

[0035] Specifically, calculate the actual number of samples n in each window k The theoretical number of samples obtained based on ρ(t) integration If the relative error exceeds a preset threshold (e.g., 5%), the window length Δt or the kernel density estimation bandwidth parameter is readjusted. For local sample loss due to sensor failure or data transmission interruption (manifested by ρ(t) suddenly dropping to 0 within a window), ρ(t) in that window is corrected using interpolation methods (e.g., linear interpolation) to ensure the continuity of the density function. The resulting temporal density function ρ(t) can be directly used to calculate sample weights in the subsequent Time Distribution Regularized Embedding (TDRE) process.

[0036] Step S103, performing time distribution regularization embedding during deep neural network pre-training, specifically, introducing the time density function ρ(t) as a time regularization factor, acting on the sample weight adjustment process in the feature embedding layer, applying penal weight compression to samples in the first density time period, and enhancing the representation weight of samples in the second density time period during the training process, where the first density is greater than the second density.

[0037] In some embodiments, a time-distributed regularized embedding is performed, such as Figure 2 As shown, the following steps are included: Step S201, calculate the time density coefficient αi Specifically, for the i-th sample in the training set, according to its timestamp T i Query the time density function ρ(t) to get the density value ρ(T i ), for ρ(T i ) is normalized to obtain ρ'(T i )=ρ(T i ) / max(ρ(t)), where max(ρ(t)) is the maximum value of ρ(t) on the entire time axis, , is a very small positive number between 1e-6 and 1e-4, so that α i ∈(0,1]; Step S202: Input weighted embedding, specifically, the original input feature x of the i-th sample i α i Element-by-element weighting to get x' i =α i ·x i , x' i Passed to subsequent network layers; Step S203: Dynamically weight the loss function. Specifically, the loss L of the i-th sample is i Multiply by α i , total loss L total =Σ(α i ·L i ) / Σα i , where Σα i is the normalization term.

[0038] Specifically, for example, based on the time density function ρ(t) obtained in the previous steps, the corresponding time density coefficient α is calculated for each sample in the training data set: i , specifically including: For the i-th sample in the training set, its timestamp is T i , by querying the time density function ρ(t), we can get the density value ρ(i) at that moment; For all samples, the density value ρ(T i ) is normalized to obtain the normalized density value ρ'(T i )=ρ(T i ) / max(ρ(t)), where max(ρ(t)) is the maximum value of the time density function ρ(t) on the entire time axis, so that ρ'(T i )∈[0,1]; Define the time density coefficient α i is the reverse mapping of the normalized density value, i.e. α i =1-ρ'(T i)+ε (ε is a very small positive number ranging from 1e-6 to 1e-4), ensuring that α i ∈(0,1], and satisfy: the higher the density of the sample time period (ρ(T i ) is larger), its corresponding α i The smaller it is; on the contrary, the sample α in the low-density time period i The bigger.

[0039] For example, the deep neural network is a RNN-based network with the following architecture: Figure 3 As shown in the following text, we will introduce an α-based i The weighted modulation of is used to adjust the intensity of the input sample. The specific process is as follows: Let the original input feature of the i-th sample be x i (a multimodal signal sequence fragment of dimension [time_steps, feature_dim]); Through the time density coefficient α i x i Perform element-by-element weighting to obtain the modulated input feature x' i =α i ·x i , where “·” represents element-wise multiplication; After modulation, x' i It is passed into subsequent network layers (such as the RNN core layer) as new input features, so that the sample input intensity in high-density time periods is suppressed and the sample input intensity in low-density time periods is enhanced, avoiding the model from focusing too much on dense samples.

[0040] For example, during the loss calculation phase of model training, α is introduced i As a loss weight factor, it adjusts the loss contribution of samples with different densities, including: Suppose the loss between the predicted value of the i-th sample after forward propagation through the network and the true label (fault label) is L i (such as cross entropy loss, mean square error loss, etc.); Calculate the total loss L total When the loss L for each sample i Multiply by its corresponding time density coefficient α i , that is, L total =Σ(α i ·L i ) / Σα i , where the denominator Σα i is a normalization term to ensure the stability of the magnitude of the total loss; Through this mechanism, the proportion of loss of samples in high-density time periods in the total loss is reduced, and the proportion of loss of samples in low-density time periods is increased, so that the model training process pays more attention to the feature information contained in sparse samples.

[0041] Optionally, in order to balance the model's learning efficiency and temporal generalization ability for the overall characteristics of the data, a dynamically adjustable time regularization coefficient λ is introduced to adaptively control the weighted intensity of TDRE. For example, the time regularization coefficient λ is dynamically adjusted. Specifically, λ is set to a smaller value at the beginning of training to weaken the weighted effect of the time distribution regularization embedding, so that the model prioritizes learning the overall distribution characteristics of the data. It is gradually increased to 1.0 in the middle of training to enhance the adjustment intensity of the time distribution regularization embedding on the sample weights and strengthen the learning of sparse time segment features. λ is maintained at 1.0 in the later stage of training to ensure that the model converges under the constraint of balanced time distribution.

[0042] Among them, λ is the global control coefficient of time distribution regularization embedding, which is used to dynamically adjust the weight intensity of time density perception. Its value range is 0.3~1.0 and changes in stages. It mainly affects the input weighted embedding and the dynamic weighting process of the loss function α i The strength of the effect determines the correction strength of the model to the uneven distribution of time; Specifically, in the early stages of training (e.g., the first 20% of iterations), set λ to a smaller value (e.g., 0.3–0.5) to weaken the weighting effect of TDRE, allowing the model to prioritize learning the overall distribution characteristics of the data. In the middle of training (e.g., 20% to 80% of iterations), gradually increase λ to 1.0 to enhance TDRE's adjustment of sample weights and strengthen the learning of sparse time segment features; In the later stages of training (e.g., the last 20% of iterations), keep λ = 1.0 to ensure that the model converges under the constraint of balanced time distribution; The adjustment strategy of λ can be dynamically optimized according to the performance of the validation set (such as the fault diagnosis accuracy) during the training process. When the performance of the validation set fluctuates, the growth rate of λ is adaptively reduced.

[0043] Through the above-mentioned input weighted embedding, loss dynamic weighting and regularization coefficient adjustment mechanism, TDRE realizes the perception and correction of the uneven sample distribution in the time dimension, enabling the model to evenly learn the equipment operation characteristics of different time periods during training, thereby improving the diagnostic stability across time periods.

[0044] Step S104 , based on the training sample data and the time distribution regularization embedding, a multi-task loss function is used to perform supervised training on the fault category, and the time regularization coefficient is dynamically adjusted during the training process to obtain a pre-trained deep neural network.

[0045] In some embodiments, as Figure 3 As shown, the deep neural network includes a feature processing layer, a time series feature extraction layer, a feature fusion layer and a prediction output layer; the feature processing layer processes the modulation feature x' output by the TDRE module. i Batch normalization operation is performed; the time series feature extraction layer adopts 1~3 layers of bidirectional gated recurrent units or bidirectional long short-term memory networks; the feature fusion layer compresses the time series dimension through global average pooling, and then performs feature mapping and fusion through 1~2 layers of fully connected layers; the prediction output layer is a fully connected layer using the Softmax activation function, and the output dimension is consistent with the total number of fault categories; the multi-task loss function is L total =λ1·(Σ(α i ·L cls _i) / Σα i )+λ2·L reg , where L cls _i is the cross entropy loss of the i-th sample, L reg is the time distribution consistency regularization term, λ1 and λ2 are the task weight coefficients in the multi-task loss function, which are used to balance the contribution ratio of the main task loss (such as fault classification loss) and the auxiliary task loss (i.e., the time distribution consistency regularization term).

[0046] Specifically, based on the feature representation of the time-distributed regularized embedding output, a deep neural network model for fault diagnosis of metallurgical electromechanical equipment is constructed, which exemplarily includes: Feature processing layer: receives the modulation feature x' output by the TDRE module i , standardize the feature distribution through batch normalization operation to reduce the impact of distribution shift of data in different time segments; Temporal feature extraction layer: uses a bidirectional gated recurrent unit (Bi-GRU) or a bidirectional long short-term memory (Bi-LSTM) network as its core structure. The Bi-GRU consists of 1 to 3 layers of forward GRU and backward GRU, with 64 to 256 hidden units per layer, to capture the dynamic evolution characteristics of multimodal signal sequences. Feature fusion layer: The output of the Bi-GRU is compressed into a time series dimension through global average pooling to obtain a fixed-length feature vector, which is then passed through one or two fully connected layers (with 128 to 512 nodes) for feature mapping and fusion. Prediction output layer: A fully connected layer with a Softmax activation function is used as the output layer. The output dimension is consistent with the total number of fault categories (such as bearing faults, winding faults, etc.), and is used to output the predicted probability distribution of each type of fault.

[0047] For example, a multi-task joint supervision loss function is used to comprehensively optimize the model's ability to distinguish fault categories and its ability to learn balanced temporal distribution, specifically including: Main task loss: using cross entropy loss L cls (Cross-Entropy Loss) calculates the difference between the predicted fault category and the true label. The formula is: L cls =-Σ(Y i ·log(P i )); where Y i is the true label of the i-th sample (in one-hot encoding form), P i is the predicted probability distribution of the model output; Auxiliary task loss: introducing the temporal distribution consistency regularization term L reg , the constraint model's representation consistency of features of different time segments is achieved by calculating the mean cosine distance of the feature vectors of samples in high-density periods and samples in low-density periods. The formula is: L reg =1-mean(cos(f high ,f low )); where f high is the feature vector set of samples in high-density period, f low is the feature vector set of samples in the low-density period, and cos() is the cosine similarity function; Total loss function: L total =λ1·(Σ(α i ·L cls _i) / Σα i )+λ2·L reg , where λ1 and λ2 are task weight coefficients (λ1∈[0.7,0.9], λ2∈[0.1,0.3]), α i is the time density coefficient, which balances the learning priorities of the main task and the auxiliary task by weight.

[0048] For example, a phased dynamic training strategy is adopted, combined with time regularization coefficient adjustment, to balance the model convergence efficiency and time generalization ability. The specific steps are as follows: Pre-training phase (iterations 0 to N1): Fixed time regularization coefficients λ = 0.3, λ1 = 0.9, λ2 = 0.1, a large learning rate (initial value 1e-3), and parameter initialization using stochastic gradient descent (SGD) or Adam optimizer to enable the model to quickly learn the basic feature distribution of the data; Reinforcement learning phase (iterations N1 to N2): linearly increase λ to 1.0, simultaneously adjust λ1 to 0.8 and λ2 to 0.2, reduce the learning rate to 5e-4, and introduce an early stopping mechanism. If the fault diagnosis accuracy of the validation set does not improve for five consecutive epochs, pause the current phase of training. Fine-tuning convergence phase (iterations N2-N3): Maintain λ=1.0, λ1=0.7, and λ2=0.3, use a smaller learning rate (1e-4) and a learning rate decay strategy (10% decay every 10 epochs), and use L2 regularization (weight decay coefficient 1e-5) to suppress overfitting until the model performance on the validation set converges stably (fluctuation <0.5%).

[0049] Through the above-mentioned model structure design, multi-task loss function and dynamic training strategy, efficient training and optimization of deep neural networks on time-unbalanced data are achieved, ensuring that the model maintains stable fault diagnosis performance in metallurgical electromechanical equipment inspection scenarios across time periods and work teams.

[0050] Step S105 , obtaining multimodal signal data of the electromechanical equipment to be diagnosed, pre-processing the multimodal signal data and inputting it into the pre-trained deep neural network, and predicting the fault type of the electromechanical equipment through the deep neural network.

[0051] In some embodiments, when obtaining multimodal signal data for spot inspection of electromechanical equipment to be diagnosed, the collected signal type is consistent with the multimodal signal data in step S101, including vibration signals, current signals, sound signals and temperature signals, and each sample data to be diagnosed has a corresponding timestamp.

[0052] Exemplarily, preprocessing the spot inspection multimodal signal data includes the following operations: Signal filtering: Using the same filtering method as the training data preprocessing, the vibration signal, current signal, and sound signal are band-pass filtered to remove high-frequency noise and low-frequency interference. The vibration signal filtering frequency band is 10Hz~5kHz, the current signal is 50Hz±10Hz, and the sound signal is 100Hz~10kHz; Signal segmentation: Use the same sliding window length as the training data (e.g., 0.5s to 5s) to segment the continuous signal, with a window sliding step of 1 / 2 to 1 of the window length, to obtain signal sequence segments with the same dimension as the training sample, with the dimension [time_steps, feature_dim]; Feature standardization: Based on the feature mean and standard deviation of the training data, the segmented signal sequence fragments are standardized. The formula is x test '=(x test-μ) / σ, where x test Represents the original signal sequence fragment of the multimodal signal data of the electromechanical equipment to be diagnosed, x test ' represents the converted signal sequence fragment, where μ is the mean of the corresponding feature of the training data, and σ is the standard deviation of the corresponding feature of the training data; The preprocessed signal sequence fragments to be diagnosed are input into a pre-trained deep neural network. After the deep neural network extracts and maps the features of the input signal, it outputs the predicted probability distribution of the fault type through the prediction output layer, and takes the category with the highest probability as the fault type prediction result; or it outputs the probability value of abnormal device operation through abnormal probability regression. When the probability value exceeds a preset threshold (such as 0.8), it is determined that the device is abnormal.

[0053] Figure 4 A fault diagnosis system 400 for electromechanical inspection equipment based on a deep neural network is shown. Figure 1 Corresponding to the method embodiment shown, the system can be specifically applied to various electronic devices.

[0054] like Figure 4 As shown, an electromechanical inspection equipment fault diagnosis system 400 based on a deep neural network provided in an embodiment of the present application includes: The acquisition module 401 is used to collect multimodal signal data of electromechanical equipment during operation in multiple time periods. Each sample data is timestamped and combined with operation and maintenance records to establish corresponding fault labels to obtain training sample data. The first construction module 402 is used to divide the training sample data into equal time windows according to the timestamp, count the number of samples in each window, and construct a time density function to reflect the distribution density of the samples on the time axis; A first embedding module 403 is configured to perform time distribution regularization embedding during deep neural network pre-training. Specifically, the time density function is introduced as a time regularization factor to act on the sample weight adjustment process in the feature embedding layer, applying penalized weight compression to samples in high-density time periods and enhancing the representation weight of samples in low-density time periods during the training process. A first training module 404 is configured to perform supervised training on the fault category based on the training sample data and the time distribution regularized embedding using a multi-task loss function, dynamically adjusting the time regularization coefficient during the training process to obtain a pre-trained deep neural network; The output module 405 is used to obtain the multimodal signal data of the electromechanical equipment to be diagnosed, pre-process the multimodal signal data and input it into the pre-trained deep neural network, and predict the fault type of the electromechanical equipment through the deep neural network.

[0055] Based on the same inventive concept, an electronic device is also provided in an embodiment of the present application. The method corresponding to the electronic device may be the method in the aforementioned embodiment, and its principle of solving the problem is similar to that of the method. The electronic device provided in an embodiment of the present application includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the methods and / or technical solutions of the aforementioned multiple embodiments of the present application.

[0056] The electronic device may be a user device, or a device formed by integrating a user device and a network device via a network, or an application running on the above device. The user device includes, but is not limited to, various terminal devices such as computers, mobile phones, tablets, smart watches, and wristbands. The network device includes, but is not limited to, network hosts, single network servers, multiple network server sets, or a collection of cloud computing-based computers, and can be used to implement some of the processing functions required for setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing. Cloud computing is a type of distributed computing, consisting of a virtual computer composed of a group of loosely coupled computers.

[0057] Figure 5 The structure of a device suitable for implementing the method and / or technical solution in the embodiment of the present application is shown. The device 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 508 into the random access memory (RAM) 503. Various programs and data required for system operation are also stored in the RAM 503. The CPU 501, ROM 502 and RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0058] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, a touch screen, a microphone, an infrared sensor, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), an LED display, an OLED display, etc., and a speaker; a storage section 508 including one or more computer-readable media such as a hard disk, an optical disk, a magnetic disk, a semiconductor memory, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet.

[0059] In particular, the methods and / or embodiments in the embodiments of the present application can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the method of the present application are performed.

[0060] Another embodiment of the present application further provides a computer-readable storage medium having computer program instructions stored thereon, which can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of the present application.

[0061] Specifically, the present embodiment can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by an instruction execution system, device or device or used in combination with it.

[0062] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0063] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0064] The flow chart or block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the equipment, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code include one or more executable instructions for realizing the logical function of the specification. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated system for hardware that performs the function or operation of the specification, or can be implemented with a combination of dedicated hardware and computer instructions.

[0065] In addition, it is obvious that the word "comprise" does not exclude other units or steps, and the singular does not exclude the plural. The words first, second, etc. are used to indicate names but do not indicate any particular order.

Claims

1. A fault diagnosis method for electromechanical inspection equipment based on deep neural network, characterized in that: The following steps are involved: Step S101: Collect multimodal signal data of electromechanical equipment during operation in multiple time periods. Each sample data is timestamped. Combined with operation and maintenance records, a corresponding fault label is established to obtain training sample data. Step S102: Divide the training sample data into equal time windows according to timestamps, count the number of samples in each window, and construct a time density function to reflect the distribution density of the samples on the time axis; Step S103, performing time distribution regularization embedding during deep neural network pre-training, specifically, introducing the time density function as a time regularization factor, acting on the sample weight adjustment process in the feature embedding layer, applying penalized weight compression to samples in a first density time period, and enhancing the representation weight of samples in a second density time period during the training process, wherein the first density is greater than the second density; Step S104: Based on the training sample data and the time distribution regularization embedding, a multi-task loss function is used to perform supervised training on the fault category, and the time regularization coefficient is dynamically adjusted during the training process to obtain a pre-trained deep neural network; Step S105 , obtaining multimodal signal data of the electromechanical equipment to be diagnosed, pre-processing the multimodal signal data and inputting it into the pre-trained deep neural network, and predicting the fault type of the electromechanical equipment through the deep neural network.

2. The method for fault diagnosis of electromechanical inspection equipment based on deep neural network according to claim 1 is characterized in that: In step S102, when the training sample data is divided into equal time windows according to timestamps, the time window length Δt is within a preset value range and is dynamically adjusted according to the characteristics of the equipment operation cycle; All time windows start from the earliest sample timestamp of the training dataset and end at the latest sample timestamp. Each window is marked as W k ; When counting the number of samples in each window, all samples are traversed and the sample timestamp T is used. i Determine the window W to which it belongs k , get the total number of samples n in each window k ; When constructing the time density function ρ(t), first calculate the average sample density ρ of each window k =n k / Δt, and then use the kernel density estimation method to estimate ρ k Smoothing is performed, and the Gaussian kernel is selected as the kernel function. The bandwidth parameter is adaptively adjusted according to the density of the sample time distribution to obtain the continuous function ρ(t) defined on the time axis.

3. The method for fault diagnosis of electromechanical inspection equipment based on deep neural network according to claim 2 is characterized in that: In step S103, the time distribution regularized embedding is performed, including the following steps: Step S201, calculate the time density coefficient α i Specifically, for the i-th sample in the training set, according to its timestamp T i Query the time density function ρ(t) to get the density value ρ(T i ), for ρ(T i ) is normalized to obtain ρ'(T i )=ρ(T i ) / max(ρ(t)), where max(ρ(t)) is the maximum value of ρ(t) on the entire time axis, , is a positive number between 1e-6 and 1e-4, so that α i ∈(0,1]; Step S202: Input weighted embedding, specifically, the original input feature x of the i-th sample i α i Element-by-element weighting to get x' i =α i ·x i , x' i Passed to subsequent network layers; Step S203: Dynamically weight the loss function. Specifically, the loss L of the i-th sample is i Multiply by α i , total loss L total =Σ(α i ·L i ) / Σα i , where Σα i is the normalization term.

4. The method for fault diagnosis of electromechanical inspection equipment based on deep neural network according to claim 3 is characterized in that: In step S104, the deep neural network includes a feature processing layer, a temporal feature extraction layer, a feature fusion layer and a prediction output layer; the feature processing layer embeds the modulation feature x' output of the temporal distribution regularization i Perform batch normalization; the time series feature extraction layer uses a 1-3 layer bidirectional gated recurrent unit or a bidirectional long short-term memory network; the feature fusion layer compresses the time series dimension through global average pooling, and then performs feature mapping and fusion through 1-2 layers of fully connected layers; the prediction output layer is a fully connected layer using the Softmax activation function, and the output dimension is consistent with the total number of fault categories; The multi-task loss function is L total =λ1·(Σ(α i ·L cls _i) / Σα i )+λ2·L reg , where L cls _i is the cross entropy loss of the i-th sample, L reg is the time distribution consistency regularization term, λ1 and λ2 are the task weight coefficients in the multi-task loss function.

5. The method for fault diagnosis of electromechanical inspection equipment based on deep neural network according to claim 2 is characterized in that: Verify the rationality of the time density function ρ(t) and calculate the actual number of samples n in each window k The relative error with the theoretical sample number obtained based on the integration of ρ(t) is adjusted if the error exceeds the preset threshold. If ρ(t) suddenly changes to 0 within a certain window, the ρ(t) of the window is corrected by interpolation to ensure the continuity of the density function.

6. The method for fault diagnosis of electromechanical inspection equipment based on deep neural network according to claim 3 is characterized in that: The time distribution regularized embedding further includes step S204, dynamically adjusting the time regularization coefficient λ. Specifically, λ is set to a first preset value at the initial stage of training to weaken the weighted effect of the time distribution regularized embedding, so that the model prioritizes learning the overall distribution characteristics of the data. The value is gradually increased to 1.0 in the middle stage of training to enhance the adjustment intensity of the time distribution regularized embedding on the sample weights and strengthen the learning of sparse time segment features. λ is maintained at 1.0 in the later stage of training to ensure that the model converges under the constraint of balanced time distribution.

7. A fault diagnosis system for electromechanical inspection equipment based on deep neural network, characterized in that: include: The acquisition module is used to collect multimodal signal data of electromechanical equipment during operation in multiple time periods. Each sample data is timestamped and combined with operation and maintenance records to establish corresponding fault labels and obtain training sample data; A first construction module is used to divide the training sample data into equal time windows according to timestamps, count the number of samples in each window, and construct a time density function to reflect the distribution density of the samples on the time axis; a first embedding module for performing time-distributed regularized embedding during deep neural network pre-training, specifically, introducing the time density function as a time regularization factor to act on the sample weight adjustment process in the feature embedding layer, applying penalized weight compression to samples in a first density time period, and enhancing the representation weight of samples in a second density time period during the training process, wherein the first density is greater than the second density; A first training module is configured to perform supervised training on the fault category based on the training sample data and the time distribution regularized embedding using a multi-task loss function, and dynamically adjust the time regularization coefficient during the training process to obtain a pre-trained deep neural network; The output module is used to obtain multimodal signal data of the electromechanical equipment to be diagnosed, pre-process the multimodal signal data and input it into the pre-trained deep neural network, and predict the type of electromechanical equipment fault through the deep neural network.

Images