A mechanical and electrical point inspection equipment fault diagnosis method based on a deep neural network

Abstract

The application belongs to the technical field of electromechanical point inspection equipment fault diagnosis based on deep neural network, and discloses a kind of electromechanical point inspection equipment fault diagnosis method based on deep neural network, the method comprises the following steps: collecting multi-time period multi-modal signal data and attaching time stamp, combining operation and maintenance record to establish the training sample of fault mark;According to time stamp, divide equal length window, count sample number and construct time density function;When pre-training, introduce time density function as regular factor, adjust sample weight in feature embedding layer, compress high-density period sample weight, and enhance low-density period sample weight;Based on training sample and time distribution regularization embedding, use multi-task loss function to supervise training, and dynamically adjust the pre-training network of time regularization coefficient;After pre-processing, input pre-training network to obtain to-be-diagnosed data, and predict equipment fault type.The application improves the stability and robustness of the model across time periods.

Description

Technical Field

[0001] This invention relates to the field of fault diagnosis technology for electromechanical inspection equipment based on deep neural networks, and more specifically, to a fault diagnosis method for electromechanical inspection equipment based on deep neural networks. Background Technology

[0002] Currently, the inspection of electromechanical equipment in metallurgical production sites has gradually adopted intelligent methods based on deep neural networks. However, due to the fluctuations in production cycles and the differences in shift operations, the sampling data of equipment operating status exhibits non-uniform and skewed distribution characteristics in the time dimension. This leads to the following problems in the training process of existing diagnostic models: the model is easily influenced by the dominant influence of densely sampled data within a certain time period, resulting in a decrease in its ability to identify operating characteristics in other time periods; when the model is deployed in different shifts or used across time periods, its generalization ability is insufficient, and the diagnostic results are prone to bias or false alarms; under the condition of limited sampling frequency of inspection data, the traditional time window moving average method cannot effectively alleviate the structural shift caused by the imbalance of time series samples.

[0003] Therefore, there is an urgent need for a fault diagnosis method to improve the stability and robustness of the model across time periods. Summary of the Invention

[0004] To address this issue, the present invention provides a method, system, electronic device, medium, and computer program product for fault diagnosis of electromechanical inspection equipment based on deep neural networks, in order to solve the aforementioned technical problems.

[0005] This invention provides a fault diagnosis method for electromechanical inspection equipment based on deep neural networks, including the following steps: Step S101, collect multimodal signal data of electromechanical equipment during operation in multiple time periods, each sample data has a timestamp information, and establish corresponding fault markers in combination with operation and maintenance records to obtain training sample data;

[0006] Step S102: Divide the training sample data into equal-length windows according to timestamps, count the number of samples in each window, and construct a time density function to reflect the distribution density of samples on the time axis.

[0007] Step S103: Temporal distribution regularization embedding is performed during the pre-training of the deep neural network. Specifically, the temporal density function is introduced as a temporal regularization factor and applied to the sample weight adjustment process in the feature embedding layer. A penalty weight compression is applied to the samples in the first high-density time period, and the representation weights of the samples in the second density time period are enhanced during the training process, wherein the first density is greater than the second density.

[0008] Step S104: Based on the training sample data and the time distribution regularized embedding, a multi-task loss function is used to supervise the training of the fault categories. During the training process, the time regularization coefficient is dynamically adjusted to obtain a pre-trained deep neural network.

[0009] Step S105: Obtain multimodal signal data of the electromechanical equipment to be diagnosed, preprocess the multimodal signal data and input it into the pre-trained deep neural network, and predict the fault type of the electromechanical equipment through the deep neural network.

[0010] Further, in step S102, when the training sample data is divided into equal-length windows according to timestamps, the time window length Δt is a preset range and is dynamically adjusted according to the characteristics of the device's operating cycle; all time windows start from the earliest sample timestamp of the training dataset and end at the latest sample timestamp, and each window is identified as W. k ;

[0011] When counting the number of samples in each window, iterate through all samples and count them based on the sample timestamp T. i Determine the window it belongs to, W k The total number of samples n for each window is obtained by statistics. k ;

[0012] When constructing the time density function ρ(t), first calculate the average sample density ρ for each window. k =n k / Δt, then use the kernel density estimation method to estimate ρ k Smoothing is performed, a Gaussian kernel is selected as the kernel function, and the bandwidth parameter is adaptively adjusted according to the density of the sample time distribution to obtain a continuous function ρ(t) defined on the time axis.

[0013] Further, in step S103, the time distribution regularization embedding includes the following steps:

[0014] Step S201, calculate the time density coefficient α i Specifically, for the i-th sample in the training set, based on its timestamp T i Querying the time density function ρ(t) yields the density value ρ(T). i ), for ρ(T) i Normalization yields ρ'(T) i )=ρ(T i ) / max(ρ(t)), where max(ρ(t)) is the maximum value of ρ(t) over the entire time axis. , For positive numbers ranging from 1e-6 to 1e-4, such that α i ∈(0,1];

[0015] Step S202, input weighted embedding, specifically, the original input feature x of the i-th sample. i via α i x' is obtained by weighting elements one by one i =α i ·x i , x' i Passed into subsequent network layers;

[0016] Step S203, dynamically weight the loss function, specifically, the loss L for the i-th sample... i Multiply by α i Total loss L total =Σ(α i ·L i ) / Σα i , where Σα i This is the normalization term.

[0017] Further, in step S104, the deep neural network includes a feature processing layer, a temporal feature extraction layer, a feature fusion layer, and a prediction output layer; the feature processing layer embeds the modulation feature x' of the output in a time-distributed regularized manner. i Batch normalization is performed; the temporal feature extraction layer uses a 1-3 layer bidirectional gated recurrent unit or a bidirectional long short-term memory network; the feature fusion layer compresses the temporal dimension through global average pooling, and then performs feature mapping and fusion through a 1-2 layer fully connected layer; the prediction output layer is a fully connected layer using the Softmax activation function, and the output dimension is consistent with the total number of fault categories.

[0018] The multi-task loss function is L total =λ1·(Σ(α i ·L cls _i) / Σα i )+λ2·L reg L cls _i is the cross-entropy loss of the i-th sample, L reg λ1 and λ2 are the time distribution consistency regularization terms, and the task weight coefficients in the multi-task loss function.

[0019] Furthermore, the rationality of the stated time density function ρ(t) is verified, and the actual number of samples n in each window is calculated. k The relative error between the theoretical sample number obtained based on the integral of ρ(t) and the error exceeds the preset threshold. If the error exceeds the preset threshold, the window length or bandwidth parameter will be readjusted. If ρ(t) suddenly becomes 0 in a certain window, the ρ(t) of that window will be corrected by interpolation to ensure the continuity of the density function.

[0020] Furthermore, the step of performing time distribution regularized embedding also includes step S204, which dynamically adjusts the time regularization coefficient λ. Specifically, in the early stage of training, λ is set to a first preset value to weaken the weighting effect of time distribution regularized embedding, so that the model prioritizes learning the overall distribution characteristics of the data. In the middle stage of training, λ is gradually increased to 1.0 to enhance the adjustment strength of time distribution regularized embedding on sample weights and strengthen the learning of sparse time segment features. In the later stage of training, λ is kept at 1.0 to ensure that the model converges under the constraint of balanced time distribution.

[0021] In another aspect, this application also provides a fault diagnosis system for electromechanical inspection equipment based on deep neural networks, including: a data acquisition module, used to acquire multimodal signal data of electromechanical equipment during operation in multiple time periods, each sample data has a timestamp information, and a corresponding fault mark is established by combining the operation and maintenance records to obtain training sample data;

[0022] The first construction module is used to divide the training sample data into equal-length windows according to timestamps, count the number of samples in each window, and construct a time density function to reflect the distribution density of samples on the time axis.

[0023] The first embedding module is used to perform time distribution regularized embedding during the pre-training of the deep neural network. Specifically, the time density function is introduced as a time regularization factor and applied to the sample weight adjustment process in the feature embedding layer. A penalty weight compression is applied to the samples in the first density time period, and the representation weights of the samples in the second density time period are enhanced during the training process, wherein the first density is greater than the second density.

[0024] The first training module is used to supervise the training of fault categories based on training sample data and the time distribution regularized embedding, and dynamically adjust the time regularization coefficient during the training process to obtain a pre-trained deep neural network.

[0025] The output module is used to acquire multimodal signal data of the electromechanical equipment to be diagnosed, preprocess the multimodal signal data and input it into the pre-trained deep neural network, and predict the fault type of the electromechanical equipment through the deep neural network.

[0026] The solution provided in this application effectively solves the problem of uneven time dimension distribution in metallurgical electromechanical equipment inspection data caused by production cycle fluctuations and differences in shift operations by introducing a time distribution regularization embedding mechanism. By constructing a time density function to perceive the time distribution characteristics of samples, penalizing weights are applied to samples in high-density periods and enhancing the representation weights of samples in low-density periods, avoiding model bias due to dense data dominance. A multi-task loss function and dynamic regularization coefficient adjustment strategy balance fault classification accuracy and time generalization ability, improving the model's diagnostic stability across time cycles and shifts. Deep fusion of preprocessed multimodal signals and timestamp information enables the model to more comprehensively capture equipment operating characteristics, reducing false alarms and biases caused by data distribution shifts, ultimately significantly enhancing the robustness and reliability of deep neural networks in diagnosing electromechanical equipment faults in complex metallurgical production environments. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0028] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0029] Figure 1 A flowchart of a fault diagnosis method for electromechanical inspection equipment based on deep neural networks is provided for an embodiment of the present invention.

[0030] Figure 2 This is a schematic diagram of the time distribution regularization embedding process provided in an embodiment of the present invention.

[0031] Figure 3 A schematic diagram of a deep neural network architecture is provided for embodiments of the present invention.

[0032] Figure 4 This is a schematic diagram of the structure of a fault diagnosis system for electromechanical inspection equipment based on a deep neural network, provided in an embodiment of the present invention.

[0033] Figure 5 This is a schematic diagram of the structure of a device provided in an embodiment of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0035] This application proposes a fault diagnosis method for electromechanical inspection equipment based on deep neural networks. The technical solution of this application will be described in detail below with reference to various embodiments.

[0036] like Figure 1 As shown, this embodiment of the invention discloses a fault diagnosis method 100 for electromechanical inspection equipment based on deep neural networks, including the following method steps:

[0037] Step S101: Collect multimodal signal data of electromechanical equipment during operation over multiple time periods. Each sample data has a timestamp information. Combine the data with the operation and maintenance records to establish corresponding fault markers and obtain training sample data.

[0038] In some embodiments, for metallurgical electromechanical equipment (such as the main drive motor of a rolling mill, conveyor belt motor, etc.), multiple types of sensors are deployed to collect multimodal signal data of the equipment during different operating periods. The multimodal signals include, but are not limited to:

[0039] Vibration signals: Acquired by accelerometers installed in key parts of the equipment such as bearing housings and housings, with a sampling frequency of 1kHz~10kHz, recording the vibration amplitude, frequency and phase characteristics during equipment operation;

[0040] Current signal: The three-phase current data of the equipment drive circuit is collected through a current transformer, with a sampling frequency of 50Hz~200Hz, reflecting the changes in motor load and electromagnetic characteristics;

[0041] Sound signal: Acoustic signals are collected by an acoustic sensor during device operation, with a sampling frequency of 44.1kHz, capturing characteristic sounds such as abnormal friction and impact;

[0042] Temperature signal: Temperature data of key components of the equipment (such as windings and bearings) are collected through infrared sensors or thermocouples at a sampling frequency of 1Hz~10Hz to monitor the thermal changes of the equipment.

[0043] Each acquired signal data is accompanied by a precise timestamp, recording the specific time the signal was generated. Each sample data also includes the corresponding device number, operating parameters (such as load rate and speed), and sampling sensor number.

[0044] In some embodiments, the collected multimodal signal data is fault-marked by combining the operation and maintenance records of metallurgical electromechanical equipment (including but not limited to equipment fault repair orders, maintenance work orders, and periodic inspection reports):

[0045] For fault events clearly recorded in the operation and maintenance records (such as bearing wear, motor overload, gear tooth breakage, etc.), the signal data collected within that time period are marked as the corresponding fault category (such as "bearing outer ring wear", "stator winding short circuit", etc.) according to the time range of the fault occurrence.

[0046] For time periods in which there are no fault records in the maintenance logs and the equipment operating parameters meet the normal standards, the signal data during that time period will be marked as "normal operation".

[0047] For signal data that is suspected to be abnormal (such as parameter fluctuations but no fault alarm is triggered) and the fault type has not yet been confirmed, it is marked as "to be verified" and the mark will be updated in the future based on the results of manual review or equipment shutdown inspection.

[0048] The final result is a structured dataset containing "timestamp-multimodal signal-fault label", where the fault label is encoded in one-hot or integer form and is used as supervision information for model training.

[0049] The multimodal signal data preprocessing transforms it into a form suitable for model input. For example, signal filtering: for vibration, current, and sound signals, bandpass filtering of the corresponding frequency band is used to remove high-frequency noise and low-frequency interference. For example, vibration signals retain the 10Hz~5kHz frequency band.

[0050] Signal segmentation: The continuous signal is segmented based on a sliding window. For example, the sliding step size of the window is 1 / 2 to 1 of the window length to obtain signal sequence segments with fixed dimensions. The segments of vibration, current, sound and temperature signals are matched one by one based on the timestamp to form sample units containing multi-dimensional features. For example, the dimensions are unified as [time_steps, feature_dim] (where feature_dim is the total number of features of the multimodal signal).

[0051] Feature standardization: The features of the segmented samples are normalized to eliminate the influence of differences in signal magnitude. Specifically, the mean μ and standard deviation σ of each feature dimension in the training set are calculated; the feature values ​​of each sample are standardized: x train '=(x train -μ) / σ, where x train x represents the feature values ​​of the original training samples. train ' represents the transformed feature values ​​of the training samples, so that the processed data conforms to a normal distribution with a mean of 0 and a standard deviation of 1, thereby improving the model's learning efficiency for different modal features.

[0052] Step S102: Divide the training sample data into equal-length windows according to timestamps, count the number of samples in each window, and construct a time density function ρ(t) to reflect the distribution density of samples on the time axis.

[0053] In some embodiments, when the training sample data is divided into equal-length windows according to timestamps, the time window length Δt is a preset range and is dynamically adjusted according to the device operating cycle characteristics; all time windows start from the earliest sample timestamp of the training dataset and end at the latest sample timestamp, and each window is identified as W. k When counting the number of samples in each window, iterate through all samples and count them based on the sample timestamp T. i Determine the window it belongs to, W k The total number of samples n for each window is obtained by statistics. k When constructing the time density function ρ(t), first calculate the average sample density ρ for each window. k =n k / Δt, then use the kernel density estimation method to estimate ρ k Smoothing is performed, a Gaussian kernel is selected as the kernel function, and the bandwidth parameter is adaptively adjusted according to the density of the sample time distribution to obtain a continuous function ρ(t) defined on the time axis.

[0054] For example, specifically, based on the timestamp information of all samples in the training data, equal-length time windows are defined on the time axis, specifically including:

[0055] Time window length setting: Based on the operating cycle characteristics of metallurgical electromechanical equipment (such as shift intervals and production cycle), the time axis is divided into continuous and non-overlapping time windows of length Δt. The value range of Δt is 5 minutes to 2 hours (which can be dynamically adjusted according to the actual operating rhythm of the equipment. For example, for rolling mill equipment, Δt can be set to 30 minutes to match the rolling batch interval).

[0056] Window coverage: The start time of all time windows starts from the earliest sample timestamp in the training dataset and ends at the latest sample timestamp in the training dataset, ensuring that the time distribution range of all training samples is covered.

[0057] Window Identifier: Each time window is uniquely identified and denoted as W. k (k=1,2,...,N, where N is the total number of windows), where W k The time interval is [t] {k-1} ,t k ), t0 is the earliest timestamp, t N It is the latest timestamp, and t k -t {k-1} =Δt.

[0058] For example, for each time window W k The number of samples within the sample was statistically analyzed, specifically including:

[0059] Iterate through all samples in the training dataset, based on the sample's timestamp T. i Determine the time window W to which it belongs k (that is, satisfying t) {k-1} ≤T i <t k );

[0060] Statistics for each window W k The total number of samples n contained within k This forms a sample size sequence {n1, n2, ..., n}. N}, where n k ≥0 (n if there are no samples in the window) k =0).

[0061] For example, based on the above sample size statistics, a time density function ρ(t) reflecting the sample distribution density on the time axis is constructed. The specific steps are as follows:

[0062] Basic density calculation: for each time window W k Calculate its average sample density ρ k =n k / Δt represents the number of samples per unit time within this window;

[0063] Density function smoothing: The kernel density estimation (KDE) method is used to smooth the discrete ρ function. k Smoothing is performed to eliminate local fluctuations caused by window division. A Gaussian kernel is selected as the kernel function, and the bandwidth parameter is adaptively adjusted according to the density of the sample time distribution (for example, reducing the bandwidth in dense sample regions to preserve details and increasing the bandwidth in sparse regions to smooth noise).

[0064] Functional definition: After smoothing, the time density function ρ(t) is defined on the time axis [t0, t... N A continuous function on [a], where for any time t, the value of ρ(t) represents the sample distribution density per unit time in the vicinity of that time, and ρ(t)≥0.

[0065] Optionally, the method also includes verifying the rationality of the time density function ρ(t) and calculating the actual number of samples n for each window. kThe relative error between the theoretical sample number obtained based on the integral of ρ(t) and the error exceeds the preset threshold. If the error exceeds the preset threshold, the window length or bandwidth parameter will be readjusted. If ρ(t) suddenly becomes 0 in a certain window, the ρ(t) of that window will be corrected by interpolation to ensure the continuity of the density function.

[0066] Specifically, calculate the actual number of samples n in each window. k Compared with the theoretical sample size obtained based on the integral of ρ(t) The relative error is calculated. If the error exceeds a preset threshold (e.g., 5%), the window length Δt or the bandwidth parameter of the kernel density estimation is readjusted. For local sample loss caused by sensor failure or data transmission interruption (manifested as ρ(t) abruptly becoming 0 within a certain window), ρ(t) of that window is corrected using interpolation methods (e.g., linear interpolation) to ensure the continuity of the density function. The final time density function ρ(t) can be directly used to calculate sample weights in subsequent Time Distribution Regularized Embedding (TDRE).

[0067] Step S103 involves performing time distribution regularization embedding during the pre-training of the deep neural network. Specifically, the time density function ρ(t) is introduced as a time regularization factor and applied to the sample weight adjustment process in the feature embedding layer. A punitive weight compression is applied to the samples in the first density time period, and the representation weights of the samples in the second density time period are enhanced during the training process, wherein the first density is greater than the second density.

[0068] In some embodiments, time distribution regularization embedding is performed, such as Figure 2 As shown, it includes the following steps:

[0069] Step S201, calculate the time density coefficient α i Specifically, for the i-th sample in the training set, based on its timestamp T i Querying the time density function ρ(t) yields the density value ρ(T). i ), for ρ(T) i Normalization yields ρ'(T) i )=ρ(T i ) / max(ρ(t)), where max(ρ(t)) is the maximum value of ρ(t) over the entire time axis. , For a minimum positive number between 1e-6 and 1e-4, such that α i ∈(0,1];

[0070] Step S202, input weighted embedding, specifically, the original input feature x of the i-th sample. i via α ix' is obtained by weighting elements one by one i =α i ·x i , x' i Passed into subsequent network layers;

[0071] Step S203, dynamically weight the loss function, specifically, the loss L for the i-th sample... i Multiply by α i Total loss L total =Σ(α i ·L i ) / Σα i , where Σα i This is the normalization term.

[0072] Specifically, for example, based on the time density function ρ(t) obtained in the aforementioned steps, the corresponding time density coefficient α is calculated for each sample in the training dataset. i Specifically, it includes:

[0073] For the i-th sample in the training set, its timestamp is T. i The density value ρ(i) at that moment is obtained by querying the time density function ρ(t);

[0074] Density values ​​ρ(T) for all samples i The normalized density value ρ'(T) is obtained by normalizing the density. i )=ρ(T i ) / max(ρ(t)), where max(ρ(t)) is the maximum value of the time density function ρ(t) over the entire time axis, such that ρ'(T) i )∈[0,1];

[0075] Define the time density coefficient α i This is the inverse mapping of the normalized density value, i.e., α i =1-ρ'(T i )+ε (where ε is a very small positive number, ranging from 1e-6 to 1e-4), ensuring α i ∈(0,1], and satisfy: the higher the density of the time period in which the sample is located (ρ(T)), the better (ρ(T)). i The larger the α, the greater its corresponding α. i The smaller the value, the better; conversely, the smaller the value, the better for samples in low-density time periods. i The larger.

[0076] For example, a deep neural network is an RNN-based network, and its architecture is as follows: Figure 3 As shown below, further explanation will follow, introducing an α-based... i The weighted modulation is used to adjust the intensity of the input sample. The specific process is as follows:

[0077] Let x be the original input feature of the i-th sample. i (A multimodal signal sequence segment with dimensions [time_steps, feature_dim]);

[0078] Through the time density coefficient α i For x i Element-wise weighting is performed to obtain the modulated input feature x' i =α i ·x i , where “·” represents element-wise multiplication;

[0079] Modulated x' i This new input feature is fed into subsequent network layers (such as the RNN core layer), which suppresses the sample input intensity in high-density time periods and enhances the sample input intensity in low-density time periods, thus preventing the model from over-focusing on dense samples.

[0080] For example, α is introduced during the loss calculation phase of model training. i As a loss weighting factor, it adjusts the loss contribution of samples with different densities, specifically including:

[0081] Let L be the loss between the predicted value of the i-th sample after forward propagation through the network and the true label (fault marker). i (such as cross-entropy loss, mean squared error loss, etc.);

[0082] Calculate the total loss L total At that time, the loss L for each sample i Multiply by its corresponding time density coefficient α i L total =Σ(α i ·L i ) / Σα i , where the denominator Σα i This is a normalization term to ensure the stability of the magnitude of the total loss;

[0083] This mechanism reduces the proportion of loss from high-density time period samples in the total loss and increases the proportion of loss from low-density time period samples, making the model training process focus more on the feature information contained in sparse samples.

[0084] Optionally, to balance the model's learning efficiency of the overall data features with its temporal generalization ability, a dynamically adjustable time regularization coefficient λ is introduced to adaptively control the weighting strength of TDRE. For example, the time regularization coefficient λ is dynamically adjusted by setting λ to a small value in the early stage of training to weaken the weighting effect of the time distribution regularization embedding, so that the model prioritizes learning the overall distribution features of the data. In the middle stage of training, λ is gradually increased to 1.0 to enhance the adjustment strength of the time distribution regularization embedding on the sample weights and strengthen the learning of sparse time segment features. In the later stage of training, λ is kept at 1.0 to ensure that the model converges under the constraint of balanced time distribution.

[0085] Here, λ is the global adjustment coefficient for time-distribution regularized embedding, used to dynamically adjust the weight strength of time density perception. Its value varies in stages between 0.3 and 1.0, mainly affecting α during the input weighted embedding and dynamic weighting of the loss function. i The strength of the effect determines the strength of the model's correction to the unevenness of time distribution;

[0086] Specifically, in the early stages of training (such as the first 20% of iterations), λ is set to a small value (such as 0.3~0.5) to reduce the weighting effect of TDRE and allow the model to learn the overall distribution characteristics of the data first.

[0087] In the middle of training (e.g., 20%~80% of the iterations), λ is gradually increased to 1.0 to enhance the adjustment strength of TDRE on sample weights and strengthen the learning of sparse time segment features;

[0088] In the later stages of training (such as the last 20% of iterations), keep λ=1.0 to ensure that the model converges under the constraint of a balanced time distribution;

[0089] The adjustment strategy for λ can be dynamically optimized based on the performance of the validation set (such as fault diagnosis accuracy) during the training process. When the performance of the validation set fluctuates, the growth rate of λ is adaptively reduced.

[0090] Through the aforementioned input weighted embedding, loss dynamic weighting, and regularization coefficient adjustment mechanisms, TDRE achieves the perception and correction of sample distribution imbalance in the time dimension, enabling the model to learn the equipment operation characteristics of different time periods in a balanced manner during training, thereby improving the diagnostic stability across time periods.

[0091] Step S104: Based on the training sample data and the time distribution regularized embedding, a multi-task loss function is used to supervise the training of the fault categories. During the training process, the time regularization coefficient is dynamically adjusted to obtain a pre-trained deep neural network.

[0092] In some embodiments, such as Figure 3As shown, the deep neural network includes a feature processing layer, a temporal feature extraction layer, a feature fusion layer, and a prediction output layer; the feature processing layer processes the modulation feature x' output by the TDRE module. i Batch normalization is performed; the temporal feature extraction layer uses a 1-3 layer bidirectional gated recurrent unit or bidirectional long short-term memory network; the feature fusion layer compresses the temporal dimension through global average pooling, and then performs feature mapping and fusion through 1-2 fully connected layers; the prediction output layer is a fully connected layer using the Softmax activation function, and the output dimension is consistent with the total number of fault categories; the multi-task loss function is L... total =λ1·(Σ(α i ·L cls _i) / Σα i )+λ2·L reg L cls _i is the cross-entropy loss of the i-th sample, L reg λ1 and λ2 are the task weight coefficients in the multi-task loss function, which are used to balance the contribution ratio of the main task loss (such as fault classification loss) and the auxiliary task loss (i.e., the time distribution consistency regularization term).

[0093] Specifically, based on the feature representation of the time-distribution regularized embedded output, a deep neural network model for fault diagnosis of metallurgical electromechanical equipment is constructed, which, for example, includes:

[0094] Feature processing layer: Receives modulation features x' output from the TDRE module. i By standardizing the feature distribution through batch normalization, the impact of distribution offset of data in different time segments can be reduced.

[0095] Temporal feature extraction layer: Bidirectional gated recurrent unit (Bi-GRU) or bidirectional long short-term memory network (Bi-LSTM) is used as the core structure. The Bi-GRU contains 1 to 3 layers of forward GRU and backward GRU, with 64 to 256 hidden units in each layer, which is used to capture the dynamic evolution features in multimodal signal sequences.

[0096] Feature fusion layer: The output of Bi-GRU is compressed in time dimension through global average pooling to obtain a fixed-length feature vector, which is then processed by 1 to 2 fully connected layers (128 to 512 nodes) for feature mapping and fusion.

[0097] Predictive output layer: A fully connected layer with the Softmax activation function is used as the output layer. The output dimension is consistent with the total number of fault categories (such as bearing faults, winding faults, etc.) and is used to output the predicted probability distribution of various faults.

[0098] For example, a multi-task joint supervised loss function is adopted to comprehensively optimize the model's ability to distinguish fault categories and its ability to learn the balance of time distribution, specifically including:

[0099] Main task loss: Cross-entropy loss L is used. cls (Cross-Entropy Loss) calculates the difference between the predicted fault category and the true label, using the following formula:

[0100] L cls =-Σ(Y i ·log(P i ));

[0101] Where Y i For the true label of the i-th sample (one-hot encoded form), P i This represents the predicted probability distribution output by the model.

[0102] Auxiliary task loss: Introducing a time distribution consistency regularization term L reg To ensure the consistency of feature representation across different time segments, the constraint model is achieved by calculating the mean cosine distance between the feature vectors of samples from high-density time periods and samples from low-density time periods. The formula is as follows:

[0103] L reg =1-mean(cos(f high ,f low ));

[0104] Where f high f is the set of feature vectors for high-density time period samples. low is the set of feature vectors for samples in low-density time periods, and cos() is the cosine similarity function;

[0105] Total loss function: L total =λ1·(Σ(α i ·L cls _i) / Σα i )+λ2·L reg Where λ1 and λ2 are task weight coefficients (λ1∈[0.7,0.9], λ2∈[0.1,0.3]), α i The time density coefficient is used to balance the learning priorities of the main task and auxiliary tasks through weighted balancing.

[0106] For example, a phased dynamic training strategy is adopted, combined with the adjustment of the time regularization coefficient, to balance the model's convergence efficiency and time generalization ability. The specific steps are as follows:

[0107] Pre-training phase (iterations 0~N1): Fixed time regularization coefficients λ=0.3, λ1=0.9, λ2=0.1, using a large learning rate (initial value 1e-3), parameter initialization is performed through stochastic gradient descent (SGD) or Adam optimizer, enabling the model to quickly learn the basic feature distribution of the data;

[0108] Reinforcement learning phase (iterations N1~N2): Linearly increase λ to 1.0, synchronously adjust λ1=0.8, λ2=0.2, reduce the learning rate to 5e-4, and introduce an early stopping mechanism. When the accuracy of fault diagnosis on the validation set does not improve for 5 consecutive epochs, the current stage of training is paused.

[0109] Fine-tuning convergence phase (iterations N2~N3): Keep λ=1.0, λ1=0.7, λ2=0.3, adopt a smaller learning rate (1e-4) and a learning rate decay strategy (decay by 10% every 10 epochs), use L2 regularization (weight decay coefficient 1e-5) to suppress overfitting, until the model's performance on the validation set is stable and converges (fluctuation <0.5%).

[0110] Through the above model structure design, multi-task loss function and dynamic training strategy, the deep neural network is efficiently trained and optimized on time-distributed data, ensuring that the model maintains stable fault diagnosis performance in metallurgical electromechanical equipment inspection scenarios across time periods and shifts.

[0111] Step S105: Obtain multimodal signal data of the electromechanical equipment to be diagnosed, preprocess the multimodal signal data and input it into the pre-trained deep neural network, and predict the fault type of the electromechanical equipment through the deep neural network.

[0112] In some embodiments, when acquiring multimodal signal data of the electromechanical equipment to be diagnosed, the type of signal collected is consistent with the multimodal signal data in step S101, including vibration signal, current signal, sound signal and temperature signal, and each sample data to be diagnosed has a corresponding timestamp.

[0113] For example, the preprocessing of the inspection multimodal signal data includes the following operations:

[0114] Signal filtering: The same filtering method as the training data preprocessing is used to perform bandpass filtering on vibration signal, current signal and sound signal to remove high frequency noise and low frequency interference. The filtering frequency band for vibration signal is 10Hz~5kHz, current signal is 50Hz±10Hz, and sound signal is 100Hz~10kHz.

[0115] Signal segmentation: The continuous signal is segmented using the same sliding window length as the training data (e.g., 0.5s~5s). The window sliding step size is 1 / 2~1 of the window length, resulting in signal sequence segments with the same dimension as the training samples, and the dimension is [time_steps, feature_dim].

[0116] Feature standardization: Based on the feature mean and standard deviation of the training data, the segmented signal sequence is standardized using the formula x. test '=(x test -μ) / σ, where x test This represents a fragment of the original signal sequence from the multimodal signal data of the equipment under inspection for diagnosis, x. test ' represents the transformed signal sequence segment, where μ is the mean of the corresponding feature in the training data and σ is the standard deviation of the corresponding feature in the training data;

[0117] The preprocessed signal sequence fragment to be diagnosed is input into a pre-trained deep neural network. After the deep neural network extracts and maps the features of the input signal, it outputs the predicted probability distribution of the fault type through the prediction output layer and takes the category with the highest probability as the fault type prediction result; or it outputs the probability value of the device operation abnormal through abnormal probability regression. When the probability value exceeds a preset threshold (such as 0.8), it is determined that the device is abnormal.

[0118] Figure 4 A fault diagnosis system 400 for electromechanical inspection equipment based on deep neural networks is shown. The system embodiment is similar to... Figure 1 Corresponding to the method embodiments shown, the system can be specifically applied to various electronic devices.

[0119] like Figure 4 As shown in the figure, an embodiment of this application provides a fault diagnosis system 400 for electromechanical inspection equipment based on deep neural networks, comprising:

[0120] The acquisition module 401 is used to acquire multimodal signal data of electromechanical equipment during operation over multiple time periods. Each sample data has a timestamp information. Combined with the operation and maintenance records, a corresponding fault marker is established to obtain training sample data.

[0121] The first construction module 402 is used to divide the training sample data into equal-length windows according to the timestamp, count the number of samples in each window, and construct a time density function to reflect the distribution density of the samples on the time axis.

[0122] The first embedding module 403 is used to perform time distribution regularization embedding during the pre-training of the deep neural network. Specifically, the time density function is introduced as a time regularization factor and applied to the sample weight adjustment process in the feature embedding layer. A penalty weight compression is applied to samples in high-density time periods, and the representation weights of samples in low-density time periods are enhanced during the training process.

[0123] The first training module 404 is used to supervise the training of fault categories based on training sample data and the time distribution regularized embedding, and dynamically adjust the time regularization coefficient during the training process to obtain a pre-trained deep neural network.

[0124] The output module 405 is used to acquire multimodal signal data of the electromechanical equipment to be diagnosed, preprocess the multimodal signal data and input it into the pre-trained deep neural network, and predict the fault type of the electromechanical equipment through the deep neural network.

[0125] Based on the same inventive concept, this application also provides an electronic device. The method corresponding to the electronic device can be the method in the foregoing embodiments, and its problem-solving principle is similar to that method. The electronic device provided in this application includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the methods and / or technical solutions of the foregoing embodiments of this application.

[0126] The electronic device can be a user device, or a device formed by integrating user devices and network devices through a network, or it can be an application running on the aforementioned devices. The user device includes, but is not limited to, various terminal devices such as computers, mobile phones, tablets, smartwatches, and smart bands. The network device includes, but is not limited to, network hosts, single network servers, multiple network server sets, or cloud computing-based computer sets, and can be used to implement some processing functions when setting an alarm clock. Here, the cloud consists of a large number of hosts or network servers based on cloud computing. Cloud computing is a type of distributed computing, consisting of a virtual computer composed of a group of loosely coupled computer sets.

[0127] Figure 5The diagram illustrates the structure of an apparatus suitable for implementing the methods and / or technical solutions in the embodiments of this application. The apparatus 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes based on a program stored in a read-only memory (ROM) 502 or a program loaded from a storage portion 508 into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0128] The following components are connected to I / O interface 505: input section 506 including keyboard, mouse, touch screen, microphone, infrared sensor, etc.; output section 507 including cathode ray tube (CRT), liquid crystal display (LCD), LED display, OLED display, etc., and speakers, etc.; storage section 508 including one or more computer-readable media such as hard disk, optical disk, magnetic disk, semiconductor memory, etc.; and communication section 509 including network interface card such as LAN (local area network) card, modem, etc. Communication section 509 performs communication processing via a network such as the Internet.

[0129] In particular, the methods and / or embodiments in this application can be implemented as computer software programs. For example, the embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. When the computer program is executed by the central processing unit (CPU) 501, it performs the functions defined in the methods of this application.

[0130] Another embodiment of this application provides a computer-readable storage medium having computer program instructions stored thereon, which can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of this application described above.

[0131] Specifically, this embodiment may employ any combination of one or more computer-readable media. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0132] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0133] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0134] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0135] Furthermore, it is clear that the word "including" does not exclude other units or steps, and the singular does not exclude the plural. Terms such as "first," "second," etc., are used to indicate names, but do not indicate any specific order.

Claims

1. A fault diagnosis method for electromechanical inspection equipment based on deep neural networks, characterized in that, Includes the following steps: Step S101: Collect multimodal signal data of electromechanical equipment during operation over multiple time periods. Each sample data has a timestamp information. Combine the operation and maintenance records to establish corresponding fault markers and obtain training sample data. Step S102: Divide the training sample data into equal-length windows according to timestamps, count the number of samples in each window, and construct a time density function to reflect the distribution density of samples on the time axis. Step S103: Temporal distribution regularization embedding is performed during the pre-training of the deep neural network. Specifically, the temporal density function is introduced as a temporal regularization factor and applied to the sample weight adjustment process in the feature embedding layer. A penalty weight compression is applied to the samples in the first density time period, and the representation weights of the samples in the second density time period are enhanced during the training process, wherein the first density is greater than the second density. Step S104: Based on the training sample data and the time distribution regularized embedding, a multi-task loss function is used to supervise the training of the fault categories. During the training process, the time regularization coefficient is dynamically adjusted to obtain a pre-trained deep neural network. Step S105: Obtain multimodal signal data of the electromechanical equipment to be diagnosed, preprocess the multimodal signal data and input it into the pre-trained deep neural network, and predict the fault type of the electromechanical equipment through the deep neural network.

2. The method for fault diagnosis of electromechanical inspection equipment based on deep neural networks according to claim 1, characterized in that, In step S102, when the training sample data is divided into equal-length windows according to timestamps, the length of the time window Δt is a preset range and is dynamically adjusted according to the characteristics of the device's operating cycle. All time windows begin with the earliest sample timestamp in the training dataset and end with the latest sample timestamp. Each window is labeled W. k ; When counting the number of samples in each window, iterate through all samples and count them based on the sample timestamp T. i Determine the window it belongs to, W k The total number of samples n for each window is obtained by statistics. k ; When constructing the time density function ρ(t), first calculate the average sample density ρ for each window. k =n k / Δt, then use the kernel density estimation method to estimate ρ k Smoothing is performed, a Gaussian kernel is selected as the kernel function, and the bandwidth parameter is adaptively adjusted according to the density of the sample time distribution to obtain a continuous function ρ(t) defined on the time axis.

3. The method for fault diagnosis of electromechanical inspection equipment based on deep neural networks according to claim 2, characterized in that, In step S103, the time distribution regularization embedding includes the following steps: Step S201, calculate the time density coefficient α i Specifically, for the i-th sample in the training set, based on its timestamp T i Querying the time density function ρ(t) yields the density value ρ(T). i ), for ρ(T) i Normalization yields ρ'(T) i )=ρ(T i ) / max(ρ(t)), where max(ρ(t)) is the maximum value of ρ(t) over the entire time axis. , For positive numbers ranging from 1e-6 to 1e-4, such that α i ∈(0,1]; Step S202, input weighted embedding, specifically, the original input feature x of the i-th sample. i via α i x' is obtained by weighting elements one by one i =α i ·x i , x' i Passed into subsequent network layers; Step S203, dynamically weight the loss function, specifically, the loss L for the i-th sample... i Multiply by α i Total loss L total =Σ(α i ·L i ) / Σα i , where Σα i This is the normalization term.

4. The method for fault diagnosis of electromechanical inspection equipment based on deep neural networks according to claim 3, characterized in that, In step S104, the deep neural network includes a feature processing layer, a temporal feature extraction layer, a feature fusion layer, and a prediction output layer; the feature processing layer embeds the modulation feature x' of the output in a time-distributed regularized manner. i Batch normalization is performed; the temporal feature extraction layer uses a 1-3 layer bidirectional gated recurrent unit or a bidirectional long short-term memory network; the feature fusion layer compresses the temporal dimension through global average pooling, and then performs feature mapping and fusion through a 1-2 layer fully connected layer; the prediction output layer is a fully connected layer using the Softmax activation function, and the output dimension is consistent with the total number of fault categories. The multi-task loss function is L total =λ1·(Σ(α i ·L cls _i) / Σα i )+λ2·L reg L cls _i is the cross-entropy loss of the i-th sample, L reg λ1 and λ2 are the time distribution consistency regularization terms, and the task weight coefficients in the multi-task loss function.

5. The method for fault diagnosis of electromechanical inspection equipment based on deep neural networks according to claim 2, characterized in that, The rationality of the stated time density function ρ(t) is verified, and the actual number of samples n in each window is calculated. k The relative error between the theoretical sample number obtained based on the integral of ρ(t) and the error exceeds the preset threshold. If the error exceeds the preset threshold, the window length or bandwidth parameter will be readjusted. If ρ(t) suddenly becomes 0 in a certain window, the ρ(t) of that window will be corrected by interpolation to ensure the continuity of the density function.

6. The method for fault diagnosis of electromechanical inspection equipment based on deep neural networks according to claim 3, characterized in that, The step of performing time distribution regularized embedding further includes step S204, which dynamically adjusts the time regularization coefficient λ. Specifically, in the early stage of training, λ is set to a first preset value to weaken the weighting effect of time distribution regularized embedding, so that the model prioritizes learning the overall distribution characteristics of the data. In the middle stage of training, λ is gradually increased to 1.0 to enhance the adjustment strength of time distribution regularized embedding on sample weights and strengthen the learning of sparse time segment features. In the later stage of training, λ is kept at 1.0 to ensure that the model converges under the constraint of balanced time distribution.

7. A fault diagnosis system for electromechanical inspection equipment based on deep neural networks, characterized in that, include: The acquisition module is used to collect multimodal signal data of electromechanical equipment during operation over multiple time periods. Each sample data has a timestamp information. Combined with operation and maintenance records, corresponding fault markers are established to obtain training sample data. The first construction module is used to divide the training sample data into equal-length windows according to timestamps, count the number of samples in each window, and construct a time density function to reflect the distribution density of samples on the time axis. The first embedding module is used to perform time distribution regularized embedding during the pre-training of the deep neural network. Specifically, the time density function is introduced as a time regularization factor and applied to the sample weight adjustment process in the feature embedding layer. A penalty weight compression is applied to the samples in the first density time period, and the representation weights of the samples in the second density time period are enhanced during the training process, wherein the first density is greater than the second density. The first training module is used to supervise the training of fault categories based on training sample data and the time distribution regularized embedding, and dynamically adjust the time regularization coefficient during the training process to obtain a pre-trained deep neural network. The output module is used to acquire multimodal signal data of the electromechanical equipment to be diagnosed, preprocess the multimodal signal data and input it into the pre-trained deep neural network, and predict the fault type of the electromechanical equipment through the deep neural network.

Images