Fault quantitative diagnosis method for multi-class imbalance based on improved per and deep reinforcement learning

Abstract

The application discloses a multi-class imbalance fault quantitative diagnosis method based on improved PER and deep reinforcement learning. In view of the problems of low precision and poor generalization of intelligent fault quantitative diagnosis of rotating machinery under long-tail distribution, the PER-EADRL new paradigm of D3QN based on improved prioritized experience replay (PER) is proposed based on deep reinforcement learning. Firstly, the priority of experience data is evaluated by using reward value and TD-error (Temporal-difference error), and high-value information is efficiently reused; secondly, a deep Q network is constructed by taking ResNet as a backbone network and deeply integrating ECA attention and CBAM attention mechanisms, and the high-order discriminative features of potential space are explored in continuous trials; meanwhile, a reward mechanism is constructed based on the imbalance rate, and the real-time accuracy is used as model feedback to conservatively fine-tune the reward function; further, a high-quality virtual environment is constructed by using synchronous compression wavelet transform to obtain a time-frequency graph. Finally, the CWRU data is verified, and the method can realize self-accurate fault quantitative identification under class imbalance conditions.

Description

Technical Field

[0001] This invention belongs to the field of quantitative diagnosis of rolling bearing faults under unbalanced distribution, specifically involving a quantitative diagnosis method for multiple unbalanced faults based on improved PER and deep reinforcement learning. Background Technology

[0002] In the context of Industry 4.0, rotating machinery plays a key role in industries such as intelligent manufacturing, rail transportation, and aerospace. The ability to accurately and quickly diagnose faults in mechanical transmission systems is crucial for ensuring the high stability and reliability of mechanical equipment and avoiding huge economic losses and serious catastrophic accidents.

[0003] In the era of the Internet of Everything, data-driven next-generation artificial intelligence empowers technological leapfrog development and industrial optimization and upgrading. However, industrial data exhibits a long-tail distribution, meaning that healthy data far outweighs fault data, fault types are diverse, and fault symptoms are complex, making accurate identification and prediction of various fault anomalies challenging. Research on class-imbalanced fault diagnosis has strong application value and practical significance for effectively processing real industrial data. Addressing the scarcity problem of typical faults under long-tail distribution, current research mainly falls into three categories: class rebalancing, information augmentation, and module design. Class rebalancing strategies attempt to rebalance the number of samples during training. In terms of data, this includes oversampling, undersampling, and mixed sampling techniques; from an algorithmic perspective, cost-sensitive learning aims to reduce the impact of class imbalance by adjusting the weights of different class objective functions. Information augmentation aims to introduce additional information during training to solve the class-imbalanced classification problem; current research mainly focuses on generative learning and transfer learning. Broadly speaking, improving or designing network strategies can be divided into four categories: metric learning, feature extractor design, classifier design, and ensemble learning.

[0004] With the accelerated innovation and vigorous development of emerging technologies such as the Industrial Internet of Things and artificial intelligence, the health status monitoring of key objects such as rolling bearings has entered the era of big data and intelligence. A large amount of industrial data has been recorded, providing strong support for a comprehensive understanding of the health and service status of mechanical equipment groups, and injecting continuous power into promoting the high-quality development of related enterprises with "specialization, refinement, and innovation". Therefore, carrying out data-driven fault diagnosis is of great significance for the health inspection of large mechanical equipment. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a quantitative diagnosis method for multi-class imbalanced faults based on improved PER and deep reinforcement learning, which solves the problem that the imbalance between the number of fault samples and normal samples seriously affects the performance of intelligent fault diagnosis models in the prior art.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] 1. A quantitative diagnosis method for multi-class imbalance faults based on improved PER and deep reinforcement learning, comprising the following steps:

[0008] S1. Signal acquisition and preprocessing: Obtain raw vibration data of rolling bearings under different health conditions and different degrees of fault damage, divide the vibration data in a non-overlapping manner, normalize the segmented signal samples, and construct initial training samples and test samples.

[0009] S2. Time-frequency graph generation: The initial training samples and test samples are subjected to synchronous compressed wavelet transform to obtain a two-dimensional time-frequency graph, which is then normalized. An unbalanced training set and test set are constructed by randomly deleting samples.

[0010] S3. Establish a quantitative diagnostic model for multi-class imbalanced faults based on improved PER and deep reinforcement learning. This model includes a deep Q-network, a reward function, an experience pool, a priority experience replay mechanism, and a state transition function. A deep Q-network is constructed by fusing ResNet with ECA and CBAM modules. A curiosity-driven two-stage reward function is constructed based on the imbalance ratio and accuracy. The priority experience replay mechanism is improved based on the reward function and TD-error. An experience pool is constructed for reusing experience data. The state transition function randomly assigns states according to the sample class order.

[0011] S4. Construct a virtual environment based on the training set, and obtain an intelligent fault quantitative diagnosis model through cyclical interaction between the intelligent agent and the virtual environment.

[0012] S5. The intelligent fault quantitative diagnosis model is validated using the application test set.

[0013] In S3, the quantitative diagnostic model for multi-class imbalanced faults based on improved PER and deep reinforcement learning includes a state space S, an action space A, and a reward function R. The state space S represents an imbalanced distribution training set constructed from multi-channel time-frequency maps, where each environmental state s... t Each corresponds to a training sample; the action space A corresponds to K categories of rolling bearing health states, A = {0, 1, 2, ..., K-1}, where K represents the number of categories; the curiosity-driven two-stage reward function R is:

[0014] R(s t ,a t )=λR1(s t ,a t )+R2(s t ,a t )

[0015] In the formula, R1(s)t ,a t R2(s) is the first-stage reward function. t ,a t ) is the second-stage reward function, and λ is an adjustable hyperparameter used to balance the reward weights.

[0016] λ is an adjustable hyperparameter used to balance reward weights; in this method, λ = 1.

[0017] R1(s t ,a t ) is the first-stage reward function, a t It is the action performed by the agent at time t, y t It is state s t The true label, t is the time step for training a multi-class imbalance fault quantitative diagnosis model based on PER-ECADRL and deep reinforcement learning, R1(s) t ,a t The following formula is used for calculation:

[0018]

[0019] R2(s t ,a t The second-stage reward function is calculated using the following formula:

[0020]

[0021] In the formula, ρ is the unbalance ratio, |D F | represents the total number of fault samples, |D N | represents the number of all correct samples, D F D represents all fault samples. N σ represents all normal samples, and σ is the feedback of the second-stage reward function.

[0022] σ is the feedback of the second-stage reward function, which is a vector with the same dimension as the number of categories. It is the feedback component σ of the reward function for the j-th class. j It is designed based on the accuracy of the training set and is calculated using the following formula:

[0023]

[0024] In the formula, This represents the number of correctly predicted samples in the j-th class. This represents the number of all samples in class j.

[0025] In S3, the priority experience replay mechanism is improved based on the reward function and TD-error. The core part ρ of the reward function is integrated into the priority ranking to increase the reuse frequency of small samples. The improvement is based on ranking-based priority, calculated using the following formula:

[0026]

[0027] In S3, the input to the deep Q-network is a time-frequency map. The input information first undergoes a 7×7 convolution to initially extract fault features. The resulting feature vectors are then passed through a batch normalization layer, a ReLU activation layer, and a max-pooling layer to eliminate differences in fault feature distribution, enhance the nonlinear expression of fault features, and filter redundant information. Then, complex nonlinear fault features are extracted layer by layer through four deep residual blocks that fuse ECA channel attention, and key saliency features are weighted and fused. Finally, the output features are processed by average pooling to reduce the computational cost of the model, and a plug-and-play CBAM lightweight attention module is embedded to refocus key features in the channel and spatial dimensions. The model output is the sum of the outputs of two fully connected layers. The structure of the deep Q-network is as follows: Figure 1 As shown.

[0028] S4 includes the following steps:

[0029] S4.1: Set the maximum number of autonomous training iterations (Episode). During each autonomous training iteration, the virtual environment initiates T diagnostic requests, each corresponding to an environment state (s). t ;

[0030] S4.2: The virtual environment randomly initiates a diagnostic request; the current state is s. t ∈S, s t Inputting a deep Q-network, the agent selects the current action 'a' according to a dynamic ε-greedy algorithm. t ∈A, the virtual environment returns a quantified reward r to the agent based on a curiosity-driven two-stage reward function. t ∈R, and randomly initiate the next diagnostic request, that is, according to the state transition function, transition to the next state s. t+1 The resulting empirical data e t ={s t ,a t ,r t ,s t+1 Store it in the experience pool with the highest priority;

[0031] S4.3: Repeat S4.2 until the T diagnostic requests in this round end, and output the cumulative reward obtained in this iteration;

[0032] S4.4: After the experience pool is full, randomly sample batches of experience data E={e0,e1,e2,…,e...} from the experience pool.n-1 Based on empirical data E={e0,e1,e2,…,e n-1 The gradient descent method is used to update the online deep Q network, and the hard update method is used to update the target deep Q network at an update frequency C. The priority of updating empirical data is based on the reward function and TD-error.

[0033] S4.5: Begin the next round of autonomous training. Repeat steps S4.2 to S4.4 until the maximum number of autonomous training rounds is reached, at which point the autonomous training process ends.

[0034] S4.6: Save the model parameters obtained from each round of diagnostic requests during autonomous training with an accuracy rate higher than a certain threshold, as the intelligent fault quantitative diagnostic model learned by the agent.

[0035] S4.2 includes the following steps:

[0036] S4.2.1: Initialize ε = 1, attenuation rate ε decay =0.998, minimum value ε min =0.02;

[0037] S4.2.2: Issue the predicted action a t Beforehand, the intelligent quantitative diagnostic model randomly generates a random number between [0, 1]. If the random number belongs to the range [0, ε], a prediction action a is randomly selected from the action space A. t If the random number belongs to the range (ε, 1], the predicted action corresponding to the maximum Q value output by the current depth Q-network is the action to be executed, a. t ;

[0038] S4.2.3: After each diagnostic request ends, dynamically iterate and update the ε value. If ε is less than ε min , ε=ε min Otherwise ε = ε × ε decay .

[0039] The original vibration signal is segmented to construct initial training samples and test samples, each containing 1650 data points.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1. As a cutting-edge direction in artificial intelligence, deep reinforcement learning fully integrates the perception capabilities of deep learning and the decision-making advantages of reinforcement learning, and is a precise and efficient end-to-end fault diagnosis model.

[0042] 2. Efficiently utilize existing scarce high-value samples, and use TD-error (Temporal difference error) and reward function value as priority measurement criteria to improve the efficiency of experience replay and reduce the number of interactions.

[0043] 3. ResNet is used to replace the deep Q network to fully perceive key environmental information. The embedded ECA and CBAM modules adaptively assign weights to the learned complex nonlinear fault features, thereby improving the robustness of the model.

[0044] 4. Adapt to imbalanced datasets and design a specific two-stage reward function. By influencing the agent's decision-making ability through cumulative rewards, the agent is guided to fully explore the optimal strategy, thereby improving the diagnostic accuracy of the fault diagnosis model for minority classes.

[0045] 5. The proposed method PER-ECADRL was validated using the CWRU bearing dataset. Experiments show that the method achieves accurate and efficient quantitative fault identification even under extreme class imbalance conditions. Attached Figure Description

[0046] Figure 1 This is the network structure of the deep Q-network in the method of this invention.

[0047] Figure 2 This is a flowchart of the quantitative fault diagnosis process of the present invention.

[0048] Figure 3 Comparison of the results of hierarchical tri-fold cross-validation for all methods.

[0049] Figure 4 Normalized confusion matrix diagram of the method proposed in this invention.

[0050] Figure 5 A 3D feature visualization of the method proposed in this invention. Detailed Implementation

[0051] The structure and working process of the present invention will be further described below with reference to the accompanying drawings.

[0052] To address the challenges of traditional intelligent fault diagnosis models, such as low accuracy, poor robustness, and poor versatility under imbalanced data distribution, this paper proposes a new paradigm for quantitative fault diagnosis (PRE-ECADRL) based on Deep reinforcement learning (DRL), which utilizes Prioritized experience replay (PRE) and a lightweight attention mechanism, namely D3QN (Dueling Double Deep Q Network). First, a 2D multi-channel time-frequency map is obtained using Synchrosqueezed Wavelet Transform (SWT) to effectively construct a high-quality virtual environment. Second, a deep Q-network is constructed using ResNet as the backbone network and deeply integrating Efficient Channel Attention (ECA) and Convolutional Block Attention Module (CBAM). This network learns high-order discriminative features of the latent space through continuous trial and error, effectively improving sample efficiency and agent reasoning ability. Simultaneously, it adapts to data distribution and cleverly constructs a curiosity-driven two-stage reward function based on imbalanced ratios (IRs). Accuracy is used as real-time feedback to conservatively fine-tune the reward function, promptly correcting the agent's decision-making direction. TD-error (Temporal-difference error) and timely reward value are used as criteria for judging the importance of experience, prioritizing experience pool data and efficiently reusing rare experiences. These components are used to construct the agent's mental model, providing a rational basis for the agent's enlightenment and accelerating the agent's advanced evolution. Finally, the proposed method PRE-EADRL was experimentally validated using an experimental dataset. This method autonomously and independently achieves accurate quantitative fault identification under class-imbalanced operating conditions, and has achieved significant improvements in performance, data efficiency, and generalization ability.

[0053] A quantitative diagnosis method for multi-class imbalance faults based on improved PER and deep reinforcement learning includes the following steps:

[0054] S1. Signal acquisition and preprocessing: Obtain raw vibration data of rolling bearings under different health conditions and different degrees of fault damage, divide the vibration data in a non-overlapping manner, normalize the segmented signal samples, and construct initial training samples and test samples.

[0055] S2. Time-frequency graph generation: The initial training samples and test samples are subjected to synchronous compressed wavelet transform to obtain a two-dimensional time-frequency graph, which is then normalized. An unbalanced training set and test set are constructed by randomly deleting samples.

[0056] S3. Establish a quantitative diagnostic model for multi-class imbalanced faults based on improved PER and deep reinforcement learning. The model includes a deep Q-network, a reward function, an experience pool, a priority experience replay mechanism, and a state transition function. A deep Q-network is constructed by fusing ResNet with ECA and CBAM modules. A curiosity-driven two-stage reward function is constructed based on the imbalance ratio and accuracy. An experience pool for reusing experience data is constructed based on the reward function and TD-error. The state transition function randomly gives the state according to the order of sample categories.

[0057] S4. Construct a virtual environment based on the training set, and obtain an intelligent fault quantitative diagnosis model through cyclical interaction between the intelligent agent and the virtual environment.

[0058] S5. The intelligent fault quantitative diagnosis model is validated using the application test set.

[0059] Specific embodiments, such as Figures 1 to 5 As shown,

[0060] The quantitative diagnostic model for multi-class imbalance faults based on improved PER and deep reinforcement learning includes the following steps:

[0061] 1) Data Acquisition and Preprocessing

[0062] SWT is a novel method similar to EMD. SWT redistributes the spectrum after wavelet transform, applying the principle that phase is unaffected by scale changes after wavelet transform. It adds scales of the same frequency and compresses wavelet coefficients around the same frequency. Finally, it transforms the spectrum into a time-frequency plane through a special mapping relationship, obtaining a time-frequency representation of the coefficient set. This method reduces spectral ambiguity caused by wavelet energy diffusion, sharpens the time-frequency resolution, and allows for further study of the internal composition of non-steady-state signals. It is suitable for analyzing rolling bearing fault signals under complex operating conditions. Its basic formula is:

[0063]

[0064]

[0065]

[0066] In the formula, the continuous wavelet transform of signal u(t) is W u (a,b), where ξ represents the angular frequency. Let denot ω represent the Fourier transform, A be the modulus of the signal u(t), and ψ represent the wavelet basis, which tends to zero in the negative frequency domain. u (a, b) represents the instantaneous frequency. TFR u (ω l b) represents synchronous compression transformation, ω l Representing l discrete angular frequencies, a k For the k-th discrete scale, Δω = ω k -ω k-1 ,(△a) k =a k -a k-1 The result of synchronous compression transformation depends on the continuous interval [ω]. l -(1 / 2)△ω,ω l The center of +(1 / 2)△ω] ω l It compresses the ambiguity of the scale / frequency direction of wavelet transform into a concentrated region, thereby significantly improving the readability of the time-frequency distribution.

[0067] The obtained TFRs are normalized to further improve data quality. An imbalanced dataset is constructed by randomly deleting a certain number of fault samples.

[0068] 2) Autonomous training of intelligent agents

[0069] In each game iteration, the environment randomly generates T states. At each time step t, the virtual environment follows the state transition function P(s). t+1 ,a t+1 |s t ,a t ) Generate state s t Interacting with the agent, the agent acts according to the current policy π θ and state s t Make an action a t Environmental assessment action a t The accuracy of the result is used to provide a quantitative reward to the intelligent agent according to the reward rules. t A two-way feedback mechanism is formed between the agent and the environment, and the environment then responds again based on P(s). t+1 ,a t+1 |s t ,a t Randomly generate state s t+1 and the above data {s t ,a t ,r t ,s t+1 The experience pool B is stored with the highest priority. By continuously iterating the above process, the agent learns the optimal behavioral strategy that is both robust and practical by maximizing the accumulated rewards. Once the experience pool B is full, a small batch of data is randomly sampled for experience replay to reduce sequence correlation and improve sampling efficiency. Model updates are achieved through gradient descent combined with the Adam optimization algorithm, updating the priority of experience pool data based on the reward function and TD-error. The PER-ECADRL diagnostic model is iteratively trained until the network converges, and the optimal diagnostic strategy is saved for online testing.

[0070] S4 includes the following steps:

[0071] S4.1: Set the maximum number of autonomous training iterations (Episode). During each autonomous training iteration, the virtual environment initiates T diagnostic requests, each corresponding to an environment state (s). t ;

[0072] S4.2: The virtual environment randomly initiates a diagnostic request; the current state is s. t ∈S, s t Inputting a deep Q-network, the agent selects the current action 'a' according to a dynamic ε-greedy algorithm. t ∈A, the virtual environment returns a quantified reward r to the agent based on a curiosity-driven two-stage reward function. t ∈R, and randomly initiate the next diagnostic request, that is, according to the state transition function, transition to the next state s. t+1 The resulting empirical data e t ={s t ,a t ,r t ,s t+1 Store it in the experience pool with the highest priority;

[0073] S4.3: Repeat S4.2 until the T diagnostic requests in this round end, and output the cumulative reward obtained in this iteration;

[0074] S4.4: After the experience pool is full, randomly sample batches of experience data E={e0,e1,e2,…,e...} from the experience pool. n-1 Based on empirical data E={e0,e1,e2,…,e n-1 The gradient descent method is used to update the online deep Q network, and the hard update method is used to update the target deep Q network at an update frequency C. The priority of updating empirical data is based on the reward function and TD-error.

[0075] S4.5: Begin the next round of autonomous training. Repeat steps S4.2 to S4.4 until the maximum number of autonomous training rounds is reached, at which point the autonomous training process ends.

[0076] S4.6: Save the model parameters obtained from each round of diagnostic requests during autonomous training with an accuracy rate higher than a certain threshold, as the intelligent fault quantitative diagnostic model learned by the agent.

[0077] S4.2 includes the following steps:

[0078] S4.2.1: Initialize ε = 1, attenuation rate ε decay =0.998, minimum value ε min =0.02;

[0079] S4.2.2: Issue the predicted action a t Beforehand, the intelligent quantitative diagnostic model randomly generates a random number between [0, 1]. If the random number belongs to the range [0, ε], a prediction action a is randomly selected from the action space A. t If the random number belongs to the range (ε, 1], the predicted action corresponding to the maximum Q value output by the current depth Q-network is the action to be executed, a. t ;

[0080] S4.2.3: After each diagnostic request ends, dynamically iterate and update the ε value. If ε is less than ε min , ε=ε min Otherwise ε = ε × ε decay .

[0081] The original vibration signal is segmented to construct initial training samples and test samples, each containing 1650 data points.

[0082] A computer storage medium storing computer instructions, which, when invoked, are used to execute all or part of the steps of the method.

[0083] D3QN, combining Double DQN and Dueling DQN, is a robust algorithm that jointly learns, explores, and makes decisions based on DQN. Assume that at time t, the agent perceives the current state s. t ∈S, according to strategy π:S→A, execute action a t ∈A, the degree of correctness of the environmental assessment action is fed back to the agent as a reward r. t ∈R, then the environment is determined according to the state transition function P(s) t+1 |s t ,a t The next state s is given randomly. t+1 The above process is continuously updated and iterated, and the agent obtains the optimal policy π by maximizing the cumulative reward. * , can be represented as:

[0084]

[0085] In the formula, This represents the cumulative reward, where T is the termination time step, and r is the cumulative reward. t′ γ is the reward obtained at time t′, where t′ represents the next time step, and γ is a weighted parameter used to measure real-time rewards and delayed rewards.

[0086] In D3QN, by introducing a Dueling network, the neural network no longer directly outputs the Q-value, but instead outputs the state-value function V(s; θ). V ) and the advantage function A(s,a;θ) A The summation of ) and the action value function Q(s,a;θ) is modeled as:

[0087] Q(s,a;θ)=V(s;θ) V )+A(s,a;θ A (5)

[0088] In the formula, θ is the parameter of the Q-network, and θ V and θ A V(s); θ V ) and A(s,a;θ A The parameters of ).

[0089] The above formula suffers from the problem that the modeling of V and A values ​​is not unique, which can lead to fluctuations in model training. To address this issue, Dueling DQN forces the advantage function of the optimal action to actually output 0, i.e.:

[0090]

[0091] In the formula, |A| represents the action dimension. Subtracting the average value from the advantage function makes the average of all advantage functions zero, yielding the unique value function V(s). Although the above formula does not satisfy the Bellman optimality equation, it is relatively stable in practical applications.

[0092] Combining DDQN, the optimal estimate of D3QN can be expressed as:

[0093]

[0094] Finally, by randomly drawing a small batch of empirical data E = {e0, e1, e2, ... e} from the experience pool B M-1 The model parameters are updated by minimizing the mean square error in each iteration. The derivative of the loss function L(θ) is shown in the following equation, and the target Q network is hard-updated with a delay of N steps.

[0095]

[0096] In the formula, e~U(B) represents random uniform sampling of the experience pool.

[0097] 3) Online testing

[0098] The PER-ECADRL optimal diagnostic strategy was used for online fault diagnosis. The fault diagnosis model trained on unbalanced rolling bearing data was used to identify the fault type of the current sample.

[0099] The following simulation experiment, which involves qualitative and quantitative diagnosis of rolling bearing faults, will provide a detailed explanation of this scheme.

[0100] I. Experimental Setup:

[0101] Experimental data were provided by Case Western Reserve University. The bearing under test was driven by a three-phase AC motor with a power of approximately 1.5 kW. The experiment used a vibration signal with a drive-end load of 1 hp and a sampling frequency of 48 kHz, with each sample containing 1650 data points. All faults were single-point faults. Healthy states included normal (N), outer raceway fault (OF), inner raceway fault (IF), and roller fault (RF), with damage diameters of 0.007 inches, 0.014 inches, and 0.021 inches for each fault. In summary, 10 different rolling bearing operating states were designed, abbreviated as IR7, IR14, IR21, OR7, OR14, OR21, RF7, RF14, RF21, and N. The pixel texture shape and color of the normalized TFRs are significantly different, providing richer information than the original vibration signal, making them suitable for quantitative fault diagnosis under imbalanced data conditions. The constructed imbalanced dataset is shown in Table 1.

[0102] Table 1. Settings for the imbalanced dataset

[0103]

[0104] II. Comparison Methods

[0105] To objectively evaluate the performance of the PRE-ECADRL method, benchmark comparison experiments were conducted against several mainstream imbalanced classification algorithms, including SVM, CLUSTER, SMOTE, Focal Loss, and DQN. Ablation experiments were performed to illustrate the effectiveness of the overall improved model, the curiosity-driven two-stage reward function, adaptive data-first experience replay, and deep Q-networks, respectively denoted as D3QN, REWARD, ECA, and PER. CNNs, with their sparse connectivity and weight-sharing characteristics, used the cross-entropy loss function as the basic classifier.

[0106] SVM uses 10 typical time-domain statistical indicators as input and GridSearchCV is used for optimal parameter grid search. CLUSTE, as a general downsampling algorithm, balances the dataset by randomly reducing negative samples. SMOTE calculates the distance between a specific minority class sample and its nearest neighbor based on Euclidean theorem, and improves data balance by constructing new minority class samples. Based on the binary cross-entropy loss function, Focal Loss improves the model's imbalanced classification performance by dynamically reducing the weights of easily distinguishable samples during training and quickly focusing the algorithm's focus on difficult-to-classify samples. DQN uses an imbalance ratio to design the reward function, with other settings the same as the original model. The reward function of the D3QN method is set as follows: +1 point if the agent guesses the sample class correctly; -1 point if it does not guess correctly. The state transition function is set to randomly generate the next state. REWARD: To verify the effect of the curiosity-driven two-stage reward function, the curiosity-driven two-stage reward function module is removed from the proposed method, and the reward function is set as follows: +1 reward if guessed correctly; -1 reward otherwise. The state transition function is set to randomly generate the next state, and the priority of empirical replay is sorted according to TD-error, while other modules remain unchanged; ECA: In order to verify the necessity of the proposed improved deep Q network, the original CNN network is used instead, while other modules remain unchanged; PER: In order to verify the effectiveness of the proposed adaptive data-priority replay, the adaptive data-priority empirical replay is removed from the proposed method, while other modules remain unchanged.

[0107] III. Experimental Results and Analysis:

[0108] The results of 10 comparative and ablation tests are shown below. Figure 3The following conclusions can be drawn: (1) The proposed method improves the average F1-score by more than 26.76% compared with SVM, reaching 96.90%; (2) The average F1-score of the proposed method is improved by more than 5.16%, 1.94%, 2.97%, and 8.46% compared with CLUSTER, SMOTE, Focal Loss, and DQN, respectively. DQN has large fluctuations, and the RL algorithm is time-consuming. (3) In the three imbalanced scenarios, the proposed method improves the F1-score by 5.06%, 6.62%, and 16.31% compared with D3QN, respectively. (4) The proposed method improves the F1-score by 78.48%, 75.50%, and 75.47% compared with REWARD. This shows that the optimized state transition function and curiosity-driven reward function greatly improve the model performance. (5) The proposed method improves the average F1-score by 6.95% and reduces Std by 1.01% compared with ECA. It can be seen that the designed deep Q network effectively enhances the agent's ability to identify key fault symptoms. (6) The proposed method improves the PER F1-score by 0.067%, 2.99% and 1.79% respectively. Adaptive data PER greatly shortens the model time while taking into account the robustness of the model.

[0109] With an IR ratio of 1:10, the specific classification of each health category is given by the confusion matrix, such as... Figure 4 As shown, the recognition rates for RF7 and RF21 are relatively low, at 60% and 68%, respectively. 24% of RF7 samples were misclassified as OR14, and 20% of RF21 samples were misclassified as IR14. There is some similarity among the misclassified time-frequency plots. The proposed method demonstrates excellent performance in predicting samples of other categories.

[0110] IV. Visualizing High-Dimensional Features:

[0111] To demonstrate the feature adaptation performance of the proposed method, t-SNE is used to reduce the dimensionality of the output features, such as... Figure 5 As shown, the features extracted by the proposed method have relatively clear decision boundaries, are very compact within the same category, and are relatively dispersed between different categories, although some features overlap in a small number of samples. This clearly demonstrates that the highly abstracted hidden features obtained through PRE-ECADRL have good discriminative power.

[0112] A computer storage medium storing computer instructions, which, when invoked, are used to execute all or part of the steps of the method.

[0113] Those skilled in the art should understand that variations can be implemented by combining existing technology and the above embodiments. Such variations do not affect the substantive content of this solution and will not be elaborated here.

[0114] It should be understood that this solution is not limited to the specific embodiments described above. Devices and structures not described in detail herein should be understood as being implemented in a manner common to the art. Any person skilled in the art can make many possible variations and modifications to this solution, or modify it into equivalent embodiments, without departing from the scope of this solution, using the methods and techniques disclosed above. This does not affect the substantive content of this solution. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of this solution, without departing from its scope, still fall within the protection scope of this solution.

Claims

1. A quantitative diagnosis method for multiple types of imbalanced faults based on improved priority experience replay (PER) and deep reinforcement learning, characterized by: Includes the following steps: S1. Signal acquisition and preprocessing: Obtain raw vibration data of rolling bearings under different health conditions and different degrees of fault damage, divide the vibration data in a non-overlapping manner, normalize the segmented signal samples, and construct initial training samples and test samples. S2. Time-frequency graph generation: The initial training samples and test samples are subjected to synchronous compressed wavelet transform to obtain a two-dimensional time-frequency graph. An unbalanced training set and test set are constructed by randomly deleting samples. S3. Establish a quantitative diagnostic model for multi-class imbalance faults based on improved PER and deep reinforcement learning. The quantitative diagnostic model for multi-class imbalance faults based on improved PER and deep reinforcement learning includes a state space. Action space reward function Wherein, the state space S represents the unbalanced distribution training set constructed from the multi-channel time-frequency graphs, and each environmental state The corresponding training samples; the action space A corresponds to the K categories of rolling bearing health states. Where K represents the number of classification categories; the basic components of the quantitative diagnosis model for multi-class imbalanced faults based on improved PER and deep reinforcement learning include a deep Q-network, a reward function, an experience pool, adaptive data-first experience replay, and a state transition function; a deep Q-network is constructed by fusing ResNet with ECA and CBAM modules; a reward function is constructed based on the imbalance ratio and accuracy; an adaptive data-first experience replay mechanism is constructed based on the reward function and temporal difference error (TD-error); an experience pool is constructed for reusing experience data; the state transition function randomly assigns states according to the order of sample categories; the reward function R is: In the formula, It is an adjustable hyperparameter used to balance reward weights. This is the first-stage reward function. It is the second-stage reward function. It is an adjustable hyperparameter used to balance reward weights. It is the agent's response to the state at time t. The action performed. S4. Construct a virtual environment based on the training set, and obtain an intelligent fault quantitative diagnosis model through cyclical interaction between the intelligent agent and the virtual environment. S5. The intelligent fault quantitative diagnosis model is validated using the application test set.

2. The quantitative diagnosis method for multi-class imbalance faults based on improved PER and deep reinforcement learning according to claim 1, characterized in that: In S3, This is the first-stage reward function. It is the action taken by the agent at time t. It is a state The true label is t, where t is the time step for training a quantitative diagnostic model for multi-class imbalance faults based on PER-ECADRL and deep reinforcement learning. The following formula is used for calculation: 。 3. The quantitative diagnosis method for multi-class imbalance faults based on improved PER and deep reinforcement learning according to claim 1, characterized in that: In S3, This is the second-stage reward function, calculated using the following formula: In the formula, For the unbalanced ratio, Represents the total number of fault samples. This represents the total number of all correct samples. Represents all fault samples, Represents all normal samples. This is the feedback from the second-stage reward function.

4. The quantitative diagnosis method for multiple imbalance faults based on improved PER and deep reinforcement learning according to claim 1, characterized in that: It is the feedback of the second-stage reward function, i.e., a vector with the same dimension as the number of categories, and the feedback component of the reward function for the j-th class. It is designed based on the accuracy of various types of training sets and is calculated using the following formula: In the formula, This represents the number of correctly predicted samples in the j-th class. This represents the number of all samples in class j.

5. The quantitative diagnosis method for multi-class imbalance faults based on improved PER and deep reinforcement learning according to claim 1, characterized in that: In S3, based on the reward function and the TD-error improved priority experience replay mechanism, the core part of the reward function is... Integrating this into priority ranking improves the reuse frequency of small samples. Improvements are made based on ranking-based priority, calculated using the following formula: 。 6. The quantitative diagnosis method for multiple imbalance faults based on improved PER and deep reinforcement learning according to claim 1, characterized in that: In step S3, the input to the deep Q-network is a time-frequency graph, and the input information first passes through... Convolutional processing extracts fault features, and the resulting feature vectors are sequentially passed through batch normalization layers, ReLU activation layers, and pooling layers to eliminate differences in fault feature distribution, enhance the nonlinear expression of fault features, and filter redundant information. Then, complex nonlinear fault features are extracted layer by layer through four deep residual blocks that fuse ECA channel attention, and key saliency features are weighted and fused. Finally, the output features are processed by average pooling to reduce the computational power required by the model, and a plug-and-play CBAM lightweight attention module is embedded to refocus key features in the channel and spatial dimensions. The model output is the sum of the outputs of the two fully connected layers.

7. The quantitative diagnosis method for multiple imbalance faults based on improved PER and deep reinforcement learning according to claim 1, characterized in that: S4 includes the following steps: S4.1: Set the maximum number of autonomous training iterations (Episode). During each autonomous training iteration, the virtual environment initiates T diagnostic requests, each corresponding to an environment state. ; S4.2: The virtual environment randomly initiates a diagnostic request; current state. ,Will Inputting a deep Q-network, the agent selects the current action according to a dynamic ε-greedy algorithm. The virtual environment returns a quantified reward to the agent based on the reward function. It then randomly initiates the next diagnostic request, that is, it transitions to the next state according to the state transition function. The resulting empirical data Stored in the experience pool with the highest priority; S4.3: Repeat S4.2 until the T diagnostic requests in this round end, and output the cumulative reward obtained in this iteration; S4.4: After the experience pool is full, random sample batches of experience data are taken from the experience pool according to priority, and the data is then used as the basis for further processing. The gradient descent method is used to update the online deep Q network, and the hard update method is used to update the target deep Q network at an update frequency C. The priority of updating empirical data is based on the reward function and TD-error. S4.5: Begin the next round of autonomous training. Repeat steps S4.2 to S4.4 until the maximum number of autonomous training rounds is reached, at which point the autonomous training process ends. S4.6: Save the model parameters obtained from each round of diagnostic requests during autonomous training with an accuracy rate higher than a certain threshold, as the intelligent fault quantitative diagnostic model learned by the agent.

Images