Dynamic cost sensitive fault diagnosis system based on AC-DCC model

Abstract

The invention discloses a dynamic cost sensitive fault diagnosis system based on an AC-DCC model. The real-time performance and the accuracy of fault diagnosis are improved in an auxiliary mode. The system can automatically position fault equipment by analyzing multi-dimensional data such as time difference, equipment utilization rate, historical maintenance record, output deviation and alarm record, and can quickly adjust a production plan according to a diagnosis result, thereby reducing the negative influence of faults on production. In order to solve the problem of data imbalance in the industrial process, a dynamic cost sensitive classifier (AC-DCC) based on an Actor-Critic model is introduced into the system; according to the model, through strategy gradient optimization, sample weights are learned in a self-adaptive mode, and particularly, weight distribution of minority samples in the pseudo label generation and classification process is dynamically adjusted for the minority samples. According to the AC-DCC model, a new reward function is designed, attention to minority class fault samples is enhanced, the classification performance of minority class faults is improved, interference of the faults on production can be reduced through intelligent scheduling, and the efficiency and reliability of industrial production are optimized.

Description

Technical Field

[0001] The present invention belongs to the field of industrial fault diagnosis and intelligent control, and relates to a fault diagnosis system based on machine learning and reinforcement learning, in particular to a dynamic cost-sensitive classification technology for unbalanced data processing in industrial processes. Background Art

[0002] In the industrial manufacturing process, equipment failure and performance degradation are one of the main factors affecting production efficiency and product quality. Traditional equipment fault diagnosis methods often rely on sensor monitoring or manual intervention to troubleshoot faults. However, with the increasing complexity of manufacturing systems, single sensor data cannot fully reflect the health status of the equipment, and excessive reliance on manual labor also increases response time, affecting the timeliness of fault handling. The unbalanced data problem in industrial processes refers to the phenomenon that fault data of different categories are unevenly distributed during equipment fault detection, classification or prediction. In the case of uneven distribution of fault data, most samples in the data set belong to certain common fault types, while a few samples belong to uncommon fault types, which leads to the classification model's good prediction of common faults, but insufficient prediction of rare faults.

[0003] The problem of unbalanced data in industrial fault diagnosis mainly focuses on frequent faults, occasional faults, complex faults, and sensor or data acquisition problems.

[0004] Frequent failures are caused by long-term high load or extreme working conditions, such as high temperature, high pressure, and overload, which lead to frequent wear and fatigue of equipment. The sample data volume is large, which is a common phenomenon in industrial failure data sets. The high frequency of these failures has become the main focus of equipment monitoring and maintenance, but it also leads to an imbalance in the data set, that is, the amount of common failure data is large, and the amount of rare failure data is scarce.

[0005] Sporadic failures are caused by external environmental factors or complex interactions within the equipment. They have a low probability of occurrence and are difficult to predict. The limited amount of sample data for this type of failure makes it difficult to train related models, and because they are rare, the problem of data scarcity is even more serious. Complex failures involve the coordinated operation of multiple devices or components, and are affected by multiple factors such as the external environment, operating methods, and control logic. Their occurrence patterns are complex and do not have a single cause. They are difficult to detect and predict through traditional diagnostic methods, further increasing the difficulty of data imbalance.

[0006] In addition, sensor or data collection problems also exacerbate data imbalance. Due to sensor failure or incomplete data collection, some faults are not fully recorded or monitored, resulting in a scarcity of fault data. These missing data are due to the complex interaction of multiple components in the system, especially in automated production lines, where failures in the coordination of multiple devices can easily cause faults that cannot be captured by a single sensor. Summary of the invention

[0007] (I) Purpose of the invention: The dynamic cost-sensitive fault diagnosis system based on the AC-DCC model described in the present invention uses multi-dimensional data such as time difference, equipment operation time and utilization, historical repair and maintenance records, production efficiency and output deviation, alarm and abnormal event records to realize automatic equipment fault identification and diagnosis. The dynamic cost-sensitive fault diagnosis system based on the AC-DCC model described in the present invention aims to improve the real-time and accuracy of fault diagnosis, and to respond to faults in a timely manner, so as to maximize production efficiency by quickly adjusting subsequent production plans.

[0008] The purpose of the present invention is to combine the differences between production plans and actual operation execution, and use existing key feature data to realize an efficient automatic fault identification and diagnosis system. The system automatically locates faulty equipment by analyzing multi-dimensional data such as time difference, equipment utilization, historical maintenance records, output deviations, and alarm records, and quickly adjusts production plans to reduce the impact of faults on production. In the actual collected industrial process monitoring data set, the data distribution of each category is usually unbalanced. Generally, some fault data is large (the number of frequent faults is large), and some fault data is small (the number of occasional faults is small).

[0009] The present invention discloses a dynamic cost-sensitive fault diagnosis system based on AC-DCC model, which aims at the cost-sensitive problem of unbalanced data in industrial processes. It proposes a dynamic cost-sensitive classifier (AC-DCC) based on ActorCritic model and designs a cost-sensitive strategy based on policy gradient to learn sample weights. The new cost-sensitive strategy adopts the alternating iteration method of ActorCritic, which can adaptively learn the cost matrix and dynamically generate sample weights during the model learning process. The newly designed reward function is used to guide the sample weight learning process in the ActorCritic mechanism. The effectiveness and practicality of the proposed ACDCC method will be verified through real-world data sets and industrial process benchmarks.

[0010] (II) Specific technical solution of the invention: Operation phase of a dynamic cost-sensitive fault diagnosis system based on AC-DCC model

[0011] The present invention discloses a dynamic cost-sensitive fault diagnosis system based on AC-DCC model, which is divided into two stages, namely, stage one is label generation (preprocessing, data set division and pseudo label generation), and stage two is training and application of fault classification model.

[0012] Stage (I) Label Generation (Preprocessing, Dataset Partitioning and Pseudo-label Generation)

[0013] By preprocessing the data and generating pseudo labels, the problem of unbalanced data in the industrial process is solved, so that the dynamic cost-sensitive mechanism of reinforcement learning optimizes the processing of minority samples, ensuring that minority faults can receive more attention in subsequent model training, providing a more comprehensive data foundation for subsequent classification model training.

[0014] Specifically, the functions of label generation (preprocessing, data set partitioning and pseudo-label generation) are as follows: 1. Perform data cleaning and standardization to ensure the uniformity and integrity of the data; 2. Generate high-quality pseudo-labels for unlabeled samples to supplement labeled scarce samples; 3. Dynamically adjust sample weights and focus on minority fault samples through reinforcement learning mechanism.

[0015] Specifically, the main technical means used in label generation (preprocessing, data set division and pseudo-label generation) are as follows: data collection is to obtain production plan data, actual operation data and historical data from the APS system and actual equipment, and then standardize and normalize the data, and deal with missing values ​​and outliers in the data; unlabeled sample processing is to generate pseudo labels for unlabeled data through a pseudo-label generator (TPG) to dynamically supplement the data set; the introduction of the AC-DCC model is to dynamically generate sample weights through the Actor-Critic reinforcement learning model, and use cost-sensitive classification methods to increase the attention of minority samples, especially the pseudo-label generation of minority faults.

[0016] Stage 2: Training and application of fault classification model

[0017] By training and applying the equipment fault classification model, accurate fault identification and real-time diagnosis can be achieved, especially improving the detection capability of a few types of faults in complex industrial environments; at the same time, the dynamic cost-sensitive fault diagnosis system based on the AC-DCC model described in the present invention can automatically adjust the production plan according to the detected fault conditions, reduce the impact of equipment failures on the production process, and improve production efficiency and the intelligence level of the system.

[0018] Specifically, the functions of training and applying the fault classification model are: first, to complete the training and evaluation of the fault classification model to ensure that the model can accurately identify different fault types of the equipment; second, to conduct a real-time diagnostic system to detect equipment failures and generate corresponding alarms; third, to realize the dynamic production plan adjustment function to ensure that the production line can maintain continuous operation during failures.

[0019] Specifically, the main technical means used in label generation (preprocessing and pseudo-label generation) are: the integration of pseudo-labels and real labels is to combine the generated pseudo-labels with known real labels to construct a complete labeled data set; supervised classification model training is to use SVM, random forest, XGBoost or neural network models to train the fault classification model through labeled data, and the model evaluation is carried out through commonly used classification indicators (such as accuracy, precision, recall rate, F1 score); real-time online fault classification is that the system receives sensor data in real time during the operation of the equipment, and predicts and diagnoses faults based on the trained model; real-time production plan adjustment is that after a fault is detected, the system automatically adjusts the production plan according to the diagnosis results and reallocates tasks to ensure the continuity of the production line.

[0020] (2) The present invention provides a dynamic cost-sensitive fault diagnosis system based on an AC-DCC model, comprising the following steps:

[0021] Stage (I) Label Generation (Preprocessing, Dataset Partitioning and Pseudo-label Generation)

[0022] S1. Data collection: Collect production, actual operation, and maintenance data from multiple sources such as APS system, equipment sensors, and historical data, and obtain data in real time through automation systems and sensor networks; and use multi-dimensional data sources to build a complete data set, covering the difference between production plan and actual execution, alarms, and historical data, to ensure that equipment fault classification has a comprehensive input basis;

[0023] S2. Data preprocessing: Process multiple types of raw data (such as time and task assignment), provide personalized standardization and normalization strategies for each data type, and provide high-quality input for subsequent modeling; use linear interpolation to fill missing values, and use statistical methods to detect and remove outliers; standardize time data in seconds to ensure feature consistency;

[0024] S3. Dataset division: Stratified sampling technology is used to ensure that the proportion of minority and majority class samples in the training set, validation set, and test set is consistent; the unlabeled samples are pseudo-labeled through TPG to improve the diversity of the data set; TPG pseudo-label generation technology is introduced to expand the effective use of unlabeled samples, ensure that minority class samples are evenly distributed in different data sets, and solve the problem of data imbalance;

[0025] S4. Pre-trained TPG (task pseudo-label generator): The TPG model generates pseudo-labels using the Softmax activation function through a multi-layer perceptron (MLP) structure and pre-trains with labeled data to improve the accuracy of pseudo-label generation. The pre-training mechanism is introduced to initialize model parameters with labeled data to provide high-quality pseudo-labels for unlabeled data, thereby improving the utilization rate of unlabeled data.

[0026] S5. Training offline TPG: Introduce the AC-DCC reinforcement learning framework, combined with the Actor-Critic mechanism, dynamically adjust the weight distribution of minority class samples through reinforcement learning, feedback the minority class classification performance through the Critic network and optimize the pseudo-label generation strategy to ensure that the minority class samples are paid attention to during the pseudo-label generation process, and improve the minority class classification performance;

[0027] S6. Reinforcement learning process: By designing a reward function based on the minority class recall rate, the error cost of minority class classification is increased through dynamic cost matrix adjustment; the strategy of TPG to generate pseudo labels is dynamically adjusted using the policy gradient algorithm to optimize the processing of minority class samples, ensure the priority of minority class samples in the entire reinforcement learning process, and improve the recognition effect of minority class faults;

[0028] Stage 2: Training and application of fault classification model

[0029] S7. Fault classification model training (offline training): In the offline training of the fault classification model, pseudo labels and real labels are combined to build a complete labeled data set, including fault types and related feature data (such as time difference, equipment operation time); then select a suitable supervised classification model (such as SVM, random forest, XGBoost or neural network) for training to improve the equipment fault classification ability; finally, use the validation set to evaluate the model performance, and optimize the model through accuracy, precision, recall and F1 value indicators to ensure the efficiency and accuracy of fault classification;

[0030] S8. Online fault classification: Real-time detection and classification of the trained model; the newly collected production data (such as time difference, equipment operation time, alarm record) is first input into the AC-DCC model, and the AC-DCC model predicts the fault label in real time based on these features, and identifies whether the equipment is in a fault state, and outputs the specific fault type and alarm information; if a fault is detected, the system will immediately generate an alarm and mark the faulty equipment; the system automatically adjusts the production plan according to the diagnosis results, and reallocates tasks to other equipment to ensure the continuity of the production line.

[0031] (3) In the dynamic cost-sensitive fault diagnosis system based on the AC-DCC model described in the present invention, each step further includes the following sub-steps:

[0032] S1. Data collection: Furthermore, the data collection in S1 of the present invention also includes the collection of production plan data, the collection of actual operation data and the collection of historical data; the specific operations are:

[0033] S1.1 Collection of production plan data: Obtain production plan data through the APS system, including the planned start time, completion time, expected output, operating time, equipment utilization, and maintenance plan of the equipment; these production plan data serve as the benchmark for equipment diagnosis and are used to compare the actual execution status;

[0034] S1.2 Collection of actual operation data: The actual operation data of the equipment is collected through equipment sensors and control systems, including start and completion time, output, operation time, alarm information, and downtime, which are used to monitor the equipment status in real time and identify deviations;

[0035] S1.3 Collection of historical data: Obtain historical data from equipment management and production systems, such as historical maintenance records, fault records, and production efficiency, to analyze equipment performance trends and predict potential failures.

[0036] S2. Data preprocessing: Furthermore, the S2. data preprocessing of the present invention also includes standardization and normalization, missing value processing and outlier detection, time data processing, estimated output processing, estimated operating time processing, equipment utilization plan and maintenance plan processing, and production task allocation processing; the specific operations are:

[0037] S2.1 Standardization and S2.2 Normalization: Standardize and normalize the collected numerical data to ensure that the data is on a uniform scale and improve the model learning effect;

[0038] S2.3 Missing value processing and S2.4 Outlier detection: Use forward filling or linear interpolation methods to fill missing data, and use statistical methods to detect and remove outliers to ensure data integrity and consistency;

[0039] S2.5 Time data processing: convert the time data of plan start and completion time into numerical values ​​and perform standardization or normalization;

[0040] S2.6 estimated output and S2.7 estimated running time processing: standardize or normalize the output and running time to avoid feature differences affecting model performance;

[0041] S2.8 Processing of equipment utilization plan and maintenance plan: Standardize or normalize equipment utilization and maintenance plan data to ensure feature consistency;

[0042] S2.9 Production task allocation processing: Process the production task allocation data through digitization or one-hot encoding method, and fill in missing values.

[0043] S3. Dataset division:Furthermore, the S3. data set division of the present invention further includes label sample and unlabel sample processing, stratified sampling division, unlabel sample and pseudo-label sample processing, and sample category analysis and weighting; the specific operations are:

[0044] S3.1 Labeled sample and unlabeled sample processing: Unlabeled samples generate pseudo labels through TPG to enhance the diversity of the dataset. Labeled samples are used for training, and pseudo labels are used to expand unlabeled samples;

[0045] S3.2 Stratified sampling: In the case of imbalanced data, stratified sampling is used to ensure that the proportion of minority and majority class samples in the training, validation, and test sets is consistent;

[0046] S3.3 Unlabeled sample and pseudo-label sample processing: The pseudo-labels generated by TPG are used to increase the unlabeled sample data and enhance the generalization ability of the classification model;

[0047] S3.4 Sample category analysis and weighting: Analyze sample distribution and set weights for minority class samples to ensure that minority class samples receive more attention during model training.

[0048] S4. Pre-training TPG (Task Pseudo Label Generator): Furthermore, the S4. pre-training TPG (task pseudo label generator) of the present invention also includes using TPG, network structure and pre-training strategy; the specific operation is:

[0049] S4.1 Using TPG: Use labeled data to pre-train the TPG model and generate pseudo labels for data expansion of unlabeled samples;

[0050] S4.2 Network structure and S4.3 pre-training strategy: TPG adopts MLP structure, uses Softmax activation function to generate pseudo-label probability distribution, uses cross-entropy loss function to calculate the error between pseudo-label and true label, and improves the quality of pseudo-label generation through pre-training.

[0051] S5. Training offline TPG: Furthermore, the S5. offline TPG training of the present invention also includes introducing AC-DCC model, strategy optimization and dynamic cost-sensitive mechanism and reinforcement learning process; the specific operations are:

[0052] S5.1 introduces the AC-DCC model: The weights of pseudo-label samples are dynamically adjusted through the Actor-Critic mechanism, especially giving higher weights to minority class samples to improve the quality of pseudo-label generation;

[0053] S5.2 Strategy optimization and dynamic cost-sensitive mechanism: Through the cost-sensitive mechanism of reinforcement learning, the weights and cost matrices of minority samples are dynamically adjusted to ensure the importance of minority samples in model training;

[0054] S5.3 Reinforcement learning process: Through reward function design, minority class recall rate and overall classification accuracy are introduced as reward targets to dynamically adjust the strategy of TPG to generate pseudo labels.

[0055] S6. Reinforcement learning process: Furthermore, the S6. reinforcement learning process of the present invention also includes reward function design, cost matrix construction and batch training, strategy optimization and iterative training; the specific operations are:

[0056] S6.1 Reward function design: Introduce the minority class recall rate and overall classification accuracy, and construct a reward function based on these two indicators to ensure that the classification performance of minority class faults receives sufficient attention;

[0057] S6.2 Cost Matrix Construction and S6.3 Batch Training: Generate a cost matrix based on category distribution, use the Actor network to generate weights, and train the model in batches to ensure the stability of the model in processing large-scale data;

[0058] S6.4 Strategy optimization and S6.5 iterative training: Through the policy gradient optimization algorithm, the strategy of TPG to generate pseudo labels is dynamically adjusted, and the model is continuously optimized in each round of reinforcement learning.

[0059] S7. Fault classification model training (offline training) : Further, the S7. fault classification model training (offline training) of the present invention also includes pseudo-label and real label integration, input data preparation and model selection and classification model training and model evaluation; the specific operations are:

[0060] S7.1 Integration of pseudo labels and real labels: Combine pseudo labels and real labels to build a complete training dataset;

[0061] S7.2 Input data preparation and model selection: Input the labeled data set into the supervised classification model (such as SVM, XGBoost) for training, select the appropriate model and optimize the parameters;

[0062] S7.3 Classification model training and model evaluation: Use the training set to train the classification model, and use the validation set to evaluate and optimize the classifier performance.

[0063] S8. Online fault classification: Furthermore, the S8. online fault classification of the present invention also includes online diagnosis process and real-time adjustment and alarm processing; the specific operations are:

[0064] S8.1 Online diagnosis process: Input the real-time collected production data into the trained classification model to predict and identify equipment failures in real time and generate alarms;

[0065] S8.2 Real-time adjustment and alarm processing: The system dynamically adjusts the production plan based on the detected fault information to ensure the continuity of the production line.

[0066] (4) In the dynamic cost-sensitive fault diagnosis system based on the AC-DCC model described in the present invention, each step further includes the following formulas and methods:

[0067] The data preprocessing in S2 also includes the following methods and formulas:

[0068] S2.1 Standardization: Convert the data to a normal distribution with a mean of 0 and a standard deviation of 1 so that the system model can converge better. The formula is:

[0069]

[0070] Where x is the data point, μ is the mean, and σ is the standard deviation;

[0071] S2.2 Normalization: Scale the data to a fixed range (ranging from 0 to 1) to ensure that all features are on the same scale and to avoid excessive impact of some features on the model due to dimensional differences. The formula is:

[0072]

[0073] Among them, x min and x max are the minimum and maximum values ​​of the data set, respectively;

[0074] S2.3 Missing value processing: Use forward filling, linear interpolation or other methods to fill missing values ​​in the data set to avoid affecting subsequent model learning; the forward filling is to fill missing values ​​with the previous valid data point; the linear interpolation is to fill missing values ​​with a linear formula, the formula is:

[0075]

[0076] Where x is the missing value; xprev and xnext are the known data points before and after the missing value; t is the time when the missing value is located, and tprev and tnext are the times of the known data points before and after respectively.

[0077] S2.4 Outlier detection and removal: Use statistical methods or rules to detect and remove abnormal data points;

[0078] S2.5 Handling of planned start time and planned completion time;

[0079] S2.6 Treatment of expected production;

[0080] S2.7 Estimated operating time processing;

[0081] S2.8 Equipment utilization planning processing;

[0082] S2.9 Equipment maintenance plan processing

[0083] S2.10 Production task allocation and processing.

[0084] S1. Data collection also includes the following steps, methods and formulas:

[0085] S1.1 Collection of production plan data: The APS system automatically extracts production plan data, including the start and end time of the equipment, output, planned operating time, equipment utilization and maintenance plan, for comparison with the actual execution effect and as a benchmark for equipment diagnosis;

[0086] S1.2 Collection of actual operation data: The actual execution data of equipment operation, such as start and completion time, actual output, and equipment utilization, are collected through sensors and control systems for real-time monitoring and comparison with planned data to identify deviations or failures in production;

[0087] S1.3 Collection of historical data: Collect historical maintenance records, fault records and production efficiency data of equipment as a reference for equipment status diagnosis, for comparison with current data, to assist in identifying trends and preventing faults.

[0088] S2. Data preprocessing also includes the following methods and formulas:

[0089] S2.1 Standardization: Convert the data to a normal distribution with a mean of 0 and a standard deviation of 1 so that the system model can converge better. The formula is:

[0090]

[0091] Where x is the data point, μ is the mean, and σ is the standard deviation;

[0092] S2.2 Normalization: Scale the data to a fixed range (ranging from 0 to 1) to ensure that all features are on the same scale and to avoid excessive impact of some features on the model due to dimensional differences. The formula is:

[0093]

[0094] Among them, x min and x max are the minimum and maximum values ​​of the data set, respectively;

[0095] S2.3 Missing value processing: Use forward filling, linear interpolation or other methods to fill missing values ​​in the data set to avoid affecting subsequent model learning; the forward filling is to fill missing values ​​with the previous valid data point; the linear interpolation is to fill missing values ​​with a linear formula, the formula is:

[0096]

[0097] Where x is the missing value; xprev and xnext are the known data points before and after the missing value; t is the time when the missing value is located, and tprev and tnext are the times of the known data points before and after respectively.

[0098] S2.4 Outlier detection and removal: Use statistical methods or rules to detect and remove abnormal data points;

[0099] S2.5 Handling of planned start time and planned completion time;

[0100] S2.6 Treatment of expected production;

[0101] S2.7 Estimated operating time processing;

[0102] S2.8 Equipment utilization planning processing;

[0103] S2.9 Equipment maintenance plan processing

[0104] S2.10 Production task allocation and processing.

[0105] S3. The data set division also includes the following methods and formulas:

[0106] S3.1 Processing of labeled and unlabeled samples: Unlabeled samples are dynamically added to the training set through pseudo labels generated by TPG, which enhances data diversity and improves model classification capabilities. Labeled samples are used for training and verification, which expands the utilization of unlabeled samples.

[0107] S3.2 Stratified sampling: Enhance the training data set through pseudo-label generation technology to ensure that the distribution of fault categories in the training set, validation set, and test set is consistent, avoiding data imbalance that affects model performance;

[0108] S3.3 Processing of unlabeled samples and pseudo-labeled samples: Pseudo-labels are generated for unlabeled samples during training and added to the training set to enhance data diversity; pseudo-labels generated by TPG are dynamically generated through a reinforcement learning framework, which can improve the classification ability of the model;

[0109] S3.4 Sample category analysis and preliminary weighting: In the preprocessing stage, different weights are set for sample imbalance through a weighting mechanism to ensure that minority samples receive more attention in model training; a higher initial weight is set for minority samples, and a lower initial weight is set for frequent failure data (majority class); the preliminary weighting formula is:

[0110]

[0111] Among them, x iis the initial weight of category i, n i is the number of samples of category i.

[0112] S4. Pre-training TPG also includes the following methods and formulas:

[0113] S4.1 Use TPG: To give the model a good initial parameter, TPG will be pre-trained using labeled training data;

[0114] S4.2 Network structure: A multi-layer perceptron (MLP) architecture is used. The input is the process variable feature and the output is the probability distribution of the pseudo label. The output layer uses the Softmax activation function to represent the probability of each pseudo label.

[0115] S4.3 Pre-training strategy: Use labeled training data to pre-train TPG to ensure that TPG can generate high-quality pseudo labels for unlabeled samples.

[0116] S5. Training offline TPG also includes the following methods and formulas:

[0117] S5.1 introduces the AC-DCC model: In TPG training, the Actor-Critic mechanism is used to dynamically adjust sample weights, especially to improve the quality of pseudo-label generation for minority samples, and to ensure classification accuracy under unbalanced data;

[0118] S5.2 Policy optimization of the AC-DCC model: The Actor network generates weights based on unlabeled samples, the Critic network evaluates the classification performance and provides feedback, and the Actor network uses the policy gradient method to adjust the weight distribution based on this feedback to ensure that minority samples receive more attention;

[0119] S5.3 Cost-sensitive mechanism of reinforcement learning: By introducing a cost matrix to dynamically adjust the classification cost of minority class samples, the model is ensured to give priority to minority class samples during training, and the sample weights are optimized through the Actor-Critic mechanism to improve the accuracy of pseudo-label generation and the classification performance of minority class samples;

[0120] S5.4 Model training process: By inputting the process variables and pseudo labels of unlabeled samples into the AC-DCC model, the Actor network dynamically generates weights, the Critic network evaluates based on the classification performance feedback, and optimizes the weight distribution through the policy gradient method.

[0121] S6. The reinforcement learning process also includes the following methods and formulas:

[0122] S6.1 Reward function design: By introducing the weight parameters of minority class recall rate and overall classification accuracy, the reward function formula is constructed to ensure that the overall classification accuracy is maintained while improving the minority class classification performance;

[0123] S6.2 Cost matrix construction: By analyzing the category distribution, a higher error cost is set for the minority class, and the minority class samples are ensured to be evenly distributed in the stratified sampling of the training, validation, and test sets, and the weighted cross entropy is used to optimize the model's classification performance for the minority class.

[0124] S6.3 Training process: Generate sample weights through batch training, use the validation set to evaluate performance and give feedback to the Actor network to adjust the strategy, and use the Adam optimizer to dynamically adjust the learning rate;

[0125] S6.4 Policy Optimization Optimize the pseudo-label generation strategy of the TPG model through a policy gradient method (such as REINFORCE), calculate the gradient of the strategy and adjust the model parameters based on reward feedback;

[0126] S6.5 Iterative training process: TPG generates pseudo labels for unlabeled data classification. After training the classifier, the reward is calculated based on the performance of the validation set, and the pseudo label generation strategy is updated and adjusted through the policy gradient optimization algorithm (such as REINFORCE).

[0127] S7. The fault classification model training also includes the following methods and formulas:

[0128] S7.1 Integration of pseudo labels and real labels: Generate labels for unlabeled data through the pseudo label generator and integrate them with the real labels to build a complete training dataset;

[0129] S7.2 Input data preparation: Input the labeled data set into the classification model, and then preprocess and standardize or normalize it to ensure that the data format is unified and feature engineering is completed;

[0130] S7.3 Model selection: Select an appropriate supervised classification model based on the complexity of the problem and the characteristics of the data;

[0131] S7.4 Classification model training: Input the labeled data set into the supervised classification model, improve the classification effect through parameter optimization, and enhance the model's ability to classify faults;

[0132] S7.5 Model evaluation: Use accuracy, precision, recall, and F1 value indicators on the validation set to evaluate the generalization ability of the model and optimize its performance.

[0133] (5) In the dynamic cost-sensitive fault diagnosis system based on the AC-DCC model described in the present invention, each sub-step further includes the following formula and method:

[0134] S2.5 The processing of planned start time and planned completion time also includes the following methods and formulas:

[0135] S2.51 Time data standardization: Convert the planned start time and completion time into seconds and then perform standardization. The formula for the converted timestamp standardization is:

[0136] and

[0137]

[0138] Among them, x start and x end is the time in seconds between the scheduled start and finish times, μ end and μ start are the means of all time points in the training data; σ start and σ start is the standard deviation of the time points;

[0139] S2.52 Normalization of time data: Alternatively, time data can be normalized directly without being standardized, and converted to a value between 0 and 1 to ensure that the time data is on the same scale; the formula is:

[0140] and

[0141]

[0142] Among them, x start-min and x start-max is the minimum and maximum value of the timestamp; x end-min and x end-max is the corresponding timestamp range.

[0143] S2.6 The processing of estimated production also includes the following formulas and methods:

[0144] S2.61 Standardization: Convert the expected output to a standard normal distribution with a mean of 0 and a standard deviation of 1 so that different features are processed on the same scale; the formula is:

[0145]

[0146] Among them, x output is the expected output of a certain equipment; μ output is the average expected output of all equipment in the dataset; σ output is the standard deviation of the estimated output of all devices in the data set;

[0147] S2.62 Normalization: Scale the estimated production to between 0 and 1 to ensure that data of different dimensions are in the same range; the formula is:

[0148]

[0149] Among them, xoutput-min is the minimum value of expected output; x output-max is the maximum expected output.

[0150] S2.7 Estimated running time processing also includes the following formulas and methods:

[0151] S2.71 Standardization: Through standardization, the estimated running time is converted into a standard normal distribution with a mean of 0 and a standard deviation of 1, reducing the impact caused by different feature dimensions; the formula is:

[0152]

[0153] Among them, x runtime is the expected operating time of a device; μ runtime is the average running time of all devices in the dataset; μ runtime is the standard deviation of the operating time of all devices;

[0154] S2.72 Normalization: Assuming the estimated running time ranges from 100 to 1000 minutes, scale the estimated running time to a fixed range between 0 and 1 to ensure that the eigenvalues ​​work on a uniform scale; the formula is:

[0155]

[0156] Among them, x runtime-min and x runtime-max are the minimum and maximum estimated run times.

[0157] S2.8 Equipment utilization planning also includes the following formulas and methods:

[0158] S2.81 Standardization: Equipment utilization is usually a percentage, and the value range is usually between 0 and 100. The planned utilization of the equipment is converted into a standard normal distribution with a mean of 0 and a standard deviation of 1. The formula is:

[0159]

[0160] Among them, x utilization is the planned utilization rate of the equipment; μ utilization is the average value of equipment utilization; μ utilization is the standard deviation of equipment utilization;

[0161] S2.82 Normalization: Normalize the equipment utilization directly to between 0 and 1; the formula is:

[0162]

[0163] Among them, x utilization is the planned utilization of the equipment (in percent).

[0164] S2.9 Equipment maintenance plan processing also includes the following formulas and methods:

[0165] S2.91 Timestamp conversion to seconds: First, convert the timestamp to seconds or minutes; when processing time data, convert it to relative seconds or minutes numerical time data;

[0166] S2.92 Standardization: Standardize the time data of the equipment maintenance plan to ensure that the mean is 0 and the standard deviation is 1; the formula is:

[0167]

[0168] Among them, x maintenance is the maintenance plan time (converted into seconds); μ maintenance is the mean of all time points in the training data; σ maintenance is the standard deviation of all time points in the training data;

[0169] S2.93 Normalization: Scale the time data of the equipment maintenance plan to a fixed range between 0 and 1; the formula is:

[0170]

[0171] Among them, x maintenance-min and x maintenance-max are the minimum and maximum values ​​in the data set.

[0172] S2.10 Production task allocation processing also includes the following formulas and methods:

[0173] S2.101 Convert task assignments to numerical values: Convert production task assignments to numerical form (such as task IDs or one-hot encoding) to ensure that classification tasks can be processed as numerical data input models;

[0174] S2.102 Standardization / normalization processing: Normalize the task ID to a value between 0 and 1 to ensure that it is input into the model at the same scale as other numerical features to maintain the balance of data input;

[0175] S2.103 Missing sensor data processing: For missing sensor data, use forward filling or linear interpolation to fill in the missing sensor data;

[0176] The forward filling is to fill in the missing data using the nearest known data point before the missing data;

[0177] The linear interpolation method estimates and fills in missing values ​​based on known data points before and after the missing data points. The linear interpolation formula is:

[0178]

[0179] Among them, xi is the missing value to be filled; x i-1 is the known data point before the missing value; x i+1 is the known data point after the missing value; t i is the time at which the missing value is located; t i-1 and t i+1 are the times corresponding to the known data points;

[0180] S2.104 Outlier detection and removal: Detect and remove outlier data points by setting reasonable upper and lower limits or using statistical methods (such as Z-score).

[0181] S4.3 pre-training strategy also includes the following formulas and methods:

[0182] S4.31 Using the cross entropy loss function (L CE ): Calculate the error between the predicted pseudo label and the true label, and measure the difference between the model output and the true value; the formula is:

[0183]

[0184] Among them, y i is the true label, The label probability predicted by the model;

[0185] S4.32 Initial parameter setting: Use labeled samples to initialize the weights of the TPG model.

[0186] The S5.4 model training process also includes the following formulas and methods:

[0187] S5.41 State input: The process variables of the unlabeled samples (such as time difference, equipment utilization) are spliced ​​with pseudo labels into a complete state vector as the input feature of the equipment operation status for training of the AC-DCC model; the vector representation of the equipment state is:

[0188] S i =[time diffutilization rate, output deviation,…]

[0189] S5.42 Action generation: The Actor network of AC-DCC generates weights according to the sample status, dynamically adjusts the sample priority when generating pseudo labels, especially pays more attention to minority class samples, and outputs the final classification results; pseudo labels are classified, and the given sample classification is:

[0190] A i =Predict\_Label(S i )

[0191] Among them, if the output is a probability distribution [P1 ,P 2 ,P 3 ], the final action is to select the category corresponding to the maximum probability;

[0192] S5.43 Classification and Feedback: The Critic network evaluates the quality of pseudo-label generation (recall rate and F1 score of minority classes) and provides feedback on classification performance to the Actor network to help optimize the weight allocation strategy.

[0193] S5.44 Strategy Update: Optimize the Actor network through the policy gradient method, dynamically adjust the sample weights, ensure that minority class samples receive more attention in subsequent training, and optimize the quality of pseudo-label generation.

[0194] S6.1 The reward function design also includes the following formulas and methods:

[0195] S6.11 Reward function based on minority class recall rate: The minority class recall rate is introduced as the reward target, and the learning focus of the model is adjusted through the weight parameter α;

[0196] S6.12 Combined with overall classification accuracy: The overall classification accuracy is introduced and balanced by the weight parameter β;

[0197] S6.13 Construct the reward function formula: Construct the reward function formula based on the weights of the minority class recall rate and the overall classification accuracy rate; each time the model correctly classifies a minority class fault, a higher reward is given; if the minority class is misclassified, a penalty is given; the reward function formula is:

[0198] R = α × Recall minority +β×Accuracy overall

[0199] Among them, α and β are weight parameters that control different objectives, ensuring that the model maintains the overall accuracy while improving the recall rate of the minority class.

[0200] S6.4 strategy optimization also includes the following formulas and methods:

[0201] S6.41 The goal of policy gradient optimization is to find the optimal strategy through reinforcement learning to maximize the long-term accumulated rewards, thereby generating higher quality pseudo labels to better complete the classification task;

[0202] S6.42 Policy Gradient Optimization Algorithm: Use a policy gradient optimization algorithm (such as REINFORCE) to calculate the policy gradient and update the model's pseudo-label generation strategy and optimize model parameters based on reward feedback; the policy gradient optimization formula is:

[0203]

[0204] Among them, πθ (a|s) is the probability of choosing action a in state s, and R(s,a) is the corresponding reward.

[0205] (6) The data information collected by the dynamic cost-sensitive fault diagnosis system based on the AC-DCC model of the present invention includes the following contents:

[0206] Preferably, the production plan data comes from the APS system, which is mainly used to compare the actual execution status as a basis for diagnosis. The specific fields are: the planned start time is the scheduled start time of the equipment; the planned completion time is the scheduled end time of the equipment; the expected output is the expected output of the equipment in each production cycle; the expected operating time is the planned operating time of the equipment in each production cycle; the equipment utilization plan is the expected utilization based on the production plan and equipment capacity; the equipment maintenance plan is the time schedule for regular maintenance and overhaul of the equipment; the production task allocation is the production tasks, process arrangements and planned progress corresponding to each equipment.

[0207] Preferably, the specific fields of the actual operation data are: the actual start-up time is the time when the equipment is actually started; the actual completion time is the time when the equipment is actually completed; the actual output is the output actually completed by the equipment in each production cycle; the actual operating time is the actual operating time of the equipment in each production cycle; the equipment utilization rate is based on the ratio of the actual operating time to the expected time, reflecting the utilization rate of the equipment; the downtime is the downtime of the equipment due to failure, maintenance or other reasons; the alarm data is the alarm information during equipment operation, including the alarm type (such as over-temperature, over-pressure, overload), alarm frequency and time; the abnormal event record is a record of abnormal conditions that occur during the operation of the equipment, such as an operating state that exceeds the normal working range; the operation log is the equipment operation information recorded by the operator, including manual intervention and fault repair data.

[0208] Preferably, the specific fields of the historical data are: the historical maintenance and overhaul records are the past maintenance and repair time, content, and frequency; the historical fault records are the past fault time, type, and repair process of the equipment; the historical production efficiency is the efficiency and output of the equipment in previous production cycles and the deviation from actual execution.

[0209] (II) The core principle of the dynamic cost-sensitive fault diagnosis system based on the AC-DCC model described in the present invention

[0210] The present invention discloses a dynamic cost-sensitive fault diagnosis system based on the AC-DCC model. Its core principle revolves around the AC-DCC model and combines multiple technical means such as reinforcement learning, pseudo-label generation, and cost-sensitive mechanism to solve the fault classification problem in unbalanced data.

[0211] (1) Introduction of the AC-DCC model: The AC-DCC model is introduced by dynamically adjusting sample weights through the Actor-Critic architecture in reinforcement learning, so that pseudo-label generation and fault classification of minority samples are given higher priority when processing unbalanced data.

[0212] In terms of Actor-Critic architecture, the AC-DCC model uses the Actor-Critic architecture of reinforcement learning to dynamically adjust the weights of samples, making pseudo-label generation and classification more accurate in unbalanced data environments, especially for minority samples.

[0213] In terms of the Actor network, the Actor network is responsible for generating sample weights based on the status of unlabeled samples, especially assigning higher weights to minority samples to enhance their influence in pseudo-label generation and model training.

[0214] In terms of the Critic network, the Critic network provides feedback signals to the Actor network by evaluating the performance of the classifier (such as the recall rate of the minority class and the F1 score), thereby optimizing its strategy for generating weights.

[0215] (2) Combination of cost matrix and AC-DCC: The cost-sensitive mechanism sets a higher classification error cost for minority class samples, allowing the system to pay more attention to minority class samples during training, thereby improving the detection capability of minority class faults.

[0216] In terms of the cost matrix, the classification error cost of minority class samples is higher, so the AC-DCC model needs to pay more attention to minority class samples during training. Then, by dynamically adjusting the cost matrix, the AC-DCC model adaptively adjusts the cost weights of minority class samples according to the feedback of the classifier to optimize its classification performance.

[0217] In terms of dynamic adjustment, as the feedback changes during the training process, the AC-DCC model adjusts the classification weights of minority class samples in real time, so that their weights and importance in pseudo-label generation and fault classification are optimized as the data distribution changes.

[0218] (4) Use of Pseudo-Tag Generator (TPG)

[0219] For unlabeled data, pseudo labels can be generated through the pseudo label generator (TPG), and these pseudo labels can be dynamically optimized using the reinforcement learning framework, thereby solving the problem that unlabeled data cannot be effectively utilized, enhancing the diversity of the data set, and improving the classification ability of the AC-DCC model.

[0220] In terms of the use of the pseudo-label generator (TPG), TPG generates pseudo-labels for unlabeled samples, and uses the reinforcement learning framework to dynamically optimize these pseudo-labels, so that unlabeled data can be effectively utilized, the diversity of the data set is increased, and the generalization ability and classification effect of the model are improved;

[0221] In terms of MLP and Softmax, the pseudo-label generation network adopts a multi-layer perceptron (MLP) architecture and generates the probability distribution of pseudo-labels through the Softmax function to ensure the flexibility and computational efficiency of pseudo-label generation.

[0222] (5) Cost-sensitive mechanisms in reinforcement learning

[0223] The introduction of reinforcement learning framework, especially the strategy based on dynamic adjustment of cost matrix, enables AC-DCC model to optimize the quality of pseudo label generation according to the feedback during training. The system of the present invention dynamically adjusts the weight and cost distribution of samples by monitoring the performance indicators of the classifier on the validation set (such as minority class recall rate, F1 score).

[0224] In terms of combining the multi-layer perceptron (MLP) network with the Softmax activation function, the pseudo-label generation network adopts the multi-layer perceptron (MLP) architecture and uses the Softmax activation function to generate the probability distribution of pseudo-labels. The flexibility and computational efficiency of this structure ensure the high-quality generation of pseudo-labels and the effective training of subsequent models.

[0225] (6) Dynamic weight adjustment mechanism:

[0226] In the pseudo-label generation process, the Actor network generates weights based on the state of the unlabeled samples, so that minority samples have a greater weight in pseudo-label generation and model training. Dynamic weight adjustment ensures that minority samples are not ignored when the model deals with data imbalance, and the AC-DCC model can better handle minority fault classification.

[0227] (7) Combination of real-time diagnosis and dynamic production scheduling

[0228] After fault detection, the system can automatically adjust the production plan according to the diagnosis results and reallocate tasks to other equipment to ensure the continuity and efficiency of production.

[0229] (IV) Beneficial technical effects: Compared with the existing technology, the dynamic cost-sensitive fault diagnosis system based on the AC-DCC model described in the present invention has the following beneficial effects:

[0230] First, the system improves the recall rate and classification accuracy of minority class samples through the dynamic weight adjustment and cost-sensitive mechanism of the AC-DCC model, effectively solving the problem that traditional fault diagnosis systems are easily overlooked when dealing with minority class faults.

[0231] Second, the system uses the TPG pseudo-label generator to fully utilize unlabeled data, expanding the scale and diversity of the training dataset, thereby significantly improving the generalization ability and classification performance of the model;

[0232] Third, the system can identify equipment failures in real time and generate corresponding alarms. At the same time, according to the type and location of the failure, the system automatically optimizes the production plan and performs dynamic task allocation, thus improving the flexibility and stability of the production line.

[0233] Fourth, the system dynamically adjusts the cost matrix and reinforcement learning framework to better identify minority class faults in unbalanced data sets, effectively solving the classification bias problem caused by data imbalance.

[0234] Fifth, the system optimizes the speed and accuracy of model training by using batch training and Adam optimizer. Based on the performance feedback on the validation set, the system dynamically adjusts the model's learning rate and weight generation strategy, further improving the accuracy and reliability of fault diagnosis.

[0235] Sixth, by introducing policy gradient optimization in reinforcement learning (such as the REINFORCE algorithm), the system can adaptively adjust the pseudo-label generation and classification strategies to ensure the maximization of long-term accumulated rewards, making the pseudo-label generation strategy more in line with the needs of actual classification tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0236] Figure 1 A flowchart of the steps of a dynamic cost-sensitive fault diagnosis system based on an AC-DCC model of the present invention;

[0237] Figure 2 A TPG fault classification flow chart of a dynamic cost-sensitive fault diagnosis system based on an AC-DCC model of the present invention;

[0238] Figure 3 A cost-sensitive training TPG flow chart based on Actor-Critic of a dynamic cost-sensitive fault diagnosis system based on AC-DCC model of the present invention;

[0239] Figure 4 The figure is a schematic diagram of the overall steps based on Actor-Critic of a dynamic cost-sensitive fault diagnosis system based on AC-DCC model of the present invention. DETAILED DESCRIPTION

[0240] The present invention discloses a dynamic cost-sensitive fault diagnosis system based on an AC-DCC model. The three technical cores of the fault diagnosis system based on the AC-DCC model are: pseudo-label generation and dynamic weight adjustment, cost-sensitive strategy of reinforcement learning mechanism, and real-time application of fault classification model and production plan adjustment.

[0241] The three technical cores of a fault diagnosis system based on the AC-DCC model are to solve the data scarcity problem by dynamically adjusting sample weights, to improve the detection capability of minority class faults by using the cost-sensitive mechanism of reinforcement learning, and to ensure the continuity of production by real-time fault detection and production adjustment functions. A dynamic cost-sensitive fault diagnosis system based on the AC-DCC model achieves fault classification optimization in processing unbalanced data and complex industrial environments through these three core mechanisms.

[0242] From the perspective of pseudo-label generation by TPG and dynamic weight adjustment by AC-DCC, the present invention discloses a dynamic cost-sensitive fault diagnosis system based on the AC-DCC model. By dynamically adjusting sample weights, AC-DCC assigns higher weights to minority class samples, so that unlabeled data can effectively participate in training through pseudo-labels generated by TPG, thereby expanding the scale of the training data set, thereby improving the overall classification performance of the model and solving the problem of data scarcity, especially for the detection of minority class faults.

[0243] From the perspective of the cost-sensitive mechanism of reinforcement learning, the present invention describes a dynamic cost-sensitive fault diagnosis system based on the AC-DCC model. By utilizing the Actor-Critic architecture in reinforcement learning, the system introduces a cost matrix for dynamically adjusting the weights of samples of different categories, especially for minority class faults. Thus, the cost-sensitive mechanism significantly improves the recall rate of minority class faults, and the model can automatically adjust when processing unbalanced data to ensure that the focus is on minority class faults, thereby optimizing the overall classification performance.

[0244] From the perspective of real-time application of fault classification models and production plan adjustment, the dynamic cost-sensitive fault diagnosis system based on the AC-DCC model described in the present invention can detect equipment failures in real-time applications, and can also automatically generate alarms to remind users, and dynamically adjust production plans according to equipment failure conditions. It has real-time fault detection and production plan adjustment functions, can quickly identify and handle faults, and production plan adjustment ensures that tasks can be reallocated after equipment failure, avoiding production interruptions, reducing downtime, and improving production efficiency.

[0245] Example 1 Overall implementation scheme of a dynamic cost-sensitive fault diagnosis system based on AC-DCC model described in the present invention

[0246] In a specific embodiment, a dynamic cost-sensitive fault diagnosis system based on the AC-DCC model is used to optimize pseudo-label generation of minority class samples using reinforcement learning, and supervised learning is combined to improve the performance of the fault classification model. In actual production applications, the system realizes real-time fault diagnosis and dynamic adjustment of production plans, and can effectively deal with complex faults and unbalanced data problems in industrial production.

[0247] In this embodiment, the specific implementation steps of the dynamic cost-sensitive fault diagnosis system based on the AC-DCC model are as follows:

[0248] Phase 1: Label generation (preprocessing and pseudo-label generation)

[0249] S1. Data collection: S1.1 Collection of production plan data: Obtain production plan data from APS (Advanced Planning and Scheduling System), including the scheduled start and end time of the equipment, output, planned operating time, equipment utilization, maintenance plan, and production task allocation data. This is used to collect data from the APS system and compare and analyze it with the actual data to determine whether the equipment execution effect meets expectations, which serves as a benchmark for equipment diagnosis.

[0250] The data source is automated data extraction through the APS system;

[0251] The data fields involve various time and planning parameters related to production, such as planned start time, planned completion time, expected output, and equipment utilization planning information;

[0252] The production plan data comes from the APS system and is mainly used to compare the actual execution status as a diagnostic benchmark. The specific fields are:

[0253] The planned start time is the scheduled start time of the device;

[0254] The planned completion time is the scheduled end time of the equipment;

[0255] The estimated output is the output that the equipment is expected to complete in each production cycle;

[0256] The estimated operating time is the planned operating time of each production cycle equipment;

[0257] The equipment utilization rate plan is an estimated utilization rate based on the production plan and equipment capacity;

[0258] The equipment maintenance plan is a schedule for regular maintenance and overhaul of the equipment;

[0259] The production tasks are allocated to each device corresponding to the production tasks, process arrangements and planned progress;

[0260] S1.2 Collection of actual operation data: Through sensors and control systems, the actual execution data of the equipment in operation is collected, including start-up and completion time, actual output, operation time, equipment utilization, downtime, alarm information, abnormal events and operation log information, which are used to compare the planned data with the actual operation data, identify whether there are deviations or failures in the equipment during the production process, realize real-time data collection and monitoring, and provide comprehensive equipment status information;

[0261] The data collection methods include automatic sensor collection and control system integration;

[0262] The data fields are various actual values ​​during the actual execution of the device, such as actual start time, actual completion time, device operation time, and device alarm record information;

[0263] The actual operation data is obtained through equipment sensors and control systems, and truly reflects the operation status of the equipment; the specific fields are:

[0264] The actual startup time is the time when the device is actually started;

[0265] The actual completion time is the time when the equipment is actually completed;

[0266] The actual output is the output actually completed by the equipment in each production cycle;

[0267] The actual operating time is the actual operating time of the equipment in each production cycle;

[0268] The equipment utilization rate is based on the ratio of actual operating time to expected time, reflecting the utilization rate of the equipment;

[0269] The downtime is the downtime of the equipment due to failure, maintenance or other reasons;

[0270] The alarm data is the alarm information during equipment operation, including the alarm type (such as over-temperature, over-pressure, over-load), alarm frequency and time;

[0271] The abnormal event record is a record of abnormal conditions that occur during the operation of the equipment, such as an operating state that exceeds the normal working range;

[0272] The operation log is the equipment operation information recorded by the operator, including manual intervention and fault repair data.

[0273] S1.3 Collection of historical data: Collect historical maintenance records, fault records, and production efficiency data of equipment as auxiliary information for analysis. The system can obtain historical data from equipment management and production systems as an important reference for equipment status diagnosis and for comparison and analysis with current data to help identify trends and prevent faults.

[0274] The data sources are maintenance records and historical fault information, which are generally from equipment management systems (such as CMMS), and historical production efficiency data are from production systems or ERP;

[0275] The data fields include historical maintenance time, content, failure time and repair details, equipment past production efficiency and execution deviation; the specific fields are:

[0276] The historical maintenance and repair records include the time, content and frequency of past maintenance and repairs;

[0277] The historical fault records are the time, type and repair process of the faults that occurred in the past.

[0278] The historical production efficiency refers to the deviation between the efficiency and output of the equipment in previous production cycles and the actual execution.

[0279] S2. Data preprocessing:

[0280] S2.1 Standardization: Convert the data to a normal distribution with a mean of 0 and a standard deviation of 1 to ensure that the data is on the same scale and has the same mean and standard deviation. This is a common data processing technique that is suitable for processing numerical data so that the system model can converge better. The formula is:

[0281]

[0282] Where x is the data point, μ is the mean, and σ is the standard deviation;

[0283] S2.2 Normalization: Scale the data to a fixed range (ranging from 0 to 1) to ensure that all features are on the same scale and to avoid excessive impact of some features on the model due to dimensional differences. The formula is:

[0284]

[0285] Among them, x min and x max are the minimum and maximum values ​​of the data set, respectively;

[0286] S2.3 Missing value processing: Use forward filling, linear interpolation or other methods to fill missing values ​​in the data set to avoid affecting subsequent model learning; the forward filling is to fill missing values ​​with the previous valid data point; the linear interpolation is to fill missing values ​​with a linear formula, the formula is:

[0287]

[0288] Where x is the missing value; xprev and xnext are the known data points before and after the missing value; t is the time when the missing value is located, and tprev and tnext are the times of the known data points before and after respectively.

[0289] S2.4 Outlier detection and removal: Use statistical methods or rules to detect and remove abnormal data points to ensure data quality and consistency and avoid system model performance degradation due to missing values ​​during training;

[0290] The statistical method is to use the box plot method and the 3σ rule to determine which data points deviate too far from the mean;

[0291] The elimination method is to eliminate abnormal data points or replace them with the mean or median.

[0292] S2.5 Processing of planned start time and planned completion time

[0293] S2.51 Standardization of time data: Convert the planned start time and completion time into seconds, and then perform standardization processing to facilitate subsequent standardization or normalization processing so that the time data has a unified unit; the standardized formula for the converted timestamp is:

[0294] and

[0295]

[0296] Among them, x start and x end is the time in seconds between the scheduled start and finish times, μ end and μ start are the means of all time points in the training data; σ start and σ start is the standard deviation of the time points;

[0297] S2.52 Normalization of time data: Alternatively, time data can be normalized directly without being standardized, and converted to a value between 0 and 1 to ensure that the time data is on the same scale; the formula is:

[0298] and

[0299]

[0300] Among them, x start-min and x start-max is the minimum and maximum value of the timestamp; x end-min and x end-max is the corresponding timestamp range.

[0301] S2.6 Treatment of estimated production

[0302] S2.61 Standardization: Convert the expected output into a standard normal distribution with a mean of 0 and a standard deviation of 1, so that different features are processed on the same scale to avoid extreme differences in the data affecting the learning effect of the model; the formula is:

[0303]

[0304] Among them, x output is the expected output of a certain equipment; μ output is the average expected output of all equipment in the dataset; σ output is the standard deviation of the estimated output of all devices in the data set;

[0305] S2.62 Normalization: Scale the expected output to between 0 and 1 to ensure that data of different dimensions are in the same range to avoid certain features having too much influence on the model; the formula is:

[0306]

[0307] Among them, x output-min is the minimum value of expected output; x output-max is the maximum value of the expected output;

[0308] S2.7 Estimated operating time processing

[0309] S2.71 Standardization: Through standardization, the estimated running time is converted to a standard normal distribution with a mean of 0 and a standard deviation of 1 to ensure the uniformity of the scale of data features, especially during model training, which can reduce the impact of different feature dimensions; the formula is:

[0310]

[0311] Among them, x runtime is the expected operating time of a device; μ runtime is the average running time of all devices in the dataset; μ runtime is the standard deviation of the operating time of all devices;

[0312] S2.72 Normalization: Assume that the estimated running time ranges from 100 to 1000 minutes, and scale the estimated running time to a fixed range between 0 and 1 to ensure that the feature values ​​work at a uniform scale and prevent some features from having too much impact on model training due to a large range of values; the formula is:

[0313]

[0314] Among them, x runtime-min and x runtime-max are the minimum and maximum estimated running times;

[0315] S2.8 Equipment utilization planning processing

[0316] S2.81 Standardization: Equipment utilization is usually a percentage, and the value range is usually between 0 and 100. The planned utilization of the equipment is converted into a standard normal distribution with a mean of 0 and a standard deviation of 1 to ensure the uniformity of the data scale and reduce the impact of differences between different features. The formula is:

[0317]

[0318] Among them, x utilization is the planned utilization rate of the equipment; μ utilization is the average value of equipment utilization; μ utilization is the standard deviation of equipment utilization;

[0319] S2.82 Normalization: Normalize the equipment utilization directly to between 0 and 1 to ensure that the equipment utilization data is within a uniform range; the formula is:

[0320]

[0321] Among them, x utilization is the planned utilization of the equipment (in percent);

[0322] S2.9 Equipment maintenance plan processing

[0323] S2.91 Timestamp conversion to seconds: First, convert the timestamp (such as 2024-09-10 08:00:00) to seconds or minutes to ensure that the data can be used for subsequent standardization or normalization; the timestamp usually represents absolute time, but when processing time data, it is converted into relative numerical data (such as seconds, minutes) to facilitate model learning and analysis;

[0324] S2.92 Standardization: Standardize the time data of the equipment maintenance plan to ensure that the mean is 0 and the standard deviation is 1, so that the data features have consistent performance on the same scale. The standardized data can ensure the input of the machine learning model and avoid differences between different data dimensions; the formula is:

[0325]

[0326] Among them, x maintenance is the maintenance plan time (converted into seconds); μ maintenance is the mean of all time points in the training data; σ maintenance is the standard deviation of all time points in the training data;

[0327] S2.93 Normalization: Scale the time data of the equipment maintenance plan to a fixed range between 0 and 1 to ensure that the model's processing of different features is not affected by dimensional differences, making the model processing more intuitive and consistent; the formula is:

[0328]

[0329] Among them, x maintenance-min and x maintenance-max are the minimum and maximum values ​​in the data set;

[0330] S2.10 Production task allocation and processing

[0331] S2.101 Convert task assignments into numerical values: Convert the production task assignments into numerical values ​​(such as task IDs) by assigning values, or converting them using One-Hot Encoding to ensure that the classification tasks become processable numerical data input models;

[0332] The task ID conversion value is to assign a unique ID to each production task to identify and distinguish the task;

[0333] The one-hot encoding is to convert categorical variables into numerical representations acceptable to the model by creating binary variables; for example, if there are three task types, task 1, task 2, and task 3, they are represented by (1,0,0), (0,1,0), and (0,0,1);

[0334] S2.102 Standardization / normalization processing: Normalize the task ID directly to ensure that the data is on the same scale; for classification problems, the task ID can be converted to a range between 0 and 1 through normalization; normalization can keep the classification task consistent with other numerical features to ensure the balance of the input;

[0335] S2.103 Missing sensor data processing: For missing sensor data, use forward filling or linear interpolation to fill in the missing sensor data;

[0336] The forward filling is to fill in the missing data using the nearest known data point before the missing data;

[0337] The linear interpolation method estimates and fills in missing values ​​based on known data points before and after the missing data points. The linear interpolation formula is:

[0338]

[0339] Among them, x i is the missing value to be filled; x i-1 is the known data point before the missing value; x i+1is the known data point after the missing value; t i is the time at which the missing value is located; t i-1 and t i+1 are the times corresponding to the known data points;

[0340] S2.104 Outlier detection and removal: Detect and remove outlier data points by setting reasonable upper and lower limits or using statistical methods (such as Z-score); the outliers are caused by sensor failure, data entry errors, or equipment abnormalities;

[0341] S3. Dataset division

[0342] S3.1 Processing of labeled samples and unlabeled samples: Pseudo labels are generated for unlabeled samples during training and added to the training set to enhance data diversity; pseudo labels generated by TPG are dynamically generated through a reinforcement learning framework, which can improve the classification ability of the model; in the data set, labeled samples and unlabeled samples need to be processed separately; when using TPG (pseudo label generator), the system requires unlabeled samples to generate pseudo labels, and labeled samples are used for model training and verification; pseudo label generation by TPG is used to expand unlabeled samples and improve the utilization rate of unlabeled samples;

[0343] The labeled samples have real fault category labels and are used in the pre-training phase of training a pseudo label generator (TPG) and as a benchmark for subsequent training of a fault classification model.

[0344] The unlabeled samples have no real labels. After generating pseudo labels through TPG, they are added to the training set to increase the amount of model training data and enhance the generalization ability of the classification model.

[0345] S3.2 Stratified sampling division: Since production fault data is unbalanced, with very little data for some fault categories (minority class) and more data for other fault categories (majority class), it is necessary to use pseudo-label generation technology to enhance the training data set when dividing the data set to ensure that the distribution of fault categories is consistent in the training set, validation set, and test set to avoid the impact of data imbalance on model performance;

[0346] The stratified sampling is to ensure that the proportion of each category in the data set is consistent when dividing the training set, validation set and test set, especially to ensure that the minority class samples have a certain proportion in each subset, so that the minority class samples and the majority class samples appear in the training, validation and test sets in the same proportion;

[0347] S3.3 Processing of unlabeled samples and pseudo-labeled samples: Pseudo-labels are generated for unlabeled samples during training and added to the training set to enhance data diversity; the pseudo-labels generated by TPG are dynamically generated through a reinforcement learning framework, which can improve the classification ability of the model.

[0348] The unlabeled samples can be divided into a training set and a validation set in a similar ratio; although the unlabeled samples have no labels at the beginning, after generating pseudo labels through TPG, they can be used together with labeled data to train the classifier;

[0349] The pseudo-label samples: pseudo-label data generated by TPG will be dynamically added to the training set to enhance the diversity of the data set and the generalization ability of the training model;

[0350] S3.4 Sample category analysis and preliminary weighting: In the preprocessing stage, it is necessary to analyze the distribution of sample categories, especially when the data is unbalanced, to set different weights for minority and majority class samples, so that more attention can be paid to minority classes in subsequent model training; the weighting mechanism is a compensation strategy for sample imbalance, ensuring the importance of minority class samples in training;

[0351] The sample category analysis is to count the number of samples in each category;

[0352] The initial weight setting is to set a higher initial weight for minority class samples and a lower initial weight for frequent fault data (majority class); the initial weighting is performed using the following formula:

[0353]

[0354] Among them, x i is the initial weight of category i, n i is the number of samples of category i;

[0355] S4. Pre-training TPG (Task Pseudo Label Generator)

[0356] S4.1 Using TPG: TPG is a neural network used to generate pseudo labels for unlabeled samples. In order to give the model a good initial parameter, TPG will be pre-trained using labeled training data.

[0357] The task pseudo-label generator (TPG) is used to generate pseudo-labels for unlabeled samples. By dynamically generating pseudo-labels, the model generates label distribution probabilities of different categories.

[0358] The unlabeled sample processing is a special processing of unlabeled data by TPG, which converts unlabeled data into trainable data by generating pseudo labels, and uses unlabeled samples for model training;

[0359] S4.2 Network structure: The input of TPG is the process variable characteristics, and the output is the probability distribution of pseudo labels of different categories; the network adopts a multi-layer perceptron (MLP) architecture, the input is the process variable characteristics, and the output is the probability distribution of pseudo labels; the output layer uses the Softmax activation function to represent the probability of each pseudo label;

[0360] The multi-layer perceptron (MLP) is used to process process variable features, and the neural network converts the input features into probability distribution of pseudo labels through layer-by-layer propagation;

[0361] The Softmax activation function is used in the output layer to map the probability distribution of different pseudo labels to the range of 0 to 1, which is used to generate the probability of label classification and can convert the output into the probability distribution of pseudo labels;

[0362] S4.3 Pre-training strategy: Use labeled training data to pre-train TPG to ensure that TPG can generate high-quality pseudo labels for unlabeled samples;

[0363] S4.31 Using the cross entropy loss function (L CE ): Calculate the error between the predicted pseudo label and the true label, and measure the difference between the model output and the true value; the formula is:

[0364]

[0365] Among them, y i is the true label, The label probability predicted by the model;

[0366] S4.32 Initial parameter setting: Use labeled samples to initialize the weights of the TPG model to ensure that the TPG network has good generalization performance when training unlabeled data.

[0367] S5. Training offline TPG

[0368] S5.1 introduces the AC-DCC model: During the TPG training process, the AC-DCC model using the Actor-Critic mechanism improves the quality of pseudo-label generation by dynamically adjusting sample weights, especially giving higher weights to pseudo-label generation of minority samples. When AC-DCC generates pseudo-labels for unlabeled data through a reinforcement learning framework, it assigns higher weights to minority samples, thereby improving the quality of pseudo-labels.

[0369] The Actor-Critic architecture is an Actor-Critic framework based on reinforcement learning. The Actor is used to dynamically adjust the sample weights, and the Critic provides reward feedback based on the classification performance to evaluate the effectiveness of the current strategy and determine the effect of the dynamic adjustment of the entire sample weights.

[0370] The dynamic weight adjustment is for minority samples in unbalanced data. AC-DCC can assign higher weights to them to solve the problem of minority samples being ignored. The weights are dynamically adjusted according to the importance of samples during training, especially to enhance the influence of minority samples and ensure the accuracy of pseudo-label generation.

[0371] S5.11 State input: The state is the input of the Actor network. The process variable characteristics of the unlabeled training samples and the information after the pseudo-label splicing serve as the input state of the Actor network.

[0372] The process variable characteristics include time difference, equipment utilization, and output deviation, which are used to describe the characteristics of each sample;

[0373] The pseudo label information is generated by the TPG network, is part of the state input, and is used to provide category information of the sample;

[0374] S5.12 Action output: The Actor network of AC-DCC generates corresponding weights for each unlabeled sample according to the state. The weights are used to dynamically adjust the importance of the samples and affect the generation of pseudo labels.

[0375] The Actor network outputs weights: The weight of each sample is generated based on the current state and is used to determine the influence of the sample. In particular, the weight of minority samples will be dynamically adjusted according to actual needs.

[0376] The sample weight generation (action output) is the output of the Actor network, which generates appropriate sample weights through the reinforcement learning process to solve data imbalance;

[0377] S5.13 Reward mechanism: The reward is evaluated based on the performance of the classifier on the validation set (such as minority class recall, F1 score) and is used to optimize the current sample weight allocation strategy;

[0378] The performance indicators are the minority class recall rate and the F1 score performance indicators used for reward feedback, which reflect the classification effect of the model on minority class samples;

[0379] The reward feedback mechanism provides reward feedback based on the performance of the classifier to optimize the weight strategy generated by the Actor network. Through the feedback of the validation set, the Critic network can provide a reward signal for adjusting the weight of the Actor network to help the Actor improve its strategy.

[0380] S5.2 AC-DCC Strategy Optimization

[0381] S5.21 Actor network weight generation: Generate weights based on the state to ensure that minority class samples receive higher weights during pseudo-label generation.

[0382] The Actor network generates weights based on the state to ensure that minority samples receive higher weights;

[0383] The state input is the process variable and pseudo-label information input of the Actor network based on the unlabeled training samples. The sample weight is dynamically adjusted based on the state information by the Actor network to ensure that the model pays more attention to minority samples in the pseudo-label generation process, thereby generating dynamic weights;

[0384] The weight generation is for samples of different categories, especially minority samples. The Actor network will give them higher weights to enhance their influence in model training.

[0385] S5.22 Critic network feedback: The Critic network provides reward feedback to the Actor network by evaluating the performance of the classifier (such as minority class recall, F1 score), helping it optimize the weight strategy;

[0386] The classification performance feedback is that the Critic network evaluates the pros and cons of the current sample weight strategy based on the performance of the classifier on the validation set, especially the processing of minority class samples;

[0387] The reward feedback mechanism provides feedback signals based on classification performance to the Actor network through the Critic network. It is the key to the optimization of the Actor network, ensuring that the weights generated by the Actor network can improve the overall performance of the classifier, especially when processing minority samples. The reward feedback provided by the Critic network directly affects the Actor network's adjustment of sample weights, optimizing the Actor network's strategy through the reinforcement learning mechanism.

[0388] S5.3 Cost-Sensitive Mechanisms in Reinforcement Learning

[0389] S5.31 introduces the cost matrix: To solve the problem of unbalanced data, the system introduces the cost matrix; the minority class samples in the cost matrix are assigned higher classification error costs, ensuring that the model pays more attention to these samples during training;

[0390] The cost-sensitive mechanism is to allocate a higher cost to minority class samples, as the risk of misclassification is greater, so that the model regards the correct classification of minority class samples as a more important task during training;

[0391] The dynamic cost adjustment means that the cost matrix not only fixes high costs for minority classes, but can also be dynamically adjusted with feedback during the training process, so that the model can more adaptively handle the problem of unbalanced sample distribution;

[0392] S5.32 Dynamically adjust the cost-sensitive classification strategy: The AC-DCC model dynamically adjusts the cost distribution in the cost matrix according to the performance of the classifier on the validation set; the recall rate and F1 score indicators of the minority class determine the next step of optimizing the cost matrix;

[0393] The cost feedback mechanism is to monitor the recall rate, accuracy rate, and F1 score indicators of minority class classification. The AC-DCC model can dynamically adjust the weight of the cost matrix to improve the performance of the model on minority classes.

[0394] The reinforcement learning mechanism is that the AC-DCC model needs to dynamically adjust the allocation of the cost matrix according to the classification performance, and then through reinforcement learning, the model can adaptively optimize the cost-sensitive classification strategy to ensure that minority class samples are better processed; using the Actor-Critic model in reinforcement learning, the adjustment of the cost matrix is ​​used as part of the strategy optimization, and the model performance is improved through continuous feedback and optimization.

[0395] S5.33 Weight adjustment of minority samples: Through the Actor-Critic mechanism of the AC-DCC model, the weights of minority samples are adjusted according to the cost matrix to ensure that they receive more attention in the pseudo-label generation process;

[0396] The weight adjustment strategy is based on the weight distribution of the cost matrix. The AC-DCC model generates higher weights for minority class samples through the Actor network, and the Critic network provides reward feedback based on the classification performance to further optimize the weight generation process.

[0397] The pseudo-label generation optimization is achieved by adjusting the weights of reinforcement learning. The model dynamically assigns weights to minority class samples to ensure that the classification performance of minority class samples in an unbalanced data environment is improved, thereby improving the accuracy of pseudo-label generation and ensuring that minority class samples in the training data set are assigned more reasonable pseudo-labels.

[0398] S5.4 Model training process

[0399] S5.41 State input: The process variables of the unlabeled samples and the states after the pseudo labels are spliced ​​are input into the AC-DCC model;

[0400] The process variable characteristics include time difference, equipment utilization, and output deviation, which are used to indicate the current operating status of the equipment;

[0401] The state representation is to construct the state of the current sample using the concatenated feature information;

[0402] The process variable splicing is to splice the process variables (such as time difference, equipment utilization rate) of the unlabeled sample with the pseudo label to form a complete sample state;

[0403] The quantification of equipment operation characteristics: it is necessary to quantify the equipment operation data and production deviation characteristics into inputs that can be processed by the model;

[0404] The constructed state vector is a specific process variable, such as time difference, equipment utilization, and output deviation, which represents the current operating status of the equipment and is input into the model for training; and the pseudo-label is spliced ​​together with these features as auxiliary information to form a state input;

[0405] The state is the operating data recorded for a certain device during the production process, including the start-up time, the operating time, the deviation between the planned output and the actual output, and the equipment utilization rate. This information constitutes the vector representation of the device state;

[0406] S i =[time diffutilization rate, output deviation,…]

[0407] S5.42 Action Generation: The Actor network of AC-DCC outputs the weight of each sample, which affects the pseudo-label generation process;

[0408] The weight generation strategy is that the Actor network generates weights to ensure that a higher attention weight is given to minority class samples and the importance of samples in the training process is dynamically adjusted;

[0409] The weight allocation mechanism is that the Actor network generates weights according to the current state, which is used to adjust the influence of minority samples when generating pseudo labels. It must ensure that the generated weights can dynamically adjust the priority of samples, especially giving higher weights to minority samples;

[0410] The dynamic adjustment mechanism is that the model generates actions based on the current state information and generates appropriate pseudo labels for unlabeled samples;

[0411] The action generation is that in a certain state (i.e., the operating data characteristics of the equipment), the TPG model outputs a pseudo-label set; this set represents the probability that the pseudo-label belongs to different categories; the final action is to select a pseudo-label for classification, such as classifying a given sample as "normal" or "fault category 1", "fault category 2";

[0412] A i =Predict\_Label(S i)

[0413] Among them, if the output is a probability distribution [P 1 ,P 2 ,P 3 ], the final action is to select the category corresponding to the maximum probability;

[0414] S5.43 Classification and Feedback: The Critic network evaluates the quality of pseudo-label generation and generates feedback based on the classification performance (e.g., minority class recall, F1 score);

[0415] The performance evaluation and feedback uses the performance of the classifier (especially the recall rate and F1 score of minority class samples) to evaluate the effect of pseudo label generation;

[0416] The feedback loop is that the Critic network feeds back the performance of the classifier to the Actor network to help optimize the weight allocation strategy;

[0417] The classification evaluation is that the Critic network generates rewards or penalties based on the performance of the classifier on the validation set to help the Actor network optimize the weight strategy. The feedback information must be transmitted to the Actor network in a timely manner for optimization;

[0418] S5.44 Policy Update: Use the policy gradient method to optimize the Actor network and adjust the sample weights to ensure that minority samples receive more attention;

[0419] The policy gradient optimization is based on feedback information. The Actor network will be optimized or updated through a policy gradient algorithm (such as REINFORCE) to dynamically adjust the strategy for pseudo-label generation to ensure that the model can focus on minority class samples.

[0420] The dynamic weight update is to dynamically adjust the weight generation strategy of the Actor network according to the classification feedback, to ensure that the minority class samples receive more attention in subsequent training, and to ensure dynamic optimization of the sample weight distribution and the quality of pseudo-label generation.

[0421] S6. Reinforcement Learning Process

[0422] S6.1 Reward Function Design

[0423] S6.11 Reward function based on minority class recall: In order to make the model pay more attention to the classification performance of minority class faults, the minority class recall is introduced as the reward target in the reward function; each time the model correctly classifies a minority class fault, a higher reward will be given; the minority class recall is introduced into the reward function design and its weight is controlled to ensure that the model pays more attention to the identification of minority classes;

[0424] The minority class recall rate is an indicator for evaluating the model's ability to detect minority class samples, and is used to measure the model's performance in identifying minority class faults;

[0425] The weight parameter α is a weight parameter of the minority class recall rate, and the target weight of the model in the reinforcement learning process is adjusted by this parameter to ensure that the minority class recall rate is given enough attention;

[0426] S6.12 Combined with overall classification accuracy: In order to ensure that the model does not affect the overall classification performance while improving the minority class recall, the overall classification accuracy is also added to the reward function; correct minority class classification will increase the reward, and incorrect minority class classification will be punished; the overall classification accuracy is introduced and its weight is set to ensure a balance between the minority class recall and the overall classification accuracy;

[0427] The overall classification accuracy is used to measure the classification performance of the model on all categories, ensuring that the accuracy of the model is not neglected due to excessive attention to a minority class;

[0428] The weight parameter β can control the weight of the overall classification accuracy, ensuring that the overall accuracy of the model is maintained while paying attention to the recall rate of the minority class;

[0429] S6.13 Construct the reward function formula: Construct the reward function formula based on the weights of the minority class recall rate and the overall classification accuracy rate; each time the model correctly classifies a minority class fault, a higher reward is given; if the minority class is misclassified, a penalty is given. The reward function formula is:

[0430] R = α × Recall minority +β×Accuracy overall

[0431] Among them, α and β are weight parameters that control different objectives, ensuring that the model maintains the overall accuracy while improving the recall rate of the minority class;

[0432] S6.2 Cost Matrix Construction

[0433] S6.21 Cost matrix construction: First, analyze the category distribution based on the initial data set, paying special attention to the ratio of minority class to majority class; then, generate a cost matrix based on the category distribution, and set the classification error cost of minority class samples to be higher than that of majority class samples, so as to ensure that minority class samples receive more attention during model training; finally, apply the generated cost matrix to the critic network as part of the weighted cross entropy loss function, and ensure that the model can learn and identify minority class samples more accurately during training by weighting the minority class error cost;

[0434] The category distribution analysis is to determine the proportion of each category by statistically analyzing the distribution of sample categories, especially analyzing the sample proportions of the minority class and the majority class;

[0435] The cost matrix generation can assign corresponding cost values ​​to each category according to the analysis results, and the error cost of the minority class is usually much higher than that of the majority class;

[0436] The weighted cross entropy loss is a weighted cross entropy loss function based on the cost matrix, which ensures that the minority class has a greater influence in training; the loss function in the Critic network must use weighted cross entropy to optimize the classification performance of the minority class based on this matrix;

[0437] S6.22 Data partitioning: First, the total data set is divided into a training set (70%), a validation set (15%), and a test set (15%) according to common proportions; during the partitioning process, a stratified sampling method is used to ensure that minority class samples are evenly distributed in the training, validation, and test sets, thereby ensuring the generalization performance of the model on different data sets and the consistency of the treatment of minority class samples;

[0438] The data set division is to ensure that the model has enough samples in different stages (training, validation, and testing) through a reasonable division strategy;

[0439] The stratified sampling is to ensure that the ratio of minority class and majority class samples in each data set is consistent with the total data set, and to ensure that minority class samples are evenly distributed in each data set, thereby avoiding the model from being unable to effectively process minority class samples during testing;

[0440] S6.3 Training Process

[0441] S6.31 Batch training: Actor network weight generation, training the data set in batches; each batch of data first passes through the Actor network to generate sample weights, and then is classified in the Critic network to assist in processing large-scale data and improve the stability and efficiency of the model;

[0442] The Actor network is used to generate dynamic weights for balancing minority and majority class samples;

[0443] The Critic network classifies samples according to their weights to ensure the accuracy of the model;

[0444] S6.32 Performance evaluation and feedback: After each round of training, the performance of the model is evaluated using the validation set, and then the model classification effect is evaluated through indicators such as minority class recall and F1 score. Based on the evaluation results, the reward function is fed back to the Actor network to update the strategy.

[0445] The validation set is used to evaluate the performance of the model and ensure that the model is continuously optimized during the training process;

[0446] The reward function is fed back to the Actor network based on the classification effect to dynamically adjust the strategy;

[0447] S6.33 Optimizer optimization parameters: Use the Adam optimizer to optimize the parameters of the Actor and Critic networks respectively; the initial learning rate is set to 0.001, and the model adjusts the learning rate according to the training performance to speed up or slow down the learning process;

[0448] The Adam optimizer, as a kind of gradient descent algorithm, optimizes the model through momentum and adaptive learning rate;

[0449] The learning rate adjustment strategy is used to dynamically adapt to the model training situation;

[0450] S6.4 Strategy Optimization

[0451] S6.41 The goal of policy gradient optimization is to optimize the strategy of the TPG model through the policy gradient method, find an optimal strategy to maximize the long-term accumulated rewards, and make the pseudo-labels generated in the future better meet the requirements of the classification task;

[0452] The optimization goal is to enable the model to learn to generate higher quality pseudo labels through reinforcement learning;

[0453] The long-term accumulated reward maximization strategy is to find an optimal strategy to maximize the long-term accumulated rewards;

[0454] S6.42 Policy Gradient Optimization Algorithm: Use a policy gradient optimization algorithm (such as REINFORCE) to update the model's strategy for generating pseudo labels; and optimize the model parameters based on reward feedback by calculating the gradient of the strategy; the policy gradient optimization formula is:

[0455]

[0456] Among them, π θ (α|s) is the probability of choosing action a in state s, and R(s,a) is the corresponding reward;

[0457] The REINFORCE policy gradient optimization algorithm is used to adjust the pseudo label generation strategy of the model;

[0458] The calculating of the policy gradient to guide the optimization process is to optimize the model parameters based on reward feedback by calculating the gradient of the policy;

[0459] S6.5 Iterative Training Process

[0460] S6.51 Generate pseudo labels: In each round of reinforcement learning, TPG first generates pseudo labels to provide initial classification information for unlabeled data samples;

[0461] The TPG generates pseudo labels using a neural network or other generators; through model learning, unlabeled samples are converted into samples with pseudo labels;

[0462] S6.52 Train classifier: Use samples with real labels and generated pseudo labels to train the classifier and improve the ability to classify data;

[0463] S6.53 Calculate rewards: Calculate rewards based on the performance of the classifier on the validation set, especially the recall rate of the minority class;

[0464] S6.54 Update strategy: Based on the calculated reward feedback, use a policy gradient optimization algorithm (such as REINFORCE) to update the model parameters and update and adjust the TPG strategy for generating pseudo labels.

[0465] Phase 2: Training and application of fault classification model

[0466] S7. Fault classification model training (offline training)

[0467] S7.1 Integration of pseudo labels and real labels: Use the pseudo label generator to generate labels for unlabeled data, and integrate the pseudo labels with the real labels to construct a complete training dataset and a complete dataset;

[0468] The data set includes a label of the fault type and characteristic data related to the equipment;

[0469] S7.2 Input data preparation: Take the labeled data set as input, including time difference, equipment utilization, output deviation, alarm data, and historical maintenance record feature data; preprocess the data (such as time difference, equipment utilization) and standardize or normalize it to ensure that the data format is unified and suitable for input classification model, and complete data preprocessing and feature engineering;

[0470] S7.3 Model selection: Select an appropriate supervised classification model, such as support vector machine (SVM), random forest, XGBoost, or neural network, based on the complexity of the problem; Select and configure supervised learning models, and select an appropriate model based on the complexity of the problem;

[0471] The different models are suitable for different complexities and data distributions, and the model can be selected based on data characteristics, model interpretability, and performance;

[0472] S7.4 Classification model training: Input the constructed labeled data set into the classification supervised learning model to optimize the parameters; improve the classification effect by optimizing the model parameters; the model will improve the fault classification ability by learning the mapping relationship between data features and fault labels;

[0473] S7.5 Model evaluation: Use the validation set to evaluate the generalization ability of the model, use common classification indicators such as accuracy, precision, recall, and F1 value to comprehensively evaluate the model performance, and optimize the model.

[0474] S8. Online fault classification

[0475] S8.1 Online diagnostic process:

[0476] S8.11 Sample input: Input newly collected production data (such as time difference, equipment operation time, alarm records) into the trained fault classification model for real-time data collection and processing;

[0477] The characteristics of the data input include time difference, equipment operation time, and alarm information. The data needs to be imported into the model in real time for prediction;

[0478] S8.12 Label prediction: The model outputs the predicted label of the equipment failure in real time based on the characteristics of the input samples;

[0479] The online model prediction is that the model classifies the input sample data in real time and outputs the fault category. The real-time response capability of the model can quickly and accurately identify the fault type;

[0480] S8.13 Fault Identification: The model can detect in real time whether the equipment is in a fault state, identify the specific type of fault, and generate corresponding alarm information;

[0481] The fault identification and alarm means that the model can timely identify the fault of the equipment and generate an alarm through the system to identify the location and type of the equipment fault;

[0482] S8.2 Real-time Adjustment:

[0483] S8.21 Alarm handling: If a fault is detected, the system will immediately generate an alarm and mark the faulty device and the type of fault;

[0484] S8.22 Production plan adjustment: The system automatically adjusts the production plan based on the diagnosis results of the faulty equipment and reallocates tasks to other equipment to ensure the continuity of the production line;

[0485] The linkage between the production scheduling system and the fault classification model is that when a certain equipment fails, the production scheduling system can automatically adjust the workload to ensure the continuity of production;

[0486] The dynamic task allocation is to ensure that the system can flexibly allocate tasks to avoid affecting the overall production due to the failure of a single device.

[0487] Example 2 Specific implementation plan based on hybrid model design of offline and online fault classification

[0488] like Figure 2 As shown in the figure, in a specific embodiment, around industrial data sets, by combining offline and online data processing and fault classification techniques, it is demonstrated how to efficiently classify and predict equipment failures. The specific operation process is shown in the figure, which is divided into two main modules: offline data processing and classification and online fault classification.

[0489] Data collection and division is that the system of the present invention first obtains data from industrial equipment, including offline and online data. The offline data set is used for model training and verification, and the online data set is used for real-time fault classification. In the offline data processing module, the data is divided into a training set and a verification set, and the online data is directly used for subsequent online fault classification.

[0490] Data standardization is the process of standardizing both offline and online data to ensure that feature data of different dimensions (such as time difference, equipment utilization, and output deviation) are on the same scale. Standardized data is more suitable for model training and prediction.

[0491] In offline fault classification preprocessing, the labeled samples are labeled samples in offline data. The system will train the classification model through preliminary pre-training steps and use the existing fault labels to improve the initial prediction ability of the model. The unlabeled samples need to be processed by the task pseudo-label generator (TPG). The TPG model generates pseudo labels for unlabeled data through existing labeled samples, thereby expanding the scale of the training set.

[0492] In pseudo-label generation and offline fault classification, the generated pseudo-labels are merged with labeled samples to form a complete data set. Then, the system uses the generated pseudo-labels and real labels to train the offline fault classification model and build an efficient classification model.

[0493] In online fault classification, online fault classification is based on the trained offline model. The online data collected in real time is input into the model after being standardized, and the system monitors the operating status of the equipment in real time and classifies faults. When a fault is detected, the system can immediately generate an alarm message and output the fault type. At the same time, the classification results are also used for dynamic adjustment of production scheduling to ensure the continuity of the production line.

[0494] Once the offline fault classification model is trained, the system deploys it to the online monitoring system. The system collects new data from the equipment in real time through sensors and control systems, and inputs these data into the trained classification model after standardization. The model predicts the operating status of the equipment based on the input feature data (such as time difference, equipment utilization, and alarm records) and determines whether a fault has occurred.

[0495] If a fault is detected, the system will generate a corresponding alarm message and mark the faulty device and the type of fault (such as overheating of the device motor, communication failure). At the same time, the system dynamically adjusts the production plan based on these classification results and reallocates tasks to other devices to ensure that the production line can run continuously and avoid downtime losses.

[0496] In the feedback and adjustment of classification results, for online data classification results, the system will dynamically adjust the production plan based on the equipment's alarm information and classification results. If the equipment is identified as a fault state, the system will immediately reallocate tasks to other equipment to avoid production interruptions, and ensure timely maintenance of the equipment through fault alarms.

[0497] In this embodiment, TPG is used to generate pseudo labels, combined with offline model training and online real-time fault classification, to form a complete industrial fault classification and production adjustment system. In practical applications, it can not only improve the classification model's ability to identify minority faults, but also optimize the production process and improve equipment utilization.

[0498] Example 3 Specific implementation plan of cost-sensitive training TPG based on Actor-Critic

[0499] In a specific embodiment, cost-sensitive training is performed on unlabeled samples using an Actor-Critic reinforcement learning framework around a task pseudo-label generator (TPG). Its main purpose is to improve the pseudo-label generation quality and classification accuracy of minority class samples by dynamically adjusting the weights of samples, especially in an unbalanced data environment.

[0500] In terms of unlabeled data input, unlabeled process data is obtained from the industrial production system, including time difference during equipment operation, equipment utilization, output deviation, and alarm record data; the unlabeled data cannot be directly used for training the classification model, and corresponding labels need to be generated for these data through a pseudo label generator (TPG).

[0501] In sample feature extraction, for each unlabeled sample, the system first extracts its process variable features, for example:

[0502] sample Process variable characteristics Time difference The time difference between the start and stop of the device Equipment utilization Equipment efficiency Output Deviation The difference between actual output and planned output Alarm information The alarm type and time of the device

[0503] These features are input into the Actor network as states to characterize the running state and characteristics of each sample.

[0504] In terms of sample weights generated by the Actor network, the Actor network generates corresponding weights based on the state characteristics of the unlabeled samples. For minority samples, the Actor network will give them higher weights to ensure that these samples receive more attention in the subsequent pseudo-label generation. This dynamic weight mechanism solves the problem of data imbalance and makes the pseudo-label generation quality of minority samples higher.

[0505] In pseudo-label generation, the TPG model is used to infer the input unlabeled samples and generate pseudo-labels. Pseudo-labels represent the fault type or operating status (such as "normal", "fault 1", "fault 2") corresponding to the unlabeled data. The pseudo-label of each sample depends not only on its features, but also on the weights generated by the Actor network, thereby improving the reliability of pseudo-label generation.

[0506] In the Critic network evaluation, the Critic network is responsible for evaluating the quality of pseudo-label generation, especially the classification performance of minority class samples. The evaluation indicators include:

[0507] index Evaluation Content Minority class recall Whether the model can correctly identify all minority class samples. F1 score The weighted average of precision and recall for the minority class.

[0508] The Critic network provides feedback signals based on these indicators to determine whether the current sample weight strategy is effective.

[0509] In terms of reward feedback, if the Critic network evaluates that the quality of pseudo-label generation is high, especially in the classification of minority samples, the system will generate a positive reward signal to improve the weight allocation strategy of the Actor network. If the effect is not good, the Critic network will generate a penalty signal to prompt the Actor network to adjust the sample weight strategy.

[0510] In terms of strategy updating, the Actor network adjusts the weight generation strategy based on the feedback signal from the Critic network through a policy gradient optimization algorithm (such as REINFORCE). The Actor network dynamically adjusts the sample weight distribution, especially optimizing the weights of minority class samples, so that they receive more attention in the subsequent pseudo-label generation and improve classification accuracy.

[0511] In terms of cost matrix optimization, while evaluating the critic network, the system introduces a dynamic cost matrix mechanism to assign higher misclassification costs to minority samples. This mechanism uses a weighted cross entropy loss function to ensure that the model pays more attention to the correct classification of minority samples during training to avoid ignoring them.

[0512] In iterative training, the system continuously optimizes the pseudo-label generation and weight distribution strategy by iterating the above steps multiple times. After each round of iteration, the Actor network dynamically adjusts according to the feedback from the Critic network to gradually improve the quality of pseudo-label generation and classification effect. Through this continuously optimized reinforcement learning mechanism, the model's classification performance for minority class samples continues to improve.

[0513] At the final model evaluation and termination, when the Critic network detects that the quality of the model's pseudo-label generation reaches the preset target, the system stops training. At this time, the minority class recall rate and F1 score indicators meet the requirements, indicating that the model's recognition ability for minority class samples has been significantly improved. Finally, the system outputs high-quality pseudo-labels and classification models.

[0514] This embodiment effectively solves the problem of pseudo-label generation and insufficient classification performance of minority class samples in unbalanced data scenarios by introducing the Actor-Critic reinforcement learning framework. The system dynamically adjusts the weights of minority class samples to ensure that they receive more attention during the training process, ultimately improving the classification model's recognition accuracy for equipment faults, especially in the detection of minority class faults.

[0515] Example 4 Dynamic weight adjustment fault classification based on AC-DCC model

[0516] The introduction of the AC-DCC model corresponds to the following steps: S5 offline TPG training and S6 reinforcement learning process. S5.1 introduces the AC-DCC model. During the TPG training process, the weights of samples, especially the weights of minority samples, are dynamically adjusted through the reinforcement learning framework of the AC-DCC model. During the training process of S6.3, during batch training, the Actor network of AC-DCC generates dynamic weights, and the Critic network optimizes the weight allocation strategy based on the classification performance feedback.

[0517] In a specific embodiment, the step of introducing the AC-DCC model corresponds to S5 training offline TPG and S6 reinforcement learning process in the technical solution of the present invention. The core function of the introduction of the AC-DCC model in the TPG training process (S5.1) and the entire reinforcement learning process (S6.3) is to dynamically adjust the sample weights, especially the weights of minority class samples, through the Actor-Critic mechanism to ensure that minority class samples can be more effectively identified during the generation of pseudo labels and fault classification. This dynamic adjustment mechanism enables the model to perform better in the case of unbalanced data.

[0518] This example demonstrates the process of dynamically adjusting sample weights to improve the classification performance of minority samples in an unbalanced dataset through the AC-DCC model. It is divided into two stages: the TPG (pseudo label generator) training stage and the weight optimization stage based on reinforcement learning.

[0519] Phase 1: TPG training process (mainly for the introduction of AC-DCC model in S5.1)

[0520] During the TPG training phase, the AC-DCC model was introduced to dynamically adjust sample weights using the Actor-Critic architecture in reinforcement learning. In the dataset, unlabeled samples and minority samples are usually difficult to classify effectively due to sparse data or unclear labels. To this end, AC-DCC dynamically adjusts sample weights through a reinforcement learning framework, especially assigning higher weights to minority samples to ensure the accuracy of pseudo-label generation.

[0521] In state input: the system first collects the operating data of industrial production equipment, including equipment utilization, time difference, and output deviation variables, as the input state; in addition, for unlabeled samples, the pseudo-labels generated by TPG are also spliced ​​into the state to form a complete input state.

[0522] For example, for a certain device on a production line, the input status includes: the startup time and actual running time of the device; the deviation between the actual output of the device and the planned output; and the pseudo-label distribution generated by TPG, which indicates the probability that the sample belongs to fault category 1, fault category 2, or normal operation.

[0523] In the Actor network generation action, based on the input state information, the Actor network generates a dynamic weight for each unlabeled sample. By analyzing the feature information of process variables and pseudo labels, the Actor network is able to assign higher weights to minority class samples to ensure that their influence is enhanced during the training process.

[0524] For example, for a minority class sample predicted to be fault class 2, the Actor network generates a higher weight such as 0.8, and for a majority class sample, a lower weight such as 0.2.

[0525] In the Critic network feedback, the Critic network generates feedback signals based on the performance of the classifier on the validation set, especially the recall rate and F1 score indicators of the minority class; the Critic judges the rationality of the sample weight by analyzing the current classification performance. If the recall rate of the minority class sample is insufficient, the Critic will provide negative feedback to optimize the weight strategy generated by the Actor network.

[0526] For example, if the recall rate of minority class samples is low in a certain round of classification, the Critic will give negative feedback, prompting the Actor network to give higher weights to minority class samples in the next round of training.

[0527] In the reward mechanism, after each round of training, the system calculates rewards based on the performance of the classifier. Correct classification of minority class samples will bring higher rewards, while incorrect classification will result in penalties. The Critic network determines the reward or penalty for the Actor network by comparing the minority class recall rate and F1 score key indicators, thereby optimizing the weight allocation strategy for the next round. Phase 2: Reinforcement learning process based on the AC-DCC model (S6.3 Cost-sensitive mechanism of reinforcement learning)

[0528] In order to solve the problem of data imbalance, a cost matrix is ​​introduced in this embodiment. The minority class samples in the cost matrix are assigned a higher classification error cost, so that the model pays more attention to the correct classification of minority class samples during the training process.

[0529] In the cost matrix construction, the cost matrix is ​​first constructed according to the ratio of minority class to majority class in the data set. The error cost of minority class samples is set higher than that of majority class, so that the classification error cost of minority class samples has a greater impact on the model loss function.

[0530] For example, for fault category 1 (minority class), the cost matrix sets the misclassification cost to 5; while for normal operation (majority class), the misclassification cost is set to 1.

[0531] Dynamically adjust the cost matrix. During the training process, the AC-DCC model dynamically adjusts the cost distribution of the cost matrix, especially the cost of minority class samples will be adjusted according to the training results. If the recall rate or F1 score of minority class samples is low, the system will automatically increase the classification error cost of this category to ensure that in the next round of training, the classifier will pay more attention to the accurate classification of minority class samples.

[0532] For example, in one round of training, the recall of fault class 2 (the minority class) decreases, and the system increases the error cost of this class from 5 to 7 to further emphasize its importance.

[0533] In action generation and feedback, during each reinforcement learning process, the AC-DCC model inputs the state information of unlabeled samples (including process variables and pseudo labels) into the Actor network. The Actor network generates corresponding weights for each sample based on these input states. The system uses these weights to adjust the priority of pseudo label generation to ensure that minority samples occupy a more important position in training.

[0534] For example, if the pseudo label of a sample is "fault category 1", the Actor network assigns a weight of 0.85 to the sample, ensuring that it receives a higher priority in the subsequent classification process.

[0535] In the classification and feedback loop, after each round of training, the Critic network evaluates the performance of the classifier on the validation set, especially the recall rate and F1 score of minority class samples. Based on the changes in these indicators, the Critic network generates feedback signals and passes them to the Actor network to help it optimize the weight generation strategy for the next round.

[0536] For example, if the classification performance of minority class samples in the validation set is significantly improved, the Critic will give the Actor network a positive reward, thereby encouraging it to continue to maintain the corresponding weight distribution in subsequent training.

[0537] Through this embodiment, the dynamic cost-sensitive fault diagnosis system based on the AC-DCC model described in the present invention effectively solves the problem of difficulty in classifying minority class samples caused by data imbalance. After the introduction of the AC-DCC model, the recall rate and classification accuracy of minority class samples are significantly improved, the quality of pseudo-label generation is improved, and the overall performance of the model is optimized.

[0538] Example 5 Application of AC-DCC model based on cost matrix in fault classification

[0539] The cost matrix is ​​combined with AC-DCC, corresponding to steps: S5.3 Cost-sensitive mechanism of reinforcement learning and S6.2 Cost matrix construction. The cost matrix introduced in S5.31 is to solve the problem of unbalanced data, and the cost matrix is ​​used to set a higher classification error cost for minority class samples. The cost matrix construction in S6.2 generates a cost matrix based on the data category distribution and applies it to the Critic network, so that more attention is paid to minority class samples during training.

[0540] In a specific embodiment, it is demonstrated how to solve the data imbalance problem by introducing a cost matrix combined with an AC-DCC model, thereby effectively improving the detection capability of minority class samples in fault classification tasks. This method is mainly used in industrial equipment fault detection scenarios, where the data set includes a large number of normal samples (majority class) and relatively few fault samples (minority class). To this end, the system of the present invention sets a higher classification error cost for minority class samples through a cost matrix, and dynamically adjusts the weight of the samples through reinforcement learning to improve the detection capability of minority class samples.

[0541] Phase 1: Cost-sensitive mechanism of reinforcement learning (S5.31 introduces the cost matrix)

[0542] Introducing the cost matrix, in the case of data imbalance, minority class samples are easily ignored during model training. In order to ensure that the model can better focus on minority class samples, we introduced the cost matrix. The cost matrix assigns a higher classification error cost to minority class samples, making the model more sensitive to classification errors of minority class samples during training.

[0543] When performing category distribution analysis, we first analyze the industrial data set and count the distribution of different fault categories. Assume that in the industrial data set, "normal operation" samples account for 80%, while "fault category 1" and "fault category 2" samples account for 10% each. Such category imbalance will cause the model to be more biased towards the majority class, that is, the normal operation samples, and ignore the minority class samples during training.

[0544] For example, the dataset categories are distributed as follows:

[0545] Dataset category Distribution ratio Normal operation sample 80％ Fault category 1 (minority category) 10％ Fault Category 2 (Minority Category) 10％

[0546] In the cost matrix setting, a cost matrix is ​​generated according to the data distribution results to weight the classification error cost. In order to enhance the model's attention to minority class samples, the cost matrix assigns a higher error cost to minority class samples. For example, the following table:

[0547] Misalignment Cost set up The cost of misclassification of normal running samples (majority class) 1 Misclassification cost for fault category 1 (minority class) 5 Misclassification cost for fault category 2 (minority class) 5

[0548] Therefore, the cost matrix is ​​expressed as:

[0549] Cost matrix = [1 5 5]

[0550] With this cost distribution strategy, if the model misclassifies minority samples during training, the cost will be very high, forcing the model to be more sensitive to these samples during training.

[0551] The cost matrix is ​​used in training. In each round of training, the cost matrix is ​​embedded in the model's loss function. The weighted cross entropy loss function is used to ensure that minority class samples have a greater weight in the loss calculation. The weighted cross entropy loss function formula is:

[0552]

[0553] For example, for normal operation samples, the cost of misclassification is low (weight is 1); for fault category 1 or fault category 2 samples, the cost of misclassification is high (weight is 5).

[0554] Phase 2: Combining the cost matrix with the AC-DCC model (S5.32 Dynamically adjust the cost-sensitive classification strategy)

[0555] During the training process, the AC-DCC model dynamically adjusts the cost distribution in the cost matrix so that the minority class samples get higher weights. The dynamic adjustment of the cost matrix is ​​to optimize the classification strategy in real time based on the performance of the classifier on the validation set (such as the minority class recall rate and F1 score).

[0556] When monitoring classification performance, the system evaluates the current model's processing effect on minority class samples by monitoring the recall rate, precision rate, and F1 score indicators of minority class samples. If the recall rate of minority class samples in the validation set is low, the AC-DCC model will dynamically increase the classification error cost of this category.

[0557] For example, in a validation set evaluation, the recall rate of fault category 1 is low, only 40%. At this time, the system will increase the classification error cost of this category from 5 to 6 so that the model will pay more attention to this category in the next round of training.

[0558] In the cost feedback mechanism, the AC-DCC model evaluates the current classification performance through the Critic network and dynamically adjusts the sample weights generated by the Actor network based on the feedback. This feedback mechanism ensures that the weight distribution of minority class samples is dynamic and optimized at any time based on the classification performance.

[0559] For example, the Critic network detects that the recall rate and F1 score of fault category 1 have greatly improved, giving the Actor network a positive feedback signal to maintain a high weight allocation for this category. If the recall rate and F1 score of fault category 2 perform poorly in a certain round of training, the Critic network will give negative feedback, prompting the Actor network to allocate a higher weight to this category in the next round.

[0560] Phase 3: Training process and strategy optimization (S6.21 cost matrix construction and S6.22 batch training and performance feedback)

[0561] The system of the present invention preliminarily constructs a cost matrix based on the category distribution, and dynamically adjusts the cost distribution of the matrix based on training feedback, ultimately ensuring that the cost of each class of samples matches the actual classification performance. The adjustment of the cost matrix directly affects the loss calculation of the Critic network.

[0562] For example, the dataset categories are distributed as follows:

[0563] Dataset category Distribution ratio Normal operation sample 70％ Fault category 1 (minority category) 15％ Fault Category 2 (Minority Category) 15％

[0564] In terms of data set division, the total data set is first divided into 70% training set, 15% validation set and 15% test set. In the data division process, a stratified sampling strategy is adopted to ensure that the proportion of minority class and majority class samples in each data set remains consistent.

[0565] For example, in the training set, normal operation samples account for 70%, and fault category 1 and fault category 2 each account for 15%. The category distribution in the validation set and test set is consistent with the training set, ensuring that the model has consistency and generalization ability on different data sets.

[0566] In the batch training and performance feedback phase, during each batch training process, the AC-DCC model dynamically generates sample weights through the Actor network and performs classification and cost evaluation in the Critic network. The Critic network calculates the classification loss of each sample through the weighted cross entropy loss function combined with the cost matrix.

[0567] In terms of strategy optimization and update, the Actor network continuously optimizes the weight distribution strategy through the feedback signal of the Critic network. After each training, the system calculates the reward or penalty based on the classification performance and updates the Actor network's strategy through the policy gradient optimization algorithm (such as REINFORCE).

[0568] For example, after a training run, the recall and F1 score of fault category 1 are significantly improved, and the Critic network rewards the Actor network to encourage it to continue to assign high weights to this category in future training. If fault category 2 performs poorly, the Actor network will adjust the weight generation strategy through policy gradient optimization to assign a higher weight to this category.

[0569] Through this embodiment, the system of the present invention successfully solves the problem of data imbalance. The cost matrix is ​​combined with the AC-DCC model, so that the model can dynamically adjust the weights and classification error costs of minority class samples according to the classification performance, significantly improving the recall rate and classification accuracy of minority class samples. In actual verification, the recall rates of fault category 1 and fault category 2 increased by 15% and 20% respectively, and the F1 score also increased significantly. The dynamic adjustment of the cost matrix ensures that the AC-DCC model can adaptively handle the problem of uneven sample distribution, effectively improving the classification performance of the model.

[0570] Example 6: Pseudo-label generation and model training of unlabeled samples based on TPG (task pseudo-label generator)

[0571] The use of the pseudo-label generator (TPG) corresponds to steps: S4 pre-training TPG and S6.5 iterative training process. S4.1 Using TPG means that TPG is used to generate pseudo labels for unlabeled samples to ensure that unlabeled data can be used in model training. S6.51 Generating pseudo labels means that TPG generates pseudo labels in each round of reinforcement learning to provide initial classification information.

[0572] In a specific embodiment, it is demonstrated how to generate pseudo labels for unlabeled samples through a task pseudo label generator (TPG), so as to effectively use unlabeled data in the training process of the classification model and improve the classification performance of the model. This method is suitable for industrial fault detection scenarios where there is a large amount of unlabeled data and minority class samples are difficult to obtain, so as to enhance the diversity of data sets and the generalization ability of models.

[0573] Phase 1: Pre-training TPG (S4)

[0574] In this embodiment, TPG is a multi-layer perceptron (MLP) neural network, which is used to generate pseudo labels for unlabeled samples. The input of TPG is process variable characteristics (such as time difference, equipment utilization, output deviation), and the output is the probability distribution of pseudo labels of different categories. To ensure that TPG can generate accurate pseudo labels for unlabeled samples, TPG is first pre-trained using labeled samples.

[0575] In the process variable feature input, the input includes various data of the equipment in the production process, such as time difference, equipment running time, and equipment utilization variables. The feature vector of each sample includes:

[0576] The feature vector of each sample Connotation Time difference The difference between the device's scheduled startup time and the actual startup time Equipment utilization The difference between the actual production volume and the planned production volume of the equipment Output Deviation The difference between the actual production volume and the planned production volume of the equipment Alarm data Equipment alarm type and frequency

[0577] The network structure is a multi-layer perceptron (MLP), in which each layer is a fully connected layer. The input layer receives the above process variable features, propagates through the hidden layer layer by layer, and the output layer generates pseudo-label probability distributions of different categories through the Softmax activation function.

[0578] Pre-training with labeled samples. In order to give the TPG network better initial parameters, first pre-training with samples with known labels is used. In this process, TPG uses labeled samples to learn how to map input features to corresponding fault categories. Cross-entropy loss function. During the pre-training process, TPG uses the cross-entropy loss function to measure the difference between the pseudo-label predicted by the model and the true label. By continuously minimizing the loss, TPG gradually learns how to generate high-quality pseudo-labels. After the initial parameters are set and the pre-training is completed, the parameters of TPG are optimized to generate more accurate pseudo-labels. At this point, the TPG network has good generalization ability and can generate high-quality pseudo-labels for unlabeled samples.

[0579] Phase 2: Iterative training process (S6.51 generates pseudo labels)

[0580] In the iterative training process of reinforcement learning, TPG is responsible for generating initial pseudo labels for unlabeled samples. These pseudo labels provide additional labeled data for classifier training, so that unlabeled samples can also be effectively utilized.

[0581] In the unlabeled sample input, the system inputs the collected unlabeled sample data into the TPG network, and TPG generates pseudo labels based on the process variable characteristics. For example:

[0582] The characteristic vector of the input sample is: [time difference = 5 minutes, equipment utilization = 85%, output deviation = 3%];

[0583] After processing, TPG outputs the probability distribution of pseudo labels, for example: [fault category 1 = 0.2. fault category 20.7, normal = 0.1];

[0584] The system selects "fault category 2" as the pseudo label for this sample based on the maximum probability.

[0585] Pseudo labels are used in model training, and the generated pseudo labels are added to the training set and used together with the real labeled data to train the classifier. This process significantly expands the scale and diversity of the training data and improves the generalization ability of the model.

[0586] During the training process, samples with pseudo labels and real labels are input into the classifier for training. The classifier learns how to correctly classify new samples based on the input sample features and their labels.

[0587] In classification model training, samples with pseudo labels and real labels are input into the classifier for training. Common classification models include support vector machine (SVM), random forest or neural network.

[0588] For example, a batch of training data contains 80% real label samples and 20% pseudo label samples. The classifier continuously learns the mapping relationship between process variable features and fault category labels through this data.

[0589] In reward calculation, after each round of training, the system calculates rewards based on the performance of the classifier on the validation set (especially the recall rate and F1 score of the minority class) to evaluate the generation quality and classification effect of the current pseudo-label.

[0590] In the performance evaluation, the validation set is used to evaluate the performance of the classifier, and the system focuses on the recall, precision, and F1 score of minority class samples. For example, in the validation set evaluation, the classifier has a recall of 70% and an F1 score of 0.75 for minority class samples, which indicates that the quality of pseudo-label generation is high.

[0591] In reward calculation, the system calculates rewards based on the performance of the classifier. If the classifier performs well on minority samples, a positive TPG reward is given; if the performance is poor, negative feedback is given. The reward function can be calculated according to the following formula:

[0592] Reward = α × minority class recall + β × overall classification accuracy

[0593] Among them, α and β are weight parameters that control different objectives, ensuring that the overall classification accuracy is maintained while optimizing the recall rate of the minority class.

[0594] In the update strategy, based on the calculated reward feedback, TPG will update its pseudo-label generation strategy through a policy gradient optimization algorithm (such as REINFORCE) to improve the quality of subsequent pseudo-labels. In policy gradient optimization, TPG adjusts model parameters through the policy gradient optimization method in reinforcement learning. The policy gradient optimization formula is as follows:

[0595]

[0596] Among them, π θ (a|s) is the probability of choosing action a in state s, and R(s,a) is the corresponding reward.

[0597] When updating TPG parameters, the network weights of TPG are continuously adjusted according to the policy gradient optimization algorithm to ensure that the generated pseudo labels can better match the real fault labels, especially to improve the quality of pseudo label generation for minority class samples.

[0598] Through this example, TPG successfully generated high-quality pseudo labels for unlabeled samples. These pseudo labels were used together with real labeled samples to train the classifier, significantly improving the classification performance of the model. Specifically,

[0599] index Initial training value Improved value Recall rate of minority class samples 60％ 75％ F1 score 0.65 0.78

[0600] Conclusion: The reward mechanism of the system in reinforcement learning ensures that TPG continuously improves the accuracy and reliability of its generated pseudo labels.

[0601] Example 7 Application of TPG pre-training and network structure

[0602] The multi-layer perceptron (MLP) network is combined with the Softmax activation function, corresponding to step: S4.2 network structure. Among them, the S4.2 network structure is that the TPG network adopts the multi-layer perceptron (MLP) architecture and uses the Softmax activation function to generate the probability distribution of pseudo labels to improve the classification effect.

[0603] In a specific embodiment, a warehouse logistics production line is equipped with multiple devices such as stackers, elevators, shuttles, and conveying equipment, which are widely used in logistics handling. Among these devices, some devices often have alarms, maintenance, and failure shutdowns during operation. In order to improve the efficiency of system fault diagnosis, TPG (task pseudo-label generator) is used to generate pseudo labels for unlabeled samples, and predictions are made in combination with equipment operation records to improve the accuracy of equipment fault classification.

[0604] The following equipment data collection:

[0605]

[0606]

[0607]

[0608]

[0609] The pseudo labels are generated by the TPG model using the unlabeled data from the above equipment. In order to give the TPG network good initial parameters, it is first pre-trained using labeled training data.

[0610] Input data: Equipment utilization, downtime, alarm frequency and type, maintenance record process variables are used as feature inputs.

[0611] In the network structure, the multi-layer perceptron (MLP) architecture, the input layer accepts the multi-dimensional feature vector of the device, the hidden layer propagates the features layer by layer through the neural network, and the output layer uses the Softmax activation function to generate pseudo labels for each category.

[0612] For example, the categories of pseudo labels include: normal, mechanical failure, motor failure, communication problem.

[0613]

[0614] In the TPG model, the following network structure is used:

[0615] Input features: including equipment utilization, alarm type, maintenance duration, and actual operating duration process variables.

[0616] Softmax output layer: used to output the probability distribution of each pseudo-label category. For example, the pseudo-label of the stacker is generated as the failure probability distribution of the Softmax activation function output device)

[0617] Label probability distribution = [normal: 0.8, motor failure: 0.15, mechanical failure: 0.05]

[0618] Through the Softmax function, it is ensured that the sum of all output probabilities is 1.

[0619] Example 8 Application of dynamic weight adjustment mechanism in generating pseudo labels for minority class samples

[0620] In a specific embodiment, in the equipment fault classification task, minority class samples (such as rare fault types) are ignored in model training due to data imbalance. Therefore, through the dynamic weight adjustment mechanism of the AC-DCC model, the weight of minority class samples in the pseudo-label generation process is increased, so that it has a greater impact on the final classification. This mechanism uses the Actor-Critic architecture, the Actor network dynamically generates sample weights, and the Critic network provides feedback based on the performance of the classifier (such as the recall rate and F1 score of the minority class) to optimize the Actor network strategy.

[0621]

[0622]

[0623] The implementation steps of this embodiment are as follows:

[0624] S1. State input and pseudo-label generation (corresponding to S5.42 action generation)

[0625]

[0626] For example, for minority class samples, the weight will be set to a higher value, such as 0.8, while the weight of majority class samples is 0.2. In this way, minority class samples can have a greater influence in the subsequent pseudo-label generation.

[0627] S2. Dynamic feedback and strategy update (corresponding to S5.44 strategy update)

[0628] The Critic network feedback uses the classifier to evaluate the current pseudo-label generation, paying special attention to the recall and F1 score of minority class samples. For example, in the initial training phase, the recall of minority class samples is only 60% and the F1 score is 0.65. The Critic network generates a reward signal based on these indicators.

[0629] The Actor network updates the weight generation strategy based on feedback information. Through the policy gradient method, the updated Actor network will further increase the weight of minority samples. For example, adjusting the weight to 0.85 will increase the priority of minority samples in subsequent training.

[0630] S3. Iterative training process and model optimization

[0631] Pseudo-label generation and classification optimization is that in each round of reinforcement learning, the pseudo-label generator (TPG) generates new pseudo-labels and assigns higher weights to minority class samples according to the dynamic weight adjustment mechanism. After multiple rounds of training, the recall rate of minority class samples gradually increased from the initial 60% to 75%, and the F1 score also increased from 0.65 to 0.78.

[0632] Final classification effect: Through the dynamic weight adjustment mechanism, the performance of the classifier in dealing with minority class failures is significantly improved, and finally the classification accuracy of minority class samples is significantly improved.

[0633] stage Minority class recall F1 score Initial training 60％ 0.65 Implementation of dynamic weight adjustment mechanism 75％ 0.78

[0634] Conclusion: Through the dynamic weight adjustment mechanism of the AC-DCC model, the quality of pseudo-label generation for minority class samples has been effectively improved, and the performance of the classification model on minority classes has been significantly improved.

[0635] Example 9 Application of real-time diagnosis combined with dynamic production scheduling in industrial production

[0636] Real-time diagnosis is combined with dynamic production scheduling, corresponding to the steps: S8 online fault classification. The S8.1 online diagnosis process is that after fault detection, the system diagnoses the equipment status through real-time classification results. The S8.2 real-time adjustment is that after a fault is detected, the system dynamically adjusts the production plan according to the diagnosis results and reallocates tasks to ensure the continuity of the production line.

[0637] In a specific embodiment, in a modern intelligent manufacturing system, equipment failure leads to stagnation or reduced efficiency of the production line. Therefore, it is crucial to combine real-time fault diagnosis with dynamic production scheduling. Through the AC-DCC model, the system can automatically analyze the fault type after detecting equipment failure and adjust the production plan according to the real-time situation, reallocate tasks, and ensure the continuity of production.

[0638] Automated warehousing and production systems include multiple key equipment, such as stackers, elevators, conveyors, and shipping equipment. These devices work together to ensure that materials and finished products flow on time. After equipment failure detection, the system needs to reallocate production tasks based on the diagnosis results to ensure efficient operation of the production line and reduce downtime.

[0639] Dataset and settings:

[0640]

[0641] The implementation steps of this embodiment are as follows:

[0642] S1. Online diagnostic process (corresponding to S8.1 online diagnostic process)

[0643] S1.1 Real-time data collection: The system collects real-time equipment status data through sensors and control systems, such as equipment utilization, alarm records, temperature, and voltage key variables. The data is updated once a minute.

[0644] S1.2 Fault classification model operation: The system uses the pre-trained AC-DCC fault classification model to analyze real-time data and diagnose the operating status of the equipment in real time.

[0645] For example, a certain elevator (LFT002) detected a "motor overheating" alarm on October 1, 2024. The classifier determined through real-time data that this was a more serious motor failure.

[0646] S1.3 Diagnostic result: The system identified a motor failure in a certain hoist (LFT002) and automatically marked the status as "faulty". The equipment needs to be shut down for maintenance.

[0647] S2. Real-time adjustment and task redistribution (corresponding to S8.2 real-time adjustment)

[0648] S2.1 Production task reallocation: After the elevator (LFT002) stops, the system immediately re-plans the production tasks. The material handling task originally assigned to LFT002 is automatically transferred to another spare elevator LFT003.

[0649] At this time, the scheduling system updates the production plan according to the real-time status of the equipment. For example, cancel the material handling task originally planned on a certain elevator (LFT002); and then reallocate the task to LFT003 to ensure the continuity of the production line.

[0650] S2.2 Production plan adjustment: After the tasks are reallocated, the system updates the overall production plan. The new plan takes into account the utilization of spare equipment and current production requirements. For example, the operating time of LFT003 will be increased from 12 hours to 16 hours to take over the work of LFT002.

[0651] S2.3 Maintenance scheduling of downtime equipment: At the same time, the system marks a certain elevator (LFT002) of the faulty equipment as "under maintenance" and sends a notification to the maintenance team. The system automatically generates a maintenance work order and dispatches the maintenance department to handle the motor failure. The estimated repair time is 4 hours. After the maintenance is completed, the system will reactivate a certain elevator (LFT002) and put it into production tasks.

[0652] S3.Continuous operation of the production line

[0653] S3.1 Task monitoring and feedback: The system continuously monitors the execution of the reallocated tasks. For example, if the spare elevator LFT003 also has high load or other problems, the system will further adjust the production tasks or enable more spare equipment. If the spare elevator LFT003 operates normally as expected, the production plan can be completed smoothly to ensure that the delivery time is not affected.

[0654] S4.Maintenance feedback and equipment recovery

[0655] S4.1 Equipment repair: After the maintenance team completes the repair of the motor fault of LFT002, the system will receive a repair confirmation signal. LFT002 is back online, and the system automatically marks its status as "available" and reallocates part of the next production task to LFT002, reducing the workload of LFT003.

[0656]

[0657]

[0658] Conclusion: In this example, through the combination of real-time fault diagnosis and dynamic production scheduling, the production line can continue to operate when the elevator (LFT002) fails. The whole process avoids the phenomenon of production suspension due to equipment failure, and the production task is smoothly transferred to other equipment. At the same time, the faulty equipment elevator (LFT002) is maintained in time and put into use quickly after repair, ensuring the smooth execution of the production plan.

[0659] Pseudo code related to the present invention:

[0660]

[0661]

Claims

1. A dynamic cost-sensitive fault diagnosis system based on AC-DCC model, characterized in that: The following steps are involved: S1. Data collection: Collect production, operation and maintenance data from multiple sources through the APS system, equipment sensors and historical data to build a complete data set covering production plans, execution differences, alarms and historical records, ensuring a comprehensive input basis for equipment fault classification; S2. Data preprocessing: By performing personalized standardization and normalization on multiple types of data, using linear interpolation to fill missing values, removing outliers, and standardizing time data to seconds, we ensure the consistency and high quality of model input; S3. Dataset division: Stratified sampling is used to ensure that the proportion of minority and majority class samples is consistent, TPG is used to generate pseudo labels to expand the effective use of unlabeled samples, and the distribution of minority class samples in each data set is balanced; S4. Pre-trained TPG: The TPG model uses a multi-layer perceptron (MLP) structure and a Softmax activation function to generate pseudo labels. It uses labeled data for pre-training to improve the accuracy of pseudo label generation. It introduces a pre-training mechanism to initialize model parameters with labeled data to provide high-quality pseudo labels for unlabeled data, thereby improving the utilization rate of unlabeled data. S5. Training offline TPG: By introducing the AC-DCC reinforcement learning framework, the Actor-Critic mechanism is used to dynamically adjust the weights of minority samples, and the pseudo-label generation strategy is optimized using the Critic network feedback. Minority samples receive more attention and the classification performance is improved. S6. Reinforcement learning process: By designing a reward function and dynamic cost matrix adjustment based on the minority class recall rate, the pseudo-label generation strategy of TPG is optimized by combining the policy gradient algorithm, and the minority class samples are processed first to improve its fault recognition effect; S7. Fault classification model training (offline training): By combining pseudo labels and real labels to build a complete data set, select a suitable supervised classification model for training, and optimize the model through the validation set and classification indicators to improve the accuracy and efficiency of equipment fault classification; S8. Online fault classification: By inputting the real-time collected production data into the AC-DCC model for fault prediction, the equipment fault status is identified in real time and the fault type and alarm information are output. At the same time, the production plan is automatically adjusted according to the diagnosis results, and the tasks are reallocated to ensure the continuity of the production line.

2. A dynamic cost-sensitive fault diagnosis system based on AC-DCC model according to claim 1, characterized in that: The S1. data collection further includes the following steps: S1.1 Collection of production plan data: The APS system automatically extracts production plan data, including the start and end time of the equipment, output, planned operating time, equipment utilization and maintenance plan, for comparison with the actual execution effect and as a benchmark for equipment diagnosis; S1.2 Collection of actual operation data: The actual execution data of equipment operation, such as start and completion time, actual output, and equipment utilization, are collected through sensors and control systems for real-time monitoring and comparison with planned data to identify deviations or failures in production; S1.3 Collection of historical data: Collect historical maintenance records, fault records and production efficiency data of equipment as a reference for equipment status diagnosis, for comparison with current data, to assist in identifying trends and preventing faults.

3. The dynamic cost-sensitive fault diagnosis system based on the AC-DCC model according to claim 1, characterized in that: The data preprocessing step S2 also includes the following steps: S2.1 Standardization: Convert the data to a normal distribution with a mean of 0 and a standard deviation of 1 so that the system model can converge better. The formula is: Where x is the data point, μ is the mean, and σ is the standard deviation; S2.2 Normalization: Scale the data to a fixed range (ranging from 0 to 1) to ensure that all features are on the same scale and to avoid excessive impact of some features on the model due to dimensional differences. The formula is: Among them, x min and x max are the minimum and maximum values ​​of the data set, respectively; S2.3 Missing value processing: Use forward filling, linear interpolation or other methods to fill missing values ​​in the data set to avoid affecting subsequent model learning; the forward filling is to fill missing values ​​with the previous valid data point; the linear interpolation is to fill missing values ​​with a linear formula, the formula is: Where x is the missing value; xprev and xnext are the known data points before and after the missing value; t is the time when the missing value is located, and tprev and tnext are the times of the known data points before and after respectively. S2.4 Outlier detection and removal: Use statistical methods or rules to detect and remove abnormal data points; S2.5 Handling of planned start time and planned completion time; S2.6 Treatment of expected production; S2.7 Estimated running time processing; S2.8 Equipment utilization planning processing; S2.9 Equipment maintenance plan processing S2.10 Production task allocation and processing.

4. The dynamic cost-sensitive fault diagnosis system based on the AC-DCC model according to claim 1, characterized in that: The S3. data set division further includes the following steps: S3.1 Processing of labeled and unlabeled samples: Unlabeled samples are dynamically added to the training set through pseudo labels generated by TPG, which enhances data diversity and improves model classification capabilities. Labeled samples are used for training and verification, which expands the utilization of unlabeled samples. S3.2 Stratified sampling: Enhance the training data set through pseudo-label generation technology to ensure that the distribution of fault categories in the training set, validation set, and test set is consistent, avoiding data imbalance that affects model performance; S3.3 Processing of unlabeled samples and pseudo-labeled samples: Pseudo-labels are generated for unlabeled samples during training and added to the training set to enhance data diversity; pseudo-labels generated by TPG are dynamically generated through a reinforcement learning framework, which can improve the classification ability of the model; S3.4 Sample category analysis and preliminary weighting: In the preprocessing stage, different weights are set for sample imbalance through a weighting mechanism to ensure that minority samples receive more attention in model training; a higher initial weight is set for minority samples, and a lower initial weight is set for frequent failure data (majority class); the preliminary weighting formula is: Among them, x i is the initial weight of category i, n i is the number of samples of category i.

5. The dynamic cost-sensitive fault diagnosis system based on the AC-DCC model according to claim 1, characterized in that: The S4. pre-training TPG further includes the following steps: S4.1 Use TPG: To give the model a good initial parameter, TPG will be pre-trained using labeled training data; S4.2 Network structure: A multi-layer perceptron (MLP) architecture is used. The input is the process variable feature and the output is the probability distribution of the pseudo label. The output layer uses the Softmax activation function to represent the probability of each pseudo label. S4.3 Pre-training strategy: Use labeled training data to pre-train TPG to ensure that TPG can generate high-quality pseudo labels for unlabeled samples.

6. The dynamic cost-sensitive fault diagnosis system based on the AC-DCC model according to claim 1, characterized in that: The S5. offline TPG training further includes the following steps: S5.1 introduces the AC-DCC model: In TPG training, the Actor-Critic mechanism is used to dynamically adjust sample weights, especially to improve the quality of pseudo-label generation for minority samples, and to ensure classification accuracy under unbalanced data; S5.2 Policy optimization of the AC-DCC model: The Actor network generates weights based on unlabeled samples, the Critic network evaluates the classification performance and provides feedback, and the Actor network uses the policy gradient method to adjust the weight distribution based on this feedback to ensure that minority samples receive more attention; S5.3 Cost-sensitive mechanism of reinforcement learning: By introducing a cost matrix to dynamically adjust the classification cost of minority class samples, the model is ensured to give priority to minority class samples during training, and the sample weights are optimized through the Actor-Critic mechanism to improve the accuracy of pseudo-label generation and the classification performance of minority class samples; S5.4 Model training process: By inputting the process variables and pseudo labels of unlabeled samples into the AC-DCC model, the Actor network dynamically generates weights, the Critic network evaluates based on the classification performance feedback, and optimizes the weight distribution through the policy gradient method.

7. The dynamic cost-sensitive fault diagnosis system based on the AC-DCC model according to claim 1, characterized in that: The S6. reinforcement learning process further includes the following steps: S6.1 Reward function design: By introducing weight parameters of minority class recall and overall classification accuracy, a reward function formula is constructed to ensure that the overall classification accuracy is maintained while improving the minority class classification performance; S6.2 Cost matrix construction: By analyzing the category distribution, a higher error cost is set for the minority class, and the minority class samples are ensured to be evenly distributed in the stratified sampling of the training, validation, and test sets, and the weighted cross entropy is used to optimize the model's classification performance for the minority class. S6.3 Training process: Generate sample weights through batch training, use the validation set to evaluate performance and give feedback to the Actor network to adjust the strategy, and use the Adam optimizer to dynamically adjust the learning rate; S6.4 Policy Optimization Optimize the pseudo-label generation strategy of the TPG model through a policy gradient method (such as REINFORCE), calculate the gradient of the strategy and adjust the model parameters based on reward feedback; S6.5 Iterative training process: TPG generates pseudo labels for unlabeled data classification. After training the classifier, the reward is calculated based on the performance of the validation set, and the pseudo label generation strategy is updated and adjusted through the policy gradient optimization algorithm (such as REINFORCE).

8. The dynamic cost-sensitive fault diagnosis system based on the AC-DCC model according to claim 1, characterized in that: The S7. fault classification model training further includes the following steps: S7.1 Integration of pseudo labels and real labels: Generate labels for unlabeled data through the pseudo label generator and integrate them with the real labels to build a complete training dataset; S7.2 Input data preparation: Input the labeled data set into the classification model, and then preprocess and standardize or normalize it to ensure that the data format is unified and feature engineering is completed; S7.3 Model selection: Select an appropriate supervised classification model based on the complexity of the problem and the characteristics of the data; S7.4 Classification model training: Input the labeled data set into the supervised classification model, improve the classification effect through parameter optimization, and enhance the model's ability to classify faults; S7.5 Model evaluation: Use accuracy, precision, recall, and F1 value indicators on the validation set to evaluate the generalization ability of the model and optimize its performance.

9. The dynamic cost-sensitive fault diagnosis system based on the AC-DCC model according to claim 3, characterized in that: The processing of the planned start time and the planned completion time in S2.5 also includes the following steps: S2.51 Time data standardization: Convert the planned start time and completion time into seconds and then perform standardization. The formula for the converted timestamp standardization is: and Among them, x start and x end is the time in seconds between the scheduled start and finish times, μ end and μ start are the means of all time points in the training data; σ start and σ start is the standard deviation of the time points; S2.52 Normalization of time data: Alternatively, time data can be normalized directly without being standardized, and converted to a value between 0 and 1 to ensure that the time data is on the same scale; the formula is: and Among them, x start-min and x start-max is the minimum and maximum value of the timestamp; x end-min and x end-max is the corresponding timestamp range.

10. The dynamic cost-sensitive fault diagnosis system based on the AC-DCC model according to claim 3, characterized in that: The processing of the estimated output in S2.6 also includes the following steps: S2.61 Standardization: Convert the expected output to a standard normal distribution with a mean of 0 and a standard deviation of 1 so that different features are processed on the same scale; the formula is: Among them, x output is the expected output of a certain equipment; μ output is the average expected output of all equipment in the dataset; σ output is the standard deviation of the estimated output of all devices in the data set; S2.62 Normalization: Scale the estimated production to between 0 and 1 to ensure that data of different dimensions are in the same range; the formula is: Among them, x output-min is the minimum value of expected output; x output-max is the maximum expected output.

11. The dynamic cost-sensitive fault diagnosis system based on the AC-DCC model according to claim 3, characterized in that: The S2.7 estimated running time process further includes the following steps: S2.71 Standardization: Through standardization, the estimated running time is converted into a standard normal distribution with a mean of 0 and a standard deviation of 1, reducing the impact caused by different feature dimensions; the formula is: Among them, x runtime is the expected operating time of a device; μ runtime is the average running time of all devices in the dataset; μ runtime is the standard deviation of the operating time of all devices; S2.72 Normalization: Assuming the estimated running time ranges from 100 to 1000 minutes, scale the estimated running time to a fixed range between 0 and 1 to ensure that the eigenvalues ​​work on a uniform scale; the formula is: Among them, x runtime-min and x runtime-max are the minimum and maximum estimated run times.

12. The dynamic cost-sensitive fault diagnosis system based on the AC-DCC model according to claim 3, characterized in that: The S2.8 equipment utilization rate planning process also includes the following steps: S2.81 Standardization: Equipment utilization is a percentage, ranging from 0 to 100. The planned utilization of the equipment is converted to a standard normal distribution with a mean of 0 and a standard deviation of 1. The formula is: Among them, x utilization is the planned utilization rate of the equipment; μ utilization is the average value of equipment utilization; μ utilization is the standard deviation of equipment utilization; S2.82 Normalization: Normalize the equipment utilization directly to between 0 and 1; the formula is: Among them, x utilization is the planned utilization of the equipment (in percent).

13. The dynamic cost-sensitive fault diagnosis system based on the AC-DCC model according to claim 3, characterized in that: The S2.9 equipment maintenance plan processing also includes the following steps: S2.91 Timestamp conversion to seconds: First, convert the timestamp to seconds or minutes; when processing time data, convert it to relative seconds or minutes numerical time data; S2.92 Standardization: Standardize the time data of the equipment maintenance plan to ensure that the mean is 0 and the standard deviation is 1; the formula is: Among them, x maintenance is the maintenance plan time (converted into seconds); μ maintenance is the mean of all time points in the training data; σ maintenance is the standard deviation of all time points in the training data; S2.93 Normalization: Scale the time data of the equipment maintenance plan to a fixed range between 0 and 1; the formula is: Among them, x maintenance-min and x maintenance-max are the minimum and maximum values ​​in the data set.

14. The dynamic cost-sensitive fault diagnosis system based on the AC-DCC model according to claim 3, characterized in that: The S2.10 production task allocation process further includes the following steps: S2.101 Convert task assignments to numerical values: Convert production task assignments to numerical form (such as task IDs or one-hot encoding) to ensure that classification tasks can be processed as numerical data input models; S2.102 Standardization / normalization processing: Normalize the task ID to a value between 0 and 1 to ensure that it is input into the model at the same scale as other numerical features to maintain the balance of data input; S2.103 Missing sensor data processing: For missing sensor data, use forward filling or linear interpolation to fill in the missing sensor data; The forward filling is to fill in the missing data using the nearest known data point before the missing data; The linear interpolation method estimates and fills in missing values ​​based on known data points before and after the missing data points. The linear interpolation formula is: Among them, x i is the missing value to be filled; x i-1 is the known data point before the missing value; x i+1 is the known data point after the missing value; t i is the time at which the missing value is located; t i-1 and t i+1 are the times corresponding to the known data points; S2.104 Outlier detection and removal: Detect and remove outlier data points by setting reasonable upper and lower limits or using statistical methods (such as Z-score).

15. The dynamic cost-sensitive fault diagnosis system based on AC-DCC model according to claim 5, characterized in that: The S4.3 pre-training strategy also includes the following steps: S4.31 Using the cross entropy loss function (L CE ): Calculate the error between the predicted pseudo label and the true label, and measure the difference between the model output and the true value; the formula is: Among them, yi is the true label, The label probability predicted by the model; S4.32 Initial parameter setting: Use labeled samples to initialize the weights of the TPG model.

16. The dynamic cost-sensitive fault diagnosis system based on the AC-DCC model according to claim 6, characterized in that: The S5.4 model training process also includes the following steps: S5.41 State input: The process variables of the unlabeled samples (such as time difference, equipment utilization) are spliced ​​with pseudo labels into a complete state vector as the input feature of the equipment operation status for training of the AC-DCC model; the vector representation of the equipment state is: S i =[time diffutilization rate,output deviation,...] S5.42 Action generation: The Actor network of AC-DCC generates weights according to the sample status, dynamically adjusts the sample priority when generating pseudo labels, especially pays more attention to minority class samples, and outputs the final classification results; pseudo labels are classified, and the given sample classification is: A i =Predict\_Label(S i ) Among them, if the output is a probability distribution [P1, P2, P3], the final action is to select the category corresponding to the maximum probability; S5.43 Classification and Feedback: The Critic network evaluates the quality of pseudo-label generation (recall rate and F1 score of minority classes) and provides feedback on classification performance to the Actor network to help optimize the weight allocation strategy. S5.44 Strategy Update: Optimize the Actor network through the policy gradient method, dynamically adjust the sample weights, ensure that minority class samples receive more attention in subsequent training, and optimize the quality of pseudo-label generation.

17. A dynamic cost-sensitive fault diagnosis system based on AC-DCC model according to claim 7, characterized in that: The reward function design in S6.1 also includes the following steps: S6.11 Reward function based on minority class recall rate: The minority class recall rate is introduced as the reward target, and the learning focus of the model is adjusted through the weight parameter α; S6.12 Combined with overall classification accuracy: The overall classification accuracy is introduced and balanced by the weight parameter β; S6.13 Construct the reward function formula: Construct the reward function formula based on the weights of the minority class recall rate and the overall classification accuracy rate; each time the model correctly classifies a minority class fault, a higher reward is given; if the minority class is misclassified, a penalty is given; the reward function formula is: R=α×Recall minority +β×Accuracy overall Among them, α and β are weight parameters that control different objectives, ensuring that the model maintains the overall accuracy while improving the recall rate of the minority class.

18. The dynamic cost-sensitive fault diagnosis system based on the AC-DCC model according to claim 7, characterized in that: The S6.4 strategy optimization also includes the following steps: S6.41 The goal of policy gradient optimization is to find the optimal strategy through reinforcement learning to maximize the long-term accumulated rewards, thereby generating higher quality pseudo labels to better complete the classification task; S6.42 Policy Gradient Optimization Algorithm: Use a policy gradient optimization algorithm (such as REINFORCE) to calculate the policy gradient and update the model's pseudo-label generation strategy and optimize model parameters based on reward feedback; the policy gradient optimization formula is: Among them, π θ (a|s) is the probability of choosing action a in state s, and R(s,a) is the corresponding reward.

19. The dynamic cost-sensitive fault diagnosis system based on the AC-DCC model according to claim 3, characterized in that: The production plan data field involves various production-related time and plan parameters, such as planned start time, planned completion time, expected output, and equipment utilization plan related information; the specific fields of the production plan data are: The planned start time is the scheduled start time of the device; The planned completion time is the scheduled end time of the equipment; The estimated output is the output that the equipment is expected to complete in each production cycle; The estimated operating time is the planned operating time of each production cycle equipment; The equipment utilization rate plan is an estimated utilization rate based on the production plan and equipment capacity; The equipment maintenance plan is a schedule for regular maintenance and overhaul of the equipment; The production task allocation includes the production tasks, process arrangements and planned progress corresponding to each device.

20. The dynamic cost-sensitive fault diagnosis system based on the AC-DCC model according to claim 3, characterized in that: The production plan data segment involves various actual values ​​in the actual execution process of the equipment; the specific fields of the actual operation data are: The actual startup time is the time when the device is actually started; The actual completion time is the time when the equipment is actually completed; The actual output is the output actually completed by the equipment in each production cycle; The actual operating time is the actual operating time of the equipment in each production cycle; The equipment utilization rate is based on the ratio of actual operating time to expected time, reflecting the utilization rate of the equipment; The downtime is the downtime of the equipment due to failure, maintenance or other reasons; The alarm data is the alarm information during equipment operation, including the alarm type (such as over-temperature, over-pressure, over-load), alarm frequency and time; The abnormal event record is a record of abnormal conditions that occur during the operation of the equipment, such as an operating state that exceeds the normal working range; The operation log is the equipment operation information recorded by the operator, including manual intervention and fault repair data.

21. The dynamic cost-sensitive fault diagnosis system based on the AC-DCC model according to claim 3, characterized in that: The historical data fields include historical maintenance time, content, failure time and repair details, and related information about the equipment's past production efficiency and execution deviation. The specific fields of the historical data are: The historical maintenance and repair records include the time, content and frequency of past maintenance and repairs; The historical fault records are the time, type and repair process of the faults that occurred in the past. The historical production efficiency refers to the deviation between the efficiency and output of the equipment in previous production cycles and the actual execution.

Images