Fault Diagnosis System and Method Based on Deep Reinforcement Learning

Abstract

This invention discloses a fault diagnosis system and method based on deep reinforcement learning, relating to the field of fault diagnosis technology. The system performs reverse compensation on historical sensor output data to obtain first standard output data; kernel density estimation is performed on the first standard output data of each sensor; the fault probability of the industrial robot under different environmental factors is analyzed to obtain second standard output data; real-time sensor output data of the industrial robot is acquired, and fault types of the industrial robot are diagnosed using a deep reinforcement learning model of fault types; by analyzing the influence of different environmental data on sensor output data, reverse compensation is performed on the sensor output data to reduce the impact of environmental data differences on the diagnostic model; based on the probability density function of industrial robot faults, the fault probability of the industrial robot is made consistent under different environmental data, achieving continuous fault diagnosis in different environments and improving the efficiency of fault diagnosis.

Description

Technical Field

[0001] This invention relates to the field of fault diagnosis technology, specifically a fault diagnosis system and method based on deep reinforcement learning. Background Technology

[0002] Industrial robot fault diagnosis is a complex and challenging field. The industrial environment is dynamic and the robot's working environment may be constantly adjusted. Factors such as vibration, humidity, and temperature changes in the dynamic environment will cause changes in the physical quantities detected by the sensors, resulting in changes in the distribution of their output data. Fault diagnosis needs to be carried out according to the characteristics of the environment, which makes continuous fault diagnosis of industrial robots difficult. Therefore, how to perform continuous fault diagnosis of industrial robots in dynamic environments has become an urgent problem to be solved. Summary of the Invention

[0003] The purpose of this invention is to provide a fault diagnosis system and method based on deep reinforcement learning to solve the problems raised in the prior art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a fault diagnosis system based on deep reinforcement learning, comprising a data acquisition module, a data storage module, a data analysis module, and a fault diagnosis module; the data acquisition module acquires operational data and environmental data of the industrial robot through sensors; the data storage module stores historical operational data, environmental data, and fault data of the industrial robot; the data analysis module performs reverse compensation on the historical output data of the sensors to remove deviations caused by environmental factors, obtaining first standard output data; analyzes the fault probability of the industrial robot under different environmental factors, adjusts the reverse compensation method to remove deviations caused by environmental factors, obtaining second standard output data; the fault diagnosis module diagnoses the fault type of the industrial robot using a deep reinforcement learning model of fault types. The fault diagnosis module further includes a recovery unit and a deep reinforcement learning unit; the deep reinforcement learning unit trains a deep reinforcement learning model of fault types based on the second standard output data of the sensors, and performs fault diagnosis on the industrial robot based on the deep reinforcement learning model of fault types; the recovery unit executes recovery operations on the industrial robot according to the fault type of the industrial robot. The data analysis module further includes a fitting unit, an estimation unit, and an optimization unit. The fitting unit trains a fitting model between the difference between environmental data and sensor output data, and then performs reverse compensation on the historical sensor output data based on the fitting model to remove biases caused by environmental factors, obtaining first standard output data. The estimation unit performs kernel density estimation on the first standard output data of each sensor to obtain the probability density function between the industrial robot's failure probability and the sensor's first standard output data. The optimization unit analyzes the failure probability of the industrial robot under different environmental factors, adjusts the reverse compensation method, removes biases caused by environmental factors, and obtains second standard output data.

[0005] The optimization unit acquires historical environmental data and sensor output data of the industrial robot under environmental data other than the reference environmental data, and obtains first standard output data S based on the sensor output data; let Sbase represent the sensor output data of the industrial robot under the reference environmental data, and generate an interval of length 2e with m as the center, where e represents noise, and m∈[Se,S+e]; and obtains the fault probability P1 under the reference environmental data and the fault probability P2 under the other environmental data based on the probability density functions under the reference environmental data and the other environmental data other than the reference environmental data. Based on P1, determine the value of m in the interval [me, m+e] that minimizes the deviation between P1 and P2. Based on m, we get P2, P2 = In the formula, f1(x) represents the probability density function under the reference environmental data, and f2(x) represents the probability density function under other environmental data besides the reference environmental data; based on the determined m that minimizes the deviation between P1 and P2, the second standard output data is obtained.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] The fault diagnosis method based on deep reinforcement learning includes the following steps:

[0008] Acquire historical output data from sensors on industrial robots;

[0009] The historical output data of the sensor is reverse-compensated to remove the deviation caused by environmental factors and obtain the first standard output data.

[0010] Kernel density estimation is performed on the first standard output data of each sensor to obtain the probability density function between the failure probability of the industrial robot and the first standard output data of the sensor.

[0011] Analyze the failure probability of industrial robots under different environmental factors, adjust the reverse compensation method, remove the deviation caused by environmental factors, and obtain the second standard output data.

[0012] Based on the second standard output data of all sensors, a deep reinforcement learning model for fault types is trained; real-time sensor output data of the industrial robot is acquired, and the fault types of the industrial robot are diagnosed through the deep reinforcement learning model for fault types.

[0013] Specifically, the process of reverse-compensating the historical output data of the sensor to remove deviations caused by environmental factors and obtain the first standard output data also includes the following steps:

[0014] Acquire all environmental data during the operation of the industrial robot, select one environmental data as the reference environmental data and denote it as Eb; acquire the output data Outb and Out of the sensors on the industrial robot under environmental data Eb and other environmental data, calculate the difference between Out and Outb, and train a fitting model between the difference and the environmental data;

[0015] The output data of sensors on the industrial robot under environmental data Eb and other environmental data are divided into training and testing sets. Other environmental data are used as input, and the difference Out-Outb in the training set is used as output to train a neural network fitting model. Other environmental data in the validation set are input into the neural network fitting model to obtain the fitted value of the difference. The effect of the neural network fitting model is verified based on the fitted value of the difference and the actual value. Other environmental data are input into the validated neural network fitting model to obtain the fitted value of the difference. The fitted value of the difference is calculated with the output data under other environmental data to obtain the first standard output data.

[0016] Specifically, performing kernel density estimation on the first standard output data of each sensor to obtain the probability density function between the industrial robot's failure probability and the first standard output data of the sensor also includes the following steps:

[0017] The historical output data of the sensors of the industrial robot under all environmental data is obtained, the data with faults is marked, and the deviation caused by environmental factors is removed from the historical output data of the sensors to obtain the first standard output data.

[0018] Generate a dataset based on historical smelting data: Set nodes x1, x2, ..., xn for sensor output data, where n is the number of nodes; Let X represent the random variable of the first standard output data, and obtain the probability P{X≥a1} of the labeled data occurring when the random variable X of the first standard output data is greater than or equal to a1, where P{X≥a1}=n. a1 / N a1 In the formula n a1 N represents the number of first standard output data that are greater than or equal to a1 and have faults. a1 This represents the number of first standard output data points greater than or equal to a1; following the same method as the sensor output data node x1, obtain the probabilities P{X≥a2}, ..., P{X≥an} of the occurrence of labeled data when the first standard output data X is greater than or equal to a2, ..., an; n data points are formed by a1 and P{X≥a1}, a2 and P{X≥a2}, ..., an and P{X≥an}, and these data points are denoted as x1, x2, ..., xn to obtain the dataset;

[0019] The probability density function f(x) between the probability of fault occurrence and the first standard output data is obtained through kernel density estimation. A kernel function with non-negativity and symmetry and an integral of 1 over the real number domain R is set. A constant greater than zero is selected as the bandwidth h of the kernel function K. The kernel function is scaled according to the bandwidth h to obtain Kh, where Kh(u) = 1 / h × K(u / h), and u is the input of the kernel function. The contribution rate Kh(x-xi) of the data point xi in the dataset to the estimated point x is obtained. The contribution rates of all data points in the dataset to the estimated point x are added together to obtain the kernel density estimate f(x) at the estimated point x. f(x) = 1 / n∑Kh(x-xi). By changing the position of the estimated point x, the kernel density estimate of the probability of fault occurrence over the entire dataset is obtained. The dataset is the interval between the minimum and maximum values ​​of the sensor output data. The effect of the kernel function is verified, and the bandwidth h with the best verification effect is selected through cross-validation.

[0020] Specifically, analyzing the failure probability of industrial robots under different environmental factors, adjusting the reverse compensation method, removing the deviation caused by environmental factors, and obtaining the second standard output data also includes the following steps:

[0021] Acquire historical environmental data and sensor output data of the industrial robot under environmental data other than the reference environmental data. Obtain the first standard output data S based on the sensor output data. Let Sbase represent the sensor output data of the industrial robot under the reference environmental data. Generate an interval of length 2e centered at m, where e represents noise, and m∈[Se,S+e]. Obtain the fault probability P1 under the reference environmental data and the fault probability P2 under the other environmental data based on the probability density functions under the reference environmental data and the other environmental data. Based on P1, determine the value of m in the interval [me, m+e] that minimizes the deviation between P1 and P2. Based on m, we get P2, P2 = In the formula, f1(x) represents the probability density function under the reference environmental data, and f2(x) represents the probability density function under other environmental data besides the reference environmental data. According to the determined m that minimizes the deviation between P1 and P2, the second standard output data is obtained. Let y1 and y2 represent the first standard output data and the second standard output data, then y2=y1+(mS).

[0022] Specifically, training a deep reinforcement learning model for fault types based on the second standard output data from all sensors also includes the following steps:

[0023] The fault types of industrial robots are binary encoded, and a DQN deep neural network is constructed. The second standard output data of the sensor forms the state space as input, and the fault types of the industrial robot form the action space. The reward function and hyperparameters of the DQN neural network are set, and the DQN neural network model is trained.

[0024] Compared with the prior art, the beneficial effects of the present invention are: by analyzing the influence of different environmental data on sensor output data, reverse compensation is performed on the sensor output data to reduce the impact of environmental data differences on the diagnostic model; based on the probability density function of industrial robot faults, the fault probability of industrial robot is made consistent under different environmental data, realizing continuous fault diagnosis in different environments and improving the efficiency of fault diagnosis. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the fault diagnosis system based on deep reinforcement learning according to the present invention.

[0026] Figure 2 This is a flowchart of the fault diagnosis method based on deep reinforcement learning of the present invention;

[0027] Figure 3 This is a flowchart of the fault diagnosis network update of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] Example: Figure 1As shown, this invention provides a fault diagnosis system based on deep reinforcement learning, including a data acquisition module, a data storage module, a data analysis module, and a fault diagnosis module. The data acquisition module acquires operational data and environmental data of the industrial robot through sensors. The data storage module stores historical operational data, environmental data, and fault data of the industrial robot. The data analysis module performs reverse compensation on the historical output data of the sensors to remove deviations caused by environmental factors, obtaining first standard output data; it analyzes the fault probability of the industrial robot under different environmental factors, adjusts the reverse compensation method, and removes deviations caused by environmental factors to obtain second standard output data. The fault diagnosis module diagnoses the fault type of the industrial robot using a deep reinforcement learning model of the fault type. The fault diagnosis module also includes a recovery unit and a deep reinforcement learning unit. The deep reinforcement learning unit trains the deep reinforcement learning model of the fault type based on the second standard output data of the sensors and performs fault diagnosis on the industrial robot based on the deep reinforcement learning model of the fault type. The recovery unit executes recovery operations on the industrial robot according to the fault type. The data analysis module further includes a fitting unit, an estimation unit, and an optimization unit. The fitting unit trains a fitting model between the difference between environmental data and sensor output data, and then performs reverse compensation on the historical sensor output data based on the fitting model to remove biases caused by environmental factors, obtaining first standard output data. The estimation unit performs kernel density estimation on the first standard output data of each sensor to obtain the probability density function between the industrial robot's failure probability and the sensor's first standard output data. The optimization unit analyzes the failure probability of the industrial robot under different environmental factors, adjusts the reverse compensation method, removes biases caused by environmental factors, and obtains second standard output data.

[0030] The optimization unit acquires historical environmental data and sensor output data of the industrial robot under environmental data other than the reference environmental data, and obtains first standard output data S based on the sensor output data; let Sbase represent the sensor output data of the industrial robot under the reference environmental data, and generate an interval of length 2e with m as the center, where e represents noise, and m∈[Se,S+e]; and obtains the fault probability P1 under the reference environmental data and the fault probability P2 under the other environmental data based on the probability density functions under the reference environmental data and the other environmental data other than the reference environmental data. Based on P1, determine the value of m in the interval [me, m+e] that minimizes the deviation between P1 and P2. Based on m, we get P2, P2 = In the formula, f1(x) represents the probability density function under the reference environmental data, and f2(x) represents the probability density function under other environmental data besides the reference environmental data; based on the determined m that minimizes the deviation between P1 and P2, the second standard output data is obtained.

[0031] In another embodiment of the present invention, the present invention provides a fault diagnosis method based on deep reinforcement learning, comprising the following steps:

[0032] Acquire historical output data from sensors on industrial robots;

[0033] The historical output data of the sensor is reverse-compensated to remove the deviation caused by environmental factors and obtain the first standard output data.

[0034] Kernel density estimation is performed on the first standard output data of each sensor to obtain the probability density function between the failure probability of the industrial robot and the first standard output data of the sensor.

[0035] Analyze the failure probability of industrial robots under different environmental factors, adjust the reverse compensation method, remove the deviation caused by environmental factors, and obtain the second standard output data.

[0036] Based on the second standard output data of all sensors, a deep reinforcement learning model for fault types is trained; real-time sensor output data of the industrial robot is acquired, and the fault types of the industrial robot are diagnosed through the deep reinforcement learning model for fault types.

[0037] To reverse-compensate the historical output data of the sensor, remove deviations caused by environmental factors, and obtain the first standard output data, the following steps are also included:

[0038] Acquire all environmental data during the operation of the industrial robot, select one environmental data as the reference environmental data and denote it as Eb; acquire the output data Outb and Out of the sensors on the industrial robot under environmental data Eb and other environmental data, calculate the difference between Out and Outb, and train a fitting model between the difference and the environmental data;

[0039] The output data of sensors on the industrial robot under environmental data Eb and other environmental data are divided into training and testing sets. Other environmental data are used as input, and the difference Out-Outb in the training set is used as output to train a neural network fitting model. Other environmental data in the validation set are input into the neural network fitting model to obtain the fitted value of the difference. The effect of the neural network fitting model is verified based on the fitted value of the difference and the actual value. Other environmental data are input into the validated neural network fitting model to obtain the fitted value of the difference. The fitted value of the difference is calculated with the output data under other environmental data to obtain the first standard output data.

[0040] Sensors on industrial robots operate in dynamic environments and are susceptible to various environmental factors, leading to fluctuations or errors in their output signals. Different environments result in different outputs, complicating continuous fault diagnosis of industrial robots. Therefore, reverse compensation is used to obtain the first standard output data of the sensors. A fitting model is established using a neural network model. First, an environmental data point is selected as a reference. When the industrial robot is in the reference environment, the sensor output data is the standard output data. When the industrial robot leaves the reference environment and operates in other environments, the sensor output data will be affected by environmental factors, deviating from the standard output data. This model captures the errors caused by environmental factors in the sensor output data. Since the reference environment data is fixed, the input to the fitting model can be directly selected from other environmental data. For each type of sensor, a neural network fitting model is trained separately to obtain the first standard output data.

[0041] The process of performing kernel density estimation on the first standard output data of each sensor to obtain the probability density function between the industrial robot's failure probability and the first standard output data of the sensors also includes the following steps:

[0042] The historical output data of the sensors of the industrial robot under all environmental data is obtained, the data with faults is marked, and the deviation caused by environmental factors is removed from the historical output data of the sensors to obtain the first standard output data.

[0043] Generate a dataset based on historical smelting data: Set nodes x1, x2, ..., xn for sensor output data, where n is the number of nodes; Let X represent the random variable of the first standard output data, and obtain the probability P{X≥a1} of the labeled data occurring when the random variable X of the first standard output data is greater than or equal to a1, where P{X≥a1}=n. a1 / N a1 In the formula n a1 N represents the number of first standard output data that are greater than or equal to a1 and have faults. a1 This represents the number of first standard output data points greater than or equal to a1; following the same method as the sensor output data node x1, obtain the probabilities P{X≥a2}, ..., P{X≥an} of the occurrence of labeled data when the first standard output data X is greater than or equal to a2, ..., an; n data points are formed by a1 and P{X≥a1}, a2 and P{X≥a2}, ..., an and P{X≥an}, and these data points are denoted as x1, x2, ..., xn to obtain the dataset;

[0044] The probability density function f(x) between the probability of fault occurrence and the first standard output data is obtained through kernel density estimation. A kernel function with non-negativity and symmetry and an integral of 1 over the real number domain R is set. A constant greater than zero is selected as the bandwidth h of the kernel function K. The kernel function is scaled according to the bandwidth h to obtain Kh, where Kh(u) = 1 / h × K(u / h), and u is the input of the kernel function. The contribution rate Kh(x-xi) of the data point xi in the dataset to the estimated point x is obtained. The contribution rates of all data points in the dataset to the estimated point x are added together to obtain the kernel density estimate f(x) at the estimated point x. f(x) = 1 / n∑Kh(x-xi). By changing the position of the estimated point x, the kernel density estimate of the probability of fault occurrence over the entire dataset is obtained. The dataset is the interval between the minimum and maximum values ​​of the sensor output data. The effect of the kernel function is verified, and the bandwidth h with the best verification effect is selected through cross-validation.

[0045] The probability density function between sensor output data and industrial robot failure probability is obtained through kernel density estimation. The probability density function is obtained separately for each type of environmental data. Since environmental factors may not be entirely the same, similar environmental data can be grouped into one class using supervised or unsupervised classification methods, and the probability density function is obtained for environmental data of the same class. When using supervised classification, environmental factors are manually labeled, assigning the same label to environmental factors that have a similar impact on sensor output data, and then training the classification model. Based on the probability density function for each type of environmental data, the failure probability of the industrial robot under each type of environmental data can be obtained.

[0046] Analyzing the failure probability of industrial robots under different environmental factors, adjusting the reverse compensation method, removing the bias caused by environmental factors, and obtaining the second standard output data also includes the following steps:

[0047] Acquire historical environmental data and sensor output data of the industrial robot under environmental data other than the reference environmental data. Obtain the first standard output data S based on the sensor output data. Let Sbase represent the sensor output data of the industrial robot under the reference environmental data. Generate an interval of length 2e centered at m, where e represents noise, and m∈[Se,S+e]. Obtain the fault probability P1 under the reference environmental data and the fault probability P2 under the other environmental data based on the probability density functions under the reference environmental data and the other environmental data. Based on P1, determine the value of m in the interval [me, m+e] that minimizes the deviation between P1 and P2. Based on m, we get P2, P2 = In the formula, f1(x) represents the probability density function under the reference environmental data, and f2(x) represents the probability density function under other environmental data besides the reference environmental data. According to the determined m that minimizes the deviation between P1 and P2, the second standard output data is obtained. Let y1 and y2 represent the first standard output data and the second standard output data, then y2=y1+(mS).

[0048] For the same robot, under different environmental data conditions, the failure probability should be similar or the same if no sudden abnormality occurs. It is unreasonable for environmental changes to cause significant changes in sensor output data, leading to a substantial shift in the assessed failure probability. The second standard output data is further adjusted based on the first standard output data. Due to noise, there is an error of ±e when converting the sensor output data to the first standard output data, resulting in accurate standard output data falling within the interval [Se, S+e]. To obtain accurate standard output data, an arbitrary point m within the interval [Se, S+e] is used as the center, and the failure probability P2 is calculated based on this center under environmental data other than the reference environmental data. The requirement is met when this failure probability is closest to the failure probability under the reference environmental data. At this point, m is the second standard output data; the error is obtained based on m and the first standard output data S. The error mS can be updated periodically without analyzing every first standard output data of the sensor; after obtaining the error mS, it is added to the first standard output data of the sensor to obtain the second standard output data.

[0049] Based on the obtained second standard output data, when the environment of the industrial robot changes, the failure probability of the industrial robot will not change abruptly without the sudden occurrence of abnormal events, and the fault diagnosis of the industrial robot can be performed continuously.

[0050] Training a deep reinforcement learning model for fault types based on the second standard output data from all sensors also includes the following steps:

[0051] The fault types of industrial robots are binary encoded, and a DQN deep neural network is constructed. The second standard output data of the sensor forms the state space as input, and the fault types of the industrial robot form the action space. The reward function and hyperparameters of the DQN neural network are set, and the DQN neural network model is trained.

[0052] A fault type is defined as an action. For each possible fault type, a binary bit is used to indicate whether it exists. For example, if there are three fault types, the action space is represented by three binary bits: 000 means no fault, 001 means only fault type 3, 010 means only fault type 2, 011 means both fault type 2 and fault type 3 exist, and so on. Since the second standard output data of the sensor changes over time, the state space also changes over time.

[0053] Specifically, the following steps are included:

[0054] DQN Network Construction: Construct a DQN deep neural network. The input is the feature vector of the state space, and the output is the Q value corresponding to each action. Each action represents a possible fault type. Initialize the weights and biases of the DQN deep neural network using random initialization. Set the values ​​of hyperparameters, including but not limited to the learning rate, discount factor γ, size of the experience replay buffer, and size of mini-batch samples.

[0055] Create an empty experience replay buffer to store samples generated during the agent's interaction with the environment, including information such as state, action, reward, and next state;

[0056] At each time step, based on the current state, an action is selected using an ε-greedy strategy. This selected action is then executed in the environment, and the reward signal and the next state returned by the environment are obtained. This sample is then stored in the experience replay buffer. When selecting an action, the Q-value is calculated by the DQN deep neural network based on the current state. The calculated Q-value is the network output Q(s,a,θ), where s represents the state (i.e., the second standard output data of the sensor), a represents the action (i.e., the fault type of the industrial robot), and θ represents the parameters of the DQN deep neural network. The reward function is positively correlated with correctly diagnosed fault types and negatively correlated with incorrectly diagnosed fault types, and is connected by weighted summation.

[0057] At regular training intervals, a small batch of samples is randomly drawn from the experience replay buffer. For each sample, the maximum Q-value Qmax corresponding to the next state is calculated using the target network. The target Q-value Qtarget is calculated based on the reward r, the maximum Q-value Qmax, and the discount factor γ: Qtarget = r + γ × Qmax. The Q-value Qpre predicted by the online network is compared with the target Q-value, and the loss L is calculated. In the formula, N represents the size of the mini-batch sample. Let L2 represent the L2 norm of Qpre-Qtarget; the gradient information of the loss function is propagated back to each layer of the online network through the backpropagation algorithm, and the parameters of the online network are updated using the Adam optimization algorithm to minimize the difference between the predicted Q value and the target Q value; after the online network has been updated a certain number of times, the parameters of the online network are copied to the target network.

[0058] Since the second standard output data of the sensor changes over time, the DQN deep neural network can output a maximum Q value and the corresponding action at each time point, thus obtaining a continuous action sequence in time. This action sequence is also the fault type diagnosis sequence. Based on the obtained fault type diagnosis sequence, the fault type of the industrial robot is diagnosed.

[0059] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A fault diagnosis method based on deep reinforcement learning, characterized in that, Includes the following steps: Acquire historical output data from sensors on industrial robots; The historical output data of the sensor is reverse-compensated to remove the deviation caused by environmental factors and obtain the first standard output data. Kernel density estimation is performed on the first standard output data of each sensor to obtain the probability density function between the failure probability of the industrial robot and the first standard output data of the sensor. Analyze the failure probability of industrial robots under different environmental factors, adjust the reverse compensation method, remove the deviation caused by environmental factors, and obtain the second standard output data. Based on the second standard output data of all sensors, a deep reinforcement learning model for fault types is trained; real-time sensor output data of the industrial robot is acquired, and the fault types of the industrial robot are diagnosed through the deep reinforcement learning model for fault types. The step of performing reverse compensation on the historical output data of the sensor to remove the deviation caused by environmental factors and obtain the first standard output data also includes the following steps: Acquire all environmental data during the operation of the industrial robot, select one environmental data as the reference environmental data and denote it as Eb; acquire the output data Outb and Out of the sensors on the industrial robot under environmental data Eb and other environmental data, calculate the difference between Out and Outb, and train a fitting model between the difference and the environmental data; The output data of sensors on the industrial robot under environmental data Eb and other environmental data are divided into training set and test set. Other environmental data are used as input, and the difference Out-Outb in the training set is used as output to train a neural network fitting model. Other environmental data in the validation set are input into the neural network fitting model to obtain the fitted value of the difference. The effect of the neural network fitting model is verified based on the fitted value of the difference and the actual value. Other environmental data are input into the validated neural network fitting model to obtain the fitted value of the difference. The fitted value of the difference is calculated with the output data under other environmental data to obtain the first standard output data. The step of performing kernel density estimation on the first standard output data of each sensor to obtain the probability density function between the industrial robot failure probability and the first standard output data of the sensor further includes the following steps: The historical output data of the sensors of the industrial robot under all environmental data is obtained, the data with faults is marked, and the deviation caused by environmental factors is removed from the historical output data of the sensors to obtain the first standard output data. Generate a dataset based on historical smelting data: Set nodes x1, x2, ..., xn for sensor output data, where n is the number of nodes; Let X represent the random variable of the first standard output data, and obtain the probability P{X≥a1} of the labeled data occurring when the random variable X of the first standard output data is greater than or equal to a1, where P{X≥a1}=n. a1 / N a1 In the formula n a1 N represents the number of first standard output data that are greater than or equal to a1 and have faults. a1 This represents the number of first standard output data points greater than or equal to a1; following the same method as the sensor output data node x1, obtain the probabilities P{X≥a2}, ..., P{X≥an} of the occurrence of labeled data when the first standard output data X is greater than or equal to a2, ..., an; n data points are formed by a1 and P{X≥a1}, a2 and P{X≥a2}, ..., an and P{X≥an}, and these data points are denoted as x1, x2, ..., xn to obtain the dataset; The probability density function f(x) between the probability of fault occurrence and the first standard output data is obtained through kernel density estimation. A kernel function with non-negativity and symmetry and an integral of 1 over the real number domain R is set. A constant greater than zero is selected as the bandwidth h of the kernel function K. The kernel function is scaled according to the bandwidth h to obtain Kh, where Kh(u) = 1 / h × K(u / h), and u is the input of the kernel function. The contribution rate Kh(x-xi) of the data point xi in the dataset to the estimated point x is obtained. The contribution rates of all data points in the dataset to the estimated point x are added together to obtain the kernel density estimate f(x) at the estimated point x. f(x) = 1 / n∑Kh(x-xi). By changing the position of the estimated point x, the kernel density estimate of the probability of fault occurrence over the entire dataset is obtained. The dataset is the interval between the minimum and maximum values ​​of the sensor output data. The effect of the kernel function is verified, and the bandwidth h with the best verification effect is selected through cross-validation.

2. The fault diagnosis method based on deep reinforcement learning according to claim 1, characterized in that, The process of analyzing the failure probability of industrial robots under different environmental factors, adjusting the reverse compensation method, removing the bias caused by environmental factors, and obtaining the second standard output data also includes the following steps: Acquire historical environmental data and sensor output data of the industrial robot under environmental data other than the reference environmental data. Obtain the first standard output data S based on the sensor output data. Let Sbase represent the sensor output data of the industrial robot under the reference environmental data. Generate an interval of length 2e centered at m, where e represents noise, and m∈[Se,S+e]. Obtain the fault probability P1 under the reference environmental data and the fault probability P2 under the other environmental data based on the probability density functions under the reference environmental data and the other environmental data. Based on P1, determine the value of m in the interval [me, m+e] that minimizes the deviation between P1 and P2. Based on m, we get P2, P2 = In the formula, f1(x) represents the probability density function under the reference environmental data, and f2(x) represents the probability density function under other environmental data besides the reference environmental data. According to the determined m that minimizes the deviation between P1 and P2, the second standard output data is obtained. Let y1 and y2 represent the first standard output data and the second standard output data, then y2=y1+(mS).

3. The fault diagnosis method based on deep reinforcement learning according to claim 2, characterized in that, The process of training a deep reinforcement learning model for fault types based on the second standard output data from all sensors also includes the following steps: The fault types of industrial robots are binary encoded, and a DQN deep neural network is constructed. The second standard output data of the sensor forms the state space as input, and the fault types of the industrial robot form the action space. The reward function and hyperparameters of the DQN neural network are set, and the DQN neural network model is trained.

Images