Industrial robot health state monitoring method and monitoring system based on digital twinning

Abstract

The invention discloses an industrial robot health state monitoring method and system based on digital twinning, and the method comprises the steps: building the digital twinning of an industrial robot, collecting multi-dimensional health state parameters, carrying out the feature extraction through a deep Q network, carrying out the modeling of a feature vector time sequence through an improved hidden Markov model, and calculating the posterior probability of the health state. The health state is evaluated in a grading mode by combining a preset rule, a maintenance strategy is generated based on reinforcement learning when early warning or faults occur, and the digital twinborn state is fed back and updated. The monitoring system comprises a data acquisition and preprocessing unit, a deep Q network feature extraction unit and other seven units, and all the units work cooperatively. According to the method, models such as adaptive learning rate adjustment and environmental influence factors are introduced, the problems that a traditional method is weak in data processing capacity and poor in environmental adaptability are solved, and dynamic accurate monitoring and intelligent maintenance guidance of the health state of the industrial robot are achieved.

Description

Technical Field

[0001] The present invention relates to the field of industrial robot health status monitoring, and particularly to a method and a monitoring system for industrial robot health status monitoring based on digital twin. Background Art

[0002] Under the background of the rapid development of intelligent manufacturing, industrial robots, as the core equipment for automated production, their health status directly affects production efficiency and product quality. With the transformation of the manufacturing industry towards intelligence and flexibility, the application scenarios of industrial robots have become increasingly complex, and multi-dimensional health status parameters such as joint torque and motor current generated during operation have increased explosively. Traditional monitoring methods have been difficult to meet the requirements of real-time and accurate monitoring. At the same time, emerging technologies such as digital twin and deep learning have been continuously mature, providing new ideas and methods for industrial robot health status monitoring. There is an urgent need for a more advanced monitoring solution to address the new challenges in the operation and maintenance process of industrial robots.

[0003] Existing industrial robot health status monitoring technologies have significant defects. First, the data processing ability is insufficient. Traditional methods mostly use a single model to analyze health status parameters, and cannot fully explore the potential connections and dynamic change rules among multi-dimensional data. For example, simple threshold judgment is difficult to adapt to the fluctuations of parameters under complex working conditions, and is prone to misjudgment; while ordinary machine learning models are difficult to accurately capture the transfer characteristics of health status when dealing with massive time-series data, resulting in lagged monitoring results. Second, the environmental adaptability and decision-making ability are weak. Most monitoring systems do not consider environmental factors. When environmental conditions such as temperature and humidity change, the accuracy of monitoring results drops significantly. Moreover, after a fault warning, there is a lack of an effective maintenance strategy generation mechanism, and it is impossible to provide targeted maintenance guidance based on the actual health status of the robot and environmental information, resulting in increased equipment repair costs and extended downtime. Summary of the Invention

[0004] In order to overcome the disadvantages and deficiencies of the existing technology, the present invention provides a method and a monitoring system for industrial robot health status monitoring based on digital twin.

[0005] The technical solution adopted by the present invention is that the method for industrial robot health status monitoring based on digital twin includes the following steps:

[0006] Step S1: Using the multi-dimensional health status parameters generated during the operation of the industrial robot, construct a digital twin of the industrial robot, and the multi-dimensional health status parameters include joint torque, motor current, vibration amplitude, and temperature data;

[0007] Step S2: Input the health status parameter data collected in real time during the operation of the industrial robot into the deep Q-network at preset time intervals. The deep Q-network adopts a multi-layer convolutional neural network structure. Through feature extraction and abstraction of the input data, a feature vector containing health status features is formed;

[0008] Step S3: Use the improved hidden Markov model to perform temporal modeling on the feature vector output by the deep Q-network. By combining the dynamic transfer relationship of the industrial robot's health status parameters in the time series, obtain the state transition probability matrix and observation probability matrix of the industrial robot's health status;

[0009] Step S4: Based on the state transition probability matrix and observation probability matrix calculated by the improved hidden Markov model, combined with the observation data at the current moment, use the forward-backward algorithm to calculate the posterior probability of the industrial robot in different health states;

[0010] Step S5: Input the posterior probability into the decision-making module. Based on the preset health status evaluation rules, the decision-making module conducts a hierarchical evaluation of the health status of the industrial robot, and divides different state levels such as normal, warning, and fault;

[0011] Step S6: When the health status of the industrial robot is in the warning or fault level, based on the reinforcement learning mechanism of the deep Q-network, generate corresponding maintenance strategies according to the current health status and environmental information of the industrial robot;

[0012] Step S7: Feed back the industrial robot health status evaluation result and maintenance strategy to the digital twin body, update the state information of the digital twin body, and conduct dynamic monitoring and maintenance guidance on the health status of the industrial robot.

[0013] Furthermore, in step S2, an adaptive learning rate adjustment mechanism is introduced in the training process of the deep Q-network, and the following update formula is adopted:

[0014]

[0015] where θ t is the parameter of the deep Q-network at time t, α t is the adaptive learning rate at time t, and its value is dynamically adjusted according to the gradient change during the network training process; G t is the sum of the squares of the gradients before time t, and ∈ is a very small positive number to prevent the denominator from being zero; is the gradient of the loss function J(θ t ) with respect to the network parameter θ t . Update the deep Q-network parameters through this formula to optimize the feature extraction ability of the industrial robot health status parameters.

[0016] Further, in step S3, the improved Hidden Markov Model introduces an environmental impact factor to construct an extended state transition probability matrix. Its calculation formula is:

[0017]

[0018] Where, is the probability of transitioning from state i to state j after improvement, a ij is the original state transition probability, β is the environmental impact coefficient, which is used to measure the influence degree of environmental factors on the health state transition of industrial robots; E t is the environmental parameter vector at time t, and this vector contains environmental factor data such as temperature, humidity, and air pressure. By introducing the environmental impact factor, the description accuracy of the dynamic changes in the health state of industrial robots is improved.

[0019] Further, in step S4, during the process of calculating the posterior probability by the forward-backward algorithm, combined with the weight coefficients of each health state parameter of the industrial robot, a weighted posterior probability calculation formula is constructed:

[0020]

[0021] Where, P(S i ∣O, λ) is the posterior probability that the industrial robot is in state S i under the observation sequence O and the model parameters λ; α t (i) is the forward variable, which represents the probability of being in state S i at time t and generating a partial observation sequence; β t (i) is the backward variable, which represents the probability of being in state S i at time t and generating the remaining observation sequence; w i is the comprehensive weight coefficient of each health state parameter corresponding to state S i , which reflects the importance degree of each parameter for state evaluation under different health states. The health state of the industrial robot is evaluated through this formula.

[0022] Further, in step S5, the preset health state evaluation rule combines the feature vector extracted by the deep Q-network and the state probability calculated by the improved Hidden Markov Model to construct a health state evaluation function:

[0023]

[0024] Where, H is the health state evaluation value of the industrial robot, M is the number of evaluation indicators; c k is the weight coefficient of the kth evaluation indicator; is the kth evaluation function, is the feature vector output by the deep Q-network, To improve the state probability vector calculated by the Hidden Markov Model, the health state of the industrial robot is quantitatively graded and evaluated through this evaluation function.

[0025] Furthermore, in step S6, the reinforcement learning mechanism of the Deep Q-Network constructs a reward function R, and its calculation formula is:

[0026]

[0027] where r s is the immediate reward obtained by taking action a in the current state s, and its value is determined according to the improvement degree of the health state of the industrial robot; γ is the discount factor, which is used to measure the importance of future rewards; Q(s', a') is the Q-value of taking action a' in the subsequent state s', and this reward function is used to guide the Deep Q-Network to generate more effective maintenance strategies.

[0028] Furthermore, in step S7, the state update process of the digital twin incorporates the influence of historical health state data, and a digital twin state update formula is constructed:

[0029]

[0030] where is the state of the digital twin at time t, is the actual health state of the industrial robot at time t, is the state of the digital twin at time t-1, and δ is the update coefficient, which is used to adjust the contribution degrees of the actual health state and the historical digital twin state to the update of the current digital twin state, and maps the digital twin to the health state of the industrial robot.

[0031] Furthermore, in step S2, the health state parameter data input into the Deep Q-Network is subjected to feature fusion processing, and a feature fusion formula is constructed:

[0032]

[0033] where is the fused feature vector, L is the number of dimensions of the health state parameters, w l is the fusion weight of the l-th dimension of the health state parameters, is the l-th dimension of the original health state parameter vector, and the feature fusion improves the extraction efficiency of the Deep Q-Network for the health state features of the industrial robot.

[0034] Furthermore, in step S3, the improved Hidden Markov Model combines the mutual influence of the health states of the joints of the industrial robot to construct a joint state correlation matrix C, and its calculation formula is:

[0035]

[0036] where c ij is the state correlation degree between joint i and joint j, T is the time length of the observed data, is an indicator function, which takes the value of 1 when the health states of joint i and joint j change simultaneously at time k, otherwise 0. The modeling ability of the hidden Markov model for the overall health state of the industrial robot is optimized and improved through this joint state correlation matrix.

[0037] An industrial robot health state monitoring system based on digital twin, the system includes:

[0038] A data acquisition and preprocessing unit, which is used to acquire multi-dimensional health state parameters during the operation of the industrial robot, and perform preliminary format conversion and standardization processing;

[0039] A deep Q-network feature extraction unit, which is connected to the data acquisition and preprocessing unit, receives the processed health state parameter data, extracts features through the deep Q-network, and outputs a feature vector containing health state features;

[0040] An improved hidden Markov model processing unit, which is connected to the deep Q-network feature extraction unit, receives the feature vector, and uses the improved hidden Markov model for time series modeling to calculate the state transition probability matrix and the observation probability matrix;

[0041] A posterior probability calculation unit, which is connected to the improved hidden Markov model processing unit, and calculates the posterior probability of the industrial robot in different health states by using the forward-backward algorithm according to the model calculation results and the current observed data;

[0042] A health state evaluation unit, which is connected to the posterior probability calculation unit, receives the posterior probability data, and performs hierarchical evaluation on the health state of the industrial robot based on the preset health state evaluation rules;

[0043] A maintenance strategy generation unit, which is connected to the health state evaluation unit. When the health state of the industrial robot is in the warning or failure level, a maintenance strategy is generated based on the reinforcement learning mechanism of the deep Q-network;

[0044] A digital twin update unit, which is connected to the health state evaluation unit and the maintenance strategy generation unit, receives the health state evaluation results and the maintenance strategy, updates the state information of the industrial robot digital twin, and dynamically monitors the health state of the industrial robot.

[0045] Beneficial effects: The present invention proposes a method and a monitoring system for monitoring the health status of industrial robots based on digital twins. This method uses the multi-layer convolutional structure of the deep Q-network to extract and abstract the health status parameters, combines the feature fusion formula to enhance the data processing efficiency, and at the same time improves the hidden Markov model by introducing environmental impact factors and joint state correlation matrices to accurately capture the dynamic relationships between data and the influence of environmental factors, avoiding misjudgment and lag problems. Aiming at the defects of weak environmental adaptability and decision-making ability, the system constructs an extended state transition probability matrix containing environmental parameters to quantify the impact of environmental factors on the health state transition of the robot; after a fault warning, the deep Q-network generates a precise maintenance strategy based on the reinforcement learning mechanism and a specific reward function according to the current state and environmental information of the robot, reducing maintenance costs and downtime. In addition, the introduction of the digital twin realizes the real-time interaction between the physical entity and the virtual model. By combining the state update formula of historical data, the digital twin can accurately map the health status of the industrial robot, achieving dynamic monitoring and maintenance guidance. Each unit of the monitoring system operates in coordination, forming a complete monitoring closed-loop from data collection, feature extraction to state evaluation and strategy generation, significantly improving the accuracy, timeliness and environmental adaptability of the health status monitoring of industrial robots, and providing an efficient and reliable technical support for the intelligent operation and maintenance of industrial robots. Description of the Drawings

[0046] Figure 1 is the flowchart of the method steps of the present invention;

[0047] Figure 2 is the composition diagram of the system units of the present invention. Detailed Embodiments

[0048] It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The following further describes this application in detail with reference to the drawings and specific embodiments.

[0049] As Figure 1 shown, the method for monitoring the health status of industrial robots based on digital twins includes the following steps:

[0050] Step S1: Using the multi-dimensional health status parameters generated during the operation of the industrial robot, construct a digital twin of the industrial robot, and the multi-dimensional health status parameters include joint torque, motor current, vibration amplitude, and temperature data;

[0051] Specifically, during the operation of an industrial robot, various types of health status parameters are generated by its key components. For example, joint torque reflects the load borne by mechanical joints during movement, motor current reflects the working intensity of the motor, vibration amplitude can be used to determine whether there is abnormal vibration in components, and temperature data can directly show the heat generation situation of the device. These parameters describe the operating state of the industrial robot from different dimensions and are the basic data for constructing a digital twin. During implementation, various sensors such as torque sensors, current sensors, vibration sensors, and temperature sensors are installed at key parts such as the joints and motors of the industrial robot to collect these multi-dimensional health status parameters in real time. After the collected data undergoes preliminary format conversion and standardization processing, it is transmitted to the digital twin construction module for constructing a virtual model that corresponds one-to-one with the physical entity industrial robot.

[0052] Constructing a digital twin of an industrial robot is of great significance. A digital twin is a mapping of a physical entity in the virtual space, which can reflect the actual operating state of the industrial robot in real time. By observing and analyzing the digital twin, a comprehensive understanding of the operating conditions of the industrial robot can be obtained without directly contacting the physical device. This not only provides an intuitive visualization object for subsequent health status monitoring but also enables predicting possible future problems of the industrial robot through simulation and emulation of the digital twin, providing a basis for taking preventive maintenance measures in advance, thereby improving the reliability and service life of the industrial robot and reducing production interruptions caused by equipment failures.

[0053] Step S2: Input the health status parameter data collected in real time during the operation of the industrial robot into the deep Q-network at a preset time interval. The deep Q-network adopts a multi-layer convolutional neural network structure. By extracting and abstracting the features of the input data, a feature vector containing health status features is formed.

[0054] Specifically, during the operation of the industrial robot, the sensors continuously collect health status parameter data. To enable the deep Q-network to effectively process this data, the collected data needs to be input into the deep Q-network at a preset time interval (such as every second, every millisecond, etc., and the specific interval is determined according to actual requirements and data collection frequency). The deep Q-network adopts a multi-layer convolutional neural network structure, which has unique advantages in processing data with spatial and temporal features. During implementation, the input data first passes through the convolutional layer of the deep Q-network. The convolutional kernels in the convolutional layer perform convolutional operations on the data, extracting local features in the data through a sliding window method. Then it passes through the pooling layer, which performs dimensionality reduction on the extracted features, reducing the amount of data while retaining key features. Finally, it passes through the fully connected layer, which integrates and abstracts the features after convolution and pooling to form a feature vector containing health status features.

[0055] Inputting the health status parameter data into the deep Q-network for feature extraction and abstraction can extract the key information that best reflects the health status of the industrial robot from a large amount of raw data. The original health status parameter data often has the characteristics of high dimension and complexity, and it is difficult to directly use this data for health status assessment. The deep Q-network can automatically learn the feature patterns in the data through a multi-layer convolutional neural network structure, and transform the raw data into low-dimensional and representative feature vectors. These feature vectors contain the key information of the operating status of the industrial robot, providing a high-quality data basis for subsequent time series modeling and health status assessment using the improved hidden Markov model, and improving the accuracy and efficiency of monitoring.

[0056] Step S3: Use the improved hidden Markov model to perform time series modeling on the feature vectors output by the deep Q-network, and obtain the state transition probability matrix and the observation probability matrix of the health status of the industrial robot by combining the dynamic transition relationship of the health status parameters of the industrial robot in the time series;

[0057] Specifically, although the feature vectors output by the deep Q-network contain the key features of the health status of the industrial robot, these features are isolated and do not reflect the dynamic changes of the data in the time series. The improved hidden Markov model can handle the data with time series characteristics well. During the implementation process, the feature vectors output by the deep Q-network are input into the improved hidden Markov model in sequence according to the time order. The model will first analyze the transition law of the health status of the industrial robot between different moments based on the historical data and the currently input feature vectors, so as to calculate the state transition probability matrix, which describes the probability of the industrial robot transferring from one health status to another. At the same time, the model will also calculate the observation probability matrix according to the possible feature vector situations under each health status, which represents the probability of observing a specific feature vector under a specific health status.

[0058] Using the improved hidden Markov model for time series modeling can deeply explore the dynamic change law of the health status parameters of the industrial robot in the time series. The health status of the industrial robot is not static, but changes continuously over time. Through the obtained state transition probability matrix and observation probability matrix, this change relationship can be quantified, and the evolution process of the health status of the industrial robot can be described more accurately. This helps to predict in advance the possible health problems of the industrial robot, take timely measures before the occurrence of faults, avoid production losses caused by equipment failures, and improve the stability and reliability of industrial production.

[0059] Step S4: Based on the state transition probability matrix and the observation probability matrix calculated by the improved Hidden Markov Model, combined with the observation data at the current moment, use the forward-backward algorithm to calculate the posterior probabilities of the industrial robot in different health states;

[0060] Specifically, after obtaining the state transition probability matrix and the observation probability matrix calculated by the improved Hidden Markov Model, combined with the observation data (i.e., feature vectors) collected by the sensor at the current moment and processed by the deep Q-network, use the forward-backward algorithm to calculate the posterior probabilities of the industrial robot in different health states. The forward algorithm starts from the initial moment and gradually calculates the probabilities of being in different health states at each moment and being able to generate the current observation sequence; the backward algorithm starts from the last moment and calculates backward the probabilities of being in different health states at each moment and being able to generate the remaining observation sequence. By combining the forward probabilities and the backward probabilities, the posterior probabilities of the industrial robot in each health state can be calculated under the current observation data.

[0061] Calculating the posterior probabilities of the industrial robot in different health states can provide a quantitative basis for health state assessment. The posterior probability reflects the likelihood of the industrial robot being in different health states after considering all known information (including the state transition probability matrix, the observation probability matrix, and the current observation data). Compared with simply relying on the state transition probability and the observation probability, the posterior probability comprehensively considers the influence of the actual observation data and can more accurately judge the current health state of the industrial robot. Based on these posterior probabilities, the health state of the industrial robot can be scientifically and reasonably evaluated, providing strong support for subsequent decision-making.

[0062] Step S5: Input the posterior probabilities into the decision-making module, and this decision-making module conducts a hierarchical assessment of the health state of the industrial robot based on the preset health state assessment rules, and divides different state levels such as normal, warning, and fault;

[0063] Specifically, after calculating the posterior probabilities of the industrial robot in different health states, input these posterior probability data into the decision-making module. A series of health state assessment rules are preset inside the decision-making module, and these rules are comprehensively formulated based on various factors such as the operating characteristics of the industrial robot, historical fault data, and industry standards. For example, when the posterior probabilities of certain health states exceed a certain threshold, it is determined that the industrial robot is in a warning state; when the posterior probability reaches a higher threshold, it is determined to be in a fault state; and when the posterior probabilities of all health states are within the normal range, it is determined that the industrial robot is in a normal operating state. The decision-making module analyzes and judges the input posterior probabilities according to these preset rules, thereby dividing the health state of the industrial robot into different state levels such as normal, warning, and fault.

[0064] Grading and evaluating the health status of industrial robots can enable operation and maintenance personnel to quickly and intuitively understand the operating conditions of the equipment. Different status levels correspond to different handling methods. When the industrial robot is in a normal state, only routine maintenance and monitoring are required; when it is in a warning state, the monitoring frequency needs to be increased, and the repair resources that may be required in advance should be prepared; while when it is in a failure state, repair measures need to be taken immediately to restore the normal operation of the equipment as soon as possible. This grading and evaluation method improves the efficiency and pertinence of equipment operation and maintenance, helps to reasonably arrange maintenance resources, and reduces the equipment maintenance cost.

[0065] Step S6: When the health status of the industrial robot is at the warning or failure level, based on the reinforcement learning mechanism of the deep Q-network, according to the current health status and environmental information of the industrial robot, generate corresponding maintenance strategies;

[0066] Specifically, when the decision-making module determines that the health status of the industrial robot is at the warning or failure level, the system will start the reinforcement learning mechanism based on the deep Q-network to generate maintenance strategies. The reinforcement learning mechanism of the deep Q-network continuously interacts with the environment to learn which actions to take in different states to obtain the maximum benefit. During the implementation process, taking the current health status of the industrial robot (reflected by the posterior probability and status level) and environmental information (such as environmental parameters like temperature and humidity) as inputs, the deep Q-network will output a series of possible maintenance actions and their corresponding Q-values (indicating the expected benefit of this action in the current state) according to its internal network parameters and learned strategies. Then, the system will select the action with the highest Q-value as the final maintenance strategy, such as replacing a certain component, adjusting operating parameters, etc.

[0067] Generating maintenance strategies based on the reinforcement learning mechanism of the deep Q-network can provide intelligent and personalized solutions for the fault handling of industrial robots. Traditional maintenance strategies are often fixed solutions based on experience and lack flexible response to actual situations. While the deep Q-network can dynamically generate the most suitable maintenance strategy according to the current specific health status and environmental conditions of the industrial robot through reinforcement learning. This method can improve the accuracy and effectiveness of maintenance, solve equipment fault problems faster, reduce equipment downtime, and improve the continuity and efficiency of industrial production.

[0068] Step S7: Feed back the industrial robot health status evaluation result and maintenance strategy to the digital twin, update the status information of the digital twin, and conduct dynamic monitoring and maintenance guidance on the health status of the industrial robot.

[0069] Specifically, after completing the assessment of the health status of the industrial robot and generating a maintenance strategy, the health status assessment results (such as status levels of normal, warning, or failure) and the maintenance strategy information are fed back into the digital twin of the industrial robot. After receiving this information, the digital twin updates its own status information according to the feedback content. For example, it marks the health status of the digital twin as the corresponding level and simulates the effect after executing the maintenance strategy in the virtual model. In this way, the digital twin can always be synchronized with the actual status of the physical entity industrial robot, realizing the dynamic monitoring of the health status of the industrial robot. At the same time, the operation and maintenance personnel can better understand the health status of the industrial robot and the effectiveness of the maintenance strategy by observing the status changes of the digital twin and the simulation effect of the maintenance strategy, thus providing more intuitive and accurate guidance for subsequent operation and maintenance decisions.

[0070] Feeding back the evaluation results and the maintenance strategy to the digital twin and updating its status information form a complete closed-loop monitoring and maintenance system. The digital twin is no longer a static model but can be dynamically updated as the actual status of the industrial robot changes. This dynamic update mechanism enables the digital twin to reflect the latest situation of the industrial robot in real time, providing timely and accurate information support for the operation and maintenance personnel. By simulating the execution effect of the maintenance strategy in the digital twin, the feasibility and potential risks of the maintenance strategy can also be evaluated in advance, optimizing the maintenance plan and improving the operation and maintenance management level of the industrial robot.

[0071] Preferably, in step S2, an adaptive learning rate adjustment mechanism is introduced in the training process of the deep Q-network, and the following update formula is adopted:

[0072]

[0073] where θ t is the parameter of the deep Q-network at time t, α t is the adaptive learning rate at time t, and its value is dynamically adjusted according to the gradient change during the network training process; G t is the sum of the squares of the gradients before time t, and ∈ is a very small positive number to prevent the denominator from being zero; is the gradient of the loss function J(θ t ) with respect to the network parameter θ t . By updating the parameters of the deep Q-network through this formula, the ability to extract the characteristics of the health status parameters of the industrial robot is optimized.

[0074] Specifically, during the training process of the deep Q-network, an adaptive learning rate adjustment mechanism is introduced. This mechanism dynamically adjusts the learning rate according to the change of the gradient during network training. Specifically, when implementing, the update of network parameters comprehensively considers parameters such as the current learning rate, the sum of historical gradient squares, and the gradient of the loss function. Through this mechanism, the problems of slow convergence speed or excessive oscillation that are prone to occur in network training under the traditional fixed learning rate can be avoided. When the deep Q-network processes the health state parameters of industrial robots, it can optimize parameters more efficiently, enhance the ability to extract data features, and thus improve the capture accuracy of the health state features of industrial robots by the entire monitoring system.

[0075] Preferably, in step S3, the improved hidden Markov model introduces an environmental impact factor to construct an extended state transition probability matrix. Its calculation formula is:

[0076]

[0077] Wherein, is the probability of transitioning from state i to state j after improvement, a ij is the original state transition probability, β is the environmental impact coefficient, which is used to measure the influence degree of environmental factors on the health state transition of industrial robots; E t is the environmental parameter vector at time t, and this vector contains environmental factor data such as temperature, humidity, and air pressure. By introducing the environmental impact factor, the description accuracy of the dynamic change of the health state of industrial robots is improved.

[0078] Specifically, the improved hidden Markov model introduces an environmental impact factor to construct an extended state transition probability matrix. During implementation, according to the environmental parameter data such as temperature, humidity, and air pressure in the operating environment of industrial robots, combined with the preset environmental impact coefficient, the original state transition probability is corrected. By quantifying environmental factors and integrating them into the model, the improved hidden Markov model can more realistically reflect the dynamic transition process of the health state of industrial robots in a complex environment, effectively making up for the state prediction deviation caused by the traditional model not considering environmental factors, and improving the accuracy and reliability of the description of the health state change of industrial robots.

[0079] Preferably, in step S4, during the process of calculating the posterior probability by the forward-backward algorithm, combined with the weight coefficients of the health state parameters of industrial robots, a weighted posterior probability calculation formula is constructed:

[0080]

[0081] Wherein, P(S i ∣O, λ) is the posterior probability that the industrial robot is in state S i under the observation sequence O and the model parameters λ; αt (i) is the forward variable, representing the probability of being in state S at time t i and generating a partial observation sequence; β t (i) is the backward variable, representing the probability of being in state S at time t i and generating the remaining observation sequence; w i is the comprehensive weight coefficient of each health state parameter corresponding to state S i reflecting the importance of each parameter for state evaluation under different health states, and the health state of the industrial robot is evaluated through this formula.

[0082] Specifically, when calculating the posterior probability using the forward-backward algorithm, the weight coefficients of each health state parameter of the industrial robot are introduced. During implementation, corresponding weights are assigned to different parameters according to the importance of each health state parameter for the overall health assessment of the industrial robot. These weight coefficients participate in the calculation of the posterior probability together with the forward variable and the backward variable, enabling the calculation result of the posterior probability to more accurately reflect the contribution differences of each health state parameter to the health state evaluation of the industrial robot. Through this weighted calculation method, the evaluation one-sidedness caused by treating each parameter equally in the traditional algorithm is effectively avoided, and a more accurate quantitative evaluation of the health state of the industrial robot is achieved.

[0083] Preferably, in step S5, a health state evaluation function is constructed by combining the feature vector extracted by the deep Q-network and the state probability calculated by the improved hidden Markov model with the preset health state evaluation rules:

[0084]

[0085] where H is the health state evaluation value of the industrial robot, and M is the number of evaluation indicators; c k is the weight coefficient of the k-th evaluation indicator; is the k-th evaluation function, is the feature vector output by the deep Q-network, is the state probability vector calculated by the improved hidden Markov model, and the health state of the industrial robot is quantitatively graded and evaluated through this evaluation function.

[0086] Specifically, for the health status evaluation rule, an evaluation function that combines the feature vector of the deep Q-network and the state probability of the improved hidden Markov model is constructed. During the implementation process, according to the importance of different health status evaluation indicators of the industrial robot, weight coefficients are set for each evaluation indicator. The feature vector output by the deep Q-network and the state probability vector calculated by the improved hidden Markov model are used as inputs, and weighted calculations are performed through the evaluation function. Finally, a quantified health status evaluation value is obtained. This evaluation function integrates the advantageous information of the two models, avoids the limitations of a single data source or evaluation method, and provides a scientific and comprehensive quantitative basis for the hierarchical evaluation of the health status of industrial robots.

[0087] Preferably, in step S6, the reinforcement learning mechanism of the deep Q-network constructs a reward function R, and its calculation formula is:

[0088]

[0089] where r s is the immediate reward obtained by taking action a in the current state s, and its value is determined according to the improvement degree of the health status of the industrial robot; γ is the discount factor, which is used to measure the importance of future rewards; Q(s', a') is the Q-value of taking action a' in the subsequent state s'. Through this reward function, the deep Q-network is guided to generate a more effective maintenance strategy.

[0090] Specifically, the reinforcement learning mechanism of the deep Q-network constructs a specific reward function. When the health status of the industrial robot is in the warning or failure level, the immediate reward is set according to the current improvement degree of the health status, and the reward function value is calculated by combining the discount factor and the Q-value of the subsequent state. During implementation, the system gives corresponding immediate rewards according to the actual improvement effect of different maintenance strategies on the health status of the industrial robot, and balances the importance of current and future rewards through the discount factor. This reward function guides the deep Q-network to learn the optimal maintenance strategy in different states, enabling the system to dynamically generate more demand-oriented and effective maintenance strategies according to the actual situation of the industrial robot, and improving the pertinence and efficiency of fault handling.

[0091] Preferably, in step S7, the state update process of the digital twin incorporates the influence of historical health status data, and a digital twin state update formula is constructed:

[0092]

[0093] where is the state of the digital twin at time t, is the actual health status of the industrial robot at time t, is the state of the digital twin at time t-1, and δ is the update coefficient, which is used to adjust the contribution degrees of the actual health state and the historical digital twin state to the update of the current digital twin state, and maps the health state of the industrial robot by the digital twin.

[0094] Specifically, the update process of the digital twin state introduces an update formula that takes into account historical health state data. During implementation, based on the actual health state data collected in real time by the industrial robot and the state data of the digital twin at the previous moment, combined with the preset update coefficient, the current state of the digital twin is updated. This update method avoids misjudgment of state fluctuations caused by relying only on real-time data or lag of state update caused by relying only on historical data, enables the digital twin to respond to changes in the actual state of the industrial robot in a timely manner, and can smoothly transition and reduce noise interference, realizing a more stable and accurate mapping of the health state of the industrial robot, and providing reliable support for operation and maintenance decision-making.

[0095] Preferably, in step S2, the health state parameter data input into the deep Q network is subjected to feature fusion processing to construct a feature fusion formula:

[0096]

[0097] where is the fused feature vector, L is the number of dimensions of the health state parameters, and w l is the fusion weight of the l-th dimension of the health state parameters, is the l-th dimension of the original health state parameter vector, and the feature extraction efficiency of the deep Q network for the health state features of the industrial robot is improved through feature fusion.

[0098] Specifically, the health state parameter data input into the deep Q network is subjected to feature fusion processing. During implementation, according to the contribution degrees of each health state parameter dimension of the industrial robot to feature extraction, fusion weights are set for different dimension parameters, and the original health state parameter vectors of each dimension are weighted and summed to obtain the fused feature vector. This feature fusion processing reduces data redundancy, integrates the effective information of different parameter dimensions, reduces the complexity of data processed by the deep Q network, and at the same time enhances the network's ability to extract key features of the health state of the industrial robot, improving the feature extraction efficiency and the overall performance of the monitoring system.

[0099] Preferably, in step S3, the improved hidden Markov model combines the mutual influences of the health states of each joint of the industrial robot to construct a joint state correlation matrix C, and its calculation formula is:

[0100]

[0101] where c ijis the state correlation degree between joint i and joint j, and T is the time length of the observed data. is an indicator function that takes the value of 1 when the health states of joint i and joint j change simultaneously at time k, and 0 otherwise. The modeling ability of the hidden Markov model for the overall health state of the industrial robot is optimized through this joint state correlation matrix.

[0102] Specifically, the improved hidden Markov model considers the mutual influence of the health states of each joint of the industrial robot and constructs a joint state correlation matrix. During the implementation process, the changes in the health states of each joint of the industrial robot during operation are statistically analyzed, and the joint state correlation degree is determined by calculating the frequency of simultaneous changes in the health states between joints, forming a joint state correlation matrix. This matrix reflects the mutual influence relationship of the health states of each joint. The improved hidden Markov model optimizes the modeling process based on this matrix, fully considering the coupling effect between joints, avoiding the defect of the traditional model that analyzes the health states of each joint in isolation, and improving the comprehensiveness and accuracy of modeling the overall health state of the industrial robot.

[0103] As Figure 2 shown, the industrial robot health state monitoring system based on digital twin includes:

[0104] A data acquisition and preprocessing unit for acquiring multi-dimensional health state parameters during the operation of the industrial robot and performing preliminary format conversion and standardization processing;

[0105] A deep Q-network feature extraction unit is connected to the data acquisition and preprocessing unit, receives the processed health state parameter data, extracts features through the deep Q-network, and outputs a feature vector containing health state features;

[0106] An improved hidden Markov model processing unit is connected to the deep Q-network feature extraction unit, receives the feature vector, and uses the improved hidden Markov model for time series modeling to calculate the state transition probability matrix and the observation probability matrix;

[0107] A posterior probability calculation unit is connected to the improved hidden Markov model processing unit, and based on the model calculation results and the current observed data, uses the forward-backward algorithm to calculate the posterior probability of the industrial robot in different health states;

[0108] A health state evaluation unit is connected to the posterior probability calculation unit, receives the posterior probability data, and based on the preset health state evaluation rules, conducts a hierarchical evaluation of the health state of the industrial robot;

[0109] A maintenance strategy generation unit is connected to the health state evaluation unit. When the health state of the industrial robot is in the early warning or failure level, a maintenance strategy is generated based on the reinforcement learning mechanism of the deep Q-network;

[0110] The digital twin update unit is connected to the health status assessment unit and the maintenance strategy generation unit, receives the health status assessment results and maintenance strategies, updates the status information of the industrial robot digital twin, and dynamically monitors the health status of the industrial robot.

[0111] Through the multi-layer convolutional structure of the deep Q-network, the present invention extracts deep features from parameters such as joint torque and motor current, and integrates multi-dimensional information using the feature fusion formula, significantly improving the data processing efficiency. At the same time, the improved hidden Markov model introduces an environmental impact factor and a joint state correlation matrix to accurately depict the dynamic transfer law of the health status of industrial robots in the time series and spatial dimensions, effectively avoiding misjudgment problems caused by insufficient data feature analysis.

[0112] Aiming at the defects of poor environmental adaptability and weak decision-making ability in the traditional technology, the system constructs a dynamic monitoring and decision-making mechanism. The improved hidden Markov model incorporates environmental factors such as temperature and humidity into the modeling system by expanding the state transition probability matrix, quantifies the impact of environmental changes on the health status of the robot, and ensures the accuracy of monitoring results under complex working conditions. When a warning or fault state is detected, the deep Q-network generates accurate maintenance strategies based on the reinforcement learning mechanism and a customized reward function, combined with the real-time health status and environmental parameters, changing the situation of relying on empirical judgment and lagging maintenance plans in the traditional method, and significantly reducing the equipment downtime and maintenance costs.

[0113] The monitoring system also realizes the closed-loop management of virtual-real interaction relying on digital twin technology. The digital twin continuously iterates and optimizes through the update formula that fuses the actual health status and historical data, and accurately maps the operating status of the industrial robot. Each unit of the system (data acquisition and preprocessing unit, deep Q-network feature extraction unit, etc.) collaborates and links together to form a complete chain from data acquisition, feature analysis, status assessment to strategy generation. In addition, the application of innovative formulas such as weighted posterior probability calculation and health status assessment function further improves the scientificity of status judgment, enabling the health monitoring of industrial robots to change from passive response to active prediction and intelligent decision-making, providing an efficient and reliable technical paradigm for equipment operation and maintenance in the field of intelligent manufacturing.

[0114] In the description of the present invention, it should be noted that unless otherwise clearly specified and limited, the terms "set", "install", "connected", "connected", "fixed" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood through specific situations.

[0115] Although embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various equivalent changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for monitoring the health status of an industrial robot based on digital twin, characterized in that, It includes the following steps: Step S1: Using the multi-dimensional health state parameters generated during the operation of the industrial robot, construct a digital twin of the industrial robot. The multi-dimensional health state parameters include joint torque, motor current, vibration amplitude, and temperature data; Step S2: Input the health state parameter data collected in real time during the operation of the industrial robot into the deep Q network at preset time intervals. The deep Q network adopts a multi-layer convolutional neural network structure. Through feature extraction and abstraction of the input data, a feature vector containing health state features is formed; Step S3: Use an improved hidden Markov model to perform temporal modeling on the feature vector output by the deep Q network. By combining the dynamic transfer relationship of the health state parameters of the industrial robot in the time series, obtain the state transition probability matrix and observation probability matrix of the health state of the industrial robot; Step S4: Based on the state transition probability matrix and observation probability matrix calculated by the improved hidden Markov model, combined with the observation data at the current moment, use the forward-backward algorithm to calculate the posterior probability of the industrial robot in different health states; Step S5: Input the posterior probability into the decision-making module. Based on the preset health state evaluation rules, evaluate the health state of the industrial robot at different levels, and divide it into different state levels of normal, warning, and failure; Step S6: When the health state of the industrial robot is at the warning or failure level, based on the reinforcement learning mechanism of the deep Q network, generate corresponding maintenance strategies according to the current health state and environmental information of the industrial robot; Step S7: Feed back the health state evaluation result and maintenance strategy of the industrial robot to the digital twin, update the state information of the digital twin, and conduct dynamic monitoring and maintenance guidance on the health state of the industrial robot.

2. The method for monitoring the health status of an industrial robot based on digital twin according to claim 1, wherein, In step S2, an adaptive learning rate adjustment mechanism is introduced in the training process of the deep Q network, and the following update formula is adopted: Among them, θ t is the parameter of the deep Q-network at time t, and α t is the adaptive learning rate at time t, and its value is dynamically adjusted according to the gradient change during the network training process; G t is the sum of the squares of the gradients before time t, and ∈ is a very small positive number to prevent the denominator from being zero; is the gradient of the loss function J(θ t ) with respect to the network parameter θ t . The parameters of the deep Q-network are updated through this formula to optimize the feature extraction ability of the health state parameters of the industrial robot.

3. The method for monitoring the health status of an industrial robot based on digital twin according to claim 1, wherein In step S3, an improved Hidden Markov Model introduces an environmental impact factor to construct an extended state transition probability matrix Its calculation formula is as follows: Among them, is the probability of transitioning from state i to state j after improvement, a ij is the original state transition probability, and β is the environmental impact coefficient, which is used to measure the influence degree of environmental factors on the health state transition of industrial robots; E t is the environmental parameter vector at time t. This vector contains environmental factor data such as temperature, humidity, and air pressure. By introducing the environmental impact factor, the description accuracy of the dynamic changes in the health state of industrial robots is improved.

4. The method for monitoring the health status of an industrial robot based on digital twin according to claim 1, wherein In step S4, during the process of calculating the posterior probability by the forward-backward algorithm, combined with the weight coefficients of the health state parameters of the industrial robot, construct a weighted posterior probability calculation formula: Among them, P(S i |O, λ) is the posterior probability that the industrial robot is in state S i under the observation sequence O and the model parameter λ; α t (i) is the forward variable, representing the probability of being in state S i at time t and generating a partial observation sequence; β t (i) is the backward variable, representing the probability of being in state S i at time t and generating the remaining observation sequence; w i is the comprehensive weight coefficient of each health state parameter corresponding to state S i , reflecting the importance degree of each parameter to the state evaluation under different health states. The health state of the industrial robot is evaluated through this formula.

5. The method for monitoring the health status of an industrial robot based on digital twin according to claim 1, wherein In step S5, the preset health state evaluation rules are combined with the feature vector extracted by the deep Q network and the state probability calculated by the improved hidden Markov model to construct a health state evaluation function: Among them, H is the health status evaluation value of the industrial robot, and M is the number of evaluation indicators; c k is the weight coefficient of the k-th evaluation indicator; is the k-th evaluation function, is the feature vector output by the deep Q network, is the state probability vector calculated by the improved hidden Markov model, and the health status of the industrial robot is quantitatively graded and evaluated through this evaluation function.

6. The method for monitoring the health status of an industrial robot based on digital twin according to claim 1, wherein In step S6, the reinforcement learning mechanism of the deep Q network constructs a reward function R, and its calculation formula is: where r s is the immediate reward obtained by taking action a in the current state s, and its value is determined according to the improvement degree of the industrial robot's health status; γ is the discount factor, which is used to measure the importance of future rewards; Q(s', a') is the Q-value of taking action a' in the subsequent state s', and the deep Q-network is guided by this reward function to generate a more effective maintenance strategy.

7. The method for monitoring the health status of an industrial robot based on digital twin according to claim 1, characterized in that, In step S7, during the state update process of the digital twin, combined with the influence of historical health state data, construct a digital twin state update formula: Among them, is the state of the digital twin at time t, is the actual health state of the industrial robot at time t, is the state of the digital twin at time t-1, and δ is the update coefficient, which is used to adjust the contribution degrees of the actual health state and the historical digital twin state to the update of the current digital twin state, and maps the digital twin to the health state of the industrial robot.

8. The method for monitoring the health status of an industrial robot based on digital twin according to claim 1, wherein In step S2, perform feature fusion processing on the health state parameter data input into the deep Q network, and construct a feature fusion formula: Among them, is the fused feature vector, L is the number of dimensions of the health state parameters, and w l is the fusion weight of the l-th dimensional health state parameter, is the l-th dimensional original health state parameter vector. The extraction efficiency of the health state features of the industrial robot by the deep Q-network is improved through feature fusion.

9. The method for monitoring the health status of an industrial robot based on digital twin according to claim 1, wherein In step S3, the improved hidden Markov model combines the mutual influence of the health states of each joint of the industrial robot to construct a joint state correlation matrix C, and its calculation formula is: Among them, c ij is the state correlation degree between joint i and joint j, T is the time length of the observed data, is an indicator function, which takes the value of 1 when the health states of joint i and joint j change simultaneously at time k, and 0 otherwise. The modeling ability of the hidden Markov model for the overall health state of the industrial robot is optimized and improved through this joint state correlation matrix.

10. An industrial robot health status monitoring system based on digital twin, characterized in that, It includes: A data acquisition and preprocessing unit, which is used to collect the multi-dimensional health state parameters during the operation of the industrial robot, and perform preliminary format conversion and standardization processing; Deep Q-network feature extraction unit, connected to the data acquisition and preprocessing unit, receives the processed health status parameter data, extracts features through the deep Q-network, and outputs a feature vector containing health status features; Improved Hidden Markov Model processing unit, connected to the deep Q-network feature extraction unit, receives the feature vector, uses the improved Hidden Markov Model for time series modeling, and calculates the state transition probability matrix and the observation probability matrix; Posterior probability calculation unit, connected to the improved Hidden Markov Model processing unit, calculates the posterior probability of the industrial robot in different health states using the forward-backward algorithm based on the model calculation results and the current observation data; Health status evaluation unit, connected to the posterior probability calculation unit, receives the posterior probability data, and grades and evaluates the health status of the industrial robot based on the preset health status evaluation rules; Maintenance strategy generation unit, connected to the health status evaluation unit, generates a maintenance strategy based on the reinforcement learning mechanism of the deep Q-network when the health status of the industrial robot is in the warning or failure level; Digital twin update unit, connected to the health status evaluation unit and the maintenance strategy generation unit, receives the health status evaluation results and the maintenance strategy, updates the state information of the industrial robot digital twin, and dynamically monitors the health status of the industrial robot.

Images