Industrial robot health state assessment method based on multi-modal servo information mapping
By constructing a multimodal servo information mapping model and self-supervised comparative representation learning, the problem of accuracy in fault diagnosis of industrial robots under complex working conditions is solved, realizing accurate perception of robot health status and rapid fault diagnosis, thereby improving operation and maintenance efficiency and reliability.
Patent Information
- Application Number
- CN202511286161.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-10
- Publication Date
- 2025-11-14
AI Technical Summary
Existing technologies struggle to effectively utilize multimodal servo information under complex operating conditions, resulting in a lack of fault samples and difficulty adapting to complex conditions such as frequent start-stop and state switching, leading to insufficient accuracy in fault diagnosis for industrial robots.
A multimodal servo information mapping model is constructed. Self-supervised contrastive representation learning and incremental learning theories are adopted, combined with dynamics and kinematics theories, to establish a health perception model. Through the fusion of multimodal servo information and autonomous diagnosis, accurate perception and rapid diagnosis of the robot's health status are achieved.
It improves the accuracy of health status perception, enables rapid diagnosis of new faults and prediction of performance degradation, reduces operation and maintenance costs, and improves the reliability of robot service and the level of intelligent operation and maintenance.
Smart Images

Figure CN120941400A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of health assessment of industrial robots, and more particularly to a method for assessing the health status of industrial robots based on multimodal servo information mapping. Background Technology
[0002] Industrial robot health monitoring, a key research hotspot in the field of intelligent manufacturing, has attracted numerous scholars both domestically and internationally to conduct in-depth research and has yielded a series of significant results. In the field of fault diagnosis, early research mainly relied on the analysis of single signals such as vibration and current. In simple production scenarios, partial overload faults of the motor can be detected simply by monitoring fluctuations in motor current. However, under complex operating conditions, such as frequent start-stop cycles, high-speed heavy-load operation, and multi-task switching of industrial robots, the fault information contained in a single signal is extremely limited, making accurate fault diagnosis difficult.
[0003] In industrial robot systems, the servo system, as a core component, plays a crucial role in the precise motion control of the robot. Early research on industrial robot fault diagnosis based on servo system information mainly focused on using single electrical signals from the servo system, such as current signals, to diagnose faults. Internationally, Hitachi, by leveraging current sensing technology and intelligent analysis algorithms, can monitor the robot's fault status to a certain extent by analyzing the servo current data of industrial robots. However, the fault information that a single servo signal, represented by current, can reflect is limited. For complex mechanical faults such as joint wear or gear meshing defects, it is difficult to accurately diagnose them relying solely on a single servo signal.
[0004] In the field of intelligent fault diagnosis for industrial robots, the following problems still exist. Despite significant progress made both domestically and internationally in industrial robot health monitoring, the following issues remain:
[0005] First, there is insufficient utilization of multimodal information in servo systems. Existing research mainly focuses on single signal analysis, lacking in-depth fusion and characterization capabilities of multimodal servo information. Second, there is a scarcity of industrial robot fault samples, making it difficult for traditional supervised learning methods to meet practical needs. Third, there is poor adaptability to complex working conditions. Existing methods are mostly designed for single working conditions and are difficult to adapt to complex working conditions such as frequent start-stop and state switching. Summary of the Invention
[0006] The purpose of this invention is to provide a health status assessment method for industrial robots based on multimodal servo information mapping. By deeply analyzing the performance degradation mechanism of industrial robots under complex working conditions, a multimodal servo information mapping model is constructed to achieve accurate perception of the robot's health status. Self-supervised contrastive representation learning theory is used to address the problem of insufficient fault samples, improving the accuracy of health perception. Based on incremental learning theory of multimodal servo information, an autonomous diagnostic model is built to achieve rapid diagnosis of new faults and prediction of performance degradation.
[0007] To achieve the above objectives, the technical solution adopted by this invention is: a method for assessing the health status of industrial robots based on multimodal servo information mapping, comprising the following steps:
[0008] Step 1: Obtain the impact of complex service conditions on the performance of industrial robots and the degradation mechanism;
[0009] Step 2: Extract health perception features from the multimodal servo information collected during the operation of the industrial robot;
[0010] Step 3: Based on the health perception features collected in Step 2, establish a health perception model for industrial robots under complex working conditions.
[0011] Step 4: Establish an incremental learning model. Based on the model in Step 3, the model is continuously learned and updated as new data is added, thereby obtaining a degradation model and predicting its failure time distribution, providing decision support for the maintenance of industrial robot equipment.
[0012] Preferably, step three specifically involves: treating each data sample x and its transformed sample x(+) as a positive sample pair, with x(+) constructed based on x using noise addition, flipping, or random pruning enhancement methods; all other samples in the sample set except x are negative samples relative to x; then training a feature extractor composed of a deep neural network to extract features from the positive and negative samples. The training objective is to train a feature extractor as a mapping function f for x, mapping x to its feature f(x), such that:
[0013] D(f(x)),f(x(+)))>>D(f(x),f(x(-) i ))
[0014] In the formula, x(-) i Let D be any negative sample, and D(·,·) be a function that measures the similarity between sample features. Therefore, the training objective function of the feature extractor can be written as:
[0015]
[0016] Since D(·,·) are numbers greater than zero, the above expression approaches 0 when x is more similar to x(+), and approaches 0 when x is more similar to x(-). i When the similarity is greater than 0, the training objective of the feature extractor is to optimize its parameters so that the above expression reaches its minimum value.
[0017] Preferably, step one specifically involves: through literature review and analysis of actual cases, sorting out the operating data of industrial robots under frequent intermittent start-stop and frequent switching of operating states; using dynamics and kinematics theory, establishing motion models of the robot under different working conditions, and analyzing its impact on mechanical stress and fatigue damage of key components; and using fault tree analysis and failure mode and effect analysis methods to comprehensively sort out the basic failure modes of industrial robots, revealing the intrinsic physical mechanisms of performance degradation caused by material wear, electrical performance deterioration, etc. at the microscopic level.
[0018] Preferably, step two specifically involves comprehensively collecting and analyzing data such as current, voltage, position, speed, and torque in the servo system, using data mining algorithms to explore the potential relationship between different fault modes and servo data, designing a multimodal state monitoring method for harsh operating conditions, integrating sensor fusion technology and adaptive filtering algorithms to improve the accuracy and reliability of data acquisition, and establishing a mapping relationship between fault characteristics and servo data.
[0019] Preferably, the incremental learning model is established as follows: all nodes constitute a node set V = {X1, X2, ..., X...} n}, where element X i Let X be a random variable representing the state of the i-th node. The directed edges between nodes reflect the dependencies between variables. i The node it points to is its child node, and all nodes pointing to X i The node is its parent node, and their relationship can be represented as the conditional probability P[X]. i |Parent(X i )], i.e., X i The probability of X is affected by all its parent nodes. i If there is no parent node, it is called the root node. The overall state of the industrial robot is the joint probability distribution on the node set V, which can be represented as:
[0020]
[0021] The degradation process of each device is modeled using feature information derived from contrastive coding, thereby predicting its failure time distribution and providing decision support for the maintenance of industrial robot equipment.
[0022] Preferably, a Markov chain discrete state-space model is used to model the degradation state transition process of the component. The transition time between each pair of states is statistically analyzed on a long-term time scale. The state transition parameters in the degradation model are determined using the maximum likelihood estimation method. In practical applications, the current degradation state of the device in the degradation model is first determined based on the comparative coding features of the latest multimodal servo information. Then, based on the established degradation model, the degradation process from the current degradation state to the final state is repeatedly simulated using the Monte Carlo simulation method. The degradation time is statistically analyzed to obtain the failure time distribution of the device from the current degradation state to the failure state.
[0023] Preferably, the industrial robot equipment operates in an intermittent rather than continuous mode, and the statistical time is the actual working time of the equipment, excluding non-working time.
[0024] The technical effects of this invention are as follows:
[0025] 1. By deeply analyzing the performance degradation mechanism of industrial robots under complex working conditions, a multimodal servo information mapping model is constructed to achieve accurate perception of the robot's health status. The self-supervised contrastive representation learning theory is used to solve the problem of insufficient fault samples and improve the accuracy of health perception. Based on the incremental learning theory of multimodal servo information, an autonomous diagnostic model is built to achieve rapid diagnosis of new faults and prediction of performance degradation.
[0026] 2. This application provides new theories and methods for the field of industrial robot health monitoring, which is expected to promote the development of intelligent operation and maintenance technology for industrial robots. It can be widely applied to the health monitoring and fault diagnosis of industrial robots, and has significant economic and social benefits. First, it improves the service reliability of industrial robots by reducing the failure rate and downtime losses through real-time monitoring of the health status of industrial robots. Second, it reduces the operation and maintenance costs of industrial robots by enabling predictive maintenance through autonomous diagnosis and performance degradation prediction. Third, it promotes the development of the robot industry, provides technical support for the intelligent operation and maintenance of industrial robots, and helps the high-quality development of the manufacturing industry. Attached Figure Description
[0027] Figure 1 This is a schematic diagram of the process for assessing the health status of industrial robots based on multimodal servo information mapping.
[0028] Figure 2 This is a schematic diagram of self-supervised representation learning.
[0029] Figure 3 A schematic diagram illustrating incremental learning and autonomous diagnosis for industrial robots.
[0030] Figure 4 This is a schematic diagram of a method for predicting the degradation of industrial robots. Detailed Implementation
[0031] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.
[0032] This invention discloses a method for assessing the health status of industrial robots based on multimodal servo information mapping, such as... Figure 1 As shown, the specific description is as follows.
[0033] 1. Multimodal servo information mapping model for performance degradation of industrial robots during service
[0034] To address the performance degradation of industrial robots under complex service conditions, this paper constructs an accurate mapping model based on multimodal servo information expansion. First, it analyzes the impact and degradation mechanisms of complex service conditions on industrial robot performance. Through literature review and case studies, it gathers operational data of industrial robots under complex conditions such as frequent intermittent start-stop and frequent switching of operating states. Using dynamics and kinematics theories, it establishes motion models of the robot under different conditions, analyzing the impact on mechanical stress and fatigue damage of key components such as motors and joints. Using fault tree analysis and failure mode and effects analysis, it comprehensively identifies the basic failure modes of industrial robots, revealing the intrinsic physical mechanisms of performance degradation, such as material wear and electrical performance deterioration, at the microscopic level. For example, for the current surge during frequent motor start-stop, it establishes current-time variation curves and, combined with a motor heating model, analyzes the damage process to the insulation performance of the motor windings and the rotor mechanical structure.
[0035] Then, the effective characteristics of servo information in the industrial robot's operating state are explored. Comprehensive data collection and analysis are performed on current, voltage, position, speed, torque, and other data from the servo system. Data mining algorithms, such as association rule mining, are used to explore the potential connections between different fault modes and servo data. For harsh operating conditions, a multimodal state monitoring method is designed, integrating sensor fusion technology and adaptive filtering algorithms to improve the accuracy and reliability of data acquisition. Taking robot joint faults as an example, the changing patterns of torque and position data before and after the fault occur are studied, establishing a mapping relationship between fault characteristics and servo data to provide data support for subsequent fault diagnosis.
[0036] Finally, a multimodal representation model of the health status of industrial robot servo information is constructed. Principal component analysis and independent component analysis are used to extract features from single servo information, such as current and torque, and evaluate their ability to express the health status. On this basis, multimodal data fusion technology, such as neural network-based fusion methods, is used to fuse multimodal servo information such as current, torque, attitude, and vibration to construct a comprehensive health status representation model. The model is trained with a large number of normal and fault samples to accurately reflect the health status of the industrial robot, laying a solid data foundation for subsequent health perception and fault diagnosis.
[0037] 2. Health perception of industrial robots driven by multimodal servo information
[0038] Health perception features are extracted from the multimodal servo information collected during the operation of industrial robots. In actual use, industrial robots often fail only after long-term operation. Therefore, for a considerable period after they are put into use, only monitoring signals in the healthy state can be obtained. Thus, the health perception of industrial robots is essentially a single-class learning problem, that is, using known equipment health status monitoring data to build a model to detect unknown abnormal states. After the industrial robot has been in use for a period of time, a small amount of monitoring data in fault states will be accumulated. Experts will classify these monitoring data according to different fault modes and extract the features of each fault mode from these small number of fault mode samples so that the faults can be automatically classified in the subsequent operation of the industrial robot. Since the data sample size for each fault mode is small, it is difficult to train a classifier and extract fault mode features using supervised training.
[0039] Therefore, a health perception model for industrial robots under complex working conditions is established, specifically as follows: Figure 2 As shown, each data sample x and its transformed sample x(+) are considered as a positive sample pair. x(+) can be constructed based on x using enhancement methods such as adding noise, flipping, and random pruning. All other samples in the sample set except x are negative samples relative to x. Then, a feature extractor composed of a deep neural network is trained to extract features from the positive and negative samples. The training objective is to train a feature extractor as a mapping function f for x, which can map x to its feature f(x) such that:
[0040] D(f(x),f(x(+)))>>D(f(x),f(x(-) i ))
[0041] In the formula, x(-) i Let D be any negative sample, and D(·,·) be a function that measures the similarity between sample features. Therefore, the training objective function of the feature extractor can be written as:
[0042]
[0043] Since D(·,·) are numbers greater than zero, the above expression approaches 0 when x is more similar to x(+), and approaches 0 when x is more similar to x(-). i When the similarity is greater than 0, the training objective of the feature extractor is to optimize its parameters so that the above expression reaches its minimum value.
[0044] 3. Autonomous diagnosis of fault status throughout the entire life cycle of industrial robots
[0045] In the early stages of industrial robot service, since it is difficult to obtain effective fault label samples, the focus will be on collecting servo system data of industrial robots in healthy operation, and using this data to build a single-class incremental learning model.
[0046] use Figure 3 The incremental learning shown addresses the problem of a lack of new and new types of fault samples that occur throughout the life cycle of industrial robots. Incremental learning refers to the ability of a model to continuously learn and update itself based on existing models as new data is added, without the need to retrain the entire model.
[0047] All nodes constitute the node set V = {X1, X2, ..., X} n}, where element X i Let X be a random variable representing the state of the i-th node. The directed edges between nodes reflect the dependencies between variables. i The node it points to is its child node, and all nodes pointing to X i The node is its parent node, and their relationship can be represented as the conditional probability P[X]. i |Parent(X i )], i.e., X i The probability of X is affected by all its parent nodes. i If a node has no parent node, it is called the root node. According to Bayesian network theory, the overall state of an industrial robot is a joint probability distribution on the node set V, which can be represented as:
[0048]
[0049] The degradation process of each device is modeled using feature information derived from contrastive coding, thereby predicting its failure time distribution and providing decision support for the maintenance of industrial robot equipment.
[0050] Specific degradation prediction schemes include: Figure 4As shown in the figure, state "1" represents the initial health state of the device, state "F" represents the state of complete device failure, and other states in between represent intermediate states of device degradation (such as early degradation, moderate degradation, etc.). The process of device degradation towards different failure modes can be discretized into the same number of degradation states according to the degree of degradation.
[0051] Based on the aforementioned degradation states, discrete state-space models such as Markov chains are used to model the degradation state transition process of the component. Transition times between pairs of states are statistically analyzed on a long-term timescale, and the maximum likelihood estimation method is used to determine the state transition parameters in the degradation model. Since industrial robot equipment operates intermittently rather than continuously, the statistically analyzed times represent the actual working time of the equipment, excluding non-working time. In practical applications, the current degradation state of the equipment in the degradation model is first determined based on the comparative encoding characteristics of the latest multimodal servo information. Then, based on the established degradation model, the degradation process from the current degradation state to the final state "F" is repeatedly simulated using Monte Carlo simulation methods, and the degradation time is statistically analyzed to obtain the failure time distribution from the current degradation state to the failure state.
[0052] Typical six-axis serial industrial robots and three-axis parallel robots are selected to simulate various working conditions in actual industrial production, such as frequent start-stop, high-speed operation, and heavy-load operation. By connecting to the industrial robot servo system, multimodal servo information is collected. By setting different types of faults in the robot joints, motors and other parts, such as joint wear, motor winding short circuit, and poor gear meshing, real fault scenarios are simulated.
[0053] The proposed health perception and autonomous diagnosis methods were validated using the constructed experimental platform. The collected multimodal information was input into the constructed model to test the model's diagnostic accuracy and prediction precision. The performance of the model was evaluated by comparing the perception and diagnosis results of different models under the same fault scenario. Based on the experimental results, the perception model and diagnosis algorithm were optimized and adjusted, such as by adjusting the model structure and parameters, to improve the model performance.
[0054] It should be noted that, in this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.
[0055] This article uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only for the purpose of helping to understand the method and core ideas of the present invention. The above descriptions are only preferred embodiments of the present invention. It should be noted that due to the limitations of textual expression, while there are objectively infinite specific structures, those skilled in the art can make several improvements, modifications, or changes without departing from the principles of the present invention, and can also combine the above technical features in an appropriate manner. These improvements, modifications, changes, or combinations, or the direct application of the inventive concept and technical solution to other situations without modification, should all be considered within the scope of protection of the present invention.
Claims
1. A method for assessing the health status of industrial robots based on multimodal servo information mapping, characterized in that, Includes the following steps: Step 1: Obtain the impact of complex service conditions on the performance of industrial robots and the degradation mechanism; Step 2: Extract health perception features from the multimodal servo information collected during the operation of the industrial robot; Step 3: Based on the health perception features collected in Step 2, establish a health perception model for industrial robots under complex working conditions. Step 4: Establish an incremental learning model. Based on the model in Step 3, the model is continuously learned and updated as new data is added, thereby obtaining a degradation model and predicting its failure time distribution, providing decision support for the maintenance of industrial robot equipment.
2. The industrial robot health status assessment method based on multimodal servo information mapping according to claim 1, characterized in that, Step three specifically involves treating each data sample x and its transformed sample x(+) as a positive sample pair. x(+) is constructed based on x using methods such as adding noise, flipping, or random pruning. All other samples in the sample set, except for x, are negative samples relative to x. Then, a feature extractor composed of a deep neural network is trained to extract features from the positive and negative samples. The training objective is to train a feature extractor as a mapping function f for x, mapping x to its feature f(x), such that: D(f(x),f(x(+)))>>D(f(x),f(x(-) i )) In the formula, x(-) i Let D be any negative sample, and D(·,·) be a function that measures the similarity between sample features. Therefore, the training objective function of the feature extractor can be written as: Since D(·,·) are numbers greater than zero, the above expression approaches 0 when x is more similar to x(+), and approaches 0 when x is more similar to x(-). i When the similarity is greater than 0, the training objective of the feature extractor is to optimize its parameters so that the above expression reaches its minimum value.
3. The industrial robot health status assessment method based on multimodal servo information mapping according to claim 1, characterized in that, Step one involves: through literature review and analysis of actual cases, sorting out the operating data of industrial robots under frequent intermittent start-stop and frequent switching of operating states; using dynamics and kinematics theory, establishing motion models of the robot under different working conditions, and analyzing its impact on mechanical stress and fatigue damage of key components; and using fault tree analysis and failure mode and effects analysis methods to comprehensively sort out the basic failure modes of industrial robots, revealing the intrinsic physical mechanisms of performance degradation caused by material wear and electrical performance deterioration at the micro level.
4. The industrial robot health status assessment method based on multimodal servo information mapping according to claim 1, characterized in that, Step two involves comprehensively collecting and analyzing data such as current, voltage, position, speed, and torque in the servo system. Data mining algorithms are used to explore the potential connections between different fault modes and servo data. For harsh operating conditions, a multimodal state monitoring method is designed, integrating sensor fusion technology and adaptive filtering algorithms to improve the accuracy and reliability of data acquisition and establish a mapping relationship between fault characteristics and servo data.
5. The industrial robot health status assessment method based on multimodal servo information mapping according to claim 1, characterized in that, The incremental learning model is established as follows: All nodes constitute a node set V = {X1, X2, ..., X}. n }, where element X i Let X be a random variable representing the state of the i-th node. The directed edges between nodes reflect the dependencies between variables. i The node it points to is its child node, and all nodes pointing to X i The node is its parent node, and their relationship can be represented as the conditional probability P[X]. i |Parent(X i )], i.e., X i The probability of X is affected by all its parent nodes. i If there is no parent node, it is called the root node. The overall state of the industrial robot is the joint probability distribution on the node set V, which can be represented as: The degradation process of each device is modeled using feature information derived from contrastive coding, thereby predicting its failure time distribution and providing decision support for the maintenance of industrial robot equipment.
6. The industrial robot health status assessment method based on multimodal servo information mapping according to claim 5, characterized in that, The degradation state transition process of the component is modeled using a Markov chain discrete state-space model. The transition time between each pair of states is statistically analyzed on a long-term time scale. The state transition parameters in the degradation model are determined using the maximum likelihood estimation method. In practical applications, the current degradation state of the device in the degradation model is first determined based on the comparative coding features of the latest multimodal servo information. Then, based on the established degradation model, the degradation process from the current degradation state to the final state is repeatedly simulated using the Monte Carlo simulation method. The degradation time is statistically analyzed to obtain the failure time distribution of the device from the current degradation state to the failure state.
7. The industrial robot health status assessment method based on multimodal servo information mapping according to claim 5, characterized in that, Industrial robot equipment operates intermittently rather than continuously. The time counted is the actual working time of the equipment, excluding non-working time.
Citation Information
Cited By
Large-model-based motor running state monitoring method, device and equipment
CN121388799A
Mechanical arm teleoperation method and device, electronic equipment, storage medium and product
CN122100169A
Mechanical arm teleoperation method and device, electronic equipment, storage medium and product
CN122100169B
