Industrial robot predictive maintenance management method and system

Abstract

The present invention provides an industrial robot predictive maintenance management method and system, which relates to the technical field of equipment maintenance management, including: acquiring multimodal heterogeneous data of the industrial robot, performing intelligent preprocessing to obtain first preprocessed data, performing cross-modal correlation fusion on the first preprocessed data to generate fused data, extracting complementary redundant information and constructing a semantic feature map; constructing a health status prediction model, adding the semantic feature map to the health status prediction model, mapping it to an embedding space through linear transformation and extracting time series information, performing feature interaction to obtain interactive features, performing transformation and feature enhancement through a feedforward neural network layer to obtain a health status probability distribution; generating a comprehensive health status distribution through Bayesian reasoning fusion, combining a detection model to identify abnormal patterns and key turning points, generating an optimal maintenance strategy and performing collaborative maintenance on the industrial robot, uploading the maintenance results to a server and saving the current state.

Description

Technical Field

[0001] The present invention relates to the technical field of equipment maintenance management, and in particular to a predictive maintenance management method and system for an industrial robot. Background Art

[0002] Industrial robots are widely used in modern manufacturing and play a key role in automated production and flexible manufacturing. However, failures and unexpected downtime of industrial robots will lead to decreased production efficiency, reduced product quality and increased maintenance costs. Traditional industrial robot maintenance mainly relies on regular maintenance and post-repair, which makes it difficult to detect and prevent potential failures in time, resulting in untimely maintenance and inefficiency.

[0003] Traditional maintenance technologies lack comprehensive assessment and integrated modeling of the health status of industrial robots, making it difficult to achieve accurate health status assessment and fault prediction. In addition, they lack real-time status monitoring and data sharing mechanisms, making it difficult to achieve remote monitoring and collaborative management of industrial robots.

[0004] Therefore, a solution is urgently needed to solve the problems existing in the prior art. Summary of the invention

[0005] The embodiments of the present invention provide a method and system for predictive maintenance management of an industrial robot, which can at least solve some of the problems existing in the prior art.

[0006] A first aspect of an embodiment of the present invention provides an industrial robot predictive maintenance management method, comprising:

[0007] Acquire multimodal heterogeneous data of industrial robots in real time through a pre-set sensor network, perform intelligent preprocessing on the multimodal heterogeneous data, combine blockchain confirmation synchronization and generate adversarial network enhanced balancing operations to obtain first preprocessed data, perform cross-modal correlation fusion on the first preprocessed data through a multimodal graph convolutional neural network to generate fused data, extract complementary redundant information between the fused data and construct a semantic feature map corresponding to the industrial robot;

[0008] Based on the converter model, a health status prediction model corresponding to an industrial robot including multiple subsystems is constructed, and the semantic feature map is added as input to the health status prediction model. The input embedding layer maps the semantic feature map to the embedding space through linear transformation and extracts the time series information. The interactive features are obtained by combining the self-attention layer for feature interaction. The interactive features are transformed and enhanced through the feedforward neural network layer to obtain the health status probability distribution of each subsystem.

[0009] Based on the health status probability distribution, a comprehensive health status distribution is generated through Bayesian reasoning fusion, combined with an anomaly detection model based on active learning, combined with active exploration and uncertainty sampling to identify abnormal patterns and key turning points in the comprehensive health status distribution, based on the health status impact information, combined with a reinforcement learning algorithm to generate an optimal maintenance strategy and through game theory methods to perform collaborative maintenance on the industrial robot, the maintenance results are uploaded to the server in real time and the current status of the industrial robot is saved.

[0010] In an optional embodiment,

[0011] The multimodal heterogeneous data of the industrial robot is acquired in real time through a pre-set sensor network, the multimodal heterogeneous data is intelligently preprocessed, and the first preprocessed data is obtained by combining blockchain confirmation synchronization and generating adversarial network enhanced equilibrium operations, including:

[0012] Pre-deploy multiple types of sensors at key components and joints of the industrial robot to form a sensor network, and obtain multimodal heterogeneous data of the industrial robot in real time through the sensor network, wherein the multimodal heterogeneous data includes acceleration, vibration, temperature, current and position information;

[0013] Intelligent preprocessing is performed on the collected multimodal heterogeneous data. The original data is denoised and outliers are removed through wavelet transform operations. The heterogeneous data with different sampling rates and timestamps are mapped to the same time scale in combination with the dynamic time warping algorithm to obtain aligned data. The aligned data is normalized to generate standard data.

[0014] The standard data is enhanced and expanded in scale through a deep generative adversarial network, the data category distribution is balanced by combining an interpolation synthesis algorithm, and the data is encrypted through an asymmetric encryption algorithm. The distributed consensus mechanism based on blockchain is used to synchronize and trustfully share data on multiple terminals to obtain the first preprocessed data.

[0015] In an optional embodiment,

[0016] Performing cross-modal correlation fusion on the first preprocessed data through a multimodal graph convolutional neural network to generate fused data, extracting complementary redundant information between the fused data and constructing a semantic feature map corresponding to the industrial robot includes:

[0017] Mapping the first preprocessed data to a pre-initialized heterogeneous graph structure, wherein each sensor node is a node of the graph, the node attribute is the multi-dimensional time series feature vector of the current sensor, the edge is the association strength between sensors, the edge weight is the correlation and dependency between nodes, and the node feature and edge weight matrix are initialized;

[0018] Applying a multimodal graph convolutional neural network on the heterogeneous graph structure, extracting local features in each modality respectively through a cross-modal convolution kernel and a convolution kernel function corresponding to each modality, assigning importance weights to the local features in each modality in combination with an attention mechanism, performing cross-modal association fusion on the convolution results of multiple modalities according to the importance weights, generating a joint semantic representation of each node, stacking multiple layers of multimodal graph convolutional neural network layers and adaptively aggregating node features to generate cross-modal features;

[0019] For the cross-modal features, a correlation matrix between different modalities is calculated based on the Pearson correlation coefficient, the correlation matrix is ​​thresholded and a sparse correlation matrix is ​​generated, the sparse correlation matrix is ​​divided into communities in combination with a graph clustering algorithm, the cross-modal features are clustered into clusters, the features in each cluster represent a set of complementary and redundant semantic information, the features in each cluster are analyzed for importance, and a graph-based feature selection method is used to identify key complementary and redundant features, and the key complementary and redundant features are added to a pre-initialized redundant set;

[0020] Construct a semantic feature map, based on the redundant set, use the key complementary redundant features to fill the instance layer of the semantic feature map, clearly define the type hierarchy and attribute values, introduce the time dimension and establish semantic associations between instances, and obtain the semantic feature map.

[0021] In an optional embodiment,

[0022] Based on the converter model, a health status prediction model corresponding to an industrial robot containing multiple subsystems is constructed. The semantic feature map is added as input to the health status prediction model. The input embedding layer maps the semantic feature map to the embedding space through linear transformation and extracts the time series information. The interactive features are obtained by combining the self-attention layer for feature interaction. The interactive features are transformed and enhanced through the feedforward neural network layer to obtain the health status probability distribution of each subsystem, including:

[0023] Based on the converter model, an encoder-decoder structure is constructed and the semantic feature map corresponding to the industrial robot is added to the converter model. The nodes in the semantic feature map are mapped to low-dimensional dense vectors through node embedding technology and the graph structure is retained. The structure of the encoder and decoder is designed and the position encoding mechanism is introduced. The position embedding vector is added to the input embedding layer to obtain the health status prediction model corresponding to each subsystem.

[0024] Add the semantic feature map as input to the health status prediction model and map the nodes in the map to a low-dimensional embedding space through a linear transformation through an input embedding layer, aggregate neighbor node information in combination with a graph attention network and update the embedding representation of each node, add corresponding timestamp information to each node embedding and construct a time series embedding matrix, determine the time series information corresponding to the current node based on the time series embedding matrix, encode the semantic feature map embedded with the time series information and divide the map into multiple subgraphs, each subgraph corresponds to a subsystem of an industrial robot, perform feature extraction and feature interaction on each subgraph based on the attention mechanism, and obtain the interaction feature;

[0025] Based on the interaction features, the association patterns between different subsystems are extracted, and the interaction features are added to the health status prediction model. The health status of the next time step is recursively generated. Combined with the association patterns and the attention mechanism, the influence of different subsystems on the current prediction is dynamically adjusted. The prediction is repeated until the preset maximum number of iterations is reached to obtain the health status probability distribution corresponding to each subsystem.

[0026] In an optional embodiment,

[0027] Combined with the graph attention network, the neighbor node information is aggregated and the embedding representation of each node is updated as shown in the following formula:

[0028] ;

[0029] in, represents the updated hidden state of node i, h i Representation Node i The hidden state of K represents the number of attention heads, σ represents the activation function, Indicates k The attention head is at time step t Next, node i For neighbor nodes j The attention weight, W k Indicates k The weight matrix of the attention head, h j Representation Node j The hidden state of Indicates k The attention head is at time step t Next, node i The weight of attention to oneself, N ( i ) represents a node i The set of neighbor nodes.

[0030] In an optional embodiment,

[0031] Based on the health status probability distribution, a comprehensive health status distribution is generated through Bayesian reasoning fusion, combined with an anomaly detection model based on active learning, combined with active exploration and uncertainty sampling to identify abnormal patterns and key turning points in the comprehensive health status distribution, based on health status impact information, combined with a reinforcement learning algorithm to generate an optimal maintenance strategy and collaboratively maintain the industrial robot through a game theory method, and the maintenance results are uploaded to a server in real time and the current state of the industrial robot is saved, including:

[0032] The health status probability distribution of each subsystem is taken as an independent random variable, a joint probability distribution model is established, prior knowledge is introduced to construct the prior distribution probability, and based on Bayes' theorem, the posterior probability distribution corresponding to each subsystem is calculated by combining the prior distribution probability and the health status likelihood function of each subsystem, and the posterior probability distribution is used as the comprehensive health status distribution;

[0033] Based on the comprehensive health status distribution, an anomaly detection model based on active learning is constructed and the comprehensive health status distribution is modeled by an unsupervised learning method, samples with the largest amount of information are selected for annotation according to an active exploration strategy, the uncertainty of the current sample is evaluated by marginal probability, the sample with the highest uncertainty is repeatedly evaluated and selected, and a manual annotation label is added;

[0034] The anomaly detection model is updated according to the re-labeled samples, the decision boundary and model hyperparameters of the anomaly detection model are adaptively adjusted in combination with the incremental learning algorithm, the comprehensive health status distribution is detected according to the updated anomaly detection model, and the drift trend of the current health status distribution and the key turning point are determined in combination with the drift detection mechanism;

[0035] The failure risk, task importance, maintenance cost and remaining life span corresponding to the industrial robot are obtained and recorded as health status impact information. The health status impact information is used as the state space, and the maintenance operation is used as the action space. The state transition probability and reward function are defined, and the function value between the state and the action is updated through the reinforcement learning algorithm to generate the optimal maintenance strategy. The maintenance reward mechanism and cooperation strategy between each industrial robot are determined in combination with the game theory method. The optimization is repeated to obtain the global optimal maintenance decision and perform collaborative maintenance on the industrial robots. The state parameters of the industrial robots after maintenance are uploaded to the server and saved.

[0036] In an optional embodiment,

[0037] Based on Bayes' theorem, the posterior probability distribution corresponding to each subsystem is calculated by combining the prior probability and the health status likelihood function of each subsystem as shown in the following formula:

[0038] ;

[0039] in, S g Indicates g The health status of each subsystem, O represents the observed evidence of a composite health state, P ( O | S g ) indicates that g Subsystems are in a healthy state S g Evidence of comprehensive health status was observed under the condition O The likelihood of P ( S g | O ) indicates that the overall health status evidence O Under the conditions g Subsystems are in a healthy state S g The posterior probability of P ( S g ) indicates the g Subsystems are in a healthy state S g The prior probability of P ( O | S j ) Indicated in j Subsystems are in a healthy state S j Evidence of comprehensive health status was observed under the condition O The likelihood of P ( S j ) indicates the j Subsystems are in a healthy state S j The prior probability of n Indicates the total number of subsystem states.

[0040] A second aspect of an embodiment of the present invention provides an industrial robot predictive maintenance management system, comprising:

[0041] The first unit is used to obtain multimodal heterogeneous data of the industrial robot in real time through a preset sensor network, perform intelligent preprocessing on the multimodal heterogeneous data, combine blockchain confirmation synchronization and generate adversarial network enhanced equilibrium operations to obtain first preprocessed data, perform cross-modal association fusion on the first preprocessed data through a multimodal graph convolutional neural network to generate fused data, extract complementary redundant information between the fused data and construct a semantic feature map corresponding to the industrial robot;

[0042] The second unit is used to build a health status prediction model corresponding to an industrial robot including multiple subsystems based on a converter model, add the semantic feature map as input to the health status prediction model, map the semantic feature map to an embedding space through a linear transformation in an input embedding layer and extract temporal information, perform feature interaction in combination with a self-attention layer to obtain interactive features, transform and enhance the interactive features through a feedforward neural network layer, and obtain a health status probability distribution of each subsystem;

[0043] The third unit is used to generate a comprehensive health status distribution based on the health status probability distribution through Bayesian reasoning fusion, combine with the anomaly detection model based on active learning, combine with active exploration and uncertainty sampling to identify abnormal patterns and key turning points in the comprehensive health status distribution, generate the optimal maintenance strategy based on the health status impact information in combination with the reinforcement learning algorithm, and perform collaborative maintenance on the industrial robot through game theory methods, upload the maintenance results to the server in real time and save the current status of the industrial robot.

[0044] According to a third aspect of the embodiments of the present invention,

[0045] An electronic device is provided, comprising:

[0046] processor;

[0047] a memory for storing processor-executable instructions;

[0048] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0049] A fourth aspect of the embodiments of the present invention is:

[0050] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0051] In the present invention, a generative adversarial network is used for data enhancement and balancing operations, which effectively solves the problems of data sparsity and imbalance, and lays a good foundation for subsequent analysis and modeling. A multimodal graph convolutional neural network is used to perform cross-modal correlation fusion on the preprocessed data, fully mining the complementary and redundant information between different modal data, which can effectively capture the high-level semantic associations between modalities and generate a more comprehensive and compact fusion data representation. The Bayesian reasoning method is used for fusion to generate a comprehensive health status distribution, which can effectively combine prior knowledge and observation data to obtain a more reliable and comprehensive health status estimate. Reinforcement learning can learn the decision sequence that maximizes long-term benefits by continuously interacting with the environment, and can significantly reduce maintenance costs and improve the availability of robots. In summary, the present invention can significantly improve the health management level of industrial robots, maximize their service life, reduce unplanned downtime, thereby improving production efficiency and reducing operating costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 Schematic diagram of a process of predictive maintenance management method for industrial robots according to an embodiment of the present invention;

[0053] Figure 2 Schematic diagram of the structure of an industrial robot predictive maintenance management system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0056] Figure 1 FIG. 1 is a flow chart of a method for predictive maintenance management of an industrial robot according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0057] S1. Acquire multimodal heterogeneous data of industrial robots in real time through a pre-set sensor network, perform intelligent preprocessing on the multimodal heterogeneous data, combine blockchain confirmation synchronization and generate adversarial network enhanced balancing operations to obtain first preprocessed data, perform cross-modal correlation fusion on the first preprocessed data through a multimodal graph convolutional neural network to generate fused data, extract complementary redundant information between the fused data and construct a semantic feature map corresponding to the industrial robot;

[0058] The multimodal heterogeneous data refers to data from different sources with different formats and characteristics. The blockchain confirmation synchronization refers to the use of blockchain technology to manage and synchronize the ownership and access rights of data. The generative adversarial network enhanced balancing operation is a data enhancement technology. Synthetic data is generated by generative adversarial networks to expand the training data set and improve the generalization ability of the model. The multimodal graph convolutional neural network is a neural network model for processing multimodal data. By mapping data of different modalities to the same graph structure, cross-modal feature representations are extracted using graph convolution operations. The cross-modal association fusion refers to mining the association relationship between different modalities in multimodal data to achieve cross-modal information fusion. The complementary redundant information refers to the fact that in multimodal data, the information carried by different modalities may have certain complementarity and redundancy. The semantic feature map is a structured knowledge representation method that forms a map by constructing semantic associations between concepts such as equipment components, failure modes, and maintenance operations.

[0059] In an optional embodiment,

[0060] The multimodal heterogeneous data of the industrial robot is acquired in real time through a pre-set sensor network, the multimodal heterogeneous data is intelligently preprocessed, and the first preprocessed data is obtained by combining blockchain confirmation synchronization and generating adversarial network enhanced equilibrium operations, including:

[0061] Pre-deploy multiple types of sensors at key components and joints of the industrial robot to form a sensor network, and obtain multimodal heterogeneous data of the industrial robot in real time through the sensor network, wherein the multimodal heterogeneous data includes acceleration, vibration, temperature, current and position information;

[0062] Intelligent preprocessing is performed on the collected multimodal heterogeneous data. The original data is denoised and outliers are removed through wavelet transform operations. The heterogeneous data with different sampling rates and timestamps are mapped to the same time scale in combination with the dynamic time warping algorithm to obtain aligned data. The aligned data is normalized to generate standard data.

[0063] The standard data is enhanced and expanded in scale through a deep generative adversarial network, the data category distribution is balanced by combining an interpolation synthesis algorithm, and the data is encrypted through an asymmetric encryption algorithm. The distributed consensus mechanism based on blockchain is used to synchronize and trustfully share data on multiple terminals to obtain the first preprocessed data.

[0064] The sensor network is a distributed network system composed of a large number of sensor nodes, which is used to collect, process and transmit information from the physical world. The wavelet transform operation is a time-frequency analysis tool that can convert time domain signals into time-frequency domain representations and extract local features of signals. The dynamic time warping algorithm is a time series alignment method used to process time series data at different time scales or speeds. The interpolation synthesis algorithm is a data filling and reconstruction method used to process missing or sparse data. The asymmetric encryption algorithm is an encryption method that uses different keys for encryption and decryption operations and is often used to protect the confidentiality of data. The distributed consensus mechanism based on blockchain is a method for reaching consensus in a distributed system, which is implemented through blockchain technology.

[0065] Pre-deploy various types of sensors, such as acceleration sensors, vibration sensors, temperature sensors, current sensors, and position sensors, at key components and joints of industrial robots, such as reducers, motors, and bearings, and obtain multimodal heterogeneous data of industrial robots in real time through sensor networks, wherein the multimodal heterogeneous data includes acceleration, vibration, temperature, current, and position information;

[0066] Intelligent preprocessing of the collected multimodal heterogeneous data, using wavelet transform operation to reduce noise and remove outliers from the original data, using dynamic time warping algorithm to map heterogeneous data with different sampling rates and timestamps to the same time scale, normalizing the aligned data, scaling the data of different modes to the same numerical range, and eliminating the dimensionality effect;

[0067] Use deep generative adversarial networks (GANs) to enhance standard data, expand the scale of standard data, and build a GAN model, including a generator and a discriminator. Through adversarial training of the generator and the discriminator, the ability of the generator to generate realistic data is continuously improved, and the scale of the standard data set is expanded. Use interpolation synthesis algorithms to balance the categories of enhanced data. For fault categories with fewer samples, use interpolation algorithms to generate new synthetic samples in the feature space to increase the amount of data for sparse categories. For normal categories with more samples, reduce the number of samples by random undersampling to balance the data distribution of each category.

[0068] Use an asymmetric encryption algorithm to encrypt the balanced data to protect the confidentiality and integrity of the data, generate a public key and a private key pair, the public key is used to encrypt the data, and the private key is used to decrypt the data, use the public key to encrypt the data, and convert the plaintext data into ciphertext. Build a blockchain network, including multiple participating nodes, such as industrial robot manufacturers, equipment owners, maintenance service providers, etc., design and deploy smart contracts, specify data sharing rules and permissions, such as data ownership, access control, and scope of use, adopt appropriate consensus algorithms, such as proof of work (PoW), proof of stake (PoS), etc., reach data synchronization and consistency consensus among multiple nodes, upload the encrypted data to the blockchain network, synchronize data between nodes through the consensus mechanism, and obtain the first preprocessed data.

[0069] In this embodiment, a sensor network is formed by deploying multiple types of sensors at key components and joints of the industrial robot to collect multimodal heterogeneous data in real time, including acceleration, vibration, temperature, current and position information, so as to achieve comprehensive perception and monitoring of the robot's operating status. Wavelet transform is used to reduce noise and remove outliers from the original data, eliminate high-frequency noise and abnormal points in the data, and improve the signal-to-noise ratio and reliability of the data. The standard data is enhanced by using a deep generative adversarial network to expand the scale and diversity of the standard data. Data balancing is achieved by oversampling sparse categories and undersampling most categories, and the accuracy and reliability of fault diagnosis are improved. The distributed consensus mechanism based on blockchain realizes data synchronization and trusted sharing among multiple nodes. In summary, this embodiment realizes comprehensive perception, quality improvement, enhanced expansion, secure sharing and intelligent application of industrial robot operation data, provides key data support for predictive maintenance management of industrial robots, improves the timeliness, accuracy and efficiency of maintenance, and reduces the risk of unexpected downtime and maintenance costs.

[0070] In an optional embodiment,

[0071] Performing cross-modal correlation fusion on the first preprocessed data through a multimodal graph convolutional neural network to generate fused data, extracting complementary redundant information between the fused data and constructing a semantic feature map corresponding to the industrial robot includes:

[0072] Mapping the first preprocessed data to a pre-initialized heterogeneous graph structure, wherein each sensor node is a node of the graph, the node attribute is the multi-dimensional time series feature vector of the current sensor, the edge is the association strength between sensors, the edge weight is the correlation and dependency between nodes, and the node feature and edge weight matrix are initialized;

[0073] Applying a multimodal graph convolutional neural network on the heterogeneous graph structure, extracting local features in each modality respectively through a cross-modal convolution kernel and a convolution kernel function corresponding to each modality, assigning importance weights to the local features in each modality in combination with an attention mechanism, performing cross-modal association fusion on the convolution results of multiple modalities according to the importance weights, generating a joint semantic representation of each node, stacking multiple layers of multimodal graph convolutional neural network layers and adaptively aggregating node features to generate cross-modal features;

[0074] For the cross-modal features, a correlation matrix between different modalities is calculated based on the Pearson correlation coefficient, the correlation matrix is ​​thresholded and a sparse correlation matrix is ​​generated, the sparse correlation matrix is ​​divided into communities in combination with a graph clustering algorithm, the cross-modal features are clustered into clusters, the features in each cluster represent a set of complementary and redundant semantic information, the features in each cluster are analyzed for importance, and a graph-based feature selection method is used to identify key complementary and redundant features, and the key complementary and redundant features are added to a pre-initialized redundant set;

[0075] Construct a semantic feature map, based on the redundant set, use the key complementary redundant features to fill the instance layer of the semantic feature map, clearly define the type hierarchy and attribute values, introduce the time dimension and establish semantic associations between instances, and obtain the semantic feature map.

[0076] The multidimensional time series feature vector refers to a series of time series features extracted from multimodal heterogeneous data, and each time step corresponds to a multidimensional feature vector. The edge weight matrix refers to a matrix that represents the strength of the connection relationship between nodes in graph structure data. The cross-modal convolution kernel is a convolution kernel function used for multimodal data fusion, which can simultaneously process feature data from different modalities. The convolution kernel function is the core operation in the convolutional neural network and is used to extract local features. The sparse correlation matrix is ​​a sparse matrix that represents the correlation between variables, most of which are zero and only a few elements represent significant correlations. The key complementary redundant features refer to the complementarity and redundancy between different modal features in multimodal data. The instance layer refers to the level of specific instances in the knowledge graph. The community partitioning refers to the process of dividing nodes into different communities or modules in a complex network.

[0077] Map the first preprocessed data to a pre-initialized heterogeneous graph structure, with each sensor node as a node of the graph. The attribute of the node is the multi-dimensional time series feature vector of the current sensor, which contains the feature data collected by the sensor at different time steps. The edge in the graph represents the association strength between sensors, and the weight of the edge represents the correlation and dependency between nodes. Initialize the node feature matrix and edge weight matrix;

[0078] Apply multimodal graph convolutional neural networks on heterogeneous graph structures to extract and fuse features of multimodal data. For each modality, design a corresponding convolution kernel function to extract local features within the modality. In the convolution results of each modality, assign importance weights to local features through the attention mechanism. Based on the attention weights, perform weighted fusion on the convolution results of different modalities to generate a joint semantic representation of each node, and achieve effective integration of cross-modal information. Stack multiple layers of multimodal graph convolutional neural network layers, extract and fuse high-level semantic features layer by layer, and generate the final cross-modal features by adaptively aggregating the feature representation of each node to capture the deep association and complementary information between different modalities.

[0079] Based on the Pearson correlation coefficient, the correlation between different modal features is calculated to obtain a correlation matrix. The correlation matrix is ​​thresholded and the correlations below the threshold are set to zero to generate a sparse correlation matrix. Combined with graph clustering algorithms, such as spectral clustering and Louvain algorithm, the sparse correlation matrix is ​​divided into communities and cross-modal features are clustered into different clusters. The features in each cluster represent a set of complementary and redundant semantic information. The importance of each feature in the cluster is evaluated by a graph-based feature selection method. According to the importance score, several features with the highest scores are selected as key complementary redundant features. The selected key complementary redundant features are added to the pre-initialized redundant set to form a redundant set containing multimodal key features.

[0080] The instance layer of the semantic feature map is filled with key complementary redundant features in the redundant set. Each feature is an instance node, carrying attribute information such as the modality, time step, and feature value to which it belongs. The type hierarchy of the semantic feature map is clearly defined, such as sensor type, fault type, component type, etc., and inheritance and inclusion relationships between types are established. For each type, its attributes and value ranges are defined, such as the sensor range, sampling frequency, and severity of the fault. The attribute description of the instance node is improved, and the time dimension is introduced into the semantic feature map to represent the state changes of the instance node at different time steps. The instance nodes at different time steps are connected through time edges to form a temporal relationship. According to domain knowledge and expert experience, semantic association edges are established between instance nodes, such as causal relationships, correlations, synonymous relationships, etc., to enrich the semantic expression ability of the map and finally obtain a semantic feature map.

[0081] In this embodiment, by mapping multimodal sensor data into a heterogeneous graph structure, using nodes to represent sensors and edges to represent the associations between sensors, a graphical representation of multimodal data is achieved. Through cross-modal convolution kernels and modality-specific convolution kernel functions, local feature extraction and cross-modal feature fusion are achieved. By calculating the correlation matrix between cross-modal features and performing sparse processing and community division, the complementary and redundant relationships between different modalities are identified. The semantic feature graph forms a structured knowledge representation in the field of predictive maintenance of industrial robots, and provides semantic support for intelligent fault diagnosis and maintenance decisions. In summary, this embodiment achieves deep fusion and semantic enhancement of multimodal sensor data of industrial robots, constructs a knowledge graph for predictive maintenance, and provides important technical support for intelligent fault diagnosis, health assessment, and maintenance strategy optimization.

[0082] S2. Based on the converter model, a health status prediction model corresponding to an industrial robot including multiple subsystems is constructed, and the semantic feature map is added as input to the health status prediction model. The input embedding layer maps the semantic feature map to the embedding space through linear transformation and extracts the time series information. The interactive features are obtained by combining the self-attention layer for feature interaction. The interactive features are transformed and enhanced through the feedforward neural network layer to obtain the health status probability distribution of each subsystem.

[0083] The converter model is a deep learning model based on the self-attention mechanism, which is usually used in fields such as natural language processing and computer vision. The health status prediction model refers to a machine learning model used to predict the current and future health status of the robot in the predictive maintenance of industrial robots. The input embedding layer refers to a layer in the converter model that maps discrete input tokens to continuous vector representations. The embedding space refers to the continuous vector space to which the input embedding layer maps discrete tokens. The interaction feature refers to the interaction information between different positions calculated by the self-attention mechanism in the converter model.

[0084] In an optional embodiment,

[0085] Based on the converter model, a health status prediction model corresponding to an industrial robot containing multiple subsystems is constructed. The semantic feature map is added as input to the health status prediction model. The input embedding layer maps the semantic feature map to the embedding space through linear transformation and extracts the time series information. The interactive features are obtained by combining the self-attention layer for feature interaction. The interactive features are transformed and enhanced through the feedforward neural network layer to obtain the health status probability distribution of each subsystem, including:

[0086] Based on the converter model, an encoder-decoder structure is constructed and the semantic feature map corresponding to the industrial robot is added to the converter model. The nodes in the semantic feature map are mapped to low-dimensional dense vectors through node embedding technology and the graph structure is retained. The structure of the encoder and decoder is designed and the position encoding mechanism is introduced. The position embedding vector is added to the input embedding layer to obtain the health status prediction model corresponding to each subsystem.

[0087] Add the semantic feature map as input to the health status prediction model and map the nodes in the map to a low-dimensional embedding space through a linear transformation through an input embedding layer, aggregate neighbor node information in combination with a graph attention network and update the embedding representation of each node, add corresponding timestamp information to each node embedding and construct a time series embedding matrix, determine the time series information corresponding to the current node based on the time series embedding matrix, encode the semantic feature map embedded with the time series information and divide the map into multiple subgraphs, each subgraph corresponds to a subsystem of an industrial robot, perform feature extraction and feature interaction on each subgraph based on the attention mechanism, and obtain the interaction feature;

[0088] Based on the interaction features, the association patterns between different subsystems are extracted, and the interaction features are added to the health status prediction model. The health status of the next time step is recursively generated. Combined with the association patterns and the attention mechanism, the influence of different subsystems on the current prediction is dynamically adjusted. The prediction is repeated until the preset maximum number of iterations is reached to obtain the health status probability distribution corresponding to each subsystem.

[0089] The position encoding mechanism refers to assigning a unique position code to each position of the input sequence in the transformer model to introduce position information, and the position embedding vector refers to a unique vector representation generated for each position in the position encoding mechanism.

[0090] Based on the converter model, an encoder-decoder structure is constructed for the health status prediction of industrial robots. The encoder is responsible for encoding the input semantic feature map into hidden representation, and the decoder generates the health status prediction result based on the hidden representation. The semantic feature map corresponding to the industrial robot is added to the converter model, and the node information and edge information in the map are used to enhance the semantic representation ability of the model. Through the node embedding technology, the nodes in the semantic feature map are mapped to low-dimensional dense vectors while retaining the graph structure information. Each node is represented as a vector of fixed dimension, and each element in the vector represents the coordinate of the node in the embedding space.

[0091] A position encoding mechanism is introduced into the structure of the encoder and decoder to capture the position information in the input sequence. In the input embedding layer, in addition to mapping the node to the embedding vector, the position embedding vector is also added. The final input representation is obtained by adding or concatenating the position embedding vector with the node embedding vector. For each subsystem of the industrial robot, a corresponding health status prediction model is constructed. The model of each subsystem is based on a converter structure, including an encoder and a decoder. The encoder receives the semantic feature subgraph corresponding to the subsystem as input, and extracts features through the self-attention mechanism and feedforward neural network. The decoder generates the health status prediction for the next time step based on the output of the encoder. Through multiple iterative predictions, the health status sequence of the subsystem in the future is obtained.

[0092] The semantic feature map is input into the health status prediction model, and the map nodes are mapped to the low-dimensional embedding space through the input embedding layer. The neighbor information of each node is aggregated through the graph attention network, and the embedding representation of the node is updated. The corresponding timestamp information is added to the embedding representation of each node to construct a time series embedding matrix. Each row of the time series embedding matrix represents the embedding vector of a node at different time steps, and each column represents the embedding vector of different nodes at the same time step. The semantic feature map embedded with time series information is encoded, and then the map is divided into multiple subgraphs. Each subgraph corresponds to a subsystem of the industrial robot. Feature extraction and feature interaction are performed on each subgraph based on the attention mechanism. Through the attention mechanism, the correlation and interaction strength between different subgraphs are calculated to obtain the interaction features representing the association pattern between subsystems.

[0093] The extracted interaction features are added to the health status prediction model as additional input information. When predicting the health status of the next time step, in addition to using the node embedding and historical health status of the current time step, the interaction features are also combined to capture the association pattern between subsystems, and dynamically adjust the degree of influence of different subsystems on the current prediction, so that the prediction result can fully consider the interaction between subsystems and recursively generate the health status of the next time step until the preset maximum number of iterations is reached, and the health status probability distribution of each subsystem is obtained.

[0094] In this embodiment, the semantic feature graph corresponding to the industrial robot is added to the converter model, and the node information and edge information in the graph are used to enrich the semantic representation ability of the model. The node embedding technology maps the graph nodes into low-dimensional dense vectors, while retaining the graph structure information, so that the model can make full use of the structured knowledge of the graph, improve the quality of feature representation and the accuracy of prediction, introduce the position encoding mechanism, so that the model can capture the position information in the input sequence and distinguish the node representations of different time steps. The attention mechanism is used to extract features and interact with each subgraph to obtain interactive features representing the association pattern between subsystems. The dependency and influence mechanism between different subsystems are fully considered, cross-subsystem contextual information is provided, and the globality and correlation of health status prediction are enhanced. In summary, this embodiment makes full use of the structured knowledge of the graph and the evolution law of time series data, captures the association pattern between subsystems, provides future-oriented predictive maintenance decision support, and realizes accurate prediction and comprehensive evaluation of the health status of industrial robots.

[0095] In an optional embodiment,

[0096] Combined with the graph attention network, the neighbor node information is aggregated and the embedding representation of each node is updated as shown in the following formula:

[0097] ;

[0098] in, represents the updated hidden state of node i, h i Representation Node i The hidden state of K represents the number of attention heads, σ represents the activation function, Indicates k The attention head is at time step t Next, node i For neighbor nodes j The attention weight, W k Indicates k The weight matrix of the attention head, h j Representation Node j The hidden state of Indicates k The attention head is at time step t Next, node i The weight of attention to oneself, N ( i ) represents a node i The set of neighbor nodes.

[0099] In this embodiment, different importance weights are assigned to the neighbor nodes of each node through the attention mechanism, thereby realizing the selective aggregation of neighbor information. The adaptive weight allocation mechanism enables the graph attention network to flexibly adjust the contribution of neighbor nodes to the central node representation according to the semantic similarity and structural relationship of the nodes, thereby improving the semantic richness and distinctiveness of the node representation. By introducing multiple attention heads, the graph attention network can capture the diversified interaction patterns between nodes from different semantic subspaces, and can integrate the information of different semantic subspaces, thereby enhancing the robustness and expressiveness of the representation. By iteratively applying the graph attention network, the hidden state of the node can be propagated and updated in multiple steps in the graph structure. In summary, this embodiment makes full use of the local and global information of the graph structure, and captures the diversified interaction patterns between nodes through adaptive attention weight allocation and multi-head attention mechanism, which can effectively improve the accuracy, robustness and interpretability of the health status prediction of industrial robots, and provide strong support for intelligent operation and maintenance and predictive maintenance.

[0100] S3. Based on the health status probability distribution, a comprehensive health status distribution is generated through Bayesian reasoning fusion, combined with an anomaly detection model based on active learning, combined with active exploration and uncertainty sampling to identify abnormal patterns and key turning points in the comprehensive health status distribution, based on the health status impact information, combined with a reinforcement learning algorithm to generate the optimal maintenance strategy and through game theory methods to perform collaborative maintenance on the industrial robot, the maintenance results are uploaded to the server in real time and the current status of the industrial robot is saved.

[0101] The Bayesian reasoning is a reasoning method based on probability theory and statistics. By integrating prior knowledge and observation data, the probability distribution of unknown quantities is updated to obtain the posterior probability distribution. The active exploration is a method of collecting information, reducing uncertainty and optimizing decision-making by actively selecting behaviors. The uncertainty sampling is an active learning strategy. By selecting samples with the largest uncertainty in model predictions for annotation and learning, the generalization ability and robustness of the model are improved. The key turning point refers to an important time node in the evolution of the health status of an industrial robot, which is usually manifested as a sharp change or abnormal fluctuation in the state. The game theory is a mathematical theory that studies the strategic interaction and decision optimization between multiple participants. In the predictive maintenance of industrial robots, game theory methods can be applied to collaborative decision-making and optimization among multiple stakeholders. The collaborative maintenance refers to the completion of the predictive maintenance task of industrial robots by multiple subjects through information sharing, resource coordination and task division.

[0102] In an optional embodiment,

[0103] Based on the health status probability distribution, a comprehensive health status distribution is generated through Bayesian reasoning fusion, combined with an anomaly detection model based on active learning, combined with active exploration and uncertainty sampling to identify abnormal patterns and key turning points in the comprehensive health status distribution, based on health status impact information, combined with a reinforcement learning algorithm to generate an optimal maintenance strategy and collaboratively maintain the industrial robot through a game theory method, and the maintenance results are uploaded to a server in real time and the current state of the industrial robot is saved, including:

[0104] The health status probability distribution of each subsystem is taken as an independent random variable, a joint probability distribution model is established, prior knowledge is introduced to construct the prior distribution probability, and based on Bayes' theorem, the posterior probability distribution corresponding to each subsystem is calculated by combining the prior distribution probability and the health status likelihood function of each subsystem, and the posterior probability distribution is used as the comprehensive health status distribution;

[0105] Based on the comprehensive health status distribution, an anomaly detection model based on active learning is constructed and the comprehensive health status distribution is modeled by an unsupervised learning method, samples with the largest amount of information are selected for annotation according to an active exploration strategy, the uncertainty of the current sample is evaluated by marginal probability, the sample with the highest uncertainty is repeatedly evaluated and selected, and a manual annotation label is added;

[0106] The anomaly detection model is updated according to the re-labeled samples, the decision boundary and model hyperparameters of the anomaly detection model are adaptively adjusted in combination with the incremental learning algorithm, the comprehensive health status distribution is detected according to the updated anomaly detection model, and the drift trend of the current health status distribution and the key turning point are determined in combination with the drift detection mechanism;

[0107] The failure risk, task importance, maintenance cost and remaining life span corresponding to the industrial robot are obtained and recorded as health status impact information. The health status impact information is used as the state space, and the maintenance operation is used as the action space. The state transition probability and reward function are defined, and the function value between the state and the action is updated through the reinforcement learning algorithm to generate the optimal maintenance strategy. The maintenance reward mechanism and cooperation strategy between each industrial robot are determined in combination with the game theory method. The optimization is repeated to obtain the global optimal maintenance decision and perform collaborative maintenance on the industrial robots. The state parameters of the industrial robots after maintenance are uploaded to the server and saved.

[0108] The joint probability distribution model is used to describe the correlation and probability distribution between multiple random variables. The health state likelihood function represents the conditional probability distribution of observed data under a given health state. The unsupervised learning is a machine learning method that discovers hidden patterns and structures from unlabeled data. The marginal probability refers to the probability distribution of a variable in the joint probability distribution, without considering the values ​​of other variables. The incremental learning is a machine learning paradigm for continuous learning, which can continuously update the model when new data arrives without retraining the entire model. The decision boundary refers to the dividing line or hyperplane that divides different categories or states in the feature space. The drift detection refers to a mechanism for identifying changes in data distribution over time, which is used to promptly discover changes in the data generation process.

[0109] The industrial robot system is divided into multiple subsystems, such as robotic arms, sensors, controllers, etc. The health status of each subsystem is probabilistically modeled and regarded as an independent random variable. Prior knowledge, such as historical maintenance data and expert experience, is introduced to construct the prior probability distribution of the health status of each subsystem. Based on Bayes' theorem, the posterior probability distribution of the health status of each subsystem is calculated by combining the prior probability and the health status likelihood function of each subsystem. The posterior probability distribution of all subsystems is combined to obtain the comprehensive health status distribution of the entire industrial robot system.

[0110] Based on the pre-obtained comprehensive health status distribution, unsupervised learning methods such as Gaussian mixture model and hidden Markov model are used to model the comprehensive health status distribution. According to active exploration strategies such as uncertainty sampling and information gain, the most valuable samples for the anomaly detection model are selected, that is, the samples with the largest amount of information. The uncertainty of the currently selected samples is evaluated by calculating the marginal probability, and the samples with the highest uncertainty are selected. The selected samples are manually labeled to determine whether they are abnormal. The labeled samples are added to the training set, and the anomaly detection model is updated.

[0111] Adopt incremental learning algorithms, such as online learning and progressive learning, adjust the decision boundary and model hyperparameters of the anomaly detection model according to the newly labeled samples, use the updated anomaly detection model to detect the comprehensive health status distribution, identify potential abnormal states, introduce drift detection mechanisms, such as concept drift detection and covariance drift detection, monitor the change trend of the comprehensive health status distribution, and determine the drift trend and key turning points of the health status distribution based on the drift detection results;

[0112] Obtain the health status influencing information of the industrial robot, such as the failure risk, task importance, maintenance cost, and remaining life, as the state space of reinforcement learning. Use the optional maintenance operations, such as preventive maintenance and corrective maintenance, as the action space of reinforcement learning. Define the state transition probability and reward function, describe the impact of different maintenance operations on health status and cost, select the reinforcement learning algorithm, and update the value function between state and action through continuous interaction with the environment to generate the optimal maintenance strategy and determine the optimal maintenance operation to be taken under different health states.

[0113] For scenarios where multiple industrial robots work together, game theory methods, such as Nash equilibrium, are introduced to model the cooperative and competitive relationships between robots. The allocation plan and collaborative strategy of maintenance resources are determined based on the health status and maintenance needs of each robot. The maintenance reward mechanism and collaborative strategy of the multi-robot system are calculated through game theory algorithms, and the maintenance decisions of multiple robots are iteratively optimized. The global and local benefits are comprehensively considered to obtain the globally optimal maintenance strategy. Based on the generated optimal maintenance strategy, collaborative maintenance of industrial robots is performed, such as preventive replacement of parts and calibration of sensors. During the maintenance process, the robot's status parameters, such as running time, load, vibration, etc., are collected, and the robot's status parameters after maintenance are uploaded to the central server, and the robot's health status information is updated and saved.

[0114] In this embodiment, the global modeling method can more accurately characterize the overall health status of the robot, providing a reliable basis for subsequent anomaly detection and maintenance decisions. By intelligently selecting the most valuable samples for labeling, the learning efficiency and accuracy of the anomaly detection model are significantly improved. By continuously incorporating newly labeled samples and adjusting decision boundaries and model parameters, the anomaly detection model can always maintain high accuracy and timeliness. It comprehensively considers multiple factors such as failure risk, task importance, maintenance cost and remaining life. The reinforcement learning algorithm is used for solving, and the globally optimal maintenance decision can be obtained, which can significantly reduce maintenance costs and improve the availability and production efficiency of the robot. When multiple robots are collaboratively maintained, the game theory method is introduced to effectively coordinate the maintenance resource allocation and task collaboration between different robots to maximize the global benefits. In summary, this embodiment realizes the accurate assessment of the health status of the robot, the early warning of anomalies, the dynamic optimization of maintenance strategies and the collaborative management of multiple robots, which significantly improves the intelligence level of predictive maintenance and provides a strong guarantee for the reliable operation and efficient maintenance of industrial robots.

[0115] In an optional embodiment,

[0116] Based on Bayes' theorem, the posterior probability distribution corresponding to each subsystem is calculated by combining the prior probability and the health status likelihood function of each subsystem as shown in the following formula:

[0117] ;

[0118] in, S g Indicates g The health status of each subsystem, O represents the observed evidence of a composite health state, P ( O | S g ) indicates that g Subsystems are in a healthy state S g Evidence of comprehensive health status was observed under the condition O The likelihood of P ( S g | O ) indicates that the overall health status evidence O Under the conditions g Subsystems are in a healthy state S g The posterior probability of P ( S g ) indicates the g Subsystems are in a healthy state S g The prior probability of P ( O | S j ) Indicated in j Subsystems are in a healthy state S j Evidence of comprehensive health status was observed under the condition O The likelihood of P ( S j ) indicates the j Subsystems are in a healthy state S j The prior probability of n Indicates the total number of subsystem states.

[0119] In this embodiment, the Bayesian theorem can be used to combine the prior probability and the likelihood function, so that the posterior probability distribution not only takes into account the prior knowledge, but also makes full use of the observed data, thereby obtaining a more reliable and comprehensive health status estimation. The posterior probability distribution is used to quantify the uncertainty of the health status of each subsystem, providing richer information and supporting more flexible and precise decision-making. The recursive update method enables the health status estimation to learn and improve adaptively. In summary, this embodiment makes full use of prior knowledge and observed data, quantifies the uncertainty of the health status, supports probabilistic decision-making, and provides a solid foundation for the intelligent maintenance of industrial robots.

[0120] Figure 2 FIG. 1 is a schematic diagram of the structure of an industrial robot predictive maintenance management system according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0121] The first unit is used to obtain multimodal heterogeneous data of the industrial robot in real time through a preset sensor network, perform intelligent preprocessing on the multimodal heterogeneous data, combine blockchain confirmation synchronization and generate adversarial network enhanced equilibrium operations to obtain first preprocessed data, perform cross-modal association fusion on the first preprocessed data through a multimodal graph convolutional neural network to generate fused data, extract complementary redundant information between the fused data and construct a semantic feature map corresponding to the industrial robot;

[0122] The second unit is used to build a health status prediction model corresponding to an industrial robot including multiple subsystems based on a converter model, add the semantic feature map as input to the health status prediction model, map the semantic feature map to an embedding space through a linear transformation in an input embedding layer and extract temporal information, perform feature interaction in combination with a self-attention layer to obtain interactive features, transform and enhance the interactive features through a feedforward neural network layer, and obtain a health status probability distribution of each subsystem;

[0123] The third unit is used to generate a comprehensive health status distribution based on the health status probability distribution through Bayesian reasoning fusion, combine with the anomaly detection model based on active learning, combine with active exploration and uncertainty sampling to identify abnormal patterns and key turning points in the comprehensive health status distribution, generate the optimal maintenance strategy based on the health status impact information in combination with the reinforcement learning algorithm, and perform collaborative maintenance on the industrial robot through game theory methods, upload the maintenance results to the server in real time and save the current status of the industrial robot.

[0124] According to a third aspect of the embodiments of the present invention,

[0125] An electronic device is provided, comprising:

[0126] processor;

[0127] a memory for storing processor-executable instructions;

[0128] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0129] According to a fourth aspect of the embodiments of the present invention,

[0130] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0131] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A predictive maintenance management method for industrial robots, characterized in that: include: Acquire multimodal heterogeneous data of industrial robots in real time through a pre-set sensor network, perform intelligent preprocessing on the multimodal heterogeneous data, combine blockchain confirmation synchronization and generate adversarial network enhanced balancing operations to obtain first preprocessed data, perform cross-modal correlation fusion on the first preprocessed data through a multimodal graph convolutional neural network to generate fused data, extract complementary redundant information between the fused data and construct a semantic feature map corresponding to the industrial robot; Based on the converter model, a health status prediction model corresponding to an industrial robot including multiple subsystems is constructed, and the semantic feature map is added as input to the health status prediction model. The input embedding layer maps the semantic feature map to the embedding space through linear transformation and extracts the time series information. The interactive features are obtained by combining the self-attention layer for feature interaction. The interactive features are transformed and enhanced through the feedforward neural network layer to obtain the health status probability distribution of each subsystem. Based on the health status probability distribution, a comprehensive health status distribution is generated through Bayesian reasoning fusion, combined with an anomaly detection model based on active learning, combined with active exploration and uncertainty sampling to identify abnormal patterns and key turning points in the comprehensive health status distribution, based on the health status impact information, combined with a reinforcement learning algorithm to generate an optimal maintenance strategy and through game theory methods to perform collaborative maintenance on the industrial robot, the maintenance results are uploaded to the server in real time and the current status of the industrial robot is saved.

2. The method according to claim 1, characterized in that The multimodal heterogeneous data of the industrial robot is acquired in real time through a pre-set sensor network, the multimodal heterogeneous data is intelligently preprocessed, and the first preprocessed data is obtained by combining blockchain confirmation synchronization and generating adversarial network enhanced equilibrium operations, including: Pre-deploy multiple types of sensors at key components and joints of the industrial robot to form a sensor network, and obtain multimodal heterogeneous data of the industrial robot in real time through the sensor network, wherein the multimodal heterogeneous data includes acceleration, vibration, temperature, current and position information; Intelligent preprocessing is performed on the collected multimodal heterogeneous data. The original data is denoised and outliers are removed through wavelet transform operations. The heterogeneous data with different sampling rates and timestamps are mapped to the same time scale in combination with the dynamic time warping algorithm to obtain aligned data. The aligned data is normalized to generate standard data. The standard data is enhanced and expanded in scale through a deep generative adversarial network, the data category distribution is balanced by combining an interpolation synthesis algorithm, and the data is encrypted through an asymmetric encryption algorithm. The distributed consensus mechanism based on blockchain is used to synchronize and trustfully share data on multiple terminals to obtain the first preprocessed data.

3. The method according to claim 1, characterized in that Performing cross-modal correlation fusion on the first preprocessed data through a multimodal graph convolutional neural network to generate fused data, extracting complementary redundant information between the fused data and constructing a semantic feature map corresponding to the industrial robot includes: Mapping the first preprocessed data to a pre-initialized heterogeneous graph structure, wherein each sensor node is a node of the graph, the node attribute is the multi-dimensional time series feature vector of the current sensor, the edge is the association strength between sensors, the edge weight is the correlation and dependency between nodes, and the node feature and edge weight matrix are initialized; Applying a multimodal graph convolutional neural network on the heterogeneous graph structure, extracting local features in each modality respectively through a cross-modal convolution kernel and a convolution kernel function corresponding to each modality, assigning importance weights to the local features in each modality in combination with an attention mechanism, performing cross-modal association fusion on the convolution results of multiple modalities according to the importance weights, generating a joint semantic representation of each node, stacking multiple layers of multimodal graph convolutional neural network layers and adaptively aggregating node features to generate cross-modal features; For the cross-modal features, a correlation matrix between different modalities is calculated based on the Pearson correlation coefficient, the correlation matrix is ​​thresholded and a sparse correlation matrix is ​​generated, the sparse correlation matrix is ​​divided into communities in combination with a graph clustering algorithm, the cross-modal features are clustered into clusters, the features in each cluster represent a set of complementary and redundant semantic information, the features in each cluster are analyzed for importance, and a graph-based feature selection method is used to identify key complementary and redundant features, and the key complementary and redundant features are added to a pre-initialized redundant set; Construct a semantic feature map, based on the redundant set, use the key complementary redundant features to fill the instance layer of the semantic feature map, clearly define the type hierarchy and attribute values, introduce the time dimension and establish semantic associations between instances, and obtain the semantic feature map.

4. The method according to claim 1, characterized in that Based on the converter model, a health status prediction model corresponding to an industrial robot containing multiple subsystems is constructed. The semantic feature map is added as input to the health status prediction model. The input embedding layer maps the semantic feature map to the embedding space through linear transformation and extracts the time series information. The interactive features are obtained by combining the self-attention layer for feature interaction. The interactive features are transformed and enhanced through the feedforward neural network layer to obtain the health status probability distribution of each subsystem, including: Based on the converter model, an encoder-decoder structure is constructed and the semantic feature map corresponding to the industrial robot is added to the converter model. The nodes in the semantic feature map are mapped to low-dimensional dense vectors through node embedding technology and the graph structure is retained. The structure of the encoder and decoder is designed and the position encoding mechanism is introduced. The position embedding vector is added to the input embedding layer to obtain the health status prediction model corresponding to each subsystem. Add the semantic feature map as input to the health status prediction model and map the nodes in the map to a low-dimensional embedding space through a linear transformation through an input embedding layer, aggregate neighbor node information in combination with a graph attention network and update the embedding representation of each node, add corresponding timestamp information to each node embedding and construct a time series embedding matrix, determine the time series information corresponding to the current node based on the time series embedding matrix, encode the semantic feature map embedded with the time series information and divide the map into multiple subgraphs, each subgraph corresponds to a subsystem of an industrial robot, perform feature extraction and feature interaction on each subgraph based on the attention mechanism, and obtain the interaction feature; Based on the interaction features, the association patterns between different subsystems are extracted, and the interaction features are added to the health status prediction model. The health status of the next time step is recursively generated. Combined with the association patterns and the attention mechanism, the influence of different subsystems on the current prediction is dynamically adjusted. The prediction is repeated until the preset maximum number of iterations is reached to obtain the health status probability distribution corresponding to each subsystem.

5. The method according to claim 4, characterized in that Combined with the graph attention network, the neighbor node information is aggregated and the embedding representation of each node is updated as shown in the following formula: ； in, Represents the updated node i The hidden state of h i Representation Node i The hidden state of K represents the number of attention heads, σ represents the activation function, Indicates k The attention head is at time step t Next, node i For neighbor nodes j The attention weight, W k Indicates k The weight matrix of the attention head, h j Representation Node j The hidden state of Indicates k The attention head is at time step t Next, node i The weight of attention to oneself, N ( i ) represents a node i The set of neighbor nodes.

6. The method according to claim 1, characterized in that Based on the health status probability distribution, a comprehensive health status distribution is generated through Bayesian reasoning fusion, combined with an anomaly detection model based on active learning, combined with active exploration and uncertainty sampling to identify abnormal patterns and key turning points in the comprehensive health status distribution, based on health status impact information, combined with a reinforcement learning algorithm to generate an optimal maintenance strategy and collaboratively maintain the industrial robot through a game theory method, and the maintenance results are uploaded to a server in real time and the current state of the industrial robot is saved, including: The health status probability distribution of each subsystem is taken as an independent random variable, a joint probability distribution model is established, prior knowledge is introduced to construct the prior distribution probability, and based on Bayes' theorem, the posterior probability distribution corresponding to each subsystem is calculated by combining the prior distribution probability and the health status likelihood function of each subsystem, and the posterior probability distribution is used as the comprehensive health status distribution; Based on the comprehensive health status distribution, an anomaly detection model based on active learning is constructed and the comprehensive health status distribution is modeled by an unsupervised learning method, samples with the largest amount of information are selected for annotation according to an active exploration strategy, the uncertainty of the current sample is evaluated by marginal probability, the sample with the highest uncertainty is repeatedly evaluated and selected, and a manual annotation label is added; The anomaly detection model is updated according to the re-labeled samples, the decision boundary and model hyperparameters of the anomaly detection model are adaptively adjusted in combination with the incremental learning algorithm, the comprehensive health status distribution is detected according to the updated anomaly detection model, and the drift trend of the current health status distribution and the key turning point are determined in combination with the drift detection mechanism; The failure risk, task importance, maintenance cost and remaining life span corresponding to the industrial robot are obtained and recorded as health status impact information. The health status impact information is used as the state space, and the maintenance operation is used as the action space. The state transition probability and reward function are defined, and the function value between the state and the action is updated through the reinforcement learning algorithm to generate the optimal maintenance strategy. The maintenance reward mechanism and cooperation strategy between each industrial robot are determined in combination with the game theory method. The optimization is repeated to obtain the global optimal maintenance decision and perform collaborative maintenance on the industrial robots. The state parameters of the industrial robots after maintenance are uploaded to the server and saved.

7. The method according to claim 6, characterized in that Based on Bayes' theorem, the posterior probability distribution corresponding to each subsystem is calculated by combining the prior probability and the health status likelihood function of each subsystem as shown in the following formula: ； in, S g Indicates g The health status of each subsystem, O represents the observed evidence of a composite health state, P ( O | S g ) indicates that g Subsystems are in a healthy state S g Evidence of comprehensive health status was observed under the condition O The likelihood of P ( S g | O ) indicates that the overall health status evidence O Under the conditions g Subsystems are in a healthy state S g The posterior probability of P ( S g ) indicates the g Subsystems are in a healthy state S g The prior probability of P ( O | S j ) Indicated in j Subsystems are in a healthy state S j Evidence of comprehensive health status was observed under the condition O The likelihood of P ( S j ) indicates the j Subsystems are in a healthy state S j The prior probability of n Indicates the total number of subsystem states.

8. An industrial robot predictive maintenance management system, used to implement the method according to any one of claims 1 to 7, characterized in that: include: The first unit is used to obtain multimodal heterogeneous data of the industrial robot in real time through a preset sensor network, perform intelligent preprocessing on the multimodal heterogeneous data, combine blockchain confirmation synchronization and generate adversarial network enhanced equilibrium operations to obtain first preprocessed data, perform cross-modal association fusion on the first preprocessed data through a multimodal graph convolutional neural network to generate fused data, extract complementary redundant information between the fused data and construct a semantic feature map corresponding to the industrial robot; The second unit is used to build a health status prediction model corresponding to an industrial robot including multiple subsystems based on a converter model, add the semantic feature map as input to the health status prediction model, map the semantic feature map to an embedding space through a linear transformation in an input embedding layer and extract temporal information, perform feature interaction in combination with a self-attention layer to obtain interactive features, transform and enhance the interactive features through a feedforward neural network layer, and obtain a health status probability distribution of each subsystem; The third unit is used to generate a comprehensive health status distribution based on the health status probability distribution through Bayesian reasoning fusion, combine with the anomaly detection model based on active learning, combine with active exploration and uncertainty sampling to identify abnormal patterns and key turning points in the comprehensive health status distribution, generate the optimal maintenance strategy based on the health status impact information in combination with the reinforcement learning algorithm, and perform collaborative maintenance on the industrial robot through game theory methods, upload the maintenance results to the server in real time and save the current status of the industrial robot.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images