Industrial robot safety monitoring method and system

Abstract

The present invention provides an industrial robot safety monitoring method and system, which relates to the field of safety monitoring technology, including arranging a multimodal sensor array to collect robot status information; performing empirical mode decomposition on vibration signals to obtain intrinsic mode functions reflecting the degree of wear of the robot transmission system; performing wavelet transform on temperature signals to obtain a temperature feature set reflecting the heat dissipation of the robot; semantically representing and linking a variety of knowledge in robot components, performance parameters, degradation modes, and maintenance measures through knowledge extraction, knowledge representation, and knowledge fusion technologies to construct a management knowledge graph corresponding to the robot; performing reasoning mining on the management knowledge graph based on a rule mining algorithm for path sorting, extracting degradation features corresponding to various components of the robot in the management knowledge graph by mining reasoning paths composed of frequently co-occurring entities and relationships, and predicting the health status trend of the robot in combination with a pre-constructed time series prediction model.

Description

Technical Field

[0001] The present invention relates to safety monitoring technologies, and in particular to an industrial robot safety monitoring method and system. Background Art

[0002] With the rapid development and wide application of robot technologies, the issues of robot health management and maintenance have increasingly attracted attention. During the long-term operation of a robot, its key components and systems are vulnerable to wear, fatigue, environmental factors, etc., resulting in performance degradation and an increased risk of failures. To ensure the reliable operation of the robot and extend its service life, it is urgent to develop effective robot health condition assessment and remaining life prediction technologies.

[0003] Traditional robot health management mainly relies on regular manual inspections and maintenance. By visually inspecting the robot and conducting performance tests, etc., its health condition is evaluated. However, this method is difficult to detect potential fault hazards in a timely manner and is time-consuming and labor-intensive. In recent years, some scholars have proposed using data-driven methods for robot health management. These methods collect various types of sensor data during the operation of the robot, such as current, vibration, temperature, etc., and establish data models to achieve the monitoring and prediction of the robot's health condition.

[0004] Although the existing data-driven methods have made certain progress, there are still the following deficiencies: First, most methods only utilize single-type sensor data and lack the comprehensive utilization of multi-source heterogeneous information; second, the existing methods mainly target individual key components or subsystems and lack the assessment of the overall health condition of the robot; third, the research on the prediction of the remaining life of the robot is relatively less, and the prediction accuracy needs to be improved. Summary of the Invention

[0005] Embodiments of the present invention provide an industrial robot safety monitoring method and system, which can solve the problems in the prior art.

[0006] In the first aspect of the embodiments of the present invention,

[0007] An industrial robot safety monitoring method is provided, including:

[0008] Arranging a multi-modal sensor array to collect the state information of the robot, where the state information includes vibration signals and temperature signals, the multi-modal sensor array includes vibration sensors and temperature sensors, the vibration sensors are arranged at the joints and actuators of the robot, and the temperature sensors are arranged at the heat dissipation components of the robot; performing empirical mode decomposition on the vibration signals to obtain the intrinsic mode functions reflecting the wear degree of the robot's transmission system; performing wavelet transform on the temperature signals to obtain the temperature feature set reflecting the heat dissipation situation of the robot;

[0009] Based on the intrinsic mode functions and the temperature feature set, combined with the attribute information corresponding to the robot, through knowledge extraction, knowledge representation, and knowledge fusion technologies, semantic representation and linking of various types of knowledge in robot components, performance parameters, degradation modes, and maintenance measures are carried out to construct a management knowledge graph corresponding to the robot;

[0010] Based on the rule mining algorithm of path ranking, the management knowledge graph is inferred and mined. By mining the inference paths composed of frequently co-occurring entities and relationships, the degradation features corresponding to each component of the robot in the management knowledge graph are extracted, and combined with the pre-constructed time series prediction model, the health state trend of the robot is predicted.

[0011] In an alternative embodiment,

[0012] Performing empirical mode decomposition on the vibration signal, the intrinsic mode functions reflecting the wear degree of the robot transmission system include:

[0013] Performing empirical mode decomposition on the vibration signal of the robot transmission system, adaptively decomposing the vibration signal into a series of intrinsic mode functions, and extracting sensitive intrinsic mode function components reflecting the inherent oscillation mode of the transmission system; the empirical mode decomposition constructs local envelope lines through cyclic screening, calculates the local envelope mean as a candidate for the intrinsic mode function, and based on the extreme points and zero-crossing points, judges the conditions of the intrinsic mode function, and recursively extracts the intrinsic mode functions;

[0014] Evaluating the correlation between the extracted intrinsic mode functions and the wear degree of the robot transmission system based on the mutual information criterion, by calculating the joint probability distribution and marginal probability distribution of the intrinsic mode functions and the wear degree of the robot transmission system, and selecting the top k intrinsic mode functions with the largest mutual information value as the sensitive components of wear degradation.

[0015] In an alternative embodiment,

[0016] Performing wavelet transform on the temperature signal, the temperature feature set reflecting the heat dissipation situation of the robot includes:

[0017] Performing wavelet transform on the temperature signal, by performing multi-scale and multi-resolution decomposition on the temperature signal, obtaining low-frequency approximation coefficients and high-frequency detail coefficients;

[0018] Extracting temperature degradation features from the high-frequency detail coefficients based on the wavelet energy spectrum and wavelet singular value spectrum, calculating the energy values of the high-frequency detail coefficients of each layer, and constructing high-frequency component energy features reflecting the local temperature rise degree; performing singular value decomposition on the high-frequency detail coefficients of each layer, and extracting singular value features reflecting the temperature mutation situation;

[0019] Construct a temperature feature set characterizing the heat dissipation of the robot based on the high-frequency component energy feature and the singular value feature.

[0020] In an alternative embodiment,

[0021] Based on the intrinsic mode function and the temperature feature set, combined with the attribute information corresponding to the robot, through knowledge extraction, knowledge representation, and knowledge fusion technologies, semantically represent and link various types of knowledge in robot components, performance parameters, degradation modes, and maintenance measures, and construct a management knowledge graph corresponding to the robot, including:

[0022] The knowledge extraction uses a named entity recognition model based on BERT and a relation extraction model based on BiLSTM-CRF, and constructs a dictionary and a relation template library;

[0023] The knowledge representation uses a knowledge representation learning model based on TransE to map concepts and relations in the ontology to a low-dimensional continuous vector space;

[0024] The knowledge fusion uses a knowledge graph linking method based on ontology matching to establish a link mapping relationship by calculating the semantic similarity between ontologies.

[0025] In an alternative embodiment,

[0026] Based on the rule mining algorithm of path ranking, perform reasoning and mining on the management knowledge graph. By mining the reasoning paths composed of frequently co-occurring entities and relations, extract the degradation features corresponding to each component of the robot in the management knowledge graph, including:

[0027] For the management knowledge graph, adopt a path sampling strategy based on random walk to generate candidate reasoning paths. Randomly select an initial entity node from the management knowledge graph, and randomly select the next-hop entity node with uniform probability according to the out-edge relationship of the current entity node. Repeat this process until the preset path length is reached. Cover different regions and semantic associations of the management knowledge graph through multiple random walks. The candidate reasoning paths include a first reasoning path depicting the association between robot components and corresponding degradation features and a second reasoning path depicting the association between degradation features and potential degradation modes;

[0028] Sort and filter the candidate inference paths, and score the candidate inference paths using a path ranking criterion that combines support and confidence. The comprehensive support measures the frequency of occurrence of the candidate inference path in the management knowledge graph; the confidence measures the strength of the association between the front and back entities in the candidate inference path. Calculate the ranking score based on the comprehensive support and confidence of the candidate inference path, sort the candidate inference paths from high to low according to the score, and select the top K candidate inference paths with the highest scores as frequent paths.

[0029] Generate the degradation features corresponding to each component according to the entities and relationship types included in the frequent paths.

[0030] In an alternative embodiment,

[0031] Sorting and filtering the candidate inference paths, and scoring the candidate inference paths using a path ranking criterion that combines support and confidence includes:

[0032] ;

[0033] Among them, sup(P) represents the comprehensive support corresponding to path P, count(P) represents the number of times path P appears in the set of candidate inference paths, N represents the number of candidate inference paths, and λ represents a length penalty factor used to control the penalty strength for the path length, len(P) represents the length of path P;

[0034] ;

[0035] Among them, conf(P) represents the confidence corresponding to path P, count(e i , r i , e i+1 ) represents entity e i through relationship r i connect entity e i+1 frequency, count(e i ) represents entity e i frequency of occurrence in the candidate inference path, w(e i ) represents entity e i weight coefficient,n Represents the number of entities.

[0036] In an alternative embodiment,

[0037] Combined with a pre - constructed time - series prediction model, predicting the health state trend of the robot includes:

[0038] Using a long short - term memory network (LSTM) as a basic building block, constructing a deep LSTM network by stacking multiple LSTM layers as the time - series prediction model, adding a fully - connected layer and an activation function at the end of the time - series prediction model, and outputting the predicted health state trend based on the degradation characteristics corresponding to each component of the robot in the management knowledge graph.

[0039] In the second aspect of the embodiments of the present application,

[0040] There is provided an industrial robot safety monitoring system for implementing the method described in any one of the preceding claims 1 - 7, characterized in that it includes:

[0041] A first unit for arranging a multi - modal sensor array to collect the state information of the robot, the state information including vibration signals and temperature signals, the multi - modal sensor array including vibration sensors and temperature sensors, the vibration sensors being arranged at the joints and actuators of the robot, and the temperature sensors being arranged at the heat - dissipation components of the robot; performing empirical mode decomposition on the vibration signals to obtain the intrinsic mode functions reflecting the wear degree of the robot's transmission system; performing wavelet transform on the temperature signals to obtain a temperature feature set reflecting the heat - dissipation situation of the robot;

[0042] A second unit for, based on the intrinsic mode functions and the temperature feature set, combining the attribute information corresponding to the robot, semantically representing and linking various knowledge among robot components, performance parameters, degradation modes, and maintenance measures through knowledge extraction, knowledge representation, and knowledge fusion technologies, and constructing a management knowledge graph corresponding to the robot;

[0043] A third unit for performing reasoning and mining on the management knowledge graph based on a path - ranking rule - mining algorithm, extracting the degradation characteristics corresponding to each component of the robot in the management knowledge graph by mining the reasoning paths composed of frequently co - occurring entities and relationships, and combining a pre - constructed time - series prediction model to predict the health state trend of the robot.

[0044] In the third aspect of the embodiments of the present invention,

[0045] There is provided an electronic device, including:

[0046] A processor;

[0047] A memory for storing processor - executable instructions;

[0048] Among them, the processor is configured to call the instructions stored in the memory to execute the method described above.

[0049] In the fourth aspect of the embodiments of the present invention,

[0050] A computer-readable storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method described above is implemented.

[0051] By arranging a multi-modal sensor array to collect the vibration signal and temperature signal of the robot, a comprehensive perception of the operating state of the robot is realized. The vibration signal and temperature signal respectively reflect the wear degree and heat dissipation condition of the robot's transmission system, providing a rich data basis for subsequent health condition assessment.

[0052] Empirical mode decomposition is used to process the vibration signal, and the intrinsic mode function reflecting the wear degree of the robot's transmission system is obtained, effectively extracting the degradation characteristics in the vibration signal. At the same time, wavelet transform is used to process the temperature signal, and a temperature feature set reflecting the heat dissipation condition of the robot is obtained, revealing the relationship between temperature change and health condition. The knowledge graph technology is introduced to semantically represent and link various kinds of knowledge such as robot components, performance parameters, degradation modes, and maintenance measures, constructing a robot management knowledge graph. The construction of the knowledge graph realizes the structuring, visualization, and intelligentization of knowledge in the field of robot health management, providing comprehensive knowledge support for health condition assessment.

[0053] A rule mining algorithm based on path ranking is proposed. By mining the inference paths composed of frequently co-occurring entities and relationships in the management knowledge graph, the degradation characteristics corresponding to each component of the robot are automatically extracted. This method overcomes the limitations of traditional methods that rely on expert experience and manual feature construction, realizing the automatic discovery and extraction of degradation characteristics. Combining with the pre-constructed time series prediction model, the prediction of the trend of the robot's health state is realized. By mining the time series patterns in the historical operation data and constructing a time series prediction model of the health state, the change trend of the robot's health state in a future period of time can be dynamically predicted, providing a basis for early warning and preventive maintenance.

[0054] The present invention comprehensively utilizes multi-modal sensor information, signal processing technology, knowledge graph technology, and machine learning technology to realize the comprehensive assessment of the robot's health condition and the prediction of the remaining life. Compared with traditional methods, the present invention can more comprehensively and accurately evaluate the health condition of the robot, discover potential failure risks in advance, optimize the maintenance strategy, extend the service life of the robot, reduce the maintenance cost, and improve the reliability and availability of the robot. Brief Description of the Drawings

[0055] Figure 1 It is a schematic flowchart of the industrial robot safety monitoring method according to an embodiment of the present invention;

[0056] Figure 2 It is a schematic structural diagram of the industrial robot safety monitoring system according to an embodiment of the present invention. Specific embodiments

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0058] The technical solutions of the present invention will be described in detail below with specific embodiments. These specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0059] Figure 1 It is a schematic flowchart of the industrial robot safety monitoring method according to an embodiment of the present invention, as Figure 1 shown, the method includes:

[0060] S101. Arrange a multi-modal sensor array to collect the state information of the robot. The state information includes vibration signals and temperature signals. The multi-modal sensor array includes vibration sensors and temperature sensors. The vibration sensors are arranged at the joints and actuators of the robot, and the temperature sensors are arranged at the heat dissipation components of the robot; perform empirical mode decomposition on the vibration signals to obtain the intrinsic mode functions reflecting the wear degree of the robot's transmission system; perform wavelet transform on the temperature signals to obtain the temperature feature set reflecting the heat dissipation situation of the robot;

[0061] In an alternative embodiment,

[0062] Performing empirical mode decomposition on the vibration signals to obtain the intrinsic mode functions reflecting the wear degree of the robot's transmission system includes:

[0063] Perform empirical mode decomposition on the vibration signals of the robot's transmission system. The vibration signals are adaptively decomposed into a series of intrinsic mode functions, and the sensitive intrinsic mode function components reflecting the inherent oscillation mode of the transmission system are extracted; the empirical mode decomposition constructs local envelope lines through cyclic screening, calculates the local envelope mean as the candidate of the intrinsic mode function, and judges the conditions of the intrinsic mode function based on the extreme points and zero-crossing points, and recursively extracts the intrinsic mode functions;

[0064] Evaluate the correlation between the extracted intrinsic mode functions and the wear degree of the robot transmission system based on the mutual information criterion. By calculating the joint probability distribution and marginal probability distribution of the intrinsic mode functions and the wear degree of the robot transmission system, select the top k intrinsic mode functions with the largest mutual information values as the sensitive components of wear degradation.

[0065] Exemplarily, in order to extract sensitive features reflecting the wear degree from the vibration signals of the robot transmission system, the present invention uses the Empirical Mode Decomposition (EMD) method to adaptively decompose the vibration signals, obtaining a series of intrinsic mode functions (IMFs) reflecting the inherent oscillation modes of the transmission system, and selects sensitive IMF components highly correlated with wear degradation based on the mutual information criterion. The specific technical solution is as follows:

[0066] Let the vibration signal collected by the robot transmission system be x(t), and the process of performing EMD decomposition on it is as follows:

[0067] Identify all local extreme points of the signal x(t), including maximum points and minimum points.

[0068] Through cubic spline interpolation, connect all maximum points and minimum points respectively to construct the upper envelope u(t) and the lower envelope l(t) . Calculate the mean value m(t) of the upper and lower envelopes. Subtract the mean value from the original signal to obtain the residual signal h(t). Determine whether h(t) satisfies the IMF conditions:

[0069] Within the entire data segment, the number of extreme points is equal to or at most differs by one from the number of zero-crossing points; at any moment, the mean value of the upper and lower envelopes is zero.

[0070] If the IMF conditions are satisfied, then take it as an IMF component, denoted as c i (t), and take the residual signal r(t) = x(t) - c i (t) as the new input signal to extract the next IMF component; if the IMF conditions are not satisfied, then take h(t) as the new input signal and continue the screening until the residual signal becomes a monotonic function or the number of its extreme points is less than a certain preset threshold, and the EMD decomposition process ends.

[0071] The IMF components obtained through EMD decomposition reflect the inherent oscillation modes of the robot transmission system at different scales, but not all IMF components are highly correlated with wear degradation. To select sensitive IMF components, the present invention introduces the mutual information criterion to evaluate the correlation between each IMF component and the wear degree. Let the i-th IMF component be c i(t), the wear degree index of the robot drive system is w(t), and the mutual information between the two is defined as:

[0072] ;

[0073] where p(c i , w) is the joint probability density function of c i and w, and p(c i ) and p(w) are the marginal probability density functions of c i and w respectively. The mutual information I(c i ; w) measures the correlation between c i and w. The larger the mutual information value, the stronger the correlation between the two.

[0074] Calculate the mutual information value between each IMF component and the wear degree index, sort the IMF components in descending order according to the mutual information value, and select the top k IMF components with the largest mutual information value as the sensitive components of wear degradation.

[0075] Through the above steps, the adaptive decomposition of the vibration signal of the robot drive system and the selection of sensitive IMF components are realized. The selected sensitive IMF components can effectively reflect the wear degradation characteristics of the drive system, laying a foundation for subsequent health condition assessment and remaining life prediction.

[0076] In an optional implementation manner,

[0077] Perform wavelet transform on the temperature signal to obtain a temperature feature set reflecting the heat dissipation of the robot, including:

[0078] Perform wavelet transform on the temperature signal. By performing multi-scale and multi-resolution decomposition on the temperature signal, low-frequency approximation coefficients and high-frequency detail coefficients are obtained;

[0079] Extract temperature degradation features from the high-frequency detail coefficients based on the wavelet energy spectrum and wavelet singular value spectrum. Calculate the energy values of the high-frequency detail coefficients of each layer to construct high-frequency component energy features reflecting the degree of local temperature rise; perform singular value decomposition on the high-frequency detail coefficients of each layer to extract singular value features reflecting temperature mutation situations;

[0080] Construct a temperature feature set characterizing the heat dissipation of the robot according to the high-frequency component energy features and the singular value features.

[0081] Exemplarily, in order to extract sensitive features reflecting the heat dissipation situation from the temperature signal of the robot, the present invention uses wavelet transform (WT) to perform multi-scale and multi-resolution decomposition on the temperature signal, obtains approximation coefficients and detail coefficients in different frequency ranges, extracts temperature degradation features from the high-frequency detail coefficients, and constructs a temperature feature set characterizing the heat dissipation situation of the robot. The specific technical solution is as follows:

[0082] Let the temperature signal collected by the heat dissipation component of the robot be T(t), and select a suitable mother wavelet function and scale factor a to perform continuous wavelet transform on the temperature signal:

[0083] ;

[0084] where a is the scale factor and b is the translation factor, is the complex conjugate of the mother wavelet function.

[0085] In practical applications, discrete wavelet transform (DWT) is usually used to decompose the signal. Perform J-level wavelet decomposition on the temperature signal T(t) to obtain the J-level low-frequency approximation coefficient A J and the high-frequency detail coefficients {D 1 , D 2 , …, D J} of each layer:

[0086] ;

[0087] where, A J (t) reflects the low-frequency trend information of the temperature signal, and D j (t) reflects the high-frequency detail information of the temperature signal under the j-level decomposition, and J represents the total number of layers.

[0088] Extract two types of temperature degradation features from the high-frequency detail coefficients obtained by wavelet decomposition: high-frequency component energy features and singular value features.

[0089] The first type of feature is based on the wavelet energy spectrum. Calculate the energy values of the high-frequency detail coefficients of each layer and construct features reflecting the degree of local temperature rise. The energy of the high-frequency detail coefficient of the j-th layer is defined as:

[0090] ;

[0091] where, N j represents the length of the high-frequency detail coefficient of the j-th layer,

[0092] Among them, \(N_j\) is the length of the high-frequency detail coefficients of the \(j\)-th layer. The energy values of the high-frequency detail coefficients of each layer form a feature vector , which is used as the first type of temperature degradation feature.

[0093] The second type of feature is based on the wavelet singular value spectrum. The singular value decomposition is performed on the high-frequency detail coefficients of each layer to extract the features reflecting the temperature mutation. The singular value decomposition of the high-frequency detail coefficients of the \(j\)-th layer is as follows:

[0094] ;

[0095] where \(U\) j and \(V\) j are orthogonal matrices, is a diagonal matrix, and the elements on the diagonal are singular values , arranged in descending order. The first \(p\) largest singular values of the high-frequency detail coefficients of the \(j\)-th layer are extracted to form a feature vector. The singular value feature vectors of the high-frequency detail coefficients of each layer are concatenated to form the second type of temperature degradation feature.

[0096] By integrating the first type of high-frequency component energy feature and the second type of singular value feature, a temperature feature set characterizing the heat dissipation of the robot is constructed. The temperature feature set combines the high-frequency component energy feature reflecting the local temperature rise degree and the singular value feature reflecting the temperature mutation situation, and can comprehensively depict the heat dissipation degradation of the robot, providing information support for the subsequent health status assessment.

[0097] Through the above steps, the wavelet transform of the robot temperature signal and the extraction of sensitive temperature features are realized. The constructed temperature feature set can effectively reflect the heat dissipation degradation of the robot and provide important inputs for the health status assessment and remaining life prediction.

[0098] S102. Based on the intrinsic mode function and the temperature feature set, combined with the attribute information corresponding to the robot, through knowledge extraction, knowledge representation, and knowledge fusion technologies, various knowledge in robot components, performance parameters, degradation modes, and maintenance measures are semantically represented and linked to construct a management knowledge graph corresponding to the robot;

[0099] In an alternative embodiment,

[0100] Based on the intrinsic mode function and the temperature feature set, combined with the attribute information corresponding to the robot, through knowledge extraction, knowledge representation, and knowledge fusion technologies, various knowledge in robot components, performance parameters, degradation modes, and maintenance measures are semantically represented and linked to construct a management knowledge graph corresponding to the robot, including:

[0101] The knowledge extraction adopts a named entity recognition model based on BERT and a relation extraction model based on BiLSTM-CRF, and constructs a dictionary and a relation template library;

[0102] The knowledge representation adopts a knowledge representation learning model based on TransE to map the concepts and relations in the ontology to a low-dimensional continuous vector space;

[0103] The knowledge fusion adopts a knowledge graph linking method based on ontology matching to establish a link mapping relationship by calculating the semantic similarity between ontologies.

[0104] Exemplarily, in order to realize the knowledge-based evaluation of the robot health status and the prediction of the remaining life, the present invention constructs a robot management knowledge graph based on the extracted intrinsic mode functions and temperature feature sets, combined with the attribute information of the robot, through knowledge extraction, knowledge representation and knowledge fusion technologies. The knowledge graph semantically represents and links various kinds of knowledge such as robot components, performance parameters, degradation modes, maintenance measures, etc., to form a structured, retrievable and inferable knowledge base. The specific technical solutions are as follows:

[0105] Knowledge extraction aims to automatically identify knowledge elements such as entities and relations from unstructured text data and transform them into a structured form. The present invention adopts named entity recognition and relation extraction technologies to extract knowledge from text data such as robot specifications, maintenance manuals, and fault cases.

[0106] Named entity recognition adopts a deep learning model based on BERT (Bidirectional Encoder Representations from Transformers). BERT is a pre-trained bidirectional Transformer encoder that can generate context-related word embedding representations. The pre-trained BERT model is fine-tuned to the named entity recognition task to identify entities such as robot components, performance parameters, degradation modes, and maintenance measures in the text. At the same time, a dictionary library in the robot field is constructed, including synonyms, abbreviations, etc. of various entities, for entity linking and disambiguation.

[0107] Relation extraction adopts a sequence labeling model based on BiLSTM-CRF (Bidirectional Long Short-Term Memory-Conditional Random Field). BiLSTM can capture the context information of the text, and CRF can consider the dependency relationship between labels. The entity pair and its context words are input into the BiLSTM-CRF model to predict the relation type between the entity pairs. At the same time, a relation template library is constructed to define common relation types in the robot field and their corresponding language patterns for guiding the relation extraction process.

[0108] Knowledge representation aims to transform the extracted entities and relationships into a form that can be processed by a computer and mine the semantic information therein. The present invention adopts a knowledge representation learning model based on TransE (Translating Embedding) to map the concepts and relationships in the ontology into a low-dimensional continuous vector space.

[0109] Knowledge fusion aims to integrate multi-source heterogeneous knowledge to construct a unified knowledge graph. The present invention adopts a knowledge graph linking method based on ontology matching to establish a link mapping relationship by calculating the semantic similarity between different ontologies.

[0110] First, according to the characteristics of the robot field, a top-level ontology is designed to define core concepts and relationship types, such as components, performance parameters, degradation modes, maintenance measures, etc. Then, the knowledge subgraphs extracted from different data sources are mapped into the top-level ontology, and the semantic similarity between the concepts in the subgraph and the concepts in the top-level ontology is calculated by the ontology matching method to establish a link mapping relationship.

[0111] The calculation of semantic similarity comprehensively considers semantic, structural, and context information. For semantic similarity calculation, methods such as cosine similarity based on word vectors and semantic distance based on thesaurus are used; for structural similarity calculation, methods such as neighbor node similarity based on graphs and shortest path similarity based on paths are used; for context similarity calculation, methods such as the PersonalRank algorithm based on random walk and context topic similarity based on topic models are used. By synthesizing multiple similarity scores, the matching degree between the concepts in the subgraph and the concepts in the top-level ontology is calculated to establish a link mapping relationship.

[0112] Through ontology matching and link mapping, multi-source heterogeneous knowledge is fused into a unified knowledge graph to form a robot management knowledge base. The management knowledge graph contains various types of knowledge such as robot components, performance parameters, degradation modes, maintenance measures, etc. Through semantic links and ontology reasoning, the association, retrieval, and reasoning of knowledge can be realized, providing comprehensive knowledge support for robot health status assessment and prediction.

[0113] S103. Use a rule mining algorithm based on path ranking to perform reasoning and mining on the management knowledge graph. By mining the reasoning paths composed of frequently co-occurring entities and relationships, extract the degradation characteristics corresponding to each component of the robot in the management knowledge graph, and combine with a pre-constructed time series prediction model to predict the health state trend of the robot.

[0114] In an alternative embodiment,

[0115] The rule mining algorithm based on path ranking performs reasoning and mining on the management knowledge graph. By mining the reasoning paths composed of frequently co-occurring entities and relationships, the degradation features corresponding to each component of the robot in the management knowledge graph are extracted, including:

[0116] For the management knowledge graph, a path sampling strategy based on random walk is adopted to generate candidate reasoning paths. An initial entity node is randomly selected from the management knowledge graph, and the next-hop entity node is randomly selected with equal probability according to the out-edge relationship of the current entity node. This process is repeated until a preset path length is reached. Through multiple random walks, different regions and semantic associations of the management knowledge graph are covered. The candidate reasoning paths include a first reasoning path depicting the association between robot components and corresponding degradation features and a second reasoning path depicting the association between degradation features and potential degradation patterns.

[0117] Sort and filter the candidate reasoning paths, and use a path ranking criterion that combines support and confidence to score the candidate reasoning paths. The comprehensive support measures the frequency of the candidate reasoning path appearing in the management knowledge graph; the confidence measures the strength of the association between the front and back entities in the candidate reasoning path. Calculate the ranking score according to the comprehensive support and confidence of the candidate reasoning path, sort the candidate reasoning paths from high to low according to the score, and select the top K candidate reasoning paths with the highest scores as frequent paths.

[0118] Generate the degradation features corresponding to each component according to the entities and relationship types included in the frequent paths.

[0119] In an alternative embodiment,

[0120] Sort and filter the candidate reasoning paths, and using a path ranking criterion that combines support and confidence to score the candidate reasoning paths includes:

[0121] ;

[0122] Among them, sup(P) represents the comprehensive support corresponding to path P, count(P) represents the number of times path P appears in the set of candidate reasoning paths, N represents the number of candidate reasoning paths, and λ represents a length penalty factor used to control the penalty strength for path length, len(P) represents the length of path P;

[0123] ;

[0124] Among them, conf(P) represents the confidence corresponding to path P, count(e i, r i , e i+1 ) Represents an entity e i Through a relationship r i Connect entities e i+1 The frequency of count(e i ) Represents an entity e i The frequency of occurrence in the candidate inference path w(e i ) Represents an entity e i The weight coefficient of n Represents the number of entities.

[0125] Exemplarily, in order to mine the degradation characteristics corresponding to each component of the robot from the management knowledge graph, the present invention proposes a rule mining algorithm based on path ranking. The algorithm generates candidate inference paths through random walks, and uses a path ranking criterion that combines support and confidence to score and screen the candidate inference paths, and finally generates the degradation characteristics corresponding to each component. The specific technical solution is as follows:

[0126] For the constructed management knowledge graph, a path sampling strategy based on random walks is used to generate candidate inference paths. The specific steps are as follows:

[0127] Randomly select an entity node from the management knowledge graph as the initial node. According to the out-edge relationship of the current entity node, randomly select the next-hop entity node with uniform probability. Add the selected entity node to the candidate inference path and update the current entity node. Repeat the above steps until the length of the candidate inference path reaches the preset maximum length. Generate multiple candidate inference paths to cover different regions and semantic associations of the management knowledge graph.

[0128] The candidate inference paths include two categories: The first category is the inference paths that depict the association between the robot components and the corresponding degradation characteristics, such as "component - parameter - characteristic", "component - mode - characteristic", etc.; the second category is the inference paths that depict the association between the degradation characteristics and the potential degradation modes, such as "characteristic - parameter - mode", "characteristic - component - mode", etc.

[0129] Sort and screen the generated candidate inference paths, and use a path ranking criterion that combines support and confidence to score the candidate inference paths. Sort the candidate inference paths in descending order according to the ranking scores, and select the top K candidate inference paths with the highest scores as the frequent paths.

[0130] Based on the selected frequent paths, generate the degradation features corresponding to each component of the robot. For the frequent paths depicting the association between components and degradation features, extract the component entities and feature entities in the paths to generate "component - feature" binary tuples, indicating that the component has the corresponding degradation feature. For the frequent paths depicting the association between degradation features and potential degradation patterns, extract the feature entities and pattern entities in the paths to generate "feature - pattern" binary tuples, indicating that the degradation feature may lead to the corresponding degradation pattern.

[0131] Finally, by integrating the "component - feature" binary tuples and the "feature - pattern" binary tuples, generate the degradation feature knowledge corresponding to each component of the robot, in the form of "component A - feature B - pattern C", "component X - feature Y", etc.

[0132] Through the above steps, the present invention uses a rule mining algorithm based on path ranking to mine the degradation features corresponding to each component of the robot from the management knowledge graph. This algorithm comprehensively utilizes the global exploration ability of random walk and the local evaluation ability of the path ranking criterion, and can effectively discover the implicit association rules and reasoning patterns in the knowledge graph, providing knowledge support for the health status assessment of the robot.

[0133] In an alternative embodiment,

[0134] Combined with a pre - constructed time - series prediction model, predicting the health state trend of the robot includes:

[0135] Taking the long short - term memory network (LSTM) as the basic building block, construct a deep LSTM network by stacking multiple LSTM layers as the time - series prediction model, and add a fully - connected layer and an activation function at the end of the time - series prediction model. Based on the degradation features corresponding to each component of the robot in the management knowledge graph, output the predicted health state trend.

[0136] Exemplarily, taking the long short - term memory network (Long Short - Term Memory, LSTM) as the basic building block, construct a deep LSTM network by stacking multiple LSTM layers as the time - series prediction model for predicting the health state trend.

[0137] LSTM is a special recurrent neural network (RNN) that can effectively handle the long - term dependence problem in time - series data. The core of LSTM is the introduction of a gating mechanism, including an input gate, a forget gate, and an output gate, which determines which information needs to be remembered and which needs to be forgotten by controlling the flow of information, thereby achieving selective memory and forgetting of long - term information. By stacking multiple LSTM layers, a deep LSTM network is constructed.

[0138] Utilize the degradation features corresponding to each component of the robot mined from the management knowledge graph as the input of the deep LSTM network to predict the health state trend of the robot.

[0139] Encode the degradation features mined from the management knowledge graph to transform them into numerical feature vectors. Methods such as One-Hot encoding and embedding representation can be used to encode the degradation features. Construct a time series data set of the degradation features. According to the timestamps of the degradation features, arrange the degradation features in chronological order to form time series data. Divide the time series data of the degradation features into a training set and a test set. The training set is used to train the deep LSTM network model, and the test set is used to evaluate the prediction performance of the model. Use the training set data to train the deep LSTM network model. Calculate the predicted output through forward propagation, calculate the error between the predicted output and the true label using a loss function (such as mean square error, cross entropy, etc.), and update the model parameters through the backpropagation algorithm.

[0140] Use the trained deep LSTM network model to predict the test set data. Input the time series data of the degradation features in the test set into the model to obtain the corresponding prediction results of the health state trend. Evaluate the prediction performance of the deep LSTM network model. Adopt appropriate evaluation metrics (such as accuracy, precision, recall, F1 score, etc.) to evaluate the prediction performance of the model on the test set, and perform hyperparameter tuning to optimize the prediction ability of the model.

[0141] Through the above steps, the present invention uses the deep LSTM network model, combined with the degradation features of the robot components mined from the management knowledge graph, to realize the prediction of the health state trend of the robot. The deep LSTM network can effectively capture the long-term dependence relationships in the time series of degradation features, predict the future health state change trend of the robot by learning the time evolution law of the degradation features, and provide decision support for the predictive maintenance and life management of the robot.

[0142] Figure 2 The structure diagram of the industrial robot safety monitoring system according to the embodiment of the present invention is as Figure 2 shown, and the system includes:

[0143] A first unit for arranging a multi-modal sensor array to collect the state information of the robot. The state information includes vibration signals and temperature signals. The multi-modal sensor array includes vibration sensors and temperature sensors. The vibration sensors are arranged at the joints and actuators of the robot, and the temperature sensors are arranged at the heat dissipation components of the robot; perform empirical mode decomposition on the vibration signals to obtain the intrinsic mode functions reflecting the wear degree of the robot transmission system; perform wavelet transform on the temperature signals to obtain the temperature feature set reflecting the heat dissipation situation of the robot;

[0144] A second unit, configured to, based on the intrinsic mode functions and the temperature feature set, combine with the attribute information corresponding to the robot, and through knowledge extraction, knowledge representation, and knowledge fusion technologies, semantically represent and link various types of knowledge in robot components, performance parameters, degradation modes, and maintenance measures, and construct a management knowledge graph corresponding to the robot;

[0145] A third unit, configured to perform inference mining on the management knowledge graph based on a rule mining algorithm for path ranking, extract degradation features corresponding to each component of the robot in the management knowledge graph by mining inference paths composed of frequently co-occurring entities and relationships, and combine with a pre-constructed time series prediction model to predict the health state trend of the robot.

[0146] In a third aspect of the embodiments of the present invention,

[0147] There is provided an electronic device, including:

[0148] A processor;

[0149] A memory for storing instructions executable by the processor;

[0150] Wherein, the processor is configured to call the instructions stored in the memory to execute the method described above.

[0151] In a fourth aspect of the embodiments of the present invention,

[0152] There is provided a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method described above is implemented.

[0153] The present invention may be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium, on which computer-readable program instructions for executing various aspects of the present invention are uploaded.

[0154] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An industrial robot safety monitoring method, characterized in that: include: Arrange a multimodal sensor array to collect state information of the robot, the state information includes a vibration signal and a temperature signal, the multimodal sensor array includes a vibration sensor and a temperature sensor, the vibration sensor is arranged at the joints and actuators of the robot, and the temperature sensor is arranged at the heat dissipation component of the robot; perform empirical mode decomposition on the vibration signal to obtain an intrinsic mode function reflecting the wear degree of the robot transmission system; perform wavelet transform on the temperature signal to obtain a temperature feature set reflecting the heat dissipation condition of the robot; Based on the intrinsic modal function and the temperature feature set, combined with the attribute information corresponding to the robot, through knowledge extraction, knowledge representation and knowledge fusion technology, the robot components, performance parameters, degradation modes, maintenance measures of various knowledge are semantically represented and linked to build the management knowledge graph corresponding to the robot; The rule mining algorithm based on path sorting performs inference mining on the management knowledge graph, extracts the degradation features corresponding to each component of the robot in the management knowledge graph by mining the inference path composed of frequently co-occurring entities and relationships, and predicts the health status trend of the robot in combination with the pre-built time series prediction model; The rule mining algorithm based on path sorting performs reasoning mining on the management knowledge graph, and extracts the degradation features corresponding to each component of the robot in the management knowledge graph by mining the reasoning path composed of frequently co-occurring entities and relationships, including: For the management knowledge graph, a random walk-based path sampling strategy is used to generate candidate reasoning paths. An initial entity node is randomly selected from the management knowledge graph. The next hop entity node is randomly selected with a uniform probability according to the outgoing edge relationship of the current entity node. The process is repeated until the preset path length is reached. Different areas and semantic associations of the management knowledge graph are covered by multiple random walks. The candidate reasoning paths include a first reasoning path that describes the association between robot parts and corresponding degradation features and a second reasoning path that describes the association between degradation features and potential degradation patterns. Sorting and screening the candidate reasoning paths, and scoring the candidate reasoning paths using a path sorting criterion of comprehensive support and confidence, wherein the comprehensive support measures the frequency of occurrence of the candidate reasoning paths in the management knowledge graph; The confidence measures the strength of the association between the preceding and succeeding entities in the candidate reasoning path; the ranking score is calculated according to the comprehensive support and confidence of the candidate reasoning path, the candidate reasoning paths are ranked from high to low according to the scores, and the top K candidate reasoning paths with the highest scores are selected as frequent paths; According to the entities and relationship types contained in the frequent paths, the degenerate features corresponding to the various components are generated.

2. The method according to claim 1, characterized in that The vibration signal is subjected to empirical mode decomposition to obtain the inherent mode function reflecting the wear degree of the robot transmission system, including: The vibration signal of the robot transmission system is subjected to empirical mode decomposition, and the vibration signal is adaptively decomposed into a series of intrinsic mode functions to extract sensitive intrinsic mode function components reflecting the intrinsic oscillation mode of the transmission system; the empirical mode decomposition constructs a local envelope through cyclic screening, calculates the local envelope mean as an eigenmode function candidate, and judges the eigenmode function conditions based on extreme points and zero crossing points, and recursively extracts the intrinsic mode function; The correlation between the extracted intrinsic modal function and the wear degree of the robot transmission system is evaluated based on the mutual information criterion. By calculating the joint probability distribution and marginal probability distribution of the intrinsic modal function and the wear degree of the robot transmission system, the first k intrinsic modal functions with the largest mutual information values ​​are selected as sensitive components of wear degradation.

3. The method according to claim 1, characterized in that The temperature signal is subjected to wavelet transform to obtain a temperature feature set reflecting the heat dissipation of the robot, including: Performing wavelet transform on the temperature signal, and obtaining low-frequency approximate coefficients and high-frequency detail coefficients by performing multi-scale and multi-resolution decomposition on the temperature signal; Based on the wavelet energy spectrum and wavelet singular value spectrum, the temperature degradation characteristics are extracted from the high-frequency detail coefficients, the energy value of each layer of high-frequency detail coefficients is calculated, and the high-frequency component energy characteristics reflecting the degree of local temperature rise are constructed; the singular value decomposition of each layer of high-frequency detail coefficients is performed to extract the singular value characteristics reflecting the temperature mutation situation; A temperature feature set characterizing the heat dissipation of the robot is constructed according to the high-frequency component energy feature and the singular value feature.

4. The method according to claim 1, characterized in that Based on the intrinsic modal function and the temperature feature set, combined with the attribute information corresponding to the robot, through knowledge extraction, knowledge representation and knowledge fusion technology, the robot components, performance parameters, degradation modes, maintenance measures and various knowledge are semantically represented and linked, and the management knowledge graph corresponding to the robot is constructed, including: The knowledge extraction adopts a BERT-based named entity recognition model and a BiLSTM-CRF-based relationship extraction model, and constructs a dictionary and relationship template library; The knowledge representation adopts a TransE-based knowledge representation learning model to map concepts and relationships in the ontology into a low-dimensional continuous vector space; The knowledge fusion adopts a knowledge graph linking method based on ontology matching, and establishes a link mapping relationship by calculating the semantic similarity between ontologies.

5. The method according to claim 1, characterized in that Sorting and screening the candidate reasoning paths, and scoring the candidate reasoning paths using a path sorting criterion that combines support and confidence include: ； in, sup(P) represents the comprehensive support corresponding to path P, count(P) represents the number of times path P appears in the candidate reasoning path set, N represents the number of candidate reasoning paths, λ represents the length penalty factor, which is used to control the penalty intensity of the path length. len(P) represents the length of path P; ； in, conf(P) represents the confidence corresponding to path P, count(e i , r i , e i+1 ) Representing Entities e i Through relationships r i Connect entities e i+1 The frequency of count(e i ) Representing Entities e i The frequency of occurrence in the candidate reasoning path, w(e i ) Representing Entities e i The weight coefficient of n Indicates the number of entities.

6. The method according to claim 1, characterized in that Combined with the pre-built time series prediction model, the health status trend of the robot is predicted to include: Taking the long short-term memory network LSTM as the basic building block, a deep LSTM network is constructed as a time series prediction model by stacking multiple LSTM layers. A fully connected layer and an activation function are added at the end of the time series prediction model. The predicted health status trend is output based on the degradation features corresponding to each component of the robot in the management knowledge graph.

7. An industrial robot safety monitoring system, used to implement the method according to any one of claims 1 to 6, characterized in that: include: The first unit is used to arrange a multimodal sensor array to collect the state information of the robot, the state information includes a vibration signal and a temperature signal, the multimodal sensor array includes a vibration sensor and a temperature sensor, the vibration sensor is arranged at the joints and actuators of the robot, and the temperature sensor is arranged at the heat dissipation component of the robot; perform empirical mode decomposition on the vibration signal to obtain an intrinsic mode function reflecting the wear degree of the robot transmission system; perform wavelet transform on the temperature signal to obtain a temperature feature set reflecting the heat dissipation condition of the robot; The second unit is used to semantically represent and link various knowledge in robot parts, performance parameters, degradation modes, and maintenance measures based on the intrinsic modal function and the temperature feature set, combined with the attribute information corresponding to the robot, through knowledge extraction, knowledge representation, and knowledge fusion technology, to build a management knowledge graph corresponding to the robot; The third unit is used to perform reasoning mining on the management knowledge graph based on a path sorting rule mining algorithm, extract the degradation features corresponding to each component of the robot in the management knowledge graph by mining the reasoning path composed of frequently co-occurring entities and relationships, and predict the health status trend of the robot in combination with a pre-built time series prediction model.

8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images