Industrial robot safety monitoring method and system

Abstract

The invention provides an industrial robot safety monitoring method and system, and relates to the technical field of safety monitoring, and the method comprises the steps: arranging a multi-mode sensor array to collect the state information of a robot; performing empirical mode decomposition on the vibration signal to obtain an intrinsic mode function reflecting the abrasion degree of the robot transmission system; performing wavelet transform on the temperature signal to obtain a temperature feature set reflecting the heat dissipation condition of the robot; through knowledge extraction, knowledge representation and knowledge fusion technologies, semantic representation and linking are carried out on various knowledge in robot components, performance parameters, degradation modes and maintenance measures, and a management knowledge graph corresponding to the robot is constructed; and performing reasoning mining on the management knowledge graph based on a rule mining algorithm of path sorting, extracting degradation characteristics corresponding to each component of the robot in the management knowledge graph by mining a reasoning path formed by frequently co-occurring entities and relationships, and predicting a health state trend of the robot in combination with a pre-constructed time sequence prediction model.

Description

Technical Field

[0001] The present invention relates to safety monitoring technology, and in particular to an industrial robot safety monitoring method and system. Background Art

[0002] With the rapid development and widespread application of robotics technology, the health management and maintenance of robots have received increasing attention. During the long-term operation of robots, their key components and systems are susceptible to wear, fatigue, environmental factors, etc., resulting in performance degradation and increased failure risks. In order to ensure the reliable operation of robots and extend their service life, it is urgent to develop effective robot health status assessment and remaining life prediction technology.

[0003] Traditional robot health management mainly relies on regular manual inspection and maintenance, and evaluates the health of the robot through visual inspection, performance testing, etc. However, this method is difficult to detect potential faults in a timely manner and is time-consuming and labor-intensive. In recent years, some scholars have proposed using data-driven methods for robot health management. These methods collect various sensor data during the operation of the robot, such as current, vibration, temperature, etc., and establish data models to monitor and predict the health of the robot.

[0004] Although the existing data-driven methods have made some progress, they still have the following shortcomings: First, most methods only use a single type of sensor data and lack the comprehensive use of multi-source heterogeneous information; second, the existing methods mainly target a single key component or subsystem and lack an assessment of the overall health status of the robot; third, there is relatively little research on the prediction of the remaining life of the robot, and the prediction accuracy needs to be improved. Summary of the invention

[0005] The embodiments of the present invention provide an industrial robot safety monitoring method and system, which can solve the problems in the prior art.

[0006] According to a first aspect of the embodiments of the present invention, Provide industrial robot safety monitoring methods, including: Arrange a multimodal sensor array to collect state information of the robot, the state information includes a vibration signal and a temperature signal, the multimodal sensor array includes a vibration sensor and a temperature sensor, the vibration sensor is arranged at the joints and actuators of the robot, and the temperature sensor is arranged at the heat dissipation component of the robot; perform empirical mode decomposition on the vibration signal to obtain an intrinsic mode function reflecting the wear degree of the robot transmission system; perform wavelet transform on the temperature signal to obtain a temperature feature set reflecting the heat dissipation condition of the robot; Based on the intrinsic modal function and the temperature feature set, combined with the attribute information corresponding to the robot, through knowledge extraction, knowledge representation and knowledge fusion technology, the robot components, performance parameters, degradation modes, maintenance measures of various knowledge are semantically represented and linked to build the management knowledge graph corresponding to the robot; A rule mining algorithm based on path sorting performs inference mining on the management knowledge graph. By mining the inference path composed of frequently co-occurring entities and relationships, the degradation characteristics corresponding to each component of the robot in the management knowledge graph are extracted. Combined with a pre-built time series prediction model, the health status trend of the robot is predicted.

[0007] In an optional embodiment, The vibration signal is subjected to empirical mode decomposition to obtain the inherent mode function reflecting the wear degree of the robot transmission system, including: The vibration signal of the robot transmission system is subjected to empirical mode decomposition, and the vibration signal is adaptively decomposed into a series of intrinsic mode functions to extract sensitive intrinsic mode function components reflecting the intrinsic oscillation mode of the transmission system; the empirical mode decomposition constructs a local envelope through cyclic screening, calculates the local envelope mean as an eigenmode function candidate, and judges the eigenmode function conditions based on extreme points and zero crossing points, and recursively extracts the intrinsic mode function; The correlation between the extracted intrinsic modal function and the wear degree of the robot transmission system is evaluated based on the mutual information criterion. By calculating the joint probability distribution and marginal probability distribution of the intrinsic modal function and the wear degree of the robot transmission system, the first k intrinsic modal functions with the largest mutual information values ​​are selected as sensitive components of wear degradation.

[0008] In an optional embodiment, The temperature signal is subjected to wavelet transform to obtain a temperature feature set reflecting the heat dissipation of the robot, including: Performing wavelet transform on the temperature signal, and obtaining low-frequency approximate coefficients and high-frequency detail coefficients by performing multi-scale and multi-resolution decomposition on the temperature signal; Based on the wavelet energy spectrum and wavelet singular value spectrum, the temperature degradation characteristics are extracted from the high-frequency detail coefficients, the energy value of each layer of high-frequency detail coefficients is calculated, and the high-frequency component energy characteristics reflecting the degree of local temperature rise are constructed; the singular value decomposition of each layer of high-frequency detail coefficients is performed to extract the singular value characteristics reflecting the temperature mutation situation; A temperature feature set characterizing the heat dissipation of the robot is constructed according to the high-frequency component energy feature and the singular value feature.

[0009] In an optional embodiment, Based on the intrinsic modal function and the temperature feature set, combined with the attribute information corresponding to the robot, through knowledge extraction, knowledge representation and knowledge fusion technology, the robot components, performance parameters, degradation modes, maintenance measures and various knowledge are semantically represented and linked, and the management knowledge graph corresponding to the robot is constructed, including: The knowledge extraction adopts a BERT-based named entity recognition model and a BiLSTM-CRF-based relationship extraction model, and constructs a dictionary and relationship template library; The knowledge representation adopts a TransE-based knowledge representation learning model to map concepts and relationships in the ontology into a low-dimensional continuous vector space; The knowledge fusion adopts a knowledge graph linking method based on ontology matching, and establishes a link mapping relationship by calculating the semantic similarity between ontologies.

[0010] In an optional embodiment, The rule mining algorithm based on path sorting performs reasoning mining on the management knowledge graph, and extracts the degradation features corresponding to each component of the robot in the management knowledge graph by mining the reasoning path composed of frequently co-occurring entities and relationships, including: For the management knowledge graph, a random walk-based path sampling strategy is used to generate candidate reasoning paths. An initial entity node is randomly selected from the management knowledge graph. The next hop entity node is randomly selected with a uniform probability according to the outgoing edge relationship of the current entity node. The process is repeated until the preset path length is reached. Different areas and semantic associations of the management knowledge graph are covered by multiple random walks. The candidate reasoning paths include a first reasoning path that describes the association between robot parts and corresponding degradation features and a second reasoning path that describes the association between degradation features and potential degradation patterns. The candidate reasoning paths are sorted and screened, and the candidate reasoning paths are scored using the path sorting criteria of comprehensive support and confidence, wherein the comprehensive support measures the frequency of occurrence of the candidate reasoning paths in the management knowledge graph; the confidence measures the strength of the association between the front and back entities in the candidate reasoning paths; the sorting scores are calculated based on the comprehensive support and confidence of the candidate reasoning paths, and the candidate reasoning paths are sorted from high to low according to the scores, and the top K candidate reasoning paths with the highest scores are screened out as frequent paths; According to the entities and relationship types contained in the frequent paths, the degenerate features corresponding to the various components are generated.

[0011] In an optional embodiment, Sorting and screening the candidate reasoning paths, and scoring the candidate reasoning paths using a path sorting criterion that combines support and confidence include: ; in, sup(P) represents the comprehensive support corresponding to path P, count(P) represents the number of times path P appears in the candidate reasoning path set, N represents the number of candidate reasoning paths, λ represents the length penalty factor, which is used to control the penalty intensity of the path length. len(P) represents the length of path P; ; in, conf(P) represents the confidence corresponding to path P, count(e i , r i , e i+1 ) Representing Entities e i Through relationships r i Connect entities e i+1 The frequency of count(e i ) Representing Entities e i The frequency of occurrence in the candidate reasoning path, w(e i ) Representing Entities e i The weight coefficient of n Indicates the number of entities.

[0012] In an optional embodiment, Combined with the pre-built time series prediction model, the health status trend of the robot is predicted to include: Taking the long short-term memory network LSTM as the basic building block, a deep LSTM network is constructed as a time series prediction model by stacking multiple LSTM layers. A fully connected layer and an activation function are added at the end of the time series prediction model. The predicted health status trend is output based on the degradation features corresponding to each component of the robot in the management knowledge graph.

[0013] In the second aspect of the embodiments of the present application, An industrial robot safety monitoring system is provided, for implementing the method according to any one of claims 1 to 7, characterized in that it comprises: The first unit is used to arrange a multimodal sensor array to collect the state information of the robot, the state information includes a vibration signal and a temperature signal, the multimodal sensor array includes a vibration sensor and a temperature sensor, the vibration sensor is arranged at the joints and actuators of the robot, and the temperature sensor is arranged at the heat dissipation component of the robot; perform empirical mode decomposition on the vibration signal to obtain an intrinsic mode function reflecting the wear degree of the robot transmission system; perform wavelet transform on the temperature signal to obtain a temperature feature set reflecting the heat dissipation condition of the robot; The second unit is used to semantically represent and link various knowledge in robot parts, performance parameters, degradation modes, and maintenance measures based on the intrinsic modal function and the temperature feature set, combined with the attribute information corresponding to the robot, through knowledge extraction, knowledge representation, and knowledge fusion technology, to build a management knowledge graph corresponding to the robot; The third unit is used to perform reasoning mining on the management knowledge graph based on a path sorting rule mining algorithm, extract the degradation features corresponding to each component of the robot in the management knowledge graph by mining the reasoning path composed of frequently co-occurring entities and relationships, and predict the health status trend of the robot in combination with a pre-built time series prediction model.

[0014] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0015] According to a fourth aspect of the embodiments of the present invention, A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0016] By arranging a multimodal sensor array to collect the robot's vibration and temperature signals, a comprehensive perception of the robot's operating status is achieved. The vibration and temperature signals reflect the wear and heat dissipation of the robot's transmission system, respectively, providing a rich data basis for subsequent health status assessment.

[0017] The vibration signal was processed by empirical mode decomposition, and the intrinsic mode function reflecting the wear degree of the robot transmission system was obtained, which effectively extracted the degradation characteristics in the vibration signal. At the same time, the temperature signal was processed by wavelet transform, and the temperature feature set reflecting the heat dissipation of the robot was obtained, revealing the relationship between temperature change and health status. The knowledge graph technology was introduced to semantically represent and link various knowledge such as robot components, performance parameters, degradation modes, maintenance measures, etc., and construct a robot management knowledge graph. The construction of the knowledge graph realizes the structuring, visualization and intelligence of knowledge in the field of robot health management, and provides comprehensive knowledge support for health status assessment.

[0018] A rule mining algorithm based on path sorting is proposed. By mining the reasoning path composed of frequently co-occurring entities and relationships in the management knowledge graph, the degradation features corresponding to each component of the robot are automatically extracted. This method overcomes the limitations of traditional methods that rely on expert experience and manual feature construction, and realizes the automatic discovery and extraction of degradation features. Combined with the pre-built time series prediction model, the prediction of the health status trend of the robot is realized. By mining the time series patterns in the historical operation data and building a time series prediction model for the health status, the health status change trend of the robot in the future can be dynamically predicted, providing a basis for early warning and preventive maintenance.

[0019] The present invention makes comprehensive use of multimodal sensor information, signal processing technology, knowledge graph technology and machine learning technology to achieve a comprehensive assessment of the robot's health status and prediction of its remaining life. Compared with traditional methods, the present invention can more comprehensively and accurately assess the robot's health status, discover potential failure risks in advance, optimize maintenance strategies, extend the robot's service life, reduce maintenance costs, and improve the robot's reliability and availability. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Schematic diagram of the process of the industrial robot safety monitoring method according to an embodiment of the present invention; Figure 2 Schematic diagram of the structure of an industrial robot safety monitoring system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0023] Figure 1 FIG. 1 is a flow chart of an industrial robot safety monitoring method according to an embodiment of the present invention. Figure 1 As shown, the method includes: S101. Arrange a multimodal sensor array to collect status information of the robot, wherein the status information includes a vibration signal and a temperature signal, and the multimodal sensor array includes a vibration sensor and a temperature sensor, wherein the vibration sensor is arranged at the joints and actuators of the robot, and the temperature sensor is arranged at the heat dissipation component of the robot; perform empirical mode decomposition on the vibration signal to obtain an intrinsic mode function reflecting the degree of wear of the robot transmission system; perform wavelet transform on the temperature signal to obtain a temperature feature set reflecting the heat dissipation of the robot; In an optional embodiment, The vibration signal is subjected to empirical mode decomposition to obtain the inherent mode function reflecting the wear degree of the robot transmission system, including: The vibration signal of the robot transmission system is subjected to empirical mode decomposition, and the vibration signal is adaptively decomposed into a series of intrinsic mode functions to extract sensitive intrinsic mode function components reflecting the intrinsic oscillation mode of the transmission system; the empirical mode decomposition constructs a local envelope through cyclic screening, calculates the local envelope mean as an eigenmode function candidate, and judges the eigenmode function conditions based on extreme points and zero crossing points, and recursively extracts the intrinsic mode function; The correlation between the extracted intrinsic modal function and the wear degree of the robot transmission system is evaluated based on the mutual information criterion. By calculating the joint probability distribution and marginal probability distribution of the intrinsic modal function and the wear degree of the robot transmission system, the first k intrinsic modal functions with the largest mutual information values ​​are selected as sensitive components of wear degradation.

[0024] For example, in order to extract sensitive features reflecting the degree of wear from the vibration signal of the robot transmission system, the present invention uses the Empirical Mode Decomposition (EMD) method to adaptively decompose the vibration signal to obtain a series of intrinsic mode functions (IMF) reflecting the inherent oscillation mode of the transmission system, and selects sensitive IMF components that are highly correlated with wear degradation based on the mutual information criterion. The specific technical solution is as follows: Assume that the vibration signal collected by the robot transmission system is x(t), and the process of EMD decomposition is as follows: Identify all local extreme points of the signal x(t), including maximum and minimum points.

[0025] Through cubic spline interpolation, all the maximum and minimum points are connected respectively to construct the upper envelope u(t) and the lower envelope l(t) . Calculate the mean m(t) of the upper and lower envelopes. Subtract the mean from the original signal to get the residual signal h(t). Determine whether h(t) satisfies the IMF condition: In the entire data segment, the number of extreme points and the number of zero-crossing points are equal or differ by at most one; at any time, the mean of the upper and lower envelopes is zero.

[0026] If the IMF condition is met, it is regarded as an IMF component and denoted as c i (t), the residual signal r(t) = x(t) -c i (t) is used as the new input signal to extract the next IMF component; if the IMF condition is not met, h(t) is used as the new input signal and the screening continues until the residual signal becomes a monotonic function or the number of its extreme points is less than a preset threshold, and the EMD decomposition process ends.

[0027] The IMF components obtained by EMD decomposition reflect the inherent oscillation modes of the robot transmission system at different scales, but not all IMF components are highly correlated with wear degradation. In order to select sensitive IMF components, the mutual information criterion is introduced to evaluate the correlation between each IMF component and the degree of wear. Let the i-th IMF component be c i (t), the wear index of the robot transmission system is w(t), and the mutual information between them is defined as: ; Among them, p(c i ,w) is c i The joint probability density function of and w, p(c i ) and p(w) are c i and the marginal probability density function of w. Mutual information I(c i ;w) measures c i The correlation between and w, the larger the mutual information value, the stronger the correlation between the two.

[0028] Calculate the mutual information value between each IMF component and the wear degree index, sort the IMF components in descending order of mutual information value, and select the first k IMF components with the largest mutual information value as the sensitive components of wear degradation.

[0029] Through the above steps, the adaptive decomposition of the vibration signal of the robot transmission system and the selection of sensitive IMF components are realized. The selected sensitive IMF components can effectively reflect the wear and degradation characteristics of the transmission system, laying the foundation for subsequent health status assessment and remaining life prediction.

[0030] In an optional embodiment, The temperature signal is subjected to wavelet transform to obtain a temperature feature set reflecting the heat dissipation of the robot, including: Performing wavelet transform on the temperature signal, and obtaining low-frequency approximate coefficients and high-frequency detail coefficients by performing multi-scale and multi-resolution decomposition on the temperature signal; Based on the wavelet energy spectrum and wavelet singular value spectrum, the temperature degradation characteristics are extracted from the high-frequency detail coefficients, the energy value of each layer of high-frequency detail coefficients is calculated, and the high-frequency component energy characteristics reflecting the degree of local temperature rise are constructed; the singular value decomposition of each layer of high-frequency detail coefficients is performed to extract the singular value characteristics reflecting the temperature mutation situation; A temperature feature set characterizing the heat dissipation of the robot is constructed according to the high-frequency component energy feature and the singular value feature.

[0031] For example, in order to extract sensitive features reflecting the heat dissipation from the temperature signal of the robot, the present invention uses wavelet transform (WT) to perform multi-scale and multi-resolution decomposition on the temperature signal, obtain approximate coefficients and detail coefficients in different frequency ranges, and extract temperature degradation features from high-frequency detail coefficients to construct a temperature feature set that characterizes the heat dissipation of the robot. The specific technical solution is as follows: Assume that the temperature signal collected by the robot's heat dissipation component is T(t), select a suitable mother wavelet function And scale factor a, continuous wavelet transform is performed on the temperature signal: ; Among them, a is the scale factor, b is the translation factor, is the complex conjugate of the mother wavelet function.

[0032] In practical applications, discrete wavelet transform (DWT) is usually used to decompose the signal. Perform J-layer wavelet decomposition on the temperature signal T(t) to obtain the J-th layer of low-frequency approximation coefficient A J and each layer's high-frequency detail coefficients {D1, D2,…, D J}: ; Among them, A J (t) reflects the low-frequency trend information of the temperature signal, D j(t) reflects the high-frequency detail information of the temperature signal at the jth layer decomposition, where J represents the total number of layers.

[0033] From the high-frequency detail coefficients obtained by wavelet decomposition, two types of temperature degradation features are extracted: high-frequency component energy features and singular value features.

[0034] The first type of feature is based on the wavelet energy spectrum, which calculates the energy value of the high-frequency detail coefficients of each layer and constructs features that reflect the degree of local temperature rise. The energy of the high-frequency detail coefficients of the jth layer is defined as: ; in, N j represents the length of the j-th layer high-frequency detail coefficient, Among them, N_j is the length of the high-frequency detail coefficient of the jth layer. The energy values ​​of the high-frequency detail coefficients of each layer constitute the feature vector , as the first type of temperature degradation feature.

[0035] The second type of feature is based on the wavelet singular value spectrum, which performs singular value decomposition on the high-frequency detail coefficients of each layer to extract features that reflect temperature mutations. Perform singular value decomposition on the high-frequency detail coefficients of the jth layer: ; Among them, U j and V j is an orthogonal matrix, is a diagonal matrix, and the elements on the diagonal are singular values , arranged in descending order. Extract the first p largest singular values ​​of the j-th layer high-frequency detail coefficients to form a feature vector. Concatenate the singular value feature vectors of each layer of high-frequency detail coefficients to form the second type of temperature degradation features.

[0036] The temperature feature set characterizing the heat dissipation of the robot is constructed by combining the first type of high-frequency component energy features and the second type of singular value features. The temperature feature set combines the high-frequency component energy features reflecting the degree of local temperature rise and the singular value features reflecting the temperature mutation, which can comprehensively characterize the heat dissipation degradation of the robot and provide information support for subsequent health status assessment.

[0037] Through the above steps, the wavelet transform of the robot's temperature signal and the extraction of sensitive temperature features are realized. The constructed temperature feature set can effectively reflect the heat dissipation degradation of the robot and provide important input for health status assessment and remaining life prediction.

[0038] S102. Based on the intrinsic modal function and the temperature feature set, combined with the attribute information corresponding to the robot, through knowledge extraction, knowledge representation and knowledge fusion technology, the robot components, performance parameters, degradation modes, maintenance measures of various knowledge are semantically represented and linked to build a management knowledge graph corresponding to the robot; In an optional embodiment, Based on the intrinsic modal function and the temperature feature set, combined with the attribute information corresponding to the robot, through knowledge extraction, knowledge representation and knowledge fusion technology, the robot components, performance parameters, degradation modes, maintenance measures and various knowledge are semantically represented and linked, and the management knowledge graph corresponding to the robot is constructed, including: The knowledge extraction adopts a BERT-based named entity recognition model and a BiLSTM-CRF-based relationship extraction model, and constructs a dictionary and relationship template library; The knowledge representation adopts a TransE-based knowledge representation learning model to map concepts and relationships in the ontology into a low-dimensional continuous vector space; The knowledge fusion adopts a knowledge graph linking method based on ontology matching, and establishes a link mapping relationship by calculating the semantic similarity between ontologies.

[0039] Exemplarily, in order to realize the knowledge-based assessment of the health status of robots and the prediction of their remaining life, the present invention constructs a robot management knowledge graph based on the extracted intrinsic modal functions and temperature feature sets, combined with the attribute information of the robot, through knowledge extraction, knowledge representation and knowledge fusion technology. The knowledge graph semantically represents and links various knowledge such as robot components, performance parameters, degradation modes, maintenance measures, etc., to form a structured, searchable and inferable knowledge base. The specific technical solution is as follows: Knowledge extraction aims to automatically identify knowledge elements such as entities and relationships from unstructured text data and convert them into structured forms. This paper uses named entity recognition and relationship extraction technology to extract knowledge from text data such as robot manuals, maintenance manuals, and fault cases.

[0040] Named entity recognition uses a deep learning model based on BERT (Bidirectional Encoder Representations from Transformers). BERT is a pre-trained bidirectional Transformer encoder that can generate context-dependent word embedding representations. The pre-trained BERT model is fine-tuned to the named entity recognition task to identify entities such as robot parts, performance parameters, degradation modes, maintenance measures, etc. in the text. At the same time, a robot domain dictionary is built, which contains synonyms, abbreviations, etc. of various entities for entity linking and disambiguation.

[0041] Relation extraction uses a sequence annotation model based on BiLSTM-CRF (Bidirectional Long Short-Term Memory-Conditional Random Field). BiLSTM can capture the contextual information of the text, and CRF can consider the dependencies between tags. Entity pairs and their context words are input into the BiLSTM-CRF model to predict the relationship type between entity pairs. At the same time, a relationship template library is built to define common relationship types and their corresponding language patterns in the robotics field to guide the relationship extraction process.

[0042] Knowledge representation aims to transform the extracted entities and relationships into a form that can be processed by computers and to mine the semantic information therein. The present invention adopts a knowledge representation learning model based on TransE (Translating Embedding) to map the concepts and relationships in the ontology into a low-dimensional continuous vector space.

[0043] Knowledge fusion aims to integrate multi-source heterogeneous knowledge and build a unified knowledge graph. The present invention adopts a knowledge graph linking method based on ontology matching, and establishes a link mapping relationship by calculating the semantic similarity between different ontologies.

[0044] First, according to the characteristics of the robotics field, the top-level ontology is designed to define core concepts and relationship types, such as components, performance parameters, degradation modes, maintenance measures, etc. Then, the knowledge subgraphs extracted from different data sources are mapped to the top-level ontology, and the semantic similarity between the concepts in the subgraph and the concepts in the top-level ontology is calculated through the ontology matching method to establish a link mapping relationship.

[0045] The calculation of semantic similarity comprehensively considers semantic, structural and contextual information. Semantic similarity calculation uses methods such as cosine similarity based on word vectors and semantic distance based on synonym dictionary; structural similarity calculation uses methods such as graph-based neighbor node similarity and path-based shortest path similarity; contextual similarity calculation uses methods such as PersonalRank algorithm based on random walk and contextual topic similarity based on topic model. Combining multiple similarity scores, the matching degree between concepts in the subgraph and concepts in the top-level ontology is calculated, and a link mapping relationship is established.

[0046] Through ontology matching and link mapping, multi-source heterogeneous knowledge is integrated into a unified knowledge graph to form a robot management knowledge base. The management knowledge graph contains a variety of knowledge such as robot components, performance parameters, degradation modes, maintenance measures, etc. Through semantic links and ontology reasoning, knowledge association, retrieval and reasoning can be achieved, providing comprehensive knowledge support for robot health status assessment and prediction.

[0047] S103. Perform inference mining on the management knowledge graph based on a path sorting rule mining algorithm. By mining the inference path composed of frequently co-occurring entities and relationships, the degradation features corresponding to each component of the robot in the management knowledge graph are extracted. Combined with a pre-built time series prediction model, the health status trend of the robot is predicted.

[0048] In an optional embodiment, The rule mining algorithm based on path sorting performs reasoning mining on the management knowledge graph, and extracts the degradation features corresponding to each component of the robot in the management knowledge graph by mining the reasoning path composed of frequently co-occurring entities and relationships, including: For the management knowledge graph, a random walk-based path sampling strategy is used to generate candidate reasoning paths. An initial entity node is randomly selected from the management knowledge graph. The next hop entity node is randomly selected with a uniform probability according to the outgoing edge relationship of the current entity node. The process is repeated until the preset path length is reached. Different areas and semantic associations of the management knowledge graph are covered by multiple random walks. The candidate reasoning paths include a first reasoning path that describes the association between robot parts and corresponding degradation features and a second reasoning path that describes the association between degradation features and potential degradation patterns. The candidate reasoning paths are sorted and screened, and the candidate reasoning paths are scored using the path sorting criteria of comprehensive support and confidence, wherein the comprehensive support measures the frequency of occurrence of the candidate reasoning paths in the management knowledge graph; the confidence measures the strength of the association between the front and back entities in the candidate reasoning paths; the sorting scores are calculated based on the comprehensive support and confidence of the candidate reasoning paths, and the candidate reasoning paths are sorted from high to low according to the scores, and the top K candidate reasoning paths with the highest scores are screened out as frequent paths; According to the entities and relationship types contained in the frequent paths, the degenerate features corresponding to the various components are generated.

[0049] In an optional embodiment, Sorting and screening the candidate reasoning paths, and scoring the candidate reasoning paths using a path sorting criterion that combines support and confidence include: ; in, sup(P) represents the comprehensive support corresponding to path P, count(P) represents the number of times path P appears in the candidate reasoning path set, N represents the number of candidate reasoning paths, λ represents the length penalty factor, which is used to control the penalty intensity of the path length. len(P) represents the length of path P; ; in, conf(P) represents the confidence corresponding to path P, count(e i , r i , e i+1 ) Representing Entities e i Through relationships r i Connect entities e i+1 The frequency of count(e i ) Representing Entities e i The frequency of occurrence in the candidate reasoning path, w(e i ) Representing Entities e i The weight coefficient of n Indicates the number of entities.

[0050] For example, in order to mine the degradation features corresponding to each component of the robot from the management knowledge graph, the present invention proposes a rule mining algorithm based on path sorting. The algorithm generates candidate reasoning paths through random walks, and uses the path sorting criteria of comprehensive support and confidence to score and screen the candidate reasoning paths, and finally generates the degradation features corresponding to each component. The specific technical solution is as follows: For the constructed management knowledge graph, a random walk-based path sampling strategy is used to generate candidate reasoning paths. The specific steps are as follows: Randomly select an entity node from the management knowledge graph as the initial node. According to the outgoing edge relationship of the current entity node, randomly select the next hop entity node with uniform probability. Add the selected entity node to the candidate reasoning path and update the current entity node. Repeat the above steps until the length of the candidate reasoning path reaches the preset maximum length. Generate multiple candidate reasoning paths to cover different areas and semantic associations of the management knowledge graph.

[0051] The candidate reasoning paths include two categories: the first category is the reasoning path that describes the association between robot components and corresponding degradation features, such as "component-parameter-feature", "component-mode-feature", etc.; the second category is the reasoning path that describes the association between degradation features and potential degradation patterns, such as "feature-parameter-mode", "feature-component-mode", etc.

[0052] The generated candidate reasoning paths are sorted and screened, and the candidate reasoning paths are scored using the path sorting criteria that combines support and confidence. The candidate reasoning paths are sorted from high to low according to the sorting scores, and the top K candidate reasoning paths with the highest scores are selected as frequent paths.

[0053] According to the selected frequent paths, the degradation features corresponding to each component of the robot are generated. For the frequent paths that characterize the association between components and degradation features, the component entities and feature entities in the paths are extracted to generate "component-feature" tuples, indicating that the component has the corresponding degradation features. For the frequent paths that characterize the association between degradation features and potential degradation patterns, the feature entities and pattern entities in the paths are extracted to generate "feature-pattern" tuples, indicating that the degradation features may lead to the corresponding degradation patterns.

[0054] Finally, by integrating the "component-feature" tuples and the "feature-pattern" tuples, the degenerate feature knowledge corresponding to each component of the robot is generated, such as "component A-feature B-pattern C", "component X-feature Y", etc.

[0055] Through the above steps, the present invention uses a rule mining algorithm based on path sorting to mine the degradation characteristics corresponding to each component of the robot from the management knowledge graph. The algorithm comprehensively utilizes the global exploration ability of random walks and the local evaluation ability of path sorting criteria, and can effectively discover the implicit association rules and reasoning patterns in the knowledge graph, providing knowledge support for the assessment of the health status of the robot.

[0056] In an optional embodiment, Combined with the pre-built time series prediction model, the health status trend of the robot is predicted to include: Taking the long short-term memory network LSTM as the basic building block, a deep LSTM network is constructed as a time series prediction model by stacking multiple LSTM layers. A fully connected layer and an activation function are added at the end of the time series prediction model. The predicted health status trend is output based on the degradation features corresponding to each component of the robot in the management knowledge graph.

[0057] Exemplarily, a long short-term memory network (LSTM) is used as a basic building block, and a deep LSTM network is constructed by stacking multiple LSTM layers as a time series prediction model for health status trend prediction.

[0058] LSTM is a special recurrent neural network (RNN) that can effectively handle long-term dependency problems in time series data. The core of LSTM is the introduction of a gating mechanism, including input gate, forget gate, and output gate. By controlling the flow of information, it determines which information needs to be remembered and which needs to be forgotten, thereby achieving selective memory and forgetting of long-term information. By stacking multiple LSTM layers, a deep LSTM network is constructed.

[0059] The degradation features corresponding to each robot component mined from the management knowledge graph are used as the input of the deep LSTM network to predict the health status trend of the robot.

[0060] The degradation features mined from the management knowledge graph are encoded and converted into numerical feature vectors. One-Hot encoding, embedded representation and other methods can be used to encode the degradation features. Construct a time series data set of degradation features. According to the timestamps of the degradation features, the degradation features are arranged in chronological order to form time series data. The degradation feature time series data is divided into a training set and a test set. The training set is used to train the deep LSTM network model, and the test set is used to evaluate the prediction performance of the model. The deep LSTM network model is trained using the training set data. The prediction output is calculated through forward propagation, and the error between the prediction output and the true label is calculated using the loss function (such as mean square error, cross entropy, etc.), and the model parameters are updated through the back propagation algorithm.

[0061] Use the trained deep LSTM network model to predict the test set data. Input the degradation feature time series data in the test set into the model to obtain the corresponding health status trend prediction results. Evaluate the prediction performance of the deep LSTM network model. Use appropriate evaluation indicators (such as accuracy, precision, recall, F1 score, etc.) to evaluate the prediction performance of the model on the test set, and perform hyperparameter tuning to optimize the model's prediction ability.

[0062] Through the above steps, the present invention uses a deep LSTM network model, combined with the robot component degradation features mined from the management knowledge graph, to achieve the prediction of the robot health status trend. The deep LSTM network can effectively capture the long-term dependencies in the degradation feature time series, and predict the future health status change trend of the robot by learning the time evolution law of the degradation feature, providing decision support for the predictive maintenance and life management of the robot.

[0063] Figure 2 FIG. 1 is a schematic diagram of the structure of an industrial robot safety monitoring system according to an embodiment of the present invention. Figure 2 As shown, the system comprises: The first unit is used to arrange a multimodal sensor array to collect the state information of the robot, the state information includes a vibration signal and a temperature signal, the multimodal sensor array includes a vibration sensor and a temperature sensor, the vibration sensor is arranged at the joints and actuators of the robot, and the temperature sensor is arranged at the heat dissipation component of the robot; perform empirical mode decomposition on the vibration signal to obtain an intrinsic mode function reflecting the wear degree of the robot transmission system; perform wavelet transform on the temperature signal to obtain a temperature feature set reflecting the heat dissipation condition of the robot; The second unit is used to semantically represent and link various knowledge in robot parts, performance parameters, degradation modes, and maintenance measures based on the intrinsic modal function and the temperature feature set, combined with the attribute information corresponding to the robot, through knowledge extraction, knowledge representation, and knowledge fusion technology, to build a management knowledge graph corresponding to the robot; The third unit is used to perform reasoning mining on the management knowledge graph based on a path sorting rule mining algorithm, extract the degradation features corresponding to each component of the robot in the management knowledge graph by mining the reasoning path composed of frequently co-occurring entities and relationships, and predict the health status trend of the robot in combination with a pre-built time series prediction model.

[0064] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0065] A fourth aspect of the embodiments of the present invention is: A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0066] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An industrial robot safety monitoring method, characterized in that: include: Arrange a multimodal sensor array to collect state information of the robot, the state information includes a vibration signal and a temperature signal, the multimodal sensor array includes a vibration sensor and a temperature sensor, the vibration sensor is arranged at the joints and actuators of the robot, and the temperature sensor is arranged at the heat dissipation component of the robot; perform empirical mode decomposition on the vibration signal to obtain an intrinsic mode function reflecting the wear degree of the robot transmission system; perform wavelet transform on the temperature signal to obtain a temperature feature set reflecting the heat dissipation condition of the robot; Based on the intrinsic modal function and the temperature feature set, combined with the attribute information corresponding to the robot, through knowledge extraction, knowledge representation and knowledge fusion technology, the robot components, performance parameters, degradation modes, maintenance measures of various knowledge are semantically represented and linked to build the management knowledge graph corresponding to the robot; A rule mining algorithm based on path sorting performs inference mining on the management knowledge graph. By mining the inference path composed of frequently co-occurring entities and relationships, the degradation characteristics corresponding to each component of the robot in the management knowledge graph are extracted. Combined with a pre-built time series prediction model, the health status trend of the robot is predicted.

2. The method according to claim 1, characterized in that The vibration signal is subjected to empirical mode decomposition to obtain the inherent mode function reflecting the wear degree of the robot transmission system, including: The vibration signal of the robot transmission system is subjected to empirical mode decomposition, and the vibration signal is adaptively decomposed into a series of intrinsic mode functions to extract sensitive intrinsic mode function components reflecting the intrinsic oscillation mode of the transmission system; the empirical mode decomposition constructs a local envelope through cyclic screening, calculates the local envelope mean as an eigenmode function candidate, and judges the eigenmode function conditions based on extreme points and zero crossing points, and recursively extracts the intrinsic mode function; The correlation between the extracted intrinsic modal function and the wear degree of the robot transmission system is evaluated based on the mutual information criterion. By calculating the joint probability distribution and marginal probability distribution of the intrinsic modal function and the wear degree of the robot transmission system, the first k intrinsic modal functions with the largest mutual information values ​​are selected as sensitive components of wear degradation.

3. The method according to claim 1, characterized in that: The temperature signal is subjected to wavelet transform to obtain a temperature feature set reflecting the heat dissipation of the robot, including: Performing wavelet transform on the temperature signal, and obtaining low-frequency approximate coefficients and high-frequency detail coefficients by performing multi-scale and multi-resolution decomposition on the temperature signal; Based on the wavelet energy spectrum and wavelet singular value spectrum, the temperature degradation characteristics are extracted from the high-frequency detail coefficients, the energy value of each layer of high-frequency detail coefficients is calculated, and the high-frequency component energy characteristics reflecting the degree of local temperature rise are constructed; the singular value decomposition of each layer of high-frequency detail coefficients is performed to extract the singular value characteristics reflecting the temperature mutation situation; A temperature feature set characterizing the heat dissipation of the robot is constructed according to the high-frequency component energy feature and the singular value feature.

4. The method according to claim 1, characterized in that: Based on the intrinsic modal function and the temperature feature set, combined with the attribute information corresponding to the robot, through knowledge extraction, knowledge representation and knowledge fusion technology, the robot components, performance parameters, degradation modes, maintenance measures and various knowledge are semantically represented and linked, and the management knowledge graph corresponding to the robot is constructed, including: The knowledge extraction adopts a BERT-based named entity recognition model and a BiLSTM-CRF-based relationship extraction model, and constructs a dictionary and relationship template library; The knowledge representation adopts a TransE-based knowledge representation learning model to map concepts and relationships in the ontology into a low-dimensional continuous vector space; The knowledge fusion adopts a knowledge graph linking method based on ontology matching, and establishes a link mapping relationship by calculating the semantic similarity between ontologies.

5. The method according to claim 1, characterized in that The rule mining algorithm based on path sorting performs reasoning mining on the management knowledge graph, and extracts the degradation features corresponding to each component of the robot in the management knowledge graph by mining the reasoning path composed of frequently co-occurring entities and relationships, including: For the management knowledge graph, a random walk-based path sampling strategy is used to generate candidate reasoning paths. An initial entity node is randomly selected from the management knowledge graph. The next hop entity node is randomly selected with a uniform probability according to the outgoing edge relationship of the current entity node. The process is repeated until the preset path length is reached. Different areas and semantic associations of the management knowledge graph are covered by multiple random walks. The candidate reasoning paths include a first reasoning path that describes the association between robot parts and corresponding degradation features and a second reasoning path that describes the association between degradation features and potential degradation patterns. The candidate reasoning paths are sorted and screened, and the candidate reasoning paths are scored using the path sorting criteria of comprehensive support and confidence, wherein the comprehensive support measures the frequency of occurrence of the candidate reasoning paths in the management knowledge graph; the confidence measures the strength of the association between the front and back entities in the candidate reasoning paths; the sorting scores are calculated based on the comprehensive support and confidence of the candidate reasoning paths, and the candidate reasoning paths are sorted from high to low according to the scores, and the top K candidate reasoning paths with the highest scores are screened out as frequent paths; According to the entities and relationship types contained in the frequent paths, the degenerate features corresponding to the various components are generated.

6. The method according to claim 5, characterized in that Sorting and screening the candidate reasoning paths, and scoring the candidate reasoning paths using a path sorting criterion that combines support and confidence include: ； in, sup(P) represents the comprehensive support corresponding to path P, count(P) represents the number of times path P appears in the candidate reasoning path set, N represents the number of candidate reasoning paths, λ represents the length penalty factor, which is used to control the penalty intensity of the path length. len(P) represents the length of path P; ； in, conf(P) represents the confidence corresponding to path P, count(e i , r i , e i+1 ) Representing Entities e i Through relationships r i Connect entities e i+1 The frequency of count(e i ) Representing Entities e i The frequency of occurrence in the candidate reasoning path, w(e i ) Representing Entities e i The weight coefficient of n Indicates the number of entities.

7. The method according to claim 1, characterized in that Combined with the pre-built time series prediction model, the health status trend of the robot is predicted to include: Taking the long short-term memory network LSTM as the basic building block, a deep LSTM network is constructed as a time series prediction model by stacking multiple LSTM layers. A fully connected layer and an activation function are added at the end of the time series prediction model. The predicted health status trend is output based on the degradation features corresponding to each component of the robot in the management knowledge graph.

8. An industrial robot safety monitoring system, used to implement the method according to any one of claims 1 to 7, characterized in that: include: The first unit is used to arrange a multimodal sensor array to collect the state information of the robot, the state information includes a vibration signal and a temperature signal, the multimodal sensor array includes a vibration sensor and a temperature sensor, the vibration sensor is arranged at the joints and actuators of the robot, and the temperature sensor is arranged at the heat dissipation component of the robot; perform empirical mode decomposition on the vibration signal to obtain an intrinsic mode function reflecting the wear degree of the robot transmission system; perform wavelet transform on the temperature signal to obtain a temperature feature set reflecting the heat dissipation condition of the robot; The second unit is used to semantically represent and link various knowledge in robot parts, performance parameters, degradation modes, and maintenance measures based on the intrinsic modal function and the temperature feature set, combined with the attribute information corresponding to the robot, through knowledge extraction, knowledge representation, and knowledge fusion technology, to build a management knowledge graph corresponding to the robot; The third unit is used to perform reasoning mining on the management knowledge graph based on a path sorting rule mining algorithm, extract the degradation features corresponding to each component of the robot in the management knowledge graph by mining the reasoning path composed of frequently co-occurring entities and relationships, and predict the health status trend of the robot in combination with a pre-built time series prediction model.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images