Robot joint early fault diagnosis method and system

Abstract

The invention discloses a robot joint early-stage fault diagnosis method and system, and the method comprises the steps: obtaining multi-modal time series data, carrying out the slicing and alignment of the multi-modal time series data, and obtaining a data sample segment; preprocessing the data sample segments, performing time-frequency analysis on the preprocessed data sample segments by using a continuous wavelet transform function, and generating a time-frequency graph; constructing a hybrid model, performing feature extraction on the time-frequency graph by using the hybrid model, capturing a performance degradation trend of the robot in a standard task period according to the extracted features, performing supervised training by using historical data, and establishing a health baseline; and deploying the trained hybrid model to an edge computing gateway, constructing an early warning logic based on the health baseline, and performing adaptive early warning on the real-time data stream. According to the invention, the parallel two-dimensional convolutional neural network based on the time-frequency graph can better mine the deep discriminative features in the signal, overcomes the limitation of a single signal source, and has higher diagnosis capability for complex faults and composite faults.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of industrial robot state monitoring and predictive maintenance, in particular to a robot joint early fault diagnosis method and system. BACKGROUND

[0002] Industrial robots can perform a certain production task for a long time, significantly improve economic benefits, and are widely used due to their high intelligent level. The reducer, as a core component, is closely related to the performance of the industrial robot. The joint reducer is small in size, light in weight, and high in running precision, and is often used in the end part of the robot arm, wrist and the like. The transmission accuracy of the joint reducer in the end joint affects the overall accuracy of the industrial robot, so it is necessary to monitor whether the joint reducer has a fault in real time.

[0003] At present, the diagnosis of robot joint fault generally sets a vibration threshold value, sets sensors in each joint of the robot, monitors the vibration amplitude of the joint, and compares the vibration amplitude with the preset vibration threshold value to determine whether the joint has a fault.

[0004] However, the degradation of the robot joint is a multi-physical field coupling process, and a single signal cannot comprehensively and accurately reflect the complex fault state. Especially in the early stage of failure, the characteristic signal is weak and the signal-to-noise ratio is low, which is easy to cause misdiagnosis or missed diagnosis. At the same time, the occurrence and development of the fault is a process of evolution over time, and the existing static analysis method cannot effectively capture the weak and gradual time sequence characteristics representing performance degradation from continuous and non-stationary operation data. Therefore, the diagnosis method relying only on threshold value has the problems of lag, single diagnosis dimension, high false alarm rate and lack of effective feature extraction of non-stationary signals. SUMMARY

[0005] Therefore, the purpose of the present application is to provide a robot joint early fault diagnosis method and system, which aims to solve the problems of lag, single diagnosis dimension, high false alarm rate and lack of effective feature extraction of non-stationary signals in the diagnosis method relying only on threshold value.

[0006] To achieve the above-mentioned purpose, the present application provides a robot joint early fault diagnosis method, which comprises: obtaining multi-modal time series data of a robot in a standard task cycle, slicing and aligning the multi-modal time series data based on the standard task cycle, and obtaining a data sample segment; preprocessing the data sample segment, using a continuous wavelet transform function to perform time-frequency analysis on the preprocessed data sample segment, and generating a time-frequency graph; construct a mixed model, use the mixed model to extract features of the time-frequency graph, capture the performance degradation trend of the robot in the standard task cycle according to the extracted features, and use historical data for supervised training to establish a health baseline; Deploy the trained mixed model to the edge computing gateway, construct early warning logic based on the health baseline, and perform adaptive early warning on real-time data streams.

[0007] According to an aspect of the above technical solution, the step of acquiring multi-modal time series data of the robot in the standard task cycle, slicing and aligning the multi-modal time series data based on the standard task cycle, includes: Arranging high-frequency acceleration vibration sensors, servo driver current sensors, and temperature sensors at any joint of the robot, unifying the clock sources of the high-frequency acceleration vibration sensors, the servo driver current sensors, and the temperature sensors, and synchronously collecting multi-modal time series data in the standard task cycle through the sensors, wherein the multi-modal time series data includes vibration time series signals, current time series signals, and temperature time series signals; According to the program node signal and / or encoder position signal of the robot controller, the start and end points in any standard task cycle are identified, and the continuous multi-modal time series data is cut into several equal-length data sample segments, which are aligned according to the standard task cycle, and the data sample segments include vibration signals, current signals, and temperature signals.

[0008] According to an aspect of the above technical solution, the step of pre-processing the data sample segment, using a continuous wavelet transform function to perform time-frequency analysis on the pre-processed data sample segment, and generating a time-frequency graph includes: Deduction, band-pass filtering, and normalization preprocessing are performed on the vibration signals and current signals to improve the signal-to-noise ratio of the vibration signals and current signals; differential processing and standardization processing are performed on the temperature signals; According to the fault characteristic frequency and system resonance frequency of the monitored object, the target frequency band is determined, and the scale sequence is calculated based on the target frequency band :

[0009] wherein, is the actual frequency, is the wavelet center frequency, and Δt is the sampling interval.

[0010] According to an aspect of the above technical solution, after calculating the scale sequence, the pre-processed vibration signals and current signals are subjected to continuous wavelet transform using a calculation library:

[0011] wherein, is an analyzed signal, is a vibration signal or a current signal, is a mother wavelet after scaling and translation, is a scale sequence, is a translation parameter, is a wavelet coefficient of the analyzed signal, is a two-dimensional complex matrix; is a complex matrix is a modulus value matrix E(a, b) obtained by taking the modulus,

[0012] map the numbers of the modulus value matrix to a color space, obtain an initial time-frequency diagram, adjust the initial time-frequency diagram to a preset size through an interpolation algorithm, and perform a contrast enhancement operation on the image to generate a time-frequency diagram for visualizing the fault feature.

[0013] According to an aspect of the above technical solution, the step of constructing a hybrid model, using the hybrid model to extract features from the time-frequency diagram, capturing the performance degradation trend of the robot in the standard task cycle according to the extracted features, and using historical data for supervised training to establish a health baseline comprises: constructing a hybrid model composed of a parallel two-dimensional convolutional neural network and a time series convolutional network in series, using a feature extraction branch of the parallel two-dimensional convolutional neural network to extract feature maps from the time-frequency diagram: inputting the time-frequency diagram of the vibration signal into the two-dimensional convolutional neural network to learn time-frequency spatial features related to mechanical impact and resonance frequency through a two-dimensional convolutional layer and a pooling layer; inputting the time-frequency diagram of the current signal into the two-dimensional convolutional neural network to learn time-frequency spatial features related to load torque change and electromagnetic harmonics through a two-dimensional convolutional layer and a pooling layer; and extracting features from the processed temperature signal through a one-dimensional convolutional layer; splicing the features extracted by the three feature extraction branches through a feature fusion layer to form a multi-modal feature vector, inputting the multi-modal feature vector into the time series convolutional network to capture the performance degradation trend of the robot in the standard task cycle, and outputting a fault classification and a health score.

[0014] According to an aspect of the above technical solution, after obtaining the fault classification and the health score, historical data of the robot joint in a known healthy state and in a simulated and / or known fault state are obtained; using the historical data to train the hybrid model, inputting a large amount of data collected from the robot in the initial healthy state into the trained hybrid model, recording the output health score, and establishing a health baseline by statistical distribution.

[0015] According to an aspect of the above technical solution, the step of deploying the trained hybrid model to an edge computing gateway and constructing an early warning logic based on the health baseline to perform adaptive early warning on real-time data streams comprises: The trained hybrid model is deployed to an edge computing gateway, and real-time data in a standard task cycle is obtained, the real-time data is sliced and aligned, time-frequency analysis is performed, the real-time time-frequency graph is input into the hybrid model, and real-time fault classification and real-time health degree scores are obtained; If the number of continuous periods in which the real-time health degree score is lower than the health baseline is greater than or equal to a preset period threshold, early degradation is determined, and a first-level early warning is performed; when the probability of any category in the real-time fault classification is greater than or equal to a probability threshold, a specific fault type alarm is triggered, and a second-level early warning is performed.

[0016] The application further provides a robot joint early fault diagnosis system for implementing the robot joint early fault diagnosis method. A data acquisition module is configured to acquire multi-modal time series data of a robot in a standard task cycle, slice and align the multi-modal time series data based on the standard task cycle, and obtain a data sample segment. A time-frequency analysis module is configured to preprocess the data sample segment, perform time-frequency analysis on the preprocessed data sample segment using a continuous wavelet transform function, and generate a time-frequency graph. A baseline establishment module is configured to construct a hybrid model, extract features of the time-frequency graph using the hybrid model, capture performance degradation trends of the robot in the standard task cycle according to the extracted features, and establish a health baseline by supervised training using historical data. A warning application module is configured to deploy the trained hybrid model to an edge computing gateway, construct a warning logic based on the health baseline, and perform adaptive early warning on a real-time data stream.

[0017] The application further provides a computer readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the robot joint early fault diagnosis method.

[0018] The application further provides an electronic device including a memory, a processor, and a computer program stored on the memory and running on the processor, wherein the processor implements the robot joint early fault diagnosis method when executing the computer program.

[0019] In summary, according to the robot joint early fault diagnosis method provided by the application, the weak fault characteristics are enhanced from the time-frequency domain through wavelet transform, and the multi-modal feature fusion and the capture of the time sequence trend by the time sequence convolution network can find abnormalities in the fault budding stage and when the physical parameters do not exceed the traditional threshold, the early warning time is much earlier than the traditional method, and the detection sensitivity of the early fault is significantly improved; the parallel two-dimensional convolution neural network based on the time-frequency diagram can better mine the deep discriminative features in the signal, and the vibration, current and temperature signals are physically verified with each other, the limitation of a single signal source is overcome, the diagnosis ability for complex faults and composite faults is stronger, and the false alarm and the missed alarm are effectively reduced; the end-to-end deep learning model combines the physical interpretability of wavelet analysis and the feature learning ability of the convolution neural network, reduces the dependence on expert experience, realizes the standardization and automation of the diagnosis process, can not only identify the fault type, but also quantify the degradation degree of the joint through the health score, and provides key data input and decision support for predictive maintenance.

[0020] Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following description and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 The flow chart of the robot joint early fault diagnosis method in the embodiment one of the application is shown in the figure. Figure 2 The structural schematic diagram of the robot joint early fault diagnosis system in the embodiment two of the application is shown in the figure. Figure 3 The structural block diagram of the electronic device in the embodiment four of the application is shown in the figure. DETAILED DESCRIPTION

[0022] In order to make the purpose, features and advantages of the application more obvious and easy to understand, the specific embodiments of the application will be described in detail below with reference to the drawings. The application is shown in several embodiments in the drawings. However, the application can be realized in many different forms, and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the application more thorough and comprehensive.

[0023] It should be noted that when an element is referred to as being "fixed to" another element, it can be directly on the other element or there may be an intervening element. When an element is considered to be "connected" to another element, it can be directly connected to the other element or there may be an intervening element. The terms "vertical," "horizontal," "left," "right," "upper," "lower," and similar expressions used herein are for illustrative purposes only and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as limiting the invention.

[0024] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances. The term "and / or" as used herein includes any and all combinations of one or more of the related listed items.

[0025] Example 1 like Figure 1 The diagram shows a flowchart of an early fault diagnosis method for robot joints according to Embodiment 1 of the present invention. The early fault diagnosis method for robot joints includes the following steps S01-S04, wherein: S01. Obtain multimodal time-series data of the robot within a standard task cycle, and slice and align the multimodal time-series data based on the standard task cycle to obtain data sample segments.

[0026] Three types of sensors are deployed on the target joint of the robot: a high-frequency acceleration vibration sensor, a servo driver current sensor, and a temperature sensor. The clock source for the high-frequency acceleration vibration sensor, the servo driver current sensor, and the temperature sensor is unified to synchronously acquire vibration timing signals. Current timing signal and temperature timing signals Multimodal time series data.

[0027] Based on the program node signals and / or encoder position signals of the robot controller, the start and end points of each standard task cycle are identified. Using this as a basis, continuous multimodal time-series data is segmented into several equal-length data sample segments aligned according to the standard task cycle. These data sample segments include vibration signals. Current signal and temperature signals ， , where i represents the i-th working cycle.

[0028] S02. Preprocess the data sample segment, use the continuous wavelet transform function to perform time-frequency analysis on the preprocessed data sample segment, and generate a time-frequency diagram.

[0029] Vibration signals and current signal Before performing wavelet transform, the vibration signal and current signal Preprocessing is performed to enhance the vibration signal. and current signal Signal-to-noise ratio: removing vibration signals and current signal The linear or slowly changing trend term is used to prevent interference with time-frequency analysis; bandpass filtering is applied based on signal characteristics, for example, based on vibration signals. Applying bandpass filtering to the characteristics preserves the vibration signal. and current signal The key frequency band is used to focus on the fault characteristic frequencies of bearings and gears, while the current signal... This allows the preservation of harmonic components near the fundamental frequency; for vibration signals and current signal Amplitude normalization is performed to eliminate the influence of dimensions and absolute amplitude.

[0030] For temperature signals ， Because its changes are slow and its frequency domain information is limited, wavelet transform is not performed. The following processing method is used: the difference between adjacent periods or adjacent time points within a period is calculated to obtain the temperature change rate sequence. Temperature change rate sequences can reflect abnormal heating to some extent better than absolute temperature values; temperature sequences or difference sequences are standardized.

[0031] Furthermore, after preprocessing the data sample segments, a target frequency band with the most analytical value is determined based on the fault characteristic frequency of the object under test (such as bearings or gears) and the system resonant frequency. The scale sequence of the corresponding object under test is then calculated based on the target frequency band. This sequence is typically distributed with logarithmic intervals to match the human eye and CNN's perception of frequency (i.e., high resolution in the low-frequency region and low resolution in the high-frequency region).

[0032] in, For actual frequency, Δt is the wavelet center frequency, and Δt is the sampling interval. The corresponding scale sequence is calculated to ensure that the generated time-frequency map can completely cover and optimally present the energy distribution within the frequency band.

[0033] After calculating the scale sequence, a computational library is used to perform continuous wavelet transform on the preprocessed vibration and current signals:

[0034] in, The signal being analyzed can be a vibration signal or a current signal. For the scaled and translated mother wavelet, It is a scale sequence. For translation parameters, The wavelet coefficients for analyzing the signal are two-dimensional complex matrices. In this embodiment, the wavelet basis function can be Morl (Morlet), and the computation library can be Python's pywt.cwt.

[0035] The above calculations output a two-dimensional complex matrix. The rows correspond to different scales (frequency), and the columns correspond to different time points b.

[0036] In order to transform the original complex matrix Transform the image into one that is more conducive to recognition by convolutional neural networks, for complex matrices Modulo operation is used to obtain the modulo matrix E(a,b):

[0037] A real-valued time-frequency energy distribution matrix is ​​obtained, which amplifies the energy representation of transient impacts in the signal.

[0038] Mapping the modulus matrix E(a,b) onto a color space generates an initial time-frequency image. This nonlinear mapping can further enhance the ability of the human eye and computer vision systems to distinguish subtle energy differences. For example, a weak fault impact may not be obvious on a grayscale image, but it will appear as a striking "bright spot" or "warm area" on a pseudo-color image.

[0039] All generated initial time-frequency maps are adjusted to a preset size using an interpolation algorithm to meet the input requirements of the subsequent parallel two-dimensional convolutional neural network. At the same time, histogram equalization or contrast stretching is performed on the adjusted time-frequency maps to further highlight the differences between fault features and the background, thus generating a new time-frequency map.

[0040] S03. Construct a hybrid model, use the hybrid model to extract features from the time-frequency graph, capture the performance degradation trend of the robot within a standard task cycle based on the extracted features, and use historical data for supervised training to establish a health baseline.

[0041] A hybrid model consisting of a parallel two-dimensional convolutional neural network and a temporal convolutional network is constructed, and the feature extraction branch of the parallel two-dimensional convolutional neural network is used to extract feature maps from the time-frequency map.

[0042] Specifically, the time-frequency graph of the vibration signal is input into a two-dimensional convolutional neural network, and the time-frequency spatial features related to mechanical impact and resonant frequency are learned through two-dimensional convolutional layers and pooling layers; the time-frequency graph of the current signal is input into a two-dimensional convolutional neural network, and the time-frequency spatial features related to load torque change and electromagnetic harmonics are learned through two-dimensional convolutional layers and pooling layers; the processed temperature signal is processed through a one-dimensional convolutional layer to extract features. The features extracted by the three feature extraction branches are concatenated through a feature fusion layer to form a multimodal feature vector. The multimodal feature vector is then input into a temporal convolutional network. The temporal convolutional network uses dilated causal convolution, which has a strong ability to model long sequences and can capture the evolution patterns and long-term dependencies of multimodal features over multiple consecutive working cycles, thereby identifying the slow performance degradation trend.

[0043] The temporal convolutional network ultimately connects a fully connected layer and an output layer, with the output layer designed as a dual-output layer. Fault classification output: The Softmax activation function can be used to output the probability of belonging to each fault category, such as: healthy, bearing inner ring wear, gear pitting, poor lubrication, etc. Health score output: The Sigmoid activation function can be used to output a health score in a continuous interval of [0,1].

[0044] Historical data on robot joints in known healthy states and simulated / known fault states, such as artificially created pitting on bearings or added wear material on gears, were collected to form a labeled training dataset. This dataset was used to perform supervised training on the hybrid model. After training, a large amount of data collected from the robot in its initial healthy state was input into the trained hybrid model. The output health score was recorded, its distribution was statistically analyzed, and a health baseline was established as a reference benchmark for subsequent early warnings.

[0045] S04. Deploy the trained hybrid model to the edge computing gateway, build early warning logic based on the health baseline, and perform adaptive early warning for real-time data streams.

[0046] The trained hybrid model is deployed to the edge computing gateway and real-time data within the standard task cycle is acquired. After processing the incoming real-time data through the above steps S01-S03, the processed data samples (especially the time-frequency diagrams of vibration and current) are input into the trained hybrid model to obtain the real-time fault classification result Pfault and health score HIcurrent.

[0047] The warning logic is as follows: if the number of consecutive periods in which the real-time health score is lower than the health baseline is greater than or equal to the preset period threshold, it is judged as early degradation and a level one warning is issued; when the probability of any category in the real-time fault classification is greater than or equal to the probability threshold, a specific fault type alarm is triggered and a level two warning is issued.

[0048] Specifically, if HIcurrent remains below the healthy baseline for 10 consecutive cycles (e.g., below 2 standard deviations of the baseline average), it is considered an early degradation and a Level 1 warning is issued. If the probability of a certain category in the fault classification output Pfault exceeds a set probability threshold (e.g., 0.9), an alarm for that specific fault type is triggered, issuing a Level 2 warning. Warning information can then be published to the factory's host computer monitoring system or MES system via the MQTT protocol.

[0049] In summary, the early fault diagnosis method for robot joints provided by this invention enhances weak fault features in the time-frequency domain through wavelet transform and combines multimodal feature fusion with temporal convolutional networks to capture temporal trends. This enables the detection of anomalies at the fault initiation stage, before physical parameters exceed traditional thresholds, providing a much earlier warning time than traditional methods and significantly improving the sensitivity of early fault detection. The parallel two-dimensional convolutional neural network based on the time-frequency graph can better mine deep discriminative features in the signal. At the same time, vibration, current, and temperature signals physically corroborate each other, overcoming the limitations of a single signal source. This enhances the diagnostic capability for complex and compound faults and effectively reduces false alarms and missed alarms. The end-to-end deep learning model combines the physical interpretability of wavelet analysis with the feature learning capability of convolutional neural networks, reducing reliance on expert experience and achieving standardization and automation of the diagnostic process. It can not only identify fault types but also quantify the degree of joint degradation through health scores, providing crucial data input and decision support for predictive maintenance.

[0050] Example 2 Another aspect of the present invention provides a robot joint early fault diagnosis system, please refer to [link / reference needed]. Figure 2 The diagram shows a schematic of the early fault diagnosis system for robot joints in Embodiment 2 of the present invention. The early fault diagnosis system for robot joints includes: Data acquisition module 11 is used to acquire multimodal time-series data of the robot within a standard task cycle, and to slice and align the multimodal time-series data based on the standard task cycle to obtain data sample segments; The time-frequency analysis module 12 is used to preprocess the data sample segment, perform time-frequency analysis on the preprocessed data sample segment using a continuous wavelet transform function, and generate a time-frequency graph. Baseline establishment module 13 is used to construct a hybrid model, use the hybrid model to extract features from the time-frequency map, capture the performance degradation trend of the robot in the standard task cycle based on the extracted features, and use historical data for supervised training to establish a healthy baseline. The early warning application module 14 is used to deploy the trained hybrid model to the edge computing gateway, build early warning logic based on the health baseline, and perform adaptive early warning on the real-time data stream.

[0051] Three types of sensors are deployed on the target joint of the robot: a high-frequency acceleration vibration sensor, a servo driver current sensor, and a temperature sensor. The clock source for the high-frequency acceleration vibration sensor, the servo driver current sensor, and the temperature sensor is unified to synchronously acquire vibration timing signals. Current timing signal and temperature timing signals Multimodal time series data.

[0052] Based on the program node signals and / or encoder position signals of the robot controller, the start and end points of each standard task cycle are identified. Using this as a basis, continuous multimodal time-series data is segmented into several equal-length data sample segments aligned according to the standard task cycle. These data sample segments include vibration signals. Current signal and temperature signals ， , where i represents the i-th working cycle.

[0053] Vibration signals and current signal Before performing wavelet transform, the vibration signal and current signal Preprocessing is performed to enhance the vibration signal. and current signal Signal-to-noise ratio: removing vibration signals and current signal The linear or slowly changing trend term is used to prevent interference with time-frequency analysis; bandpass filtering is applied based on signal characteristics, for example, based on vibration signals. Applying bandpass filtering to the characteristics preserves the vibration signal. and current signal The key frequency band is used to focus on the fault characteristic frequencies of bearings and gears, while the current signal... This allows the preservation of harmonic components near the fundamental frequency; for vibration signals and current signal Amplitude normalization is performed to eliminate the influence of dimensions and absolute amplitude.

[0054] For temperature signals ， Because its changes are slow and its frequency domain information is limited, wavelet transform is not performed. The following processing method is used: the difference between adjacent periods or adjacent time points within a period is calculated to obtain the temperature change rate sequence. Temperature change rate sequences can reflect abnormal heating to some extent better than absolute temperature values; temperature sequences or difference sequences are standardized.

[0055] Furthermore, after preprocessing the data sample segments, a target frequency band with the most analytical value is determined based on the fault characteristic frequency of the object under test (such as bearings or gears) and the system resonant frequency. The scale sequence of the corresponding object under test is then calculated based on the target frequency band. This sequence is typically distributed with logarithmic intervals to match the human eye and CNN's perception of frequency (i.e., high resolution in the low-frequency region and low resolution in the high-frequency region).

[0056] in, For actual frequency, Δt is the wavelet center frequency, and Δt is the sampling interval. The corresponding scale sequence is calculated to ensure that the generated time-frequency map can completely cover and optimally present the energy distribution within the frequency band.

[0057] After calculating the scale sequence, a computational library is used to perform continuous wavelet transform on the preprocessed vibration and current signals:

[0058] in, The signal being analyzed can be a vibration signal or a current signal. For the scaled and translated mother wavelet, It is a scale sequence. For translation parameters, The wavelet coefficients for analyzing the signal are two-dimensional complex matrices. In this embodiment, the wavelet basis function can be Morl (Morlet), and the computation library can be Python's pywt.cwt.

[0059] The above calculations output a two-dimensional complex matrix. The rows correspond to different scales (frequency), and the columns correspond to different time points b.

[0060] In order to transform the original complex matrix Transform the image into one that is more conducive to recognition by convolutional neural networks, for complex matrices Modulo operation is used to obtain the modulo matrix E(a,b):

[0061] A real-valued time-frequency energy distribution matrix is ​​obtained, which amplifies the energy representation of transient impacts in the signal.

[0062] Mapping the modulus matrix E(a,b) onto a color space generates an initial time-frequency image. This nonlinear mapping can further enhance the ability of the human eye and computer vision systems to distinguish subtle energy differences. For example, a weak fault impact may not be obvious on a grayscale image, but it will appear as a striking "bright spot" or "warm area" on a pseudo-color image.

[0063] All generated initial time-frequency maps are adjusted to a preset size using an interpolation algorithm to meet the input requirements of the subsequent parallel two-dimensional convolutional neural network. At the same time, histogram equalization or contrast stretching is performed on the adjusted time-frequency maps to further highlight the differences between fault features and the background, thus generating a new time-frequency map.

[0064] A hybrid model consisting of a parallel two-dimensional convolutional neural network and a temporal convolutional network is constructed, and the feature extraction branch of the parallel two-dimensional convolutional neural network is used to extract feature maps from the time-frequency map.

[0065] Specifically, the time-frequency graph of the vibration signal is input into a two-dimensional convolutional neural network, and the time-frequency spatial features related to mechanical impact and resonant frequency are learned through two-dimensional convolutional layers and pooling layers; the time-frequency graph of the current signal is input into a two-dimensional convolutional neural network, and the time-frequency spatial features related to load torque change and electromagnetic harmonics are learned through two-dimensional convolutional layers and pooling layers; the processed temperature signal is processed through a one-dimensional convolutional layer to extract features. The features extracted by the three feature extraction branches are concatenated through a feature fusion layer to form a multimodal feature vector. The multimodal feature vector is then input into a temporal convolutional network. The temporal convolutional network uses dilated causal convolution, which has a strong ability to model long sequences and can capture the evolution patterns and long-term dependencies of multimodal features over multiple consecutive working cycles, thereby identifying the slow performance degradation trend.

[0066] The temporal convolutional network ultimately connects a fully connected layer and an output layer, with the output layer designed as a dual-output layer. Fault classification output: The Softmax activation function can be used to output the probability of belonging to each fault category, such as: healthy, bearing inner ring wear, gear pitting, poor lubrication, etc. Health score output: The Sigmoid activation function can be used to output a health score in a continuous interval of [0,1].

[0067] Historical data on robot joints in known healthy states and simulated / known fault states, such as artificially created pitting on bearings or added wear material on gears, were collected to form a labeled training dataset. This dataset was used to perform supervised training on the hybrid model. After training, a large amount of data collected from the robot in its initial healthy state was input into the trained hybrid model. The output health score was recorded, its distribution was statistically analyzed, and a health baseline was established as a reference benchmark for subsequent early warnings.

[0068] The trained hybrid model is deployed to the edge computing gateway and real-time data within the standard task cycle is acquired. After the incoming real-time data is processed by the above modules, the processed data samples (especially the time-frequency graphs of vibration and current) are input into the trained hybrid model to obtain the real-time fault classification result Pfault and health score HIcurrent.

[0069] The warning logic is as follows: if the number of consecutive periods in which the real-time health score is lower than the health baseline is greater than or equal to the preset period threshold, it is judged as early degradation and a level one warning is issued; when the probability of any category in the real-time fault classification is greater than or equal to the probability threshold, a specific fault type alarm is triggered and a level two warning is issued.

[0070] Specifically, if HIcurrent remains below the healthy baseline for 10 consecutive cycles (e.g., below 2 standard deviations of the baseline average), it is considered an early degradation and a Level 1 warning is issued. If the probability of a certain category in the fault classification output Pfault exceeds a set probability threshold (e.g., 0.9), an alarm for that specific fault type is triggered, issuing a Level 2 warning. Warning information can then be published to the factory's host computer monitoring system or MES system via the MQTT protocol.

[0071] In summary, the robot joint early fault diagnosis system provided by this invention enhances weak fault features in the time-frequency domain through wavelet transform and combines multimodal feature fusion with temporal convolutional networks to capture temporal trends. This enables the detection of anomalies at the fault initiation stage, before physical parameters exceed traditional thresholds, providing a much earlier warning time than traditional methods and significantly improving the sensitivity for early fault detection. The parallel two-dimensional convolutional neural network based on the time-frequency graph can better mine deep discriminative features in the signal. Simultaneously, vibration, current, and temperature signals physically corroborate each other, overcoming the limitations of a single signal source and enhancing the diagnostic capability for complex and compound faults, effectively reducing false alarms and missed alarms. The end-to-end deep learning model combines the physical interpretability of wavelet analysis with the feature learning capability of convolutional neural networks, reducing reliance on expert experience and achieving standardization and automation of the diagnostic process. It not only identifies fault types but also quantifies the degree of joint degradation through health scores, providing crucial data input and decision support for predictive maintenance.

[0072] Example 3 In another aspect, the present invention also proposes a computer-readable storage medium having stored thereon one or more computer programs that, when executed by a processor, implement the above-described method for early fault diagnosis of robot joints.

[0073] Those skilled in the art will understand that the logic or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable storage medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable storage medium" can mean any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0074] More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable storage media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0075] Example 4 Figure 3 This is a structural block diagram of an electronic device provided in Embodiment 4. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the robot joint early fault diagnosis method in the above embodiments. Figure 3 The electronic device 30 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0076] like Figure 3 As shown, the electronic device 30 can be manifested as a general-purpose computing device, such as a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including memory 32 and processor 31).

[0077] Bus 33 includes a data bus, an address bus, and a control bus.

[0078] The memory 32 may include volatile memory, such as RAM 321 (random access memory), and / or cache memory 322, and may further include ROM 323 (read-only memory).

[0079] The memory 32 may also include a program tool 325 having a set (at least one) of program modules 324, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0080] The processor 31 executes various functional applications and data processing by running computer programs stored in the memory 32, such as the robot joint early fault diagnosis method described above.

[0081] Electronic device 30 can also communicate with one or more external devices 34 (e.g., keyboard, pointing device, etc.). This communication can be performed via I / O interface 35 (input / output interface). Furthermore, electronic device 30 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) via network adapter 36. Figure 3 As shown, network adapter 36 communicates with other modules of the model-generated electronic device 30 via bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the model-generated electronic device 30, including but not limited to: microcode, device drivers, redundant processors, disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0082] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0083] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0084] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

Claims

1. A method for early fault diagnosis of robot joints, characterized in that, The method for early fault diagnosis of robot joints includes: Acquire multimodal time-series data of the robot within a standard task cycle, and slice and align the multimodal time-series data based on the standard task cycle to obtain data sample segments; The data sample segment is preprocessed, and the time-frequency analysis of the preprocessed data sample segment is performed using the continuous wavelet transform function to generate a time-frequency graph. A hybrid model is constructed, and the hybrid model is used to extract features from the time-frequency graph. Based on the extracted features, the performance degradation trend of the robot within a standard task cycle is captured, and historical data is used for supervised training to establish a health baseline. The trained hybrid model is deployed to an edge computing gateway, and early warning logic is built based on the health baseline to provide adaptive early warning for real-time data streams.

2. The method for early fault diagnosis of robot joints according to claim 1, characterized in that, The steps of acquiring multimodal time-series data of the robot within a standard task cycle, and slicing and aligning the multimodal time-series data based on the standard task cycle to obtain data sample segments include: A high-frequency acceleration vibration sensor, a servo driver current sensor, and a temperature sensor are arranged at any joint of the robot. The clock source of the high-frequency acceleration vibration sensor, the servo driver current sensor, and the temperature sensor are unified. Multimodal timing data within a standard task cycle is collected synchronously through the sensors. The multimodal timing data includes vibration timing signals, current timing signals, and temperature timing signals. Based on the program node signals and / or encoder position signals of the robot controller, the start and end points within any standard task cycle are identified, and the continuous multimodal time-series data is cut into several data sample segments of equal length and aligned according to the standard task cycle. The data sample segments include vibration signals, current signals, and temperature signals.

3. The method for early fault diagnosis of robot joints according to claim 1, characterized in that, The steps of preprocessing the data sample segment, performing time-frequency analysis on the preprocessed data sample segment using a continuous wavelet transform function, and generating a time-frequency graph include: The vibration and current signals are preprocessed by detrending, bandpass filtering, and normalization to improve their signal-to-noise ratio; the temperature signal is preprocessed by differential processing and standardization. Based on the fault characteristic frequency of the monitored object and the system resonant frequency, the target frequency band is determined, and the scale sequence is calculated based on the target frequency band. : in, For actual frequency, Δt is the wavelet center frequency, and Δt is the sampling interval.

4. The method for early fault diagnosis of robot joints according to claim 3, characterized in that, After calculating the scale sequence, a computational library is used to perform continuous wavelet transform on the preprocessed vibration and current signals: in, The signal being analyzed can be a vibration signal or a current signal. For the scaled and translated mother wavelet, It is a scale sequence. For translation parameters, The wavelet coefficients for analyzing the signal are two-dimensional complex matrices; For complex matrices Modulo operation is used to obtain the modulo matrix E(a,b): The digital values ​​of the modulus matrix are mapped onto the color space to obtain an initial time-frequency map. The initial time-frequency map is then adjusted to a preset size using an interpolation algorithm, and the image is subjected to contrast enhancement to generate a time-frequency map that visualizes the fault features.

5. The method for early fault diagnosis of robot joints according to claim 1, characterized in that, The steps of constructing a hybrid model, using the hybrid model to extract features from the time-frequency graph, capturing the performance degradation trend of the robot within a standard task cycle based on the extracted features, and establishing a health baseline using historical data for supervised training include: A hybrid model consisting of a parallel 2D convolutional neural network and a temporal convolutional network is constructed. The feature extraction branch of the parallel 2D convolutional neural network is used to extract feature maps from the time-frequency map. The time-frequency graph of the vibration signal is input into a two-dimensional convolutional neural network. Through two-dimensional convolutional layers and pooling layers, the time-frequency spatial features related to mechanical impact and resonant frequency are learned. The time-frequency graph of the current signal is input into a two-dimensional convolutional neural network. Through two-dimensional convolutional layers and pooling layers, the time-frequency spatial features related to load torque change and electromagnetic harmonics are learned. The processed temperature signal is processed through a one-dimensional convolutional layer to extract features. The features extracted by the three feature extraction branches are concatenated through a feature fusion layer to form a multimodal feature vector. The multimodal feature vector is then input into a temporal convolutional network to capture the performance degradation trend of the robot within a standard task cycle and output fault classification and health score.

6. The method for early fault diagnosis of robot joints according to claim 5, characterized in that, After obtaining fault classification and health score, the historical data of the mobile robot joints in known health states and simulated and / or known fault states are obtained. The hybrid model is trained using historical data. A large amount of data collected from the robot in its initial healthy state is input into the trained hybrid model. The output health score is recorded, the distribution is statistically analyzed, and a health baseline is established.

7. The method for early fault diagnosis of robot joints according to claim 1, characterized in that, The steps of deploying the trained hybrid model to an edge computing gateway, constructing early warning logic based on the health baseline, and performing adaptive early warning for real-time data streams include: The trained hybrid model is deployed to the edge computing gateway, and real-time data within the standard task cycle is obtained. After slicing and aligning the real-time data and performing time-frequency analysis, the real-time time-frequency graph is input into the hybrid model to obtain real-time fault classification and real-time health score. If the number of consecutive cycles in which the real-time health score is below the health baseline is greater than or equal to the preset cycle threshold, it is judged as early degradation and a level one warning is issued; when the probability of any category in the real-time fault classification is greater than or equal to the probability threshold, a specific fault type alarm is triggered and a level two warning is issued.

8. A robot joint early fault diagnosis system, characterized in that, The robot joint early fault diagnosis system is used to implement the robot joint early fault diagnosis method according to any one of claims 1-7, the system comprising: The data acquisition module is used to acquire multimodal time-series data of the robot within a standard task cycle, and to slice and align the multimodal time-series data based on the standard task cycle to obtain data sample segments. The time-frequency analysis module is used to preprocess the data sample segment, perform time-frequency analysis on the preprocessed data sample segment using a continuous wavelet transform function, and generate a time-frequency graph. The baseline establishment module is used to construct a hybrid model, extract features from the time-frequency graph using the hybrid model, capture the performance degradation trend of the robot within a standard task cycle based on the extracted features, and use historical data for supervised training to establish a healthy baseline. The early warning application module is used to deploy the trained hybrid model to the edge computing gateway, build early warning logic based on the health baseline, and perform adaptive early warning on the real-time data stream.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the early fault diagnosis method for robot joints as described in any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes a computer program, it implements the robot joint early fault diagnosis method as described in any one of claims 1-7.

Images