Deep learning-based industrial robot joint abnormal vibration detection method and system

Abstract

The application discloses a kind of based on deep learning's industrial robot joint abnormal vibration detection method and system, it is related to robot fault diagnosis technical field.The specific inclusion: collection robot joint vibration signal, constructs training data set;Through autocorrelation analysis and decoupling projection matrix, obtain enhanced sample;Based on deformable convolution block and physical phase coding constructs deformable convolution feature extraction network, obtains deep time sequence feature sequence;Deep time sequence feature sequence is classified by time sequence classification head and obtains classification result;Loss function based on cross-entropy classification loss and dynamic time warping regularization term is constructed, and the model is back propagated, and parameter is updated.The application can improve the visibility of abnormal features in input, so that the model can accurately capture the local abnormal waveform related to the rotation phase within the cycle, and improve the accuracy of model anomaly detection.

Description

Technical Field

[0001] This invention relates to the field of robot fault diagnosis technology, specifically to a method and system for detecting abnormal joint vibrations in industrial robots based on deep learning. Background Technology

[0002] With the continuous improvement of industrial automation, industrial robots are playing an increasingly important role in key manufacturing processes such as welding, assembly, and material handling. As the core moving parts of industrial robots, the health of joints directly determines the overall machine's operational accuracy, stability, and service life. Under complex working conditions of long-term high-speed, heavy-load, and reciprocating motion, key transmission components inside the joints, such as harmonic reducers, bearings, and gears, will inevitably experience wear, loosening, and poor lubrication. These abnormal conditions will first manifest as changes in joint vibration characteristics. If not detected and addressed in time, abnormal vibrations will gradually intensify, eventually leading to joint jamming, loss of accuracy, or even machine shutdown, causing serious economic losses and production safety accidents. Therefore, real-time and accurate abnormal vibration detection and condition identification of industrial robot joints is an important technical means to achieve predictive maintenance and ensure the efficient and stable operation of production lines.

[0003] Existing methods directly analyze raw signals captured within a fixed time window, ignoring joint rotational speed fluctuations. This results in incomplete rotational periods and inconsistent phases across different samples, causing the same physical locations of similar faults to be misaligned on the time axis, affecting the model's stable learning of periodic patterns and local anomalies. Secondly, existing methods simply concatenate triaxial acceleration and triaxial gyroscope signals, failing to utilize kinematic relationships for structured decomposition of multimodal signals. This leads to a mixture of rigid rotational components and anomalous vibration components, making it difficult for the model to distinguish between background and anomalies, increasing the difficulty of feature extraction. Thirdly, existing convolutional neural networks use regular convolutional kernels with fixed sampling positions, making it difficult to adaptively capture non-rigid temporal deformations such as local waveform stretching, compression, abrupt changes, or minor misalignments in anomalous vibration signals, resulting in insufficient ability to extract fine-grained anomaly features related to rotational phase. Finally, existing methods rely solely on cross-entropy loss to optimize category discrimination, ignoring the inherent consistency of similar anomalous samples in temporal morphology. When there is a slight shift or stretching in the time of local impact, essentially similar anomalies are easily misjudged as significantly different, lacking robustness to temporal deformation. Summary of the Invention

[0004] To improve the visibility of anomalous features in the input and accurately capture local anomalous waveforms related to the rotation phase within the period, this invention provides the following technical solution:

[0005] This invention provides a deep learning-based method for detecting abnormal joint vibrations in industrial robots, comprising the following steps:

[0006] S1. Collect raw vibration signals during the operation of the industrial robot. The raw vibration signals include acceleration signals and angular velocity signals, forming a training dataset with category labels.

[0007] S2. The dominant rotation period of each sample is estimated by autocorrelation analysis. Each sample is mapped to a uniform standardized period length by interpolation and resampling to obtain the phase-aligned vibration signal. A decoupled projection matrix is ​​constructed based on the instantaneous angular velocity. The phase-aligned vibration signal is decomposed into dynamic components and residual vibration components. Adaptive enhancement is applied to the residual vibration components and they are spliced ​​with the dynamic components to obtain the enhanced sample.

[0008] S3. Construct a deformable convolutional feature extraction network that integrates physical phase coding. Input the enhanced sample into the deformable convolutional feature extraction network to extract deep temporal feature sequences. The physical phase coding is generated based on the angular velocity signal corresponding to the sample and is used to guide the prediction of the sampling offset of the deformable convolution.

[0009] S4. Construct a temporal classification head based on one-dimensional convolution and global average pooling. Input the deep temporal feature sequence into the temporal classification head, compress and map it to obtain the predicted class probability distribution.

[0010] S5. Construct a consistency constraint for similar features based on dynamic time warping, input a deep temporal feature sequence, obtain a dynamic time warping regularization term, construct an overall loss function based on cross-entropy classification loss and dynamic time warping regularization term, and use the overall loss function to perform backpropagation and parameter update on the model to obtain the trained detection model;

[0011] S6. After the real-time acquired vibration signal to be measured is preprocessed and feature enhanced in step S2, it is input into the trained detection model and the abnormal vibration detection result is output.

[0012] As a further technical solution, S2 specifically includes:

[0013] Obtain the original vibration signal of the sample, and perform mean removal processing on each feature channel of the original vibration signal;

[0014] A preset delay search interval is set. For each candidate delay quantity in the delay search interval, the similarity between the mean-removed sequence and its delayed sequence is calculated to obtain the autocorrelation function. The delay quantity corresponding to the maximum value of the autocorrelation function is taken as the estimated rotation period length of the sample. Based on the estimated rotation period length, a complete period segment is extracted from the original vibration signal.

[0015] Each complete cycle segment is mapped to a uniform standardized time axis to obtain a phase-aligned vibration signal;

[0016] The phase-aligned vibration signal is divided into acceleration sub-signals and angular velocity sub-signals according to the sensor type, including three-axis acceleration signals and three-axis gyroscope signals, which are used as a six-dimensional input vector.

[0017] Construct a decoupled projection matrix, perform projection operation on the six-dimensional input vector using the decoupled projection matrix, extract the dynamic component dominated by rigid rotation from the mixed signal, obtain the dynamic component, and use the complement relationship to separate the residual vibration component from the phase-aligned input signal;

[0018] Based on the ratio of the energy of the dynamic component to the energy of the residual vibration component, the adaptive gain factor corresponding to the generation time is specifically expressed as follows:

[0019]

[0020] In the formula, Indicates the adaptive gain factor. This represents the enhancement intensity coefficient, used to control the gain variation range, and has a value of 2.0. Represents dynamic components Energy; Represents the residual vibration component energy, Represents the numerically stable term;

[0021] The residual vibration component is enhanced step by step using an adaptive gain factor to obtain the enhanced residual vibration component.

[0022] By traversing all time indices within the normalized period, the enhanced residual vibration component sequence is obtained. The dynamic component sequence and the enhanced residual vibration component sequence are then concatenated along the feature dimension to obtain the enhanced sample.

[0023] As a further technical solution, S1 specifically includes:

[0024] The industrial robot runs according to a preset periodic motion trajectory. For each running state, continuous signals are collected, and the continuous signals are truncated using a sliding window method to construct multiple samples with fixed time window lengths.

[0025] Based on the actual operating status of the sample, each sample is assigned a category label, specifically labeled as one of four categories: normal, slight wear, severe wear, or poor lubrication.

[0026] All samples with category labels together constitute the training dataset.

[0027] As a further technical solution, S3 specifically includes:

[0028] Multiple deformable convolutional blocks are stacked sequentially to form a deformable convolutional feature extraction network. A deformable convolutional block is a feature extraction unit that simultaneously includes an offset prediction module, deformable convolution, non-linear activation, and normalization processing.

[0029] Before inputting the feature map into the deformable convolutional block, a physical phase encoding sequence matching its time length is constructed;

[0030] The input feature map and the physical phase encoding sequence are concatenated along the channel dimension to obtain the fused feature.

[0031] The feature input offset prediction module obtains an offset set, and deformable convolution is performed on the input feature map based on the offset set;

[0032] The output of deformable convolution is sequentially input into a nonlinear activation layer and a batch normalization layer to obtain the output of the deformable convolution block;

[0033] Repeat the above process until all deformable convolutional blocks have been processed to obtain the deep temporal feature sequence.

[0034] As a further technical solution, the construction of the physical phase coding sequence includes:

[0035] The triaxial angular velocity sequence is extracted from the original angular velocity channel corresponding to the enhanced sample, the angular velocity magnitude is calculated, and the angular velocity magnitude from the start of the normalized period to the current time is accumulated and summed to obtain the cumulative angular displacement.

[0036] The total angular displacement is obtained by performing full-cycle accumulation of the angular velocity modulus over the entire normalization period, and is expressed as:

[0037]

[0038] In the formula, Represents the total angular displacement over one standard period. Represents the angular velocity modulus at each moment. Indicates the sampling interval. Indicates the length of the modulus sequence;

[0039] Remove the cumulative angular position at the current moment to obtain the total angular displacement, and then take its sine and cosine values ​​to form a two-dimensional physical phase encoding vector.

[0040] Arrange the two-dimensional physical phase encoding vectors corresponding to all time indices in chronological order to obtain the physical phase encoding sequence.

[0041] As a further technical solution, S4 specifically includes:

[0042] The deep temporal feature sequence is input into the temporal classification head, and a one-dimensional convolution is applied to the deep temporal feature sequence to integrate contextual information;

[0043] Perform global average pooling on the output of the one-dimensional convolution to obtain a fixed-length feature vector;

[0044] A fixed-length feature vector is input into a fully connected layer to obtain a category score vector. The category score vector is then normalized to obtain a category probability distribution. The category with the highest probability value in the category probability distribution is taken as the final predicted category.

[0045] As a further technical solution, S5 specifically includes:

[0046] Input multiple samples in batches, divide the samples into multiple category sets, and collect the indexes of all samples belonging to that category in the current batch to form a sample set;

[0047] For any two sample feature sequences in the same sample set, construct a cost matrix and use dynamic programming to search for the dynamic time warped distance;

[0048] The average dynamic time warping distance of samples within the category set is used to obtain the intra-class dynamic time warping cost of that category. The average intra-class dynamic time warping cost of all categories is used to obtain the intra-batch dynamic time warping regularization term.

[0049] The cross-entropy classification loss is calculated based on the true class label and predicted class probability distribution of the sample. The cross-entropy classification loss is weighted and summed with the intra-batch dynamic time regularization term to obtain the overall loss function.

[0050] The model is backpropagated and its parameters are updated using the overall loss function. The optimal parameters are saved to obtain the trained model.

[0051] As a further technical solution, S6 specifically includes:

[0052] The collected continuous real-time signals are extracted into test samples with a fixed time window length in the form of a sliding window.

[0053] The sample to be tested is preprocessed and feature-enhanced by S2 to obtain an enhanced sample;

[0054] The enhanced data is input into the trained detection model to obtain probability distributions for four categories: normal, slight wear, severe wear, and poor lubrication. The category corresponding to the maximum value in the probability distribution is taken as the abnormal vibration detection result.

[0055] This invention also provides a deep learning-based system for detecting abnormal joint vibrations in industrial robots, specifically comprising:

[0056] Data acquisition module: used to collect vibration signals from the joints of industrial robots and build training datasets;

[0057] Data augmentation module: used to make different samples strictly correspond in rotation phase to obtain augmented samples;

[0058] Feature extraction module: used to extract local anomalous waveforms in enhanced samples and construct deep temporal feature sequences;

[0059] Temporal classification module: used to construct the temporal classification head and obtain the final output class probability;

[0060] Model training module: used to construct the overall loss function based on cross-entropy classification loss and dynamic time warping regularization term, and to perform backpropagation and parameter updates on the model;

[0061] Online detection module: used to classify and predict the vibration signals to be measured that are acquired in real time, and output abnormal vibration detection results.

[0062] The present invention also provides a computer-readable storage medium storing a computer program that can be executed by a processor to implement a deep learning-based method for detecting abnormal vibrations of industrial robot joints.

[0063] Beneficial effects of this invention:

[0064] This invention proposes a rotation period synchronization alignment method based on autocorrelation analysis and cubic spline interpolation, effectively eliminating the phase misalignment problem caused by rotational speed fluctuations. This ensures that different samples strictly correspond in rotational phase, laying a stable temporal alignment foundation for subsequent feature extraction. Innovatively, a decoupled projection matrix is ​​constructed using the instantaneous angular velocity of the joints. This matrix decomposes the phase-aligned six-dimensional vibration signal into a dynamic component dominated by rigid rotation and a residual vibration component representing anomalous information. An adaptive gain factor is designed to dynamically enhance the residual vibration component, significantly improving the visibility of anomalous features in the input. A physically guided deformable convolutional feature extraction network is constructed. During offset prediction, a physical phase code generated by the integral of the angular velocity modulus is introduced, allowing the convolutional kernel sampling position to dynamically adjust with the rotational phase. This integrates physical priors into data-driven learning, accurately capturing local anomalous waveforms related to the rotational phase within the period. In classification training, a dynamic time warping regularization term is introduced as a loss constraint to force similar samples to achieve flexible morphological alignment in the time dimension, thereby enhancing the model's adaptability to time stretching, compression, and local misalignment. At the same time, the cross-entropy loss is combined to optimize the class discrimination ability, thereby improving the accuracy and stability of anomaly detection under complex working conditions. Attached Figure Description

[0065] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0066] Figure 1 This is a schematic diagram of the process of the present invention;

[0067] Figure 2 This is a comparison chart of acceleration signals and gyroscope signals;

[0068] Figure 3 A comparison diagram of the original periodic signal and the periodic synchronization aligned signal;

[0069] Figure 4 This is a diagram showing the decoupling of multimodal signals.

[0070] Figure 5 This is a comparison chart of the original signal and the enhanced sample. Detailed Implementation

[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] Example 1

[0073] S1. Acquisition and Sample Construction of Joint Vibration Signals for Industrial Robots Based on Multi-Source Sensing

[0074] Data acquisition and sample construction of joint vibration signals for industrial robots were conducted. During actual operation of the industrial robot, integrated six-dimensional inertial measurement units (IMUs) were installed at its key joints. These units included both a three-axis accelerometer and a three-axis gyroscope, used to sense the linear vibration acceleration and rotational angular velocity of the joints in three-dimensional space, respectively. The data acquisition system continuously recorded the raw signals of the industrial robot under various operating conditions at a fixed high sampling frequency, covering typical working conditions including normal operation, slight wear, severe wear, and poor lubrication.

[0075] During data collection, the industrial robot is made to run along a preset periodic motion trajectory to ensure that each sample contains a complete or several consecutive rotation cycles, so as to preserve the periodic vibration characteristics generated during joint rotation.

[0076] For each operating state, continuous signals of sufficient duration were collected, and the original long-sequence signals were truncated using a sliding window method to construct multiple original samples with fixed time window lengths. Each original sample contains six channels of time-series signals, namely three-axis acceleration signals and three-axis gyroscope signals, which together constitute the multimodal vibration data for subsequent preprocessing and feature extraction.

[0077] While constructing the samples, each sample is assigned a category label based on its actual operating state, specifically labeled as one of four categories: normal, slight wear, severe wear, or poor lubrication. All samples with category labels together constitute the training dataset, which is used for the supervised training of the subsequent deep learning model, enabling the model to learn the nonlinear mapping relationship from multimodal vibration signals to abnormal joint states.

[0078] In one embodiment, Figure 2 The image shows the X-axis acceleration, X-axis angular velocity, and a comparison of acceleration with the X-channel of a gyroscope in the joint data collected from an industrial robot corresponding to a sample. The horizontal axis represents time (s), and the vertical axis represents amplitude. The unit of acceleration is m / s², and the unit of angular velocity is rad / s. The acceleration signal has an impact anomaly (about 0.6-0.65 seconds), which reflects the coexistence of normal operation and sudden anomalies in the original signal. This proves that the data acquisition covers multiple states and provides a foundation for subsequent processing.

[0079] S2. Preprocessing based on rotational periodic synchronization alignment and multimodal signal decoupling enhancement

[0080] The raw vibration signals of industrial robot joints typically exhibit obvious periodic waveform characteristics. Furthermore, there is a physical coupling relationship between the three-axis acceleration signals and the three-axis gyroscope signals, determined by the joint's rotational motion. When directly capturing signals using a fixed-length sliding window, the signals are easily affected by rotational speed fluctuations, resulting in incomplete rotational periods covered by different samples. This leads to inconsistent period phases and misaligned feature positions. Simultaneously, if the three-axis acceleration signals and three-axis gyroscope signals are directly concatenated and input into the subsequent model, it becomes difficult to explicitly distinguish between the stable dynamic components caused by rigid rotation and the abnormal vibration components caused by wear, loosening, and impact, hindering the extraction of abnormal features.

[0081] This invention first estimates the rotation period length independently for each sample, then maps different samples to a standardized periodic coordinate system to achieve synchronous alignment of rotation phases. Next, it uses the instantaneous angular velocity of the joint to construct a multimodal signal decoupling relationship, decomposing the aligned six-dimensional vibration signal into dynamic and residual vibration components. Finally, based on the relative energy relationship between the residual and dynamic components, it adaptively enhances the residual vibration components to obtain enhanced samples more suitable for subsequent anomaly detection. The specific steps are as follows:

[0082] S201, Rotation period estimation and signal synchronization alignment

[0083] The fixed window truncation method assumes that different samples have the same period length. However, industrial robot joints experience slight fluctuations in rotational speed during actual operation, and the actual rotation periods of different samples are usually not completely consistent. If period estimation and phase alignment are not performed first, the same physical position under the same state may fall at different time indices, affecting the subsequent model's stable learning of periodic patterns and local anomalies.

[0084] This invention estimates the dominant rotation period of each sample through autocorrelation analysis, and then maps each sample to a uniform standardized period length through interpolation resampling, thereby ensuring that different samples strictly correspond in rotation phase. The specific steps are as follows:

[0085] (1) Obtain the first The original vibration signal of each sample is denoted as... , Characterizing the first The multimodal vibration sequence corresponding to each sample has a size of [size missing]. ;

[0086] in, This represents the original sampling length, indicating the number of time steps contained in the sample, with a value of 2048. This represents the feature dimension, with a value of 6, corresponding to the three-axis acceleration signal and the three-axis gyroscope signal, respectively.

[0087] Furthermore, the original vibration signal Each feature channel undergoes a mean-reduction process to eliminate the influence of the DC component on the period estimation. Specifically, the mean of each of the six channels is calculated over the entire sampling interval, and then the mean of that channel is subtracted from the sampled value of the corresponding channel at each time step.

[0088] (2) Calculate the first time within the preset delay search interval. The autocorrelation response of each sample is used, where the preset delay search interval refers to the range of sampling points where the dominant cycle may occur, which can be determined based on the common rotational speed range of industrial robot joints combined with the sampling frequency. For example, when the estimated shortest cycle corresponds to 512 sampling points and the longest cycle corresponds to 1024 sampling points, the search interval can be set to... ,in, This represents the lower bound of the delay search, with a value of 512. This indicates the upper limit of the delayed search, with a value of 1024.

[0089] In practical implementation, each candidate latency can be... The autocorrelation function is obtained by calculating the similarity between the mean-reduced sequence and its delayed sequence. , Characterizing the first Each sample has a delay amount The autocorrelation value at a given point indicates that the delay is closer to the true period of the signal. To avoid the impact of differences in amplitude between different samples on period estimation, the autocorrelation function is preferably calculated in a normalized form. That is, the mean of the pointwise product under the corresponding delay is first calculated, and then divided by the square of the signal variance or standard deviation, so that the autocorrelation values ​​of different samples are comparable.

[0090] (3) Select the autocorrelation function from all candidate delays. The delay corresponding to the attainment of the maximum value is taken as the estimated rotation period length of the sample, denoted as . , Characterizing the first The dominant rotation period length of a sample, expressed in units of sampling points, is used to determine the start and end boundaries of a single rotation period for that sample.

[0091] In practical implementation, in the autocorrelation function Within the candidate delay interval, besides the main peak corresponding to the dominant period, there may be secondary peaks caused by signal harmonic components or noise. If multiple local peaks exist within the candidate interval, the delay corresponding to the main peak with the largest amplitude and the most stable position is preferred. Multi-window verification can be used to determine whether the main peak's position is stable. Specifically, the sample... The time axis is divided into two or three overlapping segments. Autocorrelation analysis is performed independently on each segment, and local peak values ​​are extracted. If a candidate delay... If a significant local peak is observed in the analysis results of multiple segments, then the period corresponding to that delay has high stability; conversely, if it only appears in a specific window, it is considered unstable.

[0092] Furthermore, based on the estimated rotation period length From the original vibration signal Extracting a complete periodic segment from the data, where a complete periodic segment is defined as one whose length between the start and end points is approximately equal to the specified length. The rotation interval; in order to reduce the boundary truncation error, a small number of sampling points can be finely adjusted forward or backward from the starting point of the period obtained from autocorrelation analysis so that the period boundary falls better at the waveform repetition position.

[0093] (4) Map each complete period segment to a uniform standardized time axis to obtain the phase-aligned vibration signal. , Characterizing the first The vibration signal of each sample after synchronous alignment during rotation period, with a size of [missing information]. ;in, This represents the standardized period length, indicating the number of sampling points that all samples are uniformly assigned to the same period, with a value of 1024.

[0094] In the specific implementation, the coordinates of discrete sampling points on the original period segment are used as the source time axis, and the length is... The uniform sampling point sequence is used as the target time axis, and then cubic spline interpolation is used to resample each feature channel. Cubic spline interpolation is used to smoothly reconstruct the original period segment without changing the overall waveform trend, so that periods of different lengths can be mapped to a uniform length. Moreover, compared with linear interpolation, cubic spline interpolation is smoother near the peak and can better preserve the details of the period waveform.

[0095] It should be noted that when the original sample contains multiple consecutive cycles, each cycle can be resampled separately first, and then the average of the multiple resampled cycles can be taken to obtain the phase-aligned vibration signal. It can characterize the typical vibration modes of the sample within a standardized rotational period.

[0096] Rotation period estimation and phase alignment are prerequisites for subsequent multimodal signal decoupling. Only when all samples are in a unified rotating phase coordinate system can the dynamic components and abnormal vibration components be stably comparable between different samples, thereby improving the temporal alignment of similar samples during subsequent feature extraction, making periodic stable features more concentrated and abnormal local distortions easier to identify.

[0097] In one embodiment, Figure 3 The diagram shows the period estimation and synchronization alignment. The top subplot is the acceleration X channel of the original period segment (unaligned), with the horizontal axis representing time (seconds) and the vertical axis representing acceleration (m / s²). The bottom subplot is the standardized period signal after autocorrelation period estimation and cubic spline interpolation resampling, with the horizontal axis representing the unified time axis (seconds) and the vertical axis representing acceleration (m / s²). Through period estimation and interpolation alignment, original period segments of different lengths in different samples can be mapped to the same standardized period length, making the periodic waveform strictly correspond in phase.

[0098] S202, Multimodal signal decoupling and enhancement based on joint kinematic constraints

[0099] Even after phase alignment, the triaxial acceleration signal and the triaxial gyroscope signal still contain both normal dynamic components caused by rigid rotation and abnormal vibration components caused by factors such as wear, loosening, and collisions. Without structured decomposition, subsequent models need to simultaneously identify stable backgrounds and anomalous disturbances from the mixed signals, which presents a significant learning challenge.

[0100] Based on the kinematic relationship of joint rotation, this invention constructs a decoupled projection matrix, decomposes the aligned six-dimensional signal into dynamic components and residual vibration components, and then applies adaptive enhancement to the residual vibration components to highlight abnormal information. The specific steps are as follows:

[0101] (1) Vibration signal after phase alignment Based on sensor type, the signals are divided into acceleration sub-signals and angular velocity sub-signals. The first three channels are triaxial acceleration signals, denoted as... , indicating the first Each sample at time... The three-axis acceleration signal vector, where, , , These represent the time intervals of the sample. along , , Linear acceleration components in the direction, in units of This reflects the vibration intensity of the joint in three orthogonal directions; the latter three channels are three-axis gyroscope signals, denoted as... , indicating the first Each sample at time... The three-axis angular velocity vector, where, , , These represent the time intervals of the sample. Around , , The rotational angular velocity component of the shaft, in units of This reflects the rotational state of the joint at that moment.

[0102] For each time index (The range of values ​​is) to Extraction time instantaneous angular velocity vector , Characterizing the first Each sample at time... The three-dimensional angular velocity vector reflects the rotational state of the joint at that moment and is the basis for constructing the dynamic projection relationship.

[0103] (2) Based on the instantaneous angular velocity vector Constructing the decoupled projection matrix , Characterizing time The corresponding decoupled projection matrix has a size of It is used to extract the dynamic components related to rigid rotational motion from six-dimensional multimodal vibration signals.

[0104] In one implementation, first utilize... Its own transpose The direction correlation matrix is ​​then embedded into a six-dimensional space to describe the dynamic projection relationship in the acceleration signal that is consistent with the rotation direction. Specifically, the square magnitude of the instantaneous angular velocity vector is first calculated, i.e. Then Divide by This yields a normalized direction projection term, which represents the dominant direction of the current rotation in the three-dimensional acceleration space. This projection term is then arranged within the decoupled projection matrix. The acceleration component corresponding to the top left corner is filled with 0 in the remaining positions unrelated to the gyroscope residual, thus constructing... The dynamic-related parts of the six-dimensional input can be projected out, where, This represents the numerical stability term, used to prevent the denominator from approaching 0 due to excessively small angular velocities.

[0105] (3) Using the decoupled projection matrix For time The six-dimensional input vector is projected to extract the dynamic components dominated by rigid rotation from the mixed signal, thus obtaining the dynamic components. , Characterizing the first Each sample at time... The dynamic components, with a dimension of 3, contain a dominant component consistent with the rigid rotation law of the joint.

[0106] In practical implementation, the first step is to set the time... triaxial acceleration vector With the three-axis angular velocity vector By concatenating the elements, a six-dimensional column vector is formed. Then decouple the projection matrix (size is) ) and six-dimensional input vector Multiplying them yields a three-dimensional column vector, which represents the dynamic components. .

[0107] Furthermore, by sequentially arranging the dynamic components corresponding to all time indices within a standardized period, the dynamic component sequence is obtained. The size is .

[0108] (4) The residual vibration component is separated from the aligned input signal using the complement relationship to obtain... , Characterizing the first Each sample at time... The residual vibration components, with a dimension of 3, mainly contain non-rigid vibration information that cannot be explained by rigid rotational dynamics, such as micro-impacts caused by wear, additional vibrations caused by loosening, and local abnormal fluctuations caused by poor lubrication.

[0109] In one implementation, the residual vibration components are obtained by subtracting the dynamic projection result from the original aligned signal; specifically, for each time index... First, the dynamic components are obtained, and then the corresponding dynamic parts in the original six-dimensional input are removed. The parts not explained by the rigid rotation model are retained as residual vibration components, so that the obtained residual vibration components can more effectively represent the anomalous information.

[0110] Furthermore, by sequentially arranging the residual vibration components corresponding to all time indices within a standardized period, the residual vibration component sequence is obtained. The size is .

[0111] (5) Calculate the instantaneous energy for the dynamic component and the residual vibration component respectively. The instantaneous energy refers to the energy of the residual vibration component or the energy of the dynamic component, which can be represented by the sum of squares of each component of the vector. Specifically, for each time index Calculate the dynamic components The energy, and the residual vibrational components If the residual vibrational component has relatively large energy, it usually means that there is a more obvious abnormal disturbance at that moment.

[0112] Furthermore, based on the dynamic components Energy and residual vibration components The ratio between energies, generation time Corresponding adaptive gain factor , Characterizes the amplification factor of the residual vibration components;

[0113] In one implementation, the adaptive gain factor can be made to satisfy a monotonic relationship where the higher the residual energy ratio, the greater the amplification factor, expressed as:

[0114]

[0115] In the formula, Indicates the adaptive gain factor. This represents the enhancement intensity coefficient, used to control the gain variation range, and has a value of 2.0. Represents dynamic components Energy; Represents the residual vibration component energy, This represents the numerical stability term, used to prevent the denominator from approaching 0 due to excessively small angular velocities. Its value is... .

[0116] (6) Using adaptive gain factor The residual vibration component is enhanced moment by moment to obtain the enhanced residual vibration component. , Characterizing the first Each sample at time... The enhanced residual vibration component, with a dimension of 3, is used to amplify local features at anomalous moments.

[0117] In practical implementation, the time can be... The corresponding residual vibration component vectors are multiplied by a scalar gain factor. This preserves the directional information while enhancing the amplitude expression.

[0118] (7) Traverse all time indices within the normalized period to obtain the enhanced residual vibration component sequence. The size is .

[0119] Furthermore, the dynamic component sequence With the enhanced residual vibration component sequence By concatenating along the feature dimensions, we obtain the enhanced sample. , Characterizing the first The output of each sample after multimodal signal decoupling and residual enhancement has a size of [size missing]. .

[0120] It should be noted that this invention does not simply fuse the triaxial acceleration signal and the triaxial gyroscope signal directly. Instead, it processes the stable background component related to normal rotation separately from the abnormal disturbance component that has more diagnostic value. Specifically, it first establishes a decoupling relationship under physical constraints based on the instantaneous angular velocity of the joint, and then adaptively enhances the non-rigid residual vibration component. Based on this, on the one hand, it can preserve the dynamic structural information of the joint under normal motion, and on the other hand, it can significantly improve the visibility of abnormal vibration features such as wear, loosening, and impact in the input, thereby improving the ability of the subsequent classification model to identify abnormal states.

[0121] In one embodiment, Figure 4The diagram shows the decoupling of multimodal signals. The first sub-plot compares the original acceleration X-channel with the dynamic component, with the horizontal axis representing time (seconds) and the vertical axis representing amplitude (m / s²). The dynamic component is obtained by projecting the angular velocity direction, reflecting the stable components related to rigid rotation. The second sub-plot shows the residual vibration component, which is the remaining part after subtracting the dynamic component from the original acceleration, both in m / s², mainly containing abnormal information such as wear and impact. The third sub-plot shows the instantaneous ratio (dimensionless) of residual energy to dynamic energy, reflecting the relative intensity of abnormal disturbances at each moment. It can be seen that the decoupling projection matrix constructed based on angular velocity can effectively separate the rigid rotation background and abnormal vibration.

[0122] In one embodiment, Figure 5 The image shows a comparison of residual vibration enhancement. The top sub-image compares the residual vibration component (acceleration X channel) before and after enhancement, with the horizontal axis representing time (seconds) and the vertical axis representing amplitude (m / s²). Red represents the original residual, and magenta represents the enhanced residual after adaptive gain amplification. It can be seen that the amplitude is significantly increased at the abnormal impact location (approximately 0.6 seconds). The bottom sub-image compares the enhanced acceleration signal with the original aligned signal, with the horizontal axis representing time (seconds) and the vertical axis representing acceleration (m / s²). Green represents the enhanced sample, and the black dashed line represents the original aligned signal. The adaptive gain factor can dynamically amplify the residual component at the abnormal moment according to the ratio of residual to kinetic energy, making the abnormal features more prominent in the input, demonstrating the improvement of the subsequent classification model's recognition ability through enhancement processing.

[0123] S3. Construction of a deformable convolutional feature extraction network based on rotational physical information

[0124] The preprocessed enhanced samples have a unified rotation phase expression and clearer anomalous vibration components. However, intra-period anomalies are usually manifested as local waveform stretching, compression, abrupt changes, or minor misalignments. Conventional fixed convolution kernels only sample at regular sampling locations, making it difficult to adaptively capture such non-rigid temporal deformations.

[0125] This invention constructs a deformable convolutional feature extraction network and introduces physical phase encoding generated from angular velocity information during the offset prediction process. This allows the sampling position of the convolutional kernel to be dynamically adjusted with the rotation phase, thereby extracting abnormal morphological features within the period more accurately. The specific steps are as follows:

[0126] S301. Construct a deformable convolutional feature extraction network

[0127] Constructing a deep network consisting of multiple deformable convolutional blocks stacked sequentially, i.e., a deformable convolutional feature extraction network, will enhance the sample... As input to the deformable convolutional feature extraction network, the total number of network layers is denoted as... , This indicates the number of deformable convolutional blocks, with a value of 4.

[0128] Among them, deformable convolutional blocks refer to feature extraction units that simultaneously include offset prediction, deformable convolution, nonlinear activation, and normalization processing.

[0129] The input to the first deformable convolutional block is defined as and order ; for the first A deformable convolutional block, whose input is denoted as . The output is denoted as ,in The value range is 1 to ;

[0130] After all After processing by deformable convolutional blocks, a deep temporal feature sequence is obtained. , The final extracted deep temporal feature sequence is characterized by a size of ,in, This represents the time length after several layers of convolution and optional downsampling. This represents the final number of feature channels.

[0131] In one implementation, each deformable convolutional block may contain one deformable one-dimensional convolutional layer, one batch normalization layer, and one activation layer. If necessary, strided convolution or pooling operations may be added after some deformable convolutional blocks to gradually expand the receptive field and compress the time length. The convolutional kernel size may be 3, 5, or 7, and the number of output channels may be gradually increased according to the number of layers, for example, set to 32, 64, 128, and 256 respectively.

[0132] It should be noted that the output of a deformable convolutional feature extraction network is not a single global feature vector, but rather a deep feature sequence that preserves the temporal order. Because the temporal morphological consistency between feature sequences is still needed for constraint in the subsequent training phase, the temporal dimension information cannot be lost too early.

[0133] S302, Physical Information-Guided Offset Generation and Deformable Convolution Feature Extraction

[0134] (1) In the data processing of deformable convolutional feature extraction networks, the input to the first Feature map of a deformable convolutional block Construct a physical phase encoding vector that matches its time length. , The physical phase code, generated from joint angular velocity information, represents the rotational progress at each time position within the normalized period, providing a physical prior for offset prediction. Specifically...

[0135] <1.1> Extract the triaxial angular velocity sequence from the original angular velocity channel corresponding to the enhanced sample or intermediate feature. , Characterizing time within the standardized period The three-dimensional angular velocity vector, composed of three-axis gyroscope signals, is used to describe the rotation speed and rotation state at that moment.

[0136] <1.2> For each time index Calculate the angular velocity vector Length of the module The modulus refers to the square root of the sum of the squares of the three-axis angular velocity components, which is used to characterize the overall rotational intensity at that moment.

[0137] Furthermore, regarding the period from the start of the standardization cycle to the current moment... The cumulative angular displacement is obtained by summing or numerically integrating the angular velocity moduli between the two points. This cumulative angular displacement represents the total rotation progress from the start position of the cycle to the current position.

[0138] <1.3> Perform full-cycle accumulation of the angular velocity modulus over the entire normalization period to obtain the total angular displacement. , It represents the total rotational angular displacement within a standardized period and is used to normalize the cumulative angular displacement to the relative rotational progress. To avoid numerical instability caused by the total angular displacement being too small, a very small constant can be added to the denominator.

[0139] In practical implementation, full-cycle accumulation refers to numerically integrating the angular velocity magnitude at all moments within the entire standardized period to obtain the total angular displacement within one standardized period. Specifically, for time indexes within a standardized period Then calculate the angular velocity modulus at each moment. The length is obtained as The modulus sequence is obtained, and then the total angular displacement is obtained by summing the modulus sequences. Because the time index is discrete and the sampling interval is... A fixed value (based on the sampling frequency) Decision, that is The total periodic accumulation is obtained by calculating the weighted sum of the modulo-length sequences, and is expressed as:

[0140]

[0141] In the formula, Represents the total angular displacement over one standard period. Represents the angular velocity modulus at each moment. Indicates the sampling interval. This indicates the length of the modulus sequence.

[0142] <1.4> Remove the cumulative angle at the current moment to obtain the total angle displacement. The normalized rotation progress between 0 and 1 is obtained, and its sine and cosine values ​​are taken respectively to form a two-dimensional physical phase encoding vector.

[0143] That is, for each time index This generates an encoded result of length 2, where the first dimension represents the sine phase and the second dimension represents the cosine phase.

[0144] Wherein, the cumulative angular displacement at the current moment refers to the displacement from the beginning of the normalized period ( (Up to the current time) The sum of the rotational progress, that is, for from Up to now Perform cumulative summation (multiply by) (Perform physical unit conversion).

[0145] It should be noted that the use of sine and cosine joint encoding is to eliminate the discontinuity problem of single phase representation at the start and end positions of the cycle, so that adjacent rotation positions maintain a smooth transition in the encoding space.

[0146] <1.5> Arrange the two-dimensional physical phase codes corresponding to all time indices in chronological order to obtain a size of... Physical phase encoding sequence When the first When the time length of the input feature map of a deformable convolutional block has been shortened to other lengths, linear interpolation, average pooling, or downsampling consistent with the convolution stride can be performed on the original physical phase encoding sequence to make its time length similar to that of the current input feature map. Consistent.

[0147] (2) Input feature map With physical phase encoding sequence By concatenating along the channel dimension, a fused feature is obtained. Concatenation along the channel dimension means that at each time position, the original feature vector is combined with the corresponding two-dimensional physical phase encoding vector to form a longer feature vector, so as to form an input that simultaneously contains data-driven features and rotational physical priors.

[0148] (3) Input the fused features into the offset prediction convolutional layer to obtain the set of offsets required for deformable convolution. , Characterizing the first The sampling offset generated by each deformable convolutional block is used to adjust the actual sampling position of the convolutional kernel on the time axis;

[0149] In the specific implementation, the offset prediction convolutional layer adopts a one-dimensional convolutional structure. The kernel size can be the same as or smaller than that of the main convolutional layer. The number of output channels is equal to "the number of sampling points multiplied by the offset dimension of each sampling point". Since this invention is mainly based on time series processing, the sampling offset mainly changes along the time dimension. Therefore, each sampling point usually only needs to output 1 time offset. The offset can be positive, negative or 0, representing backward offset, forward offset and no offset, respectively.

[0150] In one implementation, if the deformable convolutional kernel size is 3, the offset prediction convolutional layer can output 3 offsets for each time position, corresponding to the displacement of 3 sampling points of the convolutional kernel respectively; if a larger convolutional kernel is used, the number of output offsets is consistent with the number of sampling points of the convolutional kernel.

[0151] (4) Based on the offset set For the input feature map Perform deformable convolution, where deformable convolution means that the convolution kernel is no longer fixed at regular integer time positions, but dynamically adjusted according to the actual position of each sampling point based on the offset. For example, when the convolution kernel originally samples near the current time position, if the offset of a certain sampling point is 0.6, it means that the sampling point actually takes values ​​from a continuous position between two adjacent discrete time points.

[0152] Meanwhile, since the sampling position after offset is generally not an integer time index, in the specific implementation, it is necessary to use interpolation to read the corresponding continuous position feature values ​​from the input feature map. For one-dimensional time series, linear interpolation is preferred, but higher-order interpolation can also be used if the implementation framework supports it. Interpolation can ensure that the offset sampling process is continuous and differentiable, which is convenient for optimizing the offset prediction parameters and main convolution parameters through backpropagation during network training.

[0153] (5) Input the output obtained from deformable convolution into the nonlinear activation layer and the batch normalization layer in sequence to obtain the first... The output of a deformable convolutional block In this process, the nonlinear activation layer can use the ReLU activation function, and the batch normalization layer is used to stabilize training and improve convergence speed.

[0154] (6) Repeat the above process until all are completed. After processing each deformable convolutional block, a deep temporal feature sequence is obtained. .

[0155] It should be noted that abnormal morphological changes in the joint vibration signals of industrial robots are often related to specific rotation phases. However, simply relying on data-driven offset learning can easily overlook this physical prior. This invention introduces physical phase encoding based on angular velocity information in the offset generation process of deformable convolution. This makes the sampling position of the convolution kernel not only determined by the data itself, but also guided by the physical law of rotation progress. Based on this, the convolution kernel can adaptively focus on local abnormal waveforms near the key phase within the cycle, which is beneficial for capturing fine-grained abnormal features such as slight wear, impact, and local distortion.

[0156] S4. Training and classification based on periodic features using dynamic time warping

[0157] The deep temporal feature sequences obtained through deformable convolutional feature extraction networks already contain rich morphological information within the period, but local temporal stretching or compression may still exist between similar anomalous samples. If only conventional classification loss is used for training, the model may only focus on global class separability and ignore the temporal morphological consistency of similar samples.

[0158] This invention introduces dynamic time warping constraints during classification training to make the feature sequences of samples of the same class more similar in the time dimension. Simultaneously, a temporal classification head is constructed to output the final class probability. The specific steps are as follows:

[0159] S401. Consistency Constraints for Similar Features Based on Dynamic Time Warping

[0160] (1) During the model training phase, multiple samples are input in batches. Let the current training batch size be... That is, the number of samples input into the network in each batch, which is 32.

[0161] After preprocessing and the deformable convolutional feature extraction network, each sample in the batch corresponds to a deep temporal feature sequence, denoted as... , Characterizing the first The deep temporal feature sequence of each sample, with a size of [size missing]. .

[0162] (2) Based on the true category labels of the samples in the current batch, divide the samples into multiple category sets, assuming the total number of categories is . , The number of categories in the anomaly classification task is represented by a value of 4, which includes four categories: normal, slight wear, severe wear, and poor lubrication.

[0163] For each category Collect the indexes of all samples belonging to this category in the current batch to form a sample set. That is, the category in the current batch. The corresponding sample set.

[0164] Furthermore, for sets of the same category In the sample feature sequences, the dynamic time warping distance is calculated pairwise. The dynamic time warping distance is a distance metric that calculates the minimum cumulative matching cost under the condition that the two sequences are allowed to be non-linearly aligned in time. It is used to measure the morphological similarity between two time-series feature sequences. The smaller the distance, the more similar the two feature sequences are.

[0165] In practical implementation, for any two similar sample feature sequences and First, construct a two-dimensional cost matrix. Each element in the cost matrix represents the difference between the feature vector at a certain time position in one sequence and the feature vector at a certain time position in another sequence. This difference can be calculated using the squared Euclidean distance, that is, by taking the difference, squaring and summing the two feature vectors channel by channel.

[0166] (3) Based on the cost matrix, dynamic programming is used to search for the optimal alignment path that satisfies the boundary constraints and monotonic constraints. The boundary constraints mean that the alignment path must start from the beginning position of the two sequences and reach the end position of the two sequences. The monotonic constraints mean that the path cannot backtrack on either time axis.

[0167] By accumulating the minimum cost through dynamic programming, the dynamic time-warped distance between the two feature sequences can be obtained.

[0168] (4) For the set of categories The average dynamic time warping distance of all sample pairs within the class is used to obtain the intra-class dynamic time warping cost for that class.

[0169] Furthermore, the intra-class dynamic time warping costs for all categories are averaged to obtain the intra-batch dynamic time warping regularization term. , It represents the average morphological difference of the feature sequences of similar samples in the current training batch. The smaller the value, the stronger the morphological consistency of similar samples in the time dimension.

[0170] In practical implementation, to control the amount of computation, it is not necessary to perform full calculation on all sample pairs in the same set. Instead, some sample pairs can be randomly selected, or only the dynamic time warp distance can be calculated on the nearest neighbor sample pairs in the same set. Alternatively, the feature sequence can be appropriately downsampled before calculating the dynamic time warp distance to reduce the time and space overhead during training.

[0171] It should be noted that although abnormal joint vibrations in industrial robots belong to the same category, the specific time and location of local impacts may have slight shifts or stretching. If rigid alignment comparison is directly used for time-point comparison, it is easy to misjudge essentially similar anomalies as having large differences. Dynamic time warping allows similar features to be flexibly matched on the time axis, thus more realistically constraining similar samples to learn similar temporal morphological structures. Therefore, introducing dynamic time warping regularization terms can effectively improve the robustness of feature representations to time stretching, compression, and local misalignment.

[0172] S402, Temporal Classification Prediction and Construction of Overall Loss Function

[0173] (1) Deep temporal feature sequences Input the time series classification header and generate the first... The class prediction results for each sample, where the temporal classification head is used to further compress and map the deep features that preserve the temporal structure into probabilities for each class. Specifically,

[0174] <1.1> Deep temporal feature sequences Apply one-dimensional convolution processing. One-dimensional convolution is used to further integrate contextual information within a local time window, so that the final classification decision does not depend on a single time position, but considers short-term neighborhood features. The convolution kernel size can be 3, and the number of output channels can be set according to the capacity requirements of the classification head.

[0175] <1.2> Global average pooling is performed on the one-dimensional convolution output to obtain a fixed-length feature vector. Global average pooling means averaging the feature values ​​across all time points along the time dimension, thus generating a feature vector of length [missing information]. The time-series features are compressed into a single vector, reducing the number of parameters and the risk of overfitting, while retaining the overall statistical features.

[0176] <1.3> Input the fixed-length feature vector obtained by global average pooling into the fully connected layer, and the output length is... The class score vectors are then processed by performing Softmax normalization to obtain the class probability distributions. , Characterizing the first The class probability distribution of each sample, with dimension . The sum of the probabilities of each dimension is 1, and the category with the highest probability value can be used as the final predicted category.

[0177] (2) Based on the true labels of the samples With the predicted category probability distribution The cross-entropy classification loss is calculated to measure the difference between the model's prediction and the true label. A higher predicted probability for the true class results in a smaller cross-entropy loss. Indicates the first The true class label of each sample.

[0178] (3) Combine the cross-entropy classification loss with the dynamic time regularization term. We perform a weighted summation to obtain the overall loss function. , represented as:

[0179]

[0180] In the formula, Represents the cross-entropy classification loss. This represents the balance coefficient, used to control the contribution of the dynamic time-warping regularization term to the overall loss, and has a value of 0.1. This represents a dynamic time-normalized regularization term.

[0181] (4) Using the overall loss function Backpropagation and parameter updates are performed on the entire model, which includes all trainable parameters such as the preprocessed deformable convolutional feature extraction network, the physically guided offset prediction module, and the temporal classification head.

[0182] In practice, parameter updates can employ stochastic gradient descent, Adam optimization, or other commonly used deep learning optimization methods.

[0183] It should be noted that, in order to enable the model to have both the ability to classify and the ability to model the common structural features of similar abnormal waveforms, this invention does not rely solely on the final classification result for training. Instead, it simultaneously constrains the morphological consistency of similar samples in the deep temporal feature space, thereby improving the model's adaptability to the internal temporal deformation of similar abnormal samples and enhancing the accuracy and stability of detecting and classifying abnormal vibrations of industrial robot joints under complex working conditions.

[0184] S5, Industrial Robot Joint Abnormal Vibration Detection

[0185] After completing the offline training of the above model, it can be applied to the real-time abnormal vibration detection task of industrial robot joints. In the detection phase, the real-time six-dimensional vibration signals of the industrial robot joint to be detected are first acquired online using the same data acquisition method as in the training phase, namely, the three-axis acceleration signal and the three-axis gyroscope signal.

[0186] The acquired continuous real-time signals are truncated into test samples with a fixed time window length using a sliding window. These test samples are then input into a pre-processing module that has been trained. Phase calibration based on rotational period synchronization and multi-modal signal decoupling and enhancement processing based on joint kinematic constraints are performed sequentially to obtain enhanced samples with the same standardized period length as the training samples, but with amplified abnormal vibration components. The enhanced samples are then input into a pre-trained deformable convolutional feature extraction network. This network uses physical phase encoding generated from angular velocity information to guide offset prediction, enabling the convolutional kernel sampling position to adaptively capture local abnormal waveforms related to the rotational phase in the test samples. The final output is a deep temporal feature sequence reflecting the temporal morphology within the period. Subsequently, the extracted deep temporal feature sequence is input into a temporal classification head. After one-dimensional convolution to integrate local context and global average pooling compression, a fixed-length feature vector is obtained. Then, through fully connected layers and normalization processing, the probability distribution of the test samples belonging to four categories—normal, slight wear, severe wear, and poor lubrication—is calculated. Finally, the category corresponding to the maximum value in the probability distribution is taken as the abnormal vibration detection result of the sample to be tested, and the detection result is output to the monitoring system or operation and maintenance platform of the industrial robot to judge the current health status of the joint in real time, and to provide a quantitative basis for subsequent early warning maintenance and fault diagnosis.

[0187] Example 2

[0188] A deep learning-based system for detecting abnormal joint vibrations in industrial robots, specifically comprising:

[0189] Data acquisition module: used to collect vibration signals from the joints of industrial robots and build training datasets;

[0190] Data augmentation module: used to make different samples strictly correspond in rotation phase to obtain augmented samples;

[0191] Feature extraction module: used to extract local anomalous waveforms in enhanced samples and construct deep temporal feature sequences;

[0192] Temporal classification module: used to construct the temporal classification head and obtain the final output class probability;

[0193] Model training module: used to construct the overall loss function based on cross-entropy classification loss and dynamic time warping regularization term, and to perform backpropagation and parameter updates on the model;

[0194] Online detection module: used to classify and predict the vibration signals to be measured that are acquired in real time, and output abnormal vibration detection results.

[0195] Example 3

[0196] A computer-readable storage medium storing a computer program that can be executed by a processor to implement a deep learning-based method for detecting abnormal vibrations in the joints of an industrial robot.

[0197] Readable storage media include: non-volatile memory: flash memory, ROM, EPROM, EEPROM; volatile memory: random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM); disk storage media: hard disk (HDD), floppy disk; optical storage media: optical disc (CD, DVD, Blu-ray disc), etc.

[0198] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting abnormal joint vibrations in industrial robots based on deep learning, characterized in that, Includes the following steps: S1. Collect raw vibration signals during the operation of the industrial robot. The raw vibration signals include acceleration signals and angular velocity signals, forming a training dataset with category labels. S2. The dominant rotation period of each sample is estimated by autocorrelation analysis. Each sample is mapped to a uniform standardized period length by interpolation and resampling to obtain the phase-aligned vibration signal. A decoupled projection matrix is ​​constructed based on the instantaneous angular velocity. The phase-aligned vibration signal is decomposed into dynamic components and residual vibration components. Adaptive enhancement is applied to the residual vibration components and they are spliced ​​with the dynamic components to obtain the enhanced sample. S3. Construct a deformable convolutional feature extraction network that integrates physical phase coding, input the enhanced samples into the deformable convolutional feature extraction network, and extract deep temporal feature sequences; The physical phase encoding is generated based on the angular velocity signal corresponding to the sample and is used to guide the prediction of the sampling offset in deformable convolution. The construction of the physical phase encoding sequence includes: S311. Extract the triaxial angular velocity sequence from the original angular velocity channel corresponding to the enhanced sample, calculate the angular velocity magnitude, and sum the angular velocity magnitudes from the start of the standardized period to the current time to obtain the cumulative angular displacement; S312. Perform full-cycle accumulation of the angular velocity modulus over the entire normalization period to obtain the total angular displacement, expressed as: In the formula, Represents the total angular displacement over one standard period. This represents the magnitude of the angular velocity at each moment. Indicates the sampling interval. Indicates the length of the modulus sequence; S313. Remove the cumulative angular position at the current moment to obtain the total angular displacement, then take its sine and cosine values ​​respectively to form a two-dimensional physical phase encoding vector; S314. Arrange the two-dimensional physical phase encoding vectors corresponding to all time indices in chronological order to obtain the physical phase encoding sequence; S4. Construct a temporal classification head based on one-dimensional convolution and global average pooling. Input the deep temporal feature sequence into the temporal classification head, compress and map it to obtain the predicted class probability distribution. S5. Construct a consistency constraint for similar features based on dynamic time warping, input a deep temporal feature sequence, obtain a dynamic time warping regularization term, construct an overall loss function based on cross-entropy classification loss and dynamic time warping regularization term, and use the overall loss function to perform backpropagation and parameter update on the model to obtain the trained detection model; S6. After the real-time acquired vibration signal to be measured is preprocessed and feature enhanced in step S2, it is input into the trained detection model and the abnormal vibration detection result is output.

2. The method for detecting abnormal joint vibration of industrial robots based on deep learning according to claim 1, characterized in that, S2 specifically includes: S201. Obtain the original vibration signal of the sample, and perform mean removal processing on each feature channel of the original vibration signal; S202. Preset a delay search interval, calculate the similarity between the mean-removed sequence and its delayed sequence for each candidate delay amount in the delay search interval, obtain the autocorrelation function, take the delay amount corresponding to the maximum value of the autocorrelation function as the estimated rotation period length of the sample, and extract the complete period segment from the original vibration signal based on the estimated rotation period length. S203. Map each complete cycle segment to a unified standardized time axis to obtain the phase-aligned vibration signal; S204. The phase-aligned vibration signal is divided into acceleration sub-signals and angular velocity sub-signals according to the sensor type, including three-axis acceleration signals and three-axis gyroscope signals, which are used as a six-dimensional input vector; S205. Construct a decoupled projection matrix based on the instantaneous angular velocity vector, perform projection operation on the six-dimensional input vector using the decoupled projection matrix, extract the dynamic component dominated by rigid rotation from the mixed signal, and separate the residual vibration component from the phase-aligned input signal using the complement relationship; S206. Based on the ratio of the energy of the dynamic component to the energy of the residual vibration component, an adaptive gain factor corresponding to the time is generated, specifically expressed as follows: In the formula, Indicates the adaptive gain factor. This represents the enhancement intensity coefficient, used to control the gain variation, and has a value of 2.

0. Represents dynamic components Energy; Represents the residual vibration component energy, Represents the numerically stable term; The residual vibration component is enhanced step by step using an adaptive gain factor to obtain the enhanced residual vibration component. S207. Traverse all time indices within the normalized period to obtain the enhanced residual vibration component sequence. The dynamic component sequence and the enhanced residual vibration component sequence are concatenated along the feature dimension to obtain the enhanced sample.

3. The method for detecting abnormal joint vibration of industrial robots based on deep learning according to claim 1, characterized in that, S1 specifically includes: S101. The industrial robot runs according to a preset periodic motion trajectory. For each running state, continuous signals are collected, and the continuous signals are truncated using a sliding window method to construct multiple samples with fixed time window lengths. S102. Based on the actual operating status of the sample, assign a category label to each sample, specifically labeling it as one of the four categories: normal, slight wear, severe wear, or poor lubrication. S103. All samples with class labels together constitute the training dataset.

4. The method for detecting abnormal joint vibration of industrial robots based on deep learning according to claim 1, characterized in that, S3 specifically includes: S301. A deformable convolutional block simultaneously contains an offset prediction module, deformable convolution, non-linear activation and normalization feature extraction unit, and multiple deformable convolutional blocks are stacked sequentially to form a deformable convolutional feature extraction network. S302. Input Feature Map Before inputting deformable convolutional blocks, construct a physical phase encoding sequence that matches their time length; S303. The input feature map and the physical phase encoding sequence are concatenated along the channel dimension to obtain the fused feature; S304. The feature input offset prediction module is fused to obtain an offset set, and deformable convolution is performed on the input feature map based on the offset set; S305. The output obtained from deformable convolution is sequentially input into a nonlinear activation layer and a batch normalization layer to obtain the output of the deformable convolution block; S306. Repeat the above process until all deformable convolutional blocks have been processed to obtain the deep temporal feature sequence.

5. The method for detecting abnormal joint vibration of industrial robots based on deep learning according to claim 1, characterized in that, S4 specifically includes: S401. The temporal classification head receives deep temporal feature sequences, applies one-dimensional convolution processing to the deep temporal feature sequences, and integrates contextual information; S402. Perform global average pooling on the output of the one-dimensional convolution to obtain a fixed-length feature vector; S403. Input the fixed-length feature vector into the fully connected layer to obtain the category score vector; perform normalization on the category score vector to obtain the category probability distribution, and take the category with the largest probability value in the category probability distribution as the final predicted category.

6. The method for detecting abnormal joint vibration of industrial robots based on deep learning according to claim 1, characterized in that, S5 specifically includes: S501. Input multiple samples in batches, divide the samples into multiple category sets, and for each category, collect the indexes of all samples belonging to that category in the current batch to form a sample set; S502. For any two sample feature sequences in the same sample set, construct a cost matrix and use dynamic programming to search for the dynamic time warped distance; S503. Average the dynamic time warping distances corresponding to samples within the category set to obtain the intra-class dynamic time warping cost of that category. Average the intra-class dynamic time warping costs of all categories to obtain the intra-batch dynamic time warping regularization term. S504. Calculate the cross-entropy classification loss based on the true class label and predicted class probability distribution of the sample. The cross-entropy classification loss is weighted and summed with the intra-batch dynamic time regularization term to obtain the overall loss function. S505. Perform backpropagation and parameter updates on the model using the overall loss function, save the optimal parameters, and obtain the trained detection model.

7. The method for detecting abnormal joint vibration of industrial robots based on deep learning according to claim 1, characterized in that, S6 specifically includes: The collected continuous real-time signals are extracted into test samples with a fixed time window length in the form of a sliding window. The sample to be tested is preprocessed and feature-enhanced by S2 to obtain an enhanced sample; The enhanced data is input into the trained detection model to obtain probability distributions for four categories: normal, slight wear, severe wear, and poor lubrication. The category corresponding to the maximum value in the probability distribution is taken as the abnormal vibration detection result.

8. A deep learning-based industrial robot joint abnormal vibration detection system, executing the method as described in any one of claims 1-7, characterized in that, Specifically, it includes: Data acquisition module: used to collect vibration signals from the joints of industrial robots and build training datasets; Data augmentation module: used to make different samples strictly correspond in rotation phase to obtain augmented samples; Feature extraction module: used to extract local anomalous waveforms in enhanced samples and construct deep temporal feature sequences; Temporal classification module: used to construct the temporal classification head and obtain the final output class probability; Model training module: used to construct the overall loss function based on cross-entropy classification loss and dynamic time warping regularization term, and to perform backpropagation and parameter updates on the model; Online detection module: used to classify and predict the vibration signals to be measured that are acquired in real time, and output abnormal vibration detection results.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed by a processor to implement the deep learning-based method for detecting abnormal vibrations of industrial robot joints as described in any one of claims 1-7.

Images