Robot fault detection method, device, system and equipment and storage medium

Abstract

The invention relates to a robot fault detection method, device, system and equipment and a storage medium. The method comprises the following steps: acquiring target motion signals of a plurality of joint motors in the operation process of the robot; performing signal fusion processing on each target motion signal to determine a two-dimensional image corresponding to each target motion signal; inputting each two-dimensional image into a preset fault detection model, and determining fault probability distribution corresponding to each target motion signal according to the output of the fault detection model; and determining the fault state of the robot according to each fault probability distribution. By adopting the method, the robot fault detection accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a robot fault detection method, device, system, equipment and storage medium. Background Art

[0002] Exoskeleton robots have been widely used in the military and rehabilitation fields, and in industrial production, they assist workers in working under high-intensity and high-risk conditions, improving production efficiency and economic benefits. However, under complex working conditions, exoskeleton robots are prone to a certain degree of wear and tear or failure, affecting the operator's work.

[0003] Traditional exoskeleton robot fault detection methods usually analyze signals collected by multiple physical quantity sensors. However, fault feature coupling may occur between the signals of each sensor, resulting in signal spectrum aliasing, which affects the accuracy of fault detection. Summary of the invention

[0004] Based on this, it is necessary to provide a robot fault detection method, device, system, equipment and storage medium that can improve the accuracy of robot fault detection in response to the above technical problems.

[0005] In a first aspect, the present application provides a robot fault detection method, comprising:

[0006] Obtain target motion signals of multiple joint motors during robot operation;

[0007] Performing signal fusion processing on each target motion signal to determine a two-dimensional image corresponding to each target motion signal;

[0008] Input each two-dimensional image into a preset fault detection model, and determine the fault probability distribution corresponding to each target motion signal according to the output of the fault detection model;

[0009] The fault state of the robot is determined according to the probability distribution of each fault.

[0010] In one embodiment, obtaining target motion signals of multiple joint motors during robot operation includes:

[0011] Get the robot's identification information;

[0012] Acquire the initial motion signal of each joint motor during the operation of the robot from the message queue according to the identification information of the robot, wherein the initial motion signal includes a first vibration signal and a first torque signal of the joint motor;

[0013] Each initial motion signal is preprocessed to determine a target motion signal, wherein the preprocessing includes sliding average filtering and wavelet packet decomposition, and the target motion signal includes a second vibration signal and a second torque signal.

[0014] In one embodiment, preprocessing each initial motion signal to determine the target motion signal further includes:

[0015] Determine the theoretical torque signal of each joint motor according to the robot dynamics model;

[0016] For each joint motor, a second torque signal is determined according to a residual of a theoretical torque signal and a preprocessed first torque signal.

[0017] In one embodiment, the target motion signal is a one-dimensional time series, the two-dimensional image is symmetrical, and signal fusion processing is performed on each target motion signal to determine the two-dimensional image corresponding to each target motion signal, including:

[0018] Normalizing the value of each target motion signal at each time point;

[0019] For the value of each target motion signal at each time point, the normalized value is mapped to a point in the polar coordinate system, and the radius, counterclockwise rotation angle and clockwise rotation angle of the point are determined according to the preset initial rotation angle, the preset angle magnification factor and the value.

[0020] In one embodiment, the fault detection model includes multiple branch models, each branch model has a different convolution kernel size, pooling strategy and network depth, each two-dimensional image is input into a preset fault detection model, and the fault probability distribution corresponding to each target motion signal is determined according to the output of the fault detection model, including:

[0021] Input each two-dimensional image into a different branch model, where the branch model includes multiple convolutional layers, multiple pooling layers, a global average pooling layer, and a softmax classifier;

[0022] The fault probability distribution corresponding to each target motion signal is determined according to the output of the softmax classifier of each branch model.

[0023] In one embodiment, determining the fault state of the robot according to each fault probability distribution includes:

[0024] Based on DS evidence theory, the decision fusion of each fault probability distribution is performed to obtain the fused target fault probability distribution;

[0025] Determine the robot's fault state based on the target fault probability distribution.

[0026] In a second aspect, the present application also provides a robot fault detection device, comprising:

[0027] An acquisition module is used to acquire target motion signals of multiple joint motors during the robot operation process;

[0028] A fusion module is used to perform signal fusion processing on each target motion signal to determine a two-dimensional image corresponding to each target motion signal;

[0029] A first determination module is used to input each two-dimensional image into a preset fault detection model, and determine the fault probability distribution corresponding to each target motion signal according to the output of the fault detection model;

[0030] The second determination module is used to determine the fault state of the robot according to each fault probability distribution.

[0031] In one of the embodiments, the acquisition module is specifically used to obtain the identification information of the robot; obtain the initial motion signal of each joint motor during the operation of the robot from the message queue according to the identification information of the robot, the initial motion signal includes the first vibration signal and the first torque signal of the joint motor; pre-process each initial motion signal to determine the target motion signal, the pre-processing includes sliding average filtering and wavelet packet decomposition, and the target motion signal includes a second vibration signal and a second torque signal.

[0032] In one embodiment, the acquisition module is also used to determine the theoretical torque signal of each joint motor according to the robot dynamics model; for each joint motor, the second torque signal is determined according to the residual of the theoretical torque signal and the preprocessed first torque signal.

[0033] In one of the embodiments, the target motion signal is a one-dimensional time series, the two-dimensional image has symmetry, and the fusion module is specifically used to normalize the value of each target motion signal at each time point; for the value of each target motion signal at each time point, the normalized value is mapped to a point in a polar coordinate system, and the radius, counterclockwise rotation angle and clockwise rotation angle of the point are determined according to a preset initial rotation angle, a preset angle magnification factor and the value.

[0034] In one embodiment, the fault detection model includes multiple branch models, and the convolution kernel size, pooling strategy and network depth of each branch model are different. The first determination module is specifically used to input each two-dimensional image into a different branch model, and the branch model includes multiple convolution layers, multiple pooling layers, a global average pooling layer and a softmax classifier; the fault probability distribution corresponding to each target motion signal is determined according to the output of the softmax classifier of each branch model.

[0035] In one of the embodiments, the second determination module is specifically used to perform decision fusion on each fault probability distribution based on DS evidence theory to obtain a fused target fault probability distribution; and determine the fault state of the robot according to the target fault probability distribution.

[0036] On the third aspect, the present application also provides a robot fault detection system, which includes a robot, multiple acquisition devices and a server; the robot includes a control system and an execution device, the execution device includes multiple joint motors, each joint motor is used to complete a specified action under the control of the control system; multiple acquisition devices are respectively arranged on the multiple joint motors of the robot, for collecting the first vibration signal and the first torque signal of each joint motor of the robot; the server is used to obtain the target motion signals of the multiple joint motors during the operation of the robot; signal fusion processing is performed on each target motion signal to determine the two-dimensional image corresponding to each target motion signal; each two-dimensional image is input into a preset fault detection model, and the fault probability distribution corresponding to each target motion signal is determined according to the output of the fault detection model; the fault state of the robot is determined according to each fault probability distribution.

[0037] In one of the embodiments, the robot further includes a communication module, and the communication module is used for communication between the robot and a server, or for communication between a control system of the robot and an execution device.

[0038] In a fourth aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements any of the methods described in the first aspect when executing the computer program.

[0039] In a fifth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any of the methods described in the first aspect above.

[0040] In a sixth aspect, the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements any of the methods described in the first aspect above.

[0041] The above robot fault detection method, device, system, equipment and storage medium obtain the target motion signals of multiple joint motors during the operation of the robot; perform signal fusion processing on each target motion signal to determine the two-dimensional image corresponding to each target motion signal; input each two-dimensional image into a preset fault detection model, and determine the fault probability distribution corresponding to each target motion signal according to the output of the fault detection model; determine the fault state of the robot according to each fault probability distribution. By performing signal fusion processing on the motion signal to obtain a two-dimensional image, the dynamic changes of the time series signal can be effectively captured, the data features can be accurately extracted, and the accuracy of the detection results can be improved in subsequent fault detection. At the same time, the operation signals of the multiple joint motors of the robot can be analyzed separately. After completing independent detection of each target motion signal, the fault state of the robot is determined by combining multiple detection results, which can further improve the accuracy of fault detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 A diagram showing an application environment of a robot fault detection method in one embodiment;

[0044] Figure 2 A schematic diagram of a flow chart of a robot fault detection method in one embodiment;

[0045] Figure 3 A schematic diagram of a flow chart of steps for obtaining a target motion signal in one embodiment;

[0046] Figure 4 A schematic diagram of the structure of an MQTT system in an embodiment;

[0047] Figure 5 A schematic diagram of a flow chart of steps for obtaining a target motion signal in another embodiment;

[0048] Figure 6 It is a force diagram of a single rod in one embodiment;

[0049] Figure 7 A schematic flow chart of a step of determining a two-dimensional image corresponding to each target motion signal in an embodiment;

[0050] Figure 8 A schematic diagram of a process of converting a target motion signal into a two-dimensional image in one embodiment;

[0051] Fig. 9 A schematic flow chart of a step of determining a fault probability distribution corresponding to each target motion signal in an embodiment;

[0052] Fig.10 A schematic diagram of a flow chart of steps for determining a fault condition of a robot in one embodiment;

[0053] Fig.11 A schematic diagram of a flow chart of a robot fault detection method in another embodiment;

[0054] Fig.12 is a structural block diagram of a robot fault detection device in one embodiment;

[0055] Fig.13 is a schematic diagram of the structure of a robot in one embodiment;

[0056] Fig.14 is a schematic structural diagram of a lower limb exoskeleton robot in one embodiment;

[0057] Fig.15 A framework diagram of a robot fault detection system in one embodiment;

[0058] Fig.16 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] Wearable exoskeleton robot system is an intelligent equipment designed to work with the human body. It can assist or replace the wearer in carrying weight and moving, reducing the labor intensity of work. With the continuous development of exoskeleton research, the types of exoskeletons have become more and more diverse. For example, in medical rehabilitation training or industrial applications, exoskeleton robots significantly reduce the physical burden of assembly line workers, improve work efficiency, and thus improve the work quality of the wearer. It plays an important role in ensuring the safety and efficiency of workers in complex working conditions. Exoskeleton robots have been widely used in the military and rehabilitation fields, and in industrial production, they assist workers in working under high-intensity and high-risk working conditions, improving production efficiency and economic benefits. However, under complex working conditions, exoskeleton robots are prone to a certain degree of wear and tear or failure, which affects the operator's work.

[0061] Traditional exoskeleton robot fault detection methods usually analyze signals collected by multiple physical quantity sensors. However, fault feature coupling may occur between the signals of each sensor, resulting in signal spectrum aliasing, which affects the accuracy of fault detection.

[0062] In view of this, the present application provides a robot fault detection method that can improve the accuracy of robot fault detection. The robot fault detection method provided in the embodiment of the present application can be applied to Figure 1 In the robot fault detection system 100 shown, the robot fault detection system 100 includes a robot 110, a plurality of acquisition devices 120 and a server 130; the robot 101 includes a control system 111 and an execution device 112, the execution device 112 includes a plurality of joint motors, each joint motor is used to complete a specified action under the control of the control system 111, and the plurality of acquisition devices 120 are respectively arranged on the plurality of joint motors of the robot, and are used to collect the first vibration signal and the first torque signal of each joint motor of the robot, and the server 130 is used to obtain the target motion signal of the plurality of joint motors during the operation of the robot, perform signal fusion processing on each target motion signal to determine the two-dimensional image corresponding to each target motion signal, input each two-dimensional image into a preset fault detection model, determine the fault probability distribution corresponding to each target motion signal according to the output of the fault detection model, and determine the fault state of the robot according to each fault probability distribution. Among them, the server 104 can be an edge server, specifically an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0063] In an exemplary embodiment, Figure 2 As shown, a robot fault detection method is provided, which is applied to Figure 1 The server 130 in the example is used as an example to illustrate the method, which includes the following steps 201 to 203. Among them:

[0064] S201, obtaining target motion signals of multiple joint motors during robot operation.

[0065] Optionally, the robot may be an exoskeleton robot, an industrial robot, a service robot or a special robot, etc. The execution device of the robot includes a plurality of joint motors, and motion signals of the plurality of joint motors during operation of the robot may be acquired.

[0066] In a possible implementation, the target motion signal may be obtained by preprocessing the initial motion signal received from the acquisition device. The acquisition device may be a position sensor, a speed sensor, a force / torque sensor, a posture sensor, or a biosignal sensor, and the initial motion signal may include an initial vibration signal, an initial torque signal, an initial position signal, or an initial posture signal at a robot joint motor.

[0067] In another possible implementation, the target motion signal may also be a vibration signal, a torque signal or a position signal at a robot joint motor obtained by calling an application programming interface (Application Programming Interface) or an instruction set provided by the robot control system.

[0068] Optionally, the robot includes multiple joint motors, so multiple target motion signals are obtained.

[0069] S202: Perform signal fusion processing on each target motion signal to determine a two-dimensional image corresponding to each target motion signal.

[0070] Exemplarily, the target motion signals of the robot's four joint motors include four groups of vibration signals and four groups of torque signals. After signal fusion processing of each target motion signal, eight groups of two-dimensional images can be obtained, corresponding to the four groups of vibration signals and the four groups of torque signals respectively.

[0071] Optionally, the target motion signal can be a multi-dimensional time series signal, including time series signals of the robot joint motor in three orthogonal directions. For example, taking the three orthogonal directions of X-axis, Y-axis and Z-axis as an example, the target signal can include a vibration signal along the X direction, a vibration signal along the Y direction and a vibration signal along the Z direction at the joint motor.

[0072] In one possible implementation, the signal fusion processing can be to fuse multi-dimensional target motion signals through data fusion and generate a two-dimensional image. For example, the vibration signal along the X direction, the vibration signal along the Y direction and the vibration signal along the Z direction of a joint motor are fused, and a two-dimensional image is generated based on the fused vibration signal.

[0073] Exemplarily, data fusion of a group of target motion signals can be performed by splicing and merging multi-dimensional signals of the target motion signals, and then generating a two-dimensional image based on the merged signals; or it can be performed by extracting features from each dimensional signal, then merging the extracted features, and generating a two-dimensional image based on the merged features.

[0074] Exemplarily, the recursive graph can be generated by calculating the similarity between each point in the signal after data fusion, or the relative size relationship between each point in the signal after data fusion can be calculated and the obtained gray value matrix can be used as a two-dimensional image.

[0075] In another possible implementation, the signal fusion processing may be to generate a corresponding two-dimensional image based on a set of multi-dimensional target motion signals and a preset generation algorithm.

[0076] Optionally, the preset generation algorithm may generate a two-dimensional image according to the target motion signal by means of image coding, geometric changes or other coding methods.

[0077] Exemplarily, each point in the target motion signal can be converted from a rectangular coordinate system to a polar coordinate system based on the Gram angular field method, and the obtained Gram matrix can be used as a two-dimensional image.

[0078] S203, inputting each two-dimensional image into a preset fault detection model, and determining the fault probability distribution corresponding to each target motion signal according to the output of the fault detection model.

[0079] Optionally, the fault detection model may be a generative model, such as a generative adversarial network, or a convolutional neural network (CNN), a recurrent neural network (RNN) and its variants, etc., which is not limited in the embodiments of the present application.

[0080] Optionally, the fault probability distribution may be that the fault detection model predicts the fault state of the robot and expresses the prediction result in the form of probability distribution.

[0081] Optionally, the two-dimensional images corresponding to each target motion signal can be input into the fault detection model respectively to obtain the fault probability distribution corresponding to each target motion signal. The fault detection model can be trained using the two-dimensional images and fault types corresponding to the robot's motion signal data set as training samples.

[0082] In one possible implementation, when determining the motion signal data set, a controlled fault injection method can be used. Based on the principle of fault mode equivalence, typical fault conditions are preset in the hip and knee joint drive units, and motion signals under rated load are obtained through trajectory tracking control experiments. The sampling frequency is set to 100Hz, and the motion signals at the joint motors under normal and fault conditions are fully recorded, and the time-frequency domain characteristics of the motion signals under normal and fault conditions are recorded. The motion signal data set not only contains the original physical quantity time series information, but also provides complete time-space domain basic data support for the subsequent steps of nonlinear feature extraction and fault detection model training through the spatial motion parameter coupling relationship.

[0083] Optionally, a fully trained fault detection model can be deployed on the edge server to process the target motion signal from the robot in real time and provide accurate fault diagnosis results. The deployed fault detection model will continuously monitor the operating status of the exoskeleton robot to ensure timely response when any abnormality or fault signs are detected.

[0084] Optionally, the output of the fault detection model is used to guide maintenance decisions, optimize maintenance processes, and reduce downtime caused by equipment failures. In addition, the fault data detected by the fault detection model can be used for continuous optimization and updating of the model, thereby achieving iterative improvement of model performance. In normal operation, that is, when the model diagnosis results indicate that there is no fault, the robot continues to perform its tasks, and the fault detection model maintains continuous monitoring of the equipment status to ensure that potential faults are discovered and responded to through the edge server at the first time.

[0085] S204: Determine the fault state of the robot according to each fault probability distribution.

[0086] Optionally, multiple fault probability distributions are integrated to obtain a target fault probability distribution, and then the fault state of the robot is determined according to the target fault probability distribution.

[0087] Exemplarily, the target fault probability distribution may be obtained by using a decision fusion method such as a weighted average method or Bayesian fusion, or by clustering multiple fault probability distributions according to a clustering method to obtain the target fault probability distribution.

[0088] Exemplarily, the maximum probability principle can be used to determine the fault type with the highest probability in the target fault probability distribution as the fault state of the robot. Alternatively, a preset threshold can be obtained to determine the fault type in the target fault probability whose probability is greater than a preset threshold as the fault type of the robot.

[0089] The above robot fault detection method obtains the target motion signals of multiple joint motors during the operation of the robot; performs signal fusion processing on each target motion signal to determine the two-dimensional image corresponding to each target motion signal; inputs each two-dimensional image into a preset fault detection model, and determines the fault probability distribution corresponding to each target motion signal according to the output of the fault detection model; and determines the fault state of the robot according to each fault probability distribution. By performing signal fusion processing on the motion signal to obtain a two-dimensional image, the dynamic changes of the time series signal can be effectively captured, data features can be accurately extracted, and the accuracy of the detection results can be improved in subsequent fault detection. At the same time, the operation signals of the multiple joint motors of the robot can be analyzed separately. After completing independent detection of each target motion signal, the fault state of the robot can be determined by combining multiple detection results, which can further improve the accuracy of fault detection.

[0090] In an exemplary embodiment, Figure 3 As shown, optionally, obtaining target motion signals of multiple joint motors during robot operation includes the following steps 301 to 303. Among them:

[0091] S301, obtaining identification information of the robot.

[0092] Optionally, the robot's identification information may be the robot's serial number, QR code or barcode, or may be the robot's unique identification, so that relevant data of the robot may be obtained through the robot's identification information, such as quality inspection reports, maintenance records, maintenance history, and motion signal data.

[0093] S302, obtaining the initial motion signal of each joint motor during the operation of the robot from the message queue according to the identification information of the robot.

[0094] The initial motion signal includes a first vibration signal and a first torque signal of the joint motor.

[0095] Optionally, the acquisition device may send the acquired initial motion signal to a message queue, and then obtain the initial motion signal corresponding to the robot from the message queue based on the identification information of the robot at a fixed time or when fault detection is required.

[0096] Optionally, the scalability, reliability and performance of the robot fault detection system can be improved based on the message queue. The message queue follows the producer-consumer model. The producer (i.e., the acquisition device) sends the initial motion signal to the message queue, and the consumer (i.e., the server) obtains the initial motion signal from the message queue and processes it. The message queue plays the role of storing and forwarding messages in the middle to ensure that the messages will not be lost and can be processed in a certain order. Through the message queue, the acquisition device only needs to send the initial motion signal to the queue without having to care about the specific implementation and status of the server.

[0097] Exemplarily, the message queue may be RabbitMQ, RocketMQ, Kafka, or may be implemented based on the Message Queuing Telemetry Transport (MQTT) protocol.

[0098] Among them, the structure of the MQTT system is as follows Figure 4 As shown, the MQTT system is based on the publish / subscribe model and consists of a publisher, a proxy (MQTT Broker) and a subscriber. In an embodiment of the present application, the publisher can be a collection device and the subscriber can be a server. The collection device can publish the initial motion signal as a message to a specific topic (the topic can be the identification information of the robot). The proxy is responsible for receiving the message published by the publisher and forwarding the message to the server subscribed to the topic according to the topic of the message.

[0099] S303, preprocessing each initial motion signal to determine a target motion signal, wherein the preprocessing includes sliding average filtering and wavelet packet decomposition.

[0100] The target motion signal includes a second vibration signal and a second torque signal.

[0101] Optionally, after the robot joint motor performs an action, low-frequency jitter noise may be caused by insufficient structural stiffness or dynamic imbalance, and there may be electromagnetic noise in the motor drive circuit, transient high-frequency noise caused by the wearer's gait changes or ground reaction force impact, which affects the quality of signal acquisition and makes the initial motion signal include motion noise, such as joint braking residual vibration, electromagnetic interference noise and structural resonance noise. The motion noise can be removed by signal processing to achieve noise separation.

[0102] Optionally, each initial motion signal may be preprocessed separately. For an initial motion signal, the first vibration signal may be preprocessed to obtain a second vibration signal, and the first torque signal may be preprocessed to obtain a second torque signal.

[0103] Optionally, the acquisition device may include a three-axis acceleration sensor and a three-dimensional torque sensor, the first vibration signal may be the XYZ axial vibration acceleration of the joint motor acquired by the three-axis acceleration sensor, which is a multi-dimensional time series signal, and the first vibration signal may be the XYZ axial dynamic torque of the joint motor acquired by the three-dimensional torque sensor, which is also a multi-dimensional time series signal. It is understandable that after preprocessing, the second vibration signal and the second torque signal are also multi-dimensional time series signals.

[0104] Optionally, when removing motion noise, a hierarchical progressive signal processing system can be used. First, a sliding average filter is used to coarse-grained smooth the initial motion signal in the time domain to suppress high-frequency pulse noise and baseline drift. Then, wavelet packet decomposition and threshold denoising are used to perform fine decomposition and threshold processing on the residual mid- and high-frequency noise to retain key fault features.

[0105] Optionally, the Moving Average (MA) method can be used to smooth the waveform of the initial motion signal, effectively filter out noise components in the frequency domain, and improve signal quality. The core idea of ​​the moving average method is to use the statistical characteristics of local data to replace single-point data, thereby reducing the impact of random noise. MA is achieved by simply averaging the surrounding points or by weighted averaging the surrounding points, which can be expressed by the following formula:

[0106]

[0107] Among them, x represents the sampled data value, y represents the result after smoothing, m is the data volume, N represents the number of points included in the window, and h represents the weighting factor.

[0108] Optionally, the initial motion signal is coarse-grained smoothed using a time domain sliding window, which can effectively suppress high-frequency pulse noise generated by joint motor commutation and baseline drift caused by low-frequency fluctuations of structural resonance.

[0109] Optionally, Wavelet Packet Decomposition (WPD) is an extension of the traditional wavelet transform, which forms a finer frequency band division by recursively decomposing the high-frequency and low-frequency components of the signal. Suppose the smoothed signal is s(t), and the wavelet packet decomposition is implemented by iterative low-pass filter g[n] and high-pass filter h[n]. Each layer of decomposition divides each node into two sub-nodes: low-frequency (approximate coefficient) and high-frequency (detail coefficient), which can be expressed by the following formula:

[0110]

[0111] Among them, j is the number of decomposition layers, k is the node number, and s j kRepresents the coefficient of the kth node in the jth layer.

[0112] Exemplarily, the signal s(t) can be decomposed into 12 layers using the Daubechies series of wavelets to obtain the coefficients of each subband {wj,k}. After wavelet decomposition, the amplitude of the wavelet coefficient of the signal s(t) must be greater than the amplitude of the noise coefficient. Threshold processing is used to separate the low-frequency jitter noise from the fault characteristics (high-frequency impact components) and filter out the noise.

[0113] The identification information of the robot is obtained, and the initial motion signal of each joint motor in the robot operation process is obtained from the message queue according to the identification information of the robot. The initial motion signal includes the first vibration signal and the first torque signal of the joint motor. Each initial motion signal is preprocessed to determine the target motion signal. The preprocessing includes sliding average filtering and wavelet packet decomposition. The target motion signal includes the second vibration signal and the second torque signal. In this way, the scalability, reliability and performance of the robot fault detection system can be improved. At the same time, the initial motion signal is subjected to noise separation processing, which can provide the accuracy of fault detection.

[0114] In another possible implementation, the process of preprocessing the initial motion signal can also be implemented in the robot's main control unit, that is, the acquisition device can send the collected initial motion data to the robot, and after the robot processes the initial motion signal, it sends the target motion signal to the server through the message queue.

[0115] In an exemplary embodiment, Figure 5 As shown, optionally, preprocessing each initial motion signal to determine the target motion signal also includes the following steps 501 to 502. Among them:

[0116] S501, determining the theoretical torque signal of each joint motor according to the robot dynamics model.

[0117] Optionally, the torque signal is an important physical parameter of the robot when it is working. It can directly reflect the movement intention and the mechanical load imposed by the external environment. By real-time monitoring of torque changes, the exoskeleton robot can accurately perceive the user's gait, terrain characteristics and muscle activity status. By controlling the torque, it can also coordinate multi-joint movements, optimize energy distribution through impedance control, force-position hybrid control and other algorithms, effectively respond to transient impacts and low-frequency jitter noise, and improve system stability and wearing comfort.

[0118] Optionally, the theoretical calculation of the exoskeleton robot torque signal requires dynamic analysis of the robot through the robot dynamics model. The dynamics model can be the Newton-Euler method, which is based on Newtonian mechanics and rigid body dynamics principles, and expresses the constraints and relative motion relationships between connecting rods in vector form. The core of this method is to use Newton's laws and Euler equations to establish a group of dynamic equilibrium equations to achieve a recursive solution to the side effects of each motion. Its computational complexity is linearly related to the degree of freedom of the mechanical system, and is particularly suitable for real-time control scenarios of multi-joint complex mechanisms.

[0119] The following is an introduction to the process of determining the theoretical torque signal of the joint motor according to the Newton-Euler method:

[0120] The force diagram of a single rod is as follows Figure 6 As shown, in Figure 6 In, O i-1 , O i is the origin of the coordinate system at both ends of member i, C i is the center of mass of the bar, L i K is the distance between the origins of the coordinate system at both ends of member i; i 、h i is the center of mass C i to i-1 , O i The distance, f i-1,i 、-f i,i+1 is the force and reaction force of member i-1 acting on member i, n i-1,i 、-n i,i+1 is the moment and reaction moment of member i-1 acting on member i, f i The external force acting on member i is simplified to the center of mass C i The resultant force at i V is the resultant moment of the external moment acting on member i reduced to the center of mass, ci is the translation velocity of the bar’s center of mass, ω i is the angular velocity of the center of mass of member i.

[0121] The mass of the rod is known to be m i , the center of mass is at C i The acceleration at the center of mass is a ci , the angular velocity and angular acceleration around the center of mass are ω i and ε i According to Newton's equation, the action on the bar's center of mass C i The force at is: .

[0122] According to Euler's formula, the moment acting on member i is .

[0123] Among them, ICi For the rod L i Relative to its center of mass C i The inertia tensor can be expressed by the following formula:

[0124]

[0125] Perform independent force analysis on component i, and obtain the following from the resultant force and resultant moment theorem:

[0126]

[0127]

[0128] Optionally, based on the motion characteristics of the gait phase cycle, the lower limb biomechanical system presents a periodic switch between the single-foot support phase and the double-foot support phase, and the Newton-Euler recursion algorithm is used to derive the dynamic characteristics of the joint torque in the single / double-foot support phase. Through the inverse solution method, the end rod joint driving torque is first obtained through the physical quantity of the exoskeleton rod and the force sensor signal, and then the joint torque is solved in reverse order to obtain the theoretical torque values ​​of the left and right hip joints and the left and right knee joints.

[0129] S502: For each joint motor, determine a second torque signal according to a residual of a theoretical torque signal and a preprocessed first torque signal.

[0130] Optionally, after calculating the theoretical torque value of each joint motor, a theoretical torque signal can be obtained, and then the difference between the theoretical torque signal of each joint motor and the preprocessed first torque signal is calculated to obtain the second torque signal corresponding to each joint motor.

[0131] Alternatively, existing robot joint motor fault detection methods mostly rely on direct analysis of sensor data, without fully considering the physical correlation between the robot's dynamic characteristics and joint torque, and are unable to effectively distinguish between abnormal torque caused by mechanical failure and torque fluctuations caused by environmental interference. They ignore the dynamic theoretical model of joint torque and have difficulty capturing the deep correlation between motor load characteristics and fault mechanisms.

[0132] The above method determines the theoretical torque signal of each joint motor according to the robot dynamics model, and for each joint motor, determines the second torque signal according to the residual of the theoretical torque signal and the preprocessed first torque signal. In this way, after the first torque signal is subjected to noise separation processing, the second torque signal is determined according to the difference between the theoretical torque signal and the collected torque signal, that is, the residual of the theoretical value and the observed value is introduced into the fault detection model, which reduces the dependence of the traditional fault detection method on data, improves the actual physical correlation of the fault detection model, and improves the recognition ability of the fault detection model.

[0133] In an exemplary embodiment, Figure 7 As shown, optionally, the target motion signal is a one-dimensional time series, the two-dimensional image has symmetry, and the signal fusion processing is performed on each target motion signal to determine the two-dimensional image corresponding to each target motion signal, including the following steps 701 to 702. Among them:

[0134] S701, normalizing the value of each target motion signal at each time point.

[0135] Optionally, the values ​​of each time point in the second vibration signal and the second torque signal at each joint motor are normalized, and the normalization process can be expressed by the following formula:

[0136]

[0137] Among them, x i is the amplitude value of the i-th time point in a target motion signal, and s represent the mean and standard deviation of the target motion signal, respectively.

[0138] Optionally, distinguishing indicators such as mean deviation and standard deviation are included in the normalization process, so that the subsequently generated two-dimensional image can contain more feature information, be easier to classify, and enhance the distinction between different fault states.

[0139] S702, for the value of each target motion signal at each time point, map the normalized value to a point in the polar coordinate system, and determine the radius, counterclockwise rotation angle and clockwise rotation angle of the point according to the preset initial rotation angle, the preset angle magnification factor and the value.

[0140] Optionally, after normalizing the target motion signal, the numerical value of the time point in the target motion signal is mapped to a point in a polar coordinate system to obtain a two-dimensional image corresponding to the target motion signal, which may include amplitude and frequency information of the target motion signal.

[0141] Optionally, the one-dimensional time series signal may be converted into a two-dimensional image with symmetry based on a dot plot method or a modified dot plot method.

[0142] Exemplarily, the method for modifying the dot map can determine the radius, counterclockwise rotation angle, and clockwise rotation angle of the point according to the preset initial rotation angle, the preset angle magnification coefficient, and the value. The calculation process can be expressed by the following formula:

[0143]

[0144]

[0145]

[0146] in, is the radius of the point in the polar coordinate system mapped from the value of the i-th time point in a target motion signal, The value is mapped to the clockwise rotation angle in the polar coordinate system. The value is mapped to the counterclockwise rotation angle in the polar coordinate system. To preset the initial rotation angle, is the preset angle magnification coefficient, l is the time lag coefficient, is the value at the i+lth time point after normalization.

[0147] Optional, such as Figure 8 As shown, it is a process of converting a multi-dimensional target motion signal into a two-dimensional image. The two-dimensional image can usually be snowflake-shaped or hexagonally symmetrical.

[0148] The above-mentioned normalization processing is performed on the numerical value of each target motion signal at each time point. For the numerical value of each target motion signal at each time point, the normalized numerical value is mapped to a point in the polar coordinate system, and the radius, counterclockwise rotation angle and clockwise rotation angle of the point are determined according to the preset initial rotation angle, the preset angle magnification factor and the numerical value. In this way, each target motion signal is converted into a corresponding two-dimensional image, which can intuitively display the dynamic characteristics of the target motion signal and facilitate subsequent analysis.

[0149] In an exemplary embodiment, Fig. 9 As shown, optionally, the fault detection model includes multiple branch models, and the convolution kernel size, pooling strategy and network depth of each branch model are different. Each two-dimensional image is input into a preset fault detection model, and the fault probability distribution corresponding to each target motion signal is determined according to the output of the fault detection model, including the following steps 901 to 902. Among them:

[0150] S901, input each two-dimensional image into a different branch model, where the branch model includes multiple convolutional layers, multiple pooling layers, a global average pooling layer and a softmax classifier.

[0151] Optionally, the two-dimensional images corresponding to each target motion signal are respectively input into different branch models, and each branch model is independent of each other and corresponds to the feature extraction of a single target motion signal.

[0152] Optionally, taking the branch model as 2D-CNN as an example, convolution kernels of different sizes can be used for feature extraction in each branch model, so that the model can capture a wide range of features in the two-dimensional image and reduce noise interference, enhance the model's ability to capture fault features, enable the model to understand the input data more comprehensively, and improve the accuracy of fault classification. After feature extraction, the model uses a series of convolution and pooling layers for further feature recognition processing, introduces a global average pooling layer (GAP) at the end to replace the fully connected layer, outputs a 128-256-dimensional high-dimensional feature vector, reduces the number of parameters and enhances the ability to express spatial features, and connects a softmax classifier after the GAP layer to map the feature vector to the probability distribution of the robot's fault state (such as normal, bearing wear, fault, etc.).

[0153] S902, determining the fault probability distribution corresponding to each target motion signal according to the output of the softmax classifier of each branch model.

[0154] Optionally, the fault probability distribution corresponding to each target motion signal can be determined by the output of the softmax classifier of each branch model. The output form of the softmax classifier of the i-th branch model can be P i =[p i1 ,p i2 ,…,p ik ], k is the number of fault categories.

[0155] In the above method, each two-dimensional image is input into a different branch model, and the branch model includes multiple convolutional layers, multiple pooling layers, a global average pooling layer and a softmax classifier. The fault probability distribution corresponding to each target motion signal is determined according to the output of the softmax classifier of each branch model. In this way, fault detection is performed on each target motion signal through multiple sets of parallel branch models, which can effectively avoid the interference of feature coupling of different physical quantities and reduce the redundant representation of faults by multiple perception signals.

[0156] In an exemplary embodiment, Fig.10 As shown, optionally, determining the fault state of the robot according to each fault probability distribution includes the following steps 1001 to 1002. Among them:

[0157] S1001, based on the DS evidence theory, decision fusion is performed on each fault probability distribution to obtain a fused target fault probability distribution.

[0158] Optionally, the DS evidence theory is a Dempster-Shafer evidence theory, which can effectively fuse multiple fault probability distribution information through the Dempster combination rule.

[0159] Optionally, the probability p in each fault probability distribution can be ij Transformed into the basic credibility distribution function m in DS evidence theory i (A j ), obtain the preset probability threshold θ, and only retain the fault categories with probabilities greater than or equal to the preset probability threshold in each fault probability distribution as the effective focal element A j , thereby avoiding low confidence noise interference.

[0160] Optionally, when determining the basic credit distribution function m i (A j ), the DS synthesis rule can be used to integrate the basic probability distribution of independent evidence sources. The probability of fault category A can be expressed by the following formula:

[0161]

[0162] Among them, m i represents the evidence source i, that is, the fault probability distribution obtained after the i-th target motion signal is input into the corresponding branch model; B is the focal element from m1; C is the focal element from m2; K is the conflict factor, that is , K represents the degree of conflict between two sources of evidence. When K=1, the conflict cannot be reconciled and the synthesis rule becomes invalid.

[0163] S1002, determining the fault state of the robot according to the target fault probability distribution.

[0164] Optionally, the maximum probability principle can be used to determine the robot's fault state based on the fault type with the highest probability in the target fault probability distribution. A preset threshold can also be obtained to determine the robot's fault type based on the fault type in the target fault probability whose probability is greater than a preset threshold. When there are multiple close fault probabilities (such as a difference <0.1), a "compound fault" label can be output and a manual review process can be triggered.

[0165] The above-mentioned decision fusion of each fault probability distribution based on DS evidence theory is performed to obtain the fused target fault probability distribution, and the fault state of the robot is determined according to the target fault probability distribution. The fault probability distribution corresponding to the target motion signal collected by multiple acquisition devices can be integrated, which effectively overcomes the one-sided information problem of a single sensor in fault detection and can effectively provide the accuracy of determining the robot's fault state.

[0166] As an optional implementation, Fig.11 As shown, the robot fault detection method provided in the embodiment of the present application may include the following specific steps:

[0167] S1101, obtaining identification information of the robot;

[0168] S1102, obtaining the initial motion signal of each joint motor during the operation of the robot from the message queue according to the identification information of the robot;

[0169] Wherein, the initial motion signal includes a first vibration signal and a first torque signal of the joint motor;

[0170] S1103, preprocessing each initial motion signal to determine a second vibration signal and a preprocessed first torque signal, the preprocessing comprising sliding average filtering and wavelet packet decomposition;

[0171] S1104, determining the theoretical torque signal of each joint motor according to the robot dynamics model;

[0172] S1105, for each joint motor, determining a second torque signal according to a residual of the theoretical torque signal and the preprocessed first torque signal;

[0173] S1106, determining a target motion signal according to the second vibration signal and the second torque signal;

[0174] S1107, normalizing the value of each target motion signal at each time point;

[0175] S1108, for the value of each target motion signal at each time point, map the normalized value to a point in the polar coordinate system, and determine the radius, counterclockwise rotation angle, and clockwise rotation angle of the point according to a preset initial rotation angle, a preset angle magnification factor, and the value;

[0176] S1109, inputting each two-dimensional image into a different branch model, where the branch model includes multiple convolutional layers, multiple pooling layers, a global average pooling layer, and a softmax classifier;

[0177] S1110, determining the fault probability distribution corresponding to each target motion signal according to the output of the softmax classifier of each branch model;

[0178] S1111, based on DS evidence theory, decision fusion is performed on each fault probability distribution to obtain a fused target fault probability distribution;

[0179] S1112, determining the fault state of the robot according to the target fault probability distribution.

[0180] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0181] Based on the same inventive concept, the embodiment of the present application also provides a robot fault detection device for implementing the robot fault detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more robot fault detection device embodiments provided below can refer to the limitations of the robot fault detection method above, and will not be repeated here.

[0182] In an exemplary embodiment, Fig.12 As shown, a robot fault detection device 1200 is provided, comprising: an acquisition module 1201, a fusion module 1202, a first determination module 1203 and a second determination module 1204, wherein:

[0183] An acquisition module 1201 is used to acquire target motion signals of multiple joint motors during the operation of the robot;

[0184] A fusion module 1202 is used to perform signal fusion processing on each target motion signal to determine a two-dimensional image corresponding to each target motion signal;

[0185] The first determination module 1203 is used to input each two-dimensional image into a preset fault detection model, and determine the fault probability distribution corresponding to each target motion signal according to the output of the fault detection model;

[0186] The second determination module 1204 is used to determine the fault state of the robot according to each fault probability distribution.

[0187] In one embodiment, the acquisition module 1201 is specifically used to obtain the identification information of the robot; obtain the initial motion signal of each joint motor during the operation of the robot from the message queue according to the identification information of the robot, the initial motion signal includes the first vibration signal and the first torque signal of the joint motor; pre-process each initial motion signal to determine the target motion signal, the pre-processing includes sliding average filtering and wavelet packet decomposition, and the target motion signal includes a second vibration signal and a second torque signal.

[0188] In one embodiment, the acquisition module 1201 is further used to determine the theoretical torque signal of each joint motor according to the robot dynamics model; for each joint motor, the second torque signal is determined according to the residual of the theoretical torque signal and the preprocessed first torque signal.

[0189] In one embodiment, the target motion signal is a one-dimensional time series, and the two-dimensional image has symmetry. The fusion module 1202 is specifically used to normalize the values ​​of each target motion signal at each time point; for the values ​​of each target motion signal at each time point, the normalized values ​​are mapped to a point in a polar coordinate system, and the radius, counterclockwise rotation angle and clockwise rotation angle of the point are determined according to a preset initial rotation angle, a preset angle magnification factor and the value.

[0190] In one embodiment, the fault detection model includes multiple branch models, and the convolution kernel size, pooling strategy and network depth of each branch model are different. The first determination module 1203 is specifically used to input each two-dimensional image into a different branch model, and the branch model includes multiple convolution layers, multiple pooling layers, a global average pooling layer and a softmax classifier; the fault probability distribution corresponding to each target motion signal is determined according to the output of the softmax classifier of each branch model.

[0191] In one embodiment, the second determination module 1204 is specifically used to perform decision fusion on each fault probability distribution based on DS evidence theory to obtain a fused target fault probability distribution; and determine the fault state of the robot according to the target fault probability distribution.

[0192] Each module in the above robot fault detection device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0193] In an exemplary embodiment, Figure 1As shown, a robot fault detection system 100 is provided, and the robot fault detection system 100 includes a robot 110, a plurality of acquisition devices 120 and a server 130; the robot 110 includes a control system 111 and an execution device 112, the execution device 112 includes a plurality of joint motors, each joint motor is used to complete a specified action under the control of the control system 111, and the plurality of acquisition devices 120 are respectively arranged on the plurality of joint motors of the robot, and are used to collect the first vibration signal and the first torque signal of each joint motor of the robot; the server 130 is used to obtain the target motion signals of the plurality of joint motors during the operation of the robot, perform signal fusion processing on each target motion signal to determine a two-dimensional image corresponding to each target motion signal, input each two-dimensional image into a preset fault detection model, determine the fault probability distribution corresponding to each target motion signal according to the output of the fault detection model, and determine the fault state of the robot according to each fault probability distribution.

[0194] In one embodiment, if Fig.13 As shown, the robot 110 further includes a communication module 113, and the communication module 113 is used for communication between the robot and a server, or for communication between the control system of the robot and an execution device.

[0195] Optionally, the communication module 113 includes WiFi communication, CAN communication and sensor information processing device. The WiFi communication device is used for command transmission between the host and the robot, the CAN communication device connects the system main control unit and the active joint to control the joint motor to complete the specified action, and the sensor information processing device is responsible for collecting, counting and calculating sensor information.

[0196] Optional, such as Fig.13 As shown, the robot also includes a power supply system 114, which includes a main control power supply and a management system, and is mainly used to supply power to various parts of the robot.

[0197] Optionally, the multiple joint motors in the actuator may include active joints and passive joints, forming an actuator for completing a specified action. The active joints may include hip joints and knee joints, which are responsible for driving and executing the specified action; the passive joints include ankle joints. In addition, the actuator also includes support structures such as waist and leg connecting rods.

[0198] Optionally, the control system may include a detection device, a control device and a drive device. The detection device is composed of an incremental encoder and an absolute encoder, which are used to obtain the rotation angle and speed of the motor to achieve precise control of the motor movement. The absolute encoder is installed at the active joint and is responsible for collecting the rotation position information of the joint angle. The drive device includes components such as a motor and a reducer.

[0199] For example, taking the robot as a lower limb exoskeleton robot, the structure of the lower limb exoskeleton robot is as follows: Fig.14 As shown, the lower limb exoskeleton robot includes a battery and a main control component, and its execution equipment includes a left hip joint component, a right hip joint component, a left knee joint component, a right knee joint component, a left thigh component, a right thigh component, a left calf component, a right calf component, a left sole component and a right sole component, and the joint motors can generally be installed at the hip joints and knee joints of the lower limb exoskeleton robot, and acceleration sensors and torque sensors can be installed on each joint motor, and each sensor is used as a collection device.

[0200] Among them, the hip joint is the largest joint in the human body. It is very stable in connection with the waist and can stretch freely. It supports the human torso and head upward and provides power for the thigh and calf downward. Since the hip joint of the human lower limb can move in three different directions, the motor is installed in the flexion / extension direction of the three directions. The motor is connected to one or more gears to convert the rotational motion of the motor into the linear motion of the hip joint, thereby ensuring the flexion and extension of the hip joint. A hard limit structure is designed at the connection between the hip joint and the thigh to ensure that the torsion angle of the parts does not exceed a certain limit value. The knee joint is the most commonly used joint among the three joints in the human body structure. The normal human knee joint is a flexion and extension form, so a revolute pair is generally used to flex and extend the knee joint, and a mechanical limit is added to prevent the knee joint angle from being too large and causing harm to the wearer when walking. The motor used in the exoskeleton robot is generally a high-reliability servo motor with an integrated planetary reducer. This type of motor is widely used in the fields of foot-type robots, robotic arms, automation equipment, etc. It has a large torque and is equipped with a high-precision reducer, which makes the motor output more stable. The motor also supports multiple communication protocols and can communicate with the host computer, which is convenient for users to quickly and accurately control the host computer.

[0201] In an exemplary embodiment, the robot fault detection system may further include a safety mechanism device, which is arranged on the robot and is used to ensure the safety of the robot's operation and receive the fault diagnosis results sent by the server. If a fault is detected, the safety mechanism may be activated and a warning may be issued. If there is no fault in the system, the robot may be instructed to continue the operation. Fig.15As shown, it is a possible framework diagram of a robot fault detection system. The acquisition device installed on the hip joint and the knee joint collects the first vibration signal of the left hip, the first torque signal of the left hip, the first vibration signal of the right hip, the first torque signal of the right hip, the first vibration signal of the left knee, the first torque signal of the left knee, the first vibration signal of the right knee and the first torque signal of the right knee, and sends them to the edge server through the communication module of the robot. The edge server is embedded with the robot fault detection method described in any of the above method embodiments, and can pre-process the acquired initial motion signal to obtain the target motion signal, and fuse each target motion signal through signal fusion, determine the two-dimensional image of each target motion signal according to the modified dot matrix method, and input each two-dimensional image into different 2D-CNN branch models respectively, and then fuse the classification results output by each branch model through the DS evidence theory decision fusion method to obtain the target fault probability distribution, and determine the fault state of the robot according to the target fault probability distribution; the edge server sends the fault detection result of the robot to the monitoring platform of the safety mechanism device to activate the safety mechanism and issue a warning when the robot fails.

[0202] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.16 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a robot fault detection method is implemented.

[0203] Those skilled in the art will understand that Fig.16 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0204] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps described in any of the above method embodiments when executing the computer program.

[0205] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps described in any of the above method embodiments are implemented.

[0206] In one embodiment, a computer program product is provided, including a computer program, which implements the steps described in any of the above method embodiments when executed by a processor.

[0207] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0208] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0209] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A robot fault detection method, characterized in that: The method comprises: Obtain target motion signals of multiple joint motors during robot operation; Performing signal fusion processing on each of the target motion signals to determine a two-dimensional image corresponding to each of the target motion signals; Inputting each of the two-dimensional images into a preset fault detection model, and determining the fault probability distribution corresponding to each of the target motion signals according to the output of the fault detection model; The fault state of the robot is determined according to each of the fault probability distributions.

2. The method according to claim 1, characterized in that: The step of obtaining target motion signals of multiple joint motors during robot operation includes: Obtaining identification information of the robot; Acquire, from a message queue according to the identification information of the robot, an initial motion signal of each joint motor during the operation of the robot, wherein the initial motion signal includes a first vibration signal and a first torque signal of the joint motor; Each of the initial motion signals is preprocessed to determine a target motion signal, wherein the preprocessing includes sliding average filtering and wavelet packet decomposition, and the target motion signal includes a second vibration signal and a second torque signal.

3. The method according to claim 2, characterized in that Preprocessing each of the initial motion signals to determine a target motion signal also includes: Determine the theoretical torque signal of each joint motor according to the robot dynamics model; For each of the joint motors, a second torque signal is determined according to a residual of the theoretical torque signal and the preprocessed first torque signal.

4. The method according to any one of claims 1 to 3, characterized in that: The target motion signal is a one-dimensional time series, the two-dimensional image is symmetrical, and the signal fusion processing is performed on each of the target motion signals to determine the two-dimensional image corresponding to each of the target motion signals, including: Normalizing the value of each time point in each target motion signal; For the value of each target motion signal at each time point, the normalized value is mapped to a point in a polar coordinate system, and the radius, counterclockwise rotation angle and clockwise rotation angle of the point are determined according to the preset initial rotation angle, the preset angle magnification factor and the value.

5. The method according to claim 1, characterized in that The fault detection model includes a plurality of branch models, each of which has a different convolution kernel size, pooling strategy, and network depth. The two-dimensional images are input into a preset fault detection model, and the fault probability distribution corresponding to each target motion signal is determined according to the output of the fault detection model, including: Inputting each of the two-dimensional images into a different branch model, wherein the branch model includes a plurality of convolutional layers, a plurality of pooling layers, a global average pooling layer and a softmax classifier; The fault probability distribution corresponding to each of the target motion signals is determined according to the output of the softmax classifier of each of the branch models.

6. The method according to claim 1, characterized in that Determining the fault state of the robot according to each of the fault probability distributions includes: Based on the DS evidence theory, the above fault probability distributions are fused to obtain the fused target fault probability distribution; The fault state of the robot is determined according to the target fault probability distribution.

7. A robot fault detection device, characterized in that: The device comprises: An acquisition module is used to acquire target motion signals of multiple joint motors during the robot operation process; A fusion module, used for performing signal fusion processing on each of the target motion signals to determine a two-dimensional image corresponding to each of the target motion signals; A first determination module, used for inputting each of the two-dimensional images into a preset fault detection model, and determining a fault probability distribution corresponding to each of the target motion signals according to an output of the fault detection model; The second determination module is used to determine the fault state of the robot according to each of the fault probability distributions.

8. A robot fault detection system, characterized in that: The robot fault detection system includes a robot, a plurality of acquisition devices and a server; The robot comprises a control system and an execution device, wherein the execution device comprises a plurality of joint motors, each of which is used to complete a specified action under the control of the control system; The multiple acquisition devices are respectively arranged on the multiple joint motors of the robot, and are used to acquire the first vibration signal and the first torque signal of each joint motor of the robot; The server is used to obtain target motion signals of multiple joint motors during the robot operation process; Perform signal fusion processing on each of the target motion signals to determine a two-dimensional image corresponding to each of the target motion signals; input each of the two-dimensional images into a preset fault detection model, and determine the fault probability distribution corresponding to each of the target motion signals based on the output of the fault detection model; determine the fault state of the robot based on each of the fault probability distributions.

9. The system according to claim 8, characterized in that The robot further comprises a communication module, and the communication module is used for communication between the robot and the server, or for communication between the control system of the robot and the execution device.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

Images