Fault diagnosis method, electronic device, storage medium, and program product

Abstract

The application relates to the technical field of robots, and discloses a fault diagnosis method, an electronic device, a storage medium and a program product. In the method, an electronic device obtains actual state information of N joints of a multi-joint robot at a current time; for any target joint of the N joints, the actual state information of N-1 joints except the target joint is input into a first model to obtain first state prediction information and a predicted fault token of the target joint at the current time; a first prediction residual of the first state prediction information and the actual state information of the target joint is determined; in the case that the first prediction residual is greater than or equal to a first preset residual threshold, sequence features of a prediction residual sequence of the target joint within a first preset time length are determined; and based on the sequence features, the actual state information of the target joint and the predicted fault token, a target fault mode and a target severity of the target joint are obtained. The method can improve the accuracy of fault diagnosis.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and in particular to a fault diagnosis method, electronic device, storage medium, and program product. Background Technology

[0002] With the continuous development of robotics technology, multi-jointed robots (such as quadruped robots, hexapod robots, and robotic arms) are increasingly widely used in industrial fields such as industrial manufacturing, precision assembly, welding, and outdoor exploration and rescue. The reliability of multi-jointed robot systems is directly related to production efficiency, product quality, task success rate, and operational safety.

[0003] However, robot joints, drive systems, and transmission systems are prone to various failure modes due to long-term, complex operation. These include increased joint flexibility caused by actuator wear, joint locking caused by transmission mechanism jamming, decreased drive torque due to aging motor coil insulation, state measurement errors due to sensor drift, abnormal command execution due to controller malfunction, and performance degradation caused by outdoor environmental factors (terrain changes, load fluctuations). If these failures are not located and detected in a timely and accurate manner, they may lead to mission failure, equipment damage, or even safety accidents. Therefore, fault diagnosis technology for multi-joint robots is crucial to ensuring the reliable operation of robot systems. Summary of the Invention

[0004] This application provides a fault diagnosis method, an electronic device, a storage medium, and a program product. This method can improve the accuracy of fault diagnosis.

[0005] In a first aspect, this application provides a fault diagnosis method applied to an electronic device. The method includes: acquiring the actual state information of N joints of a multi-joint robot at the current moment, where N is an integer greater than 1; for any target joint among the N joints, inputting the actual state information of the N-1 joints other than the target joint into a first model to obtain the first state prediction information and the predicted fault token of the target joint at the current moment, wherein the predicted fault token is used to represent the predicted fault mode of the target joint and the predicted severity of the predicted fault mode based on a vector representation; determining the first prediction residual between the first state prediction information and the actual state information of the target joint; if the first prediction residual is greater than or equal to a first preset residual threshold, determining the sequence features of the prediction residual sequence of the target joint within a first preset time period before the current moment, wherein the sequence features include statistical features and frequency domain features; and inputting the sequence features, the actual state information of the target joint, and the predicted fault token into a second model to obtain the target fault mode of the target joint and the target severity of the target fault mode.

[0006] This method determines the first state prediction information of the target joint and the predicted fault token representing the predicted fault mode and predicted severity of the target joint based on the actual state information of N-1 joints other than the target joint. This results in high accuracy for both the first state prediction information and the predicted fault token. Furthermore, the method also determines the target fault mode and target severity of the target joint based on the residual between the first state prediction information and the actual state information of the target joint, as well as the predicted fault token. This multi-diagnosis approach further enhances the accuracy of the target fault mode and target severity, improving the fault diagnosis efficiency of each joint in a multi-joint robot.

[0007] In one possible implementation of the first aspect, the first model is trained as follows: A dynamic model of a multi-joint robot is obtained; for any first joint among N joints, a task instruction, K preset fault modes, and L preset severity levels corresponding to each preset fault mode are input into the dynamic model to obtain a pairing dataset for the first joint under the task instruction. The pairing dataset includes: normal state information of each of the N joints at different times; and fault state information of each of the N joints at different times when the first joint has different preset fault modes and different preset severity levels, where K and L are both integers greater than or equal to 1; the first model is trained based on the pairing dataset of each first joint among the N joints, the preset fault modes corresponding to each pairing dataset, and the preset severity levels to obtain the trained first model.

[0008] Understandably, since the paired dataset used to train the first model is obtained through simulation using the dynamics model of a multi-joint robot, compared to collecting real fault data, this method of obtaining the paired dataset can systematically enumerate all preset fault modes and their corresponding preset severity levels, making the data in the paired dataset more comprehensive, with strong coverage and high repeatability. Furthermore, this method does not require damaging the robot, resulting in lower acquisition costs and a simpler, more efficient, and controllable acquisition method for the paired dataset.

[0009] In one possible implementation of the first aspect, the first model is trained based on the paired dataset of each first joint in N joints, the preset fault mode corresponding to each paired dataset, and the preset severity, including: determining the reference vectors of the preset fault mode and preset severity corresponding to each paired dataset in D-dimensional space, wherein the directions of the reference vectors of different preset fault modes are different, different preset severity of the same preset fault mode corresponds to different distances in the same reference vector direction, and D is an integer greater than 1; and the first model is trained based on the paired dataset of each first joint and the reference vector corresponding to each paired dataset.

[0010] In one possible implementation of the first aspect, the first model is trained based on the paired dataset for each first joint and the reference vector corresponding to each paired dataset, including: training the first model using an objective function comprising a classification loss function, a contrastive learning loss function, and a regression loss function based on the paired dataset for each first joint and the reference vector corresponding to each paired dataset; wherein, the classification loss function is used to measure the accuracy of the predicted fault tokens for each first joint output by the first model; the contrastive learning loss function is used to enable the first model to distinguish the joint dynamic features corresponding to different preset fault modes; and the regression loss function is used to measure the accuracy of the state prediction information for each first joint output by the first model.

[0011] It can be understood that the classification loss function is used to measure the accuracy of the predicted fault token for each first joint output by the first model, for example, to calculate the difference between the predicted fault token for each first joint output by the first model and the corresponding reference vector. This classification loss function enables the first model to output accurate predicted fault tokens during the fault diagnosis process.

[0012] The contrastive learning loss function enables the first trained model to perform implicit information redundancy fault diagnosis on multi-joint robots based on the dynamic coupling relationship between each joint, without the need for additional hardware redundancy. Therefore, the diagnosis method is simpler and more accurate.

[0013] In one possible implementation of the first aspect, different target joints among the N joints correspond to different first models.

[0014] In this method, each first model is trained (or optimized) for the corresponding target joint, which makes the first model more accurate when diagnosing faults in the corresponding target joint.

[0015] In one possible implementation of the first aspect, the training set of the second model includes: a training residual sequence for training, sequence features of the training residual sequence, training state information of the joints, training fault tokens, and reference fault modes and reference severity.

[0016] In one possible implementation of the first aspect, the method further includes: inputting a predicted fault token into a third model to obtain a fault prevention strategy for the predicted fault mode and the predicted severity.

[0017] In one possible implementation of the first aspect, the third model is trained by: acquiring a training fault token for training; and performing reinforcement learning on the third model using a reward function based on the training fault token and domain-randomized environmental information, wherein the reward objective of the reward function includes: maximizing the task completion degree and the accuracy of the fault prevention strategy output by the third model, and the environmental information includes at least one of terrain information, load information, wind information, ground roughness information, or ground material information.

[0018] The method provided in this application can generate a fault prevention strategy for fault tokens, enabling timely motion adjustments for multi-joint robots with faulty joints. This improves the task performance of multi-joint robots and increases task completion efficiency. Furthermore, because domain-randomized environmental information is used during the training of the third model, this method is particularly suitable for multi-joint robots in complex outdoor environments and can also improve the success rate of transferring the third model to real multi-joint robots.

[0019] In one possible implementation of the first aspect, determining the sequence characteristics of the predicted residual sequence of the target joint within a first preset time period before the current moment includes: obtaining P joint subsets, each subset containing joints related to the target joint, each subset containing fewer than N joints, and each subset containing the same or different number of joints, where P is an integer greater than 1; inputting the actual state information of the joints in each subset into a first model to obtain second state prediction information of the target joint; determining the second prediction residual between the second state prediction information corresponding to each subset and the actual state information of the target joint to obtain P second prediction residuals; and determining the sequence characteristics of the predicted residual sequence of the target joint within the first preset time period before the current moment if the number of second prediction residuals greater than or equal to a first preset residual threshold is greater than a number threshold.

[0020] The method provided in this application, when the first predicted residual is greater than or equal to a first preset residual threshold, further performs cross-validation using other P subsets of joints, and only after successful validation does it further determine the target fault mode of the target joint. This cross-validation mechanism can reduce the probability of false alarms caused by single-point faults, has low sensitivity to environmental interference, and high robustness, thereby improving the accuracy of fault diagnosis.

[0021] In one possible implementation of the first aspect, the method further includes: inputting the actual state information of N-1 joints (excluding the target joint) out of N joints into a first model to obtain the third state prediction information of the target joint at the next moment from the current moment; determining the third prediction residual between the third state prediction information and the actual state information of the target joint at the next moment; performing a fitting operation on the third prediction residuals at different moments to obtain a residual growth curve; and determining the remaining service life of the target joint based on the residual growth curve and a second preset residual threshold.

[0022] The method provided in this application can determine the remaining service life by predicting the future state information of the target joint, and provide early warning. For example, for outdoor multi-joint robots, it can determine whether to continue performing the task or return to the base for maintenance in advance based on the current remaining task (e.g., remaining walking distance) and remaining service life, providing a reliable basis for the maintenance plan and parts procurement of multi-joint robots, which is beneficial to the health maintenance of multi-joint robots.

[0023] In a second aspect, this application provides an electronic device comprising: at least one processor; at least one memory; the at least one memory storing at least one program, which, when executed by at least one processor, causes the electronic device to perform the fault diagnosis method of the first aspect and any possible implementation thereof.

[0024] Thirdly, this application provides a storage medium storing instructions that, when executed on an electronic device, cause the electronic device to perform the fault diagnosis method of the first aspect and any possible implementation thereof.

[0025] Fourthly, this application provides a program product comprising: instructions that, when executed on an electronic device, cause the electronic device to perform the fault diagnosis method of the first aspect and any possible implementation thereof.

[0026] The beneficial effects of the second to fourth aspects can be found in the first aspect and the beneficial effects of any possible implementation of the first aspect, and will not be repeated here. Attached Figure Description

[0027] Figure 1 According to some embodiments of this application, a flowchart of a fault diagnosis method provided in this application is shown;

[0028] Figure 2 According to some embodiments of this application, a flowchart of a training method for a first model is shown;

[0029] Figure 3According to some embodiments of this application, a flowchart of a method for determining the sequence characteristics of a predicted residual sequence of a target joint within a first preset time period is shown.

[0030] Figure 4 According to some embodiments of this application, a flowchart illustrating a method for determining the remaining service life of a target joint is shown.

[0031] Figure 5 According to some embodiments of this application, a schematic diagram of the structure of a fault diagnosis system is shown;

[0032] Figure 6 According to some embodiments of this application, a schematic diagram of the structure of an electronic device is shown. Detailed Implementation

[0033] The illustrative embodiments of this application include, but are not limited to, fault diagnosis methods, electronic devices, storage media, and program products.

[0034] As mentioned above, fault diagnosis technology for multi-joint robots is crucial to ensuring the reliable operation of robot systems.

[0035] Some fault diagnosis techniques employ a single-joint independent detection method, which determines whether a fault exists in the target joint by analyzing its own state information (e.g., vibration signal analysis, temperature detection, current signal analysis). However, this method of determining the presence of a fault based solely on the state information of the target joint is relatively simplistic, resulting in low accuracy in fault identification.

[0036] Therefore, to solve the above-mentioned technical problems, this application provides a fault diagnosis method. In this method, an electronic device acquires the actual state information of N (N is an integer greater than 1) joints of a multi-joint robot at the current moment; for any target joint among the N joints, the actual state information of the N-1 joints other than the target joint is input into a first model to obtain the first state prediction information and a predicted fault token of the target joint at the current moment. The predicted fault token is used to represent the predicted fault mode of the target joint and the predicted severity of the predicted fault mode based on a vector representation; a first prediction residual is determined between the first state prediction information and the actual state information of the target joint; if the first prediction residual is greater than or equal to a first preset residual threshold, the sequence features of the prediction residual sequence of the target joint within a first preset time period before the current moment are determined, the sequence features including statistical features and frequency domain features; the sequence features, the actual state information of the target joint, and the predicted fault token are input into a second model to obtain the target fault mode of the target joint and the target severity of the target fault mode.

[0037] This method determines the first state prediction information of the target joint and the predicted fault token representing the predicted fault mode and predicted severity of the target joint based on the actual state information of N-1 joints other than the target joint. This results in high accuracy for both the first state prediction information and the predicted fault token. Furthermore, the method also determines the target fault mode and target severity of the target joint based on the residual between the first state prediction information and the actual state information of the target joint, as well as the predicted fault token. This multi-diagnosis approach further enhances the accuracy of the target fault mode and target severity, improving the fault diagnosis efficiency of each joint in a multi-joint robot.

[0038] Furthermore, this method is applicable to the fault diagnosis process of multi-joint robots with arbitrary degrees of freedom, and does not require modification of the algorithm framework when performing fault diagnosis on different multi-joint robots, thus exhibiting good scalability.

[0039] It is understood that the fault diagnosis method provided in this application is applicable to any electronic device capable of fault diagnosis, including but not limited to computers, wearable devices, computers with wireless transceiver capabilities, virtual reality (VR) devices, augmented reality (AR) devices, wireless devices in industrial control, wireless devices in remote medical surgery, wireless devices in transportation safety, wireless devices in smart cities, wireless devices in smart homes, and so on. This application does not limit the type of electronic device.

[0040] For example, Figure 1 According to some embodiments of this application, a flowchart of a fault diagnosis method provided in this application is shown. Figure 1 The process shown is executed by electronic devices. For ease of description, this will be explained in the following introduction. Figure 1 The execution entity will not be described again in the process shown. For example... Figure 1 As shown, the method includes:

[0041] S101: Obtain the actual state information of N joints of the multi-joint robot at the current moment, where N is an integer greater than 1.

[0042] This application does not limit the type of multi-joint robot; any robot with multiple joints is acceptable. For example, a multi-joint robot can be a quadruped robot, a hexapod robot, or a robotic arm with multiple joints. This application also does not limit the application scenario of the multi-joint robot; it can be used in indoor or outdoor environments.

[0043] In this embodiment of the application, the actual state information of each of the N joints may include any information used to represent the state of the joint. For example, the actual state information includes, but is not limited to, at least one of the following: angle, angular velocity, angular acceleration, driving torque, velocity, and acceleration.

[0044] This application does not limit the method of obtaining actual state information; various sensors can be used to obtain the corresponding actual state information. For example, a torque sensor can be used to obtain the driving torque of the joint, and an inertial measurement unit (e.g., a gyroscope) can be used to obtain the angular velocity of the joint. The positions of the various sensors can be set according to the actual data acquisition requirements. For example, the sensors can be located inside each joint, near each joint, or on the torso of the multi-joint robot.

[0045] It is understandable that electronic devices can acquire the actual status information of N joints in real time, or periodically, that is, acquire the actual status information once at a preset interval. The preset interval can be set based on experience or flexibly adjusted according to the actual application scenario.

[0046] In this embodiment, the actual state information of N joints at the current moment can be represented in the form of a state vector. For example, the expression of the state vector can be found in the following formula (1).

[0047] S(t) = [s1(t), s2(t), ..., s N Formula (1) (t)].

[0048] In formula (1), S(t) represents the state vector of the actual state information of N joints at the current moment; s i (t) represents the actual state information of the i-th joint at the current moment, where i is an integer greater than or equal to 1 and less than or equal to N.

[0049] S102: For any target joint among the N joints, input the actual state information of the N-1 joints other than the target joint into the first model to obtain the first state prediction information and the predicted fault token of the target joint at the current time.

[0050] The predicted fault token is used to represent the predicted fault mode of the target joint and the predicted severity of the predicted fault mode based on the vector representation.

[0051] It is understood that the first model in this application has the ability to predict the first state of the target joint at the current moment and predict the fault token based on the actual state information of N-1 joints. The training method of the first model is described below. Figure 2 The relevant textual descriptions will not be elaborated here.

[0052] In some embodiments, N joints correspond to the same first model, meaning that the same first model can be used to diagnose faults in any one of the N joints. In other embodiments, different target joints among the N joints correspond to different first models, meaning that the first model corresponding to a specific target joint needs to be used to diagnose faults in that specific joint.

[0053] When different target joints in N joints correspond to different first models, the first model corresponding to the j-th target joint (j is an integer greater than or equal to 1 and less than or equal to N) in the N joints can be the first model f. The actual state information of the N joints (e.g., the state vector S(t) mentioned above) excluding the target joint is input into the first model f. This allows us to obtain the first state prediction information and the prediction fault token for the j-th target joint at the current moment.

[0054] For example, the expression for the first state prediction information of the j-th target joint at the current time can be found in the following formula (2).

[0055] (t)=f (s1(t),...,s j-1 (t),s j+1 (t),...,s N Formula (2).

[0056] In formula (2), (t) represents the first state prediction information of the j-th target joint at the current time.

[0057] In this method, each first model is trained (or optimized) for the corresponding target joint, which makes the first model more accurate when diagnosing faults in the corresponding target joint.

[0058] When N joints correspond to the same first model, the same first model can be used regardless of which target joint is being diagnosed. This means that, in this case, besides inputting the actual state information of the N-1 joints (excluding the target joint) into the first model, one can also input the actual state information of all N joints into the first model, and when diagnosing different target joints, input the identifier (or index) of the target joint into the first model. In this way, the first model can still obtain the first state prediction information and predicted fault token corresponding to the target joint based on the above method. The identifier of the target joint can be an integer between 0 and N-1.

[0059] This method can perform fault diagnosis on each target joint based on only a first model, requiring fewer parameters to be trained, resulting in high efficiency in both training and inference, and low computational and storage costs, making it suitable for real-time operation in robot controllers.

[0060] It is understandable that this method determines the first state prediction information of the target joint and the predicted fault token representing the predicted fault mode and predicted severity of the target joint based on the actual state information of N-1 joints other than the target joint. Compared with methods that detect each joint independently, this method can achieve higher accuracy in both the first state prediction information and the predicted fault token.

[0061] S103: Determine the first prediction residual between the first state prediction information and the actual state information of the target joint.

[0062] For example, the expression for the first prediction residual can be found in the following formula (3).

[0063] r (t)=|s (t)- (t)| Formula (3).

[0064] In formula (3), r (t) represents the first prediction residual corresponding to the j-th target joint; s (t) represents the actual state information of the j-th target joint.

[0065] S104: If the first prediction residual is greater than or equal to the first preset residual threshold, determine the sequence characteristics of the prediction residual sequence of the target joint within the first preset time period before the current time. The sequence characteristics include statistical characteristics and frequency domain characteristics.

[0066] This application does not limit the values ​​of the first preset residual threshold (e.g., θ1) and the first preset duration. They can be set based on experience or flexibly adjusted according to the actual application scenario. The upper limit of the time corresponding to the first preset duration can be the current time.

[0067] It is understood that if the first predicted residual is greater than or equal to the first preset residual threshold, the target joint can be determined as a potentially faulty joint. Further, a predicted residual sequence of the target joint within a first preset time period can be obtained. This predicted residual sequence includes multiple predicted residuals of the target joint within the first preset time period. The method for determining the multiple predicted residuals of the target joint within the first preset time period is the same as the principle for determining the first predicted residual mentioned above, and will not be repeated here.

[0068] After obtaining the predicted residual sequence of the target joint within a first preset time period, the sequence characteristics of the predicted residual sequence are determined. The sequence characteristics may include multiple statistical characteristics of the predicted residuals (e.g., mean, standard deviation, peak value, skewness, kurtosis, etc.), and may also include frequency domain characteristics. Among them, the frequency domain characteristics may include the distribution of multiple predicted residual frequency components obtained by Fourier transform.

[0069] It is understandable that if the first predicted residual is less than the first preset residual threshold, the fault diagnosis process of the target joint will end, that is, the target joint will be determined to be fault-free.

[0070] S105: Input the sequence features, the actual state information of the target joint, and the predicted fault token into the second model to obtain the target fault mode of the target joint and the target severity corresponding to the target fault mode.

[0071] In some embodiments, the second model in this application has the ability to predict discrete target failure modes and target severity of the target joint based on sequence features, actual state information of the target joint, and predicted failure tokens. In this case, the target failure mode and the predicted failure token indicated by the predicted failure token may be the same or different; the target severity and the predicted severity indicated by the predicted failure token may be the same or different.

[0072] In this scenario, the training set of the second model includes training residual sequences, sequence features of the training residual sequences, training state information of the joints, training fault tokens (used to determine the distance between the training fault tokens and the normal state), and reference fault modes and reference severity. During training, the training residual sequences, sequence features of the training residual sequences, and training fault tokens are input into the second model to obtain the predicted fault modes and predicted severity output by the second model. The model is then repeatedly trained based on the differences between the predicted fault modes and reference fault modes output by the second model, and the differences between the predicted severity output by the second model and the reference severity, until a fully trained second model is obtained. Then, the sequence features, the actual state information of the target joint, and the predicted fault tokens are input into this pre-trained second model to obtain the target fault modes and target severity of the target joint.

[0073] In other embodiments, the second model may include a classification network and a severity mapping module. The classification network is used to obtain the target failure mode based on sequence features, the actual state information of the target joint, and the predicted failure token. The severity mapping module is used to obtain the target severity based on the predicted failure token. Since the predicted failure token indicates the predicted severity, in this method, the target severity and the predicted severity are the same, while the target failure mode and the predicted failure token indicated by the predicted failure token may be the same or different.

[0074] In this scenario, the training set for the classification network includes training residual sequences, sequence features of the training residual sequences, training state information of the joints, training fault tokens, and reference fault patterns. During training, the training residual sequences, sequence features of the training residual sequences, and training fault tokens are input into the classification network to obtain the confidence scores for each reference fault pattern. The model is then repeatedly trained based on the differences between the confidence scores of each reference fault pattern and the reference fault patterns themselves, until a fully trained classification network is obtained. Then, the sequence features, the actual state information of the target joint, and the predicted fault tokens are input into this pre-trained classification network to obtain the confidence scores for each preset fault pattern. The preset fault pattern with the highest confidence score is then selected as the target fault pattern.

[0075] This application does not limit the content of the target failure mode. For example, the target failure mode of a target joint can be any one of the following: free swing (i.e., loss of joint driving torque), locked (i.e., inability to move the joint), torque limited (i.e., decrease in maximum joint torque), speed limited (i.e., decrease in maximum joint angular velocity), and damping increased (i.e., increase in joint friction or damping).

[0076] This application does not limit the form of the target severity. For example, a mask code can be used to represent the target severity. The mask code can be an integer from 1 to 255, where 10-50 represents a minor fault, 51-200 represents a moderate fault, and 201-255 represents a severe fault.

[0077] It is understood that the method provided in this application determines the target fault mode and target severity of the target joint based on the residual between the first state prediction information and the actual state information of the target joint, the predicted fault token, and other information. This multi-diagnosis method makes the accuracy of the target fault mode and target severity higher, and improves the fault diagnosis efficiency of each joint of the multi-joint robot.

[0078] In some other embodiments, in addition to outputting a predicted fault token, the first model may also output a discrete predicted severity, which can then be directly used as the target severity of the target joint.

[0079] In other embodiments, after determining the target fault mode and severity of the target joint, a ternary structured fault code can be obtained. This structured fault code may include the target joint's identifier (ID), the target fault mode, and the target severity. The target joint's identifier can be an integer from 0 to N-1. The target severity can be represented using the mask code described above. After determining the structured fault code, corresponding emergency measures can be taken, such as slowing down operation, limiting the joint's range of motion, or switching to a backup control strategy. For outdoor mobile robots, gait adjustments or navigation path changes can also be made based on the fault and current terrain to improve the reliability, stability, and task completion rate of the multi-joint robot.

[0080] The training method of the first model mentioned in S102 above will be described in detail below.

[0081] For example, Figure 2 According to some embodiments of this application, a flowchart illustrating a training method for a first model is shown. Figure 2 As shown, the method includes:

[0082] S201: Obtain the dynamic model of the multi-joint robot.

[0083] It is understood that the dynamic model in this application is used to reproduce the physical behavior of a real multi-joint robot in a virtual world. Therefore, this dynamic model possesses accurate joint dynamic parameters (e.g., mass, coefficient of friction, limits) and considers factors such as controller-level command latency and actuator characteristics, sensor measurement noise and drift, and outdoor terrain conditions (e.g., variations in ground roughness, slopes, and damping changes in soft ground). Thus, this dynamic model can more accurately simulate the physical behavior of a real outdoor robot.

[0084] In this embodiment, the dynamic model can be pre-built in a high-fidelity physics simulation environment (e.g., Isaac Gym, MuJoCo, CoppeliaSim). Thus, the dynamic model can be directly obtained when training the first model.

[0085] S202: For any first joint among N joints, input the task instruction, K preset fault modes and L preset severity levels corresponding to each preset fault mode into the dynamic model to obtain the paired dataset of the first joint under the task instruction.

[0086] The paired dataset includes: normal state information (or normal operation trajectory) of each of the N joints at different times; and fault state information (or fault operation trajectory) of each of the N joints at different times when the first joint has different preset fault modes and different preset severity levels, where K and L are both integers greater than or equal to 1.

[0087] It can be understood that the normal state information of each of the N joints at different times is the state information of each joint at different times during the completion of the task instruction, assuming that no preset fault mode exists in any joint. The fault state information of each of the N joints at different times is the state information of each joint at different times during the completion of the task instruction, assuming that the first joint has different preset fault modes and different preset severity levels.

[0088] For example, preset fault modes include free swing, locking, torque limiting, speed limiting, and increased damping (i.e., K is 5). Each preset fault mode has a different preset severity, such as preset severity of 10%, 20%, ..., 90%, 100% (i.e., L is 10). Therefore, each first joint has K×L paired datasets, and N joints have a total of N×K×L paired datasets.

[0089] This application does not limit the content of the task instructions. The task instructions can be any instructions that can be issued to the multi-joint robot, such as gait cycle related instructions, turning instructions, running instructions, etc.

[0090] Understandably, since the paired dataset used to train the first model is obtained through simulation using the dynamics model of a multi-joint robot, compared to collecting real fault data, this method of obtaining the paired dataset can systematically enumerate all preset fault modes and their corresponding preset severity levels, making the data in the paired dataset more comprehensive, with strong coverage and high repeatability. Furthermore, this method does not require damaging the robot, resulting in lower acquisition costs and a simpler, more efficient, and controllable acquisition method for the paired dataset.

[0091] S203: Based on the paired dataset of each first joint in N joints, the preset fault mode and preset severity corresponding to each paired dataset, the first model is trained to obtain the trained first model.

[0092] It can be understood that each paired dataset corresponds to a discrete preset fault mode and preset severity. Therefore, in this step, the reference vector of the preset fault mode and preset severity corresponding to each paired dataset in the D-dimensional space (or latent space) can be determined.

[0093] In this implicit space, the reference vectors for different preset fault modes have different directions, and different preset severity levels of the same preset fault mode correspond to different distances along the same reference vector direction. No preset fault mode corresponds to the zero vector or the standard reference vector z0.

[0094] In some embodiments, preset fault modes and preset severity can be converted into a D-dimensional continuous vector (i.e., a reference vector) z based on a predefined token encoder E.

[0095] After determining the reference vector corresponding to each pairing dataset, the first model is trained based on the pairing dataset of each first joint and the reference vector corresponding to each pairing dataset.

[0096] In some embodiments, the first model may be trained using an objective function that includes a classification loss function (e.g., a CrossEntropy term), a contrastive learning loss function (e.g., a Contrastive term), and a regression loss function.

[0097] The classification loss function measures the accuracy of the predicted fault token for each first joint output by the first model. For example, it calculates the difference between the predicted fault token for each first joint output by the first model and the corresponding reference vector. This classification loss function enables the first model to output accurate predicted fault tokens during fault diagnosis.

[0098] The contrastive learning loss function enables the first model to distinguish the joint dynamics features corresponding to different preset fault modes. Specifically, it teaches the first model to differentiate between the joint coupling features of each joint in a normal state and those in a fault state, thus prompting the first model to implicitly converge samples of the same fault mode and diverge samples of different fault modes. This contrastive learning loss function allows the first model to distinguish between normal joint states (e.g., normal joint states caused by environmental changes) and faulty joint states caused by faults during fault diagnosis.

[0099] It is understandable that the contrastive learning loss function enables the first model after training to perform implicit information redundancy fault diagnosis on multi-joint robots based on the dynamic coupling relationship between each joint, without the need for additional hardware redundancy. Therefore, the diagnosis method is relatively simple and has high accuracy.

[0100] The regression loss function is used to measure the accuracy of the state prediction information (e.g., including the current state prediction information) output by the first model for each first joint. It can be understood that when different joints in N joints correspond to different first models, the regression loss function is used to measure the accuracy of the state prediction information of the first joint corresponding to the first model.

[0101] During training, classification loss, contrastive learning loss, and regression loss can be determined separately to obtain the total target loss. If the target loss does not meet the training termination condition, the parameters of the first model are adjusted based on the target loss, and the target loss is determined again, and so on. If the target loss meets the training termination condition, the training process ends, and the trained first model is obtained.

[0102] After training the first model, its fault diagnosis accuracy (i.e., precision), false positive rate, and false negative rate can be verified on a simulation validation dataset to ensure that the obtained first model has good generalization ability. The false positive rate is the proportion of joints that are incorrectly classified as "positive" (e.g., faulty) out of all joints that are actually "negative" (e.g., normal or fault-free). The false negative rate is the proportion of joints that are incorrectly classified as "negative" (e.g., normal or fault-free) out of all joints that are actually "positive" (e.g., faulty).

[0103] The training and validation method for the first model provided in this application can improve the accuracy of fault diagnosis of the first model during inference (i.e., actual deployment). Furthermore, since the subsequent third model is trained based on the output of the first model to obtain the fault prevention strategy, and the parameters of the first model (e.g., weight parameters) have been initialized to a well-defined feature space, this method can significantly accelerate the convergence speed of the subsequent third model training, thus improving the training speed of the third model.

[0104] Furthermore, based on the training method of the first model described above, the output of the first model during the fault diagnosis process is not only discrete predicted fault modes and predicted severity, but also predicted fault tokens (i.e., continuous implicit vectors). These predicted fault tokens encode the semantic information of the joint fault state and can be used to represent discrete predicted fault modes and predicted severity. Therefore, regardless of whether N joints correspond to one first model or different joints correspond to different first models, all predicted fault tokens output by the first model share a unified implicit space.

[0105] Thus, in some embodiments, similarity calculations (e.g., cosine similarity or Euclidean distance (i.e., L2 distance)) can be performed on different predicted fault tokens to identify associated or cascading faults. For example, if predicted fault tokens from multiple adjacent joints cluster in the implicit space, they can be identified as cascading faults. Furthermore, interpolation can be performed on the predicted severity of the same target joint at different times to analyze the evolution trajectory of the severity of the corresponding fault modes.

[0106] In other embodiments, the first state prediction information and future state prediction information of each target joint output by the first model can also be projected into the above-mentioned implicit space to form a joint representation with the predicted fault tokens of each target joint, so as to integrate the fault data of each target joint and improve the robustness of the fault diagnosis method.

[0107] It is understandable that, in S104 above, after determining that the target joint is a potentially faulty joint when the first predicted residual is greater than or equal to the first preset residual threshold, the predicted residual sequence of the target joint within the first preset time period is directly obtained, thereby determining the sequence characteristics of the predicted residual sequence. In other embodiments, after determining that the target joint is a potentially faulty joint, cross-validation can be performed using other P joint subsets, and after successful validation, the predicted residual sequence of the target joint within the first preset time period is obtained, thereby determining the sequence characteristics of the predicted residual sequence.

[0108] For example, Figure 3 According to some embodiments of this application, a flowchart illustrating a method for determining the sequence characteristics of a predicted residual sequence of a target joint within a first preset time period is shown. Figure 3 As shown, the method includes:

[0109] S301: Obtain P joint subsets, each of which contains joints related to the target joint. The number of joints in each subset is less than N, and the number of joints in each subset may be the same or different.

[0110] In the embodiments of this application, the joints included in each of the P (P is an integer greater than 1) joint subsets may be, for example, joints adjacent to the target joint, joints farthest from the target joint, joints symmetrical to the target joint axis, etc.

[0111] S302: Input the actual state information of the joints in each joint subset into the first model to obtain the second state prediction information of the target joint.

[0112] It is understandable that the method for determining the second state prediction information is the same as the method for determining the first state prediction information in S102 above, and will not be repeated here.

[0113] It is understandable that inputting the actual state information of a subset of joints into the first model can yield a second state prediction for the target joint. Therefore, inputting the actual state information of P subsets of joints into the first model can yield P second state predictions.

[0114] S303: Determine the second prediction residual between the second state prediction information corresponding to each joint subset and the actual state information of the target joint, and obtain P second prediction residuals.

[0115] It is understandable that the method for determining the second prediction residual is the same as the method for determining the first prediction residual in S103 above, and will not be repeated here.

[0116] S304: If, among the P second prediction residuals, the number of second prediction residuals that are greater than or equal to the first preset residual threshold is greater than the number threshold, determine the sequence characteristics of the prediction residual sequence of the target joint within the first preset time period before the current time.

[0117] This application does not limit the value of the quantity threshold; it is acceptable as long as the quantity threshold is greater than or equal to 1 and less than or equal to P.

[0118] If, among the P second prediction residuals, the number of second prediction residuals greater than or equal to the first preset residual threshold is greater than the quantity threshold, it indicates that the target joint is likely to be a faulty joint. Therefore, the sequence characteristics of the prediction residual sequence corresponding to the target joint can be determined to further execute S105 above, and determine the target fault mode and target severity of the target joint.

[0119] If, among the P second prediction residuals, the number of second prediction residuals greater than or equal to the first preset residual threshold is less than or equal to the quantity threshold, then the probability that the target joint is a faulty joint is relatively low. Therefore, the fault diagnosis process for the target joint can be terminated, i.e., the target joint is determined to be fault-free.

[0120] The method provided in this application, when the first predicted residual is greater than or equal to a first preset residual threshold, further performs cross-validation using other P subsets of joints, and only after successful validation does it further determine the target fault mode of the target joint. This cross-validation mechanism can reduce the probability of false alarms caused by single-point faults, has low sensitivity to environmental interference, and high robustness, thereby improving the accuracy of fault diagnosis.

[0121] In other embodiments, the method provided in this application can also predict the remaining service life of the target joint.

[0122] Specifically, during the training of the first model, the regression loss function is used not only to measure the accuracy of the current state prediction information of each first joint output by the first model, but also to measure the accuracy of the future state prediction information of each first joint. Thus, the trained first model can output not only the first state prediction information of the target joint at the current moment, but also the future state prediction information of the target joint. Then, the remaining lifespan of the target joint is determined based on the residual between the future state prediction information and the corresponding actual state information.

[0123] For example, Figure 4 According to some embodiments of this application, a flowchart illustrating a method for determining the remaining service life of a target joint is shown. Figure 4 As shown, the method includes:

[0124] S401: Input the actual state information of N-1 joints (excluding the target joint) out of N joints into the first model to obtain the third state prediction information of the target joint at the next time step from the current time step.

[0125] It is understandable that, since the first model has the ability to output the future state prediction information of each first joint at the next moment of the current moment during the training process, in this S401, the first model can output not only the first state prediction information of the target joint, but also the future state prediction information of the target joint at the next moment of the current moment (i.e., the third state prediction information).

[0126] It is understandable that if the first model can output information about future state prediction, the first model can also be called a diagnosis and prediction (D&P) network.

[0127] The "next moment" refers to the next fault diagnosis moment after the current moment, and the interval between the current moment and the next moment can be determined based on a preset diagnosis interval. For example, the diagnosis interval can be 0.5 seconds, 1 second, 2 seconds, etc.

[0128] For example, the expression for the third-state prediction information can be found in the following formula (4).

[0129] (t+Δt)=f (s1(t),...,s j-1 (t),s j+1 (t),...,s N Formula (4).

[0130] In formula (4), (t+Δt) represents the third state prediction information of the target joint at the next time step after the current time step; Δt represents the time interval between the next time step after the current time step and the current time step.

[0131] S402: Determine the third prediction residual between the third state prediction information and the actual state information at the next moment.

[0132] For example, the expression for the third prediction residual (or prognostic residual) can be found in the following formula (5).

[0133] r (t+Δt)=|s (t+Δt)- (t+Δt)| Formula (5).

[0134] In formula (5), r (t+Δt) represents the third prediction residual corresponding to the j-th target joint; s (t+Δt) represents the actual state information of the j-th target joint at the next time step after the current time step.

[0135] S403: Perform a fitting operation on the third prediction residuals at different times to obtain the residual growth curve.

[0136] It is understandable that, based on the S402 approach, the third prediction residuals of the target joint at different times can be obtained. For example, at the next time step, the future state prediction information for the time step after the next time step (i.e., the time step after the next step) can be obtained based on the first model. Then, based on the future state prediction information for the time step after the next step and the actual state information for the time step after the next step, the corresponding third prediction residual can be obtained. Similarly, for the target joint, the corresponding third prediction residual can be obtained at each different time step in the above manner.

[0137] After obtaining the third prediction residuals at different times, the failure development trend of the target joint is predicted using multiple third prediction residuals. For example, regression methods such as exponential fitting and linear fitting can be used to obtain residual growth curves for multiple third prediction residuals. The number of third prediction residuals used to determine the residual growth curves can be set based on experience.

[0138] In some embodiments, when the predicted fault mode indicated by the predicted fault token of the target joint is a non-acute fault, a third predicted residual of the target joint at different times can be obtained, thereby obtaining a residual growth curve. Here, a non-acute fault can be a fault mode with low severity, such as a slight increase in damping.

[0139] S404: Determine the remaining service life of the target joint based on the residual growth curve and the second preset residual threshold.

[0140] In this embodiment, the residual growth curve can be represented by an exponential function or a linear function. Therefore, given a second preset residual threshold (e.g., θ2), the time when the third predicted residual reaches the second preset residual threshold can be determined based on the residual growth curve and the second preset residual threshold, thereby determining the remaining useful life (RUL) of the target joint.

[0141] The value of the second preset residual threshold can be set based on experience or flexibly adjusted according to the actual usage scenario.

[0142] In other embodiments, besides being based on the above... Figure 4 The method shown predicts the remaining service life of the target joint. It can also obtain multiple third-state predictions of the target joint within a second preset time period based on the first model. That is, the first model can output multiple third-state information of the target joint at the current moment at multiple future moments. The lower limit of the second preset time period can be the current moment.

[0143] Then, the third prediction residuals of multiple third state prediction information and the actual state information at the corresponding time are determined to obtain multiple third prediction residuals. Then, the multiple third prediction residuals are fitted to obtain the residual growth curve, so as to obtain the remaining service life of the target joint.

[0144] The method provided in this application can determine the remaining service life by predicting the future state information of the target joint, and provide early warning. For example, for outdoor multi-joint robots, it can determine whether to continue performing the task or return to the base for maintenance in advance based on the current remaining task (e.g., remaining walking distance) and remaining service life, providing a reliable basis for the maintenance plan and parts procurement of multi-joint robots, which is beneficial to the health maintenance of multi-joint robots.

[0145] In some embodiments, after obtaining the predicted fault token of the target joint based on the first model, the predicted fault token can be input into the third model to obtain a fault prevention strategy targeting the predicted fault mode and the predicted severity.

[0146] It is understandable that fault prevention strategies are action adjustment strategies (or adaptive control strategies) for target joints in predicted fault modes, such as slowing down or changing gait, in order to maintain the stability of multi-joint robots.

[0147] For example, if the predicted failure mode is locked, that is, the target joint (e.g., the leg joint) is mechanically stuck and cannot move as instructed, then the failure prevention strategy output by the third model could be to lift the faulty leg and complete the task instruction using only the healthy leg.

[0148] In this embodiment, the third model can be a reinforcement learning (RL) agent. This embodiment does not limit the training method of the third model. For example, reinforcement learning can be performed on the third model using a reward function based on training fault tokens and domain-randomized environmental information used for training. The environmental information includes at least one of terrain information (e.g., slope), load information, wind information, ground roughness information, or ground material information (e.g., soft ground). Domain-randomized environmental information means that different levels or types of environmental information are randomly input during the reinforcement learning process.

[0149] The reward function's objective includes maximizing task completion and the accuracy of the fault prevention strategy output by the third model. Thus, the fault prevention strategy output by the trained third model has high accuracy, and this strategy enables the multi-joint robot to achieve the highest task completion rate.

[0150] The method provided in this application can generate a fault prevention strategy for fault tokens, enabling timely motion adjustments for multi-joint robots with faulty joints. This improves the task performance of multi-joint robots and increases task completion efficiency. Furthermore, because domain-randomized environmental information is used during the training of the third model, this method is particularly suitable for multi-joint robots in complex outdoor environments and can also improve the success rate of transferring the third model to real multi-joint robots.

[0151] In some embodiments, after obtaining the trained third model, the third model can be transferred from the physical simulation environment (e.g., using the Sim2Real transfer method) to a real multi-joint robot (e.g., a quadruped robot, a hexapod robot, or other multi-joint robots) for real-world testing, so that the third model can maintain good performance on the real multi-joint robot and improve the generalization ability of the third model.

[0152] In some embodiments, during the training of the third model, it may not be necessary to fine-tune the parameters of the first model, meaning the first model can be used as a fixed feature extractor. Alternatively, the parameters of the first model can be fine-tuned. In the case of fine-tuning the parameters of the first model, the first model used in S102 above is based on the aforementioned... Figure 2 The model was trained as shown and then fine-tuned as described above. This method can improve the accuracy of the first model's output in predicting fault tokens.

[0153] For example, Figure 5 According to some embodiments of this application, a schematic diagram of the structure of a fault diagnosis system is shown. For example... Figure 5 As shown, the fault diagnosis system includes a simulation module, a fault injection module, a pre-training module, a reinforcement learning module, a data acquisition module, a preprocessing module, a prediction module, a residual calculation module, a fault detection and location module, a fault classification module, an implicit spatial projection module, a prognostic assessment module, and an output module.

[0154] The simulation module is used to build a dynamic model of a multi-joint robot in a high-fidelity physical simulation environment (e.g., Isaac Gym, MuJoCo, CoppeliaSim). The fault injection module is used to input the task command, K preset fault modes, and L preset severity levels corresponding to each preset fault mode into the dynamic model to obtain the paired dataset of the first joint under the task command, which is used to execute S202.

[0155] The pre-training module is used to train the first model based on the paired dataset of each first joint in N joints, the preset fault mode and preset severity corresponding to each paired dataset, to obtain the trained first model, which is used to execute S203. The reinforcement learning module is used to integrate the first model into the training process of the third model, and through the third model, obtain the predicted fault token of the target joint based on the first model, and then obtain the fault prevention strategy for the predicted fault mode and predicted severity.

[0156] The data acquisition module is used to obtain the actual state information of N joints of the multi-joint robot at the current moment, i.e., to execute S101. The preprocessing module is used to perform filtering, normalization, and other preprocessing on the acquired actual state information of the N joints at the current moment.

[0157] The prediction module is used to input the actual state information of N-1 joints (excluding the target joint) out of N joints into the first model to obtain the first state prediction information and prediction fault token of the target joint at the current moment. This is used to execute S102 and also to obtain the future state prediction information of the target joint. The implicit space projection module is used to project the prediction fault tokens of each target joint into a shared implicit space.

[0158] The residual calculation module is used to determine the first prediction residual between the first state prediction information and the actual state information of the target joint, i.e., to execute S103. The fault detection and location module is used to determine whether the target joint is a faulty joint based on the relationship between the first prediction residual and the first preset residual threshold, and is also used to determine whether the target joint is a faulty joint by cross-validation through other P joint subsets.

[0159] The fault classification module is used to obtain the target fault mode based on sequence features, the actual state information of the target joint, and the predicted fault command, i.e., for executing S105. The prognostic assessment module is used to determine the remaining service life of the target joint, i.e., for executing the aforementioned... Figure 4 The method is shown. The output module is used to generate and output structured fault codes based on the target fault mode and target severity, for use in decision-making and fault handling by the upper-level controller.

[0160] In this embodiment, the fault diagnosis system can be migrated from a physical simulation environment (e.g., using the Sim2Real transfer method) to a real multi-joint robot (e.g., a quadruped robot, a hexapod robot, or other multi-joint robots) in an outdoor environment for real-world testing. This fault diagnosis system can then operate reliably in complex outdoor environments.

[0161] In some embodiments, this application also provides a storage medium storing instructions that, when executed on an electronic device, cause the electronic device to perform the fault diagnosis method described in the above embodiments.

[0162] In some embodiments, this application also provides a program product, including: instructions that, when executed on an electronic device, cause the electronic device to implement the fault diagnosis method described in the above embodiments.

[0163] In some embodiments, this application also provides an electronic device, which includes: at least one processor; at least one memory; the at least one memory storing at least one program, which, when executed by at least one processor, causes the electronic device to perform the fault diagnosis method described in the above embodiments.

[0164] For example, Figure 6 According to some embodiments of this application, a schematic diagram of the structure of an electronic device is shown.

[0165] In some embodiments, the electronic device may include one or more processors 604, a system control logic unit 608 connected to at least one of the one or more processors 604, a system memory 612 connected to the system control logic unit 608, a non-volatile memory (NVM) 616 connected to the system control logic unit 608, and a network interface 620 connected to the system control logic unit 608.

[0166] In some embodiments, processor 604 may include one or more single-core or multi-core processors. In some embodiments, processor 604 may include any combination of general-purpose processors and special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In embodiments where the electronic device employs an enhanced node B (ENB) or radio access network (RAN) controller, processor 604 may be configured to perform various conforming embodiments, such as... Figures 1 to 4 One or more of the multiple embodiments shown.

[0167] In some embodiments, the system control logic unit 608 may include any suitable interface controller to provide any suitable interface to at least one of the processors 604 and / or any suitable device or component communicating with the system control logic unit 608.

[0168] In some embodiments, the system control logic unit 608 may include one or more memory controllers to provide an interface to the system memory 612. The system memory 612 may be used to load and store data and / or instructions. In some embodiments, the system memory 612 of the electronic device may include any suitable volatile memory, such as a suitable dynamic random access memory.

[0169] The non-volatile memory 616 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, the non-volatile memory 616 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of a hard disk drive (HDD), a compact disc (CD) drive, and a digital versatile disc (DVD) drive.

[0170] The non-volatile memory 616 may include a portion of the storage resources on a device on which an electronic device is installed, or it may be accessible by the device but is not necessarily part of the device. For example, the non-volatile memory 616 may be accessed over a network via a network interface 620.

[0171] Specifically, system memory 612 and non-volatile memory 616 may each include a temporary copy and a permanent copy of instruction 624. Instruction 624 may include instructions that, when executed by at least one of processors 604, cause electronic devices to... Figures 1 to 4 The instructions for the method shown. In some embodiments, instructions 624, hardware, firmware and / or their software components may additionally / alternatively be located in system control logic unit 608, network interface 620 and / or processor 604.

[0172] Network interface 620 may include a transceiver for providing a radio interface to an electronic device, thereby enabling communication with any other suitable device (such as a front-end module, antenna, etc.) via one or more networks. In some embodiments, network interface 620 may be integrated into other components of the electronic device. For example, network interface 620 may be integrated into at least one of processor 604, system memory 612, non-volatile memory 616, and firmware device (not shown) with instructions, which, when at least one of processor 604 executes the instructions, enable the electronic device to implement... Figures 1 to 4 The method shown.

[0173] The network interface 620 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, the network interface 620 may be a network adapter, a wireless network adapter, a telephone modem, and / or a wireless modem.

[0174] In one embodiment, at least one of the processors 604 may be packaged together with the logic of one or more controllers for the system control logic unit 608 to form a system in package (SiP). In another embodiment, at least one of the processors 604 may be integrated on the same die with the logic of one or more controllers for the system control logic unit 608 to form a system on chip (SoC).

[0175] The electronic device may further include an input / output device 628. The input / output device 628 may include a user interface enabling a user to interact with the electronic device; the peripheral component interface is designed to allow peripheral components to also interact with the electronic device. In some embodiments, the electronic device further includes sensors for determining at least one type of environmental condition and location information related to the electronic device.

[0176] In some embodiments, the peripheral component interface may include, but is not limited to, a non-volatile memory port, an audio jack, and a power interface.

[0177] In some embodiments, the sensor may include, but is not limited to, a gyroscope sensor, an accelerometer, a proximity sensor, an ambient light sensor, and a positioning unit. The positioning unit may also be part of or interact with the network interface 620 to communicate with components of the positioning network, such as GPS satellites.

[0178] It is understood that, as used herein, the term “module” may refer to or include, or be part of, an application-specific integrated circuit (ASIC), electronic circuitry, a processor (shared, dedicated, or grouped) and / or memory that executes one or more software or firmware programs, combinational logic circuitry, and / or other suitable hardware components that provide the described functionality.

[0179] It is understood that in the various embodiments of this application, the processor may be a microprocessor, a digital signal processor, a microcontroller, etc., and / or any combination thereof. According to another aspect, the processor may be a single-core processor, a multi-core processor, etc., and / or any combination thereof.

[0180] The embodiments disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0181] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as a digital signal processor (DSP), microcontroller, application-specific integrated circuit, or microprocessor.

[0182] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0183] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or via other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, CD-ROMs, magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other forms of propagated signals. Therefore, machine-readable media includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0184] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0185] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0186] It should be noted that in the examples and description of this application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0187] Although this application has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art will understand that various changes in form and detail may be made thereto without departing from the scope of this application.

Claims

1. A fault diagnosis method, characterized in that, Applied to electronic devices, the method includes: Obtain the actual state information of N joints of a multi-joint robot at the current moment, where N is an integer greater than 1; For any target joint among the N joints, the actual state information of the N-1 joints other than the target joint is input into the first model to obtain the first state prediction information and prediction fault token of the target joint at the current time. The prediction fault token is used to represent the prediction fault mode of the target joint and the prediction severity corresponding to the prediction fault mode based on a vector. Determine the first prediction residual between the first state prediction information and the actual state information of the target joint; If the first prediction residual is greater than or equal to the first preset residual threshold, the sequence characteristics of the prediction residual sequence of the target joint within a first preset time period before the current time are determined, and the sequence characteristics include statistical characteristics and frequency domain characteristics. The sequence features, the actual state information of the target joint, and the predicted fault token are input into the second model to obtain the target fault mode of the target joint and the target severity corresponding to the target fault mode.

2. The fault diagnosis method according to claim 1, characterized in that, The first model was trained in the following way: Obtain the dynamic model of the multi-joint robot; For any first joint among the N joints, the task instruction, K preset fault modes, and L preset severity levels corresponding to each preset fault mode are input into the dynamic model to obtain the pairing dataset of the first joint under the task instruction. The pairing dataset includes: normal state information of each of the N joints at different times; and fault state information of each of the N joints at different times when the first joint has different preset fault modes and different preset severity levels, where K and L are both integers greater than or equal to 1. The first model is trained based on the paired dataset of each of the N joints, the preset fault mode and the preset severity corresponding to each paired dataset, to obtain the trained first model.

3. The fault diagnosis method according to claim 2, characterized in that, The training of the first model based on the paired dataset for each of the N joints, the preset fault mode corresponding to each paired dataset, and the preset severity includes: Determine the reference vectors in D-dimensional space for the preset fault modes and preset severity corresponding to each paired dataset, wherein the directions of the reference vectors for different preset fault modes are different, and different preset severity levels of the same preset fault mode correspond to different distances in the same reference vector direction, and D is an integer greater than 1. The first model is trained based on the pairing dataset for each first joint and the reference vector corresponding to each pairing dataset.

4. The fault diagnosis method according to claim 3, characterized in that, The step of training the first model based on the pairing dataset for each first joint and the reference vector corresponding to each pairing dataset includes: Based on the paired dataset for each first joint and the reference vector corresponding to each paired dataset, the first model is trained using an objective function that includes a classification loss function, a contrastive learning loss function, and a regression loss function. Wherein, the classification loss function is used to measure the accuracy of the predicted fault tokens for each first joint output by the first model; the contrastive learning loss function is used to enable the first model to distinguish the joint dynamic features corresponding to different preset fault modes; and the regression loss function is used to measure the accuracy of the state prediction information for each first joint output by the first model.

5. The fault diagnosis method according to claim 1, characterized in that, Different target joints among the N joints correspond to different first models.

6. The fault diagnosis method according to claim 1, characterized in that, The training set for the second model includes: The training residual sequence used for training, the sequence features of the training residual sequence, the training state information of the joints, the training fault token, and the reference fault mode and reference severity.

7. The fault diagnosis method according to claim 1, characterized in that, The method further includes: The predicted fault token is input into the third model to obtain a fault prevention strategy for the predicted fault mode and the predicted severity.

8. The fault diagnosis method according to claim 7, characterized in that, The third model was trained in the following manner: Obtain the training failure token for training; Based on the training fault token and the domain randomized environmental information, the third model is subjected to reinforcement learning using a reward function. The reward objective of the reward function includes maximizing the task completion rate and the accuracy of the fault prevention strategy output by the third model. The environmental information includes at least one of terrain information, load information, wind information, ground roughness information, or ground material information.

9. The fault diagnosis method according to any one of claims 1 to 8, characterized in that, Determining the sequence characteristics of the predicted residual sequence of the target joint within a first preset time period prior to the current time includes: Obtain P joint subsets, each of which includes joints related to the target joint. The number of joints in each of the joint subsets is less than N, and the number of joints in each of the joint subsets may be the same or different. P is an integer greater than 1. The actual state information of the joints in each of the joint subsets is input into the first model to obtain the second state prediction information of the target joint; Determine the second prediction residual between the second state prediction information corresponding to each of the joint subsets and the actual state information of the target joint, and obtain P second prediction residuals; If, among the P second prediction residuals, the number of second prediction residuals greater than or equal to the first preset residual threshold is greater than the number threshold, the sequence characteristics of the prediction residual sequence of the target joint within a first preset time period before the current time are determined.

10. The fault diagnosis method according to any one of claims 1 to 8, characterized in that, The method further includes: The actual state information of N-1 joints other than the target joint among the N joints is input into the first model to obtain the third state prediction information of the target joint at the next time step at the current time step. Determine the third prediction residual between the third state prediction information and the actual state information of the target joint at the next moment; The third prediction residuals at different times are fitted to obtain the residual growth curves; Based on the residual growth curve and the second preset residual threshold, the remaining service life of the target joint is determined.

11. An electronic device, characterized in that, include: At least one processor; at least one memory; The at least one memory stores at least one program, which, when executed by the at least one processor, causes the electronic device to perform the fault diagnosis method according to any one of claims 1 to 10.

12. A storage medium, characterized in that, The storage medium stores instructions that, when executed on an electronic device, cause the electronic device to perform the fault diagnosis method according to any one of claims 1 to 10.

13. A program product, characterized in that, Includes: instructions that, when executed on an electronic device, cause the electronic device to perform the fault diagnosis method according to any one of claims 1 to 10.

Images