Robot cable state prediction method and system, electronic equipment and storage medium

Abstract

The invention provides a robot cable state prediction method and system, electronic equipment and a storage medium, and relates to the technical field of robots. The method comprises the following steps: acquiring joint motion parameters and cable electrical parameters of a target robot; performing time sequence feature extraction on the joint motion parameters and the cable electrical parameters to generate a feature sequence; performing time sequence dependence modeling on the feature sequence by adopting a time sequence dependence model to obtain a context vector representing the cable health state of the target robot; according to the context vector, the cable state of the target robot is predicted through the prediction model, and cable state information of the target robot is determined. According to the invention, the health state of the robot cable can be monitored in real time.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and more specifically, to a method, system, electronic device, and storage medium for predicting the state of a robot cable. Background Technology

[0002] With the widespread application of collaborative robots in precision assembly, electronic manufacturing, medical surgery, and other fields, the reliability and lifespan of the hollow wiring inside their joints have become key factors affecting system stability. During prolonged, large-angle torsional operation, internal cables are prone to metal fatigue, insulation wear, signal attenuation, and even breakage due to repeated bending and twisting, leading to communication interruptions, control failures, and other malfunctions.

[0003] The means for monitoring the health status of robot cables are still relatively limited, and there is a lack of real-time assessment of cable health status. Summary of the Invention

[0004] The purpose of this application is to address the shortcomings of the prior art by providing a method, system, electronic device, and storage medium for predicting the state of robot cables, so as to monitor the health status of robot cables in real time.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, embodiments of this application provide a method for predicting the state of a robot cable, the method comprising: Obtain the joint motion parameters and cable electrical parameters of the target robot; Time-series feature extraction is performed on the joint motion parameters and the cable electrical parameters to generate a feature sequence; A temporal dependency model is used to model the temporal dependency of the feature sequence to obtain a context vector representing the cable health status of the target robot; Based on the context vector, the cable status of the target robot is predicted by the prediction model to determine the cable status information of the target robot.

[0006] Optionally, the joint motion parameters include at least one of the following: joint rotation angle, joint rotation number, joint rotation speed, and joint rotation acceleration; and the cable electrical parameters include at least one of the following: voltage parameter, current parameter, resistance parameter, insulation performance parameter, and signal integrity parameter.

[0007] Optionally, the target robot has ring markers at its movable joints, and the acquisition of the joint motion parameters of the target robot includes: The circular logo image is acquired using a preset image acquisition device; The circular sign image is identified based on an image recognition algorithm to determine the sign information contained in the circular sign image and the angular interval between the sign information; The joint motion parameters are determined based on the identification information and the angular interval between the identification information.

[0008] Optionally, the step of extracting time-series features from the joint motion parameters and the cable electrical parameters to generate a feature sequence includes: The electrical parameters of the cable are converted in the frequency domain to generate the spectral characteristics of the electrical parameters of the cable. A multi-scale convolutional neural network is used to extract multi-scale time series features from the spectral characteristics of the joint motion parameters and the cable electrical parameters, generating a multi-scale feature sequence.

[0009] Optionally, the temporal dependency model includes a long short-term memory network and an attention layer. The step of using the temporal dependency model to perform temporal dependency modeling on the feature sequence to obtain a context vector representing the cable health state of the target robot includes: The long short-term memory network is used to process the feature sequence to generate the hidden state at each time step; The attention weights of the hidden states at each time step are calculated using the attention layer. The context vector is generated based on the hidden state at each time step and the attention weights.

[0010] Optionally, the cable status information includes at least one of the following: cable health status index, cable remaining service life, and cable health type information.

[0011] Optionally, the method further includes: If the cable status information indicates that the target robot's cable is abnormal, control the target robot to stop and send a maintenance prompt message.

[0012] Secondly, this application also provides a robot cable state prediction system, which includes: a processor, an image acquisition device, a ring marker and an electrical parameter acquisition module disposed at multiple moving joints of a target robot, the image acquisition device being used to acquire images of the ring markers and send the ring marker images to the processor, the processor calculating the joint motion parameters of the target robot based on the ring marker images, and the electrical parameter acquisition module being electrically connected to the cables inside the target robot and being used to acquire and send the electrical parameters of the target robot's cables to the processor; The processor executes the robot cable state prediction method as described in any of the first aspects based on the joint motion parameters and the cable electrical parameters to determine the cable state information of the target robot.

[0013] Thirdly, embodiments of this application provide a robot cable state prediction device, the device comprising: The parameter acquisition module is used to acquire the joint motion parameters and cable electrical parameters of the target robot. The feature extraction module is used to extract time-series features from the joint motion parameters and the cable electrical parameters to generate a feature sequence; The vector generation module is used to perform temporal dependency modeling on the feature sequence using a temporal dependency model to obtain a context vector representing the cable health status of the target robot. The state prediction module is used to predict the cable state of the target robot based on the context vector and through a prediction model, and to determine the cable state information of the target robot.

[0014] Optionally, the joint motion parameters include at least one of the following: joint rotation angle, joint rotation number, joint rotation speed, and joint rotation acceleration; and the cable electrical parameters include at least one of the following: voltage parameter, current parameter, resistance parameter, insulation performance parameter, and signal integrity parameter.

[0015] Optionally, the parameter acquisition module is specifically used to acquire a circular logo image of the circular logo through a preset image acquisition device; to identify the circular logo image based on an image recognition algorithm, and to determine the logo information contained in the circular logo image and the angular interval between the logo information; and to determine the joint motion parameters based on the logo information and the angular interval between the logo information.

[0016] Optionally, the feature extraction module is specifically used to perform frequency domain transformation on the electrical parameters of the cable to generate the spectral features of the electrical parameters of the cable; and to use a multi-scale convolutional neural network to perform multi-scale time series feature extraction on the spectral features of the joint motion parameters and the electrical parameters of the cable to generate a multi-scale feature sequence.

[0017] Optionally, the temporal dependency model includes a long short-term memory network and an attention layer. The vector generation module is specifically used to process the feature sequence using the long short-term memory network to generate a hidden state at each time step; to calculate the attention weight of the hidden state at each time step using the attention layer; and to generate the context vector based on the hidden state at each time step and the attention weight.

[0018] Optionally, the cable status information includes at least one of the following: cable health status index, cable remaining service life, and cable health type information.

[0019] Optionally, the device further includes: An anomaly control module is used to control the target robot to stop and send maintenance prompt information if the cable status information indicates that the target robot's cable is abnormal.

[0020] Fourthly, embodiments of this application also provide an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores program instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to perform the steps of the robot cable state prediction method as described in any of the first aspects.

[0021] Fifthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the robot cable state prediction method as described in any of the first aspects.

[0022] The beneficial effects of this application are: The robot cable status prediction method, device, electronic equipment, and storage medium provided in this application predict cable status information based on joint motion parameters and cable electrical parameters, enabling multi-dimensional and comprehensive monitoring of cable status. This allows for more accurate and comprehensive prediction of cable status, and real-time prediction and monitoring of cable status can be achieved by acquiring joint motion parameters and cable electrical parameters in real time. This transforms cable status detection from passive fault detection to active status prediction, providing early warnings before faults occur, avoiding production losses caused by sudden faults, greatly reducing equipment failure rate and maintenance costs. Furthermore, detection can be completed without affecting normal production, reducing testing costs and downtime. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is an architecture diagram of the robot cable state prediction system provided in the embodiments of this application; Figure 2 Three-dimensional and two-dimensional schematic diagrams of the ring-shaped identifier provided in the embodiments of this application; Figure 3 This is a schematic diagram of the ring-shaped marking of the entire machine provided in an embodiment of this application; Figure 4 A flowchart illustrating the robot cable state prediction method provided in this application embodiment. Figure 1 ; Figure 5 A flowchart illustrating the robot cable state prediction method provided in this application embodiment. Figure 2 ; Figure 6 A flowchart illustrating the robot cable state prediction method provided in this application embodiment. Figure 3 ; Figure 7 A flowchart illustrating the robot cable state prediction method provided in this application embodiment. Figure 4 ; Figure 8 An architecture diagram of the cable state prediction model provided in the embodiments of this application; Figure 9 This is a schematic diagram of the structure of the robot cable state prediction device provided in the embodiments of this application; Figure 10 A schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of this application, but not all embodiments.

[0026] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0027] Furthermore, the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Additionally, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] It should be noted that, where there is no conflict, the features in the embodiments of this application can be combined with each other.

[0029] One existing method for testing the condition of robot cables is the individual cable bending test. This method involves fixing a cable sample on a bending machine and repeatedly bending it at a fixed radius. The cable life is determined by monitoring resistance or signal attenuation. However, this method has drawbacks: it cannot simulate the three-dimensional torsional conditions of robot joints during actual operation, it cannot reflect the dynamic impact of robot speed and acceleration on the cable, the test environment differs significantly from real-world conditions, and the predicted results are not representative.

[0030] Another existing method for testing the condition of robot cables is the overall machine lifespan test, which involves running the robot continuously at specific speeds and angles until the cable breaks or a communication error occurs. The drawback of this method is that it cannot monitor the cable condition in real time during operation, cannot record the correlation between joint rotation counts and cable lifespan, and can only make post-failure judgments, lacking early warning capabilities.

[0031] It can be seen that the existing technology has the following main shortcomings: 1. Delayed fault detection: The system can only detect the problem after the cable is completely broken and the control system alarm is triggered. It cannot provide real-time warnings. When the cable fails, the alarm may cause production interruption and greater economic losses.

[0032] 2. Incomplete data recording: The complete process of cable performance degradation cannot be recorded during the test. Only the final state at the time of failure can be obtained. Data on the complete evolution process of the cable from health to failure is lacking.

[0033] 3. Unclear causal relationship: It is impossible to establish a quantitative relationship between the number of joint movements and cable breakage. After the test, only the total running time can be known, but the specific contribution of each joint cannot be determined.

[0034] 4. High testing costs: Whole machine testing requires investment in complete robot equipment, testing sites and a lot of time, making the testing costs significantly higher than offline testing methods.

[0035] 5. Poor repeatability: The test conditions are difficult to be completely consistent each time, resulting in poor repeatability and comparability of test results.

[0036] If only manual inspection is used, on-site troubleshooting relies heavily on the personal experience and intuition of technicians, which may lead to misdiagnosis or omission. Moreover, traditional troubleshooting methods often take a long time, especially when the fault is not obvious or is complex, requiring high labor and time costs.

[0037] The following describes the specific implementation of the robot cable state prediction method, system, electronic device and storage medium provided in the embodiments of this application.

[0038] Figure 1 This is an architecture diagram of the robot cable state prediction system provided in the embodiments of this application, such as... Figure 1 As shown, the robot cable status prediction system includes: a processor 101, an image acquisition device 201, a ring marker 202 set at multiple moving joints of the target robot, and an electrical parameter acquisition module 203.

[0039] Image acquisition device 201 is used to acquire images of ring marker 202 and send the ring marker image to processor 101. Processor 101 calculates the joint motion parameters of the target robot based on the ring marker image. Electrical parameter acquisition module 203 is electrically connected to the internal cables of the target robot and is used to acquire and send the electrical parameters of the target robot's cables to processor 101.

[0040] The processor 101 predicts the cable status information of the target robot based on the joint motion parameters and cable electrical parameters.

[0041] In this embodiment, the target robot includes multiple movable joints, which drive the robot's end effector to move during operation. Cables inside the target robot provide power and communication, passing through the multiple movable joints to control their movement. During the movement of these joints, repeated bending and twisting are inevitable, leading to cable fatigue, insulation wear, signal attenuation, and even breakage.

[0042] In some embodiments, joint motion parameters of multiple mobile joints can be obtained based on encoders and torque sensors of multiple mobile joints.

[0043] In other embodiments, a scheme is provided for determining joint motion parameters of multiple movable joints based on visual monitoring.

[0044] Specifically, a ring marker 202 is set at each moving joint of the target robot. The ring marker surrounds each moving joint and is composed of multiple different markers. An image acquisition device 201 is deployed in the working area of ​​the target robot to acquire images of the ring markers. The processor 101 determines the joint motion parameters of each moving joint based on the relationship between the position of the marker information in the ring marker image and the preset reference position.

[0045] The image acquisition device 201 can be a high-definition camera. The position and angle of the high-definition camera are adjusted to ensure that the range of motion of multiple moving joints can be covered. The camera can be an industrial-grade product with high resolution, high frame rate, and good low-light performance.

[0046] In some embodiments, Figure 2 The three-dimensional and two-dimensional schematic diagrams of the ring-shaped identifier provided in the embodiments of this application are shown below. Figure 3 This is a schematic diagram of the ring-shaped marking of the whole machine provided in the embodiments of this application, such as... Figure 2 and Figure 3 As shown, an annular mark 202 is provided on the outer surface of the rotation axis of each movable joint, which surrounds each movable joint. The annular mark can be multiple barcodes with displacement indicators.

[0047] For example, barcode markings can use high-contrast patterns, such as black and white patterns, to ensure clear identification under various lighting conditions. The layout of the barcode takes into account the full range of joint movement to ensure that there are enough markings available for identification in any position.

[0048] For example, multiple movable joints may include: 1. Base Rotary Joint: By attaching a ring-shaped marker to the base rotating platform, the motion parameters of the target robot's joints can be measured.

[0049] 2. Shoulder joint: By attaching a ring-shaped marker to the outer surface of the shoulder joint's rotation axis, the shoulder joint motion parameters of the target robot can be measured.

[0050] 3. Elbow joint: By attaching a ring-shaped marker to the outer surface of the elbow joint's rotation axis, the elbow joint motion parameters of the target robot can be measured.

[0051] 4. Wrist pitch joint: By attaching a ring mark to the outer surface of the wrist pitch axis, the up-and-down swing parameters of the target robot's wrist can be measured.

[0052] 5. Wrist yaw joint: By attaching a ring mark to the outer surface of the wrist yaw axis, the left and right swing parameters of the target robot's wrist can be measured.

[0053] 6. Wrist roll joint: By attaching a ring mark to the outer surface of the wrist roll axis, the rotational motion parameters of the end effector of the target robot can be measured.

[0054] The electrical parameter acquisition module 203 can acquire electrical parameters from the total input end of the cable, or it can acquire electrical parameters from multiple moving joints of the cable separately. Acquiring electrical parameters from the total input end of the cable requires fewer parameters and has higher computational efficiency, while acquiring electrical parameters from multiple moving joints provides higher accuracy. The specific choice can be made according to actual needs. The electrical parameter acquisition module 203 is connected to the processor 101 via an industrial bus to ensure real-time data transmission and synchronization.

[0055] The specific implementation of the robot cable state prediction method provided in this application will be described below with reference to the embodiments.

[0056] Figure 4 A flowchart illustrating the robot cable state prediction method provided in this application embodiment. Figure 1 ,like Figure 4 As shown, the method may include: S301. Obtain the joint motion parameters and cable electrical parameters of the target robot.

[0057] In this embodiment, since the cable running through the moving joint of the target robot will be repeatedly bent and twisted during the operation, the movement of the moving joint will affect the state of the cable. The joint motion parameters can directly represent the influence of the joint motion on the state of the cable, and the state of the cable will affect the electrical signal transmission of the cable. Therefore, the cable electrical parameters are the way to quantify the state of the cable.

[0058] The joint motion parameters of the target robot can be the rotation parameters of the target robot's mobile joints. The rotation parameters are divided into: parameters for the target robot's mobile joints to rotate within a 360° range and parameters for swinging within a preset angle range. For example, the base rotation joint can rotate within a 360° range, and the wrist joint can swing within a preset angle range.

[0059] In some embodiments, the joint motion parameters of the target robot can be obtained through a controller or encoder configured for the mobile joints of the target robot.

[0060] The electrical parameters of the target robot's cables can be the electrical parameters of cables at preset reference points within the target robot's body, or the electrical parameters of cables at rotating parts of multiple moving joints. These electrical parameters can be obtained through sensors or acquisition circuits.

[0061] In some embodiments, after obtaining joint motion parameters and cable electrical parameters, timestamps are used to align the joint motion parameters and cable electrical parameters to determine their temporal consistency and form a multi-dimensional dataset.

[0062] In some embodiments, after obtaining joint motion parameters and cable electrical parameters, a Kalman filter can be used to smooth the data, eliminate noise interference, and improve the accuracy of the data.

[0063] S302. Extract time-series features from joint motion parameters and cable electrical parameters to generate feature sequences.

[0064] In this embodiment, the joint motion parameters and cable electrical parameters of multiple time steps are combined to form an original data sequence with the shape of [M, N], where M is the number of time steps and N is the number of types of joint motion parameters and cable electrical parameters for each time step.

[0065] Set a sliding window of a preset size, and perform multi-dimensional feature extraction on each type of parameter over multiple time steps to determine the multi-dimensional feature sequence of each type of parameter. Among them, the multi-dimensional features of joint motion parameters can include: temporal features and morphological features. Temporal features can include at least one of the following: mean, standard deviation, root mean square, peak value, peak-to-peak value. Morphological features can include at least one of the following: slope, kurtosis.

[0066] The multi-dimensional characteristics of cable electrical parameters can include time-domain characteristics and frequency-domain characteristics. The time-domain characteristics are similar to the time-domain characteristics of joint motion parameters, while the frequency-domain characteristics can include at least one of the following: frequency amplitude, spectral centroid, and spectral bandwidth.

[0067] For example, a type of joint motion parameters includes [s1, s2, s3, ..., sM]. The sliding window size is 5 and the step size is 1. Then, multi-dimensional feature extraction is performed on [s1, s2, s3, s4, s5], [s2, s3, s4, s5, s6], ..., [s(M-4), s(M-3), s(M-2), s(M-1), sM] in sequence to determine the multi-dimensional features of a type of joint motion parameters.

[0068] Multi-dimensional features of multiple types of parameters at each time step are concatenated to generate a feature vector for each time step. Feature vectors from multiple time steps are then combined to generate a feature sequence.

[0069] For example, if the shape of the original data sequence is [M, N], the sliding window size is i, and the number of dimensions of the multi-dimensional features for each type of parameter is j, then the length of the feature vector at each time step is N*j, and the shape of the feature sequence is [M-i+1, N*j]. For instance, if the shape of the original data sequence is [20, 5], and 8 features are calculated for each type of parameter, then the length of the feature vector at each time step is 5*8=40. After sliding through a window of 5 and a step size of 1, a data sequence of 20 time steps yields feature vectors for 16 time steps, resulting in a feature sequence of [16, 40].

[0070] S303. Use a temporal dependency model to model the temporal dependency of the feature sequence to obtain a context vector representing the cable health status of the target robot.

[0071] In this embodiment, the temporal dependency model is used to understand the causal and dependency relationships between different time steps in the feature sequence. The temporal dependency model learns the feature vector of each time step in the feature sequence and the feature vector of the previous time step to update the hidden state of each time step. The hidden state of each time step contains all the valuable information from the first time step to the current time step, until the feature vector of the last time step is learned. This yields the context vector corresponding to all the health information of the cable in multiple time steps. The context vector represents the evolution of the cable's health status. Based on this context vector, the future changes in the cable's health status can be predicted.

[0072] S304. Based on the context vector, predict the cable status of the target robot using the prediction model to determine the cable status information of the target robot.

[0073] In this embodiment, the prediction model is used to predict the cable health status of the target robot. The prediction model can output cable health indicators or cable health status types. The context vector is input into the prediction model, and the context vector is identified by multiple fully connected layer neurons of the prediction model. The cable status information is then output through the activation function.

[0074] In some embodiments, the prediction model further includes a Bayesian neural network model. If the cable status information is a cable health indicator, the Bayesian neural network model is used to calculate the prediction confidence interval of the cable health indicator.

[0075] In some embodiments, since the movement frequency and degree of multiple mobile joints are different, the corresponding cable states will also be different. For the joint motion parameters and cable electrical parameters of multiple mobile joints, state prediction can be performed separately, and the cable state information of the mobile joint with the worst state can be used as the cable state information of the target robot.

[0076] The robot cable status prediction method provided in the above embodiments predicts cable status information based on joint motion parameters and cable electrical parameters, realizing multi-dimensional and all-round monitoring of cable status. It can more accurately and comprehensively predict cable status, and can also perform real-time prediction and monitoring of cable status by acquiring joint motion parameters and cable electrical parameters in real time. This transforms cable status detection from passive fault detection to active status prediction, so as to provide early warning before faults occur, avoid production losses caused by sudden faults, greatly reduce equipment failure rate and maintenance costs, and can complete the detection without affecting normal production, reducing testing costs and downtime.

[0077] In one possible implementation, the joint motion parameters include at least one of the following: joint rotation angle, joint rotation number, joint rotation speed, and joint rotation acceleration; and the cable electrical parameters include at least one of the following: voltage parameter, current parameter, resistance parameter, insulation performance parameter, and signal integrity parameter.

[0078] In this embodiment, the joint rotation angle can be measured by an encoder, potentiometer, etc., or it can be calculated based on the angle information in the control commands generated by the target robot's controller for the moving joint.

[0079] The number of joint rotations can be obtained by the number of pulse cycles recorded by the encoder or by the Hall sensor monitoring the magnet trigger signal.

[0080] The joint rotation speed can be calculated by the change in the number of pulses per unit time through encoder differential calculation, or by the ratio of the output voltage of the tachogenerator to the rotational speed.

[0081] Joint rotational acceleration can be calculated by time differentiation of the velocity signal, or by estimating torque using motor current and combining the results with other calculations.

[0082] Voltage parameters can be acquired through voltage divider resistors / isolation amplifiers. These parameters can be the cable operating voltage or voltage drop, etc.

[0083] Current parameters can be acquired through current transformers, Hall effect sensors, or sampling resistors combined with operational amplifiers. These current parameters can include cable operating current, leakage current, harmonic current, etc.

[0084] Resistance parameters can be acquired through the DC resistance of the conductor or the contact resistance. The resistance parameters are the DC resistance value of the conductor or the contact resistance value.

[0085] Insulation performance parameters can be determined by collecting the resistance value of the insulation resistance.

[0086] Signal integrity parameters can be determined by injecting a test signal and measuring its energy attenuation, signal-to-noise ratio, etc.

[0087] In one possible implementation, Figure 5 A flowchart illustrating the robot cable state prediction method provided in this application embodiment. Figure 2 ,like Figure 5 As shown, the process of obtaining the joint motion parameters of the target robot in step S301 above may include: S401. Acquire the image of the circular sign using a preset image acquisition device.

[0088] S402. Based on the image recognition algorithm, identify the circular sign image and determine the sign information contained in the circular sign image and the angular interval between the sign information.

[0089] S403. Determine the joint motion parameters based on the identification information and the angular intervals between the identification information.

[0090] In this embodiment, as Figure 2 , Figure 3 As shown, ring-shaped markers are set on the outer surface of multiple moving joints of the target robot. The ring-shaped markers consist of multiple unique identification information, which can be graphics, such as barcodes.

[0091] During the operation of the target robot, the image acquisition device acquires multiple consecutive frames of ring-shaped marker images for each mobile joint. The pre-trained image recognition algorithm is used to identify the acquired ring-shaped marker images. Based on the marker information contained in the acquired ring-shaped marker images and the angular intervals between each marker information, the joint rotation angle of each mobile joint at multiple time steps is calculated.

[0092] The cumulative number of joint rotations across multiple time steps is determined based on the joint rotation angles at multiple time steps and the zero-point crossing behavior of the joint rotation angles. For example, if the joint rotation angle at the first time step is 10°, the cumulative number of joint rotations is determined to be 1. If the joint rotation angle at the second time step is 15° and there is no zero-point crossing behavior, the cumulative number of joint rotations is still determined to be 1. If the joint rotation angle at the third time step is -5° and there is zero-point crossing behavior, the cumulative number of joint rotations is determined to be 2.

[0093] The joint rotational velocity can be calculated based on the difference in joint rotation angle and time between two adjacent time steps, while the joint rotational acceleration can be calculated by performing a time derivative on the joint rotational velocity.

[0094] In some embodiments, a pre-trained YOLO target detection algorithm can be used to identify the ring sign image, determine the sign information contained in the ring sign image and the angular interval between each sign information, and the algorithm has real-time performance and robustness, and can operate stably in complex industrial environments.

[0095] The robot cable state prediction method provided in the above embodiments uses a ring mark as a marker for joint movement and accurately calculates joint movement parameters through a visual algorithm. It is low-cost, high-precision, and easy to implement, solving the problems of complex installation and high cost of traditional encoders, and achieving high-precision parameter measurement without contact.

[0096] In one possible implementation, Figure 6 A flowchart illustrating the robot cable state prediction method provided in this application embodiment. Figure 3 ,like Figure 6 As shown, the process of extracting time-series features from joint motion parameters and cable electrical parameters in step S302 to generate a feature sequence may include: S501. Perform frequency domain conversion on the electrical parameters of the cable to generate the spectral characteristics of the electrical parameters of the cable.

[0097] In this embodiment, in order to better observe the periodicity, harmonics and noise patterns in the electrical parameters of the cable related to aging and faults, the electrical parameters of the cable can be converted from a time domain perspective to a frequency domain perspective.

[0098] Specifically, the input time-domain cable electrical parameters can be transformed using FFT to output an amplitude spectrum, which includes amplitude and phase information at multiple frequencies, thus determining the spectral characteristics of the cable electrical parameters.

[0099] In some embodiments, instead of directly extracting features from the amplitude spectrum, the spectral features of the amplitude spectrum are determined first. The spectral features may include at least one of the following: the main frequency components and amplitude of the amplitude spectrum, the spectral centroid, the spectral bandwidth, the spectral entropy, and the high-frequency band energy.

[0100] S502. A multi-scale convolutional neural network is used to extract multi-scale time series features from the spectral characteristics of joint motion parameters and cable electrical parameters, generating a multi-scale feature sequence.

[0101] In this embodiment, the spectral characteristics of joint motion parameters and cable electrical parameters at the same time step are spliced ​​together to form a multimodal input sequence.

[0102] Multiscale convolutional neural networks have multiple one-dimensional convolutional kernels of different sizes set in parallel, such as kernel_size=1,3,5,7. Small convolutional kernels have small receptive fields and are used to capture short-term, local dependencies, such as transient spikes in current signals. Large convolutional kernels have large receptive fields and are used to capture long-term, discriminatory patterns, such as the overall decreasing trend of insulation resistance.

[0103] The multimodal input sequences at multiple time steps are fed in parallel into multiple convolutional kernels of a multi-scale convolutional neural network. Each convolutional kernel extracts features from the multimodal input sequences at multiple time steps, and each convolutional kernel outputs a feature sequence. The feature sequences output by the convolutional kernels at multiple scales are concatenated along the channel dimension to form a multi-scale feature sequence.

[0104] The robot cable state prediction method provided in the above embodiments extracts multi-scale time series features of the spectral characteristics of joint motion parameters and cable electrical parameters through a multi-scale convolutional neural network. This allows for the observation of parameter features at different time scales, thereby improving the accuracy of cable state prediction.

[0105] In one possible implementation, Figure 7 A flowchart illustrating the robot cable state prediction method provided in this application embodiment. Figure 4 ,like Figure 7 As shown, the process of S303 above, which uses a temporal dependency model to model the temporal dependency of the feature sequence and obtains the context vector representing the cable health state of the target robot, may include: S601. A long short-term memory network is used to process the feature sequence to generate the hidden state at each time step.

[0106] S602. Use an attention layer to calculate the attention weights of the hidden state at each time step.

[0107] S603. Generate a context vector based on the hidden state and attention weights at each time step.

[0108] In this embodiment, the temporal dependency model consists of a Long Short-Term Memory (LSTM) network and an attention mechanism. The feature sequences from multiple time steps are output to the LSTM network. The LSTM network updates the hidden state and cell state of the previous time step based on the feature sequence of the current time step and the hidden state and cell state of the previous time step through forget gates, input gates, and output gates, thus obtaining the hidden state and cell state of the current time step. The hidden state of each time step is obtained by passing the feature sequences from multiple time steps through computation. The hidden state and cell state of the first time step can be the default values ​​of a pre-trained LSTM network. The hidden state of each time step contains a condensed vector of all historical information from the beginning of the feature sequence to the current time step.

[0109] Although the LSTM network obtains hidden states at multiple time steps, the hidden states at all time steps are treated equally. An attention mechanism is needed to determine which hidden states at which time steps are more critical during the cable state prediction process.

[0110] The hidden state sequence at multiple time steps is input into the attention layer for computation to obtain the energy score of the hidden state at each time step. The energy score is normalized to obtain the attention weight of the hidden state at each time step. The hidden states at multiple time steps are weighted and summed according to the attention weight to obtain the context vector.

[0111] The robot cable state prediction method provided in the above embodiments, by combining an LSTM network with an attention mechanism, can accurately capture the historical changes in the robot's joint motion parameters and cable electrical parameters, thereby accurately and comprehensively predicting the robot's cable state.

[0112] In one possible implementation, cable status information may include at least one of the following: cable health status index, cable remaining service life, and cable health type information.

[0113] In this embodiment, the prediction model includes a fully connected layer. The Sigmoid activation function can be used to input the context vector into the fully connected layer and output a cable health status index of 0-1, where 1 represents a brand new state and 0 represents a completely failed state. The higher the cable health status index, the better the cable health status, and the lower the cable health status index, the worse the cable health status.

[0114] The prediction model includes a fully connected layer. A linear activation function can be used to input the context vector into this fully connected layer, which outputs a specific numerical value for the remaining cable lifespan. Furthermore, a Bayesian method can be used to perform multiple predictions, outputting the confidence range of the cable's remaining lifespan.

[0115] The prediction model includes a fully connected layer that can use the softmax activation function. The context vector is input into the fully connected layer, and the output cable health status type has multiple probabilities, such as healthy, alert, warning, and fault. The cable health type information is determined based on the category with the highest probability.

[0116] In some embodiments, this application uses a pre-trained cable condition prediction model to predict cable conditions. Figure 8 The architecture diagram of the cable status prediction model provided in the embodiments of this application is as follows: Figure 8 As shown, the cable status prediction model includes: a spectrum feature extraction module, a multi-scale convolutional neural network, a long short-term memory network, an attention layer, and a fully connected layer.

[0117] Specifically, the electrical parameters of the cable are input to the spectrum feature extraction module to extract spectrum features. The joint motion parameters and spectrum features are input to the multi-scale convolutional neural network to generate a multi-scale feature sequence. The multi-scale feature sequence is input to the long short-term memory network to output the hidden state at multiple time steps. The hidden state at multiple time steps is weighted and calculated by the attention layer to obtain the context vector. The context vector is then passed through the fully connected layer to output the predicted cable state information.

[0118] The cable status prediction model is pre-trained using historical data, and the training process will not be described in detail here.

[0119] In one possible implementation, the method may further include: If the cable status information indicates that the target robot's cable is abnormal, control the target robot to stop and send a maintenance prompt message.

[0120] In this embodiment, the cable status information is the cable health status index. If the cable health status index is less than a preset index threshold, the target robot's cable is determined to be abnormal.

[0121] The cable status information is the remaining service life of the cable. If the remaining service life of the cable is less than the preset time threshold, the cable of the target robot is determined to be abnormal.

[0122] The cable status information is the cable health type information. If the cable health type information is less than the preset type, the target robot's cable is determined to be abnormal. The predicted type can be warning or fault.

[0123] In some embodiments, a decay curve of the cable health status of the target robot can be generated based on the cable health status index or the relationship between the cable's remaining service life and time.

[0124] In one possible implementation, this solution supports simultaneous monitoring of multiple robots. By combining edge computing with cloud computing, it enables health status management of large-scale robot clusters, exhibiting good scalability and practicality.

[0125] Based on the above method embodiments, this application also provides a robot cable state prediction device. Figure 9 This is a schematic diagram of the structure of the robot cable state prediction device provided in the embodiments of this application, as shown below. Figure 9 As shown, the device may include: The parameter acquisition module 701 is used to acquire the joint motion parameters and cable electrical parameters of the target robot. Feature extraction module 702 is used to extract time series features from joint motion parameters and cable electrical parameters to generate feature sequences; The vector generation module 703 is used to perform temporal dependency modeling on the feature sequence using a temporal dependency model to obtain a context vector representing the cable health status of the target robot. The state prediction module 704 is used to predict the cable state of the target robot based on the context vector and the prediction model, and to determine the cable state information of the target robot.

[0126] Optionally, the joint motion parameters include at least one of the following: joint rotation angle, joint rotation number, joint rotation speed, and joint rotation acceleration; the cable electrical parameters include at least one of the following: voltage parameter, current parameter, resistance parameter, insulation performance parameter, and signal integrity parameter.

[0127] Optionally, the parameter acquisition module 701 is specifically used to acquire the circular sign image of the circular sign through a preset image acquisition device; to identify the circular sign image based on an image recognition algorithm, and to determine the sign information contained in the circular sign image and the angular interval between the sign information; and to determine the joint motion parameters based on the sign information and the angular interval between the sign information.

[0128] Optionally, the feature extraction module 702 is specifically used to perform frequency domain transformation on the electrical parameters of the cable to generate the spectral features of the electrical parameters of the cable; and to use a multi-scale convolutional neural network to perform multi-scale time series feature extraction on the spectral features of the joint motion parameters and the electrical parameters of the cable to generate a multi-scale feature sequence.

[0129] Optionally, the temporal dependency model includes: a long short-term memory network and an attention layer. The vector generation module 703 is specifically used to process the feature sequence using the long short-term memory network to generate the hidden state at each time step; to calculate the attention weight of the hidden state at each time step using the attention layer; and to generate a context vector based on the hidden state and attention weight at each time step.

[0130] Optionally, the cable status information includes at least one of the following: cable health status index, cable remaining service life, and cable health type information.

[0131] Optionally, such as Figure 9 As shown, the device may further include: The anomaly control module 705 is used to control the target robot to stop and send maintenance prompt information if the cable status information indicates that the target robot's cable is abnormal.

[0132] The above-described device is used to execute the method provided in the foregoing embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0133] These modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more microprocessors, or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a system-on-a-chip (SOC).

[0134] Figure 10 This is a schematic diagram of an electronic device provided in an embodiment of this application. The electronic device 100 may include a processor 101, a storage medium 102, and a bus. The storage medium 102 stores program instructions executable by the processor 101. When the electronic device 100 is running, the processor 101 communicates with the storage medium 102 via the bus, and the processor 101 executes the program instructions to perform the above-described method embodiment. The specific implementation and technical effects are similar and will not be described again here.

[0135] Optionally, this application also provides a computer-readable storage medium storing a computer program, which is executed by a processor to perform the above-described method embodiments.

[0136] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0137] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0138] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0139] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0140] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for predicting the state of a robot cable, characterized in that, The method includes: Obtain the joint motion parameters and cable electrical parameters of the target robot; Time-series feature extraction is performed on the joint motion parameters and the cable electrical parameters to generate a feature sequence; A temporal dependency model is used to model the temporal dependency of the feature sequence to obtain a context vector representing the cable health status of the target robot; Based on the context vector, the cable status of the target robot is predicted by the prediction model to determine the cable status information of the target robot.

2. The method as described in claim 1, characterized in that, The joint motion parameters include at least one of the following: joint rotation angle, joint rotation number, joint rotation speed, and joint rotation acceleration. The cable electrical parameters include at least one of the following: voltage parameter, current parameter, resistance parameter, insulation performance parameter, and signal integrity parameter.

3. The method as described in claim 1, characterized in that, The target robot has ring-shaped markers at its movable joints, each marker consisting of multiple unique identifiers. The acquisition of the target robot's joint motion parameters includes: The circular logo image is acquired using a preset image acquisition device; The circular sign image is identified based on an image recognition algorithm to determine the sign information contained in the circular sign image and the angular interval between the sign information; The joint motion parameters are determined based on the identification information and the angular interval between the identification information.

4. The method as described in claim 1, characterized in that, The step of extracting time-series features from the joint motion parameters and the cable electrical parameters to generate a feature sequence includes: The electrical parameters of the cable are converted in the frequency domain to generate the spectral characteristics of the electrical parameters of the cable. A multi-scale convolutional neural network is used to extract multi-scale time series features from the spectral characteristics of the joint motion parameters and the cable electrical parameters, generating a multi-scale feature sequence.

5. The method as described in claim 1, characterized in that, The temporal dependency model includes a long short-term memory network and an attention layer. The temporal dependency model is used to model the temporal dependency of the feature sequence to obtain a context vector representing the cable health state of the target robot, including: The long short-term memory network is used to process the feature sequence to generate the hidden state at each time step; The attention weights of the hidden states at each time step are calculated using the attention layer. The context vector is generated based on the hidden state at each time step and the attention weights.

6. The method as described in claim 1, characterized in that, The cable status information includes at least one of the following: cable health status index, cable remaining service life, and cable health type information.

7. The method as described in claim 1, characterized in that, The method further includes: If the cable status information indicates that the target robot's cable is abnormal, control the target robot to stop and send a maintenance prompt message.

8. A robot cable state prediction system, characterized in that, The robot cable status prediction system includes: a processor, an image acquisition device, a ring marker and an electrical parameter acquisition module installed at multiple moving joints of the target robot. The image acquisition device is used to acquire images of the ring markers and send the ring marker images to the processor. The processor calculates the joint motion parameters of the target robot based on the ring marker images. The electrical parameter acquisition module is electrically connected to the cables inside the target robot and is used to acquire and send the electrical parameters of the target robot's cables to the processor. The processor executes the robot cable state prediction method as described in any one of claims 1-7 based on the joint motion parameters and the cable electrical parameters to determine the cable state information of the target robot.

9. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores program instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to perform the steps of the robot cable state prediction method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, performs the steps of the robot cable state prediction method as described in any one of claims 1 to 7.

Images