Industrial robot state detection method, system and device and storage medium

Abstract

The invention provides an industrial robot state detection method, system and device and a storage medium. The method comprises the following steps: acquiring sensor time sequence data of each axis of an industrial robot in a target sampling time period; for each shaft of the industrial robot, determining energy time-varying data of the corresponding shaft in the target sampling time period according to the sensor time sequence data; converting the energy time-varying data into a two-dimensional amplitude-frequency image, and performing feature extraction on the two-dimensional amplitude-frequency image to obtain a state feature vector of the corresponding axis in the target sampling time period; inputting the state feature vector into an LSTM model to obtain a state classification result of a corresponding axis output by the LSTM model in the target sampling period; wherein the LSTM model is obtained by training a sample state feature vector with a state classification label.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial robots, and in particular, to a method, system, device and storage medium for detecting the state of an industrial robot. Background Art

[0002] For the state detection of industrial robots, in related technologies, technicians usually analyze the sensor data generated during the operation of industrial robots to judge the real-time operation state of the robots. This kind of state detection requires technicians to have an in-depth understanding of the operation and maintenance processes of industrial robots, and it is difficult to accurately detect the real-time working state of industrial robots. Summary of the Invention

[0003] The present invention provides a method, system, device and storage medium for detecting the state of an industrial robot, so as to improve the accuracy of real-time state detection of industrial robots.

[0004] In a first aspect, the present invention provides a method for detecting the state of an industrial robot, the method including: Obtaining the sensor time-series data of each axis of the industrial robot in a target sampling period; For each axis of the industrial robot, determining the energy time-varying data of the corresponding axis in the target sampling period according to the sensor time-series data; Converting the energy time-varying data into a two-dimensional amplitude-frequency image, and performing feature extraction on the two-dimensional amplitude-frequency image to obtain the state feature vector of the corresponding axis in the target sampling period; Inputting the state feature vector into an LSTM model to obtain the state classification result of the corresponding axis in the target sampling period output by the LSTM model; Wherein, the LSTM model is trained using the sample state feature vectors with state classification labels.

[0005] In a second aspect, the present invention further provides a system for detecting the state of an industrial robot, the system including: A sensor module, configured to collect the operation data of each axis of the industrial robot; An edge computing gateway, communicatively connected to the sensor module, configured to process and transmit the data collected by the sensor module; A cloud platform, communicatively connected to the edge computing gateway, configured to execute the method for detecting the state of the industrial robot in the first aspect.

[0006] In a third aspect, the present invention further provides a device for detecting the state of an industrial robot, the device including: An obtaining unit, configured to obtain the sensor time-series data of each axis of the industrial robot in a target sampling period; A determining unit, configured to, for each axis of the industrial robot, determine the energy time-varying data of the corresponding axis in the target sampling period according to the sensor time-series data; An extraction unit that converts the energy time-varying data into a two-dimensional amplitude-frequency image, extracts features from the two-dimensional amplitude-frequency image, and obtains a state feature vector of the corresponding axis during the target sampling period; A detection unit for inputting the state feature vector into an LSTM model to obtain a state classification result of the corresponding axis during the target sampling period output by the LSTM model; Wherein, the LSTM model is trained using sample state feature vectors with state classification labels.

[0007] In a fourth aspect, the present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and running on the processor. When the processor executes the computer program, the industrial robot state detection method described in the first aspect above is implemented.

[0008] In a fifth aspect, the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the industrial robot state detection method described in the first aspect above is implemented.

[0009] In a sixth aspect, the present invention further provides a computer program product, which includes a computer program or instruction. When the computer program or instruction runs on a computer, the computer is made to execute the industrial robot state detection method described in the first aspect above.

[0010] The industrial robot state detection method, system, device, and storage medium provided by the present invention convert the sensor time-series data of each axis of the industrial robot into energy time-varying data by obtaining the sensor time-series data of each axis of the industrial robot, form a two-dimensional amplitude-frequency image and obtain a state feature vector, and input the feature vector into a trained LSTM model to obtain a state classification result. It can effectively detect the state of the industrial robot in real time, improve the accuracy of real-time state detection of the industrial robot, and provide support for the safe operation of the industrial robot. Description of the Drawings

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0012] Figure 1 It is a flowchart of the industrial robot state detection method provided by the embodiment of the present invention; Figure 2 It is an example diagram of a two-dimensional amplitude-frequency image provided by the embodiment of the present invention; Figure 3 is an exemplary diagram of the industrial robot status detection provided by an embodiment of the present invention; Figure 4 is a schematic diagram of the industrial robot status detection system provided by an embodiment of the present invention; Figure 5 is a schematic diagram of the data acquisition program control flow provided by an embodiment of the present invention; Figure 6 is a schematic diagram of the structure of the industrial robot status detection device provided by an embodiment of the present invention; Figure 7 is a schematic diagram of the structure of the electronic device provided by an embodiment of the present invention. Specific Embodiments

[0013] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0014] The term "and / or" in the present invention describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0015] The term "a plurality of" in the present invention means two or more, and other quantifiers are similar thereto.

[0016] The terms "first", "second", etc. in the present invention are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances so that the embodiments of the present invention can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are usually of the same type, and the number of objects is not limited. For example, the first object can be one or multiple.

[0017] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are only some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0018] To facilitate a clearer understanding of the technical solutions of the embodiments of the present invention, some technical contents related to the embodiments of the present invention are introduced first.

[0019] Industrial robot status detection integrates various sensors and monitoring means to evaluate the mechanical, electrical, and control system status of industrial robots in real time, ensuring the safe and efficient operation of the robot system. Common techniques include vibration analysis, temperature detection, electrical system detection, speed detection, etc. Vibration analysis evaluates the operating status by measuring the vibration signals of key components of the robot (such as motors, bearings, gears, etc.). Abnormal vibration patterns may indicate wear, imbalance, looseness, or other mechanical failures. Through spectrum analysis, vibrations at specific frequencies can be identified, thereby diagnosing specific fault types. Temperature detection involves using thermocouples or infrared sensors to measure the temperature of key components of the robot. Excessive temperature may indicate overload, cooling system failure, or electrical problems. Continuous temperature monitoring helps detect early signs of overheating and prevent equipment damage. Electrical system detection involves measuring electrical parameters such as current, voltage, and resistance to evaluate the health of the robot's electrical system. Abnormal electrical readings may indicate electrical component failures, loose connections, or insulation degradation. Speed detection can measure the displacement and speed of joints in real time through encoders installed on the robot joints, used to detect abnormal joint movements, such as excessive displacement deviations or speed fluctuations.

[0020] The above industrial robot status detection methods often rely on a large amount of data collection. The collection, storage, and processing of data require complex system architectures and cannot comprehensively consider multiple sensor data to judge the real-time status of industrial robots. In addition, judging the status of industrial robots based on the collected data highly depends on an in-depth understanding of the operation and maintenance processes of industrial robots, requiring enterprises to invest a large amount of manpower. Based on this, the embodiments of the present invention provide a solution. By converting the sensor time-series data into energy time-varying data to form a two-dimensional amplitude-frequency image, after feature extraction and input into the LSTM model, the status detection result can be obtained, effectively detecting the status of industrial robots in real time and improving the accuracy of real-time status detection of industrial robots.

[0021] Figure 1 It is a schematic flowchart of the industrial robot status detection method provided by the embodiment of the present invention. As Figure 1 shown, the method includes the following steps 101, 102, 103, and 104.

[0022] Step 101: Obtain the sensor time-series data of each axis of the industrial robot during the target sampling period.

[0023] Specifically, industrial robots can be used in different application fields such as handling, welding, spraying, assembly, etc., which are not limited here. Industrial robots can be robots with different mechanical structures such as Cartesian coordinate robots, planar multi-joint robots, etc., which are not limited here.

[0024] The axes of an industrial robot refer to different joints or degrees of freedom of the industrial robot. For example: a six-axis robot has six degrees of freedom.

[0025] During the real-time detection process, the target sampling period can be the most recent sampling period. Multiple samplings are carried out within one sampling period, the time interval between two adjacent sampling moments can be set, and the duration of each sampling period can be set as needed, which is not limited here.

[0026] Sensor time-series data refers to a series of data collected by sensors over a period of time, and these data are sorted in time sequence. In the embodiments of the present invention, the sensors are used to collect corresponding data of the servo motors of each axis of the industrial robot, and the number and types of sensors are not limited in the present invention.

[0027] In some embodiments, the execution subject of the method is a cloud platform. The sensors collect data of each axis servo motor in real time within each sampling period to form sensor time-series data, and then the data is transmitted to the edge computing gateway through the controller. The edge computing gateway transmits the data to the cloud platform, and the cloud platform obtains the sensor time-series data of each axis of the industrial robot in the target sampling period for state detection of each axis of the industrial robot.

[0028] Step 102: For each axis of the industrial robot, determine the energy time-varying data of the corresponding axis in the target sampling period according to the sensor time-series data.

[0029] Specifically, the energy time-varying data is data of energy changing over time within a period of time.

[0030] After obtaining the sensor time-series data of each axis of the industrial robot in the target sampling period, for each axis of the industrial robot, the corresponding energy time-varying data can be obtained according to the sensor time-series data corresponding to the axis, and the energy time-varying data is used to characterize the energy change of the axis.

[0031] Step 103: Convert the energy time-varying data into a two-dimensional amplitude-frequency image, extract features from the two-dimensional amplitude-frequency image, and obtain the state feature vector of the corresponding axis in the target sampling period.

[0032] Specifically, the two-dimensional amplitude-frequency image refers to a two-dimensional image used to display the distribution of energy values in two dimensions of amplitude and frequency, which can visualize the characteristics of the energy time-varying data. The abscissa represents the amplitude of the energy change, the ordinate represents the frequency of the energy change, and the two-dimensional amplitude-frequency image can visualize the characteristics of the energy change amplitude and frequency.

[0033] In the present invention, after converting the sensor timing data into energy time-varying data in step 102, the energy time-varying data can be first converted into a two-dimensional amplitude-frequency image, and then feature extraction is performed on the two-dimensional amplitude-frequency image, and the extracted feature vector is used as the state feature vector of the corresponding axis in the target sampling period.

[0034] In some embodiments, an energy change curve of the energy value changing with the sampling time within the target sampling period can be obtained from the energy time-varying data. Using the fast Fourier transform, the energy magnitude and energy variability in the energy time-varying data can be extracted. The energy magnitude is arranged on the horizontal axis and the energy variability is arranged on the vertical axis to form a two-dimensional amplitude-frequency image. Figure 2 An example diagram of the two-dimensional amplitude-frequency image provided by the embodiment of the present invention is as Figure 2 shown. The two-dimensional amplitude-frequency image can reflect different states of the corresponding axis of the industrial robot within a sampling period. The left side is the two-dimensional amplitude-frequency image formed by the data collected under the normal state of the corresponding axis of the industrial robot, and the right side is the two-dimensional amplitude-frequency image formed by the data collected under the abnormal state of the corresponding axis of the industrial robot. Feature extraction is performed on the formed two-dimensional amplitude-frequency image, and the extracted features can reflect the state of the corresponding axis in the target sampling period.

[0035] Step 104: Input the state feature vector into the LSTM model to obtain the state classification result of the corresponding axis in the target sampling period output by the LSTM model.

[0036] Among them, the LSTM model is trained using the sample state feature vectors with state classification labels.

[0037] Specifically, the LSTM model can adopt the structure of the existing LSTM model, which is not limited herein. The output layer of the LSTM model can be used for state classification.

[0038] In some embodiments, the LSTM model can use the sigmoid activation function. This activation function is divided into two types: the logistics sigmoid activation function and the Tanh sigmoid activation function. Among them, the error of the Tanh sigmoid activation function is smaller. Therefore, the activation function in the present invention can use the Tanh sigmoid activation function.

[0039] In some embodiments, the maximum number of iterations of the LSTM model can be set to 1000 times, and the error can be set to .

[0040] Regarding the number of hidden layer nodes in the LSTM model, when the number of nodes is too small, it will lead to insufficient learning, unable to obtain a sufficient amount of effective information, and unable to meet the requirements for handling complex problems. When the number of nodes is too large, the training time will increase significantly, unable to meet the real-time requirements, and will increase the possibility of overfitting. Through experiments, the number of hidden layer nodes in the LSTM model can be set to 6 - 12.

[0041] In some embodiments, the state classification result can be normal, abnormal, or other operating states of the industrial robot, which are not limited herein.

[0042] In some embodiments, the state classification label is used to indicate the state classification to which the sample belongs. The state classification label can be normal, abnormal, or other operating states of the industrial robot, which are not limited herein. The sample state feature vector with the state classification label refers to the sample state feature vector given the state classification. Among them, the state classification label corresponds one-to-one with the state feature vector. For example: the state feature vector 1 can be marked as normal, and the state feature vector 2 can be marked as abnormal. In step 103, the sample state feature vectors corresponding to the training set and the test set are obtained, and the LSTM model capable of performing state classification is obtained after training using the sample state feature vectors with state classification labels.

[0043] In the present invention, after obtaining the state feature vector of the corresponding axis in the target sampling period in step 103, the state feature vector is input into the trained LSTM model capable of performing state classification, and the state classification result of the corresponding axis in the target sampling period can be output.

[0044] The industrial robot state detection method provided by the embodiments of the present invention converts the sensor time-series data of the corresponding axis of the industrial robot into energy time-varying data, forms a two-dimensional amplitude-frequency image through the energy time-varying data, extracts the features of the two-dimensional amplitude-frequency image, and inputs the feature vector into the LSTM model to obtain the state classification of the corresponding axis of the industrial robot. It can effectively detect the state of the industrial robot in real time and improve the accuracy of real-time state detection of the industrial robot.

[0045] In some embodiments, when the sensor time-series data includes the time-series data collected by multiple sensors, determining the energy time-varying data of the corresponding axis in the target sampling period according to the sensor time-series data includes: Converting the sensor measurement value at each sampling moment in the sensor time-series data into an energy value according to the sensor type; Accumulating the energy values at the same sampling moment to obtain the total energy value at each sampling moment, and the time-series data of the total energy value is the energy time-varying data.

[0046] Specifically, the multiple sensors refer to multiple different types of sensors for the same axis of an industrial robot, and the data measured by the multiple sensors is all data of the same servo motor. The sensor type refers to the type of data that the sensor can measure, and the sensor type is not limited here. For example, it can be a temperature sensor that measures the temperature of the servo motor, or it can be a speed sensor that measures the speed of the servo motor.

[0047] In some embodiments, each sampling moment is each sampling time point within the target sampling period, and the sensor measurement value is the data measured by the sensor at the sampling moment. There can be multiple types of energy values, which are not limited here and can be kinetic energy values, potential energy values, thermal energy values, electrical energy values. The total energy value is obtained by accumulating different energy values converted from different sensor measurement values. The time series data of the total energy value is formed by the total energy values at multiple sampling moments within the target sampling period.

[0048] In the present invention, in the case where multiple sensors collect time series data, time series data of multiple sensors will be obtained. The types of data collected by the sensors, such as vibration or temperature, can directly or indirectly convert them into certain energy information and can be converted into energy values, such as kinetic energy or thermal energy. The multiple energy values converted from the measurement values of multiple sensors at this sampling moment can be accumulated to obtain the total energy value at this sampling moment. Within the target sampling period, the total energy values at each sampling moment can form the time series data of the total energy value, that is, the energy time-varying data.

[0049] The industrial robot state detection method provided by the embodiments of the present invention converts the sensor measurement value into an energy value according to the sensor measurement data type, and can use the energy-driven method to transform different types of data onto a unified energy value for processing, comprehensively detect the real-time state of the industrial robot based on different sensor measurement values, and improve the accuracy of detection.

[0050] In some embodiments, feature extraction of the two-dimensional amplitude-frequency image includes: Using a CNN model to perform feature extraction on the two-dimensional amplitude-frequency image.

[0051] Specifically, the input layer, convolutional layer, and pooling layer of the CNN model can adopt the structure of an existing CNN model, which is not limited here. The output layer of the CNN model only includes a fully connected layer, that is, the CNN model outputs a feature vector.

[0052] In some embodiments, considering the size of the input two-dimensional spectrum image, the number of convolutional layers of the CNN model can be set to 3 layers, the number of convolutional kernels corresponding to the 3 convolutional layers can be set to 4, 8, and 16 respectively, the size of the convolutional kernels can be set to 3×3 and 2×2, the pooling method can adopt max pooling, and the number of neurons can be set to 16.

[0053] In some embodiments, the hyperparameter tuning method of the CNN model can use grid search. For example, 48 groups of hyperparameter combination tuning can be performed.

[0054] Feature extraction is to input the two-dimensional spectrum image into the CNN model. After passing through the input layer, convolutional layer, pooling layer, and fully connected layer of the CNN model, a state feature vector is output. This state feature vector is the feature corresponding to the two-dimensional amplitude-frequency image.

[0055] In the present invention, after obtaining the two-dimensional amplitude-frequency image through the energy time-varying parameter, the CNN model can be used to perform feature extraction on the two-dimensional amplitude-frequency image and output a state feature vector.

[0056] The industrial robot state detection method provided by the embodiments of the present invention can better extract the energy features in the two-dimensional amplitude-frequency image, obtain a more accurate state feature vector, and improve the accuracy of real-time state detection of the industrial robot by using the CNN model to perform feature extraction on the two-dimensional amplitude-frequency image.

[0057] Figure 3 is an example diagram of industrial robot state detection provided by the embodiments of the present invention. As Figure 3 shown, for the field devices of each axis, there is a corresponding servo motor on each axis. Sensors can be used to collect the sensor timing data of the corresponding axis servo motor. The controller can transmit the sensor timing data to the edge computing gateway through Ethernet communication. The edge computing gateway can transmit the sensor timing data to the cloud platform through the MQTT protocol of 4G communication. The cloud platform uses the industrial robot state detection method to detect the state of the corresponding axis. First, the sensor timing data is converted into energy time-varying data, and an energy pattern image (i.e., two-dimensional amplitude-frequency image) is obtained through the energy time-varying data to extract the energy features in the sensor timing data. Then, the CNN model can be used to perform feature extraction on the energy pattern image to obtain a state feature vector. After that, the state feature vector is input into the LSTM model for state classification to obtain the state classification result of each axis of the industrial robot, and the result is visualized on the cloud platform.

[0058] In some embodiments, the multiple sensors include a temperature sensor, a current sensor, a vibration sensor, and a displacement sensor.

[0059] Specifically, the multiple sensors can respectively collect different data of the corresponding axis servo motor. The temperature sensor collects the temperature data of the servo motor, the current sensor collects the current data of the servo motor, the vibration sensor collects the vibration data of the servo motor, and the displacement sensor collects the displacement data of the servo motor.

[0060] The industrial robot state detection method provided by the embodiments of the present invention forms sensor time-series data through the data of multiple sensors, and can comprehensively judge the state of the industrial robot based on multiple data such as temperature, current, vibration, and displacement, improving the accuracy of real-time state detection of the industrial robot.

[0061] In some embodiments, according to the sensor type, the sensor measurement value at each sampling moment in the sensor time-series data is converted into an energy value, including: For temperature sensors and current sensors, convert the sensor measurement value into a heat value; For vibration sensors and displacement sensors, convert the sensor measurement value into a kinetic energy value.

[0062] Specifically, for a temperature sensor, an initial temperature can be set as , the measured temperature of the sensor at a certain moment is , and the temperature measured by the temperature sensor at this moment can be used in the formula to calculate the heat value corresponding to the temperature measurement value at the current moment , where m is the mass of the corresponding axis of the industrial robot and c is the specific heat capacity. For a current sensor, the current sensor measurement value at the current moment can be used in the formula to calculate the heat value corresponding to the current measurement value at the current moment , where I is the magnitude of the current measured by the current sensor at the current moment, R is the resistance value of the resistor, and t is the time for the current to pass through the resistor.

[0063] For a vibration sensor, the vibration sensor measurement value at the current moment can be used in the formula to calculate the kinetic energy value corresponding to the vibration measurement value at the current moment , where m is the mass of the corresponding axis of the industrial robot and v is the vibration speed of the corresponding axis of the industrial robot. For a displacement sensor, the displacement sensor measurement value at the current moment can be used in the formula to calculate the kinetic energy value corresponding to the displacement measurement value at the current moment , where m is the mass of the corresponding axis of the industrial robot and a is the acceleration of the corresponding axis of the industrial robot in motion.

[0064] Through the above method, the embodiments of the present invention can convert the measurement values of different sensors into heat values and kinetic energy values respectively, extract the energy information in different types of data, and comprehensively detect the real-time state of the industrial robot based on the measurement values of different sensors, improving the accuracy of detection.

[0065] Figure 4 is a schematic diagram of the industrial robot state detection system provided by the embodiments of the present invention, as shown in Figure 4As shown, the system includes a sensor module 410, an edge computing gateway 420, and a cloud platform 430.

[0066] The sensor module 410 is used to collect the operation data of each axis of the industrial robot; Specifically, the sensor module 410 includes two parts: a sensor and a controller. There can be one or more sensors, and the specific number is not limited here. For each axis servo motor of the industrial robot, the controller can send control signals to the sensors in each axis of the industrial robot through a data acquisition program, control whether the sensors collect servo motor data, and determine whether to transmit the data to the edge computing gateway 420. The specific transmission method of data transmission is not limited here, and Ethernet communication can be used for data transmission.

[0067] Figure 5 It is a schematic diagram of the control flow of the data acquisition program. As Figure 5 shown, the controller controls the sensor to start collecting data. In this embodiment, there are three types of sensors. The temperature sensor can collect temperature data packets, the vibration sensor can collect vibration data packets, and the current sensor can collect current data packets. The controller judges the collected temperature data packets, vibration data packets, and current data packets to determine whether they are legal data. If the collected data is legal data, it can be judged whether the communication connection between the controller and the edge computing gateway is normal. For example, whether 4G is online. If the communication connection is normal, the collected data can be transmitted to the edge computing gateway 420 and the cloud platform 430, and the data acquisition process ends. If the communication connection is abnormal, the collected data can be saved locally, and the data acquisition process can continue at the same time. If the collected data is not legal data, the data acquisition process can continue until the collected data is legal data before it can continue to be transmitted.

[0068] The edge computing gateway 420 is communicatively connected to the sensor module and is used to process and transmit the data collected by the sensor module; Specifically, the edge computing gateway 420 is communicatively connected to the sensors and the controller in the sensor module 410 respectively. There are various ways for the edge computing gateway 420 to be communicatively connected to the sensors, and the specific communication method is not limited herein. It can be wired communication, wireless communication or other communication methods. For wired communication, data can be transmitted between the sensor node and the gateway through wired communication methods such as network cables and serial ports, which can be Ethernet or RS485. For wireless communication, it can be Wi-Fi, Bluetooth, or Zigbee. The hardware environment of the embodiment of the present invention is in the factory environment of an industrial robot workstation. Considering that wired connection has a more stable signal output than wireless connection, is not easily affected by interference and signal attenuation, etc., and also considering information security and cost efficiency in multiple aspects, the embodiment of the present invention can use a wired connection between the sensor node and the gateway, and use common twisted pair cables for connection. This kind of twisted pair cable helps to reduce signal interference and crosstalk interference, and ensures the reliability and stability of data transmission.

[0069] There are various ways for the edge computing gateway 420 to be connected to the controller, and the specific connection method is not limited herein. It can be connected using an Ethernet interface, using the existing network infrastructure, reducing costs and setup complexity, and completing real-time data transmission within milliseconds.

[0070] There are various data transmission protocols between the edge computing gateway 420 and the controller, and the specific data transmission protocol is not limited herein. Since the PROFINET protocol supports flexible topologies and also supports various different network protocols, including other network protocols such as TCP / IP and UDP / IP. Therefore, the embodiment of the present invention can use the PROFINET communication protocol based on Ethernet communication for data transmission.

[0071] The edge computing gateway 420 can also perform denoising processing on the data collected by the sensor module. The denoising processing method is not limited herein and can be mean filtering, median filtering or other denoising processing methods. The edge computing gateway transmits the denoised collected data to the cloud platform 430.

[0072] The cloud platform 430 is communicatively connected to the edge computing gateway and is used to execute all the method steps implemented in the corresponding method embodiments described above.

[0073] In the present invention, there are various ways for the cloud platform 430 to be communicatively connected to the edge computing gateway 420, and the specific communication connection method is not limited herein. Considering the complex environment in the industrial field, relatively fast data transmission speed, the need for long-distance communication, and data transmission in the absence of a stable wireless local area network, etc., the edge computing gateway 420 can be connected to the cloud platform 430 using a 4G interface.

[0074] There are many types of data transmission protocols between the cloud platform 430 and the edge computing gateway 420. The specific data transmission protocol is not limited here. For example, the MQTT (Message Queuing Telemetry Transport) protocol, the CoAP (Constrained Application Protocol) protocol, and the HTTP (HyperText Transfer Protocol) protocol can be used to transmit data between the gateway and the cloud platform through the TCP / IP protocol. Both the MQTT protocol and the CoAP protocol are protocols for the Internet of Things field, with the advantages of being lightweight, having good scalability, and low power consumption, and are suitable for the scenarios of sensor networks. The HTTP protocol is more suitable for scenarios such as web browsing and data interaction. However, in the Internet of Things field, the disadvantages of the HTTP protocol such as short connections, lack of support for server push, and lack of support for multiplexing make it not the best choice. Therefore, the embodiments of the present invention can use the MQTT protocol to transmit data between the edge computing gateway 420 and the cloud platform 430.

[0075] The cloud platform builds an industrial robot status detection model through the collected sensor data. For the real-time generated sensor data, it outputs the status classification results corresponding to each axis of the industrial robot, and the status classification results can be visualized in the cloud platform.

[0076] It should be noted here that the industrial robot status detection system provided by the embodiments of the present invention can implement all the method steps implemented by the corresponding method embodiments, and can achieve the same technical effects. The same parts and beneficial effects as those in the method embodiments will not be specifically described in this embodiment.

[0077] Figure 6 The structural schematic diagram of the industrial robot status detection device provided by the embodiments of the present invention is as Figure 6 shown, and the device includes: An acquisition unit 610, configured to acquire the sensor time-series data of each axis of the industrial robot in a target sampling period; A determination unit 620, configured to, for each axis of the industrial robot, determine the energy time-varying data of the corresponding axis in the target sampling period according to the sensor time-series data; A determination unit 630, configured to convert the energy time-varying data into a two-dimensional amplitude-frequency image, extract features from the two-dimensional amplitude-frequency image, and obtain the status feature vector of the corresponding axis in the target sampling period; A detection unit 640, configured to input the status feature vector into an LSTM model to obtain the status classification result of the corresponding axis in the target sampling period output by the LSTM model; Wherein, the LSTM model is trained using the sample status feature vectors with status classification labels.

[0078] In some embodiments, when the sensor timing data includes the timing data collected by multiple sensors, determining the time-varying energy data of the corresponding axis in the target sampling period according to the sensor timing data includes: Converting the sensor measurement value at each sampling moment in the sensor timing data into an energy value according to the sensor type; Accumulating the energy values at the same sampling moment to obtain the total energy value at each sampling moment, and the timing data of the total energy value is the time-varying energy data.

[0079] In some embodiments, performing feature extraction on the two-dimensional amplitude-frequency image includes: Performing feature extraction on the two-dimensional amplitude-frequency image using a CNN model.

[0080] In some embodiments, the multiple sensors include a temperature sensor, a current sensor, a vibration sensor, and a displacement sensor.

[0081] In some embodiments, converting the sensor measurement value at each sampling moment in the sensor timing data into an energy value according to the sensor type includes: For the temperature sensor and the current sensor, converting the sensor measurement value into a heat value; For the vibration sensor and the displacement sensor, converting the sensor measurement value into a kinetic energy value.

[0082] It should be noted here that the above industrial robot state detection device provided by the embodiments of the present invention can implement all the method steps implemented by the corresponding method embodiments, and can achieve the same technical effects. The same parts and beneficial effects as those in the method embodiments will not be specifically described in this embodiment.

[0083] It should be noted that the division of modules (or units) in the embodiments of the present invention is illustrative, merely a logical function division, and there may be other division methods in actual implementation. In addition, each functional module (or unit) in various embodiments of the present invention may be integrated into one processing unit, or each module (or unit) may exist physically alone, or two or more modules (or units) may be integrated into one unit. The above integrated modules (or units) may be implemented in the form of hardware or in the form of software functional modules (or units).

[0084] When the integrated module (or unit) is implemented in the form of a software functional module (or unit) and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

[0085] Figure 7 is a schematic structural diagram of the electronic device provided by the present invention, as Figure 7 shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730, and a communication bus 740. Among them, the processor 710, the communication interface 720, and the memory 730 complete communication with each other through the communication bus 740. The processor 710 can call the logical instructions in the memory 730 to execute the above-mentioned industrial robot state detection method, including: Obtain the sensor time-series data of each axis of the industrial robot in the target sampling period; For each axis of the industrial robot, determine the energy time-varying data of the corresponding axis in the target sampling period according to the sensor time-series data; Convert the energy time-varying data into a two-dimensional amplitude-frequency image, perform feature extraction on the two-dimensional amplitude-frequency image, and obtain the state feature vector of the corresponding axis in the target sampling period; Input the state feature vector into the LSTM model to obtain the state classification result of the corresponding axis in the target sampling period output by the LSTM model; Among them, the LSTM model is trained using the sample state feature vectors with state classification labels.

[0086] In addition, when the logical instructions in the above-mentioned memory 730 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0087] It should be noted here that the above-mentioned electronic device provided by the present invention can implement all the method steps implemented by the above method embodiments and can achieve the same technical effects. The same parts and beneficial effects as those in the method embodiments will not be specifically described herein.

[0088] On the other hand, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned industrial robot state detection method is implemented.

[0089] It should be noted here that the above-mentioned non-transitory computer-readable storage medium provided by the present invention can implement all the method steps implemented by the above method embodiments and can achieve the same technical effects. The same parts and beneficial effects as those in the method embodiments will not be specifically described herein.

[0090] On another aspect, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the above-mentioned industrial robot state detection method.

[0091] It should be noted here that the above-mentioned computer program product provided by the present invention can implement all the method steps implemented by the above method embodiments and can achieve the same technical effects. The same parts and beneficial effects as those in the method embodiments will not be specifically described herein.

[0092] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories and optical memories, etc.) containing computer-usable program code.

[0093] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be realized by computer-executable instructions. These computer-executable instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for realizing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0094] These processor-executable instructions can also be stored in a processor-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the processor-readable memory generate a manufactured article including instruction means, and the instruction means realizes the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0095] These processor-executable instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for realizing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0096] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these changes and modifications.

Claims

1. An industrial robot state detection method, characterized in that, Including: Obtain the sensor time-series data of each axis of the industrial robot during the target sampling period; For each axis of the industrial robot, determine the energy time-varying data of the corresponding axis during the target sampling period according to the sensor time-series data; Convert the energy time-varying data into a two-dimensional amplitude-frequency image, extract features from the two-dimensional amplitude-frequency image, and obtain the state feature vector of the corresponding axis during the target sampling period; Input the state feature vector into the LSTM model to obtain the state classification result of the corresponding axis during the target sampling period output by the LSTM model; Wherein, the LSTM model is trained using sample state feature vectors with state classification labels.

2. The industrial robot state detection method according to claim 1, characterized in that, In the case where the sensor time-series data includes time-series data collected by multiple sensors, the determining the energy time-varying data of the corresponding axis during the target sampling period according to the sensor time-series data includes: According to the sensor type, convert the sensor measurement value at each sampling moment in the sensor time-series data into an energy value; Accumulate the energy values at the same sampling moment to obtain the total energy value at each sampling moment, and the time-series data of the total energy value is the energy time-varying data.

3. The industrial robot state detection method according to claim 1 or 2, characterized in that, The extracting features from the two-dimensional amplitude-frequency image includes: Use a CNN model to extract features from the two-dimensional amplitude-frequency image.

4. The industrial robot state detection method according to claim 2, wherein The multiple sensors include a temperature sensor, a current sensor, a vibration sensor, and a displacement sensor.

5. The industrial robot status detection method according to claim 4, characterized in that, The converting the sensor measurement value at each sampling moment in the sensor time-series data into an energy value according to the sensor type includes: For the temperature sensor and the current sensor, convert the sensor measurement value into a heat value; For the vibration sensor and the displacement sensor, convert the sensor measurement value into a kinetic energy value.

6. An industrial robot status detection system, characterized in that Including: A sensor module for collecting the operation data of each axis of the industrial robot; An edge computing gateway communicatively connected to the sensor module for processing and transmitting the data collected by the sensor module; A cloud platform communicatively connected to the edge computing gateway for executing the industrial robot state detection method according to any one of claims 1 to 5.

7. An industrial robot state detection device, characterized in that, The device includes: An obtaining unit for obtaining the sensor time-series data of each axis of the industrial robot during the target sampling period; A determining unit for, for each axis of the industrial robot, determining the energy time-varying data of the corresponding axis during the target sampling period according to the sensor time-series data; An extracting unit for converting the energy time-varying data into a two-dimensional amplitude-frequency image, extracting features from the two-dimensional amplitude-frequency image, and obtaining the state feature vector of the corresponding axis during the target sampling period; A detecting unit for inputting the state feature vector into the LSTM model to obtain the state classification result of the corresponding axis during the target sampling period output by the LSTM model; Wherein, the LSTM model is trained using sample state feature vectors with state classification labels.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the industrial robot state detection method according to any one of claims 1 to 5.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the industrial robot state detection method according to any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the industrial robot state detection method according to any one of claims 1 to 5.

Images