Industrial robot detection method and system based on machine vision

Abstract

The present invention discloses an industrial robot detection method and system based on machine vision, which relates to the technical field of industrial detection. The real-time video of the mechanical arm is captured by a visual monitoring instrument to timely discover the position offset Pd, and compare it with the preset displacement threshold. If the offset exceeds the threshold, a tracing command is issued. After receiving the command, the range tracing module uses the historical detection data set to evaluate the event probability of each joint, and can further accurately trace the joint that is most likely to fail. This method can detect potential fault points early and reduce the spread of faults. The joint analysis module monitors the strain data information and status information of each joint in real time, and extracts relevant data according to the fault tracing range. Combined with the fault prediction model to calculate the risk index Fzs, it can comprehensively evaluate the health status of each joint and form a detailed fault report to help maintenance personnel quickly locate the problem.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial detection, and in particular to an industrial robot detection method and system based on machine vision. Background Art

[0002] With the rapid development of automation technology, industrial robots have become an indispensable core equipment in modern manufacturing, and are widely used in various tasks of production lines, including manufacturing, assembly, welding and spraying. Their reliability and accuracy directly affect production efficiency and product quality. At the same time, with the advancement of technology, machine vision, as an advanced detection method, has been introduced into industrial robot systems for real-time monitoring and analysis of the working status of robots. It can observe and analyze the position status of industrial robot arms in detail through high-precision cameras and image processing algorithms.

[0003] During the operation of an industrial robot arm, due to the complexity of the mechanical parts, each joint may fail. Especially when the arm is out of position, the superposition of failures will make the tracing of the problem more complicated. Therefore, tracing the joint that failed first is a challenging task.

[0004] When faced with the problem of position offset of the industrial robot arm, traditional methods can often only provide the overall status information of the arm, but it is difficult to accurately identify which joint in the industrial robot failed first. The shortcoming of this method is that it may be difficult to handle complex fault superposition phenomena, resulting in inaccurate tracing of the source of the fault. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides an industrial robot detection method and system based on machine vision, which solves the problems in the above-mentioned background technology.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: an industrial robot detection system based on machine vision, including a detection module, a preliminary judgment module, a range tracing module, a joint analysis module and a judgment module;

[0007] The detection module is used to capture the position status of each mechanical arm of the industrial robot by using a visual monitoring instrument when the industrial robot is in a working state, extract continuous image frames from the captured video, and perform preprocessing operations;

[0008] The initial judgment module is used to obtain the position state data of each robot arm in the preprocessed image frame to determine the position coordinates of each robot arm at the corresponding time point , and according to the position coordinates of each robot arm in the corresponding time period , to analyze the offset displacement Pd, if the offset displacement Pd exceeds the displacement threshold, a traceback command is issued externally;

[0009] The range tracing module is used to obtain the historical probability of failure of each joint in the industrial robot by using the historical detection data set after receiving the tracing command. and point process probability , based on historical probability and the point process probability of the corresponding joint , calculate the event probability of obtaining the corresponding joint , according to the event probability of the corresponding joint Size, determine the fault tracing target point and obtain the fault tracing range;

[0010] The joint analysis module is used to monitor the relevant strain data information and relevant presentation status data information of each joint in the industrial robot in real time, and extract the relevant strain data information and relevant presentation status data information of the corresponding joints within the fault tracing range according to the acquired fault tracing range, and construct the risk index Fzs according to the relevant strain data information and relevant presentation status data information of the corresponding joints within the fault tracing range and in combination with the fault prediction model;

[0011] The judgment module is used to pre-set an evaluation threshold W, and compare and analyze the risk index Fzs with the evaluation threshold W to determine the source of the fault in the industrial robot again.

[0012] Preferably, the detection module includes a capture unit and a preprocessing unit;

[0013] The capture unit is used to capture the video of the industrial robot in a working state using a visual monitoring instrument, and collect and record relevant continuous image frames therefrom, and then sort the continuous image frames in sequence according to the order in which they were taken;

[0014] The preprocessing unit is used to convert the image from the RGB color space to the HSV color space, and use filtering technology to remove noise in each image frame through median filtering.

[0015] Preferably, the preliminary judgment module includes a comparison unit and a command issuing unit;

[0016] The comparison unit is used to extract the position state data of each robot arm according to the processed image frame, and to obtain the position coordinates of each robot arm at the corresponding time period. and compare it with the preset position coordinates in the control program Compare and calculate the offset displacement Pd at the corresponding time point, which can be obtained according to the following formula: ;

[0017] The command issuing unit is used to compare and analyze the offset displacement Pd obtained by the comparison unit with the displacement threshold to determine whether each mechanical arm of the current industrial robot has a risk of position offset when working. The specific comparison and analysis content is as follows:

[0018] If the offset displacement Pd at the corresponding time point exceeds the displacement threshold, it is judged that the corresponding mechanical arm of the current industrial robot has the risk of position offset during operation, and a traceback command will be issued externally at this time;

[0019] If the offset displacement Pd at the corresponding time point does not exceed the displacement threshold, it is determined that the corresponding mechanical arm of the current industrial robot does not have a risk of position offset when working.

[0020] Preferably, the scope tracing module includes a historical tracing analysis unit and a preliminary target locking unit;

[0021] The historical tracing analysis unit is used to obtain the historical probability of failure of each joint in the industrial robot by using the historical detection data set after receiving the tracing command. and point process probability , wherein the historical detection data set includes the number of times the robot position is offset by the corresponding joint during the historical monitoring period , the total number of times Zlc caused by all joints to offset the position of the robot arm during the historical monitoring period, the number of all robot arms set in the corresponding time period And the number of robot arms whose positions are offset by the corresponding joints in each time period The specific acquisition methods are as follows:

[0022] ;

[0023] Where Zlc is the total number of times the robot position is offset by all joints in the historical monitoring period. It is expressed as the number of times the position of the robot arm is offset by the i-th joint in the historical monitoring cycle;

[0024] ;

[0025] Where n represents the number of historical time periods extracted from the historical monitoring cycle, k=1, 2, 3, ..., n, It is expressed as the number of manipulators whose positions are offset by the i-th joint in the k-th time period. Represents the number of all robotic arms set for the kth time period.

[0026] Preferably, based on historical probability and the point process probability of the corresponding joint , calculate the event probability of obtaining the corresponding joint , which is obtained by the following formula:

[0027] ;

[0028] In the formula, It is expressed as the total probability of robot position deviation.

[0029] Preferably, the preliminary target locking unit is used to determine the event probability of the corresponding joint. The calculation method is to obtain the event probability of each joint respectively. , and the event probability of each joint Sort them in order of value to get the probability of the event with the maximum value. , and the probability of the event with the largest value The corresponding joint is regarded as the initial abnormal joint, and the initial abnormal joint is used as the fault tracing target point, and the tracing range is gradually spread to the surrounding area to determine the fault tracing range.

[0030] Preferably, the joint analysis module includes a monitoring unit, a tracing analysis unit and a comprehensive analysis unit;

[0031] The monitoring unit is used to monitor the relevant strain data information and relevant presentation status data information of each joint in the industrial robot in real time to construct a monitoring set, wherein the relevant strain data information in the monitoring set includes the hysteresis Hc and joint torque Gnj of each joint;

[0032] The relevant status data information in the monitoring set includes the axial clearance Zxjx, bearing clearance Zcyj and tooth side clearance Ccjx of each joint;

[0033] The tracing analysis unit is used to extract relevant strain data information and relevant presentation status data information of the corresponding joints within the fault tracing range from the monitoring set after determining the fault tracing range, and calculate the fatigue coefficient Pxs of the corresponding joint based on the extracted relevant strain data information:

[0034] ;

[0035] Where m represents the number of joints within the fault tracing range, i=1, 2, 3, ..., m, Hc i Expressed as the hysteresis of the i-th joint, Hc avg It is expressed as the mean hysteresis of all joints within the fault tracing range, Gnj i Expressed as the joint torque of the i-th joint, Gnj avg It is expressed as the mean joint torque of all joints within the fault tracing range, and All are weight values;

[0036] According to the extracted relevant presentation state data information, the mechanical state coefficient Jxs of the corresponding joint is calculated:

[0037] ;

[0038] Where Ccjx represents the tooth side clearance, Zxjx represents the axial clearance, Zcyj represents the bearing clearance, and a1, a2 and a3 are all weighted values.

[0039] Preferably, the comprehensive analysis unit is used to input the fatigue coefficient Pxs and the mechanical state coefficient Jxs of the corresponding joint into the fault prediction model to obtain the risk index Fzs of the corresponding joint:

[0040] ;

[0041] Wherein, wz represents the temperature value, C represents the correction constant, and F1, F2 and F3 are all weight values.

[0042] Preferably, the judgment module includes an analysis unit and a tracing unit;

[0043] The analysis unit is used to compare and analyze the risk index Fzs with the evaluation threshold W to determine the source of the fault in the industrial robot again. The specific analysis content is as follows:

[0044] If the risk index Fzs of the corresponding joint exceeds the evaluation threshold W, it is indicated that the corresponding joint has a fault and is marked as a faulty joint;

[0045] If the risk index Fzs of the corresponding joint does not exceed the evaluation threshold W, it means that the corresponding joint does not have a fault and no marking is performed;

[0046] The tracing unit is used to count the joints marked as faulty, and determine the joint that first fails based on the execution sequence relationship between the faulty joints, and regard the joint as the source of the position deviation of the mechanical arm of the industrial robot.

[0047] An industrial robot detection method based on machine vision comprises the following steps:

[0048] Step 1: When the industrial robot is in working state, use visual instruments to capture the position state of the robot arm of the industrial robot, extract continuous image frames from the captured video stream, and perform preprocessing operations;

[0049] Step 2: Obtain the position state data of the robot arm in the preprocessed image frame to determine the position coordinates of the robot arm in the corresponding time period , and based on the position coordinates of the robot arm at the corresponding time period , to analyze the offset displacement Pd, if the offset displacement Pd exceeds the displacement threshold, a traceback command is issued externally;

[0050] Step 3: After receiving the traceability command, use the historical detection data set to obtain the historical probability of failure of each joint in the robot and point process probability , based on historical probability and the point process probability of the corresponding joint , calculate the event probability of obtaining the corresponding joint , according to the event probability of the corresponding joint Size, determine the fault tracing target point and obtain the fault tracing range;

[0051] Step 4: Based on the acquired fault tracing scope, monitor the relevant strain data information and relevant presentation status data information of the corresponding joints within the fault tracing scope, and construct the risk index Fzs based on the relevant strain data information and relevant presentation status data information of the corresponding joints within the fault tracing scope and in combination with the fault prediction model;

[0052] Step 5: Pre-set the evaluation threshold W, and compare and analyze the risk index Fzs with the evaluation threshold W to determine the source of the fault in the industrial robot again.

[0053] The present invention provides an industrial robot detection method and system based on machine vision, which has the following beneficial effects:

[0054] (1) The real-time video of the robot arm is captured by the visual monitoring instrument to detect the position offset Pd in ​​time and compare it with the preset displacement threshold. If the offset exceeds the threshold, a traceback command is issued. After receiving the command, the range traceback module uses the historical detection data set to evaluate the event probability of each joint. , can further accurately trace the joints that are most likely to fail. This method can detect potential fault points early and reduce the spread of faults. The joint analysis module monitors the strain data and status information of each joint in real time, and extracts relevant data according to the fault tracing range. Combined with the fault prediction model to calculate the risk index Fzs, it can comprehensively evaluate the health status of each joint and form a detailed fault report. This comprehensive analysis can effectively identify the specific source of the fault and help maintenance personnel quickly locate the problem. The judgment module compares the risk index Fzs with the pre-set evaluation threshold W. The analysis results help to accurately determine the source of the fault in the industrial robot. Through this intelligent evaluation mechanism, the efficiency of fault handling can be further improved, maintenance decisions can be optimized, and downtime and production losses can also be reduced. Through real-time monitoring, rapid fault tracing and risk assessment, the system can effectively prevent the occurrence of major faults, thereby reducing frequent repairs and replacement of parts, and reducing maintenance costs. At the same time, the system's early warning and accurate fault location can intervene before the problem expands, further saving production resources. In short, the system further improves the operational reliability of industrial robots, reduces the risk of failures and maintenance costs through visual inspection, early fault tracing, comprehensive analysis, and optimized maintenance decisions, providing strong technical support for industrial production.

[0055] (2) The comparison unit extracts the robot position status data in the processed image frame and compares it with the preset position coordinates in the control program to calculate the offset displacement Pd of each robot at the corresponding time point. This precise calculation can accurately identify whether the robot has a position offset problem, thereby providing data support for subsequent fault diagnosis and maintenance. The command issuing unit can compare and analyze the offset displacement Pd with the preset displacement threshold to evaluate in real time whether the robot has a risk of position offset. This analysis process can quickly identify potential offset risks, allowing the system to intervene in the early stages of the problem and reduce the production impact caused by position offset.

[0056] (3) At the same time, the historical detection data set is used to calculate the event probability of the corresponding joint ,This probability calculation can quantify the probability of failure of each joint, which helps to prioritize those ,joints with a higher probability of failure among many joints, thereby further ,speeding up the tracing of the faulty joints, while also improving the ,efficiency and accuracy of fault location.

[0057] (4) Statistics are collected based on the marked faulty joints, and the joint that first fails is determined based on the execution sequence of the faulty joints. This method can trace the source of the fault and regard it as the root cause of the position deviation of the industrial robot's mechanical arm. This optimized fault tracing method can accurately locate the source of the fault, thereby reducing troubleshooting time and cost and improving fault handling efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a block diagram of an industrial robot detection system based on machine vision of the present invention;

[0059] Figure 2 The present invention is a flow chart of an industrial robot detection method based on machine vision. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] Example 1

[0062] See also Figure 1 ,The present invention provides an industrial robot detection system based on machine vision, including a detection module, a preliminary judgment module, a range tracing module, a joint analysis module and a judgment module;

[0063] The detection module is used to capture the position status of each mechanical arm of the industrial robot by using a visual monitoring instrument when the industrial robot is in a working state, extract continuous image frames from the captured video, and perform preprocessing operations;

[0064] The initial judgment module is used to obtain the position state data of each robot arm in the preprocessed image frame to determine the position coordinates of each robot arm at the corresponding time point , and according to the position coordinates of each robot arm in the corresponding time period , to analyze the offset displacement Pd, if the offset displacement Pd exceeds the displacement threshold, a traceback command is issued externally;

[0065] The range tracing module is used to obtain the historical probability of failure of each joint in the industrial robot by using the historical detection data set after receiving the tracing command. and point process probability , based on historical probability and the point process probability of the corresponding joint , calculate the event probability of obtaining the corresponding joint , according to the event probability of the corresponding joint Size, determine the fault tracing target point and obtain the fault tracing range;

[0066] The joint analysis module is used to monitor the relevant strain data information and relevant presentation status data information of each joint in the industrial robot in real time, and extract the relevant strain data information and relevant presentation status data information of the corresponding joints within the fault tracing range according to the acquired fault tracing range, and construct the risk index Fzs according to the relevant strain data information and relevant presentation status data information of the corresponding joints within the fault tracing range and in combination with the fault prediction model;

[0067] The judgment module is used to pre-set an evaluation threshold W, and compare and analyze the risk index Fzs with the evaluation threshold W to determine the source of the fault in the industrial robot again.

[0068] During the operation of this system, the detection module uses visual monitoring instruments to capture real-time video when the industrial robot is in working condition, and extracts continuous image frames from it for preprocessing. This process ensures high-precision monitoring of the position status of the robot arm and effectively reduces the missed faults caused by visual detection errors. The initial judgment module obtains the position information of each robot arm by analyzing the preprocessed image frames and calculates the offset displacement Pd. When the offset displacement Pd exceeds the set threshold, the system can quickly issue a traceback command. This method can detect abnormalities in the position of the robot arm in time and prevent potential fault expansion in advance. After receiving the traceback command, the range traceability module calculates the event probability of each joint based on the historical detection data set. , and determine the scope of fault tracing. This process can comprehensively consider the historical probability of each joint of the robot arm. and point process probability , thereby providing accurate fault location information; the joint analysis module monitors the strain data and status information of each joint in the industrial robot in real time, and constructs the risk index Fzs in combination with the fault prediction model. Through detailed analysis of the joints within the fault tracing range, the system can identify the components that are most likely to fail, further improving the accuracy of fault prediction. The judgment module compares and analyzes the risk index Fzs with the preset evaluation threshold W to determine the source of the fault in the industrial robot. This intelligent evaluation and analysis method can accurately identify the root location of the problem in complex fault conditions, improving the efficiency and reliability of fault handling. In short, the system further realizes comprehensive monitoring and accurate fault location of the industrial robot arm through multi-level fault detection and analysis methods, providing strong technical support for the safe operation and maintenance of the industrial robot system.

[0069] Example 2

[0070] Please refer to Figure 1 ,Specifically: the detection module includes a capture unit and a pre-processing unit;

[0071] The capture unit is used to capture the video of the industrial robot in working state by using the visual monitoring instrument, and collect and record the relevant continuous image frames therefrom to ensure that the overall working state is fully recorded, and then the continuous image frames are sorted in sequence according to the order of shooting, so as to perform pre-processing operations on them later;

[0072] When dealing with parts of different colors in the same image, it is often necessary to take a different approach for each color to ensure accurate inspection and analysis.

[0073] The preprocessing unit is used to convert the image from the RGB color space to the HSV color space so that it is easier to separate and identify industrial robot parts of different colors. For example, the hue (H) in the HSV color space can be used for color separation more intuitively, and filtering technology is used to remove noise in each image frame through median filtering to ensure the accuracy of color separation and processing.

[0074] In this embodiment, the capture unit captures the video of the industrial robot through a visual monitoring instrument, and collects and records continuous image frames from it. This process ensures that the overall working state of the robot is fully recorded, providing a comprehensive data basis for subsequent analysis; the sequential ordering of continuous image frames enables subsequent preprocessing operations to be performed in a time series, which helps to more accurately capture subtle changes in the movement of the robot arm. The preprocessing unit converts the image from the RGB color space to the HSV color space. This conversion can more effectively separate and identify industrial robot parts of different colors. In the HSV color space, hue (H) makes color separation more intuitive and accurate, thereby further improving the accuracy of component recognition. The processing of parts of different colors can significantly reduce the problem of color aliasing and enhance the detection and analysis capabilities of each part. By removing noise from the image frame through the median filtering technology, the preprocessing unit can effectively improve the clarity and accuracy of the image. This noise removal technology can ensure that the accuracy and reliability of the data are greatly improved during the color separation and analysis process, thereby further avoiding false detection due to noise interference. To sum up, through the efficient cooperation of the capture unit and the preprocessing unit, the detection module can comprehensively record the working status of the industrial robot, optimize color separation and recognition, further improve the accuracy of image processing, realize refined detection, and support efficient subsequent analysis, thereby enhancing the overall performance and reliability of the industrial robot detection system.

[0075] Example 3

[0076] Please refer to Figure 1,Specifically: the preliminary judgment module includes a comparison unit and a command issuing unit;

[0077] The comparison unit is used to extract the position state data of each robot arm according to the processed image frame, and to obtain the position coordinates of each robot arm at the corresponding time period. and compare it with the preset position coordinates in the control program Compare and calculate the offset displacement Pd at the corresponding time point, which can be obtained according to the following formula: ;

[0078] The command issuing unit is used to compare and analyze the offset displacement Pd obtained by the comparison unit with the displacement threshold to determine whether each mechanical arm of the current industrial robot has a risk of position offset when working. The specific comparison and analysis content is as follows:

[0079] If the offset displacement Pd at the corresponding time point exceeds the displacement threshold, it is judged that the corresponding mechanical arm of the current industrial robot has the risk of position offset during operation, and a traceback command will be issued externally at this time;

[0080] If the offset displacement Pd at the corresponding time point does not exceed the displacement threshold, it is determined that the corresponding mechanical arm of the current industrial robot does not have a risk of position offset when working.

[0081] In this embodiment, the comparison unit can accurately calculate the offset displacement Pd at each time point by extracting the position state data of the robot arm in the processed image frame and comparing these data with the preset position coordinates in the control program. This precise calculation ensures the understanding of the position state of the robot arm and helps to identify possible position offsets. The command issuing unit compares and analyzes the calculated offset displacement Pd with the preset displacement threshold to determine whether there is a risk of position offset of the robot arm. If the offset displacement Pd exceeds the threshold, the system will immediately issue a traceback command. This risk assessment and alarm mechanism can detect potential faults in time and prevent the problem from further deteriorating. Through real-time comparison and analysis, the initial judgment module can respond to the position change of the robot arm immediately. When the position offset Pd does not exceed the threshold, the system will continue to monitor to ensure that the working state of the robot arm remains within a safe range. This real-time monitoring capability helps to maintain the stable operation of the industrial robot and reduce unnecessary intervention. Through accurate offset displacement calculation and fast traceback command issuance, the initial judgment module can significantly improve the efficiency of troubleshooting. Identifying the risk of position offset in advance and tracing the offset robot arm provides strong support for subsequent fault analysis and repair, shortens downtime and reduces maintenance costs. The preliminary judgment module enhances the reliability of the overall system through accurate comparison and effective risk judgment. It can timely identify and handle the position offset problem of the robot arm, reduce potential faults caused by position offset, and improve the overall operation stability and production efficiency of the industrial robot system. In summary, through the efficient work of the comparison unit and the command issuing unit, the preliminary judgment module can not only accurately detect the position offset of the robot arm, but also implement an effective risk assessment and alarm mechanism, provide real-time monitoring and response, further improve the efficiency of troubleshooting, and enhance the reliability of the system.

[0082] Example 4

[0083] Please refer to Figure 1 ,Specifically: the scope tracing module includes a historical tracing analysis unit and a preliminary target locking unit;

[0084] The historical tracing analysis unit is used to obtain the historical probability of failure of each joint in the industrial robot by using the historical detection data set after receiving the tracing command. and point process probability , wherein the historical detection data set includes the number of times the robot position is offset by the corresponding joint during the historical monitoring period , the total number of times Zlc caused by all joints to offset the position of the robot arm during the historical monitoring period, the number of all robot arms set in the corresponding time period And the number of robot arms whose positions are offset by the corresponding joints in each time period The specific acquisition methods are as follows:

[0085] ;

[0086] Where Zlc is the total number of times the robot position is offset by all joints in the historical monitoring period. It is expressed as the number of times the position of the robot arm is offset by the i-th joint in the historical monitoring cycle;

[0087] ;

[0088] Where n represents the number of historical time periods extracted from the historical monitoring cycle, k=1, 2, 3, ..., n, It is expressed as the number of manipulators whose positions are offset by the i-th joint in the k-th time period. Represents the number of all robotic arms set up for the kth time period, where the number of robotic arms refers to the degree of freedom, which describes the number of independent motion capabilities of an object or system in space.

[0089] Based on historical probability and the point process probability of the corresponding joint , calculate the event probability of obtaining the corresponding joint , which is obtained by the following formula:

[0090] ;

[0091] In the formula, It is expressed as the total probability of robot position deviation.

[0092] The preliminary target locking unit is used to determine the event probability of the corresponding joint. The calculation method is to obtain the event probability of each joint respectively. , and the event probability of each joint Sort them in order of value to get the probability of the event with the maximum value. , and the probability of the event with the largest value The corresponding joint is regarded as the initial abnormal joint, and the initial abnormal joint is used as the fault tracing target point, and the tracing range is gradually spread to the surrounding area to determine the fault tracing range.

[0093] It should be noted that the fault tracing range can be determined in the following way: according to the event probability of the corresponding joint , and combined with the statistical mean algorithm, the standard deviation of the event probability of the joints in the industrial robot is obtained. , and based on the standard deviation of the event probability of the joints in the industrial robot , obtain the fault tracing range and mark it as [ Pmax − j*σP,Pmax + j*σP ] ; Where j is a constant, usually ranging from 1 to 3, corresponding to different confidence levels. The specific value is set by the user (based on actual conditions).

[0094] In this embodiment, the historical tracing analysis unit calculates the historical failure probability and point process probability of each joint in the industrial robot by using the historical detection data set. This comprehensive historical data analysis provides a detailed fault history record, which can further accurately assess the failure risk of each joint. The event probability of each joint is calculated using the historical probability and point process probability. This calculation method can quantify the probability of joint failure, provide an accurate risk assessment based on historical data, and help determine the failure possibility of each joint in the current detection cycle, and determine the joint that is most likely to have a failure problem. The preliminary target locking unit can quickly lock the joint with the largest event probability as the initial abnormal joint by sorting the calculation results of the event probability of each joint. This sorting and selection mechanism can effectively identify the joint that is most likely to fail, provide a clear target point for subsequent fault tracing, and improve the accuracy of fault location. After determining the initial abnormal joint, the system can gradually expand the tracing range around the joint based on the joint. This extended tracing strategy can cover the potentially affected area, ensure that potential fault points are not missed, and thus comprehensively analyze and solve the fault problem. By combining historical data and event probability for comprehensive analysis, this module can quickly identify the joints with the greatest risk of failure in a large amount of data. This efficient fault diagnosis method not only speeds up the fault tracing process, but also reduces the possibility of misjudgment and missed judgment, thereby improving the diagnostic efficiency of the overall system.

[0095] Example 5

[0096] Please refer to Figure 1 ,Specifically: the joint analysis module includes a monitoring unit, a tracing analysis unit and a comprehensive analysis unit;

[0097] The monitoring unit is used to monitor the relevant strain data information and relevant presentation status data information of each joint in the industrial robot in real time to construct a monitoring set, wherein the relevant strain data information in the monitoring set includes the hysteresis Hc and joint torque Gnj of each joint;

[0098] The relevant status data information in the monitoring set includes the axial clearance Zxjx, bearing clearance Zcyj and tooth side clearance Ccjx of each joint;

[0099] The tracing analysis unit is used to extract relevant strain data information and relevant presentation status data information of the corresponding joints within the fault tracing range from the monitoring set after determining the fault tracing range, and calculate the fatigue coefficient Pxs of the corresponding joint based on the extracted relevant strain data information:

[0100] ;

[0101] Where m represents the number of joints within the fault tracing range, i=1, 2, 3, ..., m, Hc i Expressed as the hysteresis of the i-th joint, Hc avg It is expressed as the mean hysteresis of all joints within the fault tracing range, Gnj i Expressed as the joint torque of the i-th joint, Gnj avg It is expressed as the mean joint torque of all joints within the fault tracing range, and are weight values, where 0 < ≤1,0< ≤1, and + =1;

[0102] The backlash Hc of each joint mentioned above is also called lost motion error. Angle sensors can be used to measure the angle changes of the input shaft and the output shaft respectively. By comparing the difference in the angle change amplitude of the input shaft and the output shaft when the input shaft rotates in the opposite direction, the lost motion error size can be obtained. The specific formula can be used to obtain it: ; Among them, Hc in Expressed as the theoretical rotation angle of the input shaft, Hc out It is expressed as the actual rotation angle corresponding to the output shaft;

[0103] The joint torque Gnj of each joint can be monitored and obtained through a torque sensor;

[0104] According to the extracted relevant presentation state data information, the mechanical state coefficient Jxs of the corresponding joint is calculated:

[0105] ;

[0106] Wherein, Ccjx represents the tooth side clearance, Zxjx represents the axial clearance, Zcyj represents the bearing clearance, a1, a2 and a3 are all weight values, among which, 0<a1≤1, 0<a2≤1, 0<a3≤1, and a1+a2+a3=1.

[0107] The axial clearance Zxjx of each joint mentioned above refers to the clearance in the direction of the joint axis, which is the axial movement in the joint assembly, that is, the free movement of the joint in the axial direction. Usually, this clearance is caused by looseness or wear between the components in the joint. It can be measured by a displacement sensor or a contact sensor to directly measure the clearance of the axial movement.

[0108] The bearing clearance Zcyj refers to the gap between the rolling element (such as ball or roller) in the bearing and the inner or outer ring of the bearing. It can be measured using an inductive displacement sensor or an optical sensor to accurately detect the internal clearance of the bearing. Excessive bearing clearance may cause unstable movement and increase mechanical vibration and noise.

[0109] Tooth side clearance Ccjx refers to the clearance between the tooth sides of the gears when the gears are meshing. This clearance exists between the tooth sides of the gears and the gear meshing surface. The actual clearance of the tooth sides is obtained by measuring the gear surface in detail using a laser scanner or surface profiler.

[0110] In this embodiment, the monitoring unit can monitor the relevant strain data information and relevant presentation status data information of each joint in the industrial robot in real time. By constructing a monitoring set, the system can obtain multi-dimensional parameters, which is helpful to understand the working status and potential loss of each joint in many aspects, thereby providing detailed basic data for subsequent analysis. The tracing analysis unit extracts relevant data within the fault tracing range from the monitoring set, and uses these data to calculate the fatigue coefficient Pxs of the joint, which can further accurately reflect the fatigue status of the joint and provide a quantitative basis for subsequent fault diagnosis. Using the extracted relevant presentation status data information, the mechanical state coefficient Jxs of the joint is calculated to further evaluate the mechanical state of the joint. This comprehensive evaluation helps to detect abnormal mechanical state and accurately identify potential mechanical problems. The comprehensive analysis unit can more accurately evaluate the overall health status of the joint by combining the calculation results of the fatigue coefficient Pxs and the mechanical state coefficient Jxs. This comprehensive analysis helps to distinguish normal wear from potential faults, thereby improving the accuracy and reliability of fault diagnosis, and further avoiding the misjudgment that may be caused by relying on a single indicator.

[0111] Example 6

[0112] Please refer to Figure 1 Specifically: the comprehensive analysis unit is used to input the fatigue coefficient Pxs and the mechanical state coefficient Jxs of the corresponding joint into the fault prediction model to obtain the risk index Fzs of the corresponding joint:

[0113] ;

[0114] Wherein, wz represents the temperature value, C represents the correction constant, F1, F2 and F3 are all weight values, wherein 0<F1≤1, 0<F2≤1, 0<F3≤1, and F1+F2+F3=1.

[0115] The temperature value wz is monitored and obtained through a temperature sensor.

[0116] Among them, the convolutional neural network technology is used to build the initial model, and the initial model is trained and tested with the relevant strain data information and relevant presentation status data information of the corresponding joints within the fault tracing range. The trained initial model is used as the state recognition model, and the feature information in the state recognition model is obtained respectively. The state recognition model is trained and tested with the obtained feature information, and finally the trained state recognition model is used as the fault prediction model.

[0117] The judgment module includes an analysis unit and a tracing unit;

[0118] The analysis unit is used to compare and analyze the risk index Fzs with the evaluation threshold W to determine the source of the fault in the industrial robot again. The specific analysis content is as follows:

[0119] If the risk index Fzs of the corresponding joint exceeds the evaluation threshold W, it is indicated that the corresponding joint has a fault and is marked as a faulty joint;

[0120] If the risk index Fzs of the corresponding joint does not exceed the evaluation threshold W, it means that the corresponding joint does not have a fault and no marking is performed;

[0121] The tracing unit is used to count the joints marked as faulty, and determine the joint that first fails based on the execution sequence relationship between the faulty joints, and regard the joint as the source of the position deviation of the mechanical arm of the industrial robot.

[0122] In this embodiment, the risk index Fzs can further accurately reflect the actual fault risk of the joint, providing a reliable basis for fault diagnosis. The judgment module can clearly determine whether there is a fault by comparing the risk index Fzs with the evaluation threshold W. If the risk index Fzs exceeds the evaluation threshold W, the joint is marked as a faulty joint; if the risk index Fzs does not exceed the threshold, it is considered that the joint is temporarily free of faults. This clear judgment mechanism can reduce misjudgment, improve the accuracy of fault identification, and ensure the effectiveness of fault handling. The traceability unit is responsible for counting the joints marked as faults, and determining the joint that first fails based on the execution order relationship between the joints. This traceability method can effectively lock the source of the fault and help quickly locate the root cause of the position offset of the robot arm. This precise traceability capability helps to shorten the fault repair time, reduce the downtime of the production line, and improve the operating efficiency of the equipment. Through accurate risk assessment and fault judgment, the comprehensive analysis and judgment module can optimize the maintenance decision-making process. Maintenance personnel can quickly identify and handle faults based on the risk index Fzs and fault markings, reducing the time and cost of traditional fault troubleshooting, and further improving the efficiency and pertinence of maintenance work. Through a systematic fault judgment and tracing mechanism, the fault expansion can be effectively prevented and the stable operation of the industrial robot can be guaranteed. Accurate positioning and timely handling of the source of the fault can help maintain the continuity and stability of the production process, thereby improving the reliability and performance of the overall production line. In summary, the comprehensive analysis unit and judgment module can further improve the performance of the industrial robot fault detection system through accurate risk assessment, effective fault judgment, reliable fault tracing, improved maintenance efficiency and enhanced system stability, providing strong technical support and guarantee for industrial production.

[0123] Example 7

[0124] Please refer to Figure 1 and Figure 2 , specifically: A method for detecting an industrial robot based on machine vision, comprising the following steps:

[0125] Step 1: When the industrial robot is in working state, use visual instruments to capture the position state of the robot arm of the industrial robot, extract continuous image frames from the captured video stream, and perform preprocessing operations;

[0126] Step 2: Obtain the position state data of the robot arm in the preprocessed image frame to determine the position coordinates of the robot arm in the corresponding time period , and based on the position coordinates of the robot arm at the corresponding time period , to analyze the offset displacement Pd, if the offset displacement Pd exceeds the displacement threshold, a traceback command is issued externally;

[0127] Step 3: After receiving the traceability command, use the historical detection data set to obtain the historical probability of failure of each joint in the robot and point process probability , based on historical probability and the point process probability of the corresponding joint , calculate the event probability of obtaining the corresponding joint , according to the event probability of the corresponding joint Size, determine the fault tracing target point and obtain the fault tracing range;

[0128] Step 4: Based on the acquired fault tracing scope, monitor the relevant strain data information and relevant presentation status data information of the corresponding joints within the fault tracing scope, and construct the risk index Fzs based on the relevant strain data information and relevant presentation status data information of the corresponding joints within the fault tracing scope and in combination with the fault prediction model;

[0129] Step 5: Pre-set the evaluation threshold W, and compare and analyze the risk index Fzs with the evaluation threshold W to determine the source of the fault in the industrial robot again.

[0130] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An industrial robot detection system based on machine vision, characterized in that: It includes detection module, preliminary judgment module, range tracing module, joint analysis module and judgment module; The detection module is used to capture the position status of each mechanical arm of the industrial robot by using a visual monitoring instrument when the industrial robot is in a working state, extract continuous image frames from the captured video, and perform preprocessing operations; The initial judgment module is used to obtain the position state data of each robot arm in the preprocessed image frame to determine the position coordinates of each robot arm at the corresponding time point , and according to the position coordinates of each robot arm in the corresponding time period , to analyze the offset displacement Pd, if the offset displacement Pd exceeds the displacement threshold, a traceback command is issued externally; The range tracing module is used to obtain the historical probability of failure of each joint in the industrial robot by using the historical detection data set after receiving the tracing command. and point process probability , based on historical probability and the point process probability of the corresponding joint , calculate the event probability of obtaining the corresponding joint , according to the event probability of the corresponding joint Size, determine the fault tracing target point and obtain the fault tracing range; The scope tracing module includes a historical tracing analysis unit and a preliminary target locking unit; The historical tracing analysis unit is used to obtain the historical probability of failure of each joint in the industrial robot by using the historical detection data set after receiving the tracing command. and point process probability , wherein the historical detection data set includes the number of times the robot position is offset by the corresponding joint during the historical monitoring period , the total number of times Zlc caused by all joints to offset the position of the robot arm during the historical monitoring period, the number of all robot arms set in the corresponding time period And the number of robot arms whose positions are offset by the corresponding joints in each time period The specific acquisition methods are as follows: ； Where Zlc is the total number of times the robot position is offset by all joints in the historical monitoring period. It is expressed as the number of times the position of the robot arm is offset by the i-th joint in the historical monitoring cycle; ； Where n represents the number of historical time periods extracted from the historical monitoring cycle, k=1, 2, 3, ..., n, It is expressed as the number of manipulators whose positions are offset by the i-th joint in the k-th time period. represents the number of all robotic arms set for the kth time period; The joint analysis module is used to monitor the relevant strain data information and relevant presentation status data information of each joint in the industrial robot in real time, and extract the relevant strain data information and relevant presentation status data information of the corresponding joints within the fault tracing range according to the acquired fault tracing range, and construct the risk index Fzs according to the relevant strain data information and relevant presentation status data information of the corresponding joints within the fault tracing range and in combination with the fault prediction model; The judgment module is used to pre-set an evaluation threshold W, and compare and analyze the risk index Fzs with the evaluation threshold W to determine the source of the fault in the industrial robot again.

2. The machine vision-based industrial robot detection system according to claim 1, characterized in that: The detection module includes a capture unit and a pre-processing unit; The capture unit is used to capture the video of the industrial robot in a working state using a visual monitoring instrument, and collect and record relevant continuous image frames therefrom, and then sort the continuous image frames in sequence according to the order in which they were taken; The preprocessing unit is used to convert the image from the RGB color space to the HSV color space, and use filtering technology to remove noise in each image frame through median filtering.

3. The machine vision-based industrial robot detection system according to claim 2, characterized in that: The preliminary judgment module includes a comparison unit and a command issuing unit; The comparison unit is used to extract the position state data of each robot arm according to the processed image frame, and to obtain the position coordinates of each robot arm at the corresponding time period. and compare it with the preset position coordinates in the control program Compare and calculate the offset displacement Pd at the corresponding time point, which can be obtained according to the following formula: ; The command issuing unit is used to compare and analyze the offset displacement Pd obtained by the comparison unit with the displacement threshold to determine whether each mechanical arm of the current industrial robot has a risk of position offset when working. The specific comparison and analysis content is as follows: If the offset displacement Pd at the corresponding time point exceeds the displacement threshold, it is judged that the corresponding mechanical arm of the current industrial robot has the risk of position offset during operation, and a traceback command will be issued externally at this time; If the offset displacement Pd at the corresponding time point does not exceed the displacement threshold, it is determined that the corresponding mechanical arm of the current industrial robot does not have a risk of position offset when working.

4. The machine vision-based industrial robot detection system according to claim 1, characterized in that: Based on historical probability and the point process probability of the corresponding joint , calculate the event probability of obtaining the corresponding joint , which is obtained by the following formula: ； In the formula, It is expressed as the total probability of robot position deviation.

5. The machine vision-based industrial robot detection system according to claim 4, characterized in that: The preliminary target locking unit is used to determine the event probability of the corresponding joint. The calculation method is to obtain the event probability of each joint respectively. , and the event probability of each joint Sort them in order of value to get the probability of the event with the maximum value. , and the probability of the event with the largest value The corresponding joint is regarded as the initial abnormal joint, and the initial abnormal joint is used as the fault tracing target point, and the tracing range is gradually spread to the surrounding area to determine the fault tracing range.

6. The machine vision-based industrial robot detection system according to claim 1, characterized in that: The joint analysis module includes a monitoring unit, a tracing analysis unit and a comprehensive analysis unit; The monitoring unit is used to monitor the relevant strain data information and relevant presentation status data information of each joint in the industrial robot in real time to construct a monitoring set, wherein the relevant strain data information in the monitoring set includes the hysteresis Hc and joint torque Gnj of each joint; The relevant status data information in the monitoring set includes the axial clearance Zxjx, bearing clearance Zcyj and tooth side clearance Ccjx of each joint; The tracing analysis unit is used to extract relevant strain data information and relevant presentation status data information of the corresponding joints within the fault tracing range from the monitoring set after determining the fault tracing range, and calculate the fatigue coefficient Pxs of the corresponding joint based on the extracted relevant strain data information: ； Where m represents the number of joints within the fault tracing range, i=1, 2, 3, ..., m, Hc i Expressed as the hysteresis of the i-th joint, Hc avg It is expressed as the mean hysteresis of all joints within the fault tracing range, Gnj i Expressed as the joint torque of the i-th joint, Gnj avg It is expressed as the mean joint torque of all joints within the fault tracing range, and All are weight values; According to the extracted relevant presentation state data information, the mechanical state coefficient Jxs of the corresponding joint is calculated: ； Where Ccjx represents the tooth side clearance, Zxjx represents the axial clearance, Zcyj represents the bearing clearance, and a1, a2 and a3 are all weighted values.

7. The machine vision-based industrial robot detection system according to claim 6, characterized in that: The comprehensive analysis unit is used to input the fatigue coefficient Pxs and the mechanical state coefficient Jxs of the corresponding joint into the fault prediction model to obtain the risk index Fzs of the corresponding joint: ； Wherein, wz represents the temperature value, C represents the correction constant, and F1, F2 and F3 are all weight values.

8. The machine vision-based industrial robot detection system according to claim 1, characterized in that: The judgment module includes an analysis unit and a tracing unit; The analysis unit is used to compare and analyze the risk index Fzs with the evaluation threshold W to determine the source of the fault in the industrial robot again. The specific analysis content is as follows: If the risk index Fzs of the corresponding joint exceeds the evaluation threshold W, it is indicated that the corresponding joint has a fault condition and is marked as a faulty joint; If the risk index Fzs of the corresponding joint does not exceed the evaluation threshold W, it means that the corresponding joint does not have a fault and no marking is performed; The tracing unit is used to count the joints marked as faulty, and determine the joint that first fails based on the execution sequence relationship between the faulty joints, and regard the joint as the source of the position deviation of the mechanical arm of the industrial robot.

9. A method for detecting an industrial robot based on machine vision, used to implement an industrial robot detection system based on machine vision as claimed in any one of claims 1 to 8, characterized in that: The following steps are included: Step 1: When the industrial robot is in working state, use visual instruments to capture the position state of the robot arm of the industrial robot, extract continuous image frames from the captured video stream, and perform preprocessing operations; Step 2: Obtain the position state data of the robot arm in the preprocessed image frame to determine the position coordinates of the robot arm in the corresponding time period , and based on the position coordinates of the robot arm at the corresponding time period , to analyze the offset displacement Pd, if the offset displacement Pd exceeds the displacement threshold, a traceback command is issued externally; Step 3: After receiving the traceability command, use the historical detection data set to obtain the historical probability of failure of each joint in the robot and point process probability , based on historical probability and the point process probability of the corresponding joint , calculate the event probability of obtaining the corresponding joint , according to the event probability of the corresponding joint Size, determine the fault tracing target point and obtain the fault tracing range; Step 4: Based on the acquired fault tracing scope, monitor the relevant strain data information and relevant presentation status data information of the corresponding joints within the fault tracing scope, and construct the risk index Fzs based on the relevant strain data information and relevant presentation status data information of the corresponding joints within the fault tracing scope and in combination with the fault prediction model; Step 5: Pre-set the evaluation threshold W, and compare and analyze the risk index Fzs with the evaluation threshold W to determine the source of the fault in the industrial robot again.

Images