A belt conveyor roller non-rotation detection system and method based on dense optical flow
Through a detection system combining dense optical flow characteristics and thermal imaging data, the accuracy of roller non-rotation detection in the prior art is solved, and the accurate judgment and fault positioning of the reasons for roller non-rotation are achieved, and belt conveying is ensured. safe operation of the machine.
Patent Information
- Application Number
- CN202210914760.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-08-01
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2042-08-01
AI Technical Summary
The prior art is difficult to accurately detect the reasons why the belt conveyor roller does not rotate. There are safety risks and incompleteness in manual inspections, and robot detection cannot distinguish between the non-contact rollers and the non-turn caused by bearing clamping resistance.
Using a detection system based on dense optical flow, combined with an RFID reader, visible light camera, thermal imaging camera and temperature sensor, the density optical flow characteristic value and thermal imaging data are calculated, and whether the roller is in contact with the conveyor belt is judged and the RFID tag is passed. Positioning the faulty roller.
It has achieved high accuracy detection of the reasons why the roller does not rotate, and timely positioned the fault rollers to ensure the safe operation of the belt conveyor, which has significant economic benefits and application value.
Smart Images

Figure CN115838062B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of belt conveyor roller fault detection, and in particular to a belt conveyor roller non-rotation detection system and method based on dense optical flow. Background Art
[0002] There are many rollers in the belt conveyor. Since the roller is one of the important parts of the conveyor, the maintenance and replacement cost of the roller is also a very important part of the belt conveyor operation cost. The quality of the roller operation has a great impact on the operation resistance, power consumption, service life of the roller and conveyor belt, maintenance workload and transportation cost of the entire conveyor. In particular, if the roller does not rotate, it will greatly increase the risk of conveyor operation.
[0003] There are four main reasons why the rollers do not rotate. One is that the conveyor belt deviates, causing the side rollers of the conveyor belt to not contact the conveyor belt; one is that the conveyor belt is not loaded enough, causing the middle rollers of the conveyor belt to not contact the conveyor belt; one is that the assembly accuracy of the rollers and the bearing seats and the sealing and lubrication problems during use cause the bearings to get stuck; one is that the roller shell is stuck or the roller end face interferes with the roller support. Each problem needs to be discovered and handled in a timely manner.
[0004] There are currently two main ways to detect rollers that are not rotating. One is manual inspection through human eye observation, but the inspection must be carried out when the conveyor is running, which poses a personal safety risk. In addition, it is possible that the inspection can only be carried out on one side of the conveyor, which limits the field of vision. In addition, due to human fatigue factors, it is impossible to fully inspect each roller, and the inspection cannot be guaranteed to be 100% accurate. The other is that the inspection robot uses thermal imaging detection. Although it can detect abnormalities that cause roller heating, it cannot accurately determine whether the heating is caused by the roller not rotating. There is no heating phenomenon for rollers that do not rotate due to lack of contact, and the detection is not comprehensive. There is also a patent that proposes a method for detecting the operating status and wear degree of rollers based on image texture information, but it is often difficult to extract texture features from the surface of the roller, and it is easily affected by the lighting environment. In addition, it is necessary to support robot mobile detection, and the image must be captured during movement, which also increases the difficulty of detecting the operating status of the roller. Summary of the invention
[0005] The purpose of the present invention is to solve the problems existing in the prior art and provide a belt conveyor roller non-rotation detection system based on dense optical flow, comprising:
[0006] Inspection track, installed under the belt conveyor roller group, set along the conveying direction of the conveyor belt;
[0007] An inspection robot is installed on the inspection track and moves back and forth along the inspection track;
[0008] An RFID reader and an RFID tag, wherein the RFID reader is installed on the inspection robot, the number of the RFID tags is equal to the number of the roller groups, and each RFID tag corresponds to one roller group, the RFID tag is used to record the position information of the corresponding roller group, and the RFID reader is used to read the position information recorded by the RFID tag;
[0009] A visible light camera is installed on the inspection robot and is used to collect video images of the roller group in real time;
[0010] A thermal imaging camera is installed on the inspection robot and is used to collect thermal imaging images of the roller group in real time;
[0011] An external temperature sensor, installed on the inspection robot, for detecting the ambient temperature near the location of the inspection robot in real time;
[0012] A computing module is installed on the inspection robot and is respectively connected to the RFID reader / writer, the visible light camera, the thermal imaging camera and the external temperature sensor, and is used to calculate the roller data and determine the roller status in real time;
[0013] An alarm is communicatively connected with the computing module.
[0014] Preferably, the roller data includes an area of the roller in continuous visible light image frames, a continuous dense optical flow, an area of the roller in thermal imaging frames, and a maximum temperature of the roller.
[0015] Preferably, the roller state includes a roller non-contact non-rotating state and a roller stuck non-rotating state.
[0016] The present invention also provides a method for detecting non-rotation of belt conveyor rollers based on dense optical flow using the above system, which is characterized by comprising the following steps:
[0017] S1. During the pre-deployment phase, the inspection robot uses a visible light camera to collect visible light images of each roller group above the inspection track, manually annotates the rollers on each roller group image, organizes them into a sample set of rollers on site, and uses the yolov5 target detection framework to train and obtain a roller detection model;
[0018] S2. During the formal deployment phase, the inspection robot sets a specified inspection speed to perform the inspection task. The visible light camera and thermal imaging camera on the inspection robot collect visible light images and thermal imaging images of the roller group in real time and transmit the signals to the computing module.
[0019] S3, using the yolov5 roller detection model to extract the ROI of the roller group in the visible light image, and determine whether the visible light image contains a complete roller group;
[0020] S4. When the image contains a complete roller group, use OpenCV to extract the dense optical flow of the roller area, calculate the dense optical flow feature value, and record it in consecutive frames; at the same time, extract the maximum temperature of the roller from the thermal imaging image, and read the external temperature sensor data on the inspection robot to obtain the ambient temperature;
[0021] S5. When the continuous frame calculation end condition is reached, the variance of the optical flow eigenvalues of the rollers of the continuous frames is calculated to determine whether it meets the empirical threshold of the roller not rotating. If so, it is determined whether there is a large temperature rise of the roller. If so, it means that the roller is stuck. If not, it means that the roller is not in contact with the conveyor belt.
[0022] S6. When it is detected that the roller does not rotate, the alarm device sounds an alarm according to the RFID tag information corresponding to the current roller group read by the RFID reader.
[0023] Preferably, the dense optical flow feature value calculated in step S4 is the dark spot pixel ratio counted after converting the dense optical flow image into a grayscale image.
[0024] Preferably, the continuous frame calculation end condition in step S5 refers to the images continuously collected by the inspection robot during movement starting from stable detection of a complete roller group until no complete roller group is detected.
[0025] Preferably, the empirical threshold value of the roller not rotating in step S5 refers to the binary classification threshold determined by respectively counting the dense optical flow features of the normally operating rollers and the non-normally operating rollers in S4, through the videos of the rollers operating normally and a series of video samples of the rollers not operating normally collected by the inspection robot in the pre-deployment stage.
[0026] Compared with the prior art, the beneficial effects of the present invention are:
[0027] The present invention aims to accurately discover the phenomenon and cause of roller failure in belt conveyor rollers. The present invention combines the RFID reader and RFID tag on the inspection robot, the external temperature sensor, the visible light camera, the thermal imaging camera, the computing module, and the inspection track to form a set of high-accuracy roller fault detection system. When the roller is detected to be non-rotating, the system can accurately determine whether the roller failure is caused by roller shaft obstruction or lack of contact with the conveyor belt by combining the dense optical flow characteristics of the visible light image, the thermal imaging temperature measurement, and the ambient temperature. The system can also locate the faulty roller in time, give an alarm in time, and remind the operator to take corresponding treatment measures to ensure the safe operation of the belt conveyor. The system has extremely high economic benefits and application value. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Attached Figure 1 It is a flow chart of the detection method of the present invention;
[0029] Attached Figure 2 A top view of the installation of the detection system of the present invention;
[0030] Attached Figure 3 A side view of the installation of the detection system of the present invention;
[0031] Attached Figure 4 This is the visible light image roller detection and optical flow feature fusion map.
[0032] Description of Reference Numerals
[0033] 1-inspection robot, 2-visible light camera, 3-thermal imaging camera, 4-RFID reader / writer, 5-inspection track, 6-roller, 7-external temperature sensor, 8-track connector, 9-conveyor belt, 10-RFID tag. DETAILED DESCRIPTION
[0034] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the application equally.
[0035] Please see attached Figure 2 To Attachment Figure 3The present embodiment provides a belt conveyor roller non-rotation detection system based on dense optical flow, comprising: an inspection track 5, installed below the belt conveyor roller group and arranged along the conveying direction of the conveyor belt 9; an inspection robot 1, installed on the inspection track 5, and reciprocating along the inspection track 5; an RFID reader 4 and an RFID tag 10, the RFID reader 4 being installed on the inspection robot 1, the number of the RFID tags 10 being equal to the number of the roller groups, and each RFID tag 10 corresponding to one roller group, the RFID tag 10 being used to record the position information of the corresponding roller group, and the RFID reader 4 being used to read the information recorded by the RFID tag 10. recorded location information; visible light camera 2, installed on the inspection robot 1, used to collect video images of the roller group in real time; thermal imaging camera 3, installed on the inspection robot 1, used to collect thermal imaging images of the roller group in real time; external temperature sensor 7, installed on the inspection robot 1, used to detect the ambient temperature near the location of the inspection robot 1 in real time; calculation module, installed on the inspection robot 1, respectively communicated with the RFID reader, visible light camera 2, thermal imaging camera 3 and external temperature sensor 7, the calculation module is used to calculate the roller data in real time and judge the roller status in real time; alarm, the alarm is communicated with the calculation module. Among them, the roller data includes the area of the roller in the continuous visible light image frame, the continuous dense optical flow, the area of the roller in the thermal imaging frame, and the maximum temperature of the roller. The roller status includes the non-contact non-rotating state of the roller and the blocked non-rotating state of the roller. The inspection track 5 can be a double track, installed directly below the roller group. The visible light camera 2 and the thermal imaging camera 3 on the inspection robot 1 have sufficient field of view to capture all rollers of the same roller group. The two tracks in the double track can be fixedly connected by a track connector 8, and the RFID tag 10 can be installed on the track connector 8.
[0036] Please see attached Figure 1 This embodiment also provides a method for detecting non-rotation of belt conveyor rollers based on dense optical flow using the above system, comprising the following steps:
[0037] S1. In the pre-deployment stage, the inspection robot 1 uses the visible light camera 2 to collect visible light images of each roller group above the inspection track 5, manually annotates the rollers on each roller group image, organizes them into a roller sample set on site, and uses the yolov5 target detection framework to train to obtain a roller detection model;
[0038] S2, during the formal deployment phase, the inspection robot 1 sets a specified inspection speed to perform the inspection task. The visible light camera 2 and the thermal imaging camera 3 on the inspection robot 1 collect the visible light image and thermal imaging image of the roller group in real time, and transmit the signal to the calculation module;
[0039] S3, using the yolov5 roller detection model to extract the ROI of the roller group in the visible light image, and determine whether the visible light image contains a complete roller group;
[0040] S4. When the image contains a complete roller group, use OpenCV to extract the dense optical flow of the roller area, calculate the dense optical flow feature value, and record it in consecutive frames; at the same time, extract the maximum temperature of the roller from the thermal imaging image, and read the data of the external temperature sensor 7 on the inspection robot 1 to obtain the ambient temperature;
[0041] S5. When the continuous frame calculation end condition is reached, the variance of the optical flow feature value of the roller of the continuous frame is calculated to determine whether it meets the empirical threshold of the roller not rotating. If it meets the threshold, it is determined whether there is a large temperature rise of the roller. If so, it means that the roller is stuck. If not, it means that the roller is not in contact with the conveyor belt 9.
[0042] S6. When it is detected that the rollers do not rotate, the alarm device sounds an alarm according to the information of the RFID tag 10 corresponding to the current roller group read by the RFID reader / writer 4.
[0043] In some embodiments, the dense optical flow feature value calculated in step S4 is the dark spot pixel ratio counted after the dense optical flow image is converted into a grayscale image.
[0044] In some embodiments, the continuous frame calculation end condition in step S5 refers to the images continuously collected by the inspection robot 1 during the movement starting from stable detection of a complete roller group until no complete roller group is detected.
[0045] In some embodiments, the empirical threshold for the roller not rotating in step S5 refers to the binary classification threshold determined by collecting videos of the rollers operating normally and a series of video samples of the rollers not operating normally during the pre-deployment stage by the inspection robot 1, and respectively counting the dense optical flow features of the rollers operating normally and the rollers not operating normally in S4.
[0046] Specifically, the implementation of this method is divided into two stages. The first stage is the pre-deployment of the inspection robot 1, and the second stage is the formal deployment of the inspection robot 1. The pre-deployment of the inspection robot 1 requires the acquisition of two data, one is the acquisition of the roller detection model, and the other is the acquisition of the roller dense feature optical flow feature threshold.
[0047] There are three steps to obtain the roller detection model: the first step is to collect images of all roller groups of the belt conveyor under different conditions through the visible light camera 2 on the inspection robot 1. The environmental conditions mainly include the lighting environment (natural light or artificial light), the dust environment, and the running and stopping status of the belt conveyor; the second step is to clean the images collected by the visible light camera 2, manually annotate (in order to detect the roller contour as accurately as possible and in real time, it is necessary to annotate the rectangular box with the rotation angle), and organize the sample set; the third step is to use the yolov5 target detection framework to train and obtain the roller detection model.
[0048] There are two steps to obtain the threshold of dense optical flow features of rollers. The first step is to collect a series of videos of rollers operating normally and a series of videos of rollers not operating normally through the visible light camera 2 on the inspection robot 1. The second step is to use yolov5 roller detection and OpenCV to process the videos of rollers operating normally and those of rollers not operating normally, respectively, and extract the classification thresholds of dense optical flow features of two types of rollers.
[0049] like Figure 4 The figure shows an image captured by the visible light camera 2 when the robot is stationary. The belt conveyor is in normal operation. In this embodiment, each roller group includes the following Figure 3 Taking the left, middle and right three rollers 6 shown as an example, the left, middle and right three rollers of the roller group can be completely detected, among which the left and middle rollers rotate normally, and the right roller and the conveyor belt 9 are not in contact and do not rotate. When the dense optical flow image is superimposed on the original image, it can be found that the optical flow features on the left, middle and right rollers are in sharp contrast, and when observing continuous frames, the optical flow feature differences when the rollers 6 rotate are also obvious due to the differences in texture or wear on the rollers; it cannot be reflected when the rollers 6 do not rotate; more generally, when the robot moves for detection, the roller rotation speed is much greater than the robot movement speed, and the image optical flow direction caused by the robot movement will be significantly different from the optical flow direction of the roller rotation.
[0050] During the formal deployment stage of the inspection robot 1, the inspection robot 1 completes the detection of the roller status in real time while moving. The belt conveyor speed can be 5m / s, that is, the linear speed of the normal operation of the roller surface. The inspection robot 1 can move forward or backward during inspection, and the moving speed can be set to 0.2m / s. The inspection robot 1 simultaneously collects visible light images and thermal imaging images in real time while moving, and the acquisition frame rate can be 25FPS.
[0051] The computing module installed on the inspection robot 1 executes the yolov5 roller detection process, extracts the ROI of the roller group in the visible light image, and the detection frame rate can exceed 50FPS, and determines whether the ROI of the left, middle, and right rollers 6 is complete. If it is incomplete, the subsequent business logic will not be executed; if it is complete, it means that the robot has entered the area where the roller group can be fully observed, and the dense optical flow extraction of the rollers begins. In order to improve the efficiency of optical flow extraction, the original image will be scaled and only the optical flow of the roller area will be extracted to ensure that the total processing frame rate of the current process is greater than 15FPS. Then, the dark point pixel ratio counted after the dense optical flow image is converted into a grayscale image is used as the dense optical flow feature value of the roller.
[0052] In the area where the roller group is completely observed, the dense optical flow feature values of each roller in continuous frames are recorded. When leaving the observation area, the feature value variance is statistically analyzed and compared with the empirical threshold to determine whether each roller 6 is operating normally.
[0053] The computing module installed on the inspection robot 1 is also executing the process of extracting the roller temperature from the thermal imaging image in parallel. In addition, the ambient temperature around the robot is obtained in real time from the external temperature sensor 7. Since the material of the roller is an excellent heat conductor, when the roller does not rotate due to no contact, its temperature should be close to the ambient temperature because there is no friction with the conveyor belt 9; when the roller does not rotate due to bearing obstruction, it generates heat due to friction with the conveyor belt 9, and its temperature should be higher than the ambient temperature. Combining the results of the roller's non-rotation detected by the visible light image and the roller's temperature detected by the thermal imaging image, it can be further determined whether the roller does not rotate due to bearing obstruction or due to no contact with the conveyor belt 9.
[0054] The RFID reader / writer 4 installed on the inspection robot 1 reads the RFID tag 10 on the track connector 8 of the corresponding roller group, and can accurately locate the roller group currently being inspected. Figure 4 In the process, it is detected that the right roller of the roller group has no contact with the conveyor belt 9 and does not rotate. If the rollers on the same side of multiple roller groups have this abnormality, an alarm will be issued to notify the maintenance personnel that "the right roller of the conveyor roller group is not in contact with the conveyor belt 9, and the conveyor belt 9 is suspected to be deviated to the left. Please deal with it as soon as possible"; if only one roller group is abnormal, an alarm will be issued to notify the maintenance personnel that "the right roller of the conveyor roller group is not in contact with the conveyor belt 9, and the roller frame is suspected to be low. Please deal with it as soon as possible"; in addition, if it is detected that the roller does not rotate due to bearing obstruction, an alarm will be issued to notify the maintenance personnel that "the right roller of the conveyor roller group is stuck, and the roller shaft is suspected to be damaged. Please deal with it as soon as possible".
[0055] When the system and method in this embodiment are used to detect the rollers, combined with the dense optical flow characteristics of the visible light image, thermal imaging temperature measurement, and ambient temperature, it can be accurately determined whether the roller does not rotate due to the obstruction of the roller shaft or the lack of contact with the conveyor belt 9. The faulty roller can also be located in time, the alarm can be issued in time, and the operator can be reminded to take corresponding treatment measures to ensure the safe operation of the belt conveyor. This has extremely high economic benefits and application value.
Claims
1. A belt conveyor roller non-rotation detection method based on dense optical flow, characterized in that: The following steps are involved: S1. During the pre-deployment phase, the inspection robot uses a visible light camera to collect visible light images of each roller group above the inspection track, manually annotates the rollers on each roller group image, organizes them into a sample set of rollers on site, and uses the yolov5 target detection framework to train and obtain a roller detection model; S2. During the formal deployment phase, the inspection robot sets a specified inspection speed to perform the inspection task. The visible light camera and thermal imaging camera on the inspection robot collect visible light images and thermal imaging images of the roller group in real time and transmit the signals to the computing module. S3, using the yolov5 roller detection model to extract the ROI of the roller group in the visible light image, and determine whether the visible light image contains a complete roller group; S4. When the image contains a complete roller group, use OpenCV to extract the dense optical flow of the roller area, calculate the dense optical flow feature value, and record it in consecutive frames; at the same time, extract the maximum temperature of the roller from the thermal imaging image, and read the external temperature sensor data on the inspection robot to obtain the ambient temperature; S5. When the continuous frame calculation end condition is reached, the variance of the optical flow eigenvalues of the rollers of the continuous frames is calculated to determine whether it meets the empirical threshold of the roller not rotating. If so, it is determined whether there is a large temperature rise of the roller. If so, it means that the roller is stuck. If not, it means that the roller is not in contact with the conveyor belt. S6. When it is detected that the roller does not rotate, the alarm sounds an alarm according to the RFID tag information corresponding to the current roller group lock read by the RFID reader.
2. According to claim 1, a belt conveyor roller non-rotation detection method based on dense optical flow is characterized in that: The dense optical flow feature value calculated in step S4 is the dark spot pixel ratio counted after the dense optical flow image is converted into a grayscale image.
3. The method for detecting non-rotation of belt conveyor rollers based on dense optical flow according to claim 1 is characterized in that: The continuous frame calculation end condition in step S5 refers to the images continuously collected by the inspection robot during movement starting from stable detection of a complete roller group until no complete roller group is detected.
4. The method for detecting non-rotation of belt conveyor rollers based on dense optical flow according to claim 1, characterized in that: The empirical threshold of the roller not rotating in step S5 refers to the binary classification threshold determined by collecting videos of the rollers operating normally and a series of video samples of the rollers not operating normally during the pre-deployment stage, and respectively counting the dense optical flow features of the rollers operating normally and the rollers not operating normally in S4.
5. A belt conveyor roller non-rotation detection system based on dense optical flow, which adopts the belt conveyor roller non-rotation detection method based on dense optical flow as described in any one of claims 1 to 4, characterized in that: include: Inspection track, installed under the belt conveyor roller group, set along the conveying direction of the conveyor belt; An inspection robot is installed on the inspection track and moves back and forth along the inspection track; An RFID reader and an RFID tag, wherein the RFID reader is installed on the inspection robot, the number of the RFID tags is equal to the number of the roller groups, and each RFID tag corresponds to one roller group, the RFID tag is used to record the position information of the corresponding roller group, and the RFID reader is used to read the position information recorded by the RFID tag; A visible light camera is installed on the inspection robot and is used to collect video images of the roller group in real time; A thermal imaging camera is installed on the inspection robot and is used to collect thermal imaging images of the roller group in real time; An external temperature sensor, installed on the inspection robot, for detecting the ambient temperature near the location of the inspection robot in real time; A computing module is installed on the inspection robot and is respectively connected to the RFID reader / writer, the visible light camera, the thermal imaging camera and the external temperature sensor, and is used to calculate the roller data and determine the roller status in real time; An alarm is communicatively connected with the computing module.
6. The belt conveyor roller non-rotation detection system based on dense optical flow according to claim 5 is characterized in that: The roller data includes the area of the roller in the continuous visible light image frames, the continuous dense optical flow, the area of the roller in the thermal imaging frame, and the maximum temperature of the roller.
7. The belt conveyor roller non-rotation detection system based on dense optical flow according to claim 6 is characterized in that: The roller states include a roller non-contact non-rotating state and a roller stuck non-rotating state.
