A Belt Deviation Detection Method Based on Deep Learning under Complex Working Conditions

Abstract

The present invention relates to a belt deviation detection method based on deep learning under complex working conditions, comprising the following steps: step 1: selecting a YOLOv5 model, and performing target detection and semantic segmentation simultaneously by adding a segmentation head to the head part; step 2: labeling the collected images through a camera device and putting them into a network for model training; step 3: fitting the straight line equation of the belt edge and the outermost connecting line of the roller according to the trained model and the constructed system; step 4: calculating the distance between the belt centerline and the centerline of the roller according to the fitted straight line equation, thereby establishing a belt deviation detection mathematical model; step 5: the staff gives the deviation standard, and selects a suitable alarm threshold for alarm according to the requirements of different workstations; the distance between the belt centerline and the centerline of the roller is calculated, and a belt deviation judgment model is established, thereby realizing accurate detection of belt deviation and realizing feature extraction of belts and rollers under complex working conditions by using deep learning.

Description

Technical Field

[0001] The present invention relates to the technical field of belt deviation detection, and in particular to a belt deviation detection method based on deep learning under complex working conditions. Background Art

[0002] Belt conveyors are one of the important equipment for coal mine transportation, but they often go off track, resulting in reduced coal mine production efficiency. Existing belt detection methods can work well in some conventional environments, but the actual working environment of belt conveyors is not ideal. There are many interference factors. For example, the vibration caused by wiping dust from the camera surface or on-site noise can cause the camera to deviate slightly, resulting in changes in the ROI area, and there are complex working conditions such as insufficient on-site light. Many belt deviation detection systems based on traditional machine vision are not very stable and are easily affected by the on-site environment.

[0003] Chinese patent publication number CN 114155494 B discloses a deep learning-based method for monitoring conveyor belt deviation. The method uses a general target detection network to detect the linear features of the conveyor belt edge and effectively determine the deviation status. This patent requires a specific annotation method, using the vertices of the target detection frame to calculate the linear equation of the belt edge. Deviation is determined by selecting points on the boundary of the camera's field of view to calculate the distance between the left and right edges of the belt and the boundary of the camera's field of view. The creation of a data set is complex, and the generalization of the model cannot be guaranteed. The established linear equation cannot ensure that the diagonal of the target detection frame coincides exactly with the belt edge. Furthermore, when the camera is slightly offset, the distance from its boundary to the belt edge will also change. The threshold for determining deviation is difficult to determine, and the detection results are inaccurate due to the influence of multiple factors.

[0004] Chinese patent publication number CN 111003446 A discloses a method for detecting belt deviation, comprising the following steps: installing multiple cameras above the belt according to the belt length; setting electronic boundaries and deviation level thresholds for both sides of the belt in the video image of each camera; acquiring the video image of each camera in real time via a network and the RTSP protocol; extracting edges on both sides of the belt in the video image; comparing the extracted edges with pre-set electronic boundaries to obtain a deviation value; comparing the obtained deviation value with a pre-set deviation level threshold to determine whether the belt has deviated and the degree of deviation; achieving real-time intelligent detection of belt deviation, thereby improving detection efficiency and accuracy. However, the method adopted in this patent requires a large amount of historical data as support, and once the model is established, it is not easy to change. Therefore, deviations are prone to occur under complex working conditions, resulting in inaccurate detection.

[0005] The present invention provides a belt deviation detection method based on deep learning under complex working conditions. Under complex working conditions, the distance between the belt centerline and the centerline of the roller is calculated, a belt deviation judgment model is established, and deep learning is used to avoid the impact of the shortcomings of traditional camera devices.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A belt deviation detection method based on deep learning under complex working conditions includes the following steps:

[0008] Step 1: Select the YOLOv5 model and add a segmentation head to the Head part to perform object detection and semantic segmentation simultaneously;

[0009] Step 2: Use the camera to label the captured images and put them into the network for model training. The entire system is divided into an image acquisition module, a belt and roller recognition module, a belt deviation judgment module, and an early warning module to build a detection system.

[0010] Step 3: Fit the straight line equation of the belt edge and the outermost line of the roller based on the trained model and the constructed system;

[0011] Step 4: Calculate the distance between the belt centerline and the roller centerline based on the fitted straight line equation, thereby establishing a mathematical model for belt deviation detection:

[0012]

[0013] Where μ is the actual deviation percentage of the belt, d is the actual deviation distance of the belt, and d max The maximum distance between the center line of the belt's actual position and the center line of the left and right rollers;

[0014] Step 5: The staff will set the deviation standard and select the appropriate alarm threshold according to the requirements of different workstations.

[0015] Furthermore, the step 1 adopts an improved YOLOv5 model, and realizes target detection and semantic segmentation at the same time by adding a segmentation head in the Head part. The collected original picture is put into the model. The trained model will search for the pixel points of the belt and roller, and when outputting, the belt, roller and background are covered with masks of different colors for easy distinction. At the same time, the rectangular box of the target detection of the belt and roller is also output.

[0016] Furthermore, in step three, the model is segmented and inferred, and a segmentation mask distribution is generated according to each identified category, with the coordinates of each pixel point. A rectangular coordinate system is established with the point in the upper left corner of the image as the origin, the horizontal direction to the right as the y-axis, and the vertical direction downward as the x-axis. The segmentation mask distribution map is regarded as a two-dimensional array, and the coordinates of the two points on the line connecting the left and right edges of the belt and the outermost side of the roller are obtained by indexing the array, thereby obtaining the equation of the straight line connecting the left and right edges and the outermost side of the roller.

[0017] Furthermore, when the belt does not deviate, the center line of the belt coincides with the center line of the roller. When the belt deviates, a certain distance will be generated between the center line of the belt and the center line of the roller. The percentage of the actual deviation distance of the belt to the maximum deviation distance is calculated, which is the mathematical model for detecting belt deviation. When only one side of the roller is detected, it can be directly judged as serious deviation. The number of rollers on both sides is judged. If the number of rollers on the left side is 0, the belt is seriously deviated to the left; if the number of rollers on the right side is 0, the belt is seriously deviated to the right.

[0018] Furthermore, the deviation standard is defined as a normal acceptable range for a belt deviation percentage of 0-20%, a mild deviation for a belt deviation of 20%-50%, and a severe deviation for a belt deviation of 50%-100%.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] 1) Calculate the distance between the belt centerline and the roller centerline, establish a belt deviation judgment model, and achieve accurate detection of belt deviation;

[0021] 2) Deep learning is used to extract the characteristics of belt conveyors. Compared with traditional camera devices, the impact is less in complex working conditions and the belts and rollers can still be extracted. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic flow chart of the detection method of the present invention.

[0023] Figure 2 It is a schematic diagram of the system functional module structure.

[0024] Figure 3 Schematic diagram of the segmentation mask distribution of the present invention.

[0025] Figure 4 It is a schematic diagram of the mathematical model for deviation detection according to the present invention. DETAILED DESCRIPTION

[0026] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings:

[0027] See Figure 1, is a flow chart of the detection method of the present invention. The present invention provides a belt deviation detection method based on deep learning under complex working conditions, comprising the following steps:

[0028] Step 1: Select the YOLOv5 model and add a segmentation head to the Head part to perform object detection and semantic segmentation simultaneously;

[0029] Step 2: Use the camera to label the captured images and put them into the network for model training. The entire system is divided into an image acquisition module, a belt and roller recognition module, a belt deviation judgment module, and an early warning module to build a detection system.

[0030] Step 3: Fit the straight line equation of the belt edge and the outermost line of the roller based on the trained model and the constructed system;

[0031] Step 4: Calculate the distance between the belt centerline and the roller centerline based on the fitted straight line equation, thereby establishing a mathematical model for belt deviation detection:

[0032]

[0033] Where μ is the actual deviation percentage of the belt, d is the actual deviation distance of the belt, and d max The maximum distance between the center line of the belt's actual position and the center line of the left and right rollers;

[0034] Step 5: The staff will set the deviation standard and select the appropriate alarm threshold according to the requirements of different workstations.

[0035] Furthermore, the step 1 adopts an improved YOLOv5 model and divides the model into three parts: Backbone, Neck and Head. The Backbone part still uses the backbone network of YOLOv5s_v6.0, mainly including three structures: CBS, C3 and SPPF. The CBS layer is composed of Conv+BN+Silu activation function; YOLOv5s_v6.0 uses C3 structure instead of the original BottleneckCSP structure, making the model more lightweight, improving the model learning ability while saving computing resources; SPPF structure only specifies one convolution kernel, and the output after each pooling will become the input of the next pooling; Neck part, uses bottom-up PAN combined with top-down FAN; Head part, uses YOLOv3 detection head, and introduces a segmentation head at the same time, and uses YOLOv5 feature extraction network for semantic segmentation, and uses semantic segmentation to segment the shapes of belts and rollers in the field of view, and uses target detection to determine the number of rollers.

[0036] See Figure 2Furthermore, in the step 2, the image acquisition module uses an on-site camera device to capture images of the belt during operation; the belt and roller recognition module uses a trained model to extract the belt and roller in the current state; the belt deviation judgment module uses an established mathematical model to judge the current running state of the belt; then the current running state of the belt is displayed in the early warning module to issue an early warning of the belt deviation; the camera device is called to obtain the belt running video under complex working conditions, and the video is cut into pictures as a data set, and the data set is randomly divided into a training set and a verification set in a ratio of 9:1, and the labeled pictures are placed in the network for training.

[0037] See Figure 3 Furthermore, in step 3, the model is segmented and inferred, and a segmentation mask distribution is generated according to each identified category. In the model segmentation inference stage, a mask distribution map corresponding to the image size is generated according to the identified belts, rollers, and background. Figure 3 In the image, the irregular, clumping area in the center represents the rollers, the stepped, triangular area represents the belt, and the rest is the background. This mask distribution map is actually a two-dimensional array with elements "0," "1," and "2," each representing a pixel in the image. Each pixel's coordinates are included, using a rectangular coordinate system with the top-left corner of the image as the origin, the y-axis extending horizontally to the right, and the x-axis extending vertically downward. The segmentation mask distribution map is considered a two-dimensional array. By indexing the array, the coordinates of two points on the line connecting the left and right edges of the belt and the outermost rollers can be determined. From this, the equation of the line connecting the left and right edges and the outermost rollers can be derived.

[0038] See Figure 4 Furthermore, according to the coordinates of each pixel point, the linear equations of the left and right edge straight lines Bl and Br of the belt and the outermost straight lines Rl and Rr of the rollers on both sides can be fitted. A point is selected on each of the straight lines Rl and Rr, and its x-axis coordinates must be equal. Then the coordinates of the two points are: Rl1(x Rl1 ,y Rl1 )、Rr1(x Rr1 ,y Rr1 ), where x Rl1 =x Rr1 , then the distance between the straight lines Rl and Rr at these two points is:

[0039] DR|=|y Rr1 -y Rl1 | (2)

[0040] According to the midpoint of the roller, the midline Rm of the distance between the left and right rollers can be obtained. According to the above method, two points Bl1 (x Bl1 ,y Bl1 )、Br1(xBr1 ,y Br1 ), where x Bl1 =x Br1 , then the distance between the straight lines Bl and Br at these two points is:

[0041] DB|=|y Br1 -y Bl1 | (3)

[0042] From this, we can get the center line Bm of the actual position of the belt; the intersection of the straight line Bm and the line segment DB is Bm2(x Bm2 ,y Bm2 ), and the intersection with line segment DR is Bm1(x Bm1 ,y Bm1 ), the intersection of straight line Rm and line segment DB is Rm2(x Rm2 ,y Rm2 ), and the intersection with line segment DR is Rm1(x Rm1 ,y Rm1 );

[0043] When the belt does not deviate, the straight line Bm coincides with the straight line Rm. When the belt deviates, the distance D1 from point Rl1 to straight line Rm is the distance from point Rl1 to point Rm1. Then:

[0044]

[0045] The distance D2 from Rl1 to the straight line Bm is the distance from point Rl1 to point Bm1, then:

[0046]

[0047] So the distance between line Bm and line Rm is:

[0048] d=|D1-D2| (6)

[0049] From this, the actual deviation distance of the belt can be obtained;

[0050] When the belt edge coincides with the connecting line outside the roller, the belt deviation reaches its maximum value. When the straight line Bl coincides with the straight line Rl, D 2min The calculation steps are the same as above, then the maximum distance between straight line Bm and straight line Rm is:

[0051] d max =|D1-D 2min | (7)

[0052] The actual deviation percentage of the belt can be obtained as follows:

[0053]

[0054] When the belt deviates to the left, D1>D2, then d>0; when the belt deviates to the right, D1<D2, then d<0; when only one side of the roller is detected, it can be directly judged as serious deviation, and the number of rollers on both sides is judged. If the number of rollers on the left is 0, the belt is seriously deviated to the left; if the number of rollers on the right is 0, the belt is seriously deviated to the right.

[0055] Furthermore, the deviation standard is defined as a normal acceptable range for a belt deviation percentage of 0-20%, a mild deviation for a belt deviation of 20%-50%, and a severe deviation for a belt deviation of 50%-100%.

[0056] The above embodiments are implemented under the premise of the technical solution of the present invention, and detailed implementation methods and specific operation processes are given, but the protection scope of the present invention is not limited to the above embodiments. The methods used in the above embodiments are conventional methods unless otherwise specified.

Claims

1. A belt deviation detection method based on deep learning under complex working conditions, characterized in that: The following steps are involved: Step 1: Select the YOLOv5 model and add a segmentation head to the head part to perform target detection and semantic segmentation simultaneously. By adding a segmentation head to the head part, target detection and semantic segmentation are achieved simultaneously. The collected original image is put into the model. The trained model will search for the pixels of the belt and roller, and when outputting, the belt, roller, and background are covered with masks of different colors for easy distinction. At the same time, the model will also output the rectangular boxes of the belt and roller target detection; Step 2: Use the camera to label the captured images and put them into the network for model training. The entire system is divided into an image acquisition module, a belt and roller recognition module, a belt deviation judgment module, and an early warning module to build a detection system. Step 3: Fit the straight line equation of the belt edge and the outermost connecting line of the roller based on the trained model and the constructed system, perform segmentation reasoning on the model, and generate a segmentation mask distribution based on each identified category, with the coordinates of each pixel point. A rectangular coordinate system is established with the point in the upper left corner of the image as the origin, the horizontal rightward direction as the y-axis, and the vertical downward direction as the x-axis. The segmentation mask distribution map is regarded as a two-dimensional array. The coordinates of the two points on the connecting line between the left and right edges of the belt and the outermost connecting line of the roller are obtained by indexing the array. From this, the straight line equation of the connecting line between the left and right edges and the outermost connecting line of the roller can be obtained. Step 4: Calculate the distance between the belt centerline and the roller centerline based on the fitted straight line equation, thereby establishing a mathematical model for belt deviation detection: Among them, μ is the actual deviation percentage of the belt, d is the actual deviation distance of the belt, d max The maximum distance between the center line of the belt's actual position and the center line of the left and right rollers; Step 5: The staff will set the deviation standard and select the appropriate alarm threshold according to the requirements of different workstations.

2. The belt deviation detection method based on deep learning under complex working conditions according to claim 1 is characterized in that: When the belt does not deviate, the center line of the belt coincides with the center line of the roller. When the belt deviates, a certain distance will be generated between the center line of the belt and the center line of the roller. The percentage of the actual deviation distance of the belt to the maximum deviation distance is obtained, which is the belt deviation detection mathematical model. When only one side of the roller is detected, it can be directly judged as serious deviation. The number of rollers on both sides is judged. If the number of rollers on the left side is 0, the belt is seriously deviated to the left; if the number of rollers on the right side is 0, the belt is seriously deviated to the right.

3. The belt deviation detection method based on deep learning under complex working conditions according to claim 1 is characterized in that: The deviation standard is defined as a belt deviation percentage of 0-20% is a normal acceptable range, a belt deviation of 20%-50% is a mild deviation, and a belt deviation of 50%-100% is a severe deviation state.

Images