A belt deviation detection method based on visible light infrared image fusion

Abstract

The application relates to a belt deviation detection method based on visible light infrared image fusion, which comprises the following steps: S1, infrared cameras and visible light cameras acquire images respectively; S2, image denoising; S3, image gray scaling; S4, a convolutional neural network extracts image features; S5, L1 regularization extracts feature norms to obtain a two-dimensional weight value mapping; S6, a weighted average fusion strategy is used to obtain a weight map of image fusion, and source infrared gray images and source visible light gray images are weighted and fused to obtain a fusion image; S7, a target detection algorithm is used to detect a belt roller exposed to a belt; and S8, the length of the belt roller exposed to the belt is calculated. The application has the advantages of simple use, and only needs to collect videos of a belt conveying area through visible light cameras and infrared cameras carried by a patrol robot, so that whether the belt deviates or not can be detected through the belt roller.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of coal mine monitoring, in particular to a belt deviation detection method based on visible light infrared image fusion. BACKGROUND

[0002] In the coal mine production system, the coal conveying belt machine is a widely used coal conveying equipment. In daily production activities, the belt on the coal conveying belt machine will deviate due to reasons such as adhesion of coal material to the belt roller, belt relaxation, uneven distribution of coal material, etc., which will cause serious production safety accidents and huge economic losses.

[0003] The belt deviation detection can be realized by identifying the length of the roller through the inspection robot, which can greatly reduce the occurrence of accidents. However, due to the poor underground environment, the inspection robot often cannot detect the belt roller under insufficient light, resulting in low detection efficiency of the belt deviation. SUMMARY

[0004] The purpose of the present application is to provide a belt deviation detection method based on visible light infrared image fusion to solve the problem of low detection efficiency of the existing inspection robot for detecting the belt deviation of the coal conveying belt machine in a poor environment.

[0005] In order to solve the above problems, the technical scheme adopted by the present application is:

[0006] The belt deviation detection method based on visible light infrared image fusion of the present application comprises the following steps:

[0007] S1. Infrared camera and visible light camera acquire images: the infrared camera and the visible light camera acquire images from the lateral side of the coal conveying belt area, the image acquired by the infrared camera is an infrared image, and the image acquired by the visible light camera is a visible light image;

[0008] S2. Image denoising: the infrared image and the visible light image in step S1 are subjected to size transformation to obtain images of the same size, and the noise in the infrared image and the visible light image after size transformation is filtered out through Gaussian filtering;

[0009] S3. Image graying: the infrared image and the visible light image after noise removal are subjected to graying processing respectively to obtain a source infrared gray image and a source visible light gray image;

[0010] S4. Convolutional neural network extracts image features: the source infrared gray image and the source visible light gray image are input into the convolutional neural network, and the image features F1 of the source infrared gray image and the image features F2 of the source visible light gray image are extracted through the convolutional neural network;

[0011] S5. L1 regularization extracts feature norm, and obtains two-dimensional weight mapping: the feature norm of the image features of the source infrared grayscale image and the source visible light grayscale image is solved respectively through L1 regularization, and the corresponding two-dimensional weight mapping of the image features is obtained respectively;

[0012] S6. The weighted average fusion strategy is adopted to obtain the weight map of image fusion, and the source infrared grayscale image and the source visible light grayscale image are weighted fused according to the weight map, so as to obtain the fusion image fused with the source infrared grayscale image and the source visible light grayscale image;

[0013] S7. The target detection algorithm is used to detect the exposed belt roller: the fusion image is input into the target detection network to obtain the optimal prediction box of the exposed belt roller;

[0014] S8. Calculate the length of the exposed belt roller: according to the target box of the exposed belt roller detected by the target detection network, the length of the exposed belt roller is calculated.

[0015] In step S3, the specific calculation method of the gray processing is:

[0016] Gray=R*0.299+G*0.587+B*0.114

[0017] Wherein, Gray represents the gray value of the corresponding gray image, R is the red channel component in the RGB color image before gray processing, G is the green channel component in the RGB color image before gray processing, and B is the blue channel component in the RGB color image before gray processing.

[0018] The specific steps of step S4 are:

[0019] The source infrared grayscale image and the source visible light grayscale image are input into the convolutional neural network VGG16 respectively, the image features are extracted through the convolutional layer in the network, and finally the activation function ReLU is used for processing, to obtain the image features F1 and F2 respectively; the convolutional layer calculation method of VGG16 for extracting image features is:

[0020]

[0021] In the formula: u l is the output of the lth convolutional layer, u l-1 is the output of the l-1 convolutional layer, w l is the convolution kernel, N is the receptive field of the input layer, a l is the bias parameter of the convolutional network, represents the convolution operation, and f is the activation function;

[0022] The calculation method of using the activation function ReLU for processing to obtain the image feature F is:

[0023]

[0024] The image features F1 and F2 are obtained by using the formula.

[0025] In step S5, the calculation process of the two-dimensional weight map of the image features F1 and F2 is obtained by L1 regularization, and is as follows:

[0026]

[0027] wherein, represents the feature norm of the image in the (x, y) coordinate system, i∈{1, 2}, i = 1 represents the infrared image, i = 2 represents the visible light image, p represents the size of the sliding window, m represents the length of the sliding window, n represents the width of the sliding window, F i (m, n) represents the image feature value in the interactive window.

[0028] In step S6, the weight map of image fusion and the calculation method of image fusion are as follows:

[0029]

[0030]

[0031] wherein, represents the weight map of the image in the (x, y) coordinate system, fused(x, y) represents the fused image obtained by fusing the source infrared gray image and the source visible light gray image, G i (x, y) represents the source gray image, i = 1 represents the source infrared gray image, and i = 2 represents the source visible light gray image.

[0032] In step S7, the specific process is as follows:

[0033] The fused image is input into the Yolov5s target detection network, the CSP module in the network is used to extract the roller feature map, the PAN-Net is used to fuse the roller feature map, then three pre-frames are generated for each pixel point in the roller feature map, and it is judged whether there is a roller target in the three pre-frames, the position of the pre-frame is optimized by using the GIoU loss function, the target class in the pre-frame is optimized by using the cross entropy loss function, and thus the optimal prediction frame of the roller is obtained.

[0034] The calculation method of optimizing the pre-frame by using the GIoU loss function is as follows:

[0035]

[0036] ​​In the formula: A and B are two arbitrary frames in the image, C represents the minimum area surrounding A and B, and IoU represents the intersection over union of the real frame and the predicted frame;

[0037] The calculation method of the cross-entropy loss function for optimizing the target class in the pre-frame is:

[0038]

[0039]

[0040] In the formula: N represents the total number of target classes, x i is the predicted value of the target class output by the neural network, y i is the predicted probability of the target class, represents the true value of the target class, and L class is the classification loss.

[0041] When the infrared camera and the visible light camera respectively capture images from the right side above the left side area of the coal conveying belt, the specific steps in step S8 are as follows:

[0042] According to the target frame of the exposed belt roller detected by the target detection network, the length L of the exposed belt roller is calculated. First, it is judged whether the length L is within the limited area. If it is within the limited area, it means that the belt is not deviated. If it is not within the limited area, it is judged whether the length L is greater than the maximum value of the limited area or less than the minimum value of the limited area. If it is greater than the maximum value of the limited area, it means that the belt is deviated to the right. If it is less than the minimum value of the limited area, it means that the belt is deviated to the left.

[0043] When the infrared camera and the visible light camera respectively capture images from the left side above the right side area of the coal conveying belt, the specific steps in step S8 are as follows:

[0044] According to the target frame of the exposed belt roller detected by the target detection network, the length L of the exposed belt roller is calculated. First, it is judged whether the length L is within the limited area. If it is within the limited area, it means that the belt is not deviated. If it is not within the limited area, it is judged whether the length L is greater than the maximum value of the limited area or less than the minimum value of the limited area. If it is greater than the maximum value of the limited area, it means that the belt is deviated to the left. If it is less than the minimum value of the limited area, it means that the belt is deviated to the right.

[0045] According to the above technical scheme, the present application has the following advantages:

[0046] 1. The present application is simple to use. Only the visible light camera and the infrared camera carried by the inspection robot are needed to capture video of the belt conveying area. The belt deviation can be detected by the belt roller.

[0047] 2. The application adopts a visible light image and infrared image fusion mode, which can effectively identify the exposed belt roller under poor lighting conditions.

[0048] 3. The application adopts a convolutional neural network feature extraction mode, which can extract more features in the image, improve the quality of the fused image, and facilitate the detection of the roller.

[0049] 4. The application uses a yolov5s target detection network, which has high detection accuracy while achieving fast detection, and can detect belt deviation in a timely manner. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 The application is a schematic diagram of the application scenario.

[0051] Figure 2 The application is a schematic diagram of the detection process.

[0052] Figure 3 The application is a schematic diagram of the length detection of the exposed belt roller.

[0053] REFERENCE SIGNS

[0054] 1, inspection robot; 2, visible light camera; 3, infrared camera; 4, left side roller of the belt; 5, belt; 6, roller target frame. DETAILED DESCRIPTION

[0055] The application will be further described in detail below in conjunction with the embodiments.

[0056] A belt deviation detection method based on visible light infrared image fusion, first uses infrared camera 3 and visible light camera 2 to acquire images of the belt transport area from the side of the belt, then uses image processing algorithms to denoise and grayscale the dual-source images, uses a convolutional neural network to extract features of the dual-source images, calculates the norm and weight map of the image features using L1 regularization, and obtains a fused image by weighting the dual-source images through the weight map, then uses a target detection algorithm to detect the roller that exposes the belt 5 in the fused image and calculates the length, and finally determines whether the belt 5 is deviated by the length of the roller that exposes the belt 5.

[0057] Embodiment 1

[0058] In this embodiment, the infrared camera and the visible light camera respectively acquire images of the left side area of the coal belt from the right side and above, and determine whether the belt is deviated by detecting the length of the left side roller that exposes the belt.

[0059] In this embodiment, as shown in Figure 2As shown, a belt deviation detection method based on visible light infrared image fusion of the present application includes the following specific steps:

[0060] S1. As Figure 1 As shown, the infrared camera 3 and the visible light camera 2 are respectively used to collect images from the lateral side of the coal conveying belt area. The image collected by the infrared camera 3 is an infrared image, and the image collected by the visible light camera 2 is a visible light image. Specifically, the infrared camera 3 and the visible light camera 2 are mounted on the inspection robot 1. In this embodiment, the infrared camera 3 and the visible light camera 2 collect images from the right side and above the coal conveying belt area. Preferably, the infrared camera 3 and the visible light camera 2 collect images from the right side and above the coal conveying belt area at a 60° direction. This embodiment is used to collect images of the area of the left side roller 4 of the belt.

[0061] S2. The infrared image and the visible light image in step S1 are size transformed to obtain images of equal size (for neural network operation), and the noise in the size-transformed infrared image and visible light image is filtered out by Gaussian filtering. In this embodiment, the image collected by the visible light camera 2 and the 1920*1080 resolution image collected by the infrared camera 3 are transformed into 608*608 resolution images of equal size for neural network calculation.

[0062] S3. The noise-removed infrared image and visible light image are respectively subjected to grayscale processing to obtain a source infrared grayscale image and a source visible light grayscale image. In step S3, the specific calculation method of grayscale processing is:

[0063] Gray = R x 0.299 + G x 0.587 + B x 0.114

[0064] Wherein, Gray represents the gray value of the corresponding grayscale image, R is the red channel component in the RGB color image before grayscale processing, G is the green channel component in the RGB color image before grayscale processing, and B is the blue channel component in the RGB color image before grayscale processing.

[0065] S4. The source infrared grayscale image and the source visible light grayscale image (here: the size of the source infrared grayscale image is equal to the size of the source visible light grayscale image) are respectively input into a convolutional neural network, and the image features F1 of the source infrared grayscale image and the image features F2 of the source visible light grayscale image are extracted by the convolutional neural network.

[0066] Wherein, the specific steps of step S4 are:

[0067] The source infrared gray image and the source visible light gray image are respectively input into the convolutional neural network VGG16, the image features are extracted through the convolutional layers in the network, and finally the image features F1 of the source infrared gray image and the image features F2 of the source visible light gray image are obtained through the processing of the activation function ReLU.

[0068]

[0069] In the formula, u l is the output of the lth convolutional layer, u l-1 is the output of the l-1th convolutional layer, w l is the convolution kernel, N is the receptive field of the input layer, a l is the bias parameter of the convolutional network, represents the convolution operation, and f is the activation function.

[0070] The calculation method of the image features F obtained through the processing of the activation function ReLU is as follows.

[0071]

[0072] The image features F1 and F2 are obtained through the formula.

[0073] In the embodiment, the convolutional neural network VGG16 includes 13 convolutional layers, 5 pooling layers and 3 fully connected layers, and the feature extraction of the corresponding source gray image is mainly realized through the 13 convolutional layers in the step. After the source infrared gray image and the source visible light gray image are input into the VGG16 network, the bottom layer features of the image are extracted through the shallow convolution, the high layer features of the image are extracted through the deep convolution, and finally the image features F1 and F2 are obtained through the ReLU function.

[0074] S5. The feature norms of the image features of the source infrared gray image and the source visible light gray image are respectively solved through L1 regularization, and the two-dimensional weight maps of the corresponding image features are respectively obtained.

[0075] In step S5, the calculation process of the two-dimensional weight maps of the image features F1 and F2 obtained through the L1 regularization is as follows.

[0076]

[0077] In the formula, F represents the feature norm of the image in the (x, y) coordinate system, i∈{1, 2}, i=1 represents the infrared image, i=2 represents the visible light image, p represents the size of the sliding window, m represents the length of the sliding window, n represents the width of the sliding window, and F i(m, n) represents the image feature value under the interaction window.

[0078] S6. Obtain the weight map of image fusion by using the weighted average fusion strategy, and perform weighted fusion on the source infrared gray image and the source visible light gray image according to the weight map to obtain a fusion image fused from the source infrared gray image and the source visible light gray image.

[0079] In step S6, the weight map of image fusion and the calculation method of image fusion are as follows:

[0080]

[0081]

[0082] wherein, represents the weight map of image fusion, fused (x, y) represents the fusion image fused from the source infrared gray image and the source visible light gray image, G i (x, y) represents the source gray image, i = 1 represents the source infrared gray image, and i = 2 represents the source visible light gray image.

[0083] S7. Detect the roller exposing the belt 5 by a target detection algorithm: input the fusion image into a target detection network to obtain the optimal prediction box of the roller exposing the belt 5.

[0084] In step S7, the specific process is as follows:

[0085] Input the fusion image into the Yolov5s target detection network, extract the roller feature map through the CSP module in the network, fuse the roller feature map through the PAN-Net, then generate three pre-selected boxes for each pixel point in the roller feature map and judge whether there is a roller target in the three pre-selected boxes, optimize the position of the pre-selected box through the GIoU loss function, and optimize the target class in the pre-selected box through the cross entropy loss function, so as to obtain the optimal prediction box of the roller.

[0086] The calculation method of optimizing the pre-selected box by the GIoU loss function is as follows:

[0087]

[0088] In the formula, A and B are two arbitrary frames in the image, C represents the minimum area enclosing A and B, and IoU represents the intersection over union of the real frame and the prediction frame.

[0089] The calculation method of optimizing the target class in the pre-selected box by the cross entropy loss function is as follows:

[0090]

[0091]

[0092] wherein N represents the total number of target categories, x i is the predicted value of the target category output by the neural network, y i is the predicted probability of the target category, represents the true value of the target category, L class is the classification loss.

[0093] In the embodiment, the fused image is input into a Yolov5 target detection network, and the roller feature map is extracted through the convolution network in the CSP module in the network. In order to make the roller feature map contain both high-level features and low-level features of the roller, the PAN-Net is used to perform top-down upsampling on the feature map output by the CSP, so as to realize fusion of the feature map. Then, three pre-frames are generated for each pixel point in the feature map, the position of the pre-frame is optimized through the GIoU loss function, and when GIoU<0.01, the current pre-frame is the optimal pre-frame; the target category in the pre-frame is optimized through the cross-entropy loss function, and when L class <0.05, the current predicted category is the optimal predicted category, and at this time, the optimal prediction frame of the roller can be obtained. The optimal prediction frame is the roller target frame 6 (see Figure 3 ) exposing the belt 5.

[0094] S8. Calculate the length of the roller exposing the belt 5: calculate the length of the roller exposing the belt 5 according to the roller target frame 6 of the belt-exposed roller detected by the target detection network.

[0095] Specifically, the step S8 comprises the following steps:

[0096] According to the roller target frame of the belt-exposed roller detected by the target detection network, the length L of the roller exposing the belt 5 is calculated. First, it is judged whether the length L is within the limited range. If it is within the limited range, it means that the belt 5 is not deviated. If it is not within the limited range, it is judged whether the length L is greater than the maximum value of the limited range or less than the minimum value of the limited range. If it is greater than the maximum value of the limited range, it means that the belt 5 is deviated to the right. If it is less than the minimum value of the limited range, it means that the belt is deviated to the left. The embodiment detects the belt deviation by the length of the belt-exposed roller on the left side of the belt 5. As shown in Figure 3 , the belt-exposed roller on the left side of the belt 5 has part of the belt exposed, and the target detection algorithm can obtain the roller target frame 6 of the exposed roller. By judging whether the target frame height is within the set range, it can be detected whether the belt is deviated.

[0097] In the embodiment, the length of the exposed belt of the roller is 10 cm without belt deviation, and the length interval of the exposed belt of the roller is [9, 11] because a slight deviation is allowed in actual production. The height of the target frame of the roller detected by the target detection network is the length L of the exposed belt of the roller, and when L is greater than 11 cm, the belt deviates to the right; and when L is less than 9 cm, the belt deviates to the left.

[0098] The application has simple use mode, and can detect whether the belt deviates through the belt roller only by video acquisition of the belt conveying area through the visible light camera and the infrared camera carried by the inspection robot; the application can effectively identify the exposed belt roller in the case of poor light condition through the visible light image and infrared image fusion mode; the application can extract more features in the image through the convolutional neural network feature extraction mode, can improve the quality of the fused image, and is beneficial to the detection of the roller; the application uses the yolov5s target detection network, can realize rapid detection while having high detection precision, and can detect the belt deviation in time.

[0099] Embodiment 2

[0100] In the embodiment, the infrared camera and the visible light camera respectively acquire images of the right side area of the coal conveying belt from the left side top, and whether the belt deviates is judged by detecting the length of the exposed belt of the right side roller. In the embodiment, the infrared camera 3 and the visible light camera 2 acquire images of the coal conveying belt area from the left side top of the belt 5, preferably, the infrared camera 3 and the visible light camera 2 acquire images of the right side area of the coal conveying belt from the left side top of the belt 5 in the direction of 60°. The embodiment is used for acquiring images of the area of the right side roller 4 of the belt.

[0101] The specific steps in step S8 are:

[0102] The length L of the exposed belt of the roller is calculated according to the target frame of the exposed belt of the roller detected by the target detection network. First, it is judged whether the length L is in the limited area. If it is in the limited area, it means that the belt is not deviated. If it is not in the limited area, it is judged whether the length L is greater than the maximum value of the limited area or less than the minimum value of the limited area. If it is greater than the maximum value of the limited area, it means that the belt is deviated to the left. If it is less than the minimum value of the limited area, it means that the belt is deviated to the right. In this embodiment, the length of the exposed belt 5 of the roller on the right side of the belt is detected to detect the belt deviation. In this embodiment, the length of the exposed belt of the roller is 10 cm when the belt is not deviated. Since a slight deviation is allowed in actual production, the length interval of the exposed belt of the roller is [9, 11]. The height of the target frame of the roller detected by the target detection network is the length L of the exposed belt of the roller. When L is greater than 11 cm, the belt is deviated to the left. When L is less than 9 cm, the belt is deviated to the right. In this embodiment, the other steps of the belt deviation detection method based on visible and infrared image fusion of the application are the same as the related steps in embodiment 1, and will not be described here.

Claims

1. A method for detecting belt deviation based on visible-infrared image fusion, characterized in that, Comprise the following steps: S1. Infrared camera, visible light camera acquires image respectively: through infrared camera and visible light camera respectively to from side to the image of coal belt area is collected, through infrared camera collected image is infrared image, through visible light camera collected image is visible light image; S2. Image denoising: the infrared image and visible light image in step S1 are size transformed to obtain equal size images, and the noise in the size transformed infrared image and visible light image is filtered through Gaussian filtering; S3. Image gray scale: the infrared image and visible light image after removing noise are respectively processed by gray scale to obtain source infrared gray scale image and source visible light gray scale image respectively; S4. Convolutional neural network extracts image features: the source infrared gray scale image and source visible light gray scale image are respectively input into the convolutional neural network, and the image features F1 of the source infrared gray scale image and the image features F2 of the source visible light gray scale image are respectively extracted through the convolutional neural network; S5. L1 regularization extracts feature norm, and obtains two-dimensional weight mapping: the feature norm of the image features of the source infrared gray scale image and the source visible light gray scale image is respectively solved through L1 regularization, and the corresponding two-dimensional weight mapping of the image features is respectively obtained; The calculation process of the two-dimensional weight mapping of the image features F1 and the image features F2 obtained through L1 regularization is: wherein, represents the feature norm of the representative image in the (x, y) coordinate system, i ∈ {1, 2}, i = 1 represents an infrared image, i = 2 represents a visible light image, p represents the size of the sliding window, m represents the length of the sliding window, n represents the width of the sliding window, F i (m, n) represents the image feature value under the interaction window; S6. The weighted average fusion strategy is adopted to obtain the weight map of image fusion, and the source infrared gray scale image and the source visible light gray scale image are weighted and fused according to the weight map to obtain the fusion image of the source infrared gray scale image and the source visible light gray scale image; S7. The exposed belt roller is detected through the target detection algorithm: the fusion image is input into the target detection network to obtain the optimal prediction box of the exposed belt roller; The specific process is: The fusion image is input into the Yolov5s target detection network, the CSP module in the network is used to extract the roller feature map, the PAN-Net is used to fuse the roller feature map, then three preselected boxes are generated for each pixel point in the roller feature map and it is judged whether there is a roller target in the three preselected boxes, the position of the preselected box is optimized through the GIoU loss function, the target class in the preselected box is optimized through the cross entropy loss function, and thus the optimal prediction box of the roller is obtained; The calculation method of optimizing the preselected box by the GIoU loss function is: In the formula: A and B are two arbitrary frames in the image, C represents the minimum area enclosing A and B, and IoU represents the intersection over union of the true frame and the prediction frame; The calculation method of optimizing the target class in the preselected box by the cross entropy loss function is: where N represents the total number of target categories, x i is the predicted value of the target category output by the neural network, y i is the predicted probability of the target category, represents the true value of the target category, L class is the classification loss; S8. Calculate the length of the roller exposed to the belt: according to the target detection network, the length of the roller exposed to the belt is calculated. 2.The belt deviation detection method based on visible and infrared image fusion according to claim 1, characterized in that, In step S3, the specific calculation method of gray scale processing is: Gray=R*0.299+G*0.587+B*0.114 Wherein, Gray represents the gray value of the corresponding gray image, R is the red channel component in the RGB color image before the gray processing, G is the green channel component in the RGB color image before the gray processing, and B is the blue channel component in the RGB color image before the gray processing. 3.The belt deviation detection method based on visible and infrared image fusion according to claim 1, characterized in that, The specific steps of step S4 are: The source infrared gray image and the source visible light gray image are respectively input into the convolutional neural network VGG16, the image features are extracted through the convolutional layer in the network, and finally the activation function ReLU is used for processing, to obtain the image features F1 and F2 respectively; the calculation method of the convolutional layer for extracting the image features in VGG16 is: wherein: u l is the output of the lth convolutional layer, u l-1 is the output of the l-1th convolutional layer, w l is the convolution kernel, N is the receptive field of the input layer, a l is the bias parameter of the convolutional network, represents the convolution operation, and f is the activation function. The calculation method of the image feature F obtained by using the activation function ReLU for processing is: The image features F1 and F2 are obtained by using the formula.

4. The belt deviation detection method based on visible and infrared image fusion according to claim 1, characterized in that: In step S6, the weight map of image fusion and the calculation method of image fusion are: wherein, a weight map representing fusion of images in the (x, y) coordinate system, fused(x, y) represents a fused image after fusion of the source infrared grayscale image and the source visible light grayscale image, G i (x, y) represents a source grayscale image, i = 1 represents a source infrared grayscale image, and i = 2 represents a source visible light grayscale image.

5. The method according to claim 1, wherein the method is characterized by: When the infrared camera and the visible light camera respectively capture the image of the left side area of the coal conveying belt from the right top, the specific steps in step S8 are: According to the target frame of the exposed belt detected by the target detection network, the length L of the exposed belt of the carrier roller is calculated, first, it is judged whether the length L is in the limited area, if it is in the limited area, it means that the belt has no deviation; If it is not in the limited area, it is judged whether the length L is greater than the maximum value of the limited area or less than the minimum value of the limited area; if it is greater than the maximum value of the limited area, it means that the belt deviates to the right; if it is less than the minimum value of the limited area, it means that the belt deviates to the left.

6. The method according to claim 1, wherein the method is characterized by: When the infrared camera and the visible light camera respectively capture the image of the right side area of the coal conveying belt from the left top, the specific steps in step S8 are: According to the target frame of the exposed belt detected by the target detection network, the length L of the exposed belt of the carrier roller is calculated, first, it is judged whether the length L is in the limited area, if it is in the limited area, it means that the belt has no deviation; If it is not in the limited area, it is judged whether the length L is greater than the maximum value of the limited area or less than the minimum value of the limited area; if it is greater than the maximum value of the limited area, it means that the belt deviates to the right; if it is less than the minimum value of the limited area, it means that the belt deviates to the left.

Images