A deep learning-based conveyor belt intelligent detection device

Abstract

The application discloses a kind of based on deep learning's transmission machine belt intelligent detection device, including GPU operation platform, embedded single-chip microcomputer, alarm device, dust removal device, camera, real-time display device and telescopic triangular support, dust removal device is used to the surface of camera cleaning, alarm device is used to according to the signal of embedded single-chip microcomputer alarm, camera is used to obtain the real-time image of transmission machine belt and transmission to GPU operation platform, real-time display device is used to obtain the detection result of GPU operation platform and display, embedded single-chip microcomputer is used to obtain the detection result of GPU operation platform and according to detection result generates alarm signal and alarm, and generates timing signal transmission to rudder machine, GPU operation platform is used to detect transmission machine belt real-time image and detection result is sent to real-time display device and embedded single-chip microcomputer.The overfitting and misjudgment problem of crack detection is reduced, and the transmission machine belt detection efficiency and precision are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of conveyor belt detection, and in particular to a conveyor belt intelligent detection device based on deep learning. BACKGROUND

[0002] In modern society, transportation is one of the important infrastructure of social economy, and is the basic need and prerequisite for economic development. Conveyor belts play a huge role in the transportation industry in China, and can save a lot of manpower and resources, so they are widely used in industries such as metallurgy and chemical industry, and are suitable for many occasions. However, due to the continuous work of conveyor belts throughout the year, the wear and tear of transported materials, and other factors, conveyor belts are prone to breakage and even deviation, which can cause great economic losses and safety hazards. Therefore, in order to ensure the stability and safety of factory production and processing, a detection system is needed to detect various tears of conveyor belts.

[0003] The current traditional conveyor belt defect detection method is divided into two categories: one is contact type defect detection, and the other is non-contact type defect detection. The principle of contact type defect detection is that when the conveyor belt is torn or deviated, the sensor on the defect detection equipment senses the change in force to alarm. This method is expensive and prone to false positives and false negatives. Non-contact detection equipment usually detects whether the conveyor belt has defects by detecting special changes such as heat, sound, and light on the conveyor belt, and then alarms. The current market non-contact conveyor belt crack detection device is large in size, high in cost, difficult to maintain, low in detection accuracy, requires multiple cameras, difficult wiring, and needs to be connected to a desktop computer, which is time-consuming and labor-intensive to deploy in a port transportation unit with dozens of conveyor belts. SUMMARY

[0004] The present application provides a conveyor belt intelligent detection device based on deep learning to overcome the above technical problems.

[0005] A conveyor belt intelligent detection device based on deep learning, comprising a GPU operation platform, an embedded single-chip microcomputer, an alarm device, a dust removal device, a camera, a real-time display device, and a telescopic triangular support,

[0006] The telescopic triangular support is a base that changes the detection position and angle by adjusting its height and inclination angle.

[0007] The dust removal device is used to clean the surface of the camera according to the signal of the embedded single-chip microcomputer, and the dust removal device comprises a rudder and a dust removal brush.

[0008] The alarm device is used to alarm according to the signal of the embedded single-chip microcomputer, and the alarm device comprises an alarm and an alarm light.

[0009] The camera is used to obtain real-time images of the conveyor belt and transmit the real-time images to the GPU operation platform through a USB data line,

[0010] The real-time display device is used to obtain and display the detection results of the GPU operation platform, and the real-time display device is connected to the GPU operation platform through a USB data line,

[0011] The embedded single-chip microcomputer is used to obtain the detection results of the GPU operation platform and generate an alarm signal according to the detection results, transmit the alarm signal to the alarm device, and generate a timing signal to the steering engine, which controls the cleaning of the surface of the camera by the dust removal brush according to the timing signal, and the embedded single-chip microcomputer is connected to the alarm device, the GPU operation platform and the steering engine,

[0012] The GPU operation platform is used to detect the real-time images of the conveyor belt and send the detection results to the real-time display device and the embedded single-chip microcomputer.

[0013] Preferably, the GPU operation platform comprises a data processing module, a target recognition module, a crack precise detection module and a conveyor belt deviation detection module,

[0014] The data processing module is used to obtain an image data set containing the conveyor belt and expand the image data set, the expansion is to obtain pictures containing cracks of the conveyor belt in the image data set, to rotate the pictures containing cracks of the conveyor belt at different angles and to zoom the pictures at different scales respectively, to add the rotated and zoomed pictures to the image data set, to label and divide the expanded image data set into a training set and a test set,

[0015] The target recognition module is used to detect the materials and cracks on the conveyor belt in the real-time images according to the YOLOv5s model, the target recognition module comprises constructing the YOLOv5s model, modifying the YOLOv5s model, the modification comprises designing an activation function, a loss function and an anchor box prediction function, training the modified YOLOv5s model using the training set, obtaining the trained YOLOv5s model, obtaining the real-time images, detecting the real-time images according to the trained YOLOv5s model, obtaining a first detection result, and sending the first detection result to the real-time display device for display,

[0016] The crack precise detection module is used for acquiring the position of the crack on the conveyor belt in the image according to the U-net model, and the crack precise detection module comprises the following steps: acquiring a training set, processing the images in the training set by using a dark channel defogging method, constructing a U-net model, training the U-net model according to the processed training set, acquiring the trained U-net model, detecting real-time images according to the trained U-net model, acquiring a second detection result, and sending the second detection result to a real-time display device for display and to an embedded single-chip microcomputer,

[0017] The conveyor belt offset detection module is used for detecting whether the conveyor belt in the image is offset, and the conveyor belt offset detection module comprises the following steps: acquiring two real-time images at the current moment and the previous moment, denoising the two real-time images, acquiring the edges of the conveyor belt in the real-time images by using a canny edge detection algorithm, performing linear fitting on the images containing the edges of the conveyor belt according to a Hough line detection algorithm to obtain fitted conveyor belt edge lines, acquiring the conveyor belt edge lines of the two real-time images respectively, judging whether the slope difference of the conveyor belt edge lines at adjacent moments meets a threshold value, if the threshold value is met, it indicates that the conveyor belt is not offset, and if the threshold value is not met, it indicates that the conveyor belt is offset, and the offset information is sent to a real-time display device for display and to an embedded single-chip microcomputer.

[0018] Preferably, the modification of the YOLOv5s model comprises designing an activation function according to formula (1), designing a loss function according to formula (2), and designing an anchor box prediction function according to formulas (3) and (4),

[0019] f(x)=x / (e -x +1)(1)

[0020] Loss=λ1Lcls+λ2Lobj+λ3Lloc(2)

[0021] bw=pw·(2·σ(tw))·(2·σ(tw))(3)

[0022] bh=ph·(2·σ(th))·(2·σ(th))(4)

[0023] Wherein, x is an input image, λ1, λ2, λ3 are balance coefficients, Lcls is a rectangular frame loss, Lobj is a confidence loss, Lloc is a classification loss, tw is a predicted target center width offset, th is a predicted target center height offset, pw is the width of an input picture, ph is the height of an input picture, σ is a Sigmoid activation function, bw and bh are the width and height of the anchor box of the predicted boundary box.

[0024] Preferably, the processing of the images in the training set by the dark channel defogging method comprises processing according to formula (5),

[0025]

[0026] Wherein, I(x) is the input image, J(x) is the output image, A is the global atmospheric light value of the input image, t0 is the transmittance threshold, t(x) represents the transmittance of the input image.

[0027] Preferably, the linear fitting of the image containing the conveyor belt edge according to the Hough line detection algorithm comprises,

[0028] S11, the image containing the conveyor belt edge is binarized,

[0029] S12, the edge pixels of the image are detected by an edge detection operator, and the detected edge pixels are obtained,

[0030] S13, the edge pixels are fitted according to the Hough transform, and the fitted straight line set is obtained,

[0031] S14, the straight line set is filtered by the outlier filtering method, and the filtered straight line set is obtained,

[0032] S15, the filtered straight line set is fitted according to the least square fitting, and the fitted conveyor belt edge line is obtained.

[0033] The application provides a kind of based on deep learning's conveyor belt intelligent detection device of transmission machine, based on GPU operation platform and embedded single-chip microcomputer, can each module is packaged, this device is small integrated equipment, volume is smaller and easy to deploy.In addition, considering that factory environment is dim, combine timer with dust brush to remove dust in front of lens regularly;Data processing module, target identification module, crack accurate detection module and conveyor belt offset detection module are encapsulated in GPU operation platform, the target image to be identified is enhanced by dark channel processing mechanism and filtering algorithm, improve target definition, make accuracy higher, first conveyor belt data set of transmission machine is preprocessed, then conveyor belt crack and offset are detected, reduce the overfitting and misjudgment of crack detection due to lack of data and data anomaly, improve conveyor belt detection efficiency.Using dark channel defogging method to enhance the target plate to be identified at the same time, improve target definition, improve detection accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are only some embodiments of the present application, and the drawings can also be obtained by those skilled in the art without creative labor.

[0035] Figure 1 is the structure diagram of the rearview device of the present application (with cover);

[0036] Figure 2 is the structure diagram of the rearview device of the present application (with cover);

[0037] Figure 3 is the structure diagram of the rearview device of the present application (with cover);

[0038] Figure 4 is the structure diagram of the rearview device of the present application (with cover);

[0039] Figure 5 is the flow chart of the YOLOv5 target recognition of the present application;

[0040] Figure 6 is the flow chart of the U-net of the present application;

[0041] Explanation of reference numerals:

[0042] 1, rudder; 2, dust removal brush; 3, camera; 4, GPU operation platform; 5, embedded single-chip microcomputer; 6, alarm device; 7, real-time display device; 8, telescopic triangular support; 9, transmission machine belt; 10, conveying belt control line; 11, power line. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0044] Figure 1 is the structure diagram of the device of the present application, as shown in Figure 1 including GPU operation platform, embedded single-chip microcomputer, alarm device, dust removal device, camera, real-time display device and telescopic triangular support,

[0045] The telescopic triangular support is a base, and the telescopic triangular support is a base, which changes the detection position and the detection angle by adjusting the height and the inclination angle of itself.

[0046] The dust removal device is used for cleaning the surface of the camera according to the signal of the embedded single-chip microcomputer, and the dust removal device comprises a rudder and a dust removal brush,

[0047] The alarm device is used for alarming according to the signal of the embedded single-chip microcomputer, and the alarm device comprises an alarm and an alarm lamp,

[0048] The camera is used for acquiring real-time images of the conveyor belt and transmitting the real-time images to the GPU operation platform through a USB data line,

[0049] The real-time display device is used for acquiring and displaying the detection results of the GPU operation platform, and the real-time display device is connected with the GPU operation platform through a USB data line,

[0050] The embedded single-chip microcomputer is used for acquiring the detection results of the GPU operation platform and generating an alarm signal according to the detection results, transmitting the alarm signal to the alarm device, and generating a timing signal and transmitting the timing signal to the rudder, so that the rudder controls the dust removal brush to clean the surface of the camera according to the timing signal, and the embedded single-chip microcomputer is connected with the alarm device, the GPU operation platform and the rudder,

[0051] The GPU operation platform is used for detecting the real-time images of the conveyor belt and sending the detection results to the real-time display device and the embedded single-chip microcomputer.

[0052] Based on the above scheme, based on the GPU operation platform and the embedded single-chip microcomputer, each module can be packaged, the device is a small integrated device, and the volume is smaller and easy to deploy. In addition, considering the dim environment of the factory, the timer is combined with the dust removal brush to clean the dust in front of the lens regularly; the data processing module, the target recognition module, the crack accurate detection module and the conveyor belt offset detection module are packaged in the GPU operation platform, the target image to be recognized is enhanced through the dark channel processing mechanism and the filtering algorithm, the target definition is improved, the accuracy is higher, the conveyor belt dataset is preprocessed, then the conveyor belt crack and offset are detected, the overfitting and misjudgment problems caused by lack of data and data anomaly in crack detection are reduced, and the conveyor belt detection efficiency is improved. At the same time, the dark channel defogging method is used to enhance the target plate to be recognized, improve the target definition, and improve the detection accuracy.

[0053] As Figure 2 , 3, 4, the connection mode of the whole device is connected by using a data line. The power line is energized to control the conveyor belt, the camera is connected with the GPU operation platform through the USB data line, after the camera collects the image, the GPU operation platform will analyze the image, and display the real-time detection picture on the real-time display device, the real-time display device is connected with the GPU operation platform through the USB data line, and there is also a power line for power supply of the real-time display device. When the detection picture of the camera appears crack or detects the offset of the conveyor belt, the GPU operation platform will analyze the real-time image, and mark the crack on the image; at the same time, the GPU operation platform sends error information to the embedded single-chip microcomputer through the serial port, the TX serial port and the RX serial port of the GPU operation platform and the single-chip microcomputer are connected correspondingly, when the single-chip microcomputer receives the error information, a signal is output to the alarm device, the alarm lamp is turned on at the same time, and the alarm also starts to alarm, and a low-level signal is output through the conveyor belt control line, so that the conveyor belt stops moving. In order to prevent dust from affecting the detection accuracy, the embedded single-chip microcomputer outputs a PWM signal through the pin to control the steering engine, and then controls the rotation of the dust removal brush, which sweeps the camera every ten seconds to prevent dust from falling on the camera. In order to better control and monitor the device, the wireless receiver of the mouse and keyboard is inserted into the GPU operation platform at the same time, so that the whole device can be better controlled. In order to facilitate deployment, a telescopic tripod is used as the base, and the detection position and angle are changed by changing the height and inclination angle of the tripod, so that the detection device can be better arranged in the industrial field.

[0054] The various modules are packaged on the GPU operation platform for real-time monitoring of the belt state. In order to better monitor the real-time situation, the GPU operation platform is connected with a small real-time display device, when a crack appears, the crack position can be better displayed, helping workers to quickly and accurately find the crack. Once the crack is found, the GPU operation platform will feedback to the embedded single-chip microcomputer, and the single-chip microcomputer controls the alarm device and the power supply, issues an alarm at the same time, and powers off the online running conveyor belt, which is convenient for factory use and management.

[0055] Because the industrial site environment is poor, there is more dust and insufficient light. Therefore, a dust cover is provided outside the camera, and a timer and a dust removal brush are provided, which clean the surface of the camera every ten seconds, and regularly remove the dust in front of the lens.

[0056] The GPU operation platform comprises a data processing module, a target recognition module, a crack accurate detection module and a transmission machine belt offset detection module,

[0057] The data processing module is used for acquiring an image data set containing the conveyor belt and expanding the image data set, the expansion being acquiring pictures containing belt cracks in the image data set, respectively rotating and zooming the pictures containing belt cracks at different angles and different scales, adding the rotated and zoomed pictures to the image data set, labeling the expanded image data set and dividing it into a training set and a test set,

[0058] Specifically, since less image data containing conveyor belt cracks is collected, data expansion is needed on the basis of the original data set. In order to expand the data set, first, generate crack images of different angles and sizes using pictures with cracks. Since there is less conveyor belt data, collect pictures of conveyor belts at a certain harbor factory in Dalian as background pictures, then use PS technology and randomly place the crack images in the background pictures, and use to expand the data set to obtain several thousand pictures. Then use the labelimg tool to label the data set, select the object to be labeled, frame it, and classify it. The labeled file is stored in xml format, and all data is randomly divided into a training set and a test set in a ratio of 9:1 for model training and testing.

[0059] The target recognition module is used for detecting materials and cracks on the conveyor belt in real-time images according to a YOLOv5s model, the target recognition module comprising constructing a YOLOv5s model, modifying the YOLOv5s model, the modification comprising designing an activation function, a loss function and an anchor box prediction function. Specifically, the YOLOv5s model is constructed, and the YOLOv5s model is modified, the modification comprising designing an activation function, a loss function and an anchor box prediction function. In selecting the activation function, sigmoid function can cause overfitting, and ReLU can cause gradient disappearance, which cannot be used for model training. SiLU is an improved version of Sigmoid and ReLU. SiLU has the characteristics of no upper bound, smoothness and non-monotonicity, and SiLU performs better than ReLU in deep models, so SiLU is selected as the activation function to improve the accuracy of model prediction. In the output layer, a loss function is used to adjust the prediction result, the loss function can measure the distance between the predicted information and the expected information of the neural network, and the closer the predicted information is to the expected information, the smaller the loss function value is. During training, there are mainly three losses: rectangle box loss (Classesloss), confidence loss (Objectnessloss) and classification loss (Locationloss). In calculating the loss, in order to eliminate sensitivity and make the calculation of the loss more accurate, the calculation formula of the predicted target height and width is adjusted. The original anchor box prediction formula is:

[0060] bw=pw·e tw(1)

[0061] bh=ph·e th (2)

[0062] The modification of the YOLOv5s model comprises designing an activation function according to formula (3), designing a loss function according to formula (4), and designing an anchor frame prediction function according to formulas (5) and (6),

[0063] f(x)=x / (e -x +1)(3)

[0064] Loss=λ1Lcls+λ2Lobj+λ3Lloc(4)

[0065] bw=pw·(2·σ(tw))·(2·σ(tw))(5)

[0066] bh=ph·(2·σ(th))·(2·σ(th))(6)

[0067] wherein x is an input image, λ1, λ2, and λ3 are balance coefficients, Lcls is a rectangular frame loss, Lobj is a confidence loss, Lloc is a classification loss, tw is a predicted target center width offset, th is a predicted target center height offset, pw is the width of an input picture, ph is the height of the input picture, and σ is a Sigmoid activation function, which limits the predicted offset to between 0 and 1, that is, the predicted center point cannot exceed the corresponding GridCell area. The output prediction frame is compared with the real frame, the offset is calculated, and then it is updated in reverse, and the parameters are iterated according to the above formulas (5) and (6) to obtain the width and height of the anchor frame of the prediction boundary frame with the maximum probability, that is, bw and bh, thereby improving the crack detection accuracy.

[0068] The modified YOLOv5s model is trained using a training set, a trained YOLOv5s model is obtained, a real-time image is obtained, the real-time image is detected according to the trained YOLOv5s model, a first detection result is obtained, and the first detection result is sent to a real-time display device for display,

[0069] Specifically, in the conveyor belt crack target recognition module, since the real-time performance, detection accuracy and detection speed of the detection device are very high, and the YOLOv5s has the characteristics of shorter training time and faster inference speed, the YOLOv5s model is used in the conveyor belt crack detection module. After the data set is configured, the YOLOv5s model first performs data enhancement on the picture. The Mosaic used in YOLOv5 is referenced from the CutMix data enhancement method proposed at the end of 2019, but CutMix only uses two pictures for splicing, while Mosaic data enhancement uses four pictures, and splices them in a random scaling, random cropping and random arrangement manner, which has good detection effect for small targets. Data enhancement can enrich the data set and reduce the GPU computation. Then the adaptive anchor box calculation is performed. During training, the best anchor box value and the ground truth (anchor box during data preprocessing) of the data in different training sets are compared, the offset of the real box position relative to the preset box is calculated, and the offset is provided for the subsequent network. Next, the image features are extracted and fused, and finally the anchor box is adjusted according to the loss function. After multiple training, the crack on the training set picture can be quickly and accurately anchored. The process is as shown in Figure 5 The YOLOv5s model includes an input end, a Backbone network, a Neck network and a Head network, and a prediction network, wherein loss is a loss function, and gradient backforword is a gradient forward propagation.

[0070] The crack precise detection module is used to obtain the position of the crack on the conveyor belt in the image according to the U-net model. The crack precise detection module includes obtaining a training set, processing the images in the training set by the dark channel defogging method, constructing a U-net model, training the U-net model according to the processed training set, obtaining the trained U-net model, detecting the to-be-detected image according to the trained U-net model, obtaining a second detection result, sending the second detection result to a real-time display device for display and to an embedded single-chip microcomputer,

[0071] Specifically, the training set is obtained, and the images in the training set are processed by the dark channel defogging method. In order to make the input image clearer, reduce detection errors and improve accuracy, the dark channel defogging algorithm is added before the image is convolved, and guided filtering optimization is performed on the obtained image. The dark channel defogging algorithm can restore high-quality images and also obtain high-quality depth maps.

[0072] The processing of the images in the training set by the dark channel defogging method includes processing according to formula (7),

[0073]

[0074] Wherein, I(x) is the input image, J(x) is the output image, A is the global atmospheric light value of the input image, t0 is the transmittance threshold value, generally taking the value of 0.1, t(x) represents the transmittance of the input image, since the brightness of the scene is usually not as bright as the light in the atmosphere, the image after removing the haze looks very dark, when the dehazing image is obtained according to formula (5), the image needs to be exposed to adjust.

[0075] In order to better identify and mark the cracks, the U-net model is used to detect the cracks, and the U-net model can well segment and identify the edges of the texture and small cracks and irregular shapes. First, the dark channel dehazing method is used to process the picture, and the dehazed image is convolved using the U-net network. The U-net network can use limited labeled data more effectively by relying on data enhancement from very few training images, and realize the positioning of the image pixels. The network classifies each pixel point in the image, and finally outputs the segmented image according to the category of the pixel point. The specific process is as shown in Figure 6

[0076] The U-net model is constructed, the U-net model is trained according to the processed training set, and the trained U-net model is obtained, the trained U-net model is used for judging the position of the crack in the image, and the U-net model is composed of a shrinking path and an expanding path, wherein the shrinking path is used for obtaining context information, and the expanding path is used for accurate positioning. U-net is mainly divided into three parts: feature extraction, splicing and up-sampling. The feature extraction part is a shrinking network, which extracts shallow information in the process of continuous down-sampling; the splicing part aims to fuse feature information, so that the information of the deep layer and the shallow layer is fused; the up-sampling part extracts deep information, and fuses the shallow information on the left side, that is, splices the left feature. The main stem of U-net is divided into symmetrical left and right parts: the left side of the original input image is subjected to four times of down-sampling through convolution-maximum pooling to obtain four levels of feature maps. After processing the image, the U-net network extracts important features of the image and reduces the image resolution, minimizes the complexity of the neural network, and maintains the segmentation accuracy; the right side of each level feature map and the feature map obtained through deconvolution is fused through the jump connection method, and then the image details are gradually repaired; the last layer is used to calculate the loss with the label to predict the semantic graph, and the crack position is accurately located. After multiple training, the crack position can be accurately calibrated.

[0077] ​The conveyor belt deviation detection module is used for detecting whether the conveyor belt deviates in the image, and the conveyor belt deviation detection module comprises: acquiring two images to be detected at a current time and a previous time, denoising the two images to be detected, acquiring the edge of the conveyor belt in the image to be detected through a canny edge detection algorithm, performing linear fitting on the image containing the edge of the conveyor belt according to a Hough straight line detection algorithm to obtain the fitted edge line of the conveyor belt, acquiring the edge line of the conveyor belt of the two images to be detected respectively, judging whether the slope difference of the edge lines of the conveyor belt at adjacent times meets a threshold value, if the threshold value is met, it indicates that the conveyor belt does not deviate, and if the threshold value is not met, it indicates that the conveyor belt deviates, and the deviation information is sent to a real-time display device for display and to an embedded single-chip microcomputer.

[0078] In transportation, the conveyor belt may deviate sometimes. In order to detect whether the conveyor belt deviates, the OpenCV edge detection algorithm is used to monitor the edge of the conveyor belt in real time. Once the conveyor belt deviates, the upper computer will immediately feed back to the lower computer, the lower computer will immediately alarm and power off the conveyor belt. The key to feature extraction of the edge of the conveyor belt lies in the sudden change of pixel brightness, that is, the edge is determined according to the gradient amplitude and gradient direction of the image. Generally, the sobel operator is used to calculate the gradient amplitude and gradient direction of the image. After the gradient amplitude and gradient direction of the image are obtained, the non-maximum suppression operation (suppressing elements that are not maximum values, searching for local maximum values) is needed to be performed on the image edge according to the obtained gradient amplitude and gradient direction. In order to detect the edge, the change existing between adjacent pixels is found. After the image is loaded, the noise in the picture is removed through filtering, and then the canny edge detection is performed to prepare for obtaining the edge of the conveyor belt.

[0079] The two images to be detected at a current time and a previous time are acquired, the two images to be detected are denoised, and the edge of the conveyor belt in the image to be detected is acquired through a canny edge detection algorithm. The canny edge detection step comprises:

[0080] 1. Use Gaussian blur to remove noise points.

[0081] 2. Perform gray scale conversion.

[0082] 3. Use the sobel operator to calculate the gradient size and gradient direction of each point.

[0083] 4. Use non-maximum suppression (only the maximum is retained) to eliminate the stray effect caused by edge detection.

[0084] 5. Apply double-threshold boundary tracking to determine the real and potential edges.

[0085] 6. The final edge detection is completed by suppressing weak edges.

[0086] The image is subjected to ROI (region of interest) extraction to obtain the exact conveyor belt edge, and is subjected to an AND operation with the previous canny image to obtain the required conveyor belt edge.

[0087] The image containing the conveyor belt edge is subjected to linear fitting according to the Hough line detection algorithm to obtain the fitted conveyor belt edge line, the conveyor belt edge lines of the two images to be detected are obtained respectively, whether the slope difference of the conveyor belt edge lines at adjacent moments satisfies a threshold value is judged, if the threshold value is satisfied, it indicates that the conveyor belt does not deviate, the image to be detected obtained in step four is returned, if the threshold value is not satisfied, it indicates that the conveyor belt deviates, and alarm and power-off processing are performed according to the deviation information.

[0088] The linear fitting of the image containing the conveyor belt edge according to the Hough line detection algorithm comprises,

[0089] S11, the image containing the conveyor belt edge is subjected to binaryzation processing,

[0090] S12, the edge pixel points of the image are detected by an edge detection operator to obtain the detected edge pixel points,

[0091] S13, the edge pixel points are fitted according to the Hough transform to obtain a set of straight lines after fitting,

[0092] S14, the set of straight lines is filtered by an outlier filtering method to obtain a set of filtered straight lines,

[0093] S15, the set of filtered straight lines is fitted according to the least square fitting to obtain the fitted conveyor belt edge line,

[0094] Specifically, first, the image is subjected to gray scale transformation, then the brightness difference between each adjacent pixel is compared, the place where the brightness suddenly changes is the edge pixel, a plurality of edge pixel points are generated, the Hough transform is performed on these edge pixel points, the Hough transform is to transform the pixels in the image from one spatial coordinate system to another coordinate system, fit a plurality of straight lines, and form an edge image.

[0095] The edge detection operator is used to find the edges where the pixel brightness suddenly changes, and when the first order derivative of the brightness function is obtained, more obvious peaks are obtained. x Gx is the first order derivative in the horizontal direction x, y Gy is the first order derivative in the vertical direction y, the gradient and direction are obtained according to the first order derivative. The gradient formula and direction formula are shown in (8) and (9).

[0096]

[0097]

[0098] The direction and size of the gradient can be obtained by substituting the slope into the above formula, and the points on the non-boundary are removed.

[0099] In order to better remove unnecessary straight lines, outlier filtering is performed to remove line segments with a large difference in slope. Two thresholds are set: a minimum value and a maximum value. First, the slope of all line segments is obtained, and then the average slope is calculated. All slopes are traversed, the difference from the average slope is calculated, and the largest slope is found. If the difference is greater than the threshold, the corresponding line segment and slope are removed from the list. The above operation is performed in a loop until all remaining line segments are less than the threshold.

[0100] Finally, least squares fitting is performed to realize the mutual fitting of the left and right line segments into a straight line to form the conveyor belt edge line. Once the edge line slope changes significantly, the GPU computing platform can immediately send a signal to the lower computer. The GPU computing platform sends a signal to the embedded single-chip microcomputer through the serial port. When deployed on the GPU computing platform, the model performance can be improved by 10 times according to the scene, and at the same time, the model is optimized using TensorRT and accelerated using CUDA.

[0101] The current market conveyor belt crack detection device is mostly large in size, needs to deploy multiple cameras, wiring is difficult, and needs to be connected to a desktop computer, which is time-consuming and laborious to deploy in a port transportation unit with dozens of conveyor belts. The equipment on the market is mostly large in size, high in cost, difficult to maintain, low in detection accuracy, and lacks a complete set of detection system that is small in size, low in price, strong in reliability, and easy to maintain. The device is based on a GPU computing platform and an embedded single-chip microcomputer, and can be packaged into a small integrated device, which is lower in price and smaller in size and easy to deploy. In the embodiment, the system is integrated in the device, which includes a GPU computing platform, an embedded single-chip microcomputer, a power supply, an alarm device, a dust removal device, and a real-time display device.

[0102] The system is deployed on a GPU operation platform as a whole in the embodiment, and the system modules are modularized and packaged, and are used for monitoring the belt state in real time. In order to better monitor the real-time condition, the GPU operation platform is connected with a small real-time display device, when a crack appears, the crack position can be better displayed, and workers are helped to quickly and accurately find the crack. Once the crack is found, the GPU operation platform feeds back to the embedded single-chip microcomputer, the single-chip microcomputer controls the alarm device and the power supply, issues an alarm, and at the same time, the power supply of the transmission belt running online is turned off, so as to facilitate the use and management of the factory. Since the industrial site environment is poor, there is more dust and insufficient light. Therefore, a dust cover is externally arranged for the camera, and a timer and a dust removal brush are provided, the camera surface is cleaned every ten seconds, and the dust in front of the lens is regularly cleaned.

[0103] Overall beneficial effects:

[0104] The application provides a transmission belt intelligent detection device based on deep learning, which is based on a GPU operation platform and an embedded single-chip microcomputer, can encapsulate each module, and is a small integrated device, has a smaller volume and is easy to deploy. In addition, considering the dim environment of the factory, a timer and a dust removal brush are combined to regularly clean the dust in front of the lens; a data processing module, a target recognition module, a crack accurate detection module and a transmission belt offset detection module are encapsulated in the GPU operation platform, the target image to be recognized is enhanced through a dark channel processing mechanism and a filtering algorithm, the target definition is improved, the accuracy is higher, the transmission belt data set is pretreated, then the transmission belt crack and offset are detected, the overfitting and misjudgment problems of crack detection caused by lack of data and data anomaly are reduced, and the transmission belt detection efficiency is improved. Meanwhile, the dark channel defogging method is used to enhance the target plate to be recognized, improve the target definition, and improve the detection accuracy.

[0105] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the application, and not to limit them; although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the application.

Claims

1. A deep learning-based intelligent detection device for conveyor belts, characterized in that, Includes a GPU computing platform, embedded microcontroller, alarm device, dust removal device, camera, real-time display device, and retractable tripod. The retractable triangular bracket serves as a base, and its height and tilt angle can be adjusted to change the detection position and detection angle. The dust removal device is used to clean the surface of the camera based on signals from the embedded microcontroller. The dust removal device includes a servo motor and a dust removal brush. The alarm device is used to trigger an alarm based on signals from an embedded microcontroller. The alarm device includes an alarm unit and an alarm light. The camera is used to acquire real-time images of the adjusted conveyor belt and transmits these images to the GPU computing platform via a USB data cable. The real-time display device is used to acquire and display the detection results of the GPU computing platform. The real-time display device is connected to the GPU computing platform via a USB data cable. The embedded microcontroller is used to acquire the detection results from the GPU computing platform and generate an alarm signal based on the detection results. The alarm signal is then transmitted to the alarm device, and a timing signal is generated and transmitted to the servo motor. The servo motor controls the dust removal brush to clean the surface of the camera according to the timing signal. The embedded microcontroller is connected to the alarm device, the GPU computing platform, and the servo motor. The GPU computing platform is used to detect real-time images of the conveyor belt and send the detection results to a real-time display device and an embedded microcontroller. The GPU computing platform includes a data processing module, a target recognition module, a crack precision detection module, and a conveyor belt offset detection module. The data processing module is used to acquire an image dataset containing a conveyor belt and to augment the image dataset. Augmentation involves acquiring images of conveyor belt cracks within the image dataset, rotating these images at different angles and scaling them at different ratios, adding the rotated and scaled images to the image dataset, and then labeling and dividing the augmented image dataset into training and testing sets. The target recognition module is used to detect materials and cracks on a conveyor belt in a real-time image based on a YOLOv5s model. The module includes constructing a YOLOv5s model, modifying the model (including designing activation functions, loss functions, and anchor box prediction functions), training the modified YOLOv5s model using a training set, obtaining the trained YOLOv5s model, acquiring a real-time image, performing detection on the real-time image based on the trained YOLOv5s model, obtaining a first detection result, and sending the first detection result to a real-time display device for display. The crack detection module is used to locate cracks on a conveyor belt in an image based on the U-net model. The module includes acquiring a training set, processing the images in the training set using a dark channel dehazing method to construct a U-net model, training the U-net model using the processed training set, obtaining the trained U-net model, detecting cracks in a real-time image using the trained U-net model, obtaining a second detection result, and sending the second detection result to a real-time display device for display and to an embedded microcontroller. The process of processing the images in the training set using the dark channel dehazing method includes processing according to formula (5). (5) in, For the input image, For the output image, The global atmospheric light value of the input image. The transmittance threshold, This represents the transmittance of the input image; The conveyor belt offset detection module is used to detect whether the conveyor belt in an image has shifted. The module includes acquiring two real-time images at the current moment and the previous moment, denoising the two real-time images, obtaining the edge of the conveyor belt in the real-time images using the Canny edge detection algorithm, fitting a straight line to the image containing the conveyor belt edge using the Hough line detection algorithm to obtain the fitted conveyor belt edge line, acquiring the conveyor belt edge line of the two real-time images respectively, and determining whether the slope difference of the conveyor belt edge line at adjacent moments meets a threshold. If it meets the threshold, it means that the conveyor belt has not shifted; if it does not meet the threshold, it means that the conveyor belt has shifted. The offset information is sent to a real-time display device for display and to an embedded microcontroller.

2. The intelligent detection device for a conveyor belt based on deep learning according to claim 1, characterized in that, The modification of the YOLOv5s model includes designing an activation function according to formula (1), a loss function according to formula (2), and anchor box prediction functions according to formulas (3) and (4). f(x)= x / ( + 1)(1) (2) bw=pw·(2·σ(tw))·(2·σ(tw))(3) bh=ph·(2·σ(th))·(2·σ(th))(4) Where x is the input image, For balance coefficient, For rectangular frame loss, For confidence loss, For classification loss, tw is the predicted target center width offset, th is the predicted target center height offset, pw is the width of the input image, ph is the height of the input image, σ is the Sigmoid activation function, and bw and bh are the width and height of the anchor boxes of the predicted bounding boxes.

3. The intelligent detection device for a conveyor belt based on deep learning according to claim 1, characterized in that, The step of performing line fitting on the image containing the edge of the conveyor belt according to the Hough line detection algorithm includes... S11. Binarize the image containing the edge of the conveyor belt. S12. Detect edge pixels in the image using an edge detection operator and obtain the detected edge pixels. S13. Fit the edge pixels according to the Hough transform to obtain the set of fitted lines. S14. Filter the set of lines using the outlier filtering method to obtain the filtered set of lines. S15. Fit the filtered set of straight lines using least squares fitting to obtain the fitted edge line of the conveyor belt.

Images