A Method and System for Detecting Longitudinal Tears in Belt Conveyors Based on Computer Vision Recognition

By using a generative adversarial network that fuses thermal infrared and visible light images to detect belt cracks, the problem of unstable detection under external environmental interference is solved, and high-accuracy belt crack identification is achieved under different conditions.

CN118037688BActive Publication Date: 2026-11-17WUHAN UNIV OF TECH
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202410234971.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-01
Publication Date
2026-11-17
Estimated Expiration
2044-03-01

AI Technical Summary

Technical Problem

Existing methods for detecting longitudinal tears in belt conveyors are unstable under external environmental conditions such as light and noise, making it difficult to accurately identify belt cracks.

Method used

A method for fusing thermal infrared and visible light images is adopted. The fused image is generated by a generative adversarial network. The generator and two discriminators are trained and combined with the YOLOv5s model for belt crack detection. The loss function of the generator includes content loss, structure loss and generative adversarial loss. Penalty terms for the visible light and thermal infrared image discriminators are designed to improve detection accuracy.

Benefits of technology

It can accurately detect belt cracks under various lighting and noise conditions, improving the stability and accuracy of detection. The generated fused image has clear texture and high brightness, enhancing the effect of belt crack detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118037688B_ABST
    Figure CN118037688B_ABST
Patent Text Reader

Abstract

This invention discloses a method and system for detecting longitudinal tears in conveyor belts based on computer vision recognition. The method includes: acquiring and processing thermal infrared and visible light images of the conveyor belt to ensure they are the same size and correspond to the same belt region; constructing a generative adversarial network (GAN), including a generator and two discriminators, one for the visible light image and one for the thermal infrared image; inputting the processed and corresponding thermal infrared and visible light images into the generator to generate a fused image, inputting the fused image and the visible light image into the visible light image discriminator, and inputting the fused image and the thermal infrared image into the thermal infrared image discriminator, thereby training the GAN; using the trained GAN to generate a fused image of the conveyor belt to be detected; and using the fused image of the conveyor belt to detect belt cracks. This invention improves accuracy by fusing the thermal infrared and visible light images of the conveyor belt before detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of machine vision technology, specifically relating to a method and system for detecting longitudinal tears in belt conveyors based on computer vision recognition. Background Technology

[0002] Belt conveyors are a common type of conveying equipment, mainly used to transport materials from one place to another. They offer the following functions: material conveying, improved efficiency, space saving, high adaptability, and automated control, significantly increasing productivity.

[0003] Belt conveyors may experience the following malfunctions during operation: belt breakage or wear, drive system failure, tensioner malfunction, belt misalignment, cleaning device failure, and electrical faults. Among these, belt breakage poses a significant hazard. A broken belt can cause the material in the conveying process to stop abruptly, potentially leading to production interruptions and equipment damage. Cracks on the belt, once they widen or spread, can also cause breakage. Cracks accelerate belt wear, leading to faster wear and more frequent replacements, increasing maintenance and replacement costs. Cracks also reduce the belt's strength, affecting the conveyor's normal conveying capacity. If a belt breaks during operation, it can cause material accumulation, blockages, and even accidental injuries. Furthermore, a broken belt may become entangled in other equipment or components, causing equipment malfunctions or personal injury.

[0004] Existing technologies mainly process signals through traditional machine vision or by combining deep learning with sound signals. However, these technologies are easily affected by external environmental factors such as lighting and noise during the detection process, which makes the accuracy of the detection results unstable. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for detecting longitudinal tears in belt conveyors based on computer vision recognition, which solves the problem that the detection results are easily affected by the external environment during the detection process, and can accurately detect under various lighting and noise conditions, thereby improving the stability of the detection.

[0006] The technical solution adopted in this invention is as follows:

[0007] In a first aspect, the present invention provides a method for detecting longitudinal tears in a belt conveyor based on computer vision recognition, the method comprising:

[0008] Acquire thermal infrared and visible light images of the belt;

[0009] The acquired thermal infrared and visible light images of the belt are processed to make them the same size and correspond to the same belt area.

[0010] A generative adversarial network is constructed, consisting of a generator and two discriminators, namely a visible light image discriminator and a thermal infrared image discriminator.

[0011] The processed and corresponding thermal infrared image and visible light image are input into the generator to generate a fused image. The fused image and visible light image are input into the visible light image discriminator. The fused image and thermal infrared image are input into the thermal infrared image discriminator to train the generative adversarial network.

[0012] The trained generative adversarial network is used to generate a fused image of the belt to be detected;

[0013] Belt crack detection is performed using the fused image of the belt to be inspected.

[0014] Furthermore, thermal infrared images and visible light images of the area under the belt were acquired using a thermal infrared imager and an industrial camera, respectively.

[0015] Furthermore, a light source is placed below the belt to provide supplemental lighting in order to obtain a clear visible light image of the belt.

[0016] Furthermore, the generator adopts a residual network structure, and the two discriminators have the same structure, both consisting of convolutional and fully connected layers.

[0017] Furthermore, the generator's loss function L includes content loss, structural loss, and generative adversarial loss, expressed as:

[0018] L = l cont +l ssim +l madv

[0019] Among them, content loss l cont The expression is:

[0020]

[0021] In the formula, I f For the fused image generated by the generator, I inf For thermal infrared images, I vis This is a visible light image, where H and W are the height and width of the image, respectively.

[0022] Structural loss l ssim The expression is:

[0023] l ssim =1-SSIM(I f ,I inf )+1-SSIM(I f ,I vis )

[0024] In the formula, SSIM stands for Structural Similarity, which is an index for measuring the similarity between two images;

[0025] Generate adversarial loss madv The expression is:

[0026]

[0027]

[0028] In the formula, D inf For thermal infrared image discriminators, D vis For visible light image discriminators, The dataset for the generated fused images; x represents the dataset P. x Data, x i Let f(x) represent the i-th data point, m be the total number of data points, and f(x) be the total number i ) is the data x i The corresponding value.

[0029] Furthermore, the expression for structural similarity (SSIM) is as follows:

[0030] SSIM(x,y)=[l(x,y) α ·c(x,y) β ·s(x,y) γ ]

[0031]

[0032]

[0033]

[0034] Setting α, β, and γ to 1, we get:

[0035]

[0036] Where, μ x It is the average value of x, μ y It is the average value of y. It is the variance of x. σ is the variance of y. xy c1 is the covariance of x and y, where x and y are the numerical matrices of the image; c1 = (k1L') 2 c2=(k2L') 2 , L is a constant used to maintain stability, L' is the dynamic range of the pixel threshold, and k1 and k2 are coefficients.

[0037] Furthermore, the loss function of the visible light image discriminator for:

[0038]

[0039] In the formula, P vis For a dataset of visible light images, P penalty For P vis and P If The area between For P penalty The image selected in Ψ is the gradient, and Ψ is the penalty parameter; the loss function uses the Wass distance. During training, The value will increase. The value of will become smaller and smaller, and the discriminator will fail to converge. Therefore, a penalty term is added to the loss function, adding a penalty parameter to all terms whose gradient magnitude is greater than or equal to 1, and summing all the penalty terms to influence D. vis The optimal solution;

[0040] Loss function of thermal infrared image discriminator for:

[0041]

[0042] In the formula, P inf For the dataset of thermal infrared images, P penalty' For P inf and The area between For P penalty' The image selected from [the source].

[0043] Furthermore, based on the YOLOv5s model, belt crack detection is performed using the fused image of the belt to be detected.

[0044] Secondly, the present invention also provides a computer vision-based detection system for detecting longitudinal tears in belt conveyors, used to implement the computer vision-based detection method for longitudinal tears in belt conveyors as described in any one of the above-mentioned methods, comprising:

[0045] The raw image acquisition module is used to acquire thermal infrared and visible light images of the belt;

[0046] The raw image processing module is used to process the acquired thermal infrared image and visible light image of the belt to make them the same size and the corresponding belt area.

[0047] The construction module is used to construct the generative adversarial network, which includes a generator and two discriminators, namely a visible light image discriminator and a thermal infrared image discriminator.

[0048] The optimization module is used to input the processed and corresponding thermal infrared image and visible light image into the generator to generate a fused image. The fused image and visible light image are input into the visible light image discriminator, and the fused image and thermal infrared image are input into the thermal infrared image discriminator to train the generative adversarial network.

[0049] The fusion module is used to generate a fused image of the belt to be detected using the trained generative adversarial network;

[0050] The detection module is used to detect belt cracks using the fused image of the belt to be detected.

[0051] Furthermore, the generator's loss function L includes content loss, structural loss, and generative adversarial loss, expressed as:

[0052] L = l cont +l ssim +l madv

[0053] Among them, content loss l cont The expression is:

[0054]

[0055] In the formula, I f For the fused image generated by the generator, I inf For thermal infrared images, I vis This is a visible light image, where H and W are the height and width of the image, respectively.

[0056] Structural loss l ssim The expression is:

[0057] l ssim =1-SSIM(I f ,I inf )+1-SSIM(I f ,I vis )

[0058] In the formula, SSIM stands for Structural Similarity, which is an index for measuring the similarity between two images;

[0059] Generate adversarial loss madv The expression is:

[0060]

[0061]

[0062] In the formula, D inf For thermal infrared image discriminators, D vis For visible light image discriminators, The dataset for the generated fused images; x represents the dataset P. x Data, x i Let f(x) represent the i-th data point, m be the total number of data points, and f(x) be the total number i ) is the data x i The corresponding value;

[0063] Loss function of visible light image discriminator for:

[0064]

[0065] In the formula, P vis For a dataset of visible light images, P penalty For P vis and The area between For P penalty The image selected in Ψ is the gradient, and Ψ is the penalty parameter; the loss function uses the Wass distance. During training, The value will increase. The value of will become smaller and smaller, and the discriminator will fail to converge. Therefore, a penalty term is added to the loss function, adding a penalty parameter to all terms whose gradient magnitude is greater than or equal to 1, and summing all the penalty terms to influence D. vis The optimal solution;

[0066] Loss function of thermal infrared image discriminator for:

[0067]

[0068] In the formula, P inf For the dataset of thermal infrared images, P penalty' For P inf and P If The area between For P penalty' The image selected from [the source].

[0069] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0070] The present invention relates to a method and system for detecting longitudinal tears in belt conveyors based on computer vision recognition. By fusing thermal infrared images and visible light images of the belt before detection, it can accurately determine when a longitudinal tear occurs, avoiding interference from light and noise and improving the accuracy of the judgment.

[0071] Furthermore, when fusing the thermal infrared and visible light images of the belt, the generator's loss function includes three parts: content loss, structure loss, and generative adversarial loss. In addition, loss functions for the visible light image discriminator and the thermal infrared image discriminator were specially designed to make the generated fused image have clearer texture and higher brightness, thus improving the accuracy of YOLOv5s in detecting belt cracks. Attached Figure Description

[0072] Figure 1 This is a block diagram of a belt conveyor longitudinal tear detection system.

[0073] Figure 2 A physical image of the original data sampling module;

[0074] Figure 3 Here is the network structure diagram of the generator;

[0075] Figure 4 This is a network structure diagram of a block;

[0076] Figure 5 This is a network structure diagram of the discriminator;

[0077] Figure 6 This is a diagram of the YOLOv5S network structure. Detailed Implementation

[0078] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0079] This invention discloses a method and system for detecting longitudinal tears in belt conveyors based on computer vision recognition. It solves the problem that the detection results are easily affected by the external environment during the detection process, and can accurately detect under various lighting and noise conditions, thus improving the stability of the detection.

[0080] The present invention provides a computer vision-based method for detecting longitudinal tears in belt conveyors, comprising:

[0081] Acquire thermal infrared and visible light images of the belt;

[0082] The acquired thermal infrared and visible light images of the belt are processed to make them the same size and correspond to the same belt area.

[0083] A generative adversarial network is constructed, consisting of a generator and two discriminators, namely a visible light image discriminator and a thermal infrared image discriminator.

[0084] The processed and corresponding thermal infrared image and visible light image are input into the generator to generate a fused image. The fused image and visible light image are input into the visible light image discriminator. The fused image and thermal infrared image are input into the thermal infrared image discriminator to train the generative adversarial network.

[0085] The trained generative adversarial network is used to generate a fused image of the belt to be detected;

[0086] Belt crack detection is performed using the fused image of the belt to be inspected.

[0087] like Figure 1 As shown, the longitudinal tear detection system for belt conveyors based on computer vision recognition of the present invention includes the following modules: original image acquisition module, original image processing module, construction module, optimization module, fusion module, detection module, and alarm module.

[0088] The original image acquisition module, such as Figure 2 As shown, the belt (not shown) is located above the idler roller and operates by the rotation of the roller. Due to the low light level below, a light source is used for supplemental illumination. A thermal infrared imager and an industrial camera located below the belt collect data on the belt running on the idler roller, acquiring thermal infrared and visible light images as raw data. These are then divided into training and test sets at a 9:1 ratio. After acquisition, the images are sent to the terminal for processing. Visible light images can clearly show cracks on the belt under good lighting conditions, but are easily affected by lighting conditions. Thermal infrared images can identify the heat information of the belt, allowing for rapid identification of scratches instantly. They can be used under any lighting conditions, but are affected by ambient temperature. Combining the information from both images to identify belt cracks significantly improves detection accuracy.

[0089] The initial image processing module crops the thermal infrared and visible light images acquired from different locations by a thermal infrared imager and an industrial camera, respectively, into images of the same size that can overlap, preparing for subsequent modules. Because the thermal infrared imager and industrial camera have different image acquisition points, the position of the same object in the images may differ. To ensure more accurate image fusion, the initial image processing module needs to crop and transform the acquired images to obtain two images of the same size that can overlap, showing the same object.

[0090] The construction module is used to build the network structure and loss function for the generator and discriminator. Please refer to [reference needed]. Figure 3 , Figure 4 and Figure 5The generator uses a residual network structure, which can retain more image information through concatenation. The two discriminators have the same structure, consisting of convolutional and fully connected layers.

[0091] The generator's loss function consists of three parts: content loss, structural loss, and generative adversarial loss, expressed as follows:

[0092] L = l cont +l ssim +l madv

[0093] Content loss cont The expression is:

[0094] Among them, I f For the fused image generated by the generator, I inf For thermal infrared images, I vis For visible light images, H and W represent the height and width of the image, respectively.

[0095] l ssim The expression is: l ssim =1-SSIM(I f ,I inf )+1-SSIM(I f ,I vis )

[0096] SSIM (structural similarity index) is a metric for measuring the similarity between two images.

[0097] SSIM(x,y)=[l(x,y) α ·c(x,y) β ·s(x,y) γ ]

[0098]

[0099]

[0100]

[0101] Setting α, β, and γ to 1, we get:

[0102]

[0103] Where μ x It is the average value of x, μ y It is the average value of y. It is the variance of x. σ is the variance of y.xy Let c1 be the covariance of x and y, and c1 = (k1L). 2 c2 = (k2L) 2 , These are constants used to maintain stability, L is the dynamic range of the pixel threshold, k1 = 0.01, k2 = 0.03.

[0104] l madv The expression is:

[0105] Among them, D inf For thermal infrared image discriminators, D vis For visible light image discriminators, This is the dataset for the generated fused images.

[0106] The loss function of the visible light image discriminator is:

[0107]

[0108] In the formula, P vis For a dataset of visible light images, P penalty For P vis and The region between, where Ψ is the penalty parameter.

[0109] The discriminator's loss function uses the Wass distance, but the Wass distance has no upper or lower bounds. This means that during training, The value will increase. The value of will become smaller and smaller, and the discriminator will fail to converge. Therefore, a penalty term is added to the loss function, adding a penalty parameter to all terms whose gradient magnitude is greater than or equal to 1, and summing all the penalty terms. This will affect D. vis The optimal solution.

[0110] The loss function of the thermal infrared image discriminator is:

[0111]

[0112] In the formula, P inf For the dataset of thermal infrared images, P penalty For P inf and The area between.

[0113] The optimization module trains the generator and discriminator separately using the training set from the original data, thereby obtaining a trained adversarial generative network.

[0114] The specific steps are as follows:

[0115] 1. Fix the discriminator, extract n visible light images and thermal infrared images from the original training data set, input them into the generator network, train the generator network, minimize the loss function, and use the trained generator network to generate the fused image.

[0116] 2. Fix the generator, input the fused image and visible light image into the visible light image discriminator, and input the fused image and thermal infrared image into the thermal infrared image discriminator. Maximize the loss function and train the two discriminators separately.

[0117] 3. Repeat the above steps to obtain the trained generative adversarial network.

[0118] The fusion module uses a trained generative adversarial network to fuse test set images, resulting in a fused image of thermal infrared and visible light images.

[0119] The detection module uses a pre-trained YOLOv5s model to detect cracks in the fused image generated by the fusion module. The YOLOv5s network structure diagram is shown below. Figure 6 As shown, the core idea of ​​YOLOv5s is to transform the object detection problem into a regression problem. It uses a neural network to predict whether an object exists in each grid cell, as well as the object's location and category. It consists of multiple convolutional and pooling layers. YOLOv5s is an efficient algorithm that can quickly and accurately detect objects in images.

[0120] The alarm module activates when the detection module detects a crack. It can display the alarm time, location information, highest temperature, average temperature, visible light image of the crack, thermal infrared image, and fused image generated by the fusion module.

[0121] In summary, this invention provides a computer vision-based method and system for detecting longitudinal tears in conveyor belts. The system includes an original image acquisition module, an original image processing module, a construction module, an optimization module, a fusion module, and a detection module. It solves the problem of being easily affected by the external environment during the detection process, and can accurately detect under various lighting and noise conditions, thus improving the stability of the detection.

[0122] It should be noted that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0123] It should be noted that, depending on the implementation needs, the various steps / components described in this application can be broken down into more steps / components, or two or more steps / components or parts of the operation of steps / components can be combined into new steps / components to achieve the purpose of this invention.

[0124] Those skilled in the art will readily understand that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting longitudinal tears in a belt conveyor based on computer vision recognition, characterized in that, The method includes: Acquire thermal infrared and visible light images of the belt; The acquired thermal infrared and visible light images of the belt are processed to make them the same size and correspond to the same belt area. A generative adversarial network is constructed, consisting of a generator and two discriminators, namely a visible light image discriminator and a thermal infrared image discriminator. The processed and corresponding thermal infrared image and visible light image are input into the generator to generate a fused image. The fused image and visible light image are input into the visible light image discriminator. The fused image and thermal infrared image are input into the thermal infrared image discriminator to train the generative adversarial network. The trained generative adversarial network is used to generate a fused image of the belt to be detected; Belt crack detection is performed using the fused image of the belt to be inspected; The generator's loss function L includes content loss, structural loss, and generative adversarial loss, expressed as: Among them, content loss The expression is: In the formula, The fused image generated by the generator. For thermal infrared images, Visible light image, H and W These are the height and width of the image, respectively. Structural loss The expression is: In the formula, SSIM stands for Structural Similarity, which is an index for measuring the similarity between two images; Generate adversarial loss The expression is: In the formula, For thermal infrared image discriminators, For visible light image discriminators, The dataset for the generated fused images; x Represents the dataset Data, Indicates the first i One data point, m For the total amount of data, For data The corresponding value; Loss function of visible light image discriminator for: In the formula, For a dataset of visible light images, For the middle and The area between From The image selected in For gradient, The penalty parameter is used; the loss function uses the Wass distance during training. The value will increase. The value of will become smaller and smaller, and the discriminator will fail to converge. Therefore, a penalty term is added to the loss function, adding a penalty parameter to all terms whose gradient magnitude is greater than or equal to 1, and summing all the penalty terms to influence . The optimal solution; Loss function of thermal infrared image discriminator for: In the formula, This is a dataset of thermal infrared images. For the middle and The area between From The image selected from [the source].

2. The method for detecting longitudinal tears in a belt conveyor based on computer vision recognition according to claim 1, characterized in that, Thermal infrared images and visible light images of the area beneath the belt were acquired using a thermal infrared imager and an industrial camera, respectively.

3. The method for detecting longitudinal tears in a belt conveyor based on computer vision recognition according to claim 2, characterized in that, A light source is placed under the belt to provide supplemental lighting in order to obtain a clear visible light image of the belt.

4. The method for detecting longitudinal tears in a belt conveyor based on computer vision recognition according to claim 1, characterized in that, The generator uses a residual network structure, and the two discriminators have the same structure, both consisting of convolutional and fully connected layers.

5. The method for detecting longitudinal tears in a belt conveyor based on computer vision recognition according to claim 1, characterized in that, The expression for structural similarity (SSIM) is as follows: Will , and Setting it to 1, we get: in, yes The average value, yes The average value, yes variance yes variance yes and covariance, and It is a digital matrix of images; , , It is a constant used to maintain stability. It is the dynamic range of the pixel threshold. k 1 and k 2 is the coefficient.

6. The method for detecting longitudinal tears in a belt conveyor based on computer vision recognition according to claim 1, characterized in that, Based on the YOLOv5s model, belt crack detection is performed using the fused image of the belt to be detected.

7. A computer vision-based detection system for longitudinal tear detection of a belt conveyor, used to implement the computer vision-based detection method for longitudinal tear detection of a belt conveyor as described in any one of claims 1 to 6, characterized in that, include: The raw image acquisition module is used to acquire thermal infrared and visible light images of the belt; The raw image processing module is used to process the acquired thermal infrared image and visible light image of the belt to make them the same size and the corresponding belt area. The construction module is used to construct the generative adversarial network, which includes a generator and two discriminators, namely a visible light image discriminator and a thermal infrared image discriminator. The optimization module is used to input the processed and corresponding thermal infrared image and visible light image into the generator to generate a fused image. The fused image and visible light image are input into the visible light image discriminator, and the fused image and thermal infrared image are input into the thermal infrared image discriminator to train the generative adversarial network. The fusion module is used to generate a fused image of the belt to be detected using the trained generative adversarial network; The detection module is used to detect belt cracks using the fused image of the belt to be detected.

8. The detection system for longitudinal tearing of a belt conveyor based on computer vision recognition according to claim 7, characterized in that, The generator's loss function L includes content loss, structural loss, and generative adversarial loss, and its expression is: Among them, content loss The expression is: In the formula, The fused image generated by the generator. For thermal infrared images, Visible light image, H and W These are the height and width of the image, respectively. Structural loss The expression is: In the formula, SSIM stands for Structural Similarity, which is an index for measuring the similarity between two images; Generate adversarial loss The expression is: In the formula, For thermal infrared image discriminators, For visible light image discriminators, The dataset for the generated fused images; x Represents the dataset Data, Indicates the first i One data point, m For the total amount of data, For data The corresponding value; Loss function of visible light image discriminator for: In the formula, For a dataset of visible light images, For the middle and The area between From The image selected in For gradient, The penalty parameter is used; the loss function uses the Wass distance during training. The value will increase. The value of will become smaller and smaller, and the discriminator will fail to converge. Therefore, a penalty term is added to the loss function, adding a penalty parameter to all terms whose gradient magnitude is greater than or equal to 1, and summing all the penalty terms to influence . The optimal solution; Loss function of thermal infrared image discriminator for: In the formula, This is a dataset of thermal infrared images. For the middle and The area between From The image selected from [the source].

Citation Information

Patent Citations

  • Longitudinal tearing infrared image data generation method of belt conveyor

    CN118411360A