Method and device for detecting defects on a belt of a belt conveyor

Abstract

This application belongs to the field of industrial equipment condition monitoring and computer vision technology, specifically relating to a method and system for detecting defects in a conveyor belt of a belt conveyor. The method includes: preprocessing an original image of the conveyor belt surface; using a YOLOv8 target detection network to detect defects in the preprocessed image, outputting the category label and position coordinates of each defect target in the original image; cropping the original image based on the position coordinates to generate a standardized defect region image with a uniform preset size; segmenting to obtain a defect segmentation mask image corresponding to the standardized defect region image; and performing a grade determination. This improves the robustness and engineering applicability of the system under complex backgrounds and multiple operating conditions.

Description

Technical Field

[0001] This application belongs to the field of industrial equipment condition monitoring and computer vision technology, specifically relating to a method and system for detecting defects in conveyor belts of belt conveyors, which is an interdisciplinary application of image processing, deep learning and industrial inspection technology. Background Technology

[0002] Belt conveyors, as key equipment for continuous material transport in industrial production, are widely used in thermal power generation, coal mining, mine transportation, and port loading and unloading. During long-term operation, conveyor belts are susceptible to factors such as material impact, friction and wear, environmental corrosion, and tension changes, resulting in various defects such as tearing, wear, holes, and misalignment. If these defects are not detected and accurately assessed in a timely manner, they will not only accelerate conveyor belt aging and increase maintenance costs, but may also lead to safety accidents such as conveyor belt breakage and material leakage, seriously affecting the safety and continuity of the production system.

[0003] Existing methods for conveyor belt defect detection mainly include manual inspection and automated detection methods based on machine vision. Manual inspection relies on human experience, resulting in low efficiency, high subjectivity, and difficulty in long-term stable implementation in dusty and complex lighting industrial environments. While machine vision-based detection methods have improved automation to some extent, they still generally suffer from the following shortcomings: different types of defects vary significantly in size and shape, leading to inconsistent defect area scales output by the target detection model, which can cause unstable feature representation when directly used for subsequent analysis; small defects are easily missed or misjudged in complex backgrounds and multi-scale conditions, making it difficult to guarantee detection accuracy; existing technologies mainly focus on determining the existence of defects, lacking quantitative analysis of defect size, shape, and other information, making it difficult to provide reliable decision-making basis for conveyor belt operation and maintenance.

[0004] Therefore, there is a need for a defect detection device and method that can accurately locate, uniformly process, finely segment, and quantitatively evaluate conveyor belt defects under complex working conditions, so as to improve the reliability and intelligence level of conveyor belt operation status monitoring. Summary of the Invention

[0005] To address the issues of diverse defect types, significant dimensional differences, and the ease with which small defects can be missed under complex working conditions, this application provides a method and system for detecting defects in conveyor belts of belt conveyors.

[0006] The first aspect of this application provides a method for detecting defects in the conveyor belt of a belt conveyor, including: The original image of the conveyor belt surface is preprocessed, and the preprocessing includes at least noise reduction, contrast enhancement and background suppression. The YOLOv8 object detection network is used to detect defects in the preprocessed image, and the category label and position coordinates of each defect target in the original image are output. The original image is cropped according to the location coordinates, the defect interest region corresponding to each defect target is extracted, and adaptive scale normalization processing is applied to the defect interest region to generate a standardized defect region image with a uniform preset size. A semantic segmentation network with an encoder-decoder structure is used to perform pixel-level segmentation on the standardized defect region image to generate a defect segmentation mask image corresponding to the standardized defect region image. Based on the defect segmentation mask image, the geometric quantization parameters of each defect target are calculated, and the severity of the defect is judged according to the preset grading standard, generating a detection result that includes defect category, location coordinates, geometric quantization parameters and severity level. Based on the severity level, a graded alarm signal is output, and the detection results are displayed visually.

[0007] Furthermore, adaptive scale normalization is performed on the region of interest of the defect, including: The defect bounding box output by the YOLOv8 object detection network is used as the cropping basis. The defect bounding box is jointly defined by the x-coordinate of the upper left corner, the y-coordinate of the upper left corner, the x-coordinate of the lower right corner, and the y-coordinate of the lower right corner of the defect region in the original image. A rectangular region bounded by the coordinates of the upper left corner and the lower right corner is extracted from the original image and used as the region of interest image for the defect. The ratios between the original width and original height of the defect region of interest image and the preset standard width and standard height are used as scaling coefficients to map the coordinates of each pixel position in the defect region of interest image. The mapped coordinates are then discretized by rounding down to generate a standardized defect region image with the standard width and standard height.

[0008] Furthermore, the pixel-level segmentation includes: The standardized defect region image is input into a semantic segmentation network with an encoder-decoder structure to obtain the predicted probability that each pixel belongs to the defect region. The predicted probability of each pixel is compared with a preset or adaptively determined segmentation threshold; Pixels with a predicted probability greater than or equal to the segmentation threshold are identified as defective pixels, and the pixel value at the corresponding position in the defect segmentation mask image is set to the first logical value. Pixels with a predicted probability less than the segmentation threshold are identified as non-defective pixels, and the pixel value at the corresponding position in the defect segmentation mask image is set to the second logical value.

[0009] Furthermore, the pixel-level segmentation includes: The standardized defect region image is input into a semantic segmentation network with an encoder-decoder structure to obtain the predicted probability that each pixel belongs to the defect region. The predicted probability of each pixel is compared with a preset or adaptively determined segmentation threshold; Pixels with a predicted probability greater than or equal to the segmentation threshold are identified as defective pixels, and the pixel value at the corresponding position in the defect segmentation mask image is set to the first logical value. Pixels with a predicted probability less than the segmentation threshold are identified as non-defective pixels, and the pixel value at the corresponding position in the defect segmentation mask image is set to the second logical value.

[0010] Furthermore, the predicted probability is obtained by performing probability normalization on the feature mapping value using an activation function; The feature mapping value is obtained by summing the product of the pixel grayscale feature value and the first weight coefficient, and the product of the local gradient feature value and the second weight coefficient.

[0011] Furthermore, the calculation of the geometric quantization parameters for each defect target includes: Connectivity analysis is performed on the defect segmentation mask image. The connectivity analysis adopts a region growth method based on pixel adjacency. When the pixel distance between any two defect pixels in the mask image is less than or equal to a preset connectivity threshold, the two are classified into the same connected region to obtain the defect pixel set corresponding to each defect target. The actual area of ​​each defective target is the product of the total number of pixels contained in each defective pixel set and the actual physical area corresponding to each unit pixel. For each defect pixel set, a minimum bounding rectangle is fitted. The length and width of each defect target are obtained by multiplying the length of the long side and the length of the short side of the fitted minimum bounding rectangle by the ratio between the pixel size and the actual physical size.

[0012] Furthermore, the determination of the severity of the defect includes: When the actual area of ​​the defective target is less than the first area threshold and the actual length is less than the first length threshold, the defect is determined to be of a mild level. When the actual area of ​​the defective target is greater than or equal to the first area threshold and less than the second area threshold, or the actual length is greater than or equal to the first length threshold and less than the second length threshold, the defect is determined to be of medium severity. When the actual area of ​​the defective target is greater than or equal to the second area threshold, or the actual length is greater than or equal to the second length threshold, the defect is determined to be of severe level. The first area threshold, the second area threshold, the first length threshold, and the second length threshold are preset or adjustable based on at least one of the following factors: conveyor belt model, operating conditions, and safety requirements.

[0013] Furthermore, the YOLOv8 target detection stage performs spatial localization and category recognition of defect targets on the conveyor belt surface image from a global perspective; The pixel-level segmentation stage performs pixel-level morphological modeling and boundary extraction of the defect target from a local perspective; The adaptive scale normalization processing of the defect region of interest is set between the YOLOv8 target detection stage and the pixel-level segmentation stage. It actively guides the input region and input scale of the pixel-level segmentation stage by using the spatial location information and scale information output by the YOLOv8 target detection stage, so as to achieve scale coordination and spatial coordination between global detection and local segmentation. The grade determination integrates the category information output from the YOLOv8 target detection stage with the spatial distribution information output from the pixel-level segmentation stage, and combines it with the geometric quantization parameters to comprehensively determine the severity of defects.

[0014] A second aspect of this application provides a defect detection device for a belt conveyor belt, comprising: The main body of the belt conveyor includes a single idler, a conveyor belt, and a support frame. The single idler is located below the conveyor belt, and the axial direction of the single idler is perpendicular to the running direction of the conveyor belt. The support frame is composed of a vertical bracket and a truss structure, and the bottom of the support frame is provided with anchor bolts for adjusting the levelness. An industrial area scan camera is mounted directly above the conveyor belt via an adjustable bracket. The distance and angle between the industrial area scan camera and the surface of the conveyor belt are adjustable so that the optical axis of the industrial area scan camera is perpendicular to the surface of the conveyor belt and the field of view of the industrial area scan camera covers the effective detection width of the conveyor belt. The industrial area scan camera is communicatively connected to an industrial computer. The industrial computer has a computer program stored inside it, which, when executed by a processor, implements the method described in the above embodiments.

[0015] Compared with the prior art, the beneficial effects of this application are as follows: This application makes targeted improvements to the detection process structure and information processing method to address the problems of diverse defect types, significant size differences and easy omission of small-sized defects in conveyor belts under complex working conditions.

[0016] Compared to similar techniques that only employ object detection networks or single visual models, this application constructs a multi-stage visual collaborative processing flow consisting of YOLOv8 defect detection, defect region of interest guidance, fine segmentation, and defect quantification analysis. Each processing stage interacts and fuses results around the same defect object, avoiding the information fragmentation between stages in traditional serial processing methods and improving the consistency and reliability of the overall detection results. Compared to similar techniques that directly segment or analyze the entire image, this application introduces a defect region of interest guidance and scale coordination mechanism based on defect detection results. This ensures that defect regions of different sizes and shapes are uniformly standardized to a standard input scale while maintaining their geometric features, significantly reducing the impact of multi-scale differences on the accuracy of subsequent fine segmentation, especially improving the accuracy of small-sized defects. The stability of defect segmentation is improved. Compared with similar patents that only output defect location or category information, this application further performs pixel-level quantitative analysis on the defect area based on fine segmentation. By calculating geometric parameters such as the area, length and width of the defect, the severity of the defect is automatically determined, thus improving the traditional judgment of whether a defect exists to the assessment of the degree of defect, enhancing the support of the detection results for the operation and maintenance decision of the conveyor belt. Compared with the existing technology where each processing module works independently, this application integrates the global information of the defect detection stage with the local information of the fine segmentation stage through a multi-stage visual collaborative processing method, and they jointly participate in the defect level determination and alarm decision, effectively reducing the impact of misjudgment by a single model on the final result, and improving the robustness and engineering applicability of the system under complex backgrounds and multiple working conditions. Attached Figure Description

[0017] Figure 1 A perspective view of a belt conveyor belt defect detection device provided in an embodiment of this application; Figure 2 A defect diagram of a belt conveyor belt provided in an embodiment of this application; Figure 3 This is an overall flowchart of the defect detection method based on multi-stage visual collaborative processing provided in the embodiments of this application; Figure 4 A flowchart of defect detection and region of interest extraction provided in this application embodiment; Figure 5 This is a flowchart of fine segmentation and defect determination provided in the embodiments of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] See Figure 1 As shown in the embodiment of this application, a belt conveyor defect detection device is provided, including a belt conveyor body, an industrial area array camera 7, an industrial computer 1, and an integrated experimental platform 2. Each component works together through mechanical connection and data communication.

[0020] The main body of the belt conveyor includes a single idler roller 5, a wear-resistant conveyor belt 4, and a support frame 6. The single idler roller 5 is installed at a pre-set support position below the conveyor belt 4. Its surface has anti-slip and wear-resistant properties. The axis of the single idler roller 5 is perpendicular to the running direction of the conveyor belt 4, serving to provide stable support and guidance for the conveyor belt 4. The wear-resistant conveyor belt 4 is made of industrial-grade rubber material, with an anti-slip texture on its surface. Typical defects such as tears, wear, holes, and misalignment can be artificially pre-fabricated to simulate actual operating conditions. The support frame 6 consists of a vertical bracket and a truss structure. Adjustable anchor bolts are installed at the bottom for fixing and adjusting the overall levelness. The support frame 6 also houses the drive mechanism, tensioning mechanism, and idler roller assembly.

[0021] The industrial area scan camera 7 is a CMOS type industrial area scan camera. It is fixedly mounted above the conveyor belt 4 via a camera bracket. The height and angle of the camera bracket can be adjusted as needed to ensure that the optical axis of the industrial area scan camera 7 is substantially perpendicular to the surface of the conveyor belt 4, and that the field of view of the industrial area scan camera 7 completely covers the effective detection width of the conveyor belt 4. The industrial area scan camera 7 is connected to the industrial computer 1 via a gigabit Ethernet interface. It supports both continuous shooting and timed trigger shooting modes to acquire dynamic image sequences of the surface of the conveyor belt 4, adapting to image acquisition needs under different operating speeds and experimental scenarios.

[0022] The industrial computer 1 is fixedly installed on the bracket of the integrated experimental platform 2. The industrial computer 1 is pre-installed with conveyor belt defect detection software, which is used to receive the conveyor belt surface image data collected by the industrial area array camera 7 in real time, and sequentially complete the processing flow such as image preprocessing, defect detection, defect region of interest extraction and standardization, fine segmentation, defect quantitative analysis and alarm display.

[0023] The integrated experimental platform 2 adopts a stainless steel rust-proof and non-slip tabletop structure. The tabletop of the integrated experimental platform 2 has multiple standard mounting holes for fixing the support frame 6, the bracket of the industrial computer 1, and the camera bracket. A control panel 3 is located on one side of the integrated experimental platform 2. The control panel 3 is used to control the start and stop of the conveyor, adjust the running speed, and switch the camera's working mode, providing an intuitive and convenient operating interface for experimental personnel.

[0024] The wear-resistant conveyor belt 4 is made of industrial-grade rubber material, and its surface is provided with anti-slip texture. Various typical defects can be pre-formed on the surface of the wear-resistant conveyor belt 4, see [link / reference]. Figure 2 The images shown include tear defects (a), hole defects (b), and wear defects (c), used to simulate various damage states of conveyor belts under actual operating conditions.

[0025] An industrial computer, internally storing a computer program, which, when executed by a processor, implements a method for detecting defects in a conveyor belt. It includes: an image preprocessing module, used to perform noise reduction, contrast enhancement, and background suppression processing on the acquired raw image of the conveyor belt surface, and output a preprocessed image; The YOLOv8 defect detection module is used to receive the preprocessed image, identify and classify the defect targets in the preprocessed image, and output the category label and position coordinates of each defect target in the original image. The defect region of interest guidance and scale coordination module is used to receive the location coordinates, crop the defect target in the original image according to the location coordinates to extract the defect region of interest, and perform adaptive scale normalization processing on the defect region of interest to generate a normalized defect region image. The defect region of interest guidance and scale coordination subsystem serves as a guidance unit connecting the YOLOv8 defect detection subsystem and the fine segmentation subsystem, and actively constrains the input region range and input image scale of the fine segmentation subsystem using the location coordinates and scale information. The fine segmentation module is used to receive the standardized defect region image, perform pixel-level segmentation on the standardized defect region image using a deep learning network with an encoder-decoder structure, and output a defect segmentation mask image. The defect fusion and judgment module is used to receive the category label and the defect segmentation mask image, calculate the quantization parameters of the defect based on the defect segmentation mask image, automatically judge the severity of the defect according to the preset defect level classification standard, and output the detection result including defect category, location coordinates, quantization parameters and severity level. The alarm and display module is used to receive the detection results, trigger the corresponding alarm mode according to the severity level, and visualize the detection results.

[0026] Please see Figure 3 As shown, this application provides a method for detecting defects in a conveyor belt of a belt conveyor based on the above-described apparatus, comprising: S101, preprocess the acquired raw image of the conveyor belt surface. The preprocessing includes at least noise reduction, contrast enhancement and background suppression. S102, the YOLOv8 target detection network is used to detect defects in the preprocessed image, and the category label and position coordinates of each defect target in the original image are output. S103, the original image is cropped according to the position coordinates, the defect interest region corresponding to each defect target is extracted, and the defect interest region is subjected to adaptive scale normalization processing to generate a standardized defect region image with a uniform preset size. S104, using a semantic segmentation network with an encoder-decoder structure, the standardized defect region image is segmented at the pixel level to generate a defect segmentation mask image corresponding to the standardized defect region image; S105, Based on the defect segmentation mask image, calculate the geometric quantization parameters of each defect target, and determine the severity of the defect according to the preset level classification standard, generating a detection result containing defect category, location coordinates, geometric quantization parameters and severity level. S106, output a graded alarm signal according to the severity level, and visualize the detection results.

[0027] During the inspection process, an industrial area array camera first continuously acquires images of the conveyor belt surface. The acquired raw images are then processed by an image preprocessing module to remove noise, enhance, and suppress background, in order to reduce the impact of complex working conditions and environmental interference on visual analysis.

[0028] The preprocessed image is input into the YOLOv8 defect detection module. The YOLOv8 target detection network in the YOLOv8 defect detection module first performs global defect detection on the conveyor belt surface image (stage one) to obtain the defect category information and its spatial location coordinates in the original image.

[0029] Subsequently, in the second stage, the defect region of interest guidance and scale coordination stage, based on the category labels corresponding to each defect target output by the detection stage and their position coordinates in the original image, the defect region is guided to be cropped, and the cropped defect region of interest is subjected to adaptive scale normalization processing, thereby forming a standardized defect region image with a unified scale. This provides a constrained and stable input for the fine segmentation stage, realizing scale coordination between the defect detection stage and the fine segmentation stage.

[0030] Building upon this foundation, Stage Three: Fine Segmentation Stage uses the standardized defect region image as input to perform pixel-level fine segmentation of the defect region, obtaining precise spatial distribution information of the defects. Subsequently, Stage Four: Defect Fusion and Judgment Stage jointly analyzes the category labels output from the YOLOv8 defect detection stage and the defect segmentation mask image output from the fine segmentation stage, calculating geometric quantization parameters such as the area, length, and width of the defects, and determining the severity level of the defects based on these geometric quantization parameters, thus achieving collaborative decision-making among the results of multiple stages of processing.

[0031] Finally, the defect level determination result is transmitted to the alarm and display module, which triggers the corresponding alarm mode according to the severity of the defect, and visualizes the defect category, location and quantitative parameters, thus forming a complete closed-loop process from multi-stage visual collaborative processing to engineering decision output.

[0032] In step S101, in the defect detection method, the image preprocessing module in the device serves as the preprocessing unit for the entire detection process, and is used to receive the original image data of the conveyor belt surface acquired and transmitted by the front-end industrial area array camera.

[0033] The image preprocessing module denoises the original image to reduce the impact of illumination changes, dust interference, and imaging noise on image quality; it also performs contrast enhancement to improve the grayscale difference between defect areas and background areas; and it reduces the interference of non-defect areas on subsequent detection and segmentation processes through background suppression.

[0034] The above preprocessing operations provide stable and clear image input for subsequent YOLOv8 defect detection and standardized processing and fine segmentation of defect regions of interest, thereby improving the robustness and accuracy of the overall detection process.

[0035] In step S102, the YOLOv8 object detection network adopts a deep learning object detection architecture. Its backbone network adopts a cross-stage local network structure to extract multi-scale feature maps from the preprocessed image; the neck network adopts a feature pyramid structure to fuse feature information of different scales to enhance the network's ability to perceive defective targets of various sizes; the head network adopts a decoupled detection head structure to output the category prediction branch and bounding box regression branch of the defective target respectively.

[0036] During the detection process, the YOLOv8 object detection network first performs convolutional feature extraction on the preprocessed image to generate multi-scale feature maps; then, it performs dense sampling on the feature maps at each scale using a sliding window method to generate multiple anchor boxes or anchor points; subsequently, it performs class prediction and bounding box regression simultaneously on each anchor box region, outputting the class confidence score and bounding box offset of each candidate region; finally, it filters redundant candidate boxes using the non-maximum suppression (NMS) algorithm, retaining candidate boxes with confidence scores higher than a preset threshold as the final detection result.

[0037] Through the above detection process, the YOLOv8 target detection network outputs the category label and position coordinates of each defect target in the original image. The category label includes at least one or more of the following: tear category, wear category, hole category, and deviation category, used to identify the type attribute of the defect; the position coordinates are represented in the form of a defect bounding box, which is jointly defined by the x-coordinate of the upper left corner, y-coordinate of the upper left corner, x-coordinate of the lower right corner, and y-coordinate of the lower right corner of the defect region in the original image, used to identify the spatial location and approximate range of the defect target in the original image.

[0038] See Figure 3 As shown, let the original input image be I, and the defect bounding box output by the YOLOv8 defect detection module be represented as: , in , and , These represent the coordinates of the top left and bottom right corners of the defect area in the original image, respectively.

[0039] In step S103, after receiving the defect bounding box information output by the YOLOv8 defect detection module, the defect region of interest guidance and scale coordination module first performs a defect region of interest extraction operation. Specifically, let the original image be I, and the defect bounding box output by the YOLOv8 defect detection module is jointly defined by the x-coordinate of the upper left corner, the y-coordinate of the upper left corner, the x-coordinate of the lower right corner, and the y-coordinate of the lower right corner of the defect region in the original image. Using the bounding box as the cropping basis, a rectangular region enclosed by the upper left and lower right corner coordinates is extracted from the original image I to obtain the defect region of interest image. This cropping operation can be represented as extracting all pixels within the coordinate range from the original image to form an independent defect region of interest image.

[0040] The defect region of interest image contains the defect itself and its adjacent local background, while effectively removing large areas of redundant background far from the defect in the original image.

[0041] Region of interest image of the defect subject Represented as: , Adaptive scale normalization is performed on the region of interest image of the defect. This includes: The defect bounding box output by the YOLOv8 object detection network is used as the cropping basis. The defect bounding box is jointly defined by the x-coordinate of the upper left corner, the y-coordinate of the upper left corner, the x-coordinate of the lower right corner, and the y-coordinate of the lower right corner of the defect region in the original image. A rectangular region bounded by the coordinates of the upper left corner and the lower right corner is extracted from the original image and used as the region of interest image for the defect. The ratios between the original width and original height of the defect region of interest image and the preset standard width and standard height are used as scaling coefficients to map the coordinates of each pixel position in the defect region of interest image. The mapped coordinates are then discretized by rounding down to generate a standardized defect region image with the standard width and standard height.

[0042] Because different types of defects vary significantly in size and shape—for example, tear defects are often elongated strips, reaching hundreds of pixels in length, while wear defects and hole defects typically appear as smaller localized areas—directly inputting regions of interest (ROIs) of defects with varying original sizes into subsequent fine-grained segmentation modules will lead to the following technical problems: First, deep learning segmentation networks are sensitive to input scale; inputs of different scales will cause a mismatch between the receptive field and the defect size during feature extraction, especially for small defects, whose features are easily submerged by background information in large-scale inputs; second, the network needs to adapt to different scale input distributions, reducing the stability of model training and the consistency of segmentation accuracy. Therefore, this application embodiment performs adaptive scale normalization processing on the aforementioned ROIs.

[0043] The defect region of interest (ROI) serves as a guiding unit bridging the YOLOv8 object detection stage and the pixel-level segmentation stage. It utilizes the positional coordinates and defect scale information output from the YOLOv8 object detection stage to actively constrain the input region range and input image scale of the pixel-level segmentation stage. This ensures that the pixel-level segmentation stage performs segmentation only within the local regions where each defect target is located, and completes feature extraction under uniform scale conditions. Let the defect be the region of interest image. The original width and original height are respectively and The preset standardized output width and height are respectively and Through scaling transformation function The region of interest image of the defect is scaled to obtain a standardized defect region image. ,Right now: , , in, To standardize the pixel coordinates in the defect area image.

[0044] Scale transformation function The original size is The image is mapped to a size of The image. In one specific embodiment, the scaling transformation function discretizes the mapped coordinates using a floor function, and its mapping rule is: standardize any pixel in the defect region image. The pixel value is equal to the pixel value at the corresponding mapped position in the original defect region of interest image. The x-coordinate of the mapped position is the ratio of the original width to the normalized width multiplied by the x-coordinate of the point and then rounded down. The y-coordinate of the mapped position is the ratio of the original height to the normalized height multiplied by the y-coordinate of the point and then rounded down.

[0045] In one example, the scaling transformation function can discretize the mapped coordinates by rounding down, and its expression is: , in, This indicates the floor function.

[0046] Through the above-mentioned scale standardization process, defect regions of different sizes are uniformly mapped to the same scale image space while maintaining their geometric morphological features. This enhances the ability of the subsequent fine segmentation module to express small-sized defect features and improves the stability and consistency of the segmentation results.

[0047] See Figure 4 As shown, after completing scale guidance and scale co-processing of the defect region of interest, the standardized defect region image is input into the fine segmentation module to perform pixel-level segmentation of the defect region.

[0048] In one embodiment, pixel-level segmentation includes: The standardized defect region image is input into a semantic segmentation network with an encoder-decoder structure to obtain the predicted probability that each pixel belongs to the defect region. The predicted probability of each pixel is compared with a preset or adaptively determined segmentation threshold; Pixels with a predicted probability greater than or equal to the segmentation threshold are identified as defective pixels, and the pixel value at the corresponding position in the defect segmentation mask image is set to the first logical value. Pixels with a predicted probability less than the segmentation threshold are identified as non-defective pixels, and the pixel value at the corresponding position in the defect segmentation mask image is set to the second logical value.

[0049] The encoder-decoder semantic segmentation network used in the fine segmentation module consists of two parts: an encoder and a decoder. The encoder employs a deep convolutional neural network structure, composed of multiple convolutional layers and pooling layers stacked sequentially. Specifically, the encoder extracts features from the input standard defect region image through several sets of convolution-batch normalization-activation function operations, and then gradually reduces the spatial resolution of the feature map and expands the receptive field through pooling layers, enabling the network to capture the contextual semantic information and macroscopic morphological features of the defect region. After multiple downsampling layers, the encoder outputs a multi-scale feature map containing high-level semantic information.

[0050] The decoder is communicatively connected to the encoder, and its structure is a mirror image of the encoder structure. The decoder sub-network first progressively restores the spatial resolution of the multi-scale feature map through deconvolutional or upsampling layers. During each upsampling stage, it fuses the high-resolution detail features retained by the corresponding encoder layer through skip connections. This allows the decoder to obtain sufficient semantic guidance while restoring the spatial details of the image, thereby generating a pixel-level probability map with the same size as the input image in the final output layer. Each pixel position in the probability map... The value of represents the predicted probability that the pixel belongs to the defect area, and the value range is [0,1].

[0051] The predicted probability of each pixel is compared with a preset or adaptively determined segmentation threshold. The comparison is performed, and a binarized defect segmentation mask image is generated based on the comparison results. The specific judgment logic is as follows: When pixel position The predicted probability is greater than or equal to the segmentation threshold. When the pixel-level discrimination unit determines that the pixel belongs to the defect region, it sets the pixel value at the corresponding position (x,y) in the defect segmentation mask image M to the first logical value (e.g., 1). When pixel position Predicted probability Less than the segmentation threshold When the pixel-level discrimination unit determines that the pixel belongs to a non-defect area (i.e., a normal conveyor belt surface area), it will add the corresponding position in the defect segmentation mask image M. The pixel value is set to the second logical value (e.g., 0). The above can be expressed as a formula: , in, Represents pixels The predicted probability of belonging to a defective region. The segmentation threshold is either preset or adaptive.

[0052] The predicted probability is obtained by normalizing the feature mapping values ​​using an activation function; The feature mapping value is obtained by summing the product of the pixel grayscale feature value and the first weight coefficient, and the product of the local gradient feature value and the second weight coefficient. , This is a pixel-level probability mapping function.

[0053] In the formula F is a feature mapping function used to map the local features of a pixel to a real number space; it is a pixel-level segmentation mapping function. The function is defined as a mapping function based on pixel probability discrimination, and its specific form is: , The feature mapping function is used to map the local features of a pixel to a real number space. Its calculation method can be based on a weighted combination of pixel grayscale features and local gradient features, and its expression is: , in , and These are the weighting coefficients. These are pixel grayscale feature values. These are local gradient feature values. In other embodiments, the feature mapping function can also be expressed as a weighted combination of multiple feature extraction operators, and its specific form is not limited to the expression described above.

[0054] The fine segmentation module uses a pixel-level mapping function F( This maps the input image to the corresponding defect segmentation result. .in The defect segmentation mask image is: , in, Represents pixels This area is a defect. Represents pixels It belongs to the non-defect area.

[0055] Through the above pixel-level fine segmentation process, the boundary between defective and normal regions can be clearly distinguished, and a standardized segmentation mask image is finally output.

[0056] In an embodiment, the segmentation threshold This is a preset fixed threshold, such as 0.5. This fixed threshold can be predetermined based on the statistical distribution of the training dataset and is suitable for application scenarios where the differences in features between defective and normal regions are significant and the distribution is stable.

[0057] In another embodiment, the segmentation threshold To adaptively determine the threshold, the pixel-level discrimination unit dynamically calculates the optimal segmentation threshold based on the statistical characteristics of the input probability map, maximizing the inter-class variance between defective and non-defective pixels, thereby achieving adaptive adjustment of the segmentation threshold for each input image. This adaptive method has stronger robustness for scenarios with large variations in defect morphology and fluctuations in defect contrast due to varying operating conditions.

[0058] Through the pixel-level discrimination processing described above, the fine segmentation module outputs a binarized defect segmentation mask image M. This image has the same width W and height H as the standardized defect region image. In the segmentation mask image M, all pixel positions with a first logical value (i.e., 1) constitute the defect pixel set, representing the precise spatial distribution of the defect region in the standardized image space; all pixel positions with a second logical value (i.e., 0) constitute the non-defect pixel set, representing the normal conveyor belt surface area.

[0059] In the defect segmentation mask image M, the boundary between the defect region and the normal region is defined by the transition of adjacent pixel values ​​(i.e., the position from 1 to 0 or from 0 to 1), thus clearly distinguishing the geometric boundary between the defect region and the normal region and completely preserving the morphological and structural information of the defect. The above pixel-level fine segmentation results can accurately reflect the edge contour and spatial distribution of the defect, providing a high-precision data foundation for the subsequent defect quantification analysis unit.

[0060] The defect fusion and judgment module first processes the received defect segmentation mask image. Connectivity analysis is performed to obtain the independent pixel set of each defect target in the standardized image space. This connectivity analysis is implemented using a region growing method based on pixel adjacency relationships: for the defect segmentation mask image... For any two defect pixels (i.e., pixels with the first logical value), if the pixel distance between them is less than or equal to a preset pixel connectivity threshold, they are grouped into the same connected region. The pixel connectivity threshold can be preset according to the actual application scenario; for example, setting τ=1 indicates the use of a 4-neighborhood connectivity method. Through the above connected component analysis, the mask image M is divided into several independent connected regions, each corresponding to an independent defect target. When multiple defects exist in the same standardized defect region image, each defect forms its own independent connected region. By performing connected component analysis on the defect segmentation mask image M, the set of independent pixels corresponding to the defect region is represented as: , in, This represents the set of connected regions composed of defective pixels. Connectivity analysis function. This can be achieved using a region growing method based on pixel adjacency relationships, which can be represented as: , in, This is the pixel spacing coefficient. This is the pixel connectivity threshold.

[0061] When multiple defect regions exist in the same image, the multiple defect regions are represented as follows: , in, Indicates the first There are 1 defect region, and after connected component analysis, N independent defect regions are obtained. Each independent defect region contains all pixel location information of the defect in the normalized image space.

[0062] Based on this, the defect category labels and location coordinates output by the YOLOv8 defect detection module are mapped one-to-one with each connected region. Since the standardized defect region image is generated by cropping and scaling the original defect bounding box region, a definite geometric transformation relationship exists between the pixel-level spatial position of the defect region and its location coordinates in the original image. Based on the inverse mapping relationship of the scaling transformation function, each connected region is mapped back to the original image coordinate system, thereby determining the precise location coordinates of each defect target in the original image.

[0063] Based on the connected component information corresponding to each defect region and its mapping position in the original image, the geometric quantization parameters of each defect target are calculated.

[0064] The geometric quantification parameters include at least the actual area, actual length, and actual width of the defect.

[0065] The actual area of ​​each defective target is the product of the total number of pixels contained in each defective pixel set and the actual physical area corresponding to each unit pixel. Let s be the actual physical area corresponding to a unit pixel (unit: square millimeters / pixel or square centimeters / pixel). This unit pixel area is obtained through camera calibration on a calibration object of known dimensions on the conveyor belt surface. Its value depends on parameters such as the imaging resolution, shooting distance, and lens focal length of the industrial area array camera. For the th... The defective area, its actual area It equals the total number of pixels contained in the defective region multiplied by the actual physical area corresponding to each pixel. Right now: , This indicates the total number of pixels contained in the defective region.

[0066] Through the above calculations, the defect area is expressed in physical size units (e.g., square millimeters), which is independent of differences in image resolution and has scale invariance.

[0067] For the For each defective region, a minimum bounding rectangle is first fitted to the pixel set of that defective region. The minimum bounding rectangle fitting uses the rotating caliper algorithm or principal component analysis to calculate the rectangle that can completely enclose the defective region with the smallest area, and then determines the orientation, length of the long side, and length of the short side of the rectangle.

[0068] , in, This is the scaling factor between pixel size and actual physical size. The actual physical width, This is the actual width. The actual physical length This is the actual length.

[0069] Determining the severity of defects includes: When the actual area of ​​the defective target is less than the first area threshold and the actual length is less than the first length threshold, the defect is determined to be of a mild level. When the actual area of ​​the defective target is greater than or equal to the first area threshold and less than the second area threshold, or the actual length is greater than or equal to the first length threshold and less than the second length threshold, the defect is determined to be of medium severity. When the actual area of ​​the defective target is greater than or equal to the second area threshold, or the actual length is greater than or equal to the second length threshold, the defect is determined to be of severe level. The first area threshold, the second area threshold, the first length threshold, and the second length threshold are preset or adjustable based on at least one of the following factors: conveyor belt model, operating conditions, and safety requirements.

[0070] In this embodiment, the severity of the defect is divided into three levels—minor, moderate, and severe—based on its impact on the safe operation of the conveyor belt. The definitions of each level are as follows: A minor defect refers to a defect that is relatively small in size and has a limited impact on the conveyor belt's load-bearing capacity and operational stability. Such defects typically correspond to minor damage or surface wear in the early stages of conveyor belt operation and will not pose a substantial threat to the safe operation of the conveyor belt in the short term. However, they need to be included in the operational monitoring scope to track their evolution.

[0071] A moderate defect level refers to a defect whose size has reached a significant level, having a noticeable impact on the local load-bearing capacity and operational reliability of the conveyor belt, but not yet reaching the level of urgency requiring immediate shutdown. Such defects require maintenance or local repairs to be scheduled within the planned maintenance cycle to prevent them from further escalating into serious defects.

[0072] A severe defect level refers to a defect whose size exceeds the maximum allowable range for safe operation, posing a serious threat to the structural integrity and operational safety of the conveyor belt. If such defects are not addressed promptly, they are highly likely to cause safety accidents such as conveyor belt breakage, tearing and propagation, or material leakage during operation. Therefore, it is necessary to immediately trigger a high-level alarm and arrange for an emergency shutdown.

[0073] The grading unit automatically grades each defect target according to the grading logic. This is expressed by the formula: , in, , , These represent the severity levels of defects: mild, moderate, and severe, respectively. , , , This is a preset threshold parameter.

[0074] The grading function is preset or adjusted based on the degree of impact of defects on the safety of conveyor belt operation, thereby achieving automatic grading assessment of defects. The threshold parameters can be adjusted according to the conveyor belt model, operating conditions, or safety requirements.

[0075] Specifically, for tear-type defects, due to the influence of the longitudinal tension of the conveyor belt during operation, they are extremely prone to rapidly propagate along the tear tip and cause complete breakage of the conveyor belt. Their hazard to conveyor belt operation safety is significantly higher than other types of defects of the same size. Therefore, for tear defects, the grading unit uses more stringent threshold parameters, ensuring that tear defects are classified as moderate or severe even when their size is still relatively small, thus triggering higher levels of attention and more timely maintenance responses.

[0076] For hole-type defects and wear-type defects, their expansion rate under stress conditions is usually slower than that of tear defects, and their harm is relatively low. Therefore, a more lenient threshold parameter can be used, or the area and length thresholds can be adjusted by setting a category weight coefficient.

[0077] After determining the level of each defect target, the determination result is integrated with the defect category, location coordinates, and geometric quantification parameters provided by the quantification analysis unit to generate a complete inspection report and output it to the alarm and display module. The data structure of each defect entry in the inspection report includes at least the following fields: Defect ID: A unique identifier generated according to the order of defect detection or spatial location; Defect category: One of the following: tear, wear, hole, or misalignment; Defect location coordinates: The pixel coordinates of the defect center in the original image or the actual physical coordinates converted to the surface of the conveyor belt; Defect area: The measured area value in square millimeters; Defect length: The measured length value in millimeters; Defect width: Measured width value in millimeters; Defect severity level: one of mild, moderate or severe; Defect level determination criteria: The threshold parameters involved in the determination and the comparison results facilitate maintenance personnel to trace the determination logic.

[0078] The generated defect detection report is transmitted to the alarm and display module for visualizing the detection results and providing tiered alarms. The alarm and display module triggers corresponding alarms based on the severity of the defect and displays the defect type, location coordinates, and quantitative parameters on the monitoring interface, allowing staff to promptly grasp the conveyor belt's operating status and take appropriate action.

[0079] In this embodiment, multi-stage visual collaborative processing is not a simple serial superposition of multiple visual algorithms, but rather a collaborative process that guides information and fuses results between different visual processing stages, enabling each processing stage to work together around the same defective object, thereby improving the overall stability and accuracy of defect detection and assessment.

[0080] The YOLOv8 target detection stage performs spatial localization and category recognition of defect targets from a global perspective on the surface image of the conveyor belt. The pixel-level segmentation stage performs pixel-level morphological modeling and boundary extraction of the defect target from a local perspective; The adaptive scale normalization processing of the defect region of interest is set between the YOLOv8 target detection stage and the pixel-level segmentation stage. It actively guides the input region and input scale of the pixel-level segmentation stage by using the spatial location information and scale information output by the YOLOv8 target detection stage, so as to achieve scale coordination and spatial coordination between global detection and local segmentation. The grade determination integrates the category information output from the YOLOv8 target detection stage with the spatial distribution information output from the pixel-level segmentation stage, and combines it with the geometric quantization parameters to comprehensively determine the severity of defects.

[0081] Specifically, the YOLOv8 defect detection stage analyzes the conveyor belt surface image from a global perspective, primarily to quickly locate the approximate spatial position of defects and determine defect category information. The fine segmentation stage, on the other hand, performs pixel-level modeling of the defect region from a local perspective, mainly to obtain the precise shape and boundary information of the defects. To avoid information fragmentation between global detection and local analysis, this embodiment introduces a defect region of interest guidance and scale coordination stage between the two, serving as a key collaborative unit connecting different visual processing stages.

[0082] The defect region of interest guidance and scale coordination stage utilizes the spatial location and scale information output from the YOLOv8 defect detection stage to actively guide the input region range and input scale of the fine segmentation stage. This ensures that the fine segmentation process is carried out only in regions highly related to the defect and that feature extraction is completed under uniform scale conditions. This reduces the interference of multi-scale differences and complex backgrounds on the segmentation results, and achieves scale coordination and spatial coordination between the detection stage and the segmentation stage.

[0083] Based on this, the defect fusion and judgment stage does not rely solely on the output of a single visual processing stage. Instead, it fuses and analyzes the category information provided by the YOLOv8 defect detection stage with the pixel-level spatial information provided by the fine segmentation stage, and combines the defect geometric quantification parameters to jointly participate in the level judgment of defect severity, thereby achieving information synergy and decision synergy among the results of multi-stage visual processing.

[0084] Through the aforementioned multi-stage visual collaborative working mechanism, this embodiment can achieve accurate positioning, fine segmentation, and quantitative evaluation of conveyor belt defects under complex backgrounds and multi-scale defect conditions, especially improving the stability of small-sized defect identification and the reliability of defect level determination.

[0085] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for detecting defects in a conveyor belt of a belt conveyor, characterized in that, include: The original image of the conveyor belt surface is preprocessed, and the preprocessing includes at least noise reduction, contrast enhancement and background suppression. The YOLOv8 object detection network is used to detect defects in the preprocessed image, and the category label and position coordinates of each defect target in the original image are output. The original image is cropped according to the location coordinates, the defect interest region corresponding to each defect target is extracted, and adaptive scale normalization processing is applied to the defect interest region to generate a standardized defect region image with a uniform preset size. A semantic segmentation network with an encoder-decoder structure is used to perform pixel-level segmentation on the standardized defect region image to generate a defect segmentation mask image corresponding to the standardized defect region image. By fusing category labels, location coordinates, and the defect segmentation mask image, the geometric quantization parameters of each defect target are calculated, and the severity of the defect is judged according to a preset grading standard, generating a detection result that includes defect category, location coordinates, geometric quantization parameters, and severity level. Based on the severity level, a graded alarm signal is output, and the detection results are displayed visually.

2. The method for detecting defects in a conveyor belt of a belt conveyor according to claim 1, characterized in that, Adaptive scale normalization is performed on the region of interest of the defect, including: The defect bounding box output by the YOLOv8 object detection network is used as the cropping basis. The defect bounding box is jointly defined by the x-coordinate of the upper left corner, the y-coordinate of the upper left corner, the x-coordinate of the lower right corner, and the y-coordinate of the lower right corner of the defect region in the original image. A rectangular region bounded by the coordinates of the upper left corner and the lower right corner is extracted from the original image and used as the region of interest image for the defect. The ratios between the original width and original height of the defect region of interest image and the preset standard width and standard height are used as scaling coefficients to map the coordinates of each pixel position in the defect region of interest image. The mapped coordinates are then discretized by rounding down to generate a standardized defect region image with the standard width and standard height.

3. The method for detecting defects in a conveyor belt of a belt conveyor according to claim 1, characterized in that, The pixel-level segmentation includes: The standardized defect region image is input into a semantic segmentation network with an encoder-decoder structure to obtain the predicted probability that each pixel belongs to the defect region. The predicted probability of each pixel is compared with a preset or adaptively determined segmentation threshold; Pixels with a predicted probability greater than or equal to the segmentation threshold are identified as defective pixels, and the pixel value at the corresponding position in the defect segmentation mask image is set to the first logical value. Pixels with a predicted probability less than the segmentation threshold are identified as non-defective pixels, and the pixel value at the corresponding position in the defect segmentation mask image is set to the second logical value.

4. The method for detecting defects in a conveyor belt of a belt conveyor according to claim 1, characterized in that, The defect region of interest serves as a guiding unit connecting the YOLOv8 target detection stage and the pixel-level segmentation stage. It utilizes the position coordinates and defect scale information output by the YOLOv8 target detection stage to actively constrain the input region range and input image scale of the pixel-level segmentation stage. This ensures that the pixel-level segmentation stage performs segmentation processing only within the local area where each defect target is located and completes feature extraction under uniform scale conditions.

5. The method for detecting defects in a conveyor belt of a belt conveyor according to claim 3, characterized in that, The predicted probability is obtained by normalizing the feature mapping value using an activation function; The feature mapping value is obtained by summing the product of the pixel grayscale feature value and the first weight coefficient, and the product of the local gradient feature value and the second weight coefficient.

6. The method for detecting defects in a conveyor belt of a belt conveyor according to claim 1, characterized in that, The geometric quantization parameters for calculating each defect target include: Connectivity analysis is performed on the defect segmentation mask image. The connectivity analysis adopts a region growth method based on pixel adjacency. When the pixel distance between any two defect pixels in the mask image is less than or equal to a preset connectivity threshold, the two are classified into the same connected region to obtain the defect pixel set corresponding to each defect target. The actual area of ​​each defective target is the product of the total number of pixels contained in each defective pixel set and the actual physical area corresponding to each unit pixel. For each defect pixel set, a minimum bounding rectangle is fitted. The length and width of each defect target are obtained by multiplying the length of the long side and the length of the short side of the fitted minimum bounding rectangle by the ratio between the pixel size and the actual physical size.

7. The method for detecting defects in a conveyor belt of a belt conveyor according to claim 1, characterized in that, The determination of the severity of defects includes: When the actual area of ​​the defective target is less than the first area threshold and the actual length is less than the first length threshold, the defect is determined to be of a mild level. When the actual area of ​​the defective target is greater than or equal to the first area threshold and less than the second area threshold, or the actual length is greater than or equal to the first length threshold and less than the second length threshold, the defect is determined to be of medium severity. When the actual area of ​​the defective target is greater than or equal to the second area threshold, or the actual length is greater than or equal to the second length threshold, the defect is determined to be of severe level. The first area threshold, the second area threshold, the first length threshold, and the second length threshold are preset or adjustable based on at least one of the following factors: conveyor belt model, operating conditions, and safety requirements.

8. The method for detecting defects in a conveyor belt of a belt conveyor according to claim 1, characterized in that, The YOLOv8 target detection stage performs spatial localization and category recognition of defect targets from a global perspective on the surface image of the conveyor belt. The pixel-level segmentation stage performs pixel-level morphological modeling and boundary extraction of the defect target from a local perspective; The adaptive scale normalization processing of the defect region of interest is set between the YOLOv8 target detection stage and the pixel-level segmentation stage. It actively guides the input region and input scale of the pixel-level segmentation stage by using the spatial location information and scale information output by the YOLOv8 target detection stage, so as to achieve scale coordination and spatial coordination between global detection and local segmentation. The grade determination integrates the category information output from the YOLOv8 target detection stage with the spatial distribution information output from the pixel-level segmentation stage, and combines it with the geometric quantization parameters to comprehensively determine the severity of defects.

9. A defect detection device for a belt conveyor belt, characterized in that, include: The main body of the belt conveyor includes a single idler, a conveyor belt, and a support frame. The single idler is located below the conveyor belt, and the axial direction of the single idler is perpendicular to the running direction of the conveyor belt. The support frame is composed of a vertical bracket and a truss structure, and the bottom of the support frame is provided with anchor bolts for adjusting the levelness. An industrial area scan camera is mounted directly above the conveyor belt via an adjustable bracket. The distance and angle between the industrial area scan camera and the surface of the conveyor belt are adjustable so that the optical axis of the industrial area scan camera is perpendicular to the surface of the conveyor belt and the field of view of the industrial area scan camera covers the effective detection width of the conveyor belt. The industrial area scan camera is communicatively connected to an industrial computer. The industrial computer has a computer program stored inside it, which, when executed by a processor, implements the method as described in any one of claims 1 to 8.

Images