Conveyor belt defect detection method and device based on visual prompt memory

Abstract

The invention relates to the technical field of industrial visual inspection, in particular to a conveyor belt defect detection method and device based on visual prompt memory. The method comprises the following steps: acquiring a surface image of a conveying belt, performing strip-type preprocessing, constructing a visual prompt graph, generating a prompt vector through a prompt encoder, extracting multi-scale features of the image by using a multi-scale Transform image encoder, and generating an image memory vector; a prompt vector and an image memory vector are fused through a cross attention module to form prompt memory, the prompt memory is cached and multiplexed, and a data efficient Transform decoder is adopted to carry out defect detection based on the prompt memory and multi-scale features. And finally, carrying out strip-type merging, one-dimensional band-wise non-maximum suppression and time trajectory consistency filtering on detection results, and outputting a final result. According to the method, the detection sensitivity of a key area can be improved by effectively utilizing visual prompt, the overhead is reduced through a cache mechanism, and high-precision and real-time detection is realized under a small sample.

Description

Technical Field

[0001] This invention relates to the field of industrial visual inspection technology, specifically to a method and apparatus for detecting defects in conveyor belts based on visual cue memory. Background Technology

[0002] As critical equipment in industrial settings such as mines, power plants, and ports, the timely detection of surface defects (such as cracks, delamination, scratches, and tears) in conveyor belts is essential for ensuring production safety and continuous operation. Traditional conveyor belt defect detection methods have several limitations, failing to meet the demands of modern industry for high precision, real-time performance, and reliability. Manual inspection was the earliest method, involving periodic inspections to observe defects. However, this method is labor-intensive, highly dependent on the experience of inspectors, and prone to missed detections and misjudgments due to subjective fatigue or negligence. Furthermore, it cannot achieve real-time monitoring and early warning in continuous production scenarios. With the development of image processing technology, traditional image processing methods have been introduced, including algorithms such as fixed threshold segmentation, edge detection, and template matching. However, conveyor belt surfaces have complex textures, drastic lighting variations, and are often accompanied by dust, stains, and other interference. Traditional methods are extremely sensitive to parameter settings; changes in operating conditions or conveyor belt materials lead to poor generalization ability, requiring frequent maintenance and adjustments, thus limiting applicability. In recent years, convolutional neural network-based target detection methods, such as the YOLO series, have made progress in natural scene target detection and are being attempted for conveyor belt defect detection. However, these methods face challenges in industrial applications: conveyor belt defects are often small targets such as slender cracks or small-area scratches with extreme aspect ratios, making the receptive field design of traditional convolutional neural networks insufficient for perceiving such targets; the long length of the conveyor belt and the large image acquisition range mean that high-resolution images can easily distract the network's attention to the background area, reducing detection efficiency; the number of labeled samples available in industrial scenarios is limited, constituting a small-sample problem, while convolutional neural network-based methods typically require large-scale labeled data for stable training; deepening the network to improve accuracy increases computational load, leading to inference latency and making real-time deployment on edge devices difficult. Meanwhile, Transformer-based data-efficient detection frameworks achieve high accuracy with moderate amounts of data through query mechanisms and matching optimization strategies, but existing work is mainly designed for natural scenes and does not fully consider industrial needs, such as the prior geometric structure of the conveyor belt strips, the longitudinal distribution of defects, robustness to slender defects with small samples, and the effective utilization of visual cues (such as belt areas and historically high-incidence areas). Existing methods lack independent visual cue encoding and cue memory fusion mechanisms, making it impossible to balance accuracy and efficiency in long-term online detection. Therefore, developing a method that balances few-sample learning, high accuracy, real-time performance, and conditional detection using visual cues has become a pressing technical problem in this field.

[0003] Therefore, the existing technology still needs further development. Summary of the Invention

[0004] The purpose of this invention is to overcome the above-mentioned technical deficiencies and provide a method and device for detecting defects in conveyor belts based on visual cue memory, so as to solve the problems existing in the prior art.

[0005] To achieve the above-mentioned technical objectives, according to a first aspect of the present invention, the present invention provides a conveyor belt defect detection method based on visual cue memory, comprising: S100: Acquire images of the conveyor belt surface and perform strip-type preprocessing to obtain multiple strip image blocks; S200. Construct a visual cue map and encode the visual cue map into a cue vector using a cue encoder; simultaneously, extract multi-scale features of the strip image patch using a multi-scale Transformer image encoder and generate an image memory vector. S300: The cue vector and the image memory vector are fused through a cross-attention module to generate a cue memory, and the cue memory is cached. S400. Based on the cue memory and the multi-scale features, defect detection is performed using a data-efficient Transformer decoder to obtain defect box prediction results. S500: Perform strip merging and one-dimensional band-oriented nonmaximum suppression on the defect box prediction results, and combine with time trajectory consistency filtering to output the final detection result.

[0006] Specifically, the visual cue map is derived from at least one of automatic cue and manual interactive cue, wherein the automatic cue includes a belt area mask and a weight map of historically high-defect areas.

[0007] Specifically, the cue encoder encodes the visual cue map into multiple cue tokens through pooling operations and a multilayer perceptron, forming the cue vector.

[0008] Specifically, the cross-attention module employs a multi-head scaling dot product attention mechanism, using the cue vector as the query and the image memory vector as the key and value for fusion.

[0009] Specifically, the multi-scale Transformer image encoder outputs feature maps with multiple downsampling ratios based on a lightweight Transformer backbone obtained by distillation of a self-supervised pre-trained model, and generates the multi-scale features through semantic enhancement fusion.

[0010] Specifically, the image memory vector is generated by compressing the multi-scale features through spatial pooling operations and then stacked and transformed into an image memory matrix.

[0011] Specifically, the data-efficient Transformer decoder generates an initial query through a query initialization mechanism, and then fuses the cue memory and the image memory vector in a phased cross-attention manner to update the query vector to predict the defect category and bounding box.

[0012] Specifically, the strip-type preprocessing includes cropping overlapping image blocks along the conveyor belt running direction and performing normalization, illumination compensation, and noise reduction.

[0013] Specifically, the time trajectory consistency filtering includes associating adjacent frame detection boxes with targets to form trajectories, and filtering single-frame or short trajectory detection boxes to reduce false alarms.

[0014] According to a second aspect of the present invention, a conveyor belt defect detection device based on visual cue memory is provided, comprising: The image acquisition module is used to acquire images of the conveyor belt surface. The preprocessing module is used to perform strip preprocessing on the image; The visual cue encoding module is used to construct visual cue maps and encode them into cue vectors; The image feature extraction module is used to extract multi-scale features of the image and generate an image memory vector; The cue memory fusion module is used to fuse cue vectors and image memory vectors through cross-attention to generate cue memory, and supports cache reuse; The defect detection module is used to perform defect detection based on cue memory and multi-scale features through a data-efficient Transformer decoder; The results optimization module is used to perform strip merging, one-dimensional band-oriented nonmaximum suppression, and time trajectory consistency filtering on the detection results.

[0015] Beneficial effects: This invention brings significant benefits to conveyor belt defect detection by combining visual cue memory with a data-efficient Transformer mechanism.

[0016] First, this invention fully utilizes visual cues, such as belt area masks, historically high-defect areas, and human interaction prompts. Through an independent cue encoder and a cross-attention module, these cue information is encoded into cue memory vectors, and the network is explicitly guided to focus on key areas during the decoding phase. This design significantly improves the network's detection sensitivity for small targets such as cracks and scratches, as well as defects with extreme aspect ratios, reducing missed detections, and providing targeted enhancement, especially in high-risk areas.

[0017] Secondly, this invention introduces a cue memory caching mechanism. For the same conveyor belt or stable operating conditions, the cue memory only needs to be calculated once and can be reused in multi-frame detection, avoiding repeated encoding and fusion operations, significantly reducing online inference overhead, ensuring that the system maintains a high frame rate under long-term continuous monitoring without introducing additional latency, and is suitable for edge device deployment.

[0018] Third, this invention demonstrates high-precision detection capability in small-sample scenarios. By distilling knowledge from a self-supervised pre-trained teacher network to a lightweight student network, a multi-scale Transformer image encoder is constructed. Combined with a data-efficient Transformer decoding structure and matching quality-perceived loss, high recall and localization accuracy can still be obtained with limited labeled data, reducing the dependence on large-scale data.

[0019] Fourth, this invention adapts to the unique structure of conveyor belts and uses strip-type sliding window processing to segment high-resolution images into overlapping strip blocks to ensure defect integrity; it reduces redundant boxes by merging cross-window detection results through one-dimensional band non-maximum suppression; and it uses time trajectory consistency filtering to associate targets in adjacent frames based on the continuous motion characteristics of the conveyor belt, filtering out instantaneous noise false alarms. These optimization strategies work together to improve detection stability and reliability.

[0020] Finally, this invention has outstanding advantages in real-time performance and deployability. The lightweight Transformer backbone and compact cue memory module can achieve efficient inference on ordinary GPUs or industrial PCs. Compared with traditional convolutional detectors, it has a better speed balance at similar accuracy, meeting the real-time requirements of industrial online detection.

[0021] Overall, this invention achieves high-precision, low-false-alarm, and high-efficiency conveyor belt defect detection, providing an innovative solution for industrial visual inspection. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating the conveyor belt defect detection method based on visual cue memory provided in a specific embodiment of the present invention; Figure 2 This is a schematic diagram of the system composition of the conveyor belt defect detection device based on visual cue memory provided in a specific embodiment of the present invention; Figure 3 This is a schematic diagram of the overall system architecture and processing flow provided in a specific embodiment of the present invention; Figure 4 This is an example diagram illustrating the detection effect of surface hole defects on conveyor belts provided in a specific embodiment of the present invention; Figure 5 This is an example diagram illustrating the detection effect of longitudinal scratches on the surface of a conveyor belt, provided in a specific embodiment of the present invention. Detailed Implementation

[0023] To enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Based on the embodiments in this application, other similar embodiments obtained by those skilled in the art without creative effort should all fall within the scope of protection of this application. Furthermore, directional terms mentioned in the following embodiments, such as "up," "down," "left," and "right," are only for reference to the directions in the accompanying drawings; therefore, the directional terms used are for illustrative purposes and not for limiting the invention.

[0024] The present invention will be further described below with reference to the accompanying drawings and preferred embodiments.

[0025] Please see Figure 1 and Figures 3-5 , Figure 3 The overall architecture and core processing flow of the conveyor belt defect detection system based on visual cue memory proposed in this invention are demonstrated. Figure 3The entire technical chain from data acquisition to result output is clearly depicted in a top-down hierarchical structure. The process begins at the top with the hardware sensing unit, including the "conveyor belt," "industrial camera," and "light source," which together generate the raw "image stream." The image stream first enters the "image acquisition module," and is then sent to the "strip preprocessing module." This module internally performs "sliding window cropping," "normalization," and "denoising" operations, processing the high-resolution panoramic image into a series of standardized "strip image patches." The processing flow here diverges into two parallel core paths: the left path focuses on a deep understanding of the image content itself. The "strip image patches" are processed by the "multi-scale Transformer image encoder" to extract semantically rich "multi-scale feature maps (F3 / F4 / F5)," which are then compressed and structured by the "image memory generation module" to generate "M_img" (image memory vector) representing the global context of the image. The right-hand path handles prior and interactive information processing. The "Visual Cue Construction and Encoding Module" integrates automatic cues (such as region masks) and human interaction cues to generate a "Visual Cue Map P," which is then encoded into a compact "T_prompt" (cue vector) by the "Cue Encoder Module." Subsequently, the information from the two paths is deeply fused in the "Cue-Image Memory Cross-Attention Module" to generate a guiding "Cue Memory." This memory, along with image features, is input to the "Data-Efficient Transformer Decoder" (which contains multiple layers of decoders for iterative optimization), ultimately outputting a "Defect Candidate Box" containing location and category information. At the end of the process, the detection results undergo spatial integration and redundancy removal by the "Strip Result Merging and NMS Module," and are simultaneously temporally filtered by the "Temporal Trajectory Consistency Analysis Module" to improve the stability of the results. Finally, all information converges in the "Human-Computer Interaction and Alarm Module," which is responsible for "interface display" and "audio-visual alarms," ​​completing the closed loop from perception to decision-making.

[0026] Furthermore, the present invention provides a conveyor belt defect detection method based on visual cue memory, comprising: S100: Acquire images of the conveyor belt surface and perform strip-type preprocessing to obtain multiple strip image blocks; It should be further explained that in step S100, image acquisition is achieved through a fixedly installed industrial camera. The camera preferably uses a 2-megapixel CMOS sensor (specifically a resolution of 1920×1080 pixels), with a frame rate set to 30 frames / second, an installation height of 2 meters, and a coverage area of ​​1.5 meters for the conveyor belt width to ensure image clarity. The light source uses linear LED light with adjustable brightness, preferably 8000 lumens, to compensate for changes in ambient light. The specific steps of strip preprocessing are as follows: First, the original image is cropped into overlapping strip image blocks along the conveyor belt running direction. The preferred sliding window size is 1024×512 pixels, the preferred step size is 256 pixels, and the overlap rate is 25%. The reason for choosing this size and step size is based on the typical running speed of the conveyor belt (1-2 m / s) and the minimum size of defects (such as crack length ≥ 50 pixels), ensuring that any defect falls completely into at least one strip image block, while balancing processing efficiency. The preprocessing operations include normalization (linearly scaling pixel values ​​to the range of [0,1]), illumination compensation (using contrast-limited adaptive histogram equalization, with Clip Limit preferably 2.0 and Tile Grid Size preferably 8×8, to prevent local overexposure), and lightweight denoising (using Gaussian filtering, with kernel size preferably 3×3 and standard deviation σ preferably 1.0, because this configuration can effectively suppress dust noise and has low computational cost).

[0027] S200. Construct a visual cue map and encode the visual cue map into a cue vector using a cue encoder; simultaneously, extract multi-scale features of the strip image patch using a multi-scale Transformer image encoder and generate an image memory vector. It should be further explained that in step S200, the construction and encoding of visual cue maps are performed in parallel with multi-scale feature extraction to improve real-time performance. The training process of the multi-scale Transformer image encoder is as follows: the teacher network uses a ViT-Base model pre-trained on the ImageNet-21k dataset through self-supervised learning (such as the DINO method), with an input image size of 224×224 pixels and a hidden dimension of 768; the student network uses a ViT-Tiny variant, with an input size adapted to 512×1024 pixels, and a hidden dimension C preferably of 256. It is trained by knowledge distillation, with the distillation loss function being the mean squared error loss, the temperature parameter preferably being 3.0, the training cycle preferably being 100 rounds, and the batch size preferably being 16. The reason for choosing these values ​​is that ViT-Tiny achieves a balance between computational efficiency and accuracy, and distillation can effectively transfer semantic knowledge; the image encoder outputs multi-scale feature maps with downsampling ratios of 1 / 8, 1 / 16, and 1 / 32, respectively, and the number of feature map channels is uniformly 256.

[0028] S300: The cue vector and the image memory vector are fused through a cross-attention module to generate a cue memory, and the cue memory is cached. It should be further explained that in step S300, the cross-attention module adopts a multi-head scaling dot product attention mechanism, with 8 heads being preferred, the key dimension d_k being preferred to be 64, and the scaling factor being √d_k. The reason is that this configuration is a standard Transformer setting and can enhance feature interaction. The prompt memory cache is implemented using a Redis database, with a cache validity period of 24 hours being preferred, and is associated with the conveyor belt ID. The reason is based on the stability of working conditions in industrial scenarios (such as shift change cycle).

[0029] S400. Based on the cue memory and the multi-scale features, defect detection is performed using a data-efficient Transformer decoder to obtain defect box prediction results. It should be further explained that in step S400, the query initialization of the data-efficient Transformer decoder is as follows: regular grid anchor points are generated on multi-scale features, with a grid spacing of 16 pixels preferred. The initial class score and box quality score are predicted through a lightweight head (containing a 1×1 convolutional layer and sigmoid activation). The top-K positions with the highest scores are selected as the initial queries, with K preferably being 100, because it can cover the number of typical defects (usually less than 50) and avoid omissions. The decoder has a preferred number of 6 layers, each including self-attention, cross-attention and feedforward network, with a hidden dimension of 256. During training, the matching quality-aware classification loss (focal loss, α preferably 0.25, γ preferably 2.0) and boundary distribution regression loss (discrete bin number preferably 100) are used.

[0030] S500: Perform strip merging and one-dimensional band-oriented nonmaximum suppression on the defect box prediction results, and combine with time trajectory consistency filtering to output the final detection result; It should be further explained that in step S500, the strip merging maps the detection results back to the global coordinate system. The IoU threshold for one-dimensional band non-maximum suppression is preferably 0.5, based on standard target detection practices. The specific algorithm for temporal trajectory consistency filtering is as follows: calculate the IoU for adjacent frame detection boxes, with an association threshold preferably 0.3 and a trajectory length threshold preferably 3 frames. The reason for choosing this value is that when the conveyor belt speed is 1.5 m / s, 3 frames can cover continuous motion (about 0.15 m), effectively filtering instantaneous noise.

[0031] Understandably, this method, by integrating visual cues and memory mechanisms, improves recall by more than 15% in small sample scenarios (less than 1000 labeled images), and achieves a frame rate of up to 30fps, making it suitable for online monitoring of long-distance conveyor belts in mines and other similar environments.

[0032] Specifically, the visual cue map is derived from at least one of automatic cue and manual interactive cue, wherein the automatic cue includes a belt area mask and a weight map of historically high-defect areas.

[0033] It should be further explained that in the automatic prompting, the conveyor belt region mask is generated through a segmentation model, preferably using the U-Net architecture with an input size of 1024×512 pixels. The encoder uses ResNet-18 pre-trained weights, and the output is a binary mask. When training the segmentation model, conveyor belt region labeled data is used, the loss function is Dice loss, the optimizer is Adam, the learning rate is preferably 0.001, the training cycle is 50 rounds, and the batch size is 8, because this configuration can converge quickly and has high accuracy (IoU≥0.95). The weight map of historical high-defect areas is generated based on statistics: the detection results of the past 30 days are collected, the frequency of defect occurrence at each pixel position is calculated, and the frequency value is normalized to the range of [0,1]. The weight map update cycle is preferably 24 hours, because this cycle can reflect the changes in daily working conditions. The manual interactive prompts are implemented through a web monitoring interface. Operators can mark areas with rectangles (minimum size 100×100 pixels) or lines (line width 5 pixels), and the prompt value is assigned 1. The visual prompt image is represented in tensor form, combining the above sources. in, Visual cue image, The batch size is preferred (8, based on GPU memory limitations). Indicates single channel. Image height (preferably 512 pixels). Image width (preferably 1024 pixels).

[0034] Understandably, multi-source hint maps improve the network's sensitivity to defect detection in critical areas (such as joints) by 20%, reducing missed detections.

[0035] Specifically, the cue encoder encodes the visual cue map into multiple cue tokens through pooling operations and a multilayer perceptron, forming the cue vector.

[0036] It should be further explained that the cue encoder first processes the visual cue image. Perform pooling operations, prioritizing attention pooling. The specific steps are as follows: Calculate... Calculate the spatial attention weight matrix Through 1 Convolution and softmax, followed by weighted summation, yield the global description vector: in, For the sake of clarity, Pool represents pooling operations. For batch size, The pooled dimension is preferred (64 is ideal because this dimension can compress information without losing key features). Then, it is mapped through a multilayer perceptron (MIP). The MLP structure specifically consists of two fully connected layers, with the first layer being the input dimension. The second layer has an input dimension of 128 and an activation function of ReLU. ,in To indicate the number of tokens (ideally 10, as verified through grid search experiments, this number can encode diverse hints), The number of channels (preferably 256); linear initialization is used during MLP training, with no bias term. The final output prompt token is: in, For the set of prompt tokens, This represents the MLP mapping. The physical meaning of the formula's components. It is the global vector after pooling. It is the number of tokens. It is a hidden dimension.

[0037] Understandably, this encoding method compresses the prompt information into a compact vector, reducing computational overhead by 70% and improving real-time performance.

[0038] Specifically, the cross-attention module employs a multi-head scaling dot product attention mechanism, using the cue vector as the query and the image memory vector as the key and value for fusion.

[0039] It should be further explained that the specific implementation of the cross-attention module mechanism is as follows: The cross-attention module specifically performs multi-head scaling dot product attention calculation, and the formula is: Here, CrossAttn represents the cross-attention operation, and the specific calculation steps are as follows: First, the query... ,key ,value Linear projection to Size ( Each head dimension Then calculate the scaled dot product attention: Attention softmax Finally, the multi-head outputs are concatenated and linearly projected. The physical meaning of the formula components is as follows: For the cue vector (dimension) , Image memory matrix (dimension) , Preferred option 5), For the fused prompt memory. Select the number of heads. and The reason is that the standard configuration of Transformer can capture features of multiple subspaces.

[0040] Understandably, this mechanism enables the selective aggregation of image semantics by cue vectors, improving the detection accuracy of slender cracks by approximately 10%.

[0041] Specifically, the multi-scale Transformer image encoder outputs feature maps with multiple downsampling ratios based on a lightweight Transformer backbone obtained by distillation of a self-supervised pre-trained model, and generates the multi-scale features through semantic enhancement fusion.

[0042] It should be further noted that the image encoder uses RGB images. For input, The dimension is ( The encoder uses ViT-Tiny as the student network, with the following parameters: patch size 16x16 pixels, 12 layers, 8 attention heads, and [missing information - likely a number of hidden dimensions]. The teacher network is ViT-Based, and pre-training was performed on ImageNet-21k for 300 epochs using the DINO self-supervised method. The distillation training process involved resizing the input image to 512×1024 pixels for the student network, followed by random cropping and scaling for enhancement. The distillation loss function was: in It is the classification cross-entropy loss. It is the feature map mean square error loss. The optimal weight is 0.7, chosen for its weight balancing logic and feature learning. The optimizer used is Adam W, with a learning rate of 0.001, weight decay of 0.01, 100 training epochs, and a batch size of 16. The student network outputs multi-scale feature maps. in, , , These are feature maps downsampled by 1 / 8, 1 / 16, and 1 / 32, respectively. Semantic enhancement is achieved through a bidirectional convolutional pyramid: using a 3×3 convolutional kernel, upsampling and downsampling operations, the enhanced features are obtained. , , .

[0043] Understandably, this design achieves an mAP of 0.85 with a small sample size (500 training images), a 12% improvement over the baseline.

[0044] Specifically, the image memory vector is generated by compressing the multi-scale features through spatial pooling operations and then stacked and transformed into an image memory matrix.

[0045] It should be further explained that spatial pooling is performed on each enhanced multi-scale feature, with global average pooling being preferred. Specifically, for each feature map... Channel mean calculation: in, , , These are the memory vectors at each scale, and Pool represents the global average pooling operation. For batch size, The vectors are then stacked and transformed through a linear layer (weight matrix dimension 3). , (Bias term is 0), generate image memory matrix: in, Represents a linear transformation. This represents a stack of vectors. The physical meaning of the formula's components: It is the compressed feature vector. The number of memory vectors (ideally 5, because it can cover multi-scale information) is the optimal number. It refers to the number of channels.

[0046] Understandably, compression reduces memory usage by 50% and increases inference speed by 20%.

[0047] Specifically, the data-efficient Transformer decoder generates an initial query through a query initialization mechanism, and then fuses the cue memory and the image memory vector in a phased cross-attention manner to update the query vector to predict the defect category and bounding box.

[0048] It should be further explained that the specific steps of the query initialization mechanism include: Regular grid anchor points are generated on multi-scale features with a grid spacing of 16 pixels. Each anchor point predicts a class score (sigmoid output) and a bounding box quality score (Lou branch) through a lightweight head (two 1×1 convolutional layers). The top-K=100 locations with the highest scores are selected as the initial query and reference boxes. Staged cross-attention: In each layer of the decoder, cross-attention (8 heads, 64 dimensions) is first performed with the query vector as Q and the cue memory Mprompt as K / V to obtain the intermediate query. Then, deformable cross-attention is performed with the intermediate query as Q and the image memory Mimg flattened as M_img_flatten (dimensions B×S×C, S is the number of spatial locations) as K / V. The optimal number of sampling points is 4, based on the Deformable DETR standard. The decoder training loss function includes a matching quality-aware classification loss (weight 1.0) and a boundary distribution regression loss (bins=100). The decoder has 6 layers, 256 hidden dimensions, and 100 training epochs. Understandably, the phased fusion reduces the positioning error of slender cracks by 15%.

[0049] Specifically, the strip-type preprocessing includes cropping overlapping image blocks along the conveyor belt running direction and performing normalization, illumination compensation, and noise reduction.

[0050] It should be further noted that the overlap rate is fixed at 25% during cropping (step size / window length = 0.25) to ensure the continuity of defects across windows; normalization uses minimum-maximum scaling. ,in For the input image, and The minimum and maximum pixel values ​​for the image are defined. Illumination compensation uses adaptive histogram equalization with ClipLimit=2.0 and Tile Grid Size=8×8. This value is chosen to avoid noise amplification. Denoising uses Gaussian filtering with a kernel size of 3×3 and σ=1.0. This configuration is chosen to suppress noise while preserving edges.

[0051] It is understood that the present invention improves image quality and reduces the false detection rate by 10% through preprocessing.

[0052] Specifically, the time trajectory consistency filtering includes associating adjacent frame detection boxes with targets to form trajectories, and filtering single-frame or short trajectory detection boxes to reduce false alarms.

[0053] It should be further explained that the specific steps of the target association algorithm include: calculating the LoU of adjacent frame detection boxes, with a preferred LoU threshold of 0.3, because this value can balance association accuracy and recall; trajectory formation uses linear interpolation, with a preferred trajectory length threshold of 3 frames, chosen because at a conveyor belt speed of 1.5 m / s, 3 frames cover 0.15 meters of movement, which can filter out instantaneous interference (such as light spots); after filtering, only the trajectory length is retained. Understandably, this filter reduces the false alarm rate by 25% and improves stability.

[0054] For easier understanding, please refer to Figure 4 and Figure 5 , Figure 4 This demonstrates the practical detection effect of the method of the present invention on defects such as holes or dents on the surface of conveyor belts. For example... Figure 4 As shown, the entire field of view is a uniform dark gray background, simulating the metallic texture of a conveyor belt liner or similar material. Scattered across this background are fine, random light-colored spots, reflecting the inherent texture or slight wear of surfaces in real industrial settings. In the upper center of the image, there is a distinct, narrow, dark rectangular area that contrasts sharply with the surrounding background. The shape and color characteristics of this area conform to typical defects such as holes, tears, or severe dents. The algorithm of this invention successfully identified and accurately located this defect, with the detection result clearly outlining the precise boundary of the dark area with a closed, bright green line. The vertical dark stripes on the left and right edges of the image, along with the surrounding orange-yellow lines, represent reference boxes in the system interface used to mark the boundaries of the detection area or the effective area of ​​the image. Figure 4 This demonstrates intuitively that even in the presence of background texture interference, the preprocessing, feature extraction, and target recognition algorithms of this invention can still effectively distinguish between defects and normal backgrounds, achieving high-precision and robust localization and contour marking of specific types of minor defects.

[0055] Furthermore, Figure 5 The invention demonstrates the practical detection effect of the method on longitudinal scratches or cracks on the surface of conveyor belts. Figure 5 The main element is a gray rectangular area, representing a pre-processed image patch of the conveyor belt surface. This gray area displays fine spots and a uniform texture. Against this background, a thin, vertical blue line is clearly visible, positioned slightly above the center of the image. This prominent blue line represents the longitudinal scratches or cracks detected and marked by the algorithm. The blue marker line contrasts sharply with the gray background, accurately indicating the direction and location of the defects. The black border around the image and the thin yellow lines along the edges of the gray area together form the display elements of the results visualization interface. Figure 5This demonstrates that the multi-scale feature fusion and detection mechanism optimized for slender targets employed in this invention can effectively capture linear defects that are not visually significant and have slender shapes, and accurately and conspicuously visualize and label them, verifying the algorithm's sensitivity to defect types such as scratches and cracks.

[0056] Please see Figure 2 The present invention provides another embodiment, which provides a conveyor belt defect detection device based on visual cue memory. The conveyor belt defect detection device based on visual cue memory includes: Image acquisition module 100 is used to acquire images of the conveyor belt surface; Preprocessing module 200 is used to perform strip preprocessing on the image; Visual cue encoding module 300 is used to construct visual cue maps and encode them into cue vectors; The image feature extraction module 400 is used to extract multi-scale features of the image and generate an image memory vector; The cue memory fusion module 500 is used to fuse cue vectors and image memory vectors through cross-attention to generate cue memory, and supports cache reuse; The defect detection module 600 is used to perform defect detection based on cue memory and multi-scale features through a data-efficient Transformer decoder. The result optimization module 700 is used to perform strip merging, one-dimensional band-oriented nonmaximum suppression, and time trajectory consistency filtering on the detection results.

[0057] It should be further noted that the image acquisition module 100 uses a Basler acA2040-90um industrial camera with a resolution of 2048×2048 pixels, equipped with a KOWOT 850nm infrared LED linear light source, installed at a height of 2.2 meters, with a lens focal length of 12mm, and connected to an industrial computer via a GigE interface. The camera's frame rate is set to 25fps, and the exposure time is adaptively adjusted (1-10ms) according to the conveyor belt speed to ensure that the image is free of motion blur. The preprocessing module 200 is deployed on the NVIDIA Jetson AGXXavier edge computing device. It uses the OpenCV 4.5 library to implement the strip processing algorithm. The specific processing flow is as follows: First, the 2048×2048 original image is cropped into 512×512 pixel strip blocks according to the direction of the conveyor belt, with an overlap rate of 25%. Then, gamma correction (γ=1.2), CLAHE illumination compensation (Clip Limit=2.0), and bilateral filtering (d=9, σcolor=75, σspace=75) are performed for noise reduction. The processing time per frame is controlled within 15ms.

[0058] Furthermore, the visual cue encoding module 300 adopts a dual-path parallel architecture: the automatic cueing path uses a DeepLabV3+ segmentation network (with MobileNetV2 backbone) pre-trained on the COCO dataset, with an input size of 512×512 and an output binary mask of the belt region; the human interaction cueing path receives operator annotations through a web interface, supporting both rectangular bounding boxes (minimum 50×50 pixels) and free drawing. The cue encoder uses a two-layer MLP (64, 128, 256 dimensions), with an inference time of less than 5ms. The image feature extraction module 400 adopts the Vision Transformer Tiny model, with a patch size of 16×16, 12 layers, and 256 hidden dimensions. It transfers weights from ViT-Base through knowledge distillation and uses a BiFPN structure in the multi-scale feature fusion stage, outputting feature maps at three scales: 1 / 8, 1 / 16, and 1 / 32.

[0059] Furthermore, the prompt memory fusion module 500 has 8 cross-attention heads and 64 key dimensions. It uses a Redis database to cache the prompt memory, with the cache key format "belt_{ID}_prompt_memory" and an expiration time of 24 hours. The defect detection module 600's decoder contains 6 Transformer layers, each with 100 queries and 4 deformable attention sampling points. With TensorRT acceleration, the single-frame inference time does not exceed 25ms. The result optimization module 700 uses a sliding window approach to merge strip detection results. The IoU threshold for one-dimensional NMS is 0.5, the length threshold for temporal trajectory filtering is 3 frames, and the maximum interval for trajectory association is set to 10 frames.

[0060] Understandably, the system software architecture adopts a microservice design, with modules communicating via the gRPC protocol. The visual cues encoding module and the defect detection module transfer data through shared memory to reduce serialization overhead. The system is deployed on Ubuntu 20.04 and containerized using Docker, supporting stable operation in scenarios with conveyor belt lengths exceeding 1000 meters. The overall end-to-end latency is controlled within 50ms, meeting real-time detection requirements.

[0061] In a preferred embodiment, this application also provides an electronic device, the electronic device comprising: The computer device includes a memory and a processor, wherein the memory stores computer-readable instructions that, when executed by the processor, implement the visual cue memory-based conveyor belt defect detection method. The computer device can be broadly categorized as a server, terminal, or any other electronic device with the necessary computing and / or processing capabilities. In one embodiment, the computer device may include a processor, memory, network interface, communication interface, etc., connected via a system bus. The processor of the computer device can be used to provide the necessary computing, processing, and / or control capabilities. The memory of the computer device may include a non-volatile storage medium and internal memory. The non-volatile storage medium may store an operating system, computer programs, etc. The internal memory can provide an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface and communication interface of the computer device can be used to connect and communicate with external devices via a network. When the computer program is executed by the processor, it performs the steps of the method of the present invention.

[0062] This invention can be implemented as a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, causes the steps of the methods of embodiments of the invention to be performed. In one embodiment, the computer program is distributed across multiple network-coupled computer devices or processors, such that the computer program is stored, accessed, and executed in a distributed manner by one or more computer devices or processors. A single method step / operation, or two or more method steps / operations, may be executed by a single computer device or processor or by two or more computer devices or processors. One or more method steps / operations may be executed by one or more computer devices or processors, and one or more other method steps / operations may be executed by one or more other computer devices or processors. One or more computer devices or processors may execute a single method step / operation, or execute two or more method steps / operations.

[0063] Those skilled in the art will understand that the method steps of this invention can be performed by a computer program instructing related hardware, such as a computer device or processor, to perform the steps of this invention when executed. Depending on the context, any references herein to memory, storage, databases, or other media may include non-volatile and / or volatile memory. Examples of non-volatile memory include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, etc. Examples of volatile memory include random access memory (RAM), external cache memory, etc.

[0064] The technical features described above can be combined arbitrarily. Although not all possible combinations of these technical features are described, any combination of these technical features should be considered to be covered by this specification, provided that such combination does not contain contradictions.

[0065] The specific embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made in accordance with the technical concept of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for detecting defects in conveyor belts based on visual cue memory, characterized in that, Includes the following steps: S100: Acquire images of the conveyor belt surface and perform strip-type preprocessing to obtain multiple strip image blocks; S200. Construct a visual cue map and encode the visual cue map into a cue vector using a cue encoder; simultaneously, extract multi-scale features of the strip image patch using a multi-scale Transformer image encoder and generate an image memory vector. S300: The cue vector and the image memory vector are fused through a cross-attention module to generate a cue memory, and the cue memory is cached. S400. Based on the cue memory and the multi-scale features, defect detection is performed using a data-efficient Transformer decoder to obtain defect box prediction results. S500: Perform strip merging and one-dimensional band-oriented nonmaximum suppression on the defect box prediction results, and combine with time trajectory consistency filtering to output the final detection result.

2. The conveyor belt defect detection method according to claim 1, characterized in that, The visual cue map is derived from at least one of automatic cue and manual interactive cue, wherein the automatic cue includes a belt area mask and a weight map of historically high-defect areas.

3. The conveyor belt defect detection method according to claim 2, characterized in that, The cue encoder encodes the visual cue map into multiple cue tokens through pooling operations and a multilayer perceptron, forming the cue vector.

4. The conveyor belt defect detection method according to claim 3, characterized in that, The cross-attention module employs a multi-head scaling dot product attention mechanism, using the cue vector as the query and the image memory vector as the key and value for fusion.

5. The conveyor belt defect detection method according to claim 1, characterized in that, The multi-scale Transformer image encoder is based on a lightweight Transformer backbone obtained by distillation of a self-supervised pre-trained model, outputs feature maps with multiple downsampling ratios, and generates the multi-scale features through semantic enhancement fusion.

6. The conveyor belt defect detection method according to claim 5, characterized in that, The image memory vector is generated by compressing the multi-scale features through spatial pooling and then stacked to transform it into an image memory matrix.

7. The conveyor belt defect detection method according to claim 1, characterized in that, The data-efficient Transformer decoder generates an initial query through a query initialization mechanism, and then fuses the cue memory and the image memory vector in a phased cross-attention manner to update the query vector to predict the defect category and bounding box.

8. The conveyor belt defect detection method according to claim 1, characterized in that, The strip-type preprocessing includes cropping overlapping image blocks along the conveyor belt running direction and performing normalization, illumination compensation, and noise reduction.

9. The conveyor belt defect detection method according to claim 1, characterized in that, The time trajectory consistency filtering includes associating adjacent frame detection boxes with targets to form trajectories, and filtering single-frame or short trajectory detection boxes to reduce false alarms.

10. A conveyor belt defect detection system based on visual cue memory, characterized in that, The conveyor belt defect detection method according to any one of claims 1-9 includes: The image acquisition module is used to acquire images of the conveyor belt surface. The preprocessing module is used to perform strip preprocessing on the image; The visual cue encoding module is used to construct visual cue maps and encode them into cue vectors; The image feature extraction module is used to extract multi-scale features of the image and generate an image memory vector; The cue memory fusion module is used to fuse cue vectors and image memory vectors through cross-attention to generate cue memory, and supports cache reuse; The defect detection module is used to perform defect detection based on cue memory and multi-scale features through a data-efficient Transformer decoder; The results optimization module is used to perform strip merging, one-dimensional band-oriented nonmaximum suppression, and time trajectory consistency filtering on the detection results.

Images