Dynamic multi-scale fusion fabric defect detection method and related device

Abstract

The invention discloses a dynamic multi-scale fusion fabric defect detection method and a related device. The method comprises the following steps: constructing a lightweight end-to-end network; constructing a training set on the basis of a json file in the ZJU-Leader data set; training images in the training set are input into the lightweight end-to-end network, gradient steady-state training processing is carried out through dynamic adjustment of convolution kernel parameters and parallel processing of high and low frequency features, and a convergent lightweight end-to-end network is formed; inputting to-be-detected fabric image data into the convergent lightweight end-to-end network, and outputting fabric defect classification and bounding box positioning results corresponding to the to-be-detected fabric image data by utilizing intra-scale feature interaction and cross-scale feature fusion processing. In the embodiment of the invention, under the condition of ensuring the light weight of the network, the efficiency of feature extraction, the convergence speed and the robustness to industrial noise are improved, the defect identification precision of the fabric is improved, and the generalization ability of the network is improved by adopting a feature fusion strategy.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and related apparatus for detecting fabric defects through dynamic multi-scale fusion. Background Technology

[0002] As a traditional foundational industry, the textile industry's production efficiency and product quality directly impact the competitiveness of the entire supply chain. With continuous advancements in textile technology, the types and uses of fabrics are becoming increasingly diverse, leading to ever-higher demands for product quality. During fabric production, various factors such as equipment malfunctions, improper operation, and raw material issues can cause defects on the fabric surface, including cracks, stains, color differences, holes, and frayed edges. These defects not only affect the fabric's appearance but may also reduce its performance. Therefore, fabric inspection has been introduced into the fabric production line. Currently, manual inspection methods can detect fabric defects with simple, uniform background textures, but they are affected by worker subjectivity, resulting in low efficiency, poor accuracy, and a high rate of missed detections. With the expansion of production scale and the increase in product types, traditional inspection methods are no longer sufficient to meet the needs of the modern textile industry, leading to the emergence of efficient and accurate automated visual inspection methods.

[0003] Existing object detection frameworks can be broadly categorized into CNN-based and Transformer-based approaches. The former can be further divided into "two-stage" approaches, represented by Faster R-CNN and RetinaNet, and "one-stage" approaches, represented by the YOLO series. One-stage methods directly regress the bounding boxes of defects in all regions of the image, while two-stage methods first find anchor boxes containing defects in the image, then classify these anchor boxes into different defect semantic categories and adjust the anchor coordinates to obtain the final defect bounding boxes. Compared to two-stage methods, one-stage methods have the advantages of lower computational load during inference and lower hardware requirements in practical applications. However, these detectors typically require confidence thresholding and non-maximum suppression (NMS) processing, which not only slows down inference but also introduces hyperparameters, leading to instability in both speed and accuracy. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art. This invention provides a dynamic multi-scale fusion method and related device for fabric defect detection, which can improve the efficiency of feature extraction, convergence speed and robustness to industrial noise while ensuring the network is lightweight, improve the accuracy of fabric defect identification, and improve the generalization ability of the network by adopting a feature fusion strategy.

[0005] To address the aforementioned technical problems, embodiments of the present invention provide a dynamic multi-scale fusion method for fabric defect detection, the method comprising: A lightweight end-to-end network is constructed, which sequentially includes a lightweight backbone network, an adaptive feature fusion neck network, and a detection head; The training set was constructed based on the JSON files in the ZJU-Leaper dataset. The training images in the training set are input into the lightweight end-to-end network. Gradient stabilization training is performed by dynamically adjusting the convolution kernel parameters and processing high and low frequency features in parallel, thus forming a convergent lightweight end-to-end network. The image data of the fabric to be detected is input into a converged lightweight end-to-end network. The network is processed by intra-scale feature interaction and cross-scale feature fusion to output the fabric defect classification and bounding box localization results corresponding to the image data of the fabric to be detected.

[0006] Optionally, the lightweight backbone network is a network with EffiRepCSANet as the backbone. EffiRepCSANet combines the CSP structure with the Efficient Layer Aggregation Network structure and performs average pooling downsampling of the feature maps through the Adown module. An EffiRepCSA module is introduced into EffiRepCSANet, which consists of EffiRep and Conv. EffiRep is the core of the EffiRepCSA module, which achieves feature enrichment and gradient stability during training through reparameterization. EffiRep contains EffiRepBottleneck, which is a basic module with a residual structure and is repeatedly stacked in the backbone. As the number of channels increases, deep feature representations are gradually built.

[0007] Optionally, the adaptive feature fusion neck network serves as a bridge connecting the lightweight backbone network and the detection head, and is used to further process and fuse the features formed by the lightweight backbone network. The adaptive feature fusion neck network adopts a Feature Pyramid Network structure and introduces an Adaptive Hi-Lo Fusion key self-attention component, which realizes feature upsampling and fusion through Upsample and Concat modules; The key self-attention component of the Adaptive Hi-Lo Fusion is to replace the standard multi-head attention with two parallel branches of high-frequency and low-frequency attention.

[0008] Optionally, the two parallel branches of the High-frequency and Low-frequency attention include a high-frequency branch and a low-frequency branch; The high-frequency branch is local window attention, which divides the received features into non-overlapping local windows in space and performs self-attention calculation independently only within each local window. The low-frequency branch is a global downsampling attention that performs large-step average pooling on the Key and Value matrices to generate a low-resolution version of them in terms of spatial dimension.

[0009] Optionally, the execution functions for the High-frequency and Low-frequency attention are as follows: ; in, To input features for High-frequency and Low-frequency attention, ; Indicates dimensional transformation; Indicates the input Perform dimensional transformation; divide the attention head into two groups, high-frequency branch and low-frequency branch, according to a preset ratio, and perform calculations separately. Indicates a high-frequency branch. For the low-frequency branch, the corresponding feature map is processed in parallel in the high-frequency branch and the low-frequency branch. This indicates that the output features of the high-frequency branch and the low-frequency branch are spliced ​​together along the channel dimension. This represents the inverse process of the initial dimensionality transformation operation, restoring the dimensions of the fused features to the standard image data format. .

[0010] Optionally, the detection head is based on the DSFR DETR Decoder module as its core component, and the DSFR DETR Decoder module performs target detection using a Transformer architecture; The DSFR DETR Decoder module converts the multi-scale features output by the adaptive feature fusion neck network into unified hidden dimension features through the input feature layer; The hidden dimension features are input into the Transformer decoder, where a deformable self-attention mechanism and a feedforward network are used to fuse the hidden dimension features to enhance the feature representation capability.

[0011] Optionally, the formula for the deformable self-attention mechanism is defined as follows: ; in, Features representing input deformable self-attention mechanisms; To index attention heads, This represents the total number of attention heads indexed. The key for index sampling, The total number of keys sampled for the index; Given a reference point, To query features, Used to represent Location, Indicates the first The first index attention head and the first The position offset of the key sampled by each index is obtained by the query through the fully connected layer; represents the weights of the deformable self-attention mechanism, with values ​​ranging from [0,1], and ; This represents the value obtained by interpolation based on the position of a given reference point; These are the weight values ​​corresponding to low-frequency semantic features. These are the weight values ​​corresponding to high-frequency semantic features, and .

[0012] In addition, embodiments of the present invention also provide a dynamic multi-scale fusion fabric defect detection device, the device comprising: Network building module: used to build a lightweight end-to-end network, which includes a lightweight backbone network, an adaptive feature fusion neck network, and a detection head in sequence; Dataset building module: Used to build training sets based on JSON files in the ZJU-Leaper dataset; Training module: Used to input training images from the training set into the lightweight end-to-end network, and perform gradient stabilization training by dynamically adjusting the convolution kernel parameters and processing high and low frequency features in parallel to form a converged lightweight end-to-end network; Defect detection module: It is used to input the image data of the fabric to be detected into a converged lightweight end-to-end network, and use intra-scale feature interaction and cross-scale feature fusion processing to output the fabric defect classification and bounding box localization results corresponding to the image data of the fabric to be detected.

[0013] In addition, embodiments of the present invention also provide an electronic device, including a processor and a memory, wherein the processor runs a computer program or code stored in the memory to implement the fabric defect detection method as described in any of the above.

[0014] In addition, embodiments of the present invention also provide a computer-readable storage medium for storing a computer program or code, which, when executed by a processor, implements the fabric defect detection method as described above.

[0015] In this embodiment of the invention, a lightweight end-to-end network is provided. When applied to a fabric defect dataset, it improves the accuracy (mAP) by 11.5% compared to Faster R-CNN and by 1.9% compared to the single-stage YOLOv8s model. During training, the gradient norm fluctuation is minimal and the value is low, resulting in more efficient and stable surface optimization trajectories with lower risk. The EffiRepCSA module is used in the lightweight backbone network, reducing GFLOPs (computational cost) by 69.2%. With fewer backbone network parameters, it is very suitable for deployment in resource-constrained industrial embedded devices. Furthermore, it performs excellently on cross-domain datasets such as TILDA and exhibits strong robustness to common industrial interferences such as motion blur and salt-and-pepper noise. Therefore, the method in this embodiment can improve the efficiency of feature extraction, convergence speed, and robustness to industrial noise while maintaining a lightweight network, thereby enhancing the accuracy of fabric defect identification and improving the network's generalization ability. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the dynamic multi-scale fusion fabric defect detection method in an embodiment of the present invention. Figure 2 This is a schematic diagram of the structural composition of the dynamic multi-scale fusion fabric defect detection device in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structural composition of the electronic device in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structural composition of the lightweight end-to-end network in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structural composition of the lightweight backbone network in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structural composition of AHLF self-attention within the adaptive feature fusion neck network in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1, please refer to Figure 1 , Figure 1 This is a flowchart illustrating the dynamic multi-scale fusion fabric defect detection method in an embodiment of the present invention.

[0020] like Figure 1 As shown, a dynamic multi-scale fusion method for fabric defect detection includes: S101: Construct a lightweight end-to-end network, which sequentially includes a lightweight backbone network, an adaptive feature fusion neck network, and a detection head; In the specific implementation of this invention, the lightweight backbone network is an EffiRepCSANet network. The EffiRepCSANet combines the CSP structure with the Efficient Layer Aggregation Network structure and performs average pooling downsampling of the feature maps through the Adown module. An EffiRepCSA module is introduced into the EffiRepCSANet. The EffiRepCSA module consists of EffiRep and Conv. EffiRep is the core of the EffiRepCSA module and achieves feature enrichment and gradient stability during training through reparameterization. The EffiRep contains the EffiRepBottleneck, which is a basic module with a residual structure. These modules are repeatedly stacked in the backbone, and as the number of channels increases, a deep feature representation is gradually constructed.

[0021] Furthermore, the adaptive feature fusion neck network serves as a bridge connecting the lightweight backbone network and the detection head, and is used to further process and fuse the features formed by the lightweight backbone network. The adaptive feature fusion neck network adopts a Feature Pyramid Network structure and introduces an Adaptive Hi-Lo Fusion key self-attention component, which realizes feature upsampling and fusion through Upsample and Concat modules. The Adaptive Hi-Lo Fusion key self-attention component replaces the standard multi-head attention with two parallel branches of high-frequency and low-frequency attention.

[0022] Furthermore, the two parallel branches of the High-frequency and Low-frequency attention include a high-frequency branch and a low-frequency branch; the high-frequency branch is a local window attention, which divides the received features into non-overlapping local windows in space and performs self-attention computation independently only within each local window; the low-frequency branch is a global downsampling attention, which performs large-stride average pooling on the Key and Value matrices to generate a low-resolution version of them in a reduced spatial dimension.

[0023] Furthermore, the execution functions for the High-frequency and Low-frequency attention are as follows: ; in, To input features for High-frequency and Low-frequency attention, ; Indicates dimensional transformation; Indicates the input Perform dimensional transformation; divide the attention head into two groups, high-frequency branch and low-frequency branch, according to a preset ratio, and perform calculations separately. Indicates a high-frequency branch. For the low-frequency branch, the corresponding feature map is processed in parallel in the high-frequency branch and the low-frequency branch. This indicates that the output features of the high-frequency branch and the low-frequency branch are spliced ​​together along the channel dimension. This represents the inverse process of the initial dimensionality transformation operation, restoring the dimensions of the fused features to the standard image data format. .

[0024] Furthermore, the detection head is based on the DSFR DETR Decoder module, which uses a Transformer architecture for target detection. The DSFR DETR Decoder module converts the multi-scale features output by the adaptive feature fusion neck network into unified hidden dimension features through the input feature layer. The hidden dimension features are then input into the Transformer decoder, where deformable self-attention mechanism and feedforward network are used to fuse the hidden dimension features to enhance feature representation capabilities.

[0025] Furthermore, the formula for the deformable self-attention mechanism is defined as follows: ; in, Features representing input deformable self-attention mechanisms; To index attention heads, This represents the total number of attention heads indexed. The key for index sampling, The total number of keys sampled for the index; Given a reference point, To query features, Used to represent Location, Indicates the first The first index attention head and the first The position offset of the key sampled by each index is obtained by the query through the fully connected layer; represents the weights of the deformable self-attention mechanism, with values ​​ranging from [0,1], and ; This represents the value obtained by interpolation based on the position of a given reference point; These are the weight values ​​corresponding to low-frequency semantic features. These are the weight values ​​corresponding to high-frequency semantic features, and .

[0026] Specifically, such as Figure 4 As shown, the constructed lightweight end-to-end network consists of three main components: a lightweight backbone network, an adaptive feature fusion neck network, and a detection head (decoder). Basic data augmentation strategies are introduced during the data preprocessing stage. In the network stage, an efficient backbone feature extractor is first used to extract features from the input image, dynamically adjusting convolutional kernel parameters to adapt to features of different sizes. The feature maps (P3, P4, P5) from the last three stages of the backbone network are input into the Neck (adaptive feature fusion neck network). The Neck transforms multi-scale features into image feature sequences using multiple convolutional branches and residual connections through intra-scale feature interaction and cross-scale feature fusion. Then, a multi-task defect detection head is executed, employing minimum uncertainty query selection to choose a fixed number of encoder features as the initial object query for the decoder. Finally, the decoder with an auxiliary prediction head iteratively optimizes the object query to generate categories and bounding boxes.

[0027] The lightweight backbone network is a lightweight end-to-end network responsible for extracting features from the input image and constructing the feature pyramid. Based on the RT-DETR-ResNet18 backbone, it improves upon it by proposing the EfficientRepresentation Cross Stage Aggregation Network (EffiRepCSANet) to replace the original ResNet18 backbone. This lightweight backbone network improves the overall feature extraction performance by combining the CSP (Cross Stage Partial) and ELAN (Efficient Layer Aggregation Network) structures. It further reduces the spatial resolution of the feature maps by implementing average pooling downsampling through the ADown module (a downsampling module used to reduce the resolution of feature maps in the backbone network, while typically increasing the number of feature channels), thereby increasing the number of channels. Furthermore, it allows the network to capture information at different scales, making it crucial for constructing the feature pyramid.

[0028] The lightweight backbone network is constructed starting with three convolutional layers for downsampling and feature extraction. First, two Conv modules expand the number of channels in the input image from 3 to 64, then to 128, halving the spatial resolution of the feature map at each step. This is achieved by setting the stride to 2, while the autopad function automatically calculates padding to maintain the spatial size of the feature map. Next, the backbone introduces the Efficient Representation Cross Stage Aggregation (EffiRepCSA) module. This module mainly consists of EffiRep and Conv, branching out into multiple convolutional layers and finally merging the feature maps from these branches to simplify the backbone network structure and enhance feature representation.

[0029] EffiRep is the core of EffiRepCSA, designed to achieve rich features and stable gradients during training through reparameterization. Structurally, Efficient Convolutional Dyn-Reparameterization Bottleneck (EffiRepBottleneck) is a basic module with a residual structure, repeatedly stacked in the backbone (similar to the C3 module in YOLOv5 and the C2f module in YOLOv8). As the number of channels increases, it gradually builds deep feature representations. Its optimization advantage stems from the multi-branch structure EffiRCon used during training. This design is considered an implicit regularization mechanism; the parallel computation of multiple branches is equivalent to integrating gradients, explicitly reducing the variance of gradient estimation. This effectively reduces gradient oscillations during training, making the optimization direction more stable and accurate.

[0030] Therefore, EffiRepCSA reduces training resource consumption without sacrificing the model's final representation capability and inference performance, making the technology more practical and scalable when applied to more complex model designs.

[0031] The last few modules of the lightweight backbone network still combine EffiRepCSA and ADown until the maximum number of channels (1024) is reached. This combined module outputs a high-dimensional feature map, providing rich feature information for subsequent neck and head detection. The improved backbone network effectively extracts image features through deep convolutional networks and residual connections, progressively constructing a feature pyramid. This design enables the network to capture multi-scale features from low to high levels, providing a foundation for accurate object detection. The specific structure of the lightweight backbone network is as follows: Figure 5 As shown.

[0032] The adaptive feature fusion neck network acts as a bridge connecting the lightweight backbone network and the detection head, responsible for further processing and fusion of the features extracted by the lightweight backbone network. The adaptive feature fusion neck network adopts the FPN (Feature Pyramid Network) structure, and realizes feature map upsampling and fusion through the Upsample and Concat modules. The key self-attention component, Adaptive Hi-Lo Fusion (AHLF), is improved by introducing the HiLo component to improve the AIFI (Attention-based Intra-scale Feature Interaction) module in the hybrid encoder architecture of the baseline RT-DETR. The AIFI module performs intra-scale feature interaction, and its core mechanism is the standard multi-head self-attention (MHSA). Before performing cross-scale feature fusion, it performs independent feature refinement and context modeling on each single-scale feature map from the backbone network. Although the standard AIFI module is effective, its MHSA has two inherent limitations: first, the computational complexity is quadratic with the spatial dimension (N=H×W), resulting in high computational costs; second, it adopts a homogeneous approach to all patterns in the feature map, whether high-frequency details or low-frequency structures. AIFI is optimized by introducing the HiLo (High-frequency and Low-frequency) attention mechanism through a frequency-domain decoupling design paradigm. This paradigm assumes that visual features can be decomposed into high-frequency and low-frequency components and then heterogeneously processed.

[0033] The improved AIFI, or AHLF, replaces the original MHSA with HiLo attention, which includes two parallel branches: (1) High-frequency branch: Local Window Attention. This branch divides the feature map into non-overlapping local windows in space and performs self-attention computation independently only within each window. This strategy restricts the scope of attention operations to the local neighborhood, thereby efficiently modeling high-frequency information with linear complexity. It aims to efficiently capture high-frequency spatial details (such as edges and textures).

[0034] (2) Low-frequency branch: Global Downsampled Attention. This branch first performs average pooling with a large stride on the Key and Value matrices to generate a low-resolution version that is significantly reduced in spatial dimension. Then, global attention is calculated using the Query at the original resolution and the downsampled Key and Value. This method significantly reduces the computational cost of obtaining the global receptive field by approximating global interaction in a low-dimensional space. It models low-frequency global contexts (such as object structure and scene layout) with low computational cost.

[0035] This design allows AHLF to maintain low computational cost while modeling both global structure and local details in images, thus improving the detection performance of targets at different scales in models such as DSFR-DETR. Simultaneously, it captures global dependencies through a self-attention mechanism, improving feature quality early on and encoding information before feature fusion to reduce the risk of information loss. Then, upsampling and concatenation operations are used to upsample the high-resolution feature map output from the backbone, followed by concatenation with lower-level feature maps to achieve multi-scale feature fusion. This process not only preserves image details but also integrates deep semantic information, providing rich feature support for head detection tasks. Finally, further downsampling, concatenation, and EffiRepCSA processing unify the feature maps at different scales. The main structure of the AHLF self-attention component is as follows: Figure 6 As shown.

[0036] The entire AHLF algorithm can be summarized as a function. The overall process can be represented as follows: ; in, To input features for High-frequency and Low-frequency attention, ; Indicates dimensional transformation; Indicates the input Perform dimensional transformation; divide the attention head into two groups, high-frequency branch and low-frequency branch, according to a preset ratio, and perform calculations separately. Indicates a high-frequency branch. For the low-frequency branch, the corresponding feature map is processed in parallel in the high-frequency branch and the low-frequency branch. This indicates that the output features of the high-frequency branch and the low-frequency branch are spliced ​​together along the channel dimension. This represents the inverse process of the initial dimensionality transformation operation, restoring the dimensions of the fused features to the standard image data format. .

[0037] The detection head is based on the DSFR DETR Decoder module, which uses a Transformer architecture for object detection. First, the multi-scale feature maps output from the previous network are transformed to a unified hidden dimension through the input feature layer. These feature maps are then fed into the Transformer decoder, which consists of the DeformableTransformerDecoderLayer from AHLF. The decoder performs deep fusion of features through a deformable self-attention mechanism and a feedforward network to enhance feature representation. Simultaneously, the deformable self-attention mechanism significantly reduces computational cost. The formula for the deformable self-attention mechanism is defined as follows: ; in, Features representing input deformable self-attention mechanisms; To index attention heads, This represents the total number of attention heads indexed. The key for index sampling, The total number of keys sampled for the index; Given a reference point, To query features, Used to represent Location, Indicates the first The first index attention head and the first The position offset of the key sampled by each index is obtained by the query through the fully connected layer; represents the weights of the deformable self-attention mechanism, with values ​​ranging from [0,1], and ; This represents the value obtained by interpolation based on the position of a given reference point; These are the weight values ​​corresponding to low-frequency semantic features. These are the weight values ​​corresponding to high-frequency semantic features, and .

[0038] S102: Construct a training set based on JSON files in the ZJU-Leaper dataset; In the specific implementation of this invention, the ZJU-Leaper dataset is used to construct the training set. The ZJU-Leaper dataset is the largest fabric image dataset to date, classified by texture, containing 19 texture categories, of which 4 are solid color fabric textures; it contains 71,127 defect-free images and 27,650 defective images, which were collected from factories or laboratories; defects in the fabric images are annotated with bounding boxes; the location and size of defects in the ZJU-Leaper dataset have high diversity; the training set is divided into 63,814 images (including defective and defect-free images) by the JSON file in the ZJU-Leaper dataset, while the test set contains all remaining images.

[0039] S103: Input the training images in the training set into the lightweight end-to-end network, and perform gradient stabilization training by dynamically adjusting the convolution kernel parameters and processing high and low frequency features in parallel to form a converged lightweight end-to-end network. In the specific implementation of this invention, before training, it is necessary to set the parameters of the lightweight end-to-end network. That is, the parameters in the lightweight end-to-end network are randomly initialized. In order to simulate the usage of actual application scenarios, the lightweight end-to-end network is trained, and the image size is set to 512×512, the number of epochs is 300 (because the curves of various indicators have not yet converged after 100 epochs), the batch size is 24, the learning rate is 0.0001, and the learning rate factor is 0.1. The parameters of the three key scaling dimensions of each model in the lightweight end-to-end network are uniformly set to [0.33, 0.50, 1024], and the same enhancement strategy is adopted. During training, it is run on the AutoDL network server, which is configured with an RTX 3090 (24GB) GPU and an AMD EPYC 9754 128-CoreProcessor CPU, 60GB of memory.

[0040] The training images from the training set are input into the lightweight end-to-end network, and then the network is trained according to the preset training hyperparameters. After training is completed, the lightweight end-to-end network is formed. At this point, it is necessary to use evaluation metrics to confirm whether the lightweight end-to-end network has converged; therefore, a test set is required.

[0041] The test set is used to test the accuracy of the trained lightweight end-to-end network in predicting defects in test images, specifically whether it can predict the location and type of defects in the test images. For defect localization, this embodiment uses the intersection-over-union ratio (IoU) to determine its accuracy, calculated as follows: ; in, and These represent the predicted bounding box and the final bounding box, respectively; if IoU > 0.5, then it is considered... It is accurate. For defects belonging to ground truth category C, the classification score of the bounding box containing the defect is used to determine whether the defect is correctly classified. When the classification score is higher than the predefined threshold tcls, the classification result is considered accurate. When both the IoU and classification score of the predicted defect bounding box satisfy the above constraints, it is a true positive (TP); otherwise, it is a false positive (FP). When the predicted defect-free bounding box contains the defect, it is a false negative (FN); otherwise, it is a true negative (TN). Therefore, precision p and recall r can be defined as follows: ; Precision and recall are two contradictory metrics; increasing the threshold tcls in a specific model determines the trade-off between precision and recall in that region, i.e., the area under the precision-recall curve when adjusting tcls; AP is estimated using eleven-point interpolation, which can be expressed as: ; Here, ρ represents the precision-recall curve; the average mean precision (mAP) across all classes is used as the core evaluation metric, where mAP50 and mAP50-95 represent the average precision under different intersection-over-union (IoU) thresholds, respectively; recall measures the completeness of the model in identifying all real targets, and this parameter directly affects the false negative rate, making it crucial in industrial detection; model parameters (Params) are used to evaluate the model's storage requirements, while floating-point operations (GFLOPs) measure the computational resources required for a single inference; in addition, frames per second (FPS) is a key indicator for evaluating the model's inference speed, with higher FPS indicating higher processing efficiency; model training efficiency is measured by the number of iterations required to reach a specific accuracy target, with fewer iterations meaning faster convergence and lower training costs; memory consumption measures the total amount of GPU memory and system memory used by the model during inference or training.

[0042] When the evaluation criteria are met, the trained lightweight end-to-end network is a converged lightweight end-to-end network. If the evaluation criteria are not met, the parameters of the trained lightweight end-to-end network are updated using the backpropagation algorithm, and the training images in the training set are used for retraining until the evaluation criteria are met, thus obtaining a converged lightweight end-to-end network.

[0043] S104: Input the image data of the fabric to be detected into a converged lightweight end-to-end network, and use intra-scale feature interaction and cross-scale feature fusion processing to output the fabric defect classification and bounding box localization results corresponding to the image data of the fabric to be detected.

[0044] In the specific implementation process of this invention, it is first necessary to obtain the image data of the fabric to be detected, then input the image data of the fabric to be detected into a converged lightweight end-to-end network for defect detection processing, and finally output the fabric defect detection result corresponding to the image data of the fabric to be detected.

[0045] In this embodiment of the invention, a lightweight end-to-end network is provided. When applied to a fabric defect dataset, it improves the accuracy (mAP) by 11.5% compared to Faster R-CNN and by 1.9% compared to the single-stage YOLOv8s model. During training, the gradient norm fluctuation is minimal and the value is low, resulting in more efficient and stable surface optimization trajectories with lower risk. The EffiRepCSA module is used in the lightweight backbone network, reducing GFLOPs (computational cost) by 69.2%. With fewer backbone network parameters, it is very suitable for deployment in resource-constrained industrial embedded devices. Furthermore, it performs excellently on cross-domain datasets such as TILDA and exhibits strong robustness to common industrial interferences such as motion blur and salt-and-pepper noise. Therefore, the method in this embodiment can improve the efficiency of feature extraction, convergence speed, and robustness to industrial noise while maintaining a lightweight network, thereby enhancing the accuracy of fabric defect identification and improving the network's generalization ability.

[0046] Implementation 2, please refer to Figure 2 , Figure 2 This is a schematic diagram of the structural composition of the dynamic multi-scale fusion fabric defect detection device in an embodiment of the present invention.

[0047] like Figure 2 As shown, a dynamic multi-scale fusion fabric defect detection device includes: Network construction module 201: used to construct a lightweight end-to-end network, wherein the lightweight end-to-end network sequentially includes a lightweight backbone network, an adaptive feature fusion neck network, and a detection head; In the specific implementation of this invention, the lightweight backbone network is an EffiRepCSANet network. The EffiRepCSANet combines the CSP structure with the Efficient Layer Aggregation Network structure and performs average pooling downsampling of the feature maps through the Adown module. An EffiRepCSA module is introduced into the EffiRepCSANet. The EffiRepCSA module consists of EffiRep and Conv. EffiRep is the core of the EffiRepCSA module and achieves feature enrichment and gradient stability during training through reparameterization. The EffiRep contains the EffiRepBottleneck, which is a basic module with a residual structure. These modules are repeatedly stacked in the backbone, and as the number of channels increases, a deep feature representation is gradually constructed.

[0048] Furthermore, the adaptive feature fusion neck network serves as a bridge connecting the lightweight backbone network and the detection head, and is used to further process and fuse the features formed by the lightweight backbone network. The adaptive feature fusion neck network adopts a Feature Pyramid Network structure and introduces an Adaptive Hi-Lo Fusion key self-attention component, which realizes feature upsampling and fusion through Upsample and Concat modules. The Adaptive Hi-Lo Fusion key self-attention component replaces the standard multi-head attention with two parallel branches of high-frequency and low-frequency attention.

[0049] Furthermore, the two parallel branches of the High-frequency and Low-frequency attention include a high-frequency branch and a low-frequency branch; the high-frequency branch is a local window attention, which divides the received features into non-overlapping local windows in space and performs self-attention computation independently only within each local window; the low-frequency branch is a global downsampling attention, which performs large-stride average pooling on the Key and Value matrices to generate a low-resolution version of them in a reduced spatial dimension.

[0050] Furthermore, the execution functions for the High-frequency and Low-frequency attention are as follows: ; in, To input features for High-frequency and Low-frequency attention, ; Indicates dimensional transformation; Indicates the input Perform dimensional transformation; divide the attention head into two groups, high-frequency branch and low-frequency branch, according to a preset ratio, and perform calculations separately. Indicates a high-frequency branch. For the low-frequency branch, the corresponding feature map is processed in parallel in the high-frequency branch and the low-frequency branch. This indicates that the output features of the high-frequency branch and the low-frequency branch are spliced ​​together along the channel dimension. This represents the inverse process of the initial dimensionality transformation operation, restoring the dimensions of the fused features to the standard image data format. .

[0051] Furthermore, the detection head is based on the DSFR DETR Decoder module, which uses a Transformer architecture for target detection. The DSFR DETR Decoder module converts the multi-scale features output by the adaptive feature fusion neck network into unified hidden dimension features through the input feature layer. The hidden dimension features are then input into the Transformer decoder, where deformable self-attention mechanism and feedforward network are used to fuse the hidden dimension features to enhance feature representation capabilities.

[0052] Furthermore, the formula for the deformable self-attention mechanism is defined as follows: ; in, Features representing input deformable self-attention mechanisms; To index attention heads, This represents the total number of attention heads indexed. The key for index sampling, The total number of keys sampled for the index; Given a reference point, To query features, Used to represent Location, Indicates the first The first index attention head and the first The position offset of the key sampled by each index is obtained by the query through the fully connected layer; represents the weights of the deformable self-attention mechanism, with values ​​ranging from [0,1], and ; This represents the value obtained by interpolation based on the position of a given reference point; These are the weight values ​​corresponding to low-frequency semantic features. These are the weight values ​​corresponding to high-frequency semantic features, and .

[0053] Specifically, such as Figure 4 As shown, the constructed lightweight end-to-end network consists of three main components: a lightweight backbone network, an adaptive feature fusion neck network, and a detection head (decoder). Basic data augmentation strategies are introduced during the data preprocessing stage. In the network stage, an efficient backbone feature extractor is first used to extract features from the input image, dynamically adjusting convolutional kernel parameters to adapt to features of different sizes. The feature maps (P3, P4, P5) from the last three stages of the backbone network are input into the Neck. The Neck transforms multi-scale features into image feature sequences using multiple convolutional branches and residual connections through intra-scale feature interaction and cross-scale feature fusion. Then, a multi-task defect detection head is executed, employing minimum uncertainty query selection to choose a fixed number of encoder features as the initial object query for the decoder. Finally, the decoder with an auxiliary prediction head iteratively optimizes the object query to generate categories and bounding boxes.

[0054] The lightweight backbone network is a lightweight end-to-end network responsible for extracting features from the input image and constructing the feature pyramid. Based on the RT-DETR-ResNet18 backbone, it improves upon it by proposing the EfficientRepresentation Cross Stage Aggregation Network (EffiRepCSANet) to replace the original ResNet18 backbone. This lightweight backbone network improves the overall feature extraction performance by combining the CSP (Cross Stage Partial) and ELAN (Efficient Layer Aggregation Network) structures. It further reduces the spatial resolution of the feature maps by implementing average pooling downsampling through the ADown module (a downsampling module used to reduce the resolution of feature maps in the backbone network, while typically increasing the number of feature channels), thereby increasing the number of channels. Furthermore, it allows the network to capture information at different scales, making it crucial for constructing the feature pyramid.

[0055] The lightweight backbone network is constructed starting with three convolutional layers for downsampling and feature extraction. First, two Conv modules expand the number of channels in the input image from 3 to 64, then to 128, halving the spatial resolution of the feature map at each step. This is achieved by setting the stride to 2, while the autopad function automatically calculates padding to maintain the spatial size of the feature map. Next, the backbone introduces the Efficient Representation Cross Stage Aggregation (EffiRepCSA) module. This module mainly consists of EffiRep and Conv, branching out into multiple convolutional layers and finally merging the feature maps from these branches to simplify the backbone network structure and enhance feature representation.

[0056] EffiRep is the core of EffiRepCSA, designed to achieve rich features and stable gradients during training through reparameterization. Structurally, Efficient Convolutional Dyn-Reparameterization Bottleneck (EffiRepBottleneck) is a basic module with a residual structure, repeatedly stacked in the backbone (similar to the C3 module in YOLOv5 and the C2f module in YOLOv8). As the number of channels increases, it gradually builds deep feature representations. Its optimization advantage stems from the multi-branch structure EffiRCon used during training. This design is considered an implicit regularization mechanism; the parallel computation of multiple branches is equivalent to integrating gradients, explicitly reducing the variance of gradient estimation. This effectively reduces gradient oscillations during training, making the optimization direction more stable and accurate.

[0057] Therefore, EffiRepCSA reduces training resource consumption without sacrificing the model's final representation capability and inference performance, making the technology more practical and scalable when applied to more complex model designs.

[0058] The last few modules of the lightweight backbone network still combine EffiRepCSA and ADown until the maximum number of channels (1024) is reached. This combined module outputs a high-dimensional feature map, providing rich feature information for subsequent neck and head detection. The improved backbone network effectively extracts image features through deep convolutional networks and residual connections, progressively constructing a feature pyramid. This design enables the network to capture multi-scale features from low to high levels, providing a foundation for accurate object detection. The specific structure of the lightweight backbone network is as follows: Figure 5 As shown.

[0059] The adaptive feature fusion neck network acts as a bridge connecting the lightweight backbone network and the detection head, responsible for further processing and fusion of the features extracted by the lightweight backbone network. The adaptive feature fusion neck network adopts the FPN (Feature Pyramid Network) structure, and realizes feature map upsampling and fusion through the Upsample and Concat modules. The key self-attention component, Adaptive Hi-Lo Fusion (AHLF), is improved by introducing the HiLo component to improve the AIFI (Attention-based Intra-scale Feature Interaction) module in the hybrid encoder architecture of the baseline RT-DETR. The AIFI module performs intra-scale feature interaction, and its core mechanism is the standard multi-head self-attention (MHSA). Before performing cross-scale feature fusion, it performs independent feature refinement and context modeling on each single-scale feature map from the backbone network. Although the standard AIFI module is effective, its MHSA has two inherent limitations: first, the computational complexity is quadratic with the spatial dimension (N=H×W), resulting in high computational costs; second, it adopts a homogeneous approach to all patterns in the feature map, whether high-frequency details or low-frequency structures. AIFI is optimized by introducing the HiLo (High-frequency and Low-frequency) attention mechanism through a frequency-domain decoupling design paradigm. This paradigm assumes that visual features can be decomposed into high-frequency and low-frequency components and then heterogeneously processed.

[0060] The improved AIFI, or AHLF, replaces the original MHSA with HiLo attention, which includes two parallel branches: (1) High-frequency branch: Local Window Attention. This branch divides the feature map into non-overlapping local windows in space and performs self-attention computation independently only within each window. This strategy restricts the scope of attention operations to the local neighborhood, thereby efficiently modeling high-frequency information with linear complexity. It aims to efficiently capture high-frequency spatial details (such as edges and textures).

[0061] (2) Low-frequency branch: Global Downsampled Attention. This branch first performs average pooling with a large stride on the Key and Value matrices to generate a low-resolution version that is significantly reduced in spatial dimension. Then, global attention is calculated using the Query at the original resolution and the downsampled Key and Value. This method significantly reduces the computational cost of obtaining the global receptive field by approximating global interaction in a low-dimensional space. It models low-frequency global contexts (such as object structure and scene layout) with low computational cost.

[0062] This design allows AHLF to maintain low computational cost while modeling both global structure and local details in images, thus improving the detection performance of targets at different scales in models such as DSFR-DETR. Simultaneously, it captures global dependencies through a self-attention mechanism, improving feature quality early on and encoding information before feature fusion to reduce the risk of information loss. Then, upsampling and concatenation operations are used to upsample the high-resolution feature map output from the backbone, followed by concatenation with lower-level feature maps to achieve multi-scale feature fusion. This process not only preserves image details but also integrates deep semantic information, providing rich feature support for head detection tasks. Finally, further downsampling, concatenation, and EffiRepCSA processing unify the feature maps at different scales. The main structure of the AHLF self-attention component is as follows: Figure 6 As shown.

[0063] The entire AHLF algorithm can be summarized as a function. The overall process can be represented as follows: ; in, To input features for High-frequency and Low-frequency attention, ; Indicates dimensional transformation; Indicates the input Perform dimensional transformation; divide the attention head into two groups, high-frequency branch and low-frequency branch, according to a preset ratio, and perform calculations separately. Indicates a high-frequency branch. For the low-frequency branch, the corresponding feature map is processed in parallel in the high-frequency branch and the low-frequency branch. This indicates that the output features of the high-frequency branch and the low-frequency branch are spliced ​​together along the channel dimension. This represents the inverse process of the initial dimensionality transformation operation, restoring the dimensions of the fused features to the standard image data format. .

[0064] The detection head is based on the DSFR DETR Decoder module, which uses a Transformer architecture for object detection. First, the multi-scale feature maps output from the previous network are transformed to a unified hidden dimension through the input feature layer. These feature maps are then fed into the Transformer decoder, which consists of the DeformableTransformerDecoderLayer from AHLF. The decoder performs deep fusion of features through a deformable self-attention mechanism and a feedforward network to enhance feature representation. Simultaneously, the deformable self-attention mechanism significantly reduces computational cost. The formula for the deformable self-attention mechanism is defined as follows: ; in, Features representing input deformable self-attention mechanisms; To index attention heads, This represents the total number of attention heads indexed. The key for index sampling, The total number of keys sampled for the index; Given a reference point, To query features, Used to represent Location, Indicates the first The first index attention head and the first The position offset of the key sampled by each index is obtained by the query through the fully connected layer; represents the weights of the deformable self-attention mechanism, with values ​​ranging from [0,1], and ; This represents the value obtained by interpolation based on the position of a given reference point; These are the weight values ​​corresponding to low-frequency semantic features. These are the weight values ​​corresponding to high-frequency semantic features, and .

[0065] Dataset building module 202: Used to build training sets based on JSON files in the ZJU-Leaper dataset; In the specific implementation of this invention, the ZJU-Leaper dataset is used to construct the training set. The ZJU-Leaper dataset is the largest fabric image dataset to date, classified by texture, containing 19 texture categories, of which 4 are solid color fabric textures; it contains 71,127 defect-free images and 27,650 defective images, which were collected from factories or laboratories; defects in the fabric images are annotated with bounding boxes; the location and size of defects in the ZJU-Leaper dataset have high diversity; the training set is divided into 63,814 images (including defective and defect-free images) by the JSON file in the ZJU-Leaper dataset, while the test set contains all remaining images.

[0066] Training module 203: is used to input the training images in the training set into the lightweight end-to-end network, and perform gradient stabilization training by dynamically adjusting the convolution kernel parameters and processing high and low frequency features in parallel to form a converged lightweight end-to-end network. In the specific implementation of this invention, before training, it is necessary to set the hyperparameters of the lightweight end-to-end network. That is, the parameters in the lightweight end-to-end network are randomly initialized. In order to simulate the usage of actual application scenarios, the lightweight end-to-end network is trained, and the image size is set to 512×512, the number of epochs is 300 (because the curves of various indicators have not yet converged after 100 epochs), the batch size is 24, the learning rate is 0.0001, and the learning rate factor is 0.1. The parameters of the three key scaling dimensions of each model in the lightweight end-to-end network are uniformly set to [0.33, 0.50, 1024], and the same enhancement strategy is adopted. During training, it is all run on the AutoDL network server, which is configured with an RTX 3090 (24GB) GPU and an AMD EPYC 9754 128-CoreProcessor CPU, 60GB of memory.

[0067] The training images from the training set are input into the lightweight end-to-end network, and then the network is trained according to the preset training parameters. After training is completed, the lightweight end-to-end network is formed. At this point, it is necessary to use evaluation metrics to confirm whether the lightweight end-to-end network has converged; therefore, a test set is required.

[0068] The test set is used to test the accuracy of the trained lightweight end-to-end network in predicting defects in test images, specifically whether it can predict the location and type of defects in the test images. For defect localization, this embodiment uses the intersection-over-union ratio (IoU) to determine its accuracy, calculated as follows: ; in, and These represent the predicted bounding box and the final bounding box, respectively; if IoU > 0.5, then it is considered... It is accurate. For defects belonging to ground truth category C, the classification score of the bounding box containing the defect is used to determine whether the defect is correctly classified. When the classification score is higher than the predefined threshold tcls, the classification result is considered accurate. When both the IoU and classification score of the predicted defect bounding box satisfy the above constraints, it is a true positive (TP); otherwise, it is a false positive (FP). When the predicted defect-free bounding box contains the defect, it is a false negative (FN); otherwise, it is a true negative (TN). Therefore, precision p and recall r can be defined as follows: ; Precision and recall are two contradictory metrics; increasing the threshold tcls in a specific model determines the trade-off between precision and recall in that region, i.e., the area under the precision-recall curve when adjusting tcls; AP is estimated using eleven-point interpolation, which can be expressed as: ; Here, ρ represents the precision-recall curve; the average precision (mAP) across all classes is used as the core evaluation metric, where mAP50 and mAP50-95 represent the average precision under different intersection-over-union (IoU) thresholds, respectively; recall measures the completeness of the model in identifying all real targets, and this parameter directly affects the false negative rate, making it crucial in industrial detection; model parameters (Params) are used to evaluate the model's storage requirements, while floating-point operations (GFLOPs) measure the computational resources required for a single inference; in addition, frames per second (FPS) is a key indicator for evaluating the model's inference speed, with higher FPS indicating higher processing efficiency; model training efficiency is measured by the number of iterations required to reach a specific accuracy target, with fewer iterations meaning faster convergence and lower training costs; memory consumption measures the total amount of GPU memory and system memory used by the model during inference or training.

[0069] When the evaluation criteria are met, the trained lightweight end-to-end network is a converged lightweight end-to-end network. If the evaluation criteria are not met, the parameters of the trained lightweight end-to-end network are updated using the backpropagation algorithm, and the training images in the training set are used for retraining until the evaluation criteria are met, thus obtaining a converged lightweight end-to-end network.

[0070] Defect detection module 204: It is used to input the image data of the fabric to be detected into a converged lightweight end-to-end network, and use intra-scale feature interaction and cross-scale feature fusion processing to output the fabric defect classification and bounding box localization results corresponding to the image data of the fabric to be detected.

[0071] In the specific implementation process of this invention, it is first necessary to obtain the image data of the fabric to be detected, then input the image data of the fabric to be detected into a converged lightweight end-to-end network for defect detection processing, and finally output the fabric defect detection result corresponding to the image data of the fabric to be detected.

[0072] In this embodiment of the invention, a lightweight end-to-end network is provided. When applied to a fabric defect dataset, it improves the accuracy (mAP) by 11.5% compared to Faster R-CNN and by 1.9% compared to the single-stage YOLOv8s model. During training, the gradient norm fluctuation is minimal and the value is low, resulting in more efficient and stable surface optimization trajectories with lower risk. The EffiRepCSA module is used in the lightweight backbone network, reducing GFLOPs (computational cost) by 69.2%. With fewer backbone network parameters, it is very suitable for deployment in resource-constrained industrial embedded devices. Furthermore, it performs excellently on cross-domain datasets such as TILDA and exhibits strong robustness to common industrial interferences such as motion blur and salt-and-pepper noise. Therefore, the method in this embodiment can improve the efficiency of feature extraction, convergence speed, and robustness to industrial noise while maintaining a lightweight network, thereby enhancing the accuracy of fabric defect identification and improving the network's generalization ability.

[0073] This invention provides a computer-readable storage medium storing a computer program. When executed by a processor, the program implements the fabric defect detection method of any of the above embodiments. The computer-readable storage medium includes, but is not limited to, any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic cards, or optical cards. In other words, the storage device includes any medium that can store or transmit information in a readable form by a device (e.g., a computer, a mobile phone), and can be a read-only memory, a disk, or an optical disk, etc.

[0074] This invention also provides a computer application running on a computer, which is used to execute the fabric defect detection method of any of the above embodiments.

[0075] also, Figure 3 This is a schematic diagram of the structural composition of the electronic device in an embodiment of the present invention.

[0076] This invention also provides an electronic device, such as... Figure 3 As shown. The electronic device includes a processor 302, a memory 303, an input unit 304, and a display unit 305, among other devices. Those skilled in the art will understand that... Figure 3 The structural components of the illustrated electronic device do not constitute a limitation on all devices and may include more or fewer components than illustrated, or combine certain components. Memory 303 can be used to store application program 301 and various functional modules. Processor 302 runs application program 301 stored in memory 303, thereby performing various functional applications and data processing of the device. Memory can be internal memory or external memory, or both. Internal memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, or random access memory. External memory may include hard disks, floppy disks, ZIP disks, USB flash drives, magnetic tapes, etc. The memory disclosed in this invention includes, but is not limited to, these types of memory. The memory disclosed in this invention is only an example and not a limitation.

[0077] Input unit 304 is used to receive signal input and user-input keywords. Input unit 304 may include a touch panel and other input devices. The touch panel can collect user touch operations on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel) and drive the corresponding connection device according to a pre-set program; other input devices may include, but are not limited to, one or more of physical keyboards, function keys (such as play control buttons, power buttons, etc.), trackballs, mice, joysticks, etc. Display unit 305 can be used to display user-input information or information provided to the user, as well as various menus of the terminal device. Display unit 305 may be in the form of a liquid crystal display, organic light-emitting diode, etc. Processor 302 is the control center of the terminal device, connecting various parts of the entire device through various interfaces and lines, and performing various functions and processing data by running or executing software programs and / or modules stored in memory 303, and calling data stored in memory.

[0078] As one embodiment, the electronic device includes: one or more processors 302, a memory 303, and one or more application programs 301, wherein the one or more application programs 301 are stored in the memory 303 and configured to be executed by the one or more processors 302, and the one or more application programs 301 are configured to perform the fabric defect detection method corresponding to any of the above embodiments.

[0079] In this embodiment of the invention, a lightweight end-to-end network is provided. When applied to a fabric defect dataset, it improves the accuracy (mAP) by 11.5% compared to Faster R-CNN and by 1.9% compared to the single-stage YOLOv8s model. During training, the gradient norm fluctuation is minimal and the value is low, resulting in more efficient and stable surface optimization trajectories with lower risk. The EffiRepCSA module is used in the lightweight backbone network, reducing GFLOPs (computational cost) by 69.2%. With fewer backbone network parameters, it is very suitable for deployment in resource-constrained industrial embedded devices. Furthermore, it performs excellently on cross-domain datasets such as TILDA and exhibits strong robustness to common industrial interferences such as motion blur and salt-and-pepper noise. Therefore, the method in this embodiment can improve the efficiency of feature extraction, convergence speed, and robustness to industrial noise while maintaining a lightweight network, thereby enhancing the accuracy of fabric defect identification and improving the network's generalization ability.

[0080] Furthermore, the above provides a detailed description of the dynamic multi-scale fusion fabric defect detection method and related apparatus provided by the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A dynamic multi-scale fusion method for fabric defect detection, characterized in that, The method includes: A lightweight end-to-end network is constructed, which sequentially includes a lightweight backbone network, an adaptive feature fusion neck network, and a detection head; The training set was constructed based on the JSON files in the ZJU-Leaper dataset. The training images in the training set are input into the lightweight end-to-end network. Gradient stabilization training is performed by dynamically adjusting the convolution kernel parameters and processing high and low frequency features in parallel, thus forming a convergent lightweight end-to-end network. The image data of the fabric to be detected is input into a converged lightweight end-to-end network. The network is processed by intra-scale feature interaction and cross-scale feature fusion to output the fabric defect classification and bounding box localization results corresponding to the image data of the fabric to be detected.

2. The fabric defect detection method according to claim 1, characterized in that, The lightweight backbone network is an EffiRepCSANet network. The EffiRepCSANet combines the CSP structure with the Efficient LayerAggregation Network structure and performs average pooling downsampling of the feature map through the Adown module. The EffiRepCSA module is introduced into the EffiRepCSANet. The EffiRepCSA module is composed of EffiRep and Conv. EffiRep is the core of the EffiRepCSA module, which achieves feature enrichment and gradient stability during training through reparameterization. EffiRep contains EffiRepBottleneck, which is a basic module with a residual structure and is repeatedly stacked in the backbone. As the number of channels increases, deep feature representations are gradually built.

3. The fabric defect detection method according to claim 1, characterized in that, The adaptive feature fusion neck network serves as a bridge connecting the lightweight backbone network and the detection head, and is used to further process and fuse the features formed by the lightweight backbone network. The adaptive feature fusion neck network adopts a Feature Pyramid Network structure and introduces the Adaptive Hi-Lo Fusion key self-attention component, which realizes feature upsampling and fusion through the Upsample and Concat modules; The key self-attention component of the Adaptive Hi-Lo Fusion is to replace the standard multi-head attention with two parallel branches of high-frequency and low-frequency attention.

4. The fabric defect detection method according to claim 3, characterized in that, The two parallel branches of the High-frequency and Low-frequency attention include a high-frequency branch and a low-frequency branch; The high-frequency branch is local window attention, which divides the received features into non-overlapping local windows in space and performs self-attention calculation independently only within each local window. The low-frequency branch is a global downsampling attention that performs large-step average pooling on the Key and Value matrices to generate a low-resolution version of them in terms of spatial dimension.

5. The fabric defect detection method according to claim 3, characterized in that, The execution functions for the High-frequency and Low-frequency attention are as follows: ； in, To input features for High-frequency and Low-frequency attention, ; Indicates dimensional transformation; Indicates the input Perform dimensional transformation; divide the attention head into two groups, high-frequency branch and low-frequency branch, according to a preset ratio, and perform calculations separately. Indicates a high-frequency branch. For the low-frequency branch, the corresponding feature map is processed in parallel in the high-frequency branch and the low-frequency branch. This indicates that the output features of the high-frequency branch and the low-frequency branch are spliced ​​together along the channel dimension. This represents the inverse process of the initial dimensionality transformation operation, restoring the dimensions of the fused features to the standard image data format. .

6. The fabric defect detection method according to claim 1, characterized in that, The detection head is based on the DSFR DETRDecoder module as its core component, and the DSFR DETR Decoder module performs target detection using a Transformer architecture. The DSFR DETR Decoder module converts the multi-scale features output by the adaptive feature fusion neck network into unified hidden dimension features through the input feature layer; The hidden dimension features are input into the Transformer decoder, where a deformable self-attention mechanism and a feedforward network are used to fuse the hidden dimension features to enhance the feature representation capability.

7. The fabric defect detection method according to claim 6, characterized in that, The formula for the deformable self-attention mechanism is defined as follows: ； in, Features representing input deformable self-attention mechanisms; To index attention heads, This represents the total number of attention heads indexed. The key for index sampling, The total number of keys sampled for the index; Given a reference point, To query features, Used to represent Location, Indicates the first The first index attention head and the first The position offset of the key sampled by each index is obtained by the query through the fully connected layer; represents the weights of the deformable self-attention mechanism, with values ​​ranging from [0,1], and ; This represents the value obtained by interpolation based on the position of a given reference point; These are the weight values ​​corresponding to low-frequency semantic features. These are the weight values ​​corresponding to high-frequency semantic features, and .

8. A dynamic multi-scale fusion fabric defect detection device, characterized in that, The device includes: Network building module: used to build a lightweight end-to-end network, which includes a lightweight backbone network, an adaptive feature fusion neck network, and a detection head in sequence; Dataset building module: Used to build training sets based on JSON files in the ZJU-Leaper dataset; Training module: Used to input training images from the training set into the lightweight end-to-end network, and perform gradient stabilization training by dynamically adjusting the convolution kernel parameters and processing high and low frequency features in parallel to form a converged lightweight end-to-end network; Defect detection module: It is used to input the image data of the fabric to be detected into a converged lightweight end-to-end network, and use intra-scale feature interaction and cross-scale feature fusion processing to output the fabric defect classification and bounding box localization results corresponding to the image data of the fabric to be detected.

9. An electronic device comprising a processor and a memory, characterized in that, The processor runs a computer program or code stored in the memory to implement the fabric defect detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium for storing computer programs or code, characterized in that, When the computer program or code is executed by a processor, the fabric defect detection method as described in any one of claims 1 to 7 is implemented.

Images