A fabric defect classification method and system fusing multi-scale dynamic convolution and graph embedding constraints

Abstract

A fabric defect classification method and system fusing multi-scale dynamic convolution and graph embedding constraints, aiming at the problems that existing methods are difficult to model irregular defects, similar defects are easy to confuse, and small sample generalization is weak, hierarchical multi-scale features are extracted through Swin Transformer; then a dynamic multi-scale refining module is designed, different receptive field features are fused through context-aware dynamic convolution, scale-dependent is modeled through cross-scale collaborative attention, and high discriminative features are output; the refined features are input into an integrated deep random vector functional connection network, and a boundary-aware graph embedding constraint based on local Fisher discriminant analysis is innovatively introduced in the optimization target, the feature manifold structure is explicitly modeled by constructing same-class intrinsic graph and different-class penalty graph, and the intra-class compactness and inter-class separation degree are strengthened. The present invention significantly reduces the misjudgment rate of similar defects, and meets the needs of industrial real-time detection.

Description

Technical Field

[0001] This invention belongs to the field of fabric defect detection and classification technology, specifically involving a fabric defect classification method and system that integrates multi-scale dynamic convolution and graph embedding constraints, and in particular a collaborative optimization architecture based on Swing Transformer, dynamic convolution fusion module, cross-scale collaborative attention and integrated deep random vector functional connection network. Background Technology

[0002] Fabric defect detection is a core aspect of quality control in the textile industry. Defects such as holes, stains, broken yarns, and mis-weaves directly affect the appearance and lifespan of products. In high-precision fabrics (such as automotive interiors and medical textiles), even the smallest flaw can lead to the scrapping of an entire batch of products. Traditional manual visual inspection methods are not only inefficient and labor-intensive, but also susceptible to subjective fatigue, resulting in highly volatile rates of missed and false detections, making them unsuitable for the real-time quality inspection requirements of modern high-speed weaving production lines.

[0003] With the development of computer vision, early defect detection methods based on handcrafted features (such as gray-level co-occurrence matrix, Gabor filter, and local binary pattern) classified defects by extracting texture statistics. However, fabric defects have diverse morphologies (points, lines, areas) and large scales (from sub-millimeter-level fuzz to centimeter-level holes), and the background texture itself has periodicity or randomness. Handcrafted features are unable to characterize complex nonlinear distortions, resulting in severely insufficient generalization ability.

[0004] In recent years, deep learning models, especially convolutional neural networks (CNNs), have made significant progress in defect classification tasks. Classic CNNs such as ResNet and VGG extract hierarchical features by stacking convolutional and pooling layers, and can automatically learn representations from edges to semantics. However, the receptive field of CNNs is limited by the size of the convolutional kernel, resulting in a weak ability to model long-distance dependencies. They are prone to misclassifying continuous defects across regions (such as longitudinal stripes or misaligned latitudes) as multiple independent small defects. To expand the receptive field, researchers have introduced dilated convolutions or feature pyramids, but this increases the number of parameters and computational complexity.

[0005] The emergence of Visual Transformer (ViT) has provided a new approach to global feature modeling. ViT divides images into block sequences and uses a self-attention mechanism to capture global dependencies, achieving performance comparable to or even surpassing CNNs in image classification tasks. However, the computational complexity of self-attention in standard ViT is proportional to the square of the number of image blocks, making it difficult to process high-resolution fabric images in real time. Swin Transformer reduces computational complexity to linear by introducing window-based self-attention and moving window self-attention, while outputting multi-scale features through a hierarchical structure (stage-by-stage downsampling), taking into account both local details and global semantics, making it more suitable for fabric defect analysis. However, Swin Transformer's window is fixed as a regular grid, which is still insufficient for adapting to non-rigid, irregularly shaped defects (such as randomly distributed oil stains and wrinkles); in addition, its self-attention mechanism mainly relies on spatial location correlation, and does not fully mine discriminative features in the channel dimension.

[0006] On the other hand, randomized neural networks (such as stochastic vector functional link networks, RVFL) have attracted attention due to their fast training speed and good generalization performance. RVFL randomly initializes and fixes the weights from the input layer to the hidden layer, and solves the output layer weights using only the least squares method, avoiding iterative optimization through backpropagation, resulting in training efficiency that is 1-2 orders of magnitude higher than traditional CNNs. However, traditional RVFLs have a single hidden layer structure, which has limited nonlinear expressive power and is difficult to handle high-dimensional and complex fabric defect features. To address this, researchers have proposed deep RVFLs and ensemble deep RVFLs, which enhance feature abstraction capabilities by stacking multiple random mapping layers and cascading between layers. However, existing ensemble deep RVFLs only consider empirical errors and L2 regularization during optimization, ignoring the geometric distribution of samples in the feature space—similar defect samples should be more compact after projection, and dissimilar defects should be far apart, especially near the decision boundary. For defect categories with similar morphologies (such as stains and bumps, holes and linear breaks), it is difficult to distinguish them using only the distance metric in the original feature space, leading to classification confusion.

[0007] Graph embedding techniques introduce manifold learning concepts into classifier training by constructing similarity graphs (intrinsic graphs) and difference graphs (penalty graphs) between samples, explicitly enhancing intra-class aggregation and inter-class separation. However, existing graph embedding methods are mostly used for dimensionality reduction or semi-supervised learning and have not yet been deeply integrated with randomized neural networks (especially ensemble deep RVFLs). Furthermore, labeled samples are scarce in industrial scenarios, and deep models are prone to overfitting under small sample conditions, necessitating a solution that combines strong discriminative power, high training efficiency, and low computational cost.

[0008] In summary, existing fabric defect classification methods still have significant shortcomings in terms of multi-scale adaptability of feature extraction, modeling of irregular defects, utilization of geometric structure in classification decisions, and generalization ability with small samples. Summary of the Invention

[0009] Existing technologies for fabric defect classification suffer from poor adaptability to complex textures, irregular shapes, and small sample scenarios. To address these technical bottlenecks, this invention proposes a collaborative optimization architecture that integrates multi-scale dynamic convolution and graph embedding constraints. It extracts hierarchical features using the Swing Transformer, refines multi-scale information through dynamic convolution fusion and cross-scale attention, and innovatively introduces graph embedding constraints based on local Fisher discriminant analysis into the integrated deep RVFL, thereby achieving high-precision and robust fabric defect classification under lightweight conditions.

[0010] This invention significantly improves classification accuracy and generalization ability through collaborative feature extraction, dynamic multi-scale refinement, and structured classification decision-making.

[0011] This invention proposes a fabric defect classification method that integrates multi-scale dynamic convolution and graph embedding constraints, comprising the following steps: (1) Preprocess the fabric image; (2) Extracting multi-scale features using Swing Transformer: The preprocessed image is input into the Swing Transformer network, the preprocessed image is divided into 4x4 non-overlapping blocks and linearly embedded, and multi-scale features are extracted through four progressive stages. In each stage, multi-head self-attention based on 7x7 regular window and multi-head self-attention based on 7x7 moving window are alternately executed to capture micro-texture distortion in local windows and to identify cross-regional structural anomalies by using information interaction between windows. After each stage, the feature map resolution is halved and the number of channels is doubled through block merging operation, and multi-scale features are output. (3) Perform dynamic multi-scale refinement on the multi-scale features extracted in step (2): The multi-scale features are processed using a context-aware dynamic convolutional fusion module, which is a 3-branch parallel structure that integrates depthwise separable convolution, dilated convolution, and deformable convolution in parallel. In each branch, global average pooling is performed on the multi-scale features to obtain a global descriptor. Adaptive fusion weights for each branch are generated through a two-layer fully connected network, and the outputs of each branch are weighted and summed according to these weights to generate multi-receptive field fusion features. The multi-receptive field fusion feature is then processed by the cross-scale collaborative attention module. Sub-features of different scales are obtained by group convolution decomposition, and the correlation between scales is dynamically modeled to obtain the correlation matrix between scales. After Softmax normalization, each sub-feature is weighted and fused, and then residual connection is performed with the multi-receptive field fusion feature to achieve adaptive fusion of fine-grained details and coarse-grained semantics. The outputs of the dynamic convolutional fusion module and the cross-scale collaborative attention module are concatenated along the channel dimension to form the refined features; (4) The refined features are integrated into a deep random vector functional connection network. The integrated deep random vector functional connection network contains multiple levels of random feature enhancement layers. Each enhancement layer independently solves the output probability. A boundary-aware graph embedding constraint based on local Fisher discriminant analysis is introduced into the optimization objective. This constraint is calculated by constructing the intrinsic graph of the same type of sample and the penalty graph of the opposite type of sample, respectively, and the graph Laplacian matrix is ​​calculated and added to the loss function in the form of a regularization term. (5) Weigh and fuse the output probabilities of each enhancement layer, and use the category with the highest probability as the classification result.

[0012] Preferably, in step (2), the number of output channels in each of the four progressive stages of the Swin Transformer is 96, 192, 384, and 768, respectively.

[0013] Preferably, the preprocessing in step (1) includes scaling, normalizing, and dynamic data augmentation of the fabric image.

[0014] Preferably, in the context-aware dynamic convolutional fusion module in step (3), the intermediate layer of the two fully connected networks uses the ReLU activation function, and the sum of the adaptive fusion weights of each branch is 1. Preferably, the boundary-aware graph embedding constraint in step (4) is constructed based on the local Fisher discriminant analysis criterion to construct a graph matrix: for samples of the same class, an intrinsic graph is constructed, and its weight is defined as the similarity between samples divided by the total number of samples of that class; for samples of different classes, a penalty graph is constructed, and its weight is calculated differently depending on whether the samples are of the same class; based on the above weight matrix, the degree matrices of the intrinsic graph and the penalty graph are calculated respectively to obtain the graph Laplacian matrix, and then the structural constraint matrix is ​​obtained; the structural constraint matrix is ​​introduced into the loss function of the integrated deep random vector functional connection network in the form of a regularization term to force intra-class compactness and inter-class separation.

[0015] Preferably, the integrated deep random vector functional network includes an input layer, a multi-level random feature enhancement layer, and a fusion layer; the input layer receives refined features as shared input features for all enhancement layers; the multi-level random feature enhancement layer specifically includes three levels, which are concatenated. Each layer performs a nonlinear transformation to generate nonlinear enhanced features through randomly initialized and fixed weight matrices and ReLU activation functions. The enhanced features of the previous level are concatenated with the shared input features and used as the input of the next level enhancement layer; the fusion layer performs weighted fusion of the output probabilities of all levels.

[0016] Preferably, the weight matrix of the multi-level random feature enhancement layer is randomly initialized to conform to the He normal distribution and remains fixed after random initialization; each enhancement layer independently solves the output weight parameters under regularization constraints, adopts a dynamic adaptive regularization strategy, and the regularization coefficient decays exponentially with the depth of the network layer and is adjusted by the feedback of the residual of the previous layer.

[0017] Preferably, the preprocessing of the fabric image includes: scaling the input image to a fixed number of pixels and performing channel normalization; performing dynamic data enhancement several times, randomly selecting 1 to 5 enhancement methods each time, the enhancement methods including geometric transformation, semantic enhancement and random blending enhancement; geometric transformation includes horizontal flipping, vertical flipping, random rotation and scaling; semantic enhancement includes adjusting brightness and contrast and adding Gaussian noise; random blending enhancement refers to fusing textures of different defect areas at random proportions.

[0018] This invention also provides a fabric defect classification system that integrates multi-scale dynamic convolution and graph embedding constraints, used to execute the fabric defect classification method that integrates multi-scale dynamic convolution and graph embedding constraints, including: The fabric image preprocessing module is used to preprocess fabric images; The Swin Transformer feature extraction module is used to extract hierarchical multi-scale features using the Swin Transformer. The dynamic multi-scale refinement module includes a context-aware dynamic convolutional fusion module and a cross-scale collaborative attention module, which are used to adaptively fuse multi-scale features and model inter-scale dependencies. The integrated deep random vector function connects to the network classifier module to receive refined features and identify defect types through multi-level random feature enhancement layers and boundary-aware map embedding constraints. The classification and recognition module is used to weight and fuse the output probabilities of each enhancement layer, and use the category with the highest probability as the classification result.

[0019] Preferably, the integrated deep random vector functional connection network classifier module embeds a graph embedding constraint unit based on local Fisher discriminant analysis, which is used to construct the intrinsic graph and the penalty graph and calculate the graph Laplacian matrix to optimize the output weights of the classifier. The overall model parameter count of the method does not exceed 30M, and the inference delay of a single image does not exceed 20ms.

[0020] Beneficial effects:

[0021] (1) Collaborative optimization architecture: The innovative combination of the moving window attention of the Swing Transformer with the backpropagation-free multi-layer random mapping of the integrated deep random vector function network avoids the overfitting and gradient failure problems in high-dimensional small sample scenarios, and the training speed is more than 80% faster than large-scale visual Transformers.

[0022] (2) Context-aware dynamic convolution fusion: By integrating depthwise separable, dilated and deformable convolutions in parallel and introducing adaptive weight learning, the model can dynamically select the best convolution combination according to the shape of the defect, which improves the separation accuracy of irregular defects by 3-4 percentage points.

[0023] (3) Boundary-aware graph embedding constraints: For the first time, the local Fisher discriminant analysis criterion is integrated into the integrated deep random vector functional connection network optimization framework. By constructing decision boundary-sensitive intrinsic graphs and penalty graphs, the misclassification rate is reduced by 12.7% in the category of defects with similar shapes (stains and protrusions), and the F1 score is improved by more than 6% in small sample scenarios.

[0024] (4) Lightweight and real-time performance: The total number of parameters is only 28.4M, and the inference delay is 18.2ms / image, which meets the real-time detection requirements of industrial production lines. Attached Figure Description

[0025] Figure 1 This is an overall architecture diagram of the method of the present invention, which shows the complete process from the input image to the Swing Transformer, the context-aware dynamic convolutional fusion module, the cross-scale collaborative attention module, the integrated deep random vector functional connection network (including graph embedding) and finally to the classification output; Figure 2 This is an example of a feature activation heatmap of fabric defects based on the model of this invention; Figure 3A and Figure 3B This is a comparison diagram of the confusion matrix between the model of this invention and the comparative model; Figure 4A and Figure 4B This is a comparison chart of the ROC curves of the model of this invention and the comparative model. Detailed Implementation

[0026] A fabric defect classification method integrating multi-scale dynamic convolution and graph embedding constraints includes: (1) Image preprocessing: The input image is uniformly scaled to a fixed pixel size (e.g., 224×224), and bilinear interpolation is used to maintain the aspect ratio. Blank areas are filled with zero values. Channel normalization is performed to linearly map RGB values ​​to the [0,1] interval. Dynamic data enhancement is performed multiple times, and 1-5 transformations are randomly selected each time (geometric transformation: horizontal / vertical flip, random rotation, scale scaling; semantic enhancement: brightness / contrast adjustment, adding Gaussian noise; random blending enhancement: fusion of different defect area textures according to random proportions). The enhanced image is mixed with the original data in a 7:3 ratio.

[0027] (2) Swin Transformer Multi-scale Feature Extraction: A pre-trained Swin Transformer (such as Swin-Tiny) is used as the backbone network. The preprocessed image is divided into 4×4 non-overlapping blocks and converted into feature vectors through linear embedding. The network contains four progressive stages, each of which alternates between multi-head self-attention based on a regular window and multi-head self-attention based on a moving window, with the window size fixed at 7×7. The regular window self-attention calculates self-attention within the local window to capture yarn-level micro-distortions (such as single yarn breakage and fuzziness); the moving window self-attention achieves cross-window information interaction by offsetting the window boundary to identify cross-regional structural anomalies (such as long-distance warp stripes and weft misalignment). After each stage, the feature map resolution is halved and the number of channels is doubled through image block merging (the first stage output is 56×56×96, the second stage is 28×28×192, the third stage is 14×14×384, and the fourth stage is 7×7×768). The final output is a hierarchical multi-scale feature that combines local details and global semantics.

[0028] (3) Dynamic multi-scale feature refinement: (3a) Context-aware dynamic convolutional fusion module: Employing a multi-branch parallel structure, it integrates depthwise separable convolutions (3×3, lightweight for extracting inter-channel features), dilated convolutions (3×3, with a dilation rate of 2, expanding the receptive field to capture context), and deformable convolutions (3×3, adaptive sampling position to fit irregular defect shapes). Global average pooling is applied to the input features to obtain a global descriptor, which is then passed through two fully connected layers (with ReLU activation in the intermediate layer) to output the fusion weights of each branch, ensuring the sum of all weights is 1. The convolutional outputs of each branch are weighted and summed to obtain the multi-receptive-field fused features. This mechanism enables the model to adaptively select the most suitable convolutional combination based on the defect morphology (point-like, line-like, area-like).

[0029] (3b) Cross-scale collaborative attention module: The input features are decomposed into multiple sub-features of different scales through grouped convolution (achieved through different downsampling rates), the correlation matrix between scales is calculated, and after Softmax normalization, each sub-feature is weighted and fused, and then connected with the residual of the original feature to achieve adaptive fusion of fine-grained details and coarse-grained semantics. The outputs of the dynamic convolution fusion module and the cross-scale collaborative attention module are concatenated along the channel dimension to form a refined feature vector.

[0030] The context-aware dynamic convolutional fusion module and the cross-scale collaborative attention module work synergistically: the former, by integrating depthwise separable, dilated, and deformable convolutions in parallel, adaptively generates fusion weights based on the global context, solving the problem that a single convolution cannot simultaneously adapt to multiple morphological defects such as point, line, and area, and outputs "multi-receptive field fusion features"; the latter decomposes the aforementioned features into sub-features of different scales through grouped convolutions, dynamically calculates the correlation weights between scales, and achieves adaptive calibration of fine-grained texture details and coarse-grained semantic information, outputting "refined features". This synergy produces the following technical effects: the dynamic convolutional fusion module provides a richer single-scale multi-receptive field representation foundation for cross-scale attention, while cross-scale attention compensates for the shortcomings of dynamic convolution, which only focuses on fusion within a single scale and ignores inter-scale dependencies. Together, they solve the problems of "irregular defect edge feature breakage" and "semantic conflict caused by simple splicing of multi-scale features" caused by the fixed window division of Swin Transformer. This collaboration improves the model's accuracy in separating irregular defects such as oil stains and wrinkles, and avoids the subjectivity of manually setting weights required by traditional multi-scale feature fusion. It provides input features with clearer structure and stronger discriminative power for the subsequent graph embedding constraint module, ultimately achieving a leap in technical performance across the entire "feature extraction-refinement-classification" chain. Such functional complementarity and effect superposition between modules cannot be foreseen by those skilled in the art based on a simple combination of existing technologies.

[0031] (4) Integrating deep random vector functionality to connect network classifiers and graph embedding constraints: The refined features are integrated into a deep stochastic vector function network. This classifier comprises an input layer, multiple levels of stochastic feature enhancement layers (three levels in this invention), and a fusion layer. The input layer receives the refined features as the shared input to all enhancement layers. Each enhancement layer performs a nonlinear transformation using a randomly initialized and fixed (He normal distribution) weight matrix and a ReLU activation function to generate nonlinear enhanced features. The enhanced features from the previous level are concatenated with the shared input features and used as the input to the next enhancement layer.

[0032] This paper innovatively introduces boundary-aware graph embedding constraints based on local Fisher discriminant analysis into the optimization objective. The specific steps are as follows: Constructing an intrinsic graph (samples of the same class): For samples belonging to the same class, the connection weight is defined as the similarity between samples divided by the total number of samples in that class.

[0033] Construct a penalty graph (outlier samples): For the connection weights between samples, different calculation methods are used depending on whether the samples are of the same class, so that outlier samples are forcibly separated near the decision boundary.

[0034] Based on the aforementioned weight matrices, the degree matrices of the intrinsic graph and the penalty graph are calculated respectively, yielding the graph Laplacian matrix, and subsequently the structural constraint matrix. This structural constraint matrix is ​​then introduced as a regularization term into the loss function of the integrated deep stochastic vector functional connection network to enforce intra-class compactness and inter-class separation.

[0035] Simultaneously, a dynamic adaptive regularization strategy is adopted: the regularization coefficient decreases exponentially with network depth (strong constraints in shallow layers, weak constraints in deep layers), and is adjusted by feedback from the residuals of previous layers (the regularization strength is automatically increased when the residuals increase). Each enhancement layer independently solves for the output weight parameters to generate preliminary classification probabilities.

[0036] The dynamic multi-scale refining module in step (3) and the graph embedding constraint integration deep RVFL module in step (4) produce a deep synergistic effect of "feature purification - explicit structural constraint": Step (3) not only solves the problem of the fixed window of the Swin Transformer cutting off the irregular defect edges through dynamic convolution fusion and cross-scale attention, but also outputs a set of high-quality refined features with "multi-receptive field adaptation and scale dependency modeling". This set of features is directly used as the input of step (4), which significantly reduces the sensitivity of the subsequent classifier to noisy features and redundant scales; while the graph embedding constraint based on local Fisher discriminant analysis in step (4) does not directly act on the original features or shallow features as in traditional methods, but specifically performs geometric manifold modeling on the "refined features" output by step (3) - by constructing decision boundary sensitive intrinsic graphs and penalty graphs, it explicitly forces intra-class compactness and inter-class separation, making up for the natural defect of dynamic convolution and attention mechanisms that only focus on the "representation ability" of features and ignore the "topological structure" of the feature space. This synergy produces non-obvious technical effects: the refined features in step (3) provide a purer optimization object for graph embedding constraints, enabling graph embedding to more accurately capture the small differences in the manifold space of similar morphological defects (such as stains and protrusions). Experiments have shown that this synergy reduces the misclassification rate of such difficult-to-distinguish samples by 12.7%. At the same time, the structured constraints of graph embedding suppress the risk of overfitting that may be introduced during dynamic convolution fusion, improving the F1 score of the model by more than 6% in small sample scenarios. Ultimately, it breaks through the traditional architectural bottleneck of "independent optimization of feature extraction and classifier" and achieves the joint optimality of feature representation and decision boundary. This cross-module functional complementarity and effect multiplication cannot be foreseen by those skilled in the art through conventional logical deduction or simple superposition.

[0037] (5) Weighted fusion output: The classification probabilities of each enhancement layer output are assigned learnable fusion weights, which are then normalized by Softmax and weighted summed. The category with the highest probability is taken as the final classification result.

[0038] Secondly, this invention proposes a fabric defect classification system that integrates multi-scale dynamic convolution and graph embedding constraints to implement the above method. The system includes: a fabric image preprocessing module, a Swin Transformer feature extraction module, a dynamic multi-scale refining module (including a context-aware dynamic convolution fusion module and a cross-scale collaborative attention module), an integrated deep random vector functional connection network classifier module (including a graph embedding constraint unit), and a classification and recognition module.

[0039] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are implemented based on the technical solution of the present invention, but the scope of protection of the present invention is not limited to the following embodiments.

[0040] Example 1: Overall Model Architecture and Implementation Figure 1 This is an overall architecture diagram of the method of the present invention, showing the complete process from the input image to the Swing Transformer, the context-aware dynamic convolutional fusion module, the cross-scale collaborative attention module, the integrated deep random vector functional connection network (including graph embedding) and finally to the classification output.

[0041] like Figure 1 As shown, the present invention proposes a fabric defect classification method that integrates multi-scale dynamic convolution and graph embedding constraints. Figure 2 This is an example of a feature activation heatmap of fabric defects based on the model of this invention. The specific implementation steps of this method are as follows: Step 1: Image Preprocessing The acquired fabric images may have issues such as inconsistent sizes and uneven lighting. This embodiment uses bilinear interpolation to scale all images to 224×224 pixels, maintaining the original aspect ratio, and padding blank areas with 0. Then, the mean of each of the RGB channels is subtracted and divided by the standard deviation to achieve normalization. Data augmentation employs a dynamic strategy: during each training iteration, augmentation is performed with a probability of 0.5. If augmentation is performed, 1 to 5 operations are randomly selected, including random horizontal flipping, random vertical flipping, random rotation (-30° to 30°), random scaling (0.8 to 1.2 times), brightness adjustment (0.8 to 1.2), contrast adjustment (0.8 to 1.2), adding Gaussian noise (standard deviation 0.01), and random blending augmentation (randomly selecting another defect image from the same batch and fusing the texture with a weight of 0.3 to 0.7). The augmented image is then mixed with the original image at a 7:3 ratio and fed into the network.

[0042] Step 2: Swing Transformer Feature Extraction This embodiment uses the Swin-Tiny network as the backbone, with the following parameters: window size 7×7, and the number of multi-head attention modules 3, 6, 12, and 24. The input image first passes through an image patching layer, resulting in a 56×56×96 feature map. In the first stage, the image passes through a regular window self-attention block, followed by a moving window self-attention block. Each block is followed by layer normalization and a multilayer perceptron, with the output size remaining unchanged. Before the second stage, the resolution is reduced to 28×28 and the number of channels is increased to 192 through image patch merging. Subsequently, the two self-attention methods are executed alternately. The third stage outputs 14×14×384, and the fourth stage outputs 7×7×768. All self-attention calculations are performed within a local window, with the moving window offset being (3,3). Finally, the output feature map of the fourth stage is used as a multi-scale feature vector.

[0043] Step 3: Dynamic Multi-Scale Feature Refinement The 7×7×768 feature map output from the fourth stage is input into the context-aware dynamic convolutional fusion module. Three convolutional branches run in parallel: depthwise separable convolution (3×3), dilated convolution (3×3, dilation rate 2), and deformable convolution (3×3). Global average pooling is performed on the input features (768 dimensions) to obtain a 768-dimensional vector. Then, two fully connected layers (256 dimensions, ReLU in the first layer; 3 dimensions, Softmax in the second layer) generate the fusion weights for the three branches. Weighted summation yields the multi-receptive-field fused feature, still with a size of 7×7×768.

[0044] The feature is then input into the cross-scale collaborative attention module: the channel is divided into 4 groups using grouped convolution, and depthwise separable convolutions of 1×1, 3×3, 5×5, and 7×7 are performed respectively (with a stride of 2 for downsampling) to obtain sub-features at different scales. These sub-features are then upsampled to the original size, and the inter-scale correlation matrix (4×4) is calculated. After weighted fusion, the matrix is ​​concatenated with the original input residual. The output size remains 7×7×768. Finally, the feature vector is flattened.

[0045] Step 4: Integrate deep random vector functionality to connect the network classifier and graph embedding constraints. A three-level random feature enhancement layer is set up, with 500 nodes in each layer. The input feature dimension is 768, so the first layer's weight matrix size is 768×500, the second layer's size is (768+500)×500, and the third layer's size is (768+500+500)×500. All weights are initialized according to a He normal distribution (mean 0, standard deviation √(2 / input dimension)) and fixed during training. The output of each layer is processed through a ReLU activation function to obtain non-linear enhanced features.

[0046] Construction of graph embedding constraints: Select all samples in the training set and calculate Gaussian similarity (bandwidth parameter is the median distance between all sample pairs). The number of classes is 5. Construct the graph matrix based on the local Fisher discriminant analysis criterion: For samples of the same class, construct an intrinsic graph, whose weight is defined as the similarity between samples divided by the total number of samples in that class; for samples of different classes, construct a penalty graph, whose weight is calculated differently depending on whether the samples are of the same class; based on the above weights, obtain the weight matrices of the corresponding intrinsic graph and penalty graph, respectively; based on the weight matrices, calculate the degree matrices of the intrinsic graph and penalty graph, respectively, to obtain the graph Laplacian matrix. The graph constraint coefficient is set to 0.1. Dynamic regularization parameters: initial regularization coefficient 0.01, depth decay factor 0.5, residual feedback coefficient 0.1. For each level, first solve for the output weights, and then calculate the prediction probability of that level. The probabilities of the outputs at each level are weighted and fused through learnable weights (initially equal), and the classification result is obtained by taking the maximum probability. During training, cross-entropy loss was used for backpropagation of the entire network (including the fine-tuned parameters of the Swin backbone, the learnable parameters in the dynamic convolutional fusion module and the cross-scale collaborative attention module, as well as the fusion weights), but the random weights of the network connected by the integrated deep stochastic vector function remained fixed. The optimizer used was AdamW, with an initial learning rate of 1e-4, a batch size of 32, and 50 training epochs.

[0047] Step 5: Experimental Verification This experiment used three open-source fabric datasets A, B, and C from the Kaggle platform, covering five types of defects: no defects, holes, stains, bumps, and linear breaks, totaling 23,716 images. After enhancement, the number of images increased to 30,831. Experimental environment: Intel i7-14700HX, 16GB RAM, NVIDIA RTX4070, PyTorch 2.6.0.

[0048] Heatmap Result Analysis: To visually demonstrate the feature localization capability of the model of this invention for fabric defects, a feature activation heatmap was generated using Class Activation Mapping (CAM). Figure 2 As can be seen from the original image, the fabric background has a highly periodic interwoven grid pattern. This strong texture characteristic brings significant background interference, posing a great challenge to the separation and extraction of micro-defects. Observing the highly activated areas (red and yellow areas) in the heatmap, typical defect types in the dataset can be clearly identified: including local dot-like "holes" and "protruding foreign objects" (such as...). Figure 2 Central discrete highlight area), "linear breaks" distributed along latitude and longitude (such as...) Figure 2The model exhibits characteristics such as edge-extended highlighted areas and highly integrated "stains" with the background. Furthermore, no abnormal activation was observed in defect-free, normal fabric. The heatmap results clearly demonstrate that even when faced with microscopic defects highly integrated with complex textures, the model can penetrate background noise to accurately locate abnormal areas without generating false alarms, fully proving its excellent feature localization capability for irregular defects against complex backgrounds.

[0049] Generalization capability verification: Training on dataset B (64×64 grayscale image) with 70% accuracy, the accuracy on the B test set was 89.55%, with a mean average precision (mAP) of 98.81%. In cross-dataset testing (B→C, 64×64 color image), the accuracy was 86.21%, significantly higher than A→C's 75.92%, verifying the robustness of graph embedding constraints to distribution shifts. Specific results are shown in Table 1.

[0050] Table 1. Validation results of model generalization ability

[0051] Ablation experiments: To verify the effectiveness of each module in this invention, the cross-scale collaborative attention module, the multi-level ensemble deep network was degenerated into a single hidden layer RVFL, the context-aware dynamic convolutional fusion module was removed, graph embedding constraints were removed, dynamic adaptive regularization was removed, and the hybrid attention mechanism was removed, and five training tests were conducted. The results show that the baseline model (including all modules) performs best in all metrics. The performance degradation is most significant when graph embedding constraints are removed or the model is degenerated into a single hidden layer, verifying the core role of graph embedding in explicitly modeling feature manifolds and the core role of multi-level cascade mechanisms in enhancing high-dimensional nonlinear representations. At the same time, removing dynamic regularization significantly reduces model stability, confirming its key value in suppressing overfitting. Specific results are shown in Table 2.

[0052] Table 2 Ablation Experiment Results

[0053] Comparative experiments: Eight mainstream methods were selected for comparison, including ResNet50, VGG16, ViT (vit_base_patch16_224), SwinTransformer (swin_base_patch4_window7_224), MobileNetV3-Large, EfficientNet-B0, Random Forest, and Support Vector Machine. Figure 3A and Figure 3B The confusion matrix of the model of this invention and the comparative model is shown in Figure 4A and 4B. Figure 4B The ROC curves are shown. The curves of the model of this invention are all close to the upper left in all categories, with a high AUC value, and it outperforms traditional methods in difficult categories such as "holes" and "linear breaks".

[0054] Computational complexity: The model of this invention has 28.4M parameters, 3.8G floating-point operations, an inference latency of 18.2ms / image, and a training convergence time of 1-2 hours. It has significant advantages compared with VGG16 (138.4M parameters, 15.5G floating-point operations) and ViT (86.4M parameters, 17.6G floating-point operations).

[0055] Comparative Example: Differences from Existing PVT-edRVFL Patents The existing patent "A Fabric Defect Classification Method and System Based on Deep Learning PVT-edRVFL" uses Pyramid Visual Transformer as the feature extractor, integrates multi-scale features through a bidirectional feature fusion path, and uses dynamic adaptive regularization in edRVFL, but does not introduce dynamic convolutional fusion, cross-scale collaborative attention, and graph embedding constraints. The essential differences between this invention and the previous one are: (1) Different feature extraction backbone: This invention adopts the moving window attention mechanism of Swing Transformer, which has linear computational complexity and stronger local-global feature collaboration capability; (2) Different multi-scale refinement mechanism: This invention adds a context-aware dynamic convolutional fusion module and a cross-scale collaborative attention module, which can adaptively fuse multi-receptive field convolutional features and model inter-scale dependencies; (3) Different classifier constraints: This invention innovatively adds boundary-aware graph embedding constraints based on local Fisher discriminant analysis to the integrated deep random vector functional connection network, and explicitly models the geometric manifold of the samples. These differences make this invention have significant performance advantages in small sample, morphologically similar defect, and cross-dataset scenarios.

[0056] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method of fabric defect classification fusing multi-scale dynamic convolution with graph embedding constraints, characterized in that, Includes the following steps: (1) Preprocess the fabric image; (2) Extracting multi-scale features using Swing Transformer: The preprocessed image is input into the Swing Transformer network, the preprocessed image is divided into 4x4 non-overlapping blocks and linearly embedded, and multi-scale features are extracted through four progressive stages. In each stage, multi-head self-attention based on 7x7 regular window and multi-head self-attention based on 7x7 moving window are alternately executed to capture micro-texture distortion in local windows and to identify cross-regional structural anomalies by using information interaction between windows. After each stage, the feature map resolution is halved and the number of channels is doubled through block merging operation, and multi-scale features are output. (3) Dynamic multi-scale refinement of the multi-scale features extracted in step (2): The multi-scale features are processed using a context-aware dynamic convolutional fusion module. The dynamic convolutional fusion module is a 3-branch parallel structure that integrates depthwise separable convolution, dilated convolution, and deformable convolution in parallel. In each branch, the multi-scale features are global average pooled to obtain a global descriptor. The adaptive fusion weights of each branch are generated through a two-layer fully connected network, and the outputs of each branch are weighted and summed according to these weights to generate multi-receptive field fusion features. The multi-receptive field fusion features are then processed using a cross-scale collaborative attention module. Sub-features of different scales are obtained by group convolution decomposition, and the correlation between scales is dynamically modeled to obtain the correlation matrix between scales. After Softmax normalization, the sub-features are weighted and fused, and then residual connections are made with the multi-receptive field fusion features to achieve adaptive fusion of fine-grained details and coarse-grained semantics. The outputs of the dynamic convolutional fusion module and the cross-scale collaborative attention module are concatenated along the channel dimension to form the refined features. (4) The refined features are integrated into a deep random vector functional connection network. The integrated deep random vector functional connection network contains multiple levels of random feature enhancement layers. Each enhancement layer independently solves the output probability. A boundary-aware graph embedding constraint based on local Fisher discriminant analysis is introduced into the optimization objective. This constraint is calculated by constructing the intrinsic graph of the same type of sample and the penalty graph of the opposite type of sample, respectively, and the graph Laplacian matrix is ​​calculated and added to the loss function in the form of a regularization term. (5) Weigh and fuse the output probabilities of each enhancement layer, and use the category with the highest probability as the classification result.

2. The method of claim 1, wherein, In step (2), the number of output channels in the four progressive stages of the Swin Transformer are 96, 192, 384, and 768 respectively.

3. The method of claim 1, wherein, The preprocessing described in step (1) includes scaling, normalizing, and dynamic data augmentation of the fabric image.

4. The method of claim 1, wherein, In the context-aware dynamic convolutional fusion module in step (3), the intermediate layer of the two fully connected networks uses the ReLU activation function, and the sum of the adaptive fusion weights of each branch is 1.

5. The method of claim 1, wherein, In step (4), the boundary-aware graph embedding constraint is constructed based on the local Fisher discriminant analysis criterion to construct the graph matrix: for samples of the same class, an intrinsic graph is constructed, and its weight is defined as the similarity between samples divided by the total number of samples of that class; for samples of different classes, a penalty graph is constructed, and its weight is calculated differently depending on whether the samples are of the same class; based on the above weights, the corresponding weight matrices of the intrinsic graph and the penalty graph are obtained respectively, and the degree matrices of the intrinsic graph and the penalty graph are calculated based on the weight matrices respectively to obtain the graph Laplacian matrix, and then the structural constraint matrix is ​​obtained; the structural constraint matrix is ​​introduced into the loss function of the integrated deep random vector functional connection network in the form of a regularization term to force intra-class compactness and inter-class separation.

6. The method of claim 1, wherein, The integrated deep random vector functional network comprises an input layer, a multi-level random feature enhancement layer, and a fusion layer. The input layer receives refined features as shared input features for all enhancement layers. The multi-level random feature enhancement layer specifically includes three levels, which are concatenated. Each level performs a nonlinear transformation using a randomly initialized and fixed weight matrix and a ReLU activation function to generate nonlinear enhanced features. The enhanced features of the previous level are concatenated with the shared input features and used as the input for the next level enhancement layer. The fusion layer performs weighted fusion of the output probabilities of all levels.

7. The method of claim 6, wherein, The weight matrix of the multi-level random feature enhancement layer is randomly initialized to conform to the He normal distribution and remains fixed after random initialization; each enhancement layer independently solves the output weight parameters under regularization constraints, and adopts a dynamic adaptive regularization strategy. The regularization coefficient decays exponentially with the depth of the network layer and is adjusted by the feedback of the residual of the previous layer.

8. The method of claim 1, wherein, The preprocessing of the fabric image includes: scaling the input image to a fixed pixel size and performing channel normalization; performing dynamic data enhancement several times, randomly selecting 1 to 5 enhancement methods each time, including geometric transformation, semantic enhancement, and random blending enhancement; geometric transformation includes horizontal flipping, vertical flipping, random rotation, and scaling; semantic enhancement includes adjusting brightness and contrast and adding Gaussian noise; random blending enhancement refers to fusing textures from different defect areas at random proportions.

9. A fabric defect classification system fusing multi-scale dynamic convolution with graph embedding constraints, configured to perform the method of any one of claims 1-8. include: The system includes a fabric image preprocessing module for preprocessing fabric images; a Swing Transformer feature extraction module for extracting hierarchical multi-scale features using Swing Transformer; a dynamic multi-scale refining module, which includes a context-aware dynamic convolutional fusion module and a cross-scale collaborative attention module, for adaptive fusion of multi-scale features and modeling of inter-scale dependencies; and an integrated deep random vector function-connected network classifier module for receiving refined features and identifying defect types through multi-level random feature enhancement layers and boundary-aware map embedding constraints. The classification and recognition module is used to weight and fuse the output probabilities of each enhancement layer, and use the category with the highest probability as the classification result.

10. The system according to claim 9, characterized in that, The integrated deep random vector functional connection network classifier module embeds a graph embedding constraint unit based on local Fisher discriminant analysis, which is used to construct the intrinsic graph and the penalty graph and calculate the graph Laplacian matrix to optimize the output weights of the classifier. The overall model parameter count of the method does not exceed 30M, and the inference latency of a single image does not exceed 20ms.

Images