Fabric defect classification method and system based on deep integrated ELM (Extreme Learning Machine)

Abstract

The invention discloses a fabric defect classification method and system based on deep integrated ELM, and belongs to the technical field of intelligent detection of textile quality. A fabric defect image data set is prepared and preprocessed; the method comprises the following steps: constructing a double-branch deep feature extraction network, wherein a first branch extracts three-level features based on the OfficientNet-B0 and embeds a channel-space attention mechanism to perform feature enhancement; the second branch is used for extracting three levels of features based on Swin Transform; after the features are subjected to multi-scale fusion, an integrated extreme learning machine is adopted for fabric defect classification, the integrated extreme learning machine comprises a plurality of differential base learners, the number of hidden layer nodes of each base learner obeys uniform distribution from 300 to 700, and common logarithms of regularization coefficients obeys uniform distribution from-6 to-2. The method is suitable for efficient and automatic quality detection and grading of fabric surface defects in an industrial production line.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent detection of textile quality, and particularly relates to a fabric defect classification method and system based on deep ensemble ELM. BACKGROUND

[0002] In the production of the textile industry, the accurate classification of fabric surface defects (such as holes, stains, and broken warp) directly determines the product quality grade and the economic benefits of enterprises. At present, the mainstream relies on manual visual inspection, which has the problems of low efficiency (the average accuracy is only 60%), strong subjectivity, and inconsistent standards. In recent years, although intelligent detection technology based on deep learning has been gradually applied, it still faces three major problems:

[0003] Insufficient feature expression: traditional CNN models (such as VGG16 and ResNet50) pay excessive attention to high-level semantic features, ignore subtle texture and shape details, and there is a large amount of redundant noise in high-dimensional features;

[0004] Difficult to capture multi-scale defects: a single model is difficult to balance local defects (such as yarn misplacement) and global distribution features (such as large-area dyeing abnormalities), and existing hybrid architectures (such as MobileViT and CoAtNet) have insufficient feature fusion due to structural limitations;

[0005] Weak model generalization ability: in a small sample scenario, traditional classifiers (such as Softmax and single ELM) are prone to overfitting, and the ensemble method lacks an adaptive weight mechanism, and the generalization performance significantly decreases in a complex fabric texture environment (such as SwinTransformer-ELM with an accuracy of less than 87%).

[0006] Although existing research attempts to combine attention mechanisms (such as SE modules) or feature pyramids (FPN), it still cannot systematically solve the above problems, which restricts the efficiency of industrial scene landing applications. SUMMARY

[0007] In view of low efficiency and strong subjectivity of traditional manual detection, and problems of insufficient feature extraction and insufficient model generalization ability of existing intelligent algorithms, the application provides a fabric defect classification method and system based on deep integrated ELM. A double-branch deep feature extraction network is constructed (combining local detail perception ability of EfficientNet-B0 and global context modeling ability of SwinTransformer), multi-scale feature fusion is performed in combination with a feature pyramid network (FPN), and a channel-spatial attention mechanism (CBAM) is embedded to enhance key feature expression; in the classification stage, a diversity-enhanced integrated extreme learning machine (ELM) strategy is adopted, difference-based learning is performed by randomizing hidden layer parameters, and adaptive weighted decision fusion is performed based on performance of a verification set. The method significantly improves the precision, robustness and generalization ability of fabric defect classification, and is suitable for efficient and automatic quality detection and grading of fabric surface defects (such as holes, dyeing, oil stains and the like) on an industrial production line.

[0008] The technical solutions adopted by the application are as follows:

[0009] In a first aspect, the application provides a fabric defect classification method based on deep integrated ELM, comprising the following steps:

[0010] (1) preparing a fabric defect image dataset and performing preprocessing;

[0011] (2) constructing a double-branch deep feature extraction network, the first branch extracts three levels of features based on EfficientNet-B0, and obtains first feature E, second feature E and third feature E, the channel number of the three levels of features increases layer by layer, and the spatial dimension decreases layer by layer; the channel-spatial attention mechanism is embedded to enhance the features of the three levels of features, and obtains first enhanced feature E, second enhanced feature E and third enhanced feature E; the second branch extracts three levels of features based on Swin Transformer, including first feature S, second feature S and third feature S, the channel number of the three levels of features increases layer by layer, and the spatial dimension decreases layer by layer;

[0012] (3) converting the first enhanced feature E, the second enhanced feature E, the third enhanced feature E, the first feature S, the second feature S and the third feature S obtained in step (2) to feature maps with the same channel and unchanged spatial dimension through a dimension conversion network, performing multi-scale fusion on the dimension-converted feature maps by using a feature pyramid network, and generating a final feature vector;

[0013] (4) the final feature vector is classified by an ensemble extreme learning machine, the ensemble extreme learning machine comprises a plurality of differentiated base learners, each base learner randomizes hidden layer parameters, including a number of hidden layer nodes and a regularization coefficient, wherein the number of hidden layer nodes is uniformly distributed from 300 to 700, and the common logarithm of the regularization coefficient is uniformly distributed from-6 to-2;

[0014] (5) training a fabric defect classification model composed of a double-branch deep feature extraction network, a dimension conversion network, a feature pyramid network and an ensemble extreme learning machine, using the trained fabric defect classification model to realize fabric defect classification and output fabric defect types.

[0015] Further, the spatial dimensions of the first feature E, the second feature E and the third feature E correspond one-to-one to the spatial dimensions of the first feature S, the second feature S and the third feature S respectively.

[0016] Further, the first feature E, the second feature E and the third feature E are respectively generated by blocks[1], blocks[3] and blocks[5] of the EfficientNet-B0 model, and the dimensions are 56x56x32, 28x28x56 and 14x14x160 respectively.

[0017] Further, in the first branch, the process of feature enhancement by embedding channel-spatial attention mechanism includes:

[0018] The first feature E, the second feature E and the third feature E are respectively input into the channel-spatial attention module, first, the channel attention module generates channel descriptors through global average pooling and maximum pooling, and generates channel attention weights through multi-layer perception and Sigmoid activation, and then the channel attention weights are multiplied with the input features element by element to obtain the feature map enhanced by channel attention; then the spatial attention module generates spatial descriptors through average pooling and maximum pooling along the channel dimension on the feature map enhanced by channel attention, and generates spatial attention weights through 7x7 convolution and Sigmoid activation, and then the spatial attention weights are multiplied with the feature map enhanced by channel attention element by element to obtain the final enhanced feature map.

[0019] Further, in the second branch, the first feature S, the second feature S and the third feature S are respectively generated by Stage 0, Stage 1 and Stage 2 of the SwinTransformer model, and the dimensions are 56x56x96, 28x28x192 and 14x14x384 respectively.

[0020] Further, step (3) comprises:

[0021] (3.1) unify the channel dimension to 128 dimensions through 1x1 convolution;

[0022] (3.2) The same spatial dimension and channel dimension features are adaptively fused by using Softmax weight distribution to generate three fused features F fused,1 , F fused,2 , and F fused,3 , and the adaptive fusion formula is as follows:

[0023] F fused,i = softmax(w)1·F′ e,i + softmax(w)2·F′ s,i

[0024] wherein w is a learnable parameter, w∈r 2 , and is converted into two weight values softmax(w)1 and softmax(w)2 between 0 and 1 and summing to 1 through the Softmax operation; F′ e,i is the i-th enhanced feature E, F′ s,i is the i-th feature S, F′ e,i and F′ s,i have the same channel dimension and spatial dimension, F fused,i is the i-th fused feature, i = 1, 2, 3;

[0025] (3.3) Feature upsampling and pyramid construction are performed through the top-down path to generate a three-level feature pyramid {P1, P2, P3}, and a final feature vector V = [GAP(P1); GAP(P2); GAP(P3)] is generated based on the three-level feature pyramid, wherein GAP(.) represents global average pooling, and [. ;. ;.] represents splicing.

[0026] Further, the three-level feature pyramid construction formula is as follows:

[0027] P3 = F fused,3

[0028] P2 = Upsample(P3) + F fused,2

[0029] P1 = Upsample(P2) + F fused,1

[0030] wherein P3, P2, and P1 represent the three-layer feature pyramid formed by the top-down path, and Upsample(.) represents upsampling.

[0031] Further, in the training stage, the ensemble extreme learning machine adaptively weights the decision fusion based on the validation set accuracy, and the weighted voting formula is as follows:

[0032]

[0033] wherein, Acc k is the accuracy of the k-th base learner on the validation set, Acc i is the accuracy of the i-th base learner on the validation set, M represents the number of base learners, and η is a temperature coefficient, w k is the weight of the k-th base learner, p k (j|x) represents the probability of the k-th base learner predicting that the final feature vector x belongs to the class j, is the prediction result.

[0034] Further, the fabric defect types include holes, dyeing, oil stains, thread end errors, and foreign matter.

[0035] In a second aspect, the application provides a fabric defect classification system based on deep ensemble ELM, which is used to realize the fabric defect classification method based on deep ensemble ELM.

[0036] The application has the following beneficial effects:

[0037] The application fuses the local detail perception ability of EfficientNet-B0 and the global context modeling ability of Swin Transformer to construct a double-branch feature extraction network, realizes adaptive fusion of multi-scale defect features in combination with a feature pyramid (FPN) and a channel-spatial attention mechanism (CBAM), and innovatively adopts an ensemble extreme learning machine (Ensemble ELM) classifier with randomized hidden layer parameters and regularization coefficients, thereby significantly improving the fabric defect classification accuracy under the premise of ensuring the real-time performance of the model (4-core CPU / 8-GB memory platform), effectively solving the problems of insufficient feature coverage of traditional single models, weak small sample generalization ability, and industrial scene noise interference, providing a high-precision, strong-robustness end-to-end solution for textile automatic quality inspection, and having extremely crucial significance for guaranteeing product quality and improving enterprise economic benefits. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 is an integrated ELM classification model architecture based on deep feature fusion;

[0039] Figure 2 is an accuracy image of different attention mechanisms;

[0040] Figure 3 is a performance comparison of different feature fusion methods;

[0041] Figure 4 is a stability analysis diagram of deep feature fusion on the BaseELM classifier;

[0042] Figure 5is a stability analysis diagram of deep feature fusion on EnsembleELM classifier. DETAILED DESCRIPTION

[0043] The present application will be further described and illustrated in connection with the specific embodiments. The embodiments are only exemplary and do not circumscribe the scope of the present disclosure. The technical features of various embodiments of the present application can be combined in a corresponding manner without conflict, provided that there is no conflict.

[0044] The accompanying drawings are merely schematic and are not necessarily drawn to scale. Some of the blocks in the block diagrams are functional entities that do not necessarily have to correspond to physically or logically independent entities. These functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0045] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all steps. For example, some steps can be further decomposed, and some steps can be combined or partially combined, so the actual execution order can be changed according to the actual situation.

[0046] As shown in Figure 1 The present application proposes a fabric defect classification method based on deep ensemble ELM. A double-branch deep feature extraction network is constructed to fuse the local detail perception ability of EfficientNet-B0 and the global context modeling ability of SwinTransformer, combined with a feature pyramid network (FPN) for multi-scale feature fusion, and embedded with a channel-spatial attention mechanism (CBAM) to enhance key feature expression. In the classification stage, a diversity-enhanced ensemble extreme learning machine (ELM) strategy is adopted to construct differentiated base learners by randomizing the hidden layer parameters, and to perform adaptive weighted decision fusion based on the performance of the validation set. It mainly includes the following steps:

[0047] Step 1: Prepare the data set and perform preprocessing

[0048] (1.1) Select the data set

[0049] The FabricDefect Dataset dataset is used in this study. This dataset comes from FabricDefect Dataset from Kaggle, Fabric Stain Dataset from Kaggle, Aitex Fabric Image Database, contains multiple defect types, balanced sample size for each type, and a total of 2739 samples; there is also a fabric defect dataset, a total of 23,716. Fabric defect types include but are not limited to holes, dyeing, oil stains, thread end errors, and foreign matter.

[0050] (1.2) Dataset preprocessing

[0051] Adjust the fabric images in the dataset to the input size of the model: 224*224*3, and divide them into training set, validation set and test set according to the ratio of 7:1.5:1.5.

[0052] Step 2: Fabric defect classification model (ESF-edELM) parameter analysis and setting

[0053] In this embodiment, based on the Python3.12 platform, the hardware configuration is 4-core CPU / 8GB memory of conventional computing resources; through the fusion of double-branch feature extraction (EfficientNet-B0+Swin Transformer) and multi-scale feature pyramid (FPN) to construct 128-dimensional unified features, and train 5 randomized ELM base learners (hidden layer nodes 300-700, regularization coefficients 10 -6 ~10 -2 ), finally realize the classification decision with adaptive weighted voting (η = 2.0).

[0054] Step 3: Image feature extraction

[0055] (3.1) Hierarchical feature extraction based on EfficientNet-B0

[0056] EfficientNet-B0 uses a compound scaling strategy, which scales the network depth, i.e. the number of layers, the width, i.e. the number of channels, and the input image resolution in a coordinated manner. Depth determines the model's abstraction level of features, width affects the model's ability to capture different features, and resolution is related to the level of detail in the image. By adjusting these parameters reasonably, EfficientNet-B0 reduces the complexity of the model and improves the computational efficiency while effectively extracting fabric defect features.

[0057] To mine the rich feature information contained in the key layers inside EfficientNet-B0, the hook mechanism commonly used in the field of deep learning is used. This mechanism can obtain the input, output or gradient information of a specific layer during the forward propagation or backward propagation of the model, so as to accurately capture the EfficientNet B0 level features meaningful for fabric defect classification. The following three feature layers are selected:

[0058] 1. blocks[1]: located in the shallow layer of the network, with an output dimension of 56x56x32. This setting can preserve the fabric texture details. The receptive field of this layer is small, but it can capture subtle texture changes on the fabric surface, such as yarn texture and weaving lines. These seemingly insignificant texture details are actually key clues for identifying small defects and can provide crucial evidence for subsequent defect classification.

[0059] 2. blocks[3]: located in the middle layer of the network, with an output dimension of 28x28x56. It can capture local defect patterns. As the network layers deepen, the receptive field gradually increases, just like extending the field of view from the microscopic world under the microscope to a small range of macroscopic scenes. This allows the integration of information within a certain area and is good at capturing local defect patterns, such as specific patterns formed by small areas of stains, local yarn disorder or slight color differences on the fabric. This can provide strong support for accurate defect type judgment.

[0060] 3. blocks[5]: located in the deep part of the network, with an output dimension of 14x14x160. It is responsible for extracting high-level semantic features. Deep networks have advantages, such as larger receptive fields and stronger feature abstraction capabilities. Just like looking down from a high place, it can grasp the features of the fabric image from a global perspective. It can identify the category of defects, accurately judge whether it is a hole, stain, broken warp or other types of defects, and also assess the severity of the defect. These high-level semantic features play a decisive role in the final classification decision, helping the model make accurate and reasonable judgments.

[0061] (3.2) CBAM attention mechanism

[0062] In the fabric defect classification model, the CBAM attention mechanism plays a crucial role and is applied after each level of feature extraction of EfficientNet-B0, aiming to suppress noise, highlight defect areas, and enhance the expression ability of key features, thereby improving the recognition accuracy of fabric defects by the model. The CBAM attention mechanism consists of a channel attention module (CAM) and a spatial attention module (SAM), which performs attention calculation on the feature map in the channel and spatial dimensions to achieve adaptive re-labeling of the features.

[0063] Channel attention module calculation process: The channel attention module is applied after the feature layers output by blocks[1], blocks[3], and blocks[5] in EfficientNet-B0. The calculation process is as follows:

[0064] 1. Perform global average pooling and global maximum pooling operations on the input feature map F to obtain two different channel descriptors. Global average pooling compresses the spatial dimension to 1, obtaining AvgPool(F) ∈ R B×C×1×1 , which reflects the average feature response of each channel in the entire spatial region; global maximum pooling obtains MaxPool(F) ∈ R B×C×1×1 , which highlights the strongest response features of each channel in space.

[0065] 2. Input AvgPool(F) and MaxPool(F) into a multi-layer perceptron (MLP). MLP contains two fully connected layers with ReLU activation function in between. Assuming the input of MLP is x, the weight matrix of the first fully connected layer is W1, the bias is b1, the weight matrix of the second fully connected layer is W2, and the bias is b2.

[0066] 3. Add MLP(AvgPool(F)) and MLP(MaxPool(F)), and then map the result to 0-1 through the Sigmoid activation function (σ) to obtain the channel attention weight Wc.

[0067] 4. Multiply the channel attention weight Wc with the original feature map F element by element to obtain the feature map F c-att enhanced by channel attention.

[0068] Spatial attention module calculation process: The spatial attention module is applied to the feature map F c-att processed by the channel attention module. The calculation process of spatial attention is as follows:

[0069] 1. For F c-attThe average pooling and the max pooling operations along the channel dimension are performed respectively to obtain two spatial descriptors. The average pooling obtains AvgPoolspatial(F c-att )∈R B×1×H×W , which reflects the average feature intensity of each spatial position on all channels; the max pooling obtains MaxPoolspatial(F c-att )∈R B×1×H×W , which retains the maximum value of each spatial position in all channels.

[0070] 2. The two spatial descriptors are spliced in the channel dimension.

[0071] 3. A 7x7 convolutional layer f 7×7 is used to perform convolution operation on the spliced feature map, and then a Sigmoid activation function is used to obtain the spatial attention weight Ws.

[0072] 4. The spatial attention weight Ws is multiplied element by element with the feature map F c-att enhanced by the channel attention to obtain the final feature map F final processed by CBAM.

[0073] Through the sequential action of the above channel attention and spatial attention, CBAM can automatically focus on the key areas and important channel information that have important influence on fabric defect classification, effectively suppress noise and irrelevant information, and significantly improve the expression ability of the features, providing more discriminative features for subsequent classification tasks.

[0074] (3.3) Hierarchical feature extraction based on Swin Transformer

[0075] Swin Transformer uses hierarchical window division for feature extraction. This unique architecture can capture information at different scales when processing images, making it particularly suitable for extracting global and local features in fabric defect classification tasks. In terms of hierarchical feature selection, the invention selects the outputs of Stage 0-2 of swin-tiny:

[0076] 1. Stage 0 (96 channels): 7x7 window captures local texture abnormalities, at this time the feature map is divided into 56x56x96. The smaller window size can focus on the fine texture structure of the fabric. In this stage, the model can find the tiny differences between yarns, local texture deformation, etc., which are important for early detection of potential defects.

[0077] 2. Stage 1 (192 channels): The receptive field is expanded to 14x14, and the feature map is divided into 28x28x192 at this time. This change allows the model to capture information from a wider area, enabling it to recognize defects that may not be apparent in a small local area but exhibit specific patterns at a larger scale.

[0078] 3. Stage 2 (384 channels): The shift window enables cross-region association, and the feature map is divided into 14x14x384 at this time. The shift window mechanism breaks the limitations of traditional fixed windows, allowing the model to pass information between different windows and enhancing its understanding of global context. For larger defects such as large areas of dyeing or damage, correlation analysis can be performed with the surrounding normal areas to accurately grasp the boundaries and overall characteristics of the defects, thereby enabling more accurate classification. In terms of dimension adaptation, the sequence output X ∈ R B×N×C permute(0,3,1,2) is used to convert it to a convolutional format X' ∈ R B×C×H×W , so that the features of SwinTransformer can be fused with the features of EfficientNet-B0 in the same feature space. This operation can be represented as: X' = permute(X, (0,3,1,2)). This conversion lays the foundation for subsequent FPN feature fusion, ensuring that features extracted by different models can work effectively together to improve the accuracy of fabric defect classification.

[0079] (3.4) FPN feature fusion

[0080] After completing the hierarchical feature extraction of EfficientNet-B0 and Swin Transformer, a key problem arises: how to effectively integrate these features from different models and different scales to fully leverage their advantages and improve the performance of fabric defect classification. These features differ greatly in terms of dimension, resolution, and contribution to classification. If these unprocessed features are directly used for classification, it will lead to information confusion, making it difficult for the model to effectively learn and distinguish defect features. On the one hand, the differences in features during the calculation process increase the difficulty and complexity of model training. On the other hand, without proper fusion of features at different scales, the model cannot fully learn the information from multiple aspects, affecting the accuracy of classification. Feature Pyramid Networks (FPN) is an effective solution to this problem. FPN has the ability to unify feature dimensions, adaptively fuse different features, and construct a feature pyramid, allowing EfficientNet-B0 and Swin Transformer to extract features and integrate them organically. This enables the model to learn and use these features, improving the accuracy and stability of fabric defect classification. The selection of FPN for feature fusion is a key step in improving the performance of the model, and the specific process is as follows:

[0081] 1. Feature dimension unification: The features of each layer of the two models are uniformly mapped to 128 dimensions by means of 1x1 convolution, and the width and height are kept unchanged, so that the features of different models and different levels are consistent in dimension, facilitating subsequent fusion operations.

[0082] 2. Adaptive fusion: After unifying the dimensions, adaptive fusion is performed to fuse the two features of the same dimension together, and the learnable parameter w∈R 2 balances the contribution of the features of the two models. The formula is:

[0083] F fused,i == softmax(w)1·F′ e,i + softmax(w)2·F′ s,i

[0084] where softmax performs a Softmax operation on the parameter w to convert it into two weight values between 0 and 1 and summing to 1, which are denoted as softmax(w)1 and softmax(w)2. F fused,i represents the i-th feature after fusion, F′ e,i represents the i-th enhanced feature E after feature dimension unification, F′ s,i represents the i-th feature S after feature dimension unification, F′ e,i and F′ s,i have the same dimension. In this way, according to the importance of different level features, the weights are adaptively allocated to achieve effective fusion of the features of EfficientNet-B0 and Swin Transformer.

[0085] 3. Feature upsampling and pyramid construction: The fused features are upsampled through bilinear interpolation in a top-down path. P3, as the top layer, does not need to be upsampled, P3=F fused,3 ; P2 is composed of the upsampled feature of P3 and F fused,2 , and the formula is: P2=Upsample(P3)+F fused,2 , where P3 is the deep fusion feature, which is upsampled by bilinear interpolation to increase its resolution, and then added to the intermediate layer fusion feature F fused,2 . Similarly, P1=Upsample(P2)+F fused,1 The final output generates a three-level feature pyramid {P1, P2, P3} with dimensions of 128x56x56, 128x28x28, and 128x14x14, respectively.

[0086] 4. Final feature vector generation: global average pooling is performed on the features at each level of the pyramid and concatenated to obtain the final feature vector: V = [GAP(P1); GAP(P2); GAP(P3)] e R 384 Through the above FPN feature fusion process, different scale features extracted by EfficientNet-B0 and SwinTransformer can be fully integrated to generate a feature vector containing rich information, providing more discriminative input for the subsequent integrated ELM classifier, thereby improving the performance of fabric defect classification.

[0087] Step 4: Integrated extreme learning machine classifier

[0088] (4.1) Base learner ELM unit

[0089] The mathematical expression of each base learner is:

[0090] H = g(XW + b)

[0091] Where: X e R n×d is a 384-dimensional feature matrix output by FPN, and n is the number of samples; W e R d×L is a random input weight matrix, and L is the number of hidden layer nodes; b e R L is a random bias vector; g(·) is a ReLU activation function, g(x) = max(0, x); H e R n ×L is the hidden layer output matrix.

[0092] (4.2) Diversity dynamic integration strategy

[0093] In the traditional ELM integration method, the base learners often have the problem of homogeneity, that is, the features learned by multiple base learners and the decisions made are similar, which cannot improve the performance of the integrated model. The diversity dynamic integration strategy proposed in the present application sets the number of hidden layer nodes of each base learner in the random hyperparameter space as L ~ U(300, 700), indicating that L is uniformly distributed from 300 to 700; the regularization coefficient is log 10 α ~ U(-6, -2), that is, the common logarithm of α is uniformly distributed from -6 to -2. By this way, different ELM base learners are constructed, which has many innovations:

[0094] 1. Breaking the homogeneity limitation of base learners: In traditional ELM integration, the structure and parameters of each base learner are similar, which makes it difficult to capture different features when processing complex fabric defect data. This strategy randomly selects different hidden layer node numbers and regularization coefficients in each base learner, so that each base learner can focus on different features of fabric defects during training.

[0095] 2. Enhance the generalization ability of the model: with the help of diversified base learners, the model can learn more extensive feature representation, reduce the dependence on specific samples or features, and improve the generalization ability. When facing different textures, defect types and complex production environment of fabric images, the model can adapt to changes and improve the accuracy and stability of classification.

[0096] 3. Improve the robustness of the model: diversified base learners can complement each other. If a base learner has a large deviation when processing specific data, other base learners can correct it accordingly. This complementary mechanism improves the robustness of the model, allowing it to maintain good performance when facing noisy data or abnormal samples.

[0097] (4.3) Adaptive decision fusion

[0098] Traditional decision fusion methods generally use fixed weights or simple voting methods. These methods cannot reflect the differentiated performance of base learners in the data set. In the training phase, the output weight of the kth base learner is defined as:

[0099]

[0100] where Acc k is the accuracy of the base learner on the validation set, and η is the temperature coefficient (set to 2.0 in the experiment). This dynamic weight distribution strategy based on accuracy can adjust the contribution of each base learner in the final classification decision according to its actual performance on the validation set. Base learners with excellent performance will have higher weights and play a greater role in decision-making; while base learners with poor performance will have relatively low weights, avoiding negative impact on the final result.

[0101] The final classification decision is realized through weighted voting, and the formula is:

[0102]

[0103] where p k (j|x) represents the probability of the kth base learner predicting that sample x belongs to class j, and M represents the number of ELM units of the base learner.

[0104] Weighted voting fully considers the reliability of each base learner compared to simple voting, making the classification decision more reasonable. Different base learners may have different predictions for the same sample when facing complex fabric defect data. Weighted voting considers the weights and prediction probabilities of each base learner to make a more realistic decision.

[0105] Step 5: Realizing fabric defect classification by using the trained ESF-edELM classification model

[0106] The training loss of the ESF-edELM classification model includes but is not limited to multi-classification cross-entropy loss and regularized least squares loss.

[0107] The deep feature fusion integrated extreme learning machine architecture plays a key role in achieving accurate classification in the fabric defect classification model, and the deep feature fusion is composed of EfficientNet-B0, SwinTransformer, CBAM attention mechanism and FPN. EfficientNet-B0 uses a compound scaling strategy to optimize the model, obtains key layer features through the hook mechanism to provide rich local information, and embeds CBAM after EfficientNet-B0 extracts local detail features to effectively suppress noise and irrelevant information and improve the expression ability of features. Swin Transformer uses hierarchical window division to extract features from multiple scales to capture subtle textures and global information, and FPN is responsible for fusing the features of the former two. It first relies on 1x1 convolution to unify the feature dimensions, then adaptively allocates weights to fuse high and low level features, and generates a feature vector containing rich scale information to provide excellent input for integrated ELM. Deep feature fusion and integrated ELM work together, and integrated ELM is responsible for the final classification decision in the entire model, which is based on extreme learning machine. Extreme learning machine belongs to single hidden layer feedforward neural network, and the connection weights between input layer and hidden layer and the hidden layer threshold are randomly set during training, only the output layer weight is calculated, and the training speed is fast. However, facing complex and diverse fabric defect data, a single extreme learning machine is difficult to fully capture features, affecting classification performance. Therefore, this architecture integrates multiple ELM base learners, which are trained in random hyperparameters, with different hidden layer node numbers and regularization coefficients, and the learning results are different. They capture different defect features and then fuse them through a specific strategy. The recognition ability of the integrated model is stronger, and the classification performance is improved.

[0108] To further explore the role of attention mechanism in the fabric defect classification model based on deep feature fusion, three groups of comparative experiments were carried out, such as Figure 2The experiment was carried out without using attention mechanism (denoted as "None") and using SE attention mechanism (denoted as "SE") at the original position of CBAM. The model accuracy of the CBAM attention mechanism reached 93.44%, the model accuracy of the SE attention mechanism reached 91.48%, and the model accuracy of the SE attention mechanism reached 92.46%. The application of CBAM was obviously higher than the model accuracy of the comparative method, which means that the CBAM attention mechanism has an excellent performance in strengthening the model's ability to capture fabric defect features compared to SE and None, and can effectively improve the accuracy of classification. The reason is that the CBAM attention mechanism uses the channel attention module and the spatial attention module to perform adaptive re-labeling operation on the feature map in the channel and spatial dimensions, respectively, effectively suppressing noise and highlighting the defect area, improving the expression ability of key features and providing more discriminative features for subsequent classification.

[0109] In order to explore the breakthrough of the double-branch architecture in feature expression ability compared with the traditional single-branch network, and the advantage compared with the existing hybrid architecture, this experiment compared the classification effect of the single-branch feature extraction network of EfficientNet-B0 and Swin Transformer in the integrated ELM classifier, as shown in Table 1.

[0110] Table 1 Single-branch feature extraction network experimental results

[0111]

[0112]

[0113] It can be seen that EfficientNet-B0-edELM performs well in accuracy, recall rate, weighted F1 score, and Cohen's Kappa, but it still has shortcomings in comprehensive capture of complex defect features. As a single-branch structure, it focuses on local feature extraction, and has limited ability to mine the distribution of defects in the global fabric and the associated features of the surrounding area. When facing complex scenes such as mixed defects, it may misjudge due to incomplete feature extraction. The indicators of SwinTransformer-edELM are relatively low, which indicates that the single-branch SwinTransformer is difficult to balance local details and global information in feature expression, and has weak ability in extracting local features such as fine texture, which limits the overall performance of the model.

[0114] FPN fusion systematically integrates the advantages of multi-layer features of the double branch through the structured design of "hierarchical alignment + cross-layer interaction + dynamic weight". Simple concatenation and average fusion lack scale perception and weight optimization, making it difficult to fully exploit the potential of multi-branch architecture. To verify the performance difference of the model feature fusion method, this experiment selects three fusion methods: simple concatenation fusion, average fusion, and FPN fusion, as shown in Figure 3 The three feature fusion methods show obvious differences in accuracy and F1 score, two key indicators. The accuracy of the simple concatenation fusion method is 88.23%, and the F1 score is 87.87%. This method lacks in-depth mining of the correlation between features and cannot effectively integrate feature information. The accuracy of the average fusion method is improved to 90.87%, and the F1 score reaches 90.79%. Compared with simple concatenation fusion, the average fusion method takes the average of features, which to some extent alleviates the differences between different features and achieves preliminary information integration. However, it does not consider the differences in importance of different features and only treats them equally, so there is a certain limitation in improving performance. The fusion scheme in this study fully considers the hierarchical structure and multi-scale information of features, and adaptively adjusts the weights of different features, allowing the model to more effectively capture key information, especially for tasks such as fabric defect classification that require consideration of both details and global features.

[0115] To verify the performance difference of the model classification, this experiment selects BaseELM and EnsembleELM two classifiers, and the experimental results are shown in Figures 4-5 For the stability analysis of ESF on two classifiers, it can be seen that the weighted F1 score of BaseELM is 0.9074, and the standard deviation reflects that its fluctuation is relatively large. The weighted F1 score of EnsembleELM is 0.9207, and the fluctuation is small. This shows that the integrated ELM method enhances the stability of the model through the synergistic effect of multiple base learners, can maintain consistent performance on different training and testing data, reduces the performance decline caused by data fluctuations, and is more reliable in practical applications.

[0116] To verify the classification performance difference of the overall model and existing methods on this dataset, this experiment selects a variety of classification models, as shown in Table 2.

[0117] Table 2 Experimental results of different classification models

[0118]

[0119] From the experimental results in Table 2, it can be seen that the present application achieves the best effect. In contrast, VGG16 and ResNet50, as classic deep convolutional neural networks, show better classification performance when used alone. The accuracy of VGG16 reaches 91.14%, and the F1 score is 91.23%; ResNet50 performs better, with an accuracy of 91.97% and an F1 score of 91.92%. The Cohen's Kappa coefficients of these two models are 85.14% and 86.42%, respectively, indicating that the models have high consistency. However, the performance of ELM combined with them decreases significantly, which may be due to the training method of ELM randomly setting the connection weights between the input layer and the hidden layer and the hidden layer threshold, which does not match the complex feature extraction mechanism of VGG16 and ResNet50. ELM cannot fully utilize the high-level features extracted by these two models, leading to information loss or incorrect judgments, which in turn affects the classification performance. The ConvNeXt model is sensitive to noise and may mistakenly consider it as a defect feature, thus extracting inaccurate features and affecting the classification performance. The Vision Transformer model divides the image into multiple fixed-size patches for processing, focusing more on global feature learning and having relatively weak local feature capture ability, which may not be able to capture these features well.

[0120] In this embodiment, a fabric defect classification system based on deep integrated ELM is also provided, which is used to implement the fabric defect classification method described above. The system includes:

[0121] A preprocessing module for preparing and preprocessing the fabric defect image dataset;

[0122] A dual-branch deep feature extraction module for constructing a dual-branch deep feature extraction network. The first branch extracts three levels of features based on EfficientNet-B0, obtaining first feature E, second feature E, and third feature E, respectively. The channel number of the three layers of features increases layer by layer, and the spatial dimension decreases layer by layer. An embedded channel-spatial attention mechanism is used to enhance the features of the three levels, obtaining first enhanced feature E, second enhanced feature E, and third enhanced feature E, respectively. The second branch extracts three levels of features based on SwinTransformer, including first feature S, second feature S, and third feature S. The channel number of the three layers of features increases layer by layer, and the spatial dimension decreases layer by layer.

[0123] A feature fusion module for converting the obtained first enhanced feature E, second enhanced feature E, third enhanced feature E, first feature S, second feature S, and third feature S through a dimension conversion network to obtain feature maps with the same channel and unchanged spatial dimension. A feature pyramid network is used to perform multi-scale fusion on the dimension-converted feature maps to generate a final feature vector.

[0124] an integrated extreme learning machine module for classifying the final feature vector by using an integrated extreme learning machine, the integrated extreme learning machine comprising a plurality of differentiated base learners, each base learner randomizing hidden layer parameters including a number of hidden layer nodes and a regularization coefficient, wherein the number of hidden layer nodes is subject to a uniform distribution of 300 to 700, and the common logarithm of the regularization coefficient is subject to a uniform distribution of -6 to -2;

[0125] a training and recognition module for training a fabric defect classification model composed of the double-branch deep feature extraction network, the dimension conversion network, the feature pyramid network and the integrated extreme learning machine, and realizing fabric defect classification by using the trained fabric defect classification model and outputting a fabric defect type.

[0126] For the system embodiment, since it basically corresponds to the method embodiment, the relevant part is described in the method embodiment, and the implementation method of the remaining modules is not described here. The system embodiment described above is only illustrative, wherein the units described as separate components can be or can not be physically separated, and the components displayed as units can be or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Some or all of the modules can be selected to achieve the purpose of the present application according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0127] The system embodiment of the present application can be applied to any device with data processing capability, which can be a device or apparatus such as a computer. The system embodiment can be realized by software, or by hardware or a combination of software and hardware. Taking software realization as an example, as a logical device, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory and running by the processor of the device with data processing capability.

[0128] The above-described embodiments only express several embodiments of the present application, which are described in detail and specifically, but cannot be understood as a limitation on the scope of the present application. For those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the protection scope of the present application.

Claims

1. A fabric defect classification method based on deep integrated ELM, characterized in that, Includes the following steps: (1) Prepare a dataset of fabric defect images and perform preprocessing; (2) Construct a dual-branch deep feature extraction network. The first branch extracts three-level features based on EfficientNet-B0, which are the first feature E, the second feature E, and the third feature E. The number of channels in the three-level features increases layer by layer, and the spatial dimension decreases layer by layer. The channel-space attention mechanism is embedded to enhance the three-level features, which are the first enhanced feature E, the second enhanced feature E, and the third enhanced feature E. The second branch extracts three-level features based on Swin Transformer, which include the first feature S, the second feature S, and the third feature S. The number of channels in the three-level features increases layer by layer, and the spatial dimension decreases layer by layer. (3) The first enhanced feature E, the second enhanced feature E, the third enhanced feature E, the first feature S, the second feature S, and the third feature S obtained in step (2) are passed through a dimension transformation network to obtain feature maps with the same channel and unchanged spatial dimension. The feature pyramid network is used to perform multi-scale fusion on the dimension-transformed feature maps to generate the final feature vector. (4) The final feature vector is classified using an integrated extreme learning machine. The integrated extreme learning machine contains multiple differential base learners. Each base learner randomizes the hidden layer parameters, including the number of hidden layer nodes and the regularization coefficient. The number of hidden layer nodes follows a uniform distribution of 300 to 700, and the common logarithm of the regularization coefficient follows a uniform distribution of -6 to -2. (5) Train a fabric defect classification model consisting of a dual-branch deep feature extraction network, a dimension transformation network, a feature pyramid network, and an ensemble extreme learning mechanism. Use the trained fabric defect classification model to classify fabric defects and output the fabric defect type.

2. The fabric defect classification method based on deep integrated ELM according to claim 1, characterized in that, The spatial dimensions of the first feature E, the second feature E, and the third feature E correspond one-to-one with the spatial dimensions of the first feature S, the second feature S, and the third feature S, respectively.

3. The fabric defect classification method based on deep integrated ELM according to claim 1 or 2, characterized in that, The first feature E, the second feature E, and the third feature E are generated by blocks[1], blocks[3], and blocks[5] of the EfficientNet-B0 model, respectively, with dimensions of 56×56×32, 28×28×56, and 14×14×160.

4. The fabric defect classification method based on deep integrated ELM according to claim 1, characterized in that, In the first branch, the process of feature enhancement using the embedded channel-space attention mechanism includes: The first feature E, the second feature E, and the third feature E are input into the channel-spatial attention module. First, the channel attention module generates channel descriptors through global average pooling and max pooling. Then, channel attention weights are generated by multilayer perceptron and sigmoid activation. The channel attention weights are multiplied element-wise with the input features to obtain the feature map enhanced by channel attention. Next, the spatial attention module performs average pooling and max pooling along the channel dimension on the channel-enhanced feature map to generate spatial descriptors. Then, spatial attention weights are generated by 7×7 convolution and sigmoid activation. The spatial attention weights are multiplied element-wise with the channel-enhanced feature map to obtain the final enhanced feature map.

5. The fabric defect classification method based on deep integrated ELM according to claim 1 or 2, characterized in that, In the second branch, the first feature S, the second feature S, and the third feature S are generated by Stage 0, Stage 1, and Stage 2 of the Swin Transformer model, respectively, with dimensions of 56×56×96, 28×28×192, and 14×14×384.

6. The fabric defect classification method based on deep integrated ELM according to claim 2, characterized in that, Step (3) includes: (3.1) The channel dimension is unified to 128 dimensions through 1×1 convolution; (3.2) Softmax weight allocation is used to achieve adaptive fusion of features with the same spatial and channel dimensions, generating three fused features F. fused,1 F fused,2 F fused,3 The adaptive fusion formula is as follows: F fused,i =softmax(w)1·F′ e,i +softmax(w)2·F′ s,i Where w is a learnable parameter, w∈R 2 The softmax operation transforms the weights into two values, softmax(w)1 and softmax(w)2, which are between 0 and 1 and sum to 1; F′ e,i It is the i-th enhanced feature E, F′ s,i It is the i-th feature S, F′ e,i and F′ s,i Both the channel dimension and the spatial dimension are the same, F fused,i It is the i-th fusion feature, where i = 1, 2, 3; (3.3) Feature upsampling and pyramid construction are performed through a top-down path to generate a three-level feature pyramid {P1,P2,P3}. Based on the three-level feature pyramid, the final feature vector V = [GAP(P1); GAP(P2); GAP(P3)] is generated, where GAP(.) represents global average pooling and [.;.;.] represents concatenation.

7. The fabric defect classification method based on deep integrated ELM according to claim 6, characterized in that, The formula for constructing a three-level feature pyramid is: P3=F fused,3 P2=Upsample(P3)+F fused,2 P1=Upsample(P2)+F fused,1 Where P3, P2, and P1 represent the three-layer feature pyramid formed by the top-down path, and Upsample(.) represents upsampling.

8. The fabric defect classification method based on deep integrated ELM according to claim 1, characterized in that, During the training phase, the ensemble extreme learning machine performs adaptive weighted decision fusion based on the validation set accuracy. The weighted voting formula is as follows: Among them, Acc k Let Acc be the accuracy of the k-th base learner on the validation set. i Let M be the accuracy of the i-th base learner on the validation set, M be the number of base learners, η be the temperature coefficient, and w be the base learner's accuracy on the validation set. k p represents the weights of the k-th base learner. k (j|x) represents the probability that the k-th base learner predicts the final feature vector x belongs to class j. This is the predicted result.

9. The fabric defect classification method based on deep integrated ELM according to claim 1, characterized in that, Fabric defects include holes, stains, oil stains, loose threads, and foreign objects.

10. A fabric defect classification system based on deep integrated ELM, used to implement the fabric defect classification method based on deep integrated ELM as described in claim 1, characterized in that, The system includes: The preprocessing module is used to prepare and preprocess the fabric defect image dataset. The dual-branch deep feature extraction module is used to construct a dual-branch deep feature extraction network. The first branch extracts three-level features based on EfficientNet-B0, resulting in first feature E, second feature E, and third feature E. The number of channels in the three-level features increases layer by layer, while the spatial dimension decreases layer by layer. An embedded channel-space attention mechanism is used to enhance the three-level features, resulting in first enhanced feature E, second enhanced feature E, and third enhanced feature E. The second branch extracts three-level features based on SwinTransformer, including first feature S, second feature S, and third feature S. The number of channels in the three-level features increases layer by layer, while the spatial dimension decreases layer by layer. The feature fusion module is used to obtain feature maps with the same channel and unchanged spatial dimension by passing the first enhanced feature E, the second enhanced feature E, the third enhanced feature E, the first feature S, the second feature S, and the third feature S through a dimension transformation network. The feature pyramid network is then used to perform multi-scale fusion on the dimension-transformed feature maps to generate the final feature vector. An integrated extreme learning machine module is used to classify the final feature vector using an integrated extreme learning machine. The integrated extreme learning machine contains multiple differential base learners. Each base learner randomizes the hidden layer parameters, including the number of hidden layer nodes and the regularization coefficient. The number of hidden layer nodes follows a uniform distribution of 300 to 700, and the common logarithm of the regularization coefficient follows a uniform distribution of -6 to -2. The training and recognition module is used to train a fabric defect classification model consisting of a dual-branch deep feature extraction network, a dimension transformation network, a feature pyramid network, and an ensemble extreme learning mechanism. The trained fabric defect classification model is used to classify fabric defects and output the fabric defect type.

Images