Deep learning-based jacquard fabric detection method and embedded device

Abstract

The invention discloses a deep learning-based jacquard fabric detection method and embedded equipment, and the method comprises the steps: carrying out the improvement and optimization of a Backbone layer of a model, a Starblock module in the model and a C2f-SP module in the model on the basis of a YOLOv8 target detection model without an anchor frame, so as to enhance the feature extraction effect of the model, and improve the detection precision of the jacquard fabric. The model is input and mapped to a high-dimensional nonlinear feature extraction space to enhance the processing capacity of the model for complex features and improve the calculation efficiency of the model, then the SCPL-YOLO model is optimized in combination with a model pruning and knowledge distillation method to realize model compression, and the storage space and calculation overhead of the model are effectively reduced on the premise of ensuring the detection precision. According to the method, on NVIDIA Jetson Orin NX 16GB equipment, the optimized model single-frame reasoning time delay can be limited to 6 ms, the real-time detection requirement can be met, the detection result can be fed back to a front-end interface in real time, and the method can be widely applied to the actual production environment of textile enterprises.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, and more specifically to a method for detecting jacquard fabrics based on deep learning. Background Technology

[0002] With the intelligent and digital transformation and upgrading of my country's textile industry, higher requirements have been placed on the accuracy and efficiency of fabric appearance quality inspection. In the textile manufacturing process, especially for high-value-added jacquard fabrics, whose surface patterns are complex and textures are delicate, even the slightest flaw can affect the aesthetics and performance of the final product. The core task of jacquard fabric inspection is to identify defects such as broken weft, broken warp, color difference, and missing patterns on its surface. This process has traditionally relied mainly on manual visual inspection. However, manual inspection suffers from low efficiency, results heavily influenced by subjective factors, and a high false positive rate due to visual fatigue, and it is difficult to meet the continuous production needs of large-scale production lines. With the expansion of production scale and the increasing demands of consumers for product quality, traditional inspection methods can no longer meet the high-precision and high-efficiency needs of textile enterprises.

[0003] In recent years, the rapid development of computer vision technology has provided a new technical approach for fabric defect detection. In particular, the emergence of deep learning models has greatly improved the ability to extract image features and recognize targets. In fabric detection tasks, deep learning algorithms can automatically learn complex texture features, avoiding the tedious process of manual feature extraction in traditional image processing methods.

[0004] However, despite the excellent performance of deep learning in many visual recognition tasks, directly applying general object detection algorithms (such as YOLO, SSD, Faster-RCNN, etc.) to jacquard fabric detection still faces multiple challenges: On the one hand, the texture of jacquard fabrics is highly heterogeneous, with frequent changes in patterns and blurred boundaries, making it difficult for models to accurately distinguish between defects and normal textures; on the other hand, the feature extraction effect of YOLO series models is unstable in complex texture backgrounds, and it is prone to missed detections and false detections, especially when detecting small defects.

[0005] Furthermore, existing deep learning detection models generally suffer from large model size and high computational complexity, making them difficult to run efficiently on resource-constrained embedded devices, thus limiting their application in the textile industry. This is mainly because textile production line detection systems often require real-time detection and feedback on embedded terminals; therefore, balancing lightweight and high accuracy in algorithm models has become a research focus. Currently, academia and industry have proposed various improvement strategies to address this issue, including using feature pyramid structures to enhance multi-scale feature fusion and introducing attention mechanisms to improve the model's responsiveness to key regions. However, the application of these methods in jacquard fabric detection still requires further optimization to achieve a balance between detection accuracy and running speed. Summary of the Invention

[0006] The problem this invention aims to solve is how to effectively and accurately distinguish the subtle differences between defect structures and jacquard textures in jacquard fabrics, given their heterogeneous characteristics and frequently changing and blurred patterns, and how to apply this to a lightweight testing terminal to achieve effective detection of defects in jacquard fabrics.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: a jacquard fabric detection method based on deep learning, the steps of which include: identifying the input image, video or real-time data from a camera according to a pre-trained detection model to obtain the detection result of jacquard fabric defects; wherein, the detection model includes: a backbone network, which uses a CBS module as the starting feature layer, sequentially connects 4 layers of StarNet-P network structure, performs dimensionality reduction on the feature map layer by layer, and transforms the dimensionality-reduced input features into a high-dimensional nonlinear space; a neck network, which connects the 2nd to 4th layers of StarNet-P network structure, and is used to: enhance the feature expression of the feature map output by the 4th layer of StarNet-P network structure through the C2f-SP module, then upsample it, and concatenate it with the feature map output by the 3rd layer of StarNet-P network structure; and perform dimensionality reduction on the 3rd and 4th layers of StarNet-P network structure. The feature map after stitching the StarNet-P network structure is fused by the C2f-SP module, then upsampled and stitched with the feature map output by the second layer of the StarNet-P network structure. The feature map after stitching the second and third layers of the StarNet-P network structure is enhanced by the C2f-SP module, then input into the CBS convolutional layer for channel compression and feature enhancement, and then stitched with the feature map output after fusion with the third and fourth layers of the StarNet-P network structure. Finally, the stitched result is input into the C2f-SP module, stitched with the feature map output by the fourth layer of the StarNet-P network structure through the CBS convolutional layer, and then output through the last three C2f-SP modules. The detection head uses a lightweight shared group convolutional (LSGC) structure to receive the features output by the C2f-SP module and outputs the detection results of jacquard fabric defects.

[0008] Optionally, in any of the deep learning-based jacquard fabric detection methods described above, each layer of the StarNet-P network structure includes: a CBR module, which consists of sequentially connected 3×3 convolutions, batch normalization (BN), and ReLU6 activation functions, used to downsample the input feature map; a Starblock module, which consists of fully connected layers, 3×3 PConv convolutional modules, and residual connections, wherein: two 3×3 PConv convolutional modules serve as the input and output ends of the Starblock module, respectively, connecting the CBR module of this layer of the StarNet-P network structure and the output feature map of the next level; three fully connected layers (FC) connect the two 3×3 PConv convolutional modules, where the two fully connected layers (FC) process the output of the input 3×3 PConv convolutional modules respectively, and then map the input features to a high-dimensional linear space through star operations, and then provide them to the output 3×3 PConv convolutional module through the third fully connected layer (FC); the output 3×3 The PConv convolution module also provides its output as a residual to the input as the input to the 3×3 PConv convolution module.

[0009] Optionally, in any of the deep learning-based jacquard fabric detection methods described above, the mathematical expression for the star operation is: Where W1 and W2 are weight matrices, and B1 and B2 are bias vectors. This indicates element-wise multiplication.

[0010] Optionally, in any of the above-described deep learning-based jacquard fabric detection methods, each C2f-SP module includes: a CBS module, respectively located at the input and output ends of the C2f-SP module; a feature segmentation module, used to split the feature map output by the input CBS module into two sub-feature maps along the channel dimension; a feature extraction module, which uses a three-level sequentially connected Starblock module to extract features from one of the sub-feature maps output by the feature segmentation module; and a stitching unit, which connects the three-level Starblock module and the other sub-feature map output by the feature segmentation module, stitches their outputs together, and provides them to the output CBS module.

[0011] Optionally, in any of the deep learning-based jacquard fabric detection methods described above, in the Starblock module used in each C2f-SP module, PConv (Partial Convolution) replaces DWConv (Depthwise Convolution), and PConv performs convolution operations only on a portion of the input channels, while the remaining channels are directly retained.

[0012] Optionally, in any of the deep learning-based jacquard fabric detection methods described above, the Lightweight Shared Group Convolution (LSGC) structure replaces the BN layer with Group Normalized Convolution (Conv_GN) to normalize the channels. Its specific structure includes: three 3×3 CBS modules for preprocessing input features of different sizes; each of the three 3×3 CBS modules is followed by a 1×1 Conv_GN unit to adjust the number of channels; and a 3×3 Conv_GN unit that performs multi-scale feature fusion on the outputs of the three 1×1 Conv_GN units. After regression branching and scale mapping, the precise location of the jacquard fabric defect in the image is output, and after classification branching, the probability corresponding to the shrinkage of the jacquard fabric defect is output.

[0013] Optionally, in any of the deep learning-based jacquard fabric detection methods described above, after the detection model has been trained for jacquard fabric defects, it is further compressed by the LAMP (Layer-Adaptive Magnitude-based Pruning) pruning algorithm and the knowledge distillation method.

[0014] Optionally, in any of the deep learning-based jacquard fabric detection methods described above, the pruning algorithm first calculates the LAMP score for each layer of weights obtained from training the detection model, then globally sorts the weights based on the LAMP scores, removes the connections with the lowest scores, and continues until the global sparsity constraint is met. Afterward, it is used as a student model and distilled using a knowledge distillation method. The knowledge distillation method uses YOLOv8x as the teacher model and the model obtained from the pruning algorithm as the student model, calculating the inter-channel correlation matrix of the teacher model and the student model respectively according to the following formula: , Then, using the distillation loss function: Total loss function: To minimize the inter-channel correlation error between the teacher model and the student model, a compressed detection model is obtained.

[0015] To achieve the above objectives, this application also provides an embedded device for detecting jacquard fabrics based on deep learning. Based on the PySide6 framework, it includes: a detection interface for receiving images, videos, or real-time data from a camera; a detection model for receiving data from the detection interface and outputting detection results according to a pre-defined detection model; and a visualization platform for displaying the detection results. The detection model includes: a backbone network that uses a CBS module as the starting feature layer, sequentially connecting four layers of StarNet-P network structure, reducing the dimensionality of the feature map layer by layer, and transforming the dimensionality-reduced input features into a high-dimensional nonlinear space; and a neck network that connects the second to fourth layers of StarNet-P network structure, used to: enhance the feature representation of the feature map output by the fourth layer of StarNet-P network structure through a C2f-SP module, then upsample it, and connect it to the third layer of StarNet-P network. The feature maps output by the structure are stitched together; the feature maps stitched together from the 3rd and 4th layers of the StarNet-P network structure are fused through the C2f-SP module, then upsampled and stitched together with the feature maps output from the 2nd layer of the StarNet-P network structure; the feature maps stitched together from the 2nd and 3rd layers of the StarNet-P network structure are enhanced through the C2f-SP module, then input to the CBS convolutional layer for channel compression and feature enhancement, and then stitched together with the feature maps output from the fusion of the 3rd and 4th layers of the StarNet-P network structure; finally, the stitched result is input into the C2f-SP module, stitched together with the feature maps output from the 4th layer of the StarNet-P network structure through the CBS convolutional layer, and then output through the last three C2f-SP modules; the detection head uses a lightweight shared group convolutional (LSGC) structure to receive the features output by the C2f-SP module and outputs the detection results of jacquard fabric defects.

[0016] Optionally, in the deep learning-based embedded device for jacquard fabric detection as described above, after the detection model has been trained for jacquard fabric defects, it is further compressed using the LAMP (Layer-Adaptive Magnitude-based Pruning) algorithm and the knowledge distillation method: First, the LAMP score is calculated for each layer weight obtained from the training of the detection model. The weights are then globally sorted based on the LAMP scores, and the connections with the smallest scores are removed until the global sparsity constraint is met. This is then used as the student model, with YOLOv8x as the teacher model. The inter-channel correlation matrix of the teacher model and the student model is calculated according to the following formula: , Then, using the distillation loss function: Total loss function: To minimize the inter-channel correlation error between the teacher model and the student model, a compressed detection model is obtained.

[0017] Beneficial effects This application discloses a deep learning-based jacquard fabric detection method and embedded device. Based on the anchor-free YOLOv8 object detection algorithm model, it improves and optimizes the model's Backbone layer, Starblock module, and C2f-SP module to enhance feature extraction performance. The model input is mapped to a high-dimensional nonlinear feature extraction space to enhance the model's ability to handle complex features and improve computational efficiency. Then, model pruning and knowledge distillation methods are combined to optimize the SCPL-YOLO model, achieving model compression and effectively reducing storage space and computational overhead while ensuring detection accuracy. On an NVIDIA Jetson Orin NX 16GB device, this application can limit the single-frame inference latency of the optimized model to 6ms, meeting real-time detection requirements and providing real-time feedback of detection results to the front-end interface. It can be widely applied in the actual production environment of textile enterprises. This invention demonstrates high precision, recall, and real-time performance in jacquard fabric detection tasks, showing promising prospects for industrial applications.

[0018] Other features and advantages of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing this application. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the present application and form part of the specification. They, together with the embodiments of the present application, serve to explain the present application but do not constitute a limitation thereof. The following detailed description, in conjunction with the accompanying drawings and embodiments of the present invention, is provided in further detail: Figure 1 This is a schematic diagram of the network architecture of the YOLOv8 model used in this application; Figure 2 A schematic diagram illustrating the optimization of the joint model pruning and knowledge distillation methods in this application; Figure 3 This diagram illustrates the proportion of each category of samples in the total dataset after data augmentation of the training samples used in the model of this application. Figure 4 This is a diagram of the StarNet-P network structure used in the model of this application; Figure 5 This is a structural diagram of the improved Starblock module used in the model of this application; Figure 6 This is a structural diagram of the LSGC module used in the model of this application; Figure 7 This is a flowchart illustrating the steps of the LAMP pruning training method used in this application. Figure 8 This is a schematic diagram of the ICC knowledge distillation structure used in this application; Figure 9 This is a structural diagram of the C2f-SP module used in the model of this application; Figure 10 This is a flowchart illustrating the model conversion process for the model in this application. Figure 11 This is the system architecture execution diagram of the model in this application; Figure 12 This is a schematic diagram of the functional interface of the jacquard fabric testing platform based on the model in this application. Detailed Implementation

[0020] To make the objectives and technical solutions of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the described embodiments of this application without creative effort are within the scope of protection of this application.

[0021] Those skilled in the art will understand that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0022] The term "connection" as used in this application can mean a direct connection between components or an indirect connection between components through other components.

[0023] YOLOv8, as an anchor-free object detection algorithm, abandons the traditional anchor-based detection method. This design optimizes its training process, significantly improving its detection flexibility and accuracy. The algorithm is classified as an anchor-free object detection algorithm during the inference phase because it does not rely on predefined anchor boxes. Although the concept of anchor boxes is still used during training, these "anchor boxes" serve only as initial estimates or references for the scale and aspect ratio of the bounding boxes. The algorithm can adjust and optimize these estimates based on the true bounding boxes of the objects in the training images to improve the model's adaptability and accuracy. However, existing YOLOv8 detection models suffer from insufficient feature extraction due to the diverse patterns in jacquard fabrics with complex textures. Furthermore, its large number of model parameters and high computational complexity make it unsuitable for integration into handheld terminals used in the textile industry, hindering real-time, high-precision fabric defect detection.

[0024] Figure 1 According to this application, a deep learning-based jacquard fabric detection model is proposed. The StarNet-P network is introduced into the improved YOLOv8 backbone network (backbone layer) to enhance the feature extraction effect of the model. In the neck network (Neck layer), PConv

[59] is used to improve the Starblock

[58] module, and the improved module replaces the Bottleneck module in the C2f module. This improves the module so that it can not only map the input to a high-dimensional nonlinear feature space to enhance the model's processing of complex features, but also improves the computational efficiency of the model. In conjunction with the use of the lightweight shared group convolutional structure LSGC to optimize the detection head of the original model in the detection head (Head layer), the model can realize the feature sharing function of feature maps of different scales to improve the accuracy of the algorithm for target localization and category prediction. Finally, this application also introduces the LAMP pruning algorithm to achieve model lightweighting in order to address the problems of large model parameters and high computational complexity. In addition, to prevent the accuracy loss caused by pruning, the ICC knowledge distillation method is used to improve the detection performance of the lightweight model.

[0025] Specifically, the detection model includes: The backbone network uses the CBS module as the starting feature layer, sequentially connects 4 layers of StarNet-P network structure, reduces the dimensionality of the feature map layer by layer, and transforms the dimensionality-reduced input features into a high-dimensional nonlinear space. The neck network, connecting layers 2 to 4 of the StarNet-P network structure, is used for: enhancing the feature map output from layer 4 of the StarNet-P network through the C2f-SP module, then upsampling it and concatenating it with the feature map output from layer 3 of the StarNet-P network structure; fusing the feature map after concatenation of layers 3 and 4 of the StarNet-P network structure through the C2f-SP module, then upsampling it and concatenating it with the feature map output from layer 2 of the StarNet-P network structure; enhancing the feature map after concatenation of layers 2 and 3 of the StarNet-P network structure through the C2f-SP module, then inputting it into the CBS convolutional layer for channel compression and feature enhancement, and then concatenating it with the feature map output after fusion of layers 3 and 4 of the StarNet-P network structure; finally, inputting the concatenated result into the C2f-SP module, concatenating it with the feature map output from layer 4 of the StarNet-P network structure through the CBS convolutional layer, and then outputting it through the last three C2f-SP modules. The detection head uses a lightweight shared group convolution (LSGC) structure to receive features output by the C2f-SP module and outputs the detection results of jacquard fabric defects.

[0026] After completing training for defects in jacquard fabrics, the model also referenced... Figure 2 As shown, the model is compressed sequentially using the LAMP (Layer-Adaptive Magnitude-based Pruning) algorithm and the knowledge distillation method: First, a LAMP score is calculated for the weights of each layer obtained from training the detection model. The weights are then globally sorted based on their LAMP scores, and the connections with the lowest scores are removed. This process continues until the global sparsity constraint is met, resulting in the SCPL-YOLO-P model. Then, using the SCPL-YOLO-P model as the student model and YOLOv8x as the teacher model, the inter-channel correlation matrices of the teacher and student models are calculated according to the following formula: , Then, using the distillation loss function: Total loss function: To minimize the inter-channel correlation error between the teacher model and the student model, a compressed detection model is obtained.

[0027] The compressed detection model can be deployed in an embedded device based on the PySide6 framework, and can receive images or videos to be detected or directly receive real-time data from a camera using the device's detection interface. The compressed detection model receives data from the detection interface and identifies the input images, videos, or real-time data from the camera based on pre-trained weight parameters to obtain the detection results of jacquard fabric defects. The results are then output to the visualization platform connected to the device to display the detection model's findings.

[0028] In a more specific implementation, the SCPL-YOLO detection model upon which this application is based refers to... Figure 4 To enhance the YOLOv8 model's multi-scale feature extraction capabilities, the StarNet-P network can be used to replace the YOLOv8 backbone. This structure improves the model's ability to perceive complex pattern edges by introducing multi-level residual connections and a spatial pyramid feature fusion mechanism. The backbone network employs a four-stage hierarchical modular architecture, primarily composed of multiple convolutional layers, fully connected layers, and a small number of PConv modules. The first processing module is the CBR module, which consists of 3x3 convolutional layers with a stride of 2, BN layers, and a ReLU6 activation function. The CBR module's role in the backbone network is to perform dimensionality reduction on the input feature map. For example, if the input feature map is 640x640 in length and width, after downsampling by the 2-stride convolutional layer in this module, the feature map's length and width will change to 320x320. The four processing modules in the backbone network are all Stage modules. Each Stage module mainly consists of a CBR module and a Starblock module. Their main function is to perform dimensionality reduction and element-wise multiplication operations to transform the input features into a high-dimensional nonlinear space, thereby enhancing the network's feature representation capability. Each layer of the StarNet-P network structure is configured to include: The CBR module, consisting of sequentially connected 3×3 convolutions, batch normalization (BN), and ReLU6 activation functions, is used to downsample the input feature map; The Starblock module is the core module of the aforementioned StarNet-P network. It consists of fully connected layers, 3×3 PConv convolutional modules with a stride of 1, and residual connections. Two 3×3 PConv convolutional modules serve as the input and output of the Starblock module, respectively, connecting the CBR module of this StarNet-P network structure and the output feature map of the next level. The two 3×3 PConv convolutional modules are connected by three fully connected layers (FC). The two FC layers process the output of the input 3×3 PConv convolutional modules and map the input features to a high-dimensional linear space through star operations. The output features are then provided to the output 3×3 PConv convolutional module through the third FC layer. The output 3×3 PConv convolutional module also provides its output as a residual to the input as the input of the 3×3 PConv convolutional module.

[0029] The Starblock module effectively reduces computational complexity while preserving spatial information of features through feature space transformation operations. The star operation, the core operation in the Starblock module, fuses two linearly transformed features through element-wise multiplication, mapping the input features to a high-dimensional nonlinear feature space. This high-dimensional feature mapping significantly enhances the network's expressive power without increasing its width, enabling it to better capture complex features and improving computational efficiency. Furthermore, the element-wise multiplication operation in the star operation improves the model's nonlinear fitting ability, similar to a multinomial kernel function. By stacking multiple layers, the star operation can exponentially increase the hidden dimension, significantly improving network performance, especially in classification and detection tasks.

[0030] In practical applications, the mathematical expression for the star operation used by the Starblock module in this StarNet-P network structure is: Where W1 and W2 are weight matrices, B1 and B2 are bias vectors, and X represents the input vector. This indicates element-wise multiplication.

[0031] After processing through four Stage modules, the network compresses the feature maps in the spatial dimension using global average pooling to generate a global feature representation. This operation not only preserves feature channel information but also reduces computational complexity. The main approach is to replace the previous SPPF module in YOLOv8 with four Stage modules from the StarNet-P network. This design allows the model to significantly improve its feature extraction capabilities and classification performance while maintaining high computational efficiency.

[0032] The SCPL-YOLO detection model constructed using the above method can be used to... Figure 1 The second and third diagrams show that each C2f-SP module in the neck network refers to... Figure 9 The method shown is configured to include: The CBS module is set at the input and output terminals of the C2f-SP module, respectively; The feature segmentation module (Split) is used to split the feature map output by the input CBS module into two sub-feature maps along the channel dimension; The feature extraction module, which uses a three-level sequentially connected Starblock module, extracts features from one of the sub-feature maps output by the feature segmentation module. The concatenation unit (Concat) connects to both the third-level Starblock module and another sub-feature map output by the feature segmentation module, concatenates their outputs, and provides them to the CBS module at the output end. For details regarding the Starblock module used within the C2f-SP module, please refer to [link / reference needed]. Figure 5 Instead of DWConv (Depthwise Convolution), PConv (Partial Convolution) performs convolution operations on only a portion of the input channels, while retaining the remaining channels.

[0033] Furthermore, the SCPL-YOLO detection model constructed using the above method can also refer to... Figure 6 The lightweight shared group convolutional (LSGC) structure used by the detection head can be replaced with group normalized convolutional (Conv_GN) layers to normalize the channels, and constructed as follows: The second, third, and fourth layers of the StarNet-P network structure in the backbone network (corresponding to P3, P4, and P5 respectively) are sequentially connected to three sets of 3×3 CBS modules. The three sets of 3×3 CBS modules preprocess the input features of different sizes provided by the three layers of the StarNet-P network structure. Then, a 1×1 Conv_GN unit is connected after each of the three 3×3 CBS modules to adjust the number of channels; Then, at its back end, a 3×3 Conv_GN unit is used to perform multi-scale feature fusion on the outputs of the three 1×1 Conv_GN units respectively; After fusion, the precise location of the jacquard fabric defect in the image is output after regression branch (Conv_Reg) and scaling (Scale). After classification branch (Conv_Cls), the probability corresponding to the shrinkage of the jacquard fabric defect is output.

[0034] The detection model constructed in the above manner can utilize jacquard fabric images in the enterprise's production environment, acquire them at high resolution using an EPSON scanner, expand the dataset using data augmentation methods (such as random rotation, flipping, brightness adjustment, etc.), and finally use the LabelImg tool to manually annotate the target area to construct a jacquard fabric detection dataset. This dataset is then used for training to determine the parameters in the model that correspond to the detection target.

[0035] To alleviate the problem of imbalanced sample size, this paper employs data augmentation techniques to expand the data of the minority class through operations such as random rotation, random flipping, random translation, and random cropping, thereby balancing the number of samples in each class in the dataset. These operations not only increase the number of minority class samples but also improve the model's generalization ability to fabric patterns of different shapes and positions. After data augmentation, the proportion of each class's sample size in the total dataset is shown below. Figure 3 As shown Jacquard fabric inspection tasks typically require operation on resource-constrained equipment. Such equipment imposes stringent limitations on the number of parameters and computational complexity of the inspection model, demanding high accuracy and low computational cost to meet the high accuracy and speed requirements of embedded devices. Therefore, this paper optimizes the SCPL-YOLO model using a combination of model pruning and knowledge distillation methods, aiming to reduce storage and computational overhead while maintaining high accuracy.

[0036] The specific execution process of pruning and distillation is as follows: Figure 2As shown, the SCPL-YOLO model obtained in the previous chapter is optimized using the LAMP (Layer-Adaptive Magnitude-based Pruning) method to reduce its parameter size and computational complexity. Specifically, a LAMP score is first calculated for the weights of each layer of the SCPL-YOLO model. The weights are then globally sorted based on their LAMP scores, and the connections with the lowest scores are removed until the global sparsity constraint is met. To distinguish between different models, the pruned model is named SCPL-YOLO-P. While pruning methods can effectively reduce the number of parameters and other performance metrics, they can also lead to a slight decrease in accuracy. Therefore, the ICC knowledge distillation method is used to compensate for this shortcoming. The higher-performing YOLOv8x is selected as the teacher model for knowledge distillation, and SCPL-YOLO-P is used as the student model. The ICC knowledge distillation method is then employed. This method calculates the inter-channel correlation matrix of the YOLOv8x network and the SCPL-YOLO-P network separately and minimizes the error between them, enabling the student model to learn the feature diversity of the teacher model, ultimately resulting in the knowledge-distilled model. The distilled model was named SCPL-YOLO-PD. The above method aims to reduce model complexity while improving model accuracy. The specific steps include: The core design principle of the LAMP pruning algorithm is minimizing model-level distortion. Specifically, during pruning, it optimizes the sparsity distribution between layers to minimize the cumulative ℓ² distortion of the model output. Traditional pruning methods typically perform pruning operations based on the weights of individual layers, minimizing only the distortion of a single layer's output, neglecting the impact on the overall model output. The LAMP algorithm, however, takes a holistic approach, explicitly approximating the impact of model-level distortion on pruning by introducing dynamically normalized LAMP scores, while simplifying the pruning process and maintaining efficiency. This mechanism allows the LAMP algorithm to automatically adapt to the weight distribution of different layers, rationally allocating sparsity for each layer, avoiding manual adjustments. By optimizing the model pruning process, the LAMP algorithm effectively reduces the computational cost and storage requirements of the model, making it particularly suitable for embedded devices with limited computing resources. The specific pruning execution flow of this algorithm is as follows: Figure 7 As shown; Pruned models typically experience slight decreases in performance metrics such as precision, recall, and mAP@50. In such cases, knowledge distillation is used to optimize the pruned model. This method allows the pruned model to learn knowledge from a highly accurate teacher model, resulting in a significant reduction in the number of parameters and computational complexity compared to the SCPL-YOLO model, while simultaneously improving performance metrics like precision. This facilitates efficient deployment and operation on embedded devices. The ICC (Inter-Channel Correlation) knowledge distillation method is used to distill the pruned model. The overall structure of the ICC knowledge distillation method is as follows: Figure 8 As shown Finally, the streamlined model is deployed to the appropriate hardware. This stage involves deploying the trained SCPL-YOLO-PD model onto an embedded hardware device, specifically an NVIDIA Jetson Orin NX 16GB. This type of hardware typically includes dedicated components such as a GPU, which can effectively accelerate the model's operation and inference process, thereby improving the system's processing efficiency.

[0037] The following section provides a more detailed and comprehensive analysis and explanation of the design key points and advantages of the SCPL-YOLO-based jacquard fabric testing model provided in this application, using a complete example.

[0038] 1- Overall Model Architecture Design This invention, based on the YOLOv8 framework, proposes an improved model called SCPL-YOLO, specifically addressing the complex textures, diverse patterns, and rich feature layers of jacquard fabrics. This model optimizes the three core modules of YOLOv8: Backbone, Neck, and Head. The overall structure is as follows: Figure 1 As shown.

[0039] Specifically, this invention introduces the StarNet-P network into the Backbone layer to enhance the feature extraction capability for complex textures; adopts an improved C2f-Starblock-PConv module (C2f-SP module for short) in the Neck layer to improve the efficiency of multi-scale feature fusion and processing; and designs a lightweight shared group convolutional structure (LSGC) in the Head layer to reduce redundant computation through a weight sharing mechanism, thereby improving the localization and classification accuracy of the detection head.

[0040] 2- StarNet-P backbone network design used in the model To overcome the problem of detail loss caused by spatial resolution compression during feature extraction in the original YOLOv8 backbone network, this invention replaces its backbone layer with the StarNet-P network. The StarNet-P network structure is as follows: Figure 4 As shown.

[0041] The StarNet-P network employs a four-stage hierarchical design, with each stage consisting of a CBR module and a Starblock module. The CBR module, composed of 3×3 convolutions, batch normalization (BN), and a ReLU6 activation function, is responsible for downsampling the input feature map. The Starblock module is the core of the network, and its structure is as follows: Figure 9 As shown, it consists of fully connected layers, 3×3 PConv convolutions, and residual connections. This module maps input features to a high-dimensional nonlinear space through "star operations," significantly improving the model's ability to express complex textures. Its mathematical expression is as follows: Where W1 and W2 are weight matrices, B1 and B2 are bias vectors, and * denotes element-wise multiplication. This operation enhances feature representation through nonlinear mapping without increasing network width, and is particularly suitable for feature extraction of complex patterns in jacquard fabrics.

[0042] 3- C2f-Starblock-PConv Feature Fusion Module To further enhance the model's ability to handle complex features, this invention designs a C2f-Starblock-PConv module (C2f-SP) in the Neck layer, the structure of which is as follows: Figure 9 As shown.

[0043] This module replaces the Bottleneck unit in the original C2f module with an improved Starblock module. The improved Starblock module uses PConv (Partial Convolution) instead of the original DWConv (Depthwise Convolution), as shown below. Figure 5 As shown, PConv performs convolution operations only on a portion of the input channels, while retaining the remaining channels. This reduces computational load and avoids the inefficiency issues caused by frequent memory accesses in DWConv.

[0044] Specifically, to address the shortcomings of the DWConv module, the PConv module is introduced to improve the Starblock module, and this improved Starblock module is also applied to the overall StarNet-P network. PConv supports flexible convolution operations on select channels, reducing redundant computation and memory access, thus extracting spatial features more efficiently. The structures of the improved Starblock and PConv modules are as follows: Figure 5 As shown, the PConv module rationally divides the input feature channels and performs convolution operations only on a portion of the input channels. Subsequently, it merges the output of this partial operation with another portion of the features, thereby significantly reducing computational cost (FLOPs) and avoiding the frequent memory access issues common in DWConv. This design enables PConv to not only achieve higher floating-point efficiency (FLOPS) at runtime, effectively reducing latency and increasing throughput, but also to more efficiently focus on the feature information of important channels. These advantages allow the PConv module to capture important feature information more quickly and accurately when processing complex features in the Starbolck module; therefore, PConv convolutions are used to replace DWConv convolutions in the Starbolck module.

[0045] 4- LSGC Lightweight Shared Detection Head To address the parameter redundancy issue caused by independent convolutions on feature maps of different scales in the original YOLOv8 detector head, this invention introduces a lightweight shared group convolutional structure (LSGC) in the Head layer, as shown in the figure below. Figure 6 As shown.

[0046] The LSGC architecture replaces traditional Batch Normalization (BN) layers with grouped normalized convolutions (Conv_GN), grouping and normalizing channels to improve training stability. Simultaneously, this architecture achieves unified processing of feature maps at three scales (80×80, 40×40, and 20×20) by sharing convolutional weights, significantly reducing the number of parameters. Specifically, LSGC first preprocesses the input features using a 3×3 CBS module, then adjusts the number of channels using a 1×1 Conv_GN, and finally uses a 3×3 Conv_GN to achieve multi-scale feature fusion. This design significantly improves the model's inference efficiency while maintaining detection accuracy.

[0047] It is simpler than the YOLOv8 Head layer. Its structure removes redundant convolution operations in the original Head layer to reduce unnecessary computation. At the same time, it uses group normalized convolution to share features at different levels, improving the model's ability to detect targets of different sizes.

[0048] This module primarily processes feature maps of different scales output by the Neck layer, with spatial resolutions of 80×80, 40×40, and 20×20. The internal structure of LSGC mainly consists of a CBS module and group-normalized convolutions (Conv_GN) of different sizes. First, a 3×3 CBS module preprocesses the output of the previous layer to improve the non-linear feature representation capability of the corresponding layer. Then, a 1×1 Conv_GN convolution operation is used to adjust the number of channels in the input feature map. Finally, a 3×3 Conv_GN convolution is used to share weights for feature information at different scales to achieve feature information fusion. Both of these Conv_GN convolution operations use group-normalized layers instead of BN layers, which can group and normalize channels and improve the stability of the model during training. The processed feature maps are passed to different task decoupling branches, mainly including Conv_Reg layers for bounding box regression and Conv_Cls layers for object classification. Conv_Reg layers of different scales perform corresponding scaling operations to adjust the scale of the prediction results. The main advantage of the LSGC module is that it improves the model's ability to detect targets of different sizes while reducing redundant computation.

[0049] 5-LAMP Pruning Algorithm To meet the lightweight model requirements of embedded devices, this invention employs the LAMP (Layer-Adaptive Magnitude-based Pruning) algorithm to compress the SCPL-YOLO model. The LAMP algorithm achieves global importance assessment and pruning of weights by minimizing model-level distortion, and its process is as follows: Figure 2 As shown.

[0050] The LAMP scoring formula is as follows: Where W_x represents the x-th weight value after sorting. This score comprehensively considers the weight magnitude and its relative importance in the global context, enabling more accurate identification and removal of redundant connections. The pruned model (SCPL-YOLO-P) reduced the number of parameters, model size, and GFLOPs by approximately 77%, 75.5%, and 50.5%, respectively, while improving inference speed by 24.4%–37.3%.

[0051] 6- ICC Knowledge Distillation Method To compensate for the accuracy loss caused by pruning, this invention proposes a knowledge distillation method based on inter-channel correlation (ICC), the structure of which is as follows: Figure 8 As shown.

[0052] This method uses YOLOv8x as the teacher model and SCPL-YOLO-P as the student model, achieving knowledge transfer by minimizing the difference in feature channel correlation between the two. The ICC matrix calculation formula is as follows: Where F_T and F_S are the feature maps of the teacher and student networks, respectively. To reduce the impact of noise, this invention divides the feature maps into n×m sub-blocks, calculates the ICC matrix of each sub-block, and then aggregates them. The distillation loss function is: The total loss function is: Experiments show that the distilled model (SCPL-YOLO-PD) improves precision, recall and mAP@50 by 3.1%, 2.4% and 2.9% respectively, while significantly reducing model size and computational cost.

[0053] 7- Model Conversion and TensorRT Acceleration To achieve efficient inference of the model on embedded devices, this invention converts the trained SCPL-YOLO-PD model into the TensorRT engine format. The conversion process is as follows: Figure 10 As shown.

[0054] The specific steps include: First, convert the PyTorch model (.pt) to the .wts intermediate format; then, use the TensorRT parser to build the network definition; then perform optimizations such as layer fusion and accuracy calibration; finally, generate the .engine format inference engine.

[0055] On the NVIDIA Jetson Orin NX 16GB device, the optimized model has a single-frame inference latency of approximately 6ms, which meets the requirements for real-time detection.

[0056] 8- Design of Jacquard Fabric Inspection System This application allows for the development of an interactive inspection system for jacquard fabrics based on the PySide6 framework. Its overall architecture is as follows: Figure 11 As shown.

[0057] The system front-end provides user login, model import, image / video / camera detection, confidence adjustment, and other functional interfaces, such as... Figure 12As shown, the backend uses a PostgreSQL database for user management and result storage, and employs a multi-threaded mechanism to call the TensorRT engine for real-time inference, feeding the results back to the frontend interface. The system supports multiple detection modes, possesses high compatibility, high stability, and ease of operation, and can be widely applied in the actual production environments of textile enterprises.

[0058] In summary, this application addresses the problems of low efficiency and high false positive rate in traditional manual inspection of jacquard fabrics by proposing a lightweight detection model that integrates feature enhancement, model compression, and knowledge distillation. A complete hardware and software collaborative detection system is also constructed to achieve efficient, accurate, and real-time detection of jacquard fabrics. This application improves the YOLOv8 network structure and combines a feature extraction enhancement module, model pruning, and knowledge distillation techniques to achieve a high-precision, high-speed jacquard fabric detection model that can be deployed on embedded devices. This invention constructs a complete intelligent detection solution for jacquard fabrics from three levels: algorithm optimization, model compression, and system implementation. The SCPL-YOLO model enhances feature extraction and classification capabilities, and LAMP pruning and ICC distillation achieve a balance between model lightweighting and accuracy, ultimately enabling efficient deployment and real-time inference on embedded devices. Experiments show that this invention has high precision, recall, and real-time performance in jacquard fabric detection tasks, demonstrating promising prospects for industrial applications.

[0059] Its specific innovations include: 1) Feature fusion optimization module design. Based on the C2f module of YOLOv8, the C2f-Starblock-PConv module is introduced. This module adopts a partial convolution mechanism to improve robustness in incomplete or complex backgrounds, and uses multi-branch convolution to enhance the feature fusion capability of the network, thereby improving the detection accuracy of multi-texture targets.

[0060] 2) Lightweight optimization of the detection head. A lightweight shared group convolution (LSGC) structure is introduced into the YOLOv8 detection head. By reducing the number of convolution kernel parameters and feature channel redundancy, the computational overhead and inference latency are reduced while ensuring classification and localization accuracy.

[0061] 3) Model pruning. The LAMP (Layer-wise Adaptive Magnitude Pruning) algorithm was used to perform structured pruning on the SCPL-YOLO model. By evaluating the importance of the weights in each layer, low-contribution parameters were pruned layer by layer, thereby significantly reducing the number of model parameters and computational cost while maintaining detection performance.

[0062] 4) Knowledge distillation optimization. To restore the accuracy of the pruned model, the ICC (Inter-Class Compactness) knowledge distillation method is adopted. The intermediate features of the teacher network are used to guide the feature learning of the student network, thereby enhancing the discriminative feature representation ability of the model.

[0063] 5) Model Deployment. The optimized model was deployed on an Nvidia Jetson Orin NX16GB embedded device using the TensorRT framework, and an interactive detection interface was developed using PySide6 to achieve real-time detection and visualization of jacquard fabrics.

[0064] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting jacquard fabrics based on deep learning, characterized in that the steps include... include: The detection model, obtained through pre-training, identifies the input images, videos, or real-time data from the camera to obtain the detection results of jacquard fabric defects. The detection model includes: The backbone network uses the CBS module as the starting feature layer, sequentially connects 4 layers of StarNet-P network structure, reduces the dimensionality of the feature map layer by layer, and transforms the dimensionality-reduced input features into a high-dimensional nonlinear space. The neck network, connecting layers 2 to 4 of the StarNet-P network structure, is used for: enhancing the feature map output from layer 4 of the StarNet-P network through the C2f-SP module, then upsampling it and concatenating it with the feature map output from layer 3 of the StarNet-P network structure; fusing the feature map after concatenation of layers 3 and 4 of the StarNet-P network structure through the C2f-SP module, then upsampling it and concatenating it with the feature map output from layer 2 of the StarNet-P network structure; enhancing the feature map after concatenation of layers 2 and 3 of the StarNet-P network structure through the C2f-SP module, then inputting it into the CBS convolutional layer for channel compression and feature enhancement, and then concatenating it with the feature map output after fusion of layers 3 and 4 of the StarNet-P network structure; finally, inputting the concatenated result into the C2f-SP module, concatenating it with the feature map output from layer 4 of the StarNet-P network structure through the CBS convolutional layer, and then outputting it through the last three C2f-SP modules. The detection head uses a lightweight shared-group convolutional structure to receive features output by the C2f-SP module and outputs the detection results of jacquard fabric defects.

2. The deep learning-based jacquard fabric detection method as described in claim 1, characterized in that, Each layer of the StarNet-P network structure includes: a CBR module, which consists of sequentially connected 3×3 convolutions, batch normalization, and ReLU6 activation functions, used to downsample the input feature map; The Starblock module consists of fully connected layers, 3×3 PConv convolutional modules, and residual connections, where: Two 3×3 PConv convolutional modules serve as the input and output of the Starblock module, respectively, connecting the CBR module of this StarNet-P network structure to the output feature map of the next level. There are three fully connected layers between the two 3×3 PConv convolutional modules. The two fully connected layers process the output of the input 3×3 PConv convolutional modules respectively, and then map the input features to a high-dimensional linear space through star operation. The third fully connected layer then provides the input features to the output 3×3 PConv convolutional module. The 3×3 PConv convolution module at the output end also provides its output as a residual to the input end as the input of the 3×3 PConv convolution module.

3. The deep learning-based jacquard fabric detection method as described in claim 1, characterized in that, The mathematical expression for the star operation is: Where W1 and W2 are weight matrices, and B1 and B2 are bias vectors. This indicates element-wise multiplication.

4. The deep learning-based jacquard fabric detection method as described in claim 1, characterized in that, Each C2f-SP module includes: The CBS module is set at the input and output terminals of the C2f-SP module, respectively; The feature segmentation module is used to split the feature map output by the input CBS module into two sub-feature maps along the channel dimension; The feature extraction module, which uses a three-level sequentially connected Starblock module, extracts features from one of the sub-feature maps output by the feature segmentation module. The splicing unit connects to both the third-level Starblock module and another sub-feature map output by the feature segmentation module. After splicing their outputs, the spliced ​​outputs are provided to the CBS module at the output end.

5. The deep learning-based jacquard fabric detection method as described in claim 4, characterized in that, In the Starblock module used in each C2f-SP module, PConv replaces DWConv. PConv performs convolution operations only on a portion of the input channels, while the remaining channels are directly retained.

6. The method for detecting jacquard fabrics based on deep learning as described in claim 1, characterized in that, The lightweight shared group convolutional structure replaces the BN layer with group-normalized convolutions, normalizing the channels by grouping them. Its specific structure includes: Three sets of 3×3 CBS modules are used to preprocess input features of different sizes respectively. Each of the three sets of 3×3 CBS modules is connected to a 1×1 Conv_GN unit to adjust the number of channels. A 3×3 Conv_GN unit performs multi-scale feature fusion on the outputs of three 1×1 Conv_GN units. After regression branching and scale mapping, it outputs the precise location of the jacquard fabric defect in the image and outputs the probability corresponding to the shrinkage of the jacquard fabric defect after classification branching.

7. The deep learning-based jacquard fabric detection method as described in any one of claims 1-5, characterized in that, After the detection model was trained to target defects in jacquard fabrics, it was compressed using the LAMP pruning algorithm and the knowledge distillation method.

8. The deep learning-based jacquard fabric detection method as described in claim 7, characterized in that, The pruning algorithm first calculates the LAMP score for each layer of weights obtained from training the detection model, then sorts the weights globally based on the LAMP score, removes the connection with the smallest score, and continues until the global sparsity constraint is met. Then, it is used as a student model and distilled according to the knowledge distillation method. The knowledge distillation method uses YOLOv8x as the teacher model and the model obtained by the pruning algorithm as the student model. The inter-channel correlation matrix of the teacher model and the student model is calculated according to the following formula: ， ， Then, using the distillation loss function: , Total loss function: , The compressed detection model is obtained by minimizing the inter-channel correlation error between the teacher model and the student model.

9. An embedded device for detecting jacquard fabrics based on deep learning, characterized in that, It is based on the PySide6 framework and includes: The detection interface is used to receive images or videos to be detected or to directly receive real-time data from the camera. The detection model receives data from the detection interface and outputs detection results according to the pre-defined detection model. A visualization platform for displaying the detection results of the detection model; The detection model includes: The backbone network uses the CBS module as the starting feature layer, sequentially connects 4 layers of StarNet-P network structure, reduces the dimensionality of the feature map layer by layer, and transforms the dimensionality-reduced input features into a high-dimensional nonlinear space. The neck network, connecting layers 2 to 4 of the StarNet-P network structure, is used for: enhancing the feature map output from layer 4 of the StarNet-P network through the C2f-SP module, then upsampling it and concatenating it with the feature map output from layer 3 of the StarNet-P network structure; fusing the feature map after concatenation of layers 3 and 4 of the StarNet-P network structure through the C2f-SP module, then upsampling it and concatenating it with the feature map output from layer 2 of the StarNet-P network structure; enhancing the feature map after concatenation of layers 2 and 3 of the StarNet-P network structure through the C2f-SP module, then inputting it into the CBS convolutional layer for channel compression and feature enhancement, and then concatenating it with the feature map output after fusion of layers 3 and 4 of the StarNet-P network structure; finally, inputting the concatenated result into the C2f-SP module, concatenating it with the feature map output from layer 4 of the StarNet-P network structure through the CBS convolutional layer, and then outputting it through the last three C2f-SP modules. The detection head uses a lightweight shared-group convolutional structure to receive features output by the C2f-SP module and outputs the detection results of jacquard fabric defects.

10. The embedded device for detecting jacquard fabrics based on deep learning as described in claim 9, characterized in that, After the detection model was trained to detect defects in jacquard fabrics, it was further compressed using the LAMP pruning algorithm and the knowledge distillation method. First, calculate the LAMP score for each layer of weights trained on the detection model. Then, globally sort the weights based on the LAMP scores and remove the connections with the lowest scores until the global sparsity constraint is met. Use this result as the student model and YOLOv8x as the teacher model. Calculate the inter-channel correlation matrix for both the teacher and student models using the following formula: Then, using the distillation loss function: , Total loss function: , The compressed detection model is obtained by minimizing the inter-channel correlation error between the teacher model and the student model.

Images